Bug 56436 - BINDINGSGENERATOR: warning BG8C00: For type Java.Util.IList, base interface Java.Util.ICollection does not exist.
Summary: BINDINGSGENERATOR: warning BG8C00: For type Java.Util.IList, base interface ...
Status: VERIFIED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: Bindings ()
Version: 7.3 (15.2)
Hardware: Macintosh Mac OS
: High normal
Target Milestone: 15.4
Assignee: Atsushi Eno
URL:
: 56435 58187 ()
Depends on:
Blocks:
 
Reported: 2017-05-15 07:31 UTC by nazar
Modified: 2017-09-29 07:17 UTC (History)
10 users (show)

Tags: bb ac
Is this bug a regression?: Yes
Last known good build:


Attachments
project to reproduce the issue (177.14 KB, application/zip)
2017-05-15 07:32 UTC, nazar
Details
updated project to reproduce (241.18 KB, application/zip)
2017-06-08 18:10 UTC, nazar
Details
Build output from Visual Studio for Mac (23.87 KB, text/plain)
2017-06-28 22:40 UTC, Mark McLemore
Details
Another repro case (937.39 KB, application/zip)
2017-07-25 14:13 UTC, Matthew Leibowitz
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
VERIFIED FIXED

Description nazar 2017-05-15 07:31:47 UTC
System information

=== Xamarin Studio Community ===

Version 6.3 (build 864)
Installation UUID: 7ff12dfc-d5e5-409a-9c27-088e6e54def6
Runtime:
	Mono 5.0.0.100 (2017-02/9667aa6) (64-bit)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 500000100

=== Xamarin.Android ===

Version: 7.3.0.13 (Xamarin Studio Community)
Android SDK: /Users/nazar/Library/Android/sdk
	Supported Android versions:
		7.1 (API level 25)

SDK Tools Version: 26.0.1
SDK Platform Tools Version: 25.0.5
SDK Build Tools Version: 25.0.2

Java SDK: /usr
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Operating System ===

Mac OS X 10.12.4
Darwin iMac-Nazar.Dlink 16.5.0 Darwin Kernel Version 16.5.0
    Fri Mar  3 16:52:33 PST 2017
    root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64

Binding generator can't generate bindings some classes, therefore our implementation od List can't be bounded.

Here is the BINDINGSGENERATOR warnings:

 Warnings:

/Users/nazar/Documents/scichart_root/XamarinBinding/XamarinBinding/XamarinBinding/XamarinBinding.csproj (Build) ->
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Bindings.targets (GenerateBindings target) ->

	BINDINGSGENERATOR:  warning BG8C00: For type Java.Util.IList, base interface Java.Util.ICollection does not exist.
	BINDINGSGENERATOR:  warning BG8502: Invalidating Java.Util.IList and all nested types because some of its interfaces were invalid.
	BINDINGSGENERATOR:  warning BG8C00: For type Com.Scichart.Xamarinlibrary.ISciList, base interface java.util.List<E> is invalid.
	BINDINGSGENERATOR:  warning BG8502: Invalidating Com.Scichart.Xamarinlibrary.ISciList and all nested types because some of its interfaces were invalid.

Please see attached project which reproduces the problem

If you need any other information to reproduce this, please let us know.
Comment 1 nazar 2017-05-15 07:32:17 UTC
Created attachment 22164 [details]
project to reproduce the issue
Comment 2 nazar 2017-05-15 07:35:24 UTC
*** Bug 56435 has been marked as a duplicate of this bug. ***
Comment 3 andrew 2017-05-23 09:36:23 UTC
This bug is still affecting us in Xamarin.Android Version: 7.3.0.13 and is blocking us from releasing our Xamarin.Android binding library for SciChart - the result of a year of investment and effort!!

Please find full system report below: 

=== Visual Studio Community 2017 for Mac (Preview) ===

Version 7.1 Preview (7.1 build 583)
Installation UUID: 47116cce-50dc-4dda-a94e-ef7bac0cefab
Runtime:
	Mono 5.2.0.104 (2017-04/4a0006f) (64-bit)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 502000104

=== NuGet ===

Version: 4.0.0.2323

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
SDK: /usr/local/share/dotnet/sdk/1.0.3/Sdks
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.5.4
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Apple Developer Tools ===

Xcode 8.3 (12169)
Build 8E162

=== Xamarin.Android ===

Version: 7.3.0.13 (Visual Studio Community)
Android SDK: /Users/aburnettthompson/Library/Android/sdk
	Supported Android versions:
		4.4    (API level 19)
		4.4.87 (API level 20)
		5.0    (API level 21)
		5.1    (API level 22)
		6.0    (API level 23)
		7.0    (API level 24)
		7.1    (API level 25)

SDK Tools Version: 26.0.1
SDK Platform Tools Version: 26.0.0
SDK Build Tools Version: 25.0.3

Java SDK: /usr
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Xamarin.iOS ===

Version: 10.11.0.126 (Visual Studio Community)
Hash: 7571635e
Branch: master
Build date: 2017-05-09 16:04:54-0400

=== Xamarin.Mac ===

Version: 3.5.0.126 (Visual Studio Community)

=== Xamarin Inspector ===

Version: 1.3.0-alpha1
Hash: 0855e01
Branch: master
Build date: Thu, 11 May 2017 04:16:13 GMT

=== Build Information ===

Release ID: 701000583
Git revision: 445a7f09feca58babb966e0c66a6b299d0bd450c
Build date: 2017-05-12 16:05:38-04
Xamarin addins: f9b72ca5f6ca5d9476d8f58353ada2afd56c549b
Build lane: monodevelop-lion-d15-3-preview

=== Operating System ===

Mac OS X 10.12.3
Darwin 16.4.0 Darwin Kernel Version 16.4.0
    Thu Dec 22 22:53:21 PST 2016
    root:xnu-3789.41.3~3/RELEASE_X86_64 x86_64



Best regards, 
Andrew
Comment 4 nazar 2017-06-01 15:47:58 UTC
hi guys, Is there anything happening in order to fix this?
Comment 5 nazar 2017-06-08 14:25:03 UTC
Hi guys, so it's almost a month when we report this. 

Have you reproduced it? Are there any ideas what should we do?

Any help is highly appreciated!
Comment 6 andrew 2017-06-08 17:38:08 UTC
I can confirm this bug still exists in Visual Studio for Mac + Xamarin/Android 7.3.1.2

We are unable to update our build server and are still stuck on Xamarin versions from 3 months ago! We are getting worried now as we've invested a year in building chart components for xamarin but code & features that used to be available are no longer compiling
Comment 7 nazar 2017-06-08 18:10:36 UTC
Created attachment 22781 [details]
updated project to reproduce
Comment 8 nazar 2017-06-08 18:12:01 UTC
so I've updated the project to reproduce, which clearly shows the place where and which class isn't generated...

let us know if you are able to reproduce.

Thanks in advance.

Best regards,
Nazar
Comment 9 nazar 2017-06-22 18:49:03 UTC
Hi guys, is there anything we can do to help you solve this serious issue?
Comment 10 nazar 2017-06-27 10:06:21 UTC
guys, could you at least let us know what is going on there? because this is quite serious since we can't use the latest build of xamarin.android and at some point, it can introduce serious problems for us... (

any help is highly appreciated
Comment 11 nazar 2017-06-27 13:26:28 UTC
also, the ticket has Importance:	--- normal which is not true, it's almost urgent because our release depends on this issue /...
Comment 12 Mark McLemore 2017-06-28 22:39:59 UTC
Thank you for taking the time to submit the bug (sorry for the delay in getting back to you!). We were able to reproduce this bug using Visual Studio for Mac Community 7.0.1 and Xamarin.Android 7.3.1.2 using the following steps:

1. Download, unzip attachment 22781 [details] project.
2. Open solution in Visual Studio for Mac.
3. Right-click XamarinBinding project > Build XamarinBinding.

I've attached the build-output.txt from these steps to help the engineering team look into the issue.
Comment 13 Mark McLemore 2017-06-28 22:40:58 UTC
Created attachment 23185 [details]
Build output from Visual Studio for Mac
Comment 14 nazar 2017-06-29 07:04:33 UTC
Hi, Mark, thank you for confirming the bug, hopefully, it shouldn't be too hard to fix.

Also, it's the same on windows. Just so you know

Thank you very much:)
Comment 15 nazar 2017-07-11 07:16:55 UTC
Hi, guys, any news on this?
Comment 16 Matthew Leibowitz 2017-07-25 05:47:20 UTC
This is still an issue in Xamarin.Android 7.4.0.15
Comment 17 Matthew Leibowitz 2017-07-25 14:13:17 UTC
Created attachment 23788 [details]
Another repro case
Comment 18 Matthew Leibowitz 2017-07-25 14:14:14 UTC
My PC info: https://gist.github.com/mattleibow/2f112e87497f64ecb9fe268548beef94
Comment 19 Matthew Leibowitz 2017-08-04 14:17:33 UTC
May be related to: https://bugzilla.xamarin.com/show_bug.cgi?id=58187
Comment 20 Matthew Leibowitz 2017-08-04 16:21:54 UTC
My full build log: https://gist.github.com/mattleibow/9f4fa0850fb920ca4b48e74a5a32c73b

    BINDINGSGENERATOR : warning BG8C00: For type Java.Util.IQueue, base interface Java.Util.ICollection does not exist.
    BINDINGSGENERATOR : warning BG8502: Invalidating Java.Util.IQueue and all nested types because some of its interfaces were invalid.
    BINDINGSGENERATOR : warning BG8C00: For type UniversalImageLoader.Core.Assist.Deque.IDeque, base interface java.util.Queue<E> is invalid.
    BINDINGSGENERATOR : warning BG8502: Invalidating UniversalImageLoader.Core.Assist.Deque.IDeque and all nested types because some of its interfaces were invalid.
    BINDINGSGENERATOR : warning BG8C00: For type UniversalImageLoader.Core.Assist.Deque.IBlockingDeque, base interface com.nostra13.universalimageloader.core.assist.deque.Deque<E> is invalid.
    BINDINGSGENERATOR : warning BG8C00: For type Java.Util.Concurrent.IBlockingQueue, base interface Java.Util.IQueue is invalid.
    BINDINGSGENERATOR : warning BG8C00: For type Java.Util.Concurrent.IBlockingQueue, base interface Java.Util.ICollection does not exist.
    BINDINGSGENERATOR : warning BG8502: Invalidating Java.Util.Concurrent.IBlockingQueue and all nested types because some of its interfaces were invalid.
    BINDINGSGENERATOR : warning BG8C00: For type UniversalImageLoader.Core.Assist.Deque.IBlockingDeque, base interface java.util.concurrent.BlockingQueue<E> is invalid.
    BINDINGSGENERATOR : warning BG8502: Invalidating UniversalImageLoader.Core.Assist.Deque.IBlockingDeque and all nested types because some of its interfaces were invalid.
    BINDINGSGENERATOR : warning BG8C00: For type Java.Util.AbstractCollection, base interface Java.Util.ICollection does not exist.
    BINDINGSGENERATOR : warning BG8C00: For type Java.Util.AbstractQueue, base interface Java.Util.IQueue is invalid.
    BINDINGSGENERATOR : warning BG8C00: For type Java.Util.AbstractQueue, base interface Java.Util.ICollection does not exist.
    BINDINGSGENERATOR : warning BG8C00: For type UniversalImageLoader.Core.Assist.Deque.LinkedBlockingDeque, base interface com.nostra13.universalimageloader.core.assist.deque.BlockingDeque<E> is invalid.
    BINDINGSGENERATOR : warning BG8C00: For type UniversalImageLoader.Core.Assist.Deque.LinkedBlockingDeque, base interface com.nostra13.universalimageloader.core.assist.deque.BlockingDeque<E> is invalid.

It appears that ICollection is the real culprit here.
Comment 21 Jonathan Pryor 2017-08-04 19:20:41 UTC
I figured out the problem.

As is often the case, I'm not sure about the solution.

The immediate problem is *not* in `generator`. The "problem" (note scare quotes) is related to `Mono.Android.dll`. Specifically, consider the build output from Attachment #23185 [details]:

> Target GenerateBindings:
>     /Library/Frameworks/Xamarin.Android.framework/Versions/Current/bin/generator obj/Debug/api.xml --csdir=obj/Debug/generated/src --enumdir=obj/Debug/generated/enums --enummetadata=obj/Debug/generated/metadata --assembly=XamarinBinding --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/Java.Interop.dll --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v7.0/Mono.Android.dll --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/System.Core.dll --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/System.dll --ref=/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/Facades/System.Runtime.dll --fixup=Transforms/Metadata.xml --enumfields=Transforms/EnumFields.xml --enummethods=Transforms/EnumMethods.xml --api-level=24 --type-map-report=obj/Debug/generated/type-mapping.txt --global --public 

We can run that command manually from the project directory, but change the path to the Mono.Android.dll assembly used (so long as you have the appropriate Xamarin.Android versions installed).

This works (command truncated for sanity):

> /Library/Frameworks/Xamarin.Android.framework/Versions/Current/bin/generator \
>   --ref=Library/Frameworks/Xamarin.Android.framework/Versions/7.2.0-7/lib/xbuild-frameworks/MonoAndroid/v7.0/Mono.Android.dll 
>   ...

While this fails:

> /Library/Frameworks/Xamarin.Android.framework/Versions/Current/bin/generator \
>   --ref=Library/Frameworks/Xamarin.Android.framework/Versions/7.3.1-2/lib/xbuild-frameworks/MonoAndroid/v7.0/Mono.Android.dll 
>   ...

Thus, the question: what would have changed in `Mono.Android.dll` between 7.2 and 7.3? API for the same `$(TargetFrameworkVersion)`, so it's not anything related to API.

There are two internal changes of consequence between 7.2 and 7.3:

1. We changed which compiler was used, from `mcs` to `csc`.
2. We changed the *build system* for building `Mono.Android.dll`, from Makefiles to MSBuild.

(1), (2), or both in combination are at fault for the "real" problem (again, note scare quotes):

The order of `.class` declarations within `Mono.Android.dll` has changed.

In Xamarin.Android 7.2:

> $ monodis --typedef /Library/Frameworks/Xamarin.Android.framework/Versions/7.2.0-7/lib/xbuild-frameworks/MonoAndroid/v2.3/Mono.Android.dll | grep 'Collection '
> 107: Android.Runtime.JavaCollection (flist=2838, mlist=4986, flags=0x100001, extends=0x340)
> 2974: Java.Security.PermissionCollection (flist=28571, mlist=52711, flags=0x100081, extends=0x340)
> 3162: Java.Util.AbstractCollection (flist=32439, mlist=58849, flags=0x100081, extends=0x340)
> 3303: Java.Util.ICollection (flist=34297, mlist=62549, flags=0xa1, extends=0x0)

Compare to Xamarin.Android 7.3:

> $ monodis --typedef /Library/Frameworks/Xamarin.Android.framework/Versions/7.3.1-2/lib/xbuild-frameworks/MonoAndroid/v2.3/Mono.Android.dll | grep 'Collection '
> 1225: Java.Util.AbstractCollection (flist=12236, mlist=28302, flags=0x100081, extends=0x1d94)
> 1268: Java.Util.ICollection (flist=12547, mlist=29415, flags=0xa1, extends=0x0)
> 1595: Java.Security.PermissionCollection (flist=14852, mlist=36764, flags=0x100081, extends=0x1d94)
> 3064: Android.Runtime.JavaCollection (flist=23714, mlist=63857, flags=0x100001, extends=0x1d94)

Why does *that* matter?

Now we get to the *real* bug, which *is* generator *and* `Mono.Android.dll`:

1. `generator` registers types in assembly declaration order:

https://github.com/xamarin/java.interop/blob/acfcc62/tools/generator/CodeGenerator.cs#L259-L269

2. `Mono.Android.dll` *aliases* `java.util.Collection` with `JavaCollection`:

https://github.com/xamarin/xamarin-android/blob/5777337/src/Mono.Android/Android.Runtime/JavaCollection.cs

Related: generator type lookup is keyed off the Java type name, but when given a managed name will look through all values to find one with the same FullName:

https://github.com/xamarin/java.interop/blob/d1deb90/tools/generator/SymbolTable.cs#L258

With Xamarin.Android 7.2's Mono.Android.dll, `generator` first registers `Android.Runtime.JavaCollection` for `java.util.Collection`, then later replaces the JavaCollection mapping with one for Java.Util.ICollection. (This is because of the order of types in the assembly.)

With Xamarin.Android 7.4's Mono.Android.dll, the reverse happens: first `java.util.Collection` is associated with `Java.Util.ICollection`, then it's overwritten with an association to `Android.Runtime.JavaCollection`. At this point, any lookup for `Java.Util.ICollection` will fail.

Order-dependent *anything* is a recipe for disaster and/or insanity. :-/

# Possible fixes to consider:

1. Don't alias `JavaCollection` with java.util.Collection, by removing this `[Register]`:

https://github.com/xamarin/xamarin-android/blob/5777337/src/Mono.Android/Android.Runtime/JavaCollection.cs#L12

While this sounds sensible, this could break anything that needs to determine the Java name from a managed type, including plausibly method generation with [Export].

2. Special-case `generator` by ignoring `Android.Runtime` types. 

3. Special-case `generator` for `java.util.Collection` (and others?). `generator` already special-cases a number of types such as `java.util.List`; why not just add `java.util.Collection`?

https://github.com/xamarin/java.interop/blob/d1deb90/tools/generator/SymbolTable.cs#L176-L225

(3) is likely the sanest solution, but raises an associated question: what other types need special-casing?
Comment 22 Jonathan Pryor 2017-08-04 20:33:29 UTC
PR for fix: https://github.com/xamarin/java.interop/pull/173
Comment 23 Bill Holmes 2017-08-07 19:51:16 UTC
@jonp reading the summary of the PR I am now thinking that 58303 could be related as well?

https://bugzilla.xamarin.com/show_bug.cgi?id=58303
Comment 24 Jonathan Pryor 2017-08-08 14:15:04 UTC
@Holmes: I fear Bug #58303 is the same but different: from PR #173:

> For example, if an assembly defines `JavaCollection` before
> `ICollection`, then a type lookup for `java.util.Collection` will
> return `ICollection`. If instead the assembly defines `ICollection`
> before `JavaCollection`, then a type lookup for `java.util.Collection`
> will instead return `JavaCollection`.
> 
> This is not necessarily desirable, but I don't see much alternative.

I think this is *exactly* what Bug #58303 is hitting. PR 173 does not change this behavior; it's still order dependent. At present, I'm not sure how to fix that either.

Then again, the fix I came up with in PR 173 isn't any of the options I mentioned in Comment #21, so maybe I'll think of something...?
Comment 26 Jon Douglas [MSFT] 2017-08-21 21:32:26 UTC
*** Bug 58187 has been marked as a duplicate of this bug. ***
Comment 27 Jon Douglas [MSFT] 2017-09-26 21:47:50 UTC
Setting to Target Milestone 15.4 based on the comments in https://bugzilla.xamarin.com/show_bug.cgi?id=56436#c25
Comment 28 Saurabh Paunikar 2017-09-29 07:17:32 UTC
Verified on

Visual Studio Enterprise 2017 for Mac (Preview) ===

Version 7.2 Preview (7.2 build 623)
Installation UUID: 2743ed70-809a-46fb-9993-ed5c01fd39e1
Runtime:
	Mono 5.4.0.201 (2017-06/71277e78f6e) (64-bit)
	GTK+ 2.24.23 (Raleigh theme)

Package version: 504000201

Xamarin.Mac
Version: 3.99.9.8 (Visual Studio Enterprise)

Xamarin.Android
Version: 8.0.0.27 (Visual Studio Enterprise)

Xamarin.iOS
Version: 11.2.0.8

Detailed Build Info: https://gist.github.com/saurabh-paunikar/f1694ba19e76b1fd9aa15f9f751c24a5

Screencast links: https://www.screencast.com/t/TlG5gvOziYDq