Bug 60336 - Add support to `desugar` Java 8 .class files to be converted to Java 7 compatible .class files before dex
Summary: Add support to `desugar` Java 8 .class files to be converted to Java 7 compat...
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: Bindings ()
Version: 8.0 (15.4)
Hardware: PC Windows
: --- normal
Target Milestone: 15.6
Assignee: Atsushi Eno
URL:
Depends on:
Blocks:
 
Reported: 2017-10-23 19:44 UTC by Jon Douglas [MSFT]
Modified: 2017-12-18 16:56 UTC (History)
4 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Jon Douglas [MSFT] 2017-10-23 19:44:10 UTC
## Description:

Based on the Java 8 feature support documentation, there is a step to "desugar" in which converts Java 8 .class files to be Java 7 compatible ones. This allows for the use of lambdas, method references and other features of Java 8.

https://developer.android.com/studio/write/java8-support.html

It seems this is an external tool used in the gradle build process:

https://android.googlesource.com/platform/external/desugar/

Currently if one tries to bind a project that includes Java 8 features, CompileToDalvik / dx.jar will fail miserably about missing support. Here's an example:

3>COMPILETODALVIK : PARSE error : 
3>  InvokeDynamic not supported (TaskId:497)
3>  ...while preparsing cst 003d at offset 0000010e (TaskId:497)
3>  ...while parsing com/cxense/cxensesdk/CxenseSdk.class (TaskId:497)
3>   (TaskId:497)

This is an example of Java 8's lambda expressions being used in which InvokeDynamic is how their object representation is created.

Thus we need to add a step to "desugar" after javac and prior to dx
Comment 1 Atsushi Eno 2017-10-25 04:28:01 UTC
Any reproducible project?
Comment 3 Atsushi Eno 2017-10-30 10:05:42 UTC
Good news: your repro builds if you add xamarin-android-d8-build NuGet package (and will fail if you don't, which proves that the NuGet package works).

Bad news: it turned out that we need another fix in Dx build task which is invoked only if we use improved fast deployment, which is https://github.com/xamarin/xamarin-android/pull/990 (and our monodroid product which makes use of it).
Comment 4 Atsushi Eno 2017-10-30 10:34:56 UTC
Actually no - d8 is not as mature as dx and it does not report any defects regarding invalid bytecode operations and thus it gives false positives as if the jars were fine. We need individual r8 invocation :-(
Comment 5 Atsushi Eno 2017-10-31 09:21:49 UTC
Your repro projectmisses some dependency jars. Here is output from *desugar* tool bundled in Android Studio 3.0 (I'm copying it into my ongoing desugar support work):

java -jar /sources/xamarin-android/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/desugar/desugar_deploy.jar --bootclasspath_entry ~/android-sdk-linux/platforms/android-27/android.jar --input obj/Release/lp/1/jl/bin/classes.jar --output classes.jar
Exception in thread "main" java.lang.NoClassDefFoundError: com/cxense/Cxense
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
	at com.google.devtools.build.android.desugar.HeaderClassLoader.findClass(HeaderClassLoader.java:66)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at com.google.devtools.build.android.desugar.LambdaDesugaring.loadFromInternal(LambdaDesugaring.java:330)
	at com.google.devtools.build.android.desugar.LambdaDesugaring.access$500(LambdaDesugaring.java:53)
	at com.google.devtools.build.android.desugar.LambdaDesugaring$InvokedynamicRewriter.createLookup(LambdaDesugaring.java:559)
	at com.google.devtools.build.android.desugar.LambdaDesugaring$InvokedynamicRewriter.visitInvokeDynamicInsn(LambdaDesugaring.java:395)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(Unknown Source)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(Unknown Source)
	at org.objectweb.asm.ClassReader.a(Unknown Source)
	at org.objectweb.asm.ClassReader.b(Unknown Source)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at com.google.devtools.build.android.desugar.Desugar.desugarClassesInInput(Desugar.java:401)
	at com.google.devtools.build.android.desugar.Desugar.desugarOneInput(Desugar.java:326)
	at com.google.devtools.build.android.desugar.Desugar.desugar(Desugar.java:280)
	at com.google.devtools.build.android.desugar.Desugar.main(Desugar.java:584)
Caused by: java.lang.ClassNotFoundException: Class com.cxense.Cxense not found
	at com.google.devtools.build.android.desugar.HeaderClassLoader.findClass(HeaderClassLoader.java:53)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 20 more
Comment 7 Jonah 2017-11-02 07:57:06 UTC
Hmm weird, he can't find the main class. @Atsushi how can I reproduce this?
Comment 8 Jonah 2017-11-09 07:55:26 UTC
@Atsushi is it possible that the main classes is linked out of the library while compiling?
Comment 9 Atsushi Eno 2017-11-09 15:27:33 UTC
What I was running was desugar_deploy.jar, and it definitely has main.

Anyways desugar support is merged at https://github.com/xamarin/xamarin-android/pull/991
Comment 10 Atsushi Eno 2017-11-09 15:33:34 UTC
One thing to note is that no one can compile Java sources that contain lambda expressions using android.jar, even with java8 javac. So for normal good Android developers who makes sure to compile their libraries with Android API, lambdas don't exist.
Comment 11 Jonah 2017-11-14 07:20:23 UTC
@Atsushi so I have to ask the developers of the 3th party library to remove the lambda? Is that the only solution?