Bug 57645 - "The "LinkAssemblies" task failed unexpectedly" due to a NullReferenceException in Mono.Linker's `ProcessLibrary()` method when project references certain assemblies, for example "Mono.Data.Sqlite.Portable" NuGet package
Summary: "The "LinkAssemblies" task failed unexpectedly" due to a NullReferenceExcepti...
Status: VERIFIED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: Linker ()
Version: 7.3 (15.2)
Hardware: PC All
: --- major
Target Milestone: 15.4
Assignee: Radek Doulik
URL:
: 56702 57733 58182 ()
Depends on:
Blocks:
 
Reported: 2017-06-20 21:04 UTC by Brendan Zagaeski (Xamarin Team, assistant)
Modified: 2017-09-26 19:24 UTC (History)
20 users (show)

Tags:
Is this bug a regression?: Yes
Last known good build: Xamarin.Android 7.2.0-7 (b16fb82)


Attachments
Test case (13.13 KB, application/zip)
2017-06-20 21:04 UTC, Brendan Zagaeski (Xamarin Team, assistant)
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
VERIFIED FIXED

Description Brendan Zagaeski (Xamarin Team, assistant) 2017-06-20 21:04:05 UTC
Created attachment 23022 [details]
Test case

"The "LinkAssemblies" task failed unexpectedly" due to a NullReferenceException in Mono.Linker's `ProcessLibrary()` method when project references certain assemblies, for example "Mono.Data.Sqlite.Portable" NuGet package




## Regression status

BAD:  Xamarin.Android 7.3.1-2 (9dbc4c5) "15.2.2 release"
GOOD: Xamarin.Android 7.2.0-7 (b16fb82) "15.1   release"




## Steps followed to test

1. Create a new "Visual C# > Android > Blank App (Android)" project.

2. Add the "Mono.Data.Sqlite.Portable" version 1.0.3.5 NuGet package.

3. Attempt to build the project in the Release configuration.

OR

1. Restore the NuGet packages in the attached test case (for example by opening it in Visual Studio for Mac).

2. Attempt to build the test case in the Release configuration, for example on the command line:

msbuild /t:Build /p:Configuration=Release BlankAppAndroid1.sln




## BAD Results with Xamarin.Android 7.3.1.2 (9dbc4c5) on either Windows or Mac

> The "LinkAssemblies" task failed unexpectedly.
> System.NullReferenceException: Object reference not set to an instance of an object.
>    at Mono.Linker.Steps.ResolveFromAssemblyStep.ProcessLibrary(LinkContext context, AssemblyDefinition assembly)
>    at Mono.Tuner.CustomizeActions.ProcessUserAssembly(AssemblyDefinition assembly)
>    at Mono.Tuner.CustomizeActions.ProcessAssembly(AssemblyDefinition assembly)
>    at Mono.Linker.Steps.BaseStep.Process(LinkContext context)
>    at Mono.Linker.Pipeline.Process(LinkContext context)
>    at MonoDroid.Tuner.Linker.Run(Pipeline pipeline, LinkContext context)
>    at MonoDroid.Tuner.Linker.Process(LinkerOptions options, LinkContext& context)
>    at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)
>    at Xamarin.Android.Tasks.LinkAssemblies.Execute()
>    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
>    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()

It seems that in particular it's the "System.Data.Portable.dll" assembly from the NuGet package that causes the problem.  Removing that reference stops the problem.

> <Reference Include="System.Data.Portable, Version=4.0.0.0, Culture=neutral, PublicKeyToken=59e704a76bc4613a, processorArchitecture=MSIL">
>   <HintPath>..\packages\Mono.Data.Sqlite.Portable.1.0.3.5\lib\MonoAndroid\System.Data.Portable.dll</HintPath>
>   <Private>True</Private>
> </Reference>



## Additional testing environment info (brief)

Mono 5.0.1.1 (2017-02/5077205) (64-bit)

Java JDK 8u65 (1.8.0_65) (64-bit)

Android SDK Tools Version:  25.2.5
Android SDK Platform Tools: 25.0.3
Android SDK Build Tools:    25

macOS 10.12.5
Comment 1 Brendan Zagaeski (Xamarin Team, assistant) 2017-06-20 21:13:39 UTC
*** Bug 56702 has been marked as a duplicate of this bug. ***
Comment 2 Christoph Engels 2017-06-21 11:55:33 UTC
Coming from bug 56702. Like stated over there, I have the same problem, but I wanted to add that neither the failing project nor my other (working) projects the failing project depends on include a reference to System.Data.Portable.dll

@Brendan Zagaeski: How did you determine the problematic reference? Is there a chance for me to identify it? I cannot remove references because then my project will most likely not compile anymore. I wonder if the name of the problematic assembly appears in any log, maybe in a verbose mode?
Comment 3 Brendan Zagaeski (Xamarin Team, assistant) 2017-06-21 16:39:44 UTC
> How did you determine the problematic reference?

I think it would be possible to attach the Visual Studio debugger to the MSBuild process to break on the NullReferenceException and then inspect the call stack to find out which assembly is being processed at the time of call to `ProcessLibrary()`.  But I used a simpler guess-and-check approach where I started with a blank Android app project and then individually tested adding a reference to each individual assembly from the original complete project until the new project started showing the issue.
Comment 4 Radek Doulik 2017-07-31 14:30:04 UTC
It looks like duplicate of #58182

*** This bug has been marked as a duplicate of bug 58182 ***
Comment 7 Radek Doulik 2017-07-31 17:28:35 UTC
*** Bug 58182 has been marked as a duplicate of this bug. ***
Comment 8 Radek Doulik 2017-07-31 17:33:17 UTC
OK, I reopened this bug. Please note https://bugzilla.xamarin.com/show_bug.cgi?id=58182#c12 which explain, what is the issue here. Matthew, would it be possible to fix the Mono.Data.Sqlite.Portable and release new version?

I will meanwhile create PR for linker to throw exception with more information about the issue in case it will possibly happen with some other assemblies as well in future.
Comment 9 Christoph Engels 2017-07-31 17:49:46 UTC
In reply to action #4, marking this a duplicate of bug 58182

My original bug 56702 was deemed a duplicate of this bug 57645.
Then this bug 57645 was deemed to be a duplicate of bug 58182.

You definitely need to stop declaring the report that existed FIRST to be the duplicate of the newer one. This doesn't make sense, it's the other way round. Flagging otherwise might even mess up the system because already collected information is disregarded and data like "since when is the bug known to exist" will be lost. :)
Comment 10 Radek Doulik 2017-07-31 18:19:31 UTC
Marek, do you know what happened to System.Data.Common.DbConnection nested types like System.Data.Common.DbConnection/DataTypes? It looks like there were removed from BCL and I am not sure if it was done on purpose (possibly when we used reference sources?) or whether it is an actual regression in our BCL.
Comment 11 Radek Doulik 2017-08-01 09:57:03 UTC
We discussed this bug with Marek and the nested forwarded type might be result of csc bug, which was fixed meanwhile in Roslyn. I will try to confirm that and create better patch for linker which might be able to handle assemblies affected by this csc bug.
Comment 12 Marek Safar 2017-08-01 14:08:03 UTC
*** Bug 57733 has been marked as a duplicate of this bug. ***
Comment 13 Marek Safar 2017-08-01 14:12:24 UTC
Indeed, this is quite complicated but we need linker workaround for this csc bug.

Native csc (pre-Roslyn version) when encountered typeforwarded type it added automatically all its nested types including private/protected/internal ones. This bug was fixed in Roslyn csc but there are still many assemblies compiled with native csc (pre VS2015) which can have typeforwarded typeref to a type which is not available because it's not included in public API.
Comment 14 Rolf Bjarne Kvinge [MSFT] 2017-08-01 14:15:23 UTC
It looks like the same thing happens for XI (when using F#): https://bugzilla.xamarin.com/show_bug.cgi?id=58063#c10
Comment 16 Radek Doulik 2017-08-01 21:54:34 UTC
This PR should fix it. https://github.com/mono/linker/pull/137
Comment 17 Radek Doulik 2017-08-01 21:58:32 UTC
To add more information, the origin of the issue is that in earlier versions of linker we didn't mark exported forwarded types at all, in ResolveFromAssemblyStep.cs.
Comment 18 Marek Safar 2017-08-03 07:08:47 UTC
*** Bug 58063 has been marked as a duplicate of this bug. ***
Comment 20 Jonathan Pryor 2017-08-09 14:57:29 UTC
Fixed in:

monodroid/master/8e3af711

https://github.com/xamarin/xamarin-android/commit/b0af9cc5465a80adaa80221082ce1e55142bc791
monodroid/d15-4/fbcc76f6
Comment 26 bulent 2017-09-03 10:27:17 UTC
When will this fix be released? Currently I'm using Visual studio 2017 with xamarin android version 7.4.0.21 and this fix isn't there?
Comment 31 Brendan Zagaeski (Xamarin Team, assistant) 2017-09-07 19:02:28 UTC
## Status update for any users watching this issue

> When will this fix be released?
- At the moment, this fix is available for Mac in the Xamarin 15.4 Preview on the Alpha channel version (in Xamarin.Android 7.5.0.13).

- The fix will also be available for Windows in the upcoming Visual Studio 2017 version 15.4.0 Preview 2 and the corresponding Alpha channel update for Visual Studio 2015.  Those updates are in-progress now.  They will be published as soon as possible, potentially within the next few business days.

- I have also asked about the possibility of having the fix backported to Visual Studio 2017 version 15.3 and the corresponding Xamarin 15.3 Release.  I will update this bug report with any updates I get about that possibility over the next couple of weeks.
Comment 32 Brendan Zagaeski (Xamarin Team, assistant) 2017-09-19 20:18:57 UTC
## Status update for any users watching this issue

The fix for this issue has now been published in Xamarin.Android 7.4.5.1, available in:

- Visual Studio 2017 version 15.3.5
- The Stable updater channel in Visual Studio 2017 for Mac
- The Stable updater channel in Xamarin for Visual Studio 2015, as part of Xamarin 4.7.9.45
Comment 33 Todd Diehl 2017-09-26 18:45:44 UTC
https://bugzilla.xamarin.com/show_bug.cgi?id=57733 was marked as a duplicate of this issue, however, it manifested on Xamarin iOS. Was this issue resolved on Xamarin.iOS as well? And if so, which version?
Comment 34 Brendan Zagaeski (Xamarin Team, assistant) 2017-09-26 19:24:09 UTC
> Was this issue resolved on Xamarin.iOS as well? And if so, which version?

Yes, the fix is included in the current Beta channel version of Xamarin.iOS (version 11.2), which is closing in on the Stable channel most likely within a few weeks.  (Apologies too that I didn't notice the applicability to iOS when I was requesting a backport after Comment 31.  Unfortunately just by chance the original scenario I was using in Comment 0 didn't hit the problem on iOS, so it had stuck in my head that Xamarin.Android was the key target for backporting.)