Bug 10317 - some dynamic code causes runtime crash
Summary: some dynamic code causes runtime crash
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: MSBuild ()
Version: 4.6.x
Hardware: PC Linux
: Normal major
Target Milestone: ---
Assignee: Jonathan Pryor
URL:
Depends on:
Blocks:
 
Reported: 2013-02-15 01:53 UTC by Atsushi Eno
Modified: 2013-06-28 14:55 UTC (History)
4 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Scratch.DynamicCSharp.zip (10.92 KB, application/zip)
2013-06-26 23:11 UTC, Jonathan Pryor
Details
mono-logging.patch (4.49 KB, patch)
2013-06-28 13:56 UTC, Jonathan Pryor
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Atsushi Eno 2013-02-15 01:53:28 UTC
monodroid-samples/DynamicTest is now broken on Release mode, due to the error below:

error opening trace file: No such file or directory (2)
GREF GC Threshold: 1800
Missing method .ctor in assembly System.Core.dll, type System.Runtime.InteropServices.ComVisibleAttribute
Can't find custom attr constructor image: System.Core.dll mtoken: 0x0a0010b2
MainActivity..ctor(): time: 1360910778129
MainActivity.onCreate: time: 1360910778183
resolveUri failed on bad bitmap uri: 
Emulator without GPU emulation detected.
Missing method .ctor in assembly Microsoft.CSharp.dll, type System.Runtime.InteropServices.ComVisibleAttribute
Can't find custom attr constructor image: Microsoft.CSharp.dll mtoken: 0x0a0000a1
Missing method .ctor in assembly Mono.CSharp.dll, type System.Runtime.InteropServices.ComVisibleAttribute
Can't find custom attr constructor image: Mono.CSharp.dll mtoken: 0x0a000785
Missing method .ctor in assembly Mono.CSharp.dll, type System.Runtime.InteropServices.ComVisibleAttribute
Can't find custom attr constructor image: Mono.CSharp.dll mtoken: 0x0a000785

Unhandled Exception:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.TypeLoadException: Could not load type 'System.Runtime.InteropServices.ComVisibleAttribute' from assembly 'Mono.CSharp'.
  at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0 
  at Mono.CSharp.ReflectionImporter.ImportAssembly (System.Reflection.Assembly assembly, Mono.CSharp.RootNamespace targetNamespace) [0x00000] in <filename unknown>:0 
  at Microsoft.CSharp.RuntimeBinder.DynamicContext.Create () [0x00000] in <filename unknown>:0 
  at Microsoft.CSharp.RuntimeBinder.CSharpInvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject target, System.Dynamic.DynamicMetaObject[] args, System.Dynamic.DynamicMetaObject errorSuggestion) [0x00000] in <filename unknown>:0 
  at System.Dynamic.InvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject target, System.Dynamic.DynamicMetaObject[] args) [0x00000] in <filename unknown
[ERROR] FATAL UNHANDLED EXCEPTION: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.TypeLoadException: Could not load type 'System.Runtime.InteropServices.ComVisibleAttribute' from assembly 'Mono.CSharp'.
  at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0 
  at Mono.CSharp.ReflectionImporter.ImportAssembly (System.Reflection.Assembly assembly, Mono.CSharp.RootNamespace targetNamespace) [0x00000] in <filename unknown>:0 
  at Microsoft.CSharp.RuntimeBinder.DynamicContext.Create () [0x00000] in <filename unknown>:0 
  at Microsoft.CSharp.RuntimeBinder.CSharpInvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject target, System.Dynamic.DynamicMetaObject[] args, System.Dynamic.DynamicMetaObject errorSuggestion) [0x00000] in <filename unknown>:0 
  at System.Dynamic.InvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject target, System.Dynamic.DynamicMetaObject[] args) [0x00000] in <fil

(This incomplete trace is the actual output I got.)

It is most likely because of the internal compiler changes that are done without any dynamic consideration.

Adding mscorlib.dll to <LinkSkip> in the project property workarounds this issue, but it should not be really a final design. mscorlib is HUGE library that should be stripped.

I also checked the linker implementation and almost disabled "remove extra attributes" step but that only results in another NRE. Nor this LinkDescription worked: https://gist.github.com/atsushieno/d7ac88d2aecd28a7c741

I don't think our compiler/runtime team can fix the issue (e.g. by very carefully change code to not break dynamic and linking), they will keep breaking things, so the only place I see we can fix this issue is our linker.
Comment 1 Marek Safar 2013-02-18 05:46:31 UTC
This should be possible to fix but will require some runtime changes and less aggressive compiler dynamic context setup than we have now.
Comment 2 Atsushi Eno 2013-02-18 07:21:12 UTC
To make it clear, it is not any fault in compiler/runtime changes (to my understanding they supposed to change) e.g. some reflection (typename) based type lookup breaks things because we have hardcoded attribute removals from code at packaging time. Making it work *now* would be certainly possible, but keeping things work is not likely.
Comment 3 Miguel de Icaza [MSFT] 2013-02-18 10:26:13 UTC
Atsushi, would you mind uploading this assembly?

So this means that we have the ComVisibleAttribtue in the assembly, but we removed the constructor. 

Seastien, does that sound right?
Comment 4 Miguel de Icaza [MSFT] 2013-02-18 10:26:37 UTC
Second question: why are we even keeping the ComVisible attribute?
Comment 5 Sebastien Pouliot 2013-02-18 10:58:00 UTC
M4A linker has grown different from MT/XM but...

Missing method .ctor in assembly Microsoft.CSharp.dll, type
System.Runtime.InteropServices.ComVisibleAttribute
...
System.TypeLoadException: Could not load type
'System.Runtime.InteropServices.ComVisibleAttribute' from assembly
'Mono.CSharp'.

^ that may be bad error messages _or_ there's more than one implementation of the type. If it's a second implementation then the linker directives (on mscorlib.dll) won't help a bit for Mono.CSharp.dll

Also...

* there's a step (at least in MT) to remove attributes that are (generally) not needed at runtime. This includes removing [ComVisible] *attributes* but not the type itself;

* the ComVisibleAttribute type will be removed only the nothing refers to it. It's likely there's some references, e.g. typeof (ComVisibleAttribute), inside the BCL so the type is part of the final binary;

* the default ComVisibleAttribute .ctor will be removed if not used (by IL). Now if it's used only thru reflection then the linker will remove it (if not preserved). The stack trace hints at reflection usage:

  at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0
Comment 6 Atsushi Eno 2013-02-19 04:14:50 UTC
I'm going to upload the assembly in the next private comment.

This is the stack trace (NRE) when we remove "remove attributes" linker step (it is Release mode build by nature, so no line info. Also somehow incomplete like the first stack trace):

error opening trace file: No such file or directory (2)
GREF GC Threshold: 1800
MainActivity..ctor(): time: 1361265092883
MainActivity.onCreate: time: 1361265092987
resolveUri failed on bad bitmap uri: 
Emulator without GPU emulation detected.
* Assertion at /svn/mono-mobile-master/mono/metadata/marshal.c:4529, condition `method' not met
Stacktrace:

  at (wrapper managed-to-native) System.MonoCustomAttrs.GetCustomAttributesDataInternal (System.Reflection.ICustomAttributeProvider) <0xffffffff>
  at System.MonoCustomAttrs.GetCustomAttributesData (System.Reflection.ICustomAttributeProvider) <0x00017>
  at System.Reflection.CustomAttributeData.GetCustomAttributes (System.Reflection.Assembly) <0x00013>
  at Mono.CSharp.ImportedAssemblyDefinition.ReadAttributes () <0x0004b>
  at Mono.CSharp.MetadataImporter.GetAssemblyDefinition (System.Reflection.Assembly) <0x0008f>
  at Mono.CSharp.ReflectionImporter.ImportAssembly (System.Reflection.Assembly,Mono.CSharp.RootNamespace) <0x0002b>
  at Microsoft.CSharp.RuntimeBinder.DynamicContext.Create () <0x0022b>
  at Microsoft.CSharp.RuntimeBinder.CSharpInvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject,System.Dynamic.DynamicMetaObject[],System.Dynamic.DynamicMetaObject) <0x0002f>
  at System.Dynamic.InvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject,System.Dynamic.DynamicMetaObject[]) <0x00018>
  at System.Dynamic.DynamicMetaObject.BindInvokeMember (System.Dynamic.InvokeMemberBinder,System.Dynamic.DynamicMetaObject[]) <0x00027>
  at System.Dynamic.InvokeMemberBinder.Bind (System.Dynamic.DynamicMetaObject,System.Dynamic.DynamicMetaObject[]) <0x00042>
  at System.Dynamic.DynamicMetaObjectBinder.Bind (object[],System.Collections.ObjectModel.ReadOnlyCollection`1<System.Linq.Expressions.ParameterExpression>,System.Linq.Expressions.LabelTarget) <0x00145>
  at System.Runtime.CompilerServices.CallSiteBinder.BindCore<T> (System.Runtime.CompilerServices.CallSite`1<T>,object[]) <0x00090>
  at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid3<T0, T1, T2> (System.Runtime.CompilerServices.CallSite,T0,T1,T2) <0x00427>
  at DynamicTest.MainActivity/<OnCreate>c__AnonStorey0.<>m__0 (object,System.Net.DownloadStringCompletedEventArgs) <0x0092b>
  at System.Net.WebClient.OnDownloadStringCompleted (System.Net.DownloadStringCompletedEventArgs) <0x0002e>
  at System.Net.WebClient.<DownloadStringAsync>m__F (object) <0x00126>
  at System.Threading.Thread.StartInternal () <0x0007f>
  at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <0xffffffff>

Unhandled Exception:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object
  at (wrapper managed-to-native) System.MonoCustomAttrs:GetCustomAttributesDataInternal (System.Reflection.ICustomAttributeProvider)
  at System.MonoCustomAttrs.GetCustomAttributesData (ICustomAttributeProvider obj) [0x00000] in <filename unknown>:0 
  at System.Reflection.CustomAttributeData.GetCustomAttributes (System.Reflection.Assembly target) [0x00000] in <filename unknown>:0 
  at Mono.CSharp.ImportedAssemblyDefinition.ReadAttributes () [0x00000] in <filename unknown>:0 
  at Mono.CSharp.MetadataImporter.GetAssemblyDefinition (System.Reflection.Assembly assembly) [0x00000] in <filename unknown>:0 
  at Mono.CSharp.ReflectionImporter.ImportAssembly (System.Reflection.Assembly assembly, Mono.CSharp.RootNamespace targetNamespace) [0x00000] in <filename unknown>:0 
  at Microsoft.CSharp.RuntimeBi
[ERROR] FATAL UNHANDLED EXCEPTION: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object
  at (wrapper managed-to-native) System.MonoCustomAttrs:GetCustomAttributesDataInternal (System.Reflection.ICustomAttributeProvider)
  at System.MonoCustomAttrs.GetCustomAttributesData (ICustomAttributeProvider obj) [0x00000] in <filename unknown>:0 
  at System.Reflection.CustomAttributeData.GetCustomAttributes (System.Reflection.Assembly target) [0x00000] in <filename unknown>:0 
  at Mono.CSharp.ImportedAssemblyDefinition.ReadAttributes () [0x00000] in <filename unknown>:0 
  at Mono.CSharp.MetadataImporter.GetAssemblyDefinition (System.Reflection.Assembly assembly) [0x00000] in <filename unknown>:0 
  at Mono.CSharp.ReflectionImporter.ImportAssembly (System.Reflection.Assembly assembly, Mono.CSharp.RootNamespace targetNamespace) [0x00000] in <filename unknown>:0 
  at Microsoft.CSh
Comment 10 Marek Safar 2013-02-20 16:44:14 UTC
Are you sure the attached mscorlib is the correct one, it has full ComVisibleAttribute type still inside it and I can use the assembly without any error locally.

I'd like to track down

 * Assertion at /svn/mono-mobile-master/mono/metadata/marshal.c:4529, condition `method' not met

can you attach the assembly which caused the problem or is there any other problem?
Comment 11 Atsushi Eno 2013-02-21 04:00:21 UTC
As I wrote on comment #8, the assembly is *after* I removed "remove attributes" step on the linker. Maybe that was not obvious for you - that is, the attached mscorlib.dll does *not* remove ComVisibleAttribute anymore.

I can attach an assembly that does link and strip ComVisibleAttribute, but I don't see any point of doing that. It is obvious that such a mscorlib.dll doesn't work.
Comment 12 Marek Safar 2013-02-21 06:17:09 UTC
I did some digging on my own and this looks similar to #10263
Comment 13 Miguel de Icaza [MSFT] 2013-03-11 10:42:38 UTC
Atsushi,

So one question: with the assembly from #8 dynamic then works?

And could you provide the assembly without the "remove attributes" step, so we can see what is taking place.
Comment 15 Atsushi Eno 2013-03-19 10:00:44 UTC
Oh I missed the question on comment #13: no, the assembly on #8 doesn't work either. And it is (and the new one at #14 is) the exactly one that is "linked without the remove attributes step".
Comment 16 Jonathan Pryor 2013-06-26 23:11:16 UTC
Created attachment 4210 [details]
Scratch.DynamicCSharp.zip

I cannot reproduce this bug anymore. Perhaps it was fixed without us realizing it?

For starters, the original testcase of monodroid-samples/DynamicTest doesn't _build_ for me, never mind failing at runtime:

    $ xbuild /p:Configuration=Release
    : error CS2001: Source file `System.Json/Properties/Resources.Designer.cs' could not be found

From commit monodroid-samples/31008c12 I see that it's supposed to be a generated file; perhaps xbuild lacks the support to generate it? Perhaps this sample requires MSBuild/Windows .NET?

If so, is there some other mechanism that can be used?

Since DynamicTest didn't work for me, I cribbed a new `dynamic` test, this one involving XML from a local acquaintance of mine. Thus the new Scratch.DynamicCSharp.zip attachment, which builds (yay) and runs in Release configuration without generating any runtime errors.
Comment 17 Jonathan Pryor 2013-06-26 23:12:04 UTC
Eno: Are you able to build monodroid-samples/DynamicTest? Does it still fail for you? (Please see Comment #16.)
Comment 18 Atsushi Eno 2013-06-27 01:45:18 UTC
It was not building with XS because it lacks designer code generation support for .resx resources. I explicitly added the designer.cs in the repo so now it should build with XS too.

And yes the bug still reproduced. The same runtime crash.

I also ran your repro at comment #16 and it did work(!). So it is likely some internals of DLR is damaged by some changes (from months ago; it was before Xamarin 2.0 launch).
Comment 19 Jonathan Pryor 2013-06-27 12:00:07 UTC
Eno: Thank you for fixing monodroid-samples.

Current master still fails similar to Comment #6, but without the NullReferenceException:

> * Assertion at /Users/builder/data/lanes/monodroid-mlion-master/294d4619/source/mono/mono/metadata/marshal.c:4806, condition `method' not met
> Stacktrace:
> 
>   at <unknown> <0xffffffff>
>   at (wrapper managed-to-native) System.MonoCustomAttrs.GetCustomAttributesDataInternal (System.Reflection.ICustomAttributeProvider) <0xffffffff>
>   at System.MonoCustomAttrs.GetCustomAttributesData (System.Reflection.ICustomAttributeProvider) <0x0001f>
>   at System.Reflection.CustomAttributeData.GetCustomAttributes (System.Reflection.Assembly) <0x00013>
>   at Mono.CSharp.ImportedAssemblyDefinition.ReadAttributes () <0x0003f>
>   at Mono.CSharp.MetadataImporter.GetAssemblyDefinition (System.Reflection.Assembly) <0x00083>
>   at Mono.CSharp.ReflectionImporter.ImportAssembly (System.Reflection.Assembly,Mono.CSharp.RootNamespace) <0x00033>
>   at Microsoft.CSharp.RuntimeBinder.DynamicContext.Create () <0x0021f>
>   at Microsoft.CSharp.RuntimeBinder.CSharpInvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject,System.Dynamic.DynamicMetaObject[],System.Dynamic.DynamicMetaObject) <0x00037>
>   at System.Dynamic.InvokeMemberBinder.FallbackInvokeMember (System.Dynamic.DynamicMetaObject,System.Dynamic.DynamicMetaObject[]) <0x00033>
>   at System.Dynamic.DynamicMetaObject.BindInvokeMember (System.Dynamic.InvokeMemberBinder,System.Dynamic.DynamicMetaObject[]) <0x0003f>
>   at System.Dynamic.InvokeMemberBinder.Bind (System.Dynamic.DynamicMetaObject,System.Dynamic.DynamicMetaObject[]) <0x0005b>
>   at System.Dynamic.DynamicMetaObjectBinder.Bind (object[],System.Collections.ObjectModel.ReadOnlyCollection`1<System.Linq.Expressions.ParameterExpression>,System.Linq.Expressions.LabelTarget) <0x0018b>
>   at System.Runtime.CompilerServices.CallSiteBinder.BindCore<T> (System.Runtime.CompilerServices.CallSite`1<T>,object[]) <0x00093>
>   at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid3<T0, T1, T2> (System.Runtime.CompilerServices.CallSite,T0,T1,T2) <0x0040b>
>   at (wrapper delegate-invoke) System.Action`4.invoke_void__this___T1_T2_T3_T4 (T1,T2,T3,T4) <0xffffffff>
>   at DynamicTest.MainActivity/<OnCreate>c__AnonStorey0.<>m__0 (object,System.Net.DownloadStringCompletedEventArgs) <0x00993>
>   at System.Net.WebClient.OnDownloadStringCompleted (System.Net.DownloadStringCompletedEventArgs) <0x0003f>
>   at System.Net.WebClient.<DownloadStringAsync>m__16 (object) <0x00177>
>   at System.Threading.Thread.StartInternal () <0x000b7>
>   at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <0xffffffff>
> 
> =================================================================
> Got a SIGSEGV while executing native code. This usually indicates
> a fatal error in the mono runtime or one of the native libraries 
> used by your application.
> =================================================================
> 
> I/ActivityManager(  390): Process DynamicTest.DynamicTest (pid 2268) has died.

Looks like I'll need to modify mscorlib.dll to figure out what it's trying to inflate...
Comment 20 Jonathan Pryor 2013-06-28 12:04:11 UTC
WORKAROUND: The sample works if a new file is added to DynamicTest.csproj, Build action LinkDescription, with the following contents:

  <?xml version="1.0" encoding="UTF-8" ?>
  <linker>
    <assembly fullname="mscorlib">
      <type fullname="System.Runtime.InteropServices.ComVisibleAttribute">
        <method name=".ctor" />
      </type>
      <type fullname="System.Reflection.CustomAttributeData">
        <method name=".ctor" />
      </type>
      <type fullname="System.Reflection.CustomAttributeTypedArgument">
        <method name=".ctor" />
      </type>
      <type fullname="System.Reflection.CustomAttributeTypedArgument">
        <method name=".ctor" />
      </type>
    </assembly>
  </linker>
Comment 21 Atsushi Eno 2013-06-28 13:27:56 UTC
It doesn't change the runtime crasher situation (neither did relevant internal fix), unless I have some bogus file. I ran monodis against the linked mscorlib.dll which is about 1.5MB (so I guess it is the expectedly linked one) but it core dumped.
Comment 22 Jonathan Pryor 2013-06-28 13:50:35 UTC
Fixed in master/7e00daf1 by merging Comment #20's workaround.
Comment 23 Jonathan Pryor 2013-06-28 13:56:48 UTC
Created attachment 4226 [details]
mono-logging.patch

> It doesn't change the runtime crasher situation (neither did relevant internal fix)

It did for me. Through some logging in libmonosgen-2.0.a (see mono-logging.patch), I saw that the the runtime asserts were generated because various constructors it was looking for (and asserting that they exist) didn't exist. As soon as I changed the linker to preserve all the types it was looking for, the program worked without any asserts.

> I ran monodis against the linked mscorlib.dll which is about 1.5MB
> (so I guess it is the expectedly linked one) but it core dumped.

`monodis` is incredibly flaky. Film at 11.

Part of this was because monodis/mono was requiring that various COM types exist from mscorlib.dll. Kumpera fixed this in Bug #12609, so monodis should be a tad less flaky with more recent versions.

(`monodis is _still_ incredibly flaky.)
Comment 24 Atsushi Eno 2013-06-28 14:55:23 UTC
OK, actually with another attempt with the latest master it did work somehow. No idea what's the difference though.