Bug 56307 - FatAppFiles test failing with 2017-04
Summary: FatAppFiles test failing with 2017-04
Status: RESOLVED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: General ()
Version: XI 10.12 (d15-3)
Hardware: PC Mac OS
: Normal normal
Target Milestone: 15.3
Assignee: Rolf Bjarne Kvinge [MSFT]
URL:
: 56308 ()
Depends on:
Blocks:
 
Reported: 2017-05-12 17:41 UTC by Andi McClure
Modified: 2017-05-17 12:29 UTC (History)
4 users (show)

Tags: 2017-04
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Andi McClure 2017-05-12 17:41:49 UTC
On our PR to bump macios mono to 2017-04

https://github.com/xamarin/xamarin-macios/pull/1960#issuecomment-300890969

We upgraded the mono from 2017-04 commit 197c23 to 4960d5. Six tests are now failing on Jenkins:

https://jenkins.mono-project.com/job/xamarin-macios-pr-builder/3706/

One of these tests is the FatAppFiles test. The failure, which I do not understand, is

https://jenkins.mono-project.com/job/xamarin-macios-pr-builder/3706/testReport/junit/(root)/Xamarin/MTouch_FatAppFiles/
                                                MESSAGE:
                                                  expected files
  Expected: <empty>
  But was:  < "System.dll", "System.aotdata.armv7", "System.aotdata.arm64" >

Although this problem seems to have been triggered by a change to the runtime, I do not believe the runtime team is in an adequate place to diagnose the problem. I do not know what the intent of this test is.

I am able to reproduce the problem on a local machine with a build of macios 60485cfd2c. The log is below.

Running mtouch.mtouch.Xamarin.MTouch.FatAppFiles ...
Test configuration:
  MONOTOUCH_PREFIX=/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git
  IOS_DESTDIR=/Users/andi/work/g/xamarin-macios/_ios-build
  SDK_VERSION=10.3
  XCODE_ROOT=/Applications/Xcode83.app/Contents/Developer
  XCODE5_ROOT=/Applications/Xcode511.app/Contents/Developer
  XCODE6_ROOT=/Applications/Xcode601.app/Contents/Developer Exists=False
  INCLUDE_IOS=True
  INCLUDE_MAC=True
  INCLUDE_TVOS=True
  INCLUDE_WATCHOS=True
/Library/Frameworks/Mono.framework/Commands/mcs  -lib:/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS  /noconfig /t:exe /nologo /out:/Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.exe /r:/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/Xamarin.iOS.dll /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.cs 
5/12/2017 12:51:44 PM Executed in 00:00:00.3211990: /Library/Frameworks/Mono.framework/Commands/mcs  -lib:/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS  /noconfig /t:exe /nologo /out:/Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.exe /r:/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/Xamarin.iOS.dll /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.cs 
/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mtouch  --dev /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.app --sdkroot /Applications/Xcode83.app/Contents/Developer  --sdk 10.3 --msym:false --dsym:false /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.exe -r:/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/Xamarin.iOS.dll --abi armv7,arm64 --cache /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory0/mtouch-test-cache
5/12/2017 12:52:04 PM Executed in 00:00:20.0086307: /Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mtouch  --dev /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.app --sdkroot /Applications/Xcode83.app/Contents/Developer  --sdk 10.3 --msym:false --dsym:false /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory/testApp.exe -r:/Users/andi/work/g/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/Xamarin.iOS.dll --abi armv7,arm64 --cache /Users/andi/work/g/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory0/mtouch-test-cache
Comment 1 Rolf Bjarne Kvinge [MSFT] 2017-05-17 10:18:39 UTC
The reason this test is failing is that System.dll and Mono.Security.dll are linked away (the test verifies that certain files are present in a built apps, and the files corresponding with these assemblies are no longer present).

This obviously raises the question: why weren't these assemblies linked away before?

Running "monodis --typedef" on the linked assemblies from the test as run on master, reveals this:

> $ monodis --typedef System.dll 
> Typedef Table
> 1: (null) (flist=1, mlist=1, flags=0x0, extends=0x0)
> 2: ObjCRuntime.INativeObject (flist=1, mlist=1, flags=0xa0, extends=0x0)
> 3: Mono.Net.CFObject (flist=1, mlist=2, flags=0x100000, extends=0x5)
> 4: Mono.Net.CFArray (flist=4, mlist=19, flags=0x100, extends=0xc)
> 5: Mono.Net.CFNumber (flist=5, mlist=32, flags=0x100100, extends=0xc)
> 6: Mono.Net.CFRange (flist=5, mlist=41, flags=0x100108, extends=0x25)
> 7: Mono.Net.CFString (flist=7, mlist=42, flags=0x100100, extends=0xc)
> 8: Mono.Net.CFData (flist=8, mlist=53, flags=0x100100, extends=0xc)
> 9: Mono.Net.CFDictionary (flist=8, mlist=63, flags=0x0, extends=0xc)
> 10: Mono.Net.CFMutableDictionary (flist=10, mlist=75, flags=0x100100, extends=0x24)
> 11: Mono.Net.CFUrl (flist=10, mlist=80, flags=0x100100, extends=0xc)
> 12: Mono.Net.CFRunLoop (flist=10, mlist=83, flags=0x100100, extends=0xc)
> 13: Mono.Net.CFBoolean (flist=10, mlist=94, flags=0x100, extends=0x5)
> 14: Mono.AppleTls.SecCertificate (flist=13, mlist=106, flags=0x100100, extends=0x5)
> 15: Mono.AppleTls.SecIdentity (flist=14, mlist=122, flags=0x100, extends=0x5)
> 16: Mono.AppleTls.SecIdentity/ImportOptions (flist=19, mlist=134, flags=0x100105, extends=0x5)
> 17: Mono.AppleTls.SecKey (flist=19, mlist=134, flags=0x100100, extends=0x5)
> 18: Mono.AppleTls.SecStatusCode (flist=21, mlist=141, flags=0x100, extends=0x69)
> 19: Mono.AppleTls.SecTrustResult (flist=395, mlist=141, flags=0x100, extends=0x69)
> 20: Mono.AppleTls.SecImportExport (flist=404, mlist=141, flags=0x100100, extends=0x5)
> 21: Mono.AppleTls.SecImportExport/<>c (flist=404, mlist=144, flags=0x102103, extends=0x5)
> 22: Mono.AppleTls.SecPolicy (flist=406, mlist=147, flags=0x100100, extends=0x5)
> 23: Mono.AppleTls.SecTrust (flist=407, mlist=154, flags=0x100100, extends=0x5)
> 24: System.Security.Cryptography.OidGroup (flist=408, mlist=174, flags=0x101, extends=0x69)
> 25: System.Security.Cryptography.Oid (flist=420, mlist=174, flags=0x100101, extends=0x5)
> 26: System.Security.Cryptography.CAPI (flist=423, mlist=176, flags=0x100180, extends=0x5)
> 27: System.Security.Cryptography.AsnEncodedData (flist=423, mlist=178, flags=0x100101, extends=0x5)
> 28: System.Security.Cryptography.X509Certificates.X509Utils (flist=424, mlist=179, flags=0x100100, extends=0x5)
> 29: System.Security.Cryptography.X509Certificates.PublicKey (flist=424, mlist=181, flags=0x100101, extends=0x5)
> 30: System.Security.Cryptography.X509Certificates.X509Certificate2 (flist=429, mlist=188, flags=0x102101, extends=0x51)
> 31: System.Security.Cryptography.X509Certificates.X509Certificate2Impl (flist=431, mlist=204, flags=0x100080, extends=0x55)
> 32: System.Security.Cryptography.X509Certificates.X509CertificateCollection (flist=431, mlist=209, flags=0x102101, extends=0x6d)
> 33: System.Security.Cryptography.X509Certificates.X509CertificateCollection/X509CertificateEnumerator (flist=431, mlist=212, flags=0x100102, extends=0x5)
> 34: System.Security.Cryptography.X509Certificates.X509Helper2 (flist=432, mlist=217, flags=0x100180, extends=0x5)
> 35: <PrivateImplementationDetails> (flist=432, mlist=218, flags=0x100, extends=0x5)
> 36: <PrivateImplementationDetails>/__StaticArrayInitTypeSize=9 (flist=433, mlist=219, flags=0x113, extends=0x25)
> 
> $ monodis --typedef Mono.Security.dll 
> Typedef Table
> 1: (null) (flist=1, mlist=1, flags=0x0, extends=0x0)
> 2: Mono.Security.ASN1 (flist=1, mlist=1, flags=0x100101, extends=0x5)> 

Which means that all these types were linked away in 2017-04.

@Sebastien, do you know if this is the expected behavior now?

The test app is just a "Console.WriteLine (typeof (ObjCRuntime.Runtime));", so it's not using any networking.
Comment 2 Rolf Bjarne Kvinge [MSFT] 2017-05-17 10:19:20 UTC
*** Bug 56308 has been marked as a duplicate of this bug. ***
Comment 3 Rolf Bjarne Kvinge [MSFT] 2017-05-17 10:56:43 UTC
This is the change that caused this to surface: https://github.com/mono/mono/commit/4960d5d2a28a08476ee4239e1746f04afce41c13

Some of the above types from System.dll implemented ObjCRuntime.INativeObject (from System.dll), which our linker detected as implementing ObjCRuntime.INativeObject (from Xamarin.iOS.dll), so these types were treated as custom NSObject subclasses, and the MarkNSObjects linker step would mark them.

With that change, these types now implement ObjCRuntimeInternal.INativeObject, and the linker does not treat them as custom NSObject subclasses anymore.

I think the new behavior is correct: these types do not actually inherit from the real NSObject/INativeObject, so the linker should not treat them as such.

This may run into different bugs because the linker might now remove more stuff than before, but that would be a different issue.
Comment 4 Rolf Bjarne Kvinge [MSFT] 2017-05-17 11:06:02 UTC
Fixed: https://github.com/xamarin/xamarin-macios/pull/1960/commits/559f7e92503ec794ca01c69bcda6091363d9e3d8

@Sebastien, do you agree with the new behavior being correct? If so, we can close this bug.
Comment 5 Sebastien Pouliot 2017-05-17 12:18:10 UTC
@Rolf, that looks fine. I think the removal of the old TLS managed code (Mono.Security) and the move/copy of AppleTLS into System.dll was the original trigger. 

The INativeObject copy just camouflaged a large part of the change until you fixed the compiler warnings :)
Comment 6 Rolf Bjarne Kvinge [MSFT] 2017-05-17 12:29:44 UTC
Closing then 👍