Bug 13620 - App started crashing on launch after upgrade to X.iOS 6.4.0.2, on a real device
Summary: App started crashing on launch after upgrade to X.iOS 6.4.0.2, on a real device
Status: RESOLVED NOT_REPRODUCIBLE
Alias: None
Product: iOS
Classification: Xamarin
Component: XI runtime ()
Version: 6.4.0
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2013-07-30 14:26 UTC by Andrew
Modified: 2013-09-30 13:22 UTC (History)
3 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
executable bin diff (chashing on the right) (14.50 KB, image/png)
2013-07-31 05:59 UTC, Andrew
Details
executable bin diff 2 (chashing on the right) (80.76 KB, image/png)
2013-07-31 06:00 UTC, Andrew
Details
symbolicated crash report (24.70 KB, text/plain)
2013-07-31 06:42 UTC, Andrew
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED NOT_REPRODUCIBLE

Description Andrew 2013-07-30 14:26:38 UTC
Hello, got a problem after upgrade: I no longer able to run my app on iphone/ipad. It crashes in about 8 of 10 attempts to start, in both release and debug configurations (simulator works fine).

Here's console after a typical crash:


  Jul 30 21:09:55 Andrews-iPhone com.apple.debugserver-199[9865] <Warning>: Connecting to com.apple.debugserver service...
  Jul 30 21:09:55 Andrews-iPhone com.apple.debugserver-199[9865] <Warning>: Got a connection, waiting for process information for launching or attaching.
  Jul 30 21:09:55 Andrews-iPhone com.apple.launchd[1] (UIKitApplication:EK4H8X28E3.com.mycompany.myapp.native[0x39a3][9866]) <Warning>: (UIKitApplication:EK4H8X28E3.com.mycompany.myapp.native[0x39a3]) Spawned and waiting for the debugger to attach before continuing...
  Jul 30 21:09:55 Andrews-iPhone amfid[9867] <Error>: Jul 30 21:09:55  SecTrustEvaluate  [leaf CriticalExtensions IssuerCommonName]
  Jul 30 21:09:55 Andrews-iPhone com.apple.debugserver-199[9865] <Warning>: Got a connection, waiting for debugger instructions.
  Jul 30 21:09:55 Andrews-iPhone com.apple.debugserver-199[9865] <Warning>: 1 +0.000000 sec [2689/0303]: error: ::ptrace (request = PT_THUPDATE, pid = 0x268a, tid = 0x2503, signal = -1) err = Invalid argument (0x00000016)
Jul 30 21:09:55 Andrews-iPhone com.apple.debugserver-199[9865] <Warning>: 2 +0.006275 sec [2689/1303]: error: ::read ( 5, 0x2ff2f9fc, 18446744069414585344 ) => -1 err = Bad file descriptor (0x00000009)
Jul 30 21:09:55 Andrews-iPhone com.apple.debugserver-199[9861] <Warning>: 1 +0.000000 sec [2685/0303]: error: ::read ( 5, 0x2fdbdebc, 1024 ) => 0 err = Bad file descriptor (0x00000009)
  Jul 30 21:09:55 Andrews-iPhone mobile_installation_proxy[9859] <Error>: main: Could not receive request from host.
  Jul 30 21:09:55 Andrews-iPhone mobile_installation_proxy[9863] <Error>: main: Could not receive request from host.
  Jul 30 21:09:56 Andrews-iPhone kernel[0] <Debug>: launchd[9866] Builtin profile: container (sandbox)
  Jul 30 21:09:56 Andrews-iPhone kernel[0] <Debug>: launchd[9866] Container: /private/var/mobile/Applications/7508F2FC-5C28-4C1D-A088-DAAE2CD9801E (sandbox)
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: Stacktrace:
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 
	Native stacktrace:
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	0   myapp                        0x008f9cf7 myapp + 8768759
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	1   myapp                        0x00907fe1 myapp + 8826849
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	2   libsystem_c.dylib                   0x3c2dde93 _sigtramp + 42
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	3   myapp                        0x00908a6f myapp + 8829551
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	4   myapp                        0x00908a6f myapp + 8829551
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	5   myapp                        0x008b9569 myapp + 8504681
Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	6   myapp                        0x008ba003 myapp + 8507395
Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	7   myapp                        0x00909023 myapp + 8831011
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	8   myapp                        0x009075f7 myapp + 8824311
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	9   myapp                        0x0090b73f myapp + 8841023
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	10  myapp                        0x0090e2eb myapp + 8852203
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	11  myapp                        0x0090e53f myapp + 8852799
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	12  myapp                        0x008e007b myapp + 8663163
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	13  myapp                        0x00947b2c myapp + 9087788
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 	14  libdyld.dylib                       0x3c296b20 <redacted> + 0
  Jul 30 21:09:56 Andrews-iPhone myapp[9866] <Error>: 
	=================================================================
	Got a SIGSEGV while executing native code. This usually indicates
	a fatal error in the mono runtime or one of the native libraries 
	used by your application.
	=================================================================
  Jul 30 21:09:57 Andrews-iPhone ReportCrash[9868] <Notice>: Formulating crash report for process myapp[9866]
  Jul 30 21:09:57 Andrews-iPhone com.apple.launchd[1] (UIKitApplication:EK4H8X28E3.com.mycompany.myapp.native[0x39a3][9866]) <Warning>: (UIKitApplication:EK4H8X28E3.com.mycompany.myapp.native[0x39a3]) Job appears to have crashed: Abort trap: 6
  Jul 30 21:09:57 Andrews-iPhone com.apple.launchd[1] (UIKitApplication:EK4H8X28E3.com.mycompany.myapp.native[0x39a3]) <Notice>: (UIKitApplication:EK4H8X28E3.com.mycompany.myapp.native[0x39a3]) Throttling respawn: Will start in 2147483646 seconds
  Jul 30 21:09:57 Andrews-iPhone backboardd[26] <Warning>: Application 'UIKitApplication:EK4H8X28E3.com.mycompany.myapp.native[0x39a3]' exited abnormally with signal 6: Abort trap: 6
  Jul 30 21:09:57 Andrews-iPhone ReportCrash[9868] <Error>: libMobileGestalt copySystemVersionDictionaryValue: Could not lookup ReleaseType from system version dictionary
  Jul 30 21:09:57 Andrews-iPhone ReportCrash[9868] <Notice>: Saved crashreport to /var/mobile/Library/Logs/CrashReporter/myapp_2013-07-30-210956_Andrews-iPhone.plist using uid: 0 gid: 0, synthetic_euid: 501 egid: 0
 

Here's the crash report:


Incident Identifier: 6458324C-F08B-49FA-9061-F518CB96D350
CrashReporter Key:   23fa444527a75f085aeb283951e201358a3baf82
Hardware Model:      iPhone4,1
Process:         myapp [9866]
Path:            /var/mobile/Applications/7508F2FC-5C28-4C1D-A088-DAAE2CD9801E/myappIOsApplication.app/myapp
Identifier:      myapp
Version:         ??? (???)
Code Type:       ARM (Native)
Parent Process:  launchd [1]

Date/Time:       2013-07-30 21:09:56.521 +0300
OS Version:      iOS 6.1.2 (10B146)
Report Version:  104

Exception Type:  EXC_BAD_ACCESS (SIGABRT)
Exception Codes: KERN_INVALID_ADDRESS at 0x00000006
Crashed Thread:  0

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x3c35d350 __pthread_kill + 8
1   libsystem_c.dylib             	0x3c2d411e pthread_kill + 54
2   libsystem_c.dylib             	0x3c31096e abort + 90
3   myapp                  	0x008f9dae 0x9d000 + 8768942
4   myapp                  	0x00907fdc 0x9d000 + 8826844
5   libsystem_c.dylib             	0x3c2dde90 _sigtramp + 40
6   myapp                  	0x00908a6a 0x9d000 + 8829546
7   myapp                  	0x00908a6a 0x9d000 + 8829546
8   myapp                  	0x008b9564 0x9d000 + 8504676
9   myapp                  	0x008b9ffe 0x9d000 + 8507390
10  myapp                  	0x0090901e 0x9d000 + 8831006
11  myapp                  	0x009075f2 0x9d000 + 8824306
12  myapp                  	0x0090b73c 0x9d000 + 8841020
13  myapp                  	0x0090e2e6 0x9d000 + 8852198
14  myapp                  	0x0090e53a 0x9d000 + 8852794
15  myapp                  	0x008e0076 0x9d000 + 8663158
16  myapp                  	0x00947b28 0x9d000 + 9087784
17  libdyld.dylib                 	0x3c296b1c start + 0

Thread 1 name:  Dispatch queue: com.apple.libdispatch-manager
Thread 1:
0   libsystem_kernel.dylib        	0x3c34d648 kevent64 + 24
1   libdispatch.dylib             	0x3c27d974 _dispatch_mgr_invoke + 792
2   libdispatch.dylib             	0x3c27d654 _dispatch_mgr_thread$VARIANT$mp + 32

Thread 2:
0   libsystem_kernel.dylib        	0x3c35dd98 __workq_kernreturn + 8
1   libsystem_c.dylib             	0x3c2abcf6 _pthread_workq_return + 14
2   libsystem_c.dylib             	0x3c2aba12 _pthread_wqthread + 362
3   libsystem_c.dylib             	0x3c2ab8a0 start_wqthread + 4

Thread 3:
0   libsystem_kernel.dylib        	0x3c35dd98 __workq_kernreturn + 8
1   libsystem_c.dylib             	0x3c2abcf6 _pthread_workq_return + 14
2   libsystem_c.dylib             	0x3c2aba12 _pthread_wqthread + 362
3   libsystem_c.dylib             	0x3c2ab8a0 start_wqthread + 4

Thread 0 crashed with ARM Thread State (32-bit):
    r0: 0x00000000    r1: 0x00000000      r2: 0x00000000      r3: 0x3de24534
    r4: 0x00000006    r5: 0x3de24b88      r6: 0x00bd6dda      r7: 0x2fd6323c
    r8: 0x00bd6a03    r9: 0x3de208a8     r10: 0x00bd6e2f     r11: 0x1cb20a3c
    ip: 0x00000148    sp: 0x2fd63230      lr: 0x3c2d4123      pc: 0x3c35d350
  cpsr: 0x00000010

Binary Images:
   0x9d000 -   0xbe4fff +myapp armv7  <cd5fa080d8e733998a940516520320fe> /var/mobile/Applications/7508F2FC-5C28-4C1D-A088-DAAE2CD9801E/myappIOsApplication.app/myapp


More chances to get it launched is to make "Clean All" before starting, but it doesn't help in 100% of launches.
It even can be 4-6 successful launches sequentially, and then a dozen crashed.

Configuration:

Xamarin.iOS
Version: 6.4.0.2 (Business Edition)
Hash: c9f7659
Branch: 

target SDK: 6.1
Link SDK assemblies only
ARMv7
no SGEN GC

Just for sure I toggled generic type sharing and incremental builds checkboxes: that doesn't affect the issue.

What should I do next for troubleshooting?
Build date: 2013-18-07 21:36:03-0400
Comment 1 Andrew 2013-07-31 05:58:33 UTC
Last hour I did about 20 sequential clean/rebuild/archive launches, two builds are crashing.
I investigated "crashing" and "not crashing" archives, there are only several differences in the executable file plus each DLL has two bytes different (which I believe a kind of build number -- changes in every archive, so unlikely it causes crashing).

I'm attaching binary comparison screenshots of the executable file just in case it can give a clue, a working file on the left, a crashing one on the right.

What do I do next to troubleshoot?

p.s. I'm not able to symbolicate Xamarin.iOS apps crash reports, despite the fact there's an archive with symbols in XCode organizer, and I followed steps found in the Internet to troubleshoot (mdimport). Any clue, what I'm doing wrong? Previously I did successful re-symbolication manually in profiling Instruments, by choosing the path to unpacked DSYM folder.
Comment 2 Andrew 2013-07-31 05:59:43 UTC
Created attachment 4494 [details]
executable bin diff (chashing on the right)

executable bin diff (chashing on the right)
Comment 3 Andrew 2013-07-31 06:00:19 UTC
Created attachment 4495 [details]
executable bin diff 2 (chashing on the right)

executable bin diff 2 (chashing on the right)
Comment 4 Andrew 2013-07-31 06:42:02 UTC
Created attachment 4496 [details]
symbolicated crash report

Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x3c698350 __pthread_kill + 8
1   libsystem_c.dylib             	0x3c60f11e pthread_kill + 54
2   libsystem_c.dylib             	0x3c64b96e abort + 90
3   myApp                  	0x00940fce mono_handle_native_sigsegv (mini-exceptions.c:2380)
4   myApp                  	0x0094f1fc mono_sigsegv_signal_handler (mini.c:6564)
5   libsystem_c.dylib             	0x3c618e90 _sigtramp + 40
6   myApp                  	0x0094fc8a mono_resolve_patch_target (mini.c:3358)
7   myApp                  	0x0094fc8a mono_resolve_patch_target (mini.c:3358)
8   myApp                  	0x00900784 load_method (aot-runtime.c:3241)
9   myApp                  	0x0090121e mono_aot_get_method (aot-runtime.c:3637)
10  myApp                  	0x0095023e mono_jit_compile_method_with_opt (mini.c:5657)
11  myApp                  	0x0094e812 mono_jit_runtime_invoke (mini.c:6274)
12  myApp                  	0x0095295c mono_runtime_invoke (object.c:2827)
13  myApp                  	0x00955506 mono_runtime_exec_main (object.c:4054)
14  myApp                  	0x0095575a mono_runtime_run_main (object.c:3678)
15  myApp                  	0x00927296 mono_jit_exec (driver.g.c:957)
16  myApp                  	0x0098ed48 main (main.m:485)
17  libdyld.dylib                 	0x3c5d1b1c start + 0
Comment 5 Rolf Bjarne Kvinge [MSFT] 2013-07-31 11:54:29 UTC
Can you attach the binaries you used to do those comparisons (you might want to use dropbox or a similar service, since you'll probably hit the attachment limit for bugzilla)?

And are you using Visual Studio or Xamarin Studio?
Comment 7 Rolf Bjarne Kvinge [MSFT] 2013-07-31 17:35:05 UTC
Unfortunately disassembling those files doesn't show any differences.

Would it be possible to get access to your project so I can try to reproduce this myself? Going through the binaries trying to figure out what part differs between them using the Mach-O file format documentation will take a while otherwise.
Comment 8 Andrew 2013-08-01 06:23:55 UTC
Unfortunately I cannot release the sources. 
I started believing that it's rather infrastructure causing the issue, not the sources. I develop on a Windows PC, project folder is shared over SMB to a Mac Book, where Xamarin Studio is launched (do build and debug). 

On the Windows server, the BIN and OBJ folders are symbolic links to RAM disk (I'll explain later why it matters). The sources are heavily shared between the iOS app and a couple other server and mobile products, and the shared projects often have a duplicate project file that includes only those files that actually needed for the iOS app. Those duplicate Xamarin Studio/VS project files are added to a separate solution (used to build the iOS app) and have own "output path" set, pointing to Mac RAM disk. Also there are projects with default output paths, i.e. SMB drive (which can actually point to a RAM drive on Windows).

(Don't ask why it's organized this way, I started a year ago with Monotouch, then upgraded to Xamarin.iOS and is trying to find the most effective way to develop the app with such heavily shared sources. I'm still in the search :-)

When I was working with Monotouch, I faced with compilation errors (on Mac), if project OBJ folder is placed on Windows RAM drive (NTFS). And there's no problem if I leave the OBJ on the physical disk (NTFS). Otherwise, I was getting some strange error message from the compiler (I don't remember the actual text, but remember good that there was no connection between the message and the actual problem). So I always leave the OBJ folder on the physical drive.

I suspect that this time I'm having this issue because of similar, unexplainable issue, not because of the sources. Statistics also tells the same: on evening a day before yesterday I got ~8 of 10 builds crashing. next morning I got ~2 of 10 builds crashing. Last hour I spent building several Xamarin sample projects (via SMB) 10-20 times each and got zero crash.

If you were able to confirm that the binary is truly "corrupted", that would confirm my version, and might guarantee that a successful launch on a device indicates the build is not corrupted and "good" for use and can be uploaded to AppStore. Can you?

Would it help if I arrange to put your UUID into our devices list and will send you an IPA for manual deployment to your device (or someone's from X team), so you can debug it?


Thanks.