Bug 26222 - Assertion in mono_thread_info_attach, condition `result' not met (mono-threads.c:159)
Summary: Assertion in mono_thread_info_attach, condition `result' not met (mono-thread...
Status: VERIFIED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: XI runtime ()
Version: XI 8.7.x (iOS 8.2 previews)
Hardware: Macintosh Mac OS
: High normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2015-01-21 02:18 UTC by Frank A. Krueger
Modified: 2015-07-29 04:20 UTC (History)
7 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
App database. Put it in the Documents directory. (Uncompress of course) (2.44 MB, application/zip)
2015-02-27 13:50 UTC, Frank A. Krueger
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
VERIFIED FIXED

Description Frank A. Krueger 2015-01-21 02:18:45 UTC
Using the unified API.

The crash says:

error: * Assertion at ../../../../../mono/mono/utils/mono-threads.c:159, condition `result' not met

Thread 15 Crashed:
0   libsystem_kernel.dylib        	0x319c0dfc __pthread_kill + 8
1   libsystem_pthread.dylib       	0x31a3ed33 pthread_kill + 59
2   libsystem_c.dylib             	0x31960905 abort + 73
3   Mocast                        	0x01fb844f mono_handle_native_sigsegv (mini-exceptions.c:2360)
4   Mocast                        	0x01fbcff7 sigabrt_signal_handler (mini-posix.c:207)
5   libsystem_platform.dylib      	0x31a3a879 _sigtramp + 41
6   libsystem_pthread.dylib       	0x31a3ed33 pthread_kill + 59
7   libsystem_c.dylib             	0x31960905 abort + 73
8   Mocast                        	0x020663d4 log_callback (runtime.m:817)
9   Mocast                        	0x020535f1 log_adapter (mono-logger.c:278)
10  Mocast                        	0x0205a515 monoeg_assertion_message (goutput.c:113)
11  Mocast                        	0x02054f65 mono_thread_info_attach (mono-threads.c:159)
12  Mocast                        	0x02054acb inner_start_thread (mono-threads-posix.c:67)
13  libsystem_pthread.dylib       	0x31a3de91 _pthread_body + 137
14  libsystem_pthread.dylib       	0x31a3de03 _pthread_start + 115
15  libsystem_pthread.dylib       	0x31a3bb8c thread_start + 4

My Xamarin.iOS:

Xamarin.iOS
Version: 8.7.0.126 (Enterprise Edition)
Hash: 3fe096a
Branch: 
Build date: 2015-01-14 17:38:04-0500
Comment 1 Rolf Bjarne Kvinge [MSFT] 2015-01-21 04:06:47 UTC
Do you have a test case?
Comment 2 Frank A. Krueger 2015-01-21 17:13:55 UTC
No just an app that reproduces it on occasion (a couple times a day with use). It seems to happen when I post 40+ network requests all at once. What can I capture in logs to better help you?

This uses System.Threading.Tasks a lot but is otherwise not special.

Source code is here: https://github.com/praeclarum/Mocast (you have access)
Comment 3 Rolf Bjarne Kvinge [MSFT] 2015-01-22 04:58:02 UTC
The project doesn't build, it seems to reference files from other repositories:

	CSC: error CS2001: Source file `../../Praeclarum/Praeclarum/ListDiff.cs' could not be found
	CSC: error CS2001: Source file `../../Praeclarum/Praeclarum.iOS/UI/Layout.cs' could not be found
	CSC: error CS2001: Source file `../../Praeclarum/Praeclarum/Log.cs' could not be found
Comment 4 Frank A. Krueger 2015-01-23 00:22:26 UTC
Rolf, sorry, added you to https://github.com/praeclarum/Praeclarum  Just put it in parallel to Mocast.

I'm working on a small repro too...
Comment 5 Frank A. Krueger 2015-01-23 12:40:39 UTC
My small repro isn't reproing so I'm still stuck with just the app itself as a test.
Comment 6 Rolf Bjarne Kvinge [MSFT] 2015-01-26 13:10:35 UTC
Are there any particular steps I can do to reproduce this? Or maybe it's automatable so that I can leave the app open overnight while it tries to reproduce it all by itself?
Comment 7 Frank A. Krueger 2015-02-12 14:05:22 UTC
For me, the app crashes essentially every time I use it due to this bug. It's very frustrating, but I have no idea how to tell you to reproduce it.

Can you put more info in the ASSERT, or can I enable logging or something?

I can't ship my app this crashy...
Comment 8 Frank A. Krueger 2015-02-12 14:11:49 UTC
Since it's 100% repro for me, can we setup a remote debug session? Or I can let you control my machine.
Comment 9 Miguel de Icaza [MSFT] 2015-02-12 14:26:05 UTC
Frank,

We can set that up.    Rolf will follow up with some times.

In the meantime, can you share those repos with me as well?

And on what hardware does this happen?

What kind of things do I need to do to trigger it once I get it?
Comment 10 Rolf Bjarne Kvinge [MSFT] 2015-02-13 05:47:27 UTC
I can do a remote debug session on Monday, at 3-12am and 1-2pm Boston time.

I have a few questions before though:

* Does this happen on both device and simulator, or only of them?
* You say it seems related to posting 40+ network requests at the same time; how can I make your app do that?
Comment 11 Frank A. Krueger 2015-02-27 13:49:28 UTC
I’m sorry I missed the email.

This only happens on the device.

--

I have since rebuilt with 8.7.2.42 and am not getting the crashes in a few minutes of use. I will keep testing and see if it's stable now.

--

I have attached my app database. If you can get this file into the Documents directory before this line in AppDelegate.cs:58 runs:

Storage = new MocastStorage ();

then you should have a pretty close setup to what I'm using. (Other settings are stored in iCloud and user prefs unfortunately.)

If you see them all listed, then just start playing around in the app and pulling to refresh a few times. If it repros as it did for me, it will crash in a just a few seconds.
Comment 12 Frank A. Krueger 2015-02-27 13:50:47 UTC
Created attachment 10093 [details]
App database. Put it in the Documents directory. (Uncompress of course)
Comment 13 Frank A. Krueger 2015-03-01 19:16:38 UTC
I still get the crash with 8.7.2.42 but a lot less often.

It definitely is triggered best by doing a full refresh in the app (pull to refresh on the first tab screen).
Comment 14 Rolf Bjarne Kvinge [MSFT] 2015-03-02 08:23:16 UTC
OK, I can reproduce now.
Comment 15 Rolf Bjarne Kvinge [MSFT] 2015-03-02 14:44:14 UTC
Rodrigo, this looks like another case where our pthread tls key is freed before Objective-C's: https://gist.github.com/rolfbjarne/de13dcd4aaee1325f41d - mono_dead_letter_dealloc is called before mono_thread_info_attach again on the same thread.

So it looks like the mono_dead_letter_dealloc logic is not good enough. 

I looked into alternatives, and it looks like the Mach API can be used, here's a very simple POC demonstrating that the Mach API can notify us upon thread exit, and those notifications are executed after pthread's key destructors: https://gist.github.com/rolfbjarne/13b9e57874e60fe3d130. The Mach API is not very well document though, so there are probably many issues with that POC.

Another alternative (which I didn't write a POC for, but ran into reading documentation [1]), is that pthread key destructors are somewhat recursive:

pthread_setspecific (key1, dtor1);
dtor1
{
    pthread_setspecific (key2, dtor2);
}
dtor2
{
   // the actual logic
}

dtor2 is supposed to be called after all the initial dtors have been called, so maybe we can abuse this to our purpose.

[1] 'man pthread_key_create': "If, after all the destructors have been called for all non-NULL values with associated destructors, there are still some non-NULL values with associated destructors, then the process is repeated."
Comment 16 Rodrigo Kumpera 2015-03-02 14:58:06 UTC
Hi Rolf,

Are you using mono_thread_detach_if_exiting?
This was added last August and was intended to fix the short comings of the dead letter approach. It is the recommended workaround for this binding bug.

On your suggestions.

The mach one doesn't work because it's racy, notification must happen in the exiting thread and happen before the thread is deallocated otherwise we'll have the GC poking into unknown ports/memory.

The pthread approach doesn't work because user keys are destroyed before system keys.
It's used to detect use-after-cleanup by mono_thread_detach_if_exiting though.
Comment 17 Rolf Bjarne Kvinge [MSFT] 2015-03-02 16:13:16 UTC
Fixed.

maccore/master: dfc42be3c998834febd5249771aa4b7c9f0a1aa5
Comment 19 Frank A. Krueger 2015-03-30 19:38:02 UTC
Sorry for the late response. I have built with Version: 8.8.2.4 and so far it is running very well.
Comment 20 Rolf Bjarne Kvinge [MSFT] 2015-03-31 05:10:51 UTC
Thanks Frank, marking as verified.
Comment 21 ohad 2015-07-28 15:52:07 UTC
Hi I have this bug as well but didn't understand how to fix. Can you please explore on that. 

Thanks. 
Ohad.
Comment 22 Rolf Bjarne Kvinge [MSFT] 2015-07-29 04:20:24 UTC
@Ohad, the fix is included in the latest stable version of Xamarin.iOS (8.10), so if you see this you're running into a different problem and need to file a new bug for it.