Bug 39879 - Assertion at threadpool-ms-io.c:370, condition ‘fd >= 0’ not met
Summary: Assertion at threadpool-ms-io.c:370, condition ‘fd >= 0’ not met
Status: RESOLVED NOT_REPRODUCIBLE
Alias: None
Product: Runtime
Classification: Mono
Component: io-layer ()
Version: 4.2.0 (C6)
Hardware: Macintosh Linux
: --- normal
Target Milestone: ---
Assignee: Ludovic Henry
URL:
Depends on:
Blocks:
 
Reported: 2016-03-24 03:41 UTC by Jim Borden
Modified: 2017-02-16 16:01 UTC (History)
3 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED NOT_REPRODUCIBLE

Description Jim Borden 2016-03-24 03:41:58 UTC
Hardware and OS selection based on this:  It is a Linux Mint VM running on OS X.

This assertion appears to be new in 4.2.3, and it happens very frequently when running my unit tests in a row (using Monodevelop NUnit test runner, or nunit-console).  One at a time is no problem but as a group this happens every time (in case you want to look, the tests I tried running are here -> https://github.com/couchbase/couchbase-lite-net/blob/71cb2ff3b879c11dc8e59eee737b284a99a0ad0a/src/Couchbase.Lite.Tests.Shared/ApiTest.cs).  I have no idea what it means.  Here is some info that it gave me:

[New LWP 59297]
[New LWP 59296]
[New LWP 59295]
[New LWP 59294]
[New LWP 59293]
[New LWP 59292]
[New LWP 59291]
[New LWP 59290]
[New LWP 59289]
[New LWP 59288]
[New LWP 59287]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
185	../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S: No such file or directory.
  Id   Target Id         Frame 
  12   Thread 0x7f7fa27ff700 (LWP 59287) "cli" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  11   Thread 0x7f7fa0715700 (LWP 59288) "Finalizer" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  10   Thread 0x7f7f9bae4700 (LWP 59289) "cli" 0x00007f7fa33c86dd in accept () at ../sysdeps/unix/syscall-template.S:81
  9    Thread 0x7f7f9b8e3700 (LWP 59290) "Timer-Scheduler" pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  8    Thread 0x7f7f9aeff700 (LWP 59291) "cli" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  7    Thread 0x7f7f9acfe700 (LWP 59292) "cli" __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
  6    Thread 0x7f7f9b24f700 (LWP 59293) "cli" 0x00007f7fa33c8ee9 in __libc_waitpid (pid=59298, stat_loc=0x7f7f9b24dd4c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:40
  5    Thread 0x7f7f98c59700 (LWP 59294) "cli" pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  4    Thread 0x7f7f9b04e700 (LWP 59295) "cli" __clock_nanosleep (clock_id=1, flags=1, req=0x7f7f9b04dd80, rem=0xffffffffffffffff) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:49
  3    Thread 0x7f7f83fff700 (LWP 59296) "Threadpool work" pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  2    Thread 0x7f7f83dfe700 (LWP 59297) "Threadpool work" pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
* 1    Thread 0x7f7fa3ee67c0 (LWP 59286) "cli" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

Thread 12 (Thread 0x7f7fa27ff700 (LWP 59287)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00000000005f9aec in ?? ()
#2  0x00007f7fa33c1182 in start_thread (arg=0x7f7fa27ff700) at pthread_create.c:312
#3  0x00007f7fa30ee47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 11 (Thread 0x7f7fa0715700 (LWP 59288)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x000000000061de28 in mono_sem_wait ()
#2  0x00000000005a2076 in ?? ()
#3  0x00000000005843d3 in ?? ()
#4  0x0000000000624666 in ?? ()
#5  0x00007f7fa33c1182 in start_thread (arg=0x7f7fa0715700) at pthread_create.c:312
#6  0x00007f7fa30ee47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 10 (Thread 0x7f7f9bae4700 (LWP 59289)):
#0  0x00007f7fa33c86dd in accept () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000000060bbdd in ?? ()
#2  0x0000000000575829 in ?? ()
#3  0x000000004132cbb1 in ?? ()
#4  0x00007f7f94001900 in ?? ()
#5  0x00007f7fa28c7468 in ?? ()
#6  0x00007f7fa28c7260 in ?? ()
#7  0x00007f7fa28c7398 in ?? ()
#8  0x00007f7fa28c6bb0 in ?? ()
#9  0x00007f7f94000e00 in ?? ()
#10 0x000000004132c6cd in ?? ()
#11 0x00007f7f9bae3770 in ?? ()
#12 0x00007f7f9bae36d0 in ?? ()
#13 0x000000004132c650 in ?? ()
#14 0x00007f7fa28b4770 in ?? ()
#15 0x000000004132c2b4 in ?? ()
#16 0x0000000000000001 in ?? ()
#17 0x00007f7f9bae3790 in ?? ()
#18 0x00007f7fa28c6cc0 in ?? ()
#19 0x6174616400797470 in ?? ()
#20 0x000000004132c610 in ?? ()
#21 0x0000000002a6d9a8 in ?? ()
#22 0x000000004132c534 in ?? ()
#23 0x000000004132c610 in ?? ()
#24 0x00007f7f9bae3880 in ?? ()
#25 0x000000004132c3ec in ?? ()
#26 0x00007f7fa28c7260 in ?? ()
#27 0x00007f7fa28b4770 in ?? ()
#28 0x00007f7f00000000 in ?? ()
#29 0x00000000412ed7b0 in ?? ()
#30 0x0000000000000001 in ?? ()
#31 0x00007f7fa31352f0 in __strcmp_ssse3 () at ../sysdeps/x86_64/multiarch/../strcmp.S:2098
#32 0x000000004132c3b0 in ?? ()
#33 0x0000000002a6d980 in ?? ()
#34 0x000000004132c2e4 in ?? ()
#35 0x000000004132c3b0 in ?? ()
#36 0x00007f7f9bae3880 in ?? ()
#37 0x000000004132c2b4 in ?? ()
#38 0x00007f7fa28c7260 in ?? ()
#39 0x000000004132c024 in ?? ()
#40 0xffff000000ff0000 in ?? ()
#41 0x6174616400797470 in ?? ()
#42 0x646e69006c657665 in ?? ()
#43 0xffffffffffffff00 in ?? ()
#44 0x00007f7fa28b36e0 in ?? ()
#45 0x0000000000000000 in ?? ()

Thread 9 (Thread 0x7f7f9b8e3700 (LWP 59290)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
#1  0x00000000005feea4 in ?? ()
#2  0x0000000000610e4c in ?? ()
#3  0x000000000058415e in ?? ()
#4  0x0000000000585309 in ?? ()
#5  0x0000000041ca027d in ?? ()
#6  0x0000000000000001 in ?? ()
#7  0x0000000000000000 in ?? ()

Thread 8 (Thread 0x7f7f9aeff700 (LWP 59291)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00000000005fef47 in ?? ()
#2  0x000000000061101b in ?? ()
#3  0x00000000005a4d2a in ?? ()
#4  0x0000000041d2e7b8 in ?? ()
#5  0x00007f7f8c001900 in ?? ()
#6  0x00007f7fa2af43d8 in ?? ()
#7  0x00007f7fa2af44a8 in ?? ()
#8  0x00007f7fa2a82fb0 in ?? ()
#9  0xffffffffffffffff in ?? ()
#10 0x00007f7f90000e00 in ?? ()
#11 0x00007f7fa28cec80 in ?? ()
#12 0x00007f7f9aefe780 in ?? ()
#13 0x00007f7f9aefe6c0 in ?? ()
#14 0x0000000041d2e621 in ?? ()
#15 0x00007f7fa2a82e28 in ?? ()
#16 0x0000000000000001 in ?? ()
#17 0x00007f7f9aefe850 in ?? ()
#18 0x0000000041d2e5bf in ?? ()
#19 0x00007f7fa2a82fb0 in ?? ()
#20 0x0000000041d2e40f in ?? ()
#21 0x00007f7fa2af4301 in ?? ()
#22 0x00007f7fa2a82fb0 in ?? ()
#23 0x00007f7fa2af43d8 in ?? ()
#24 0x00007f7fa3dcea20 in ?? ()
#25 0x00007f7f9aefe850 in ?? ()
#26 0x00007f7fa2af44a8 in ?? ()
#27 0x00007f7fa2af43d8 in ?? ()
#28 0x00007f7fa2af43d8 in ?? ()
#29 0x00007f7f9aefe850 in ?? ()
#30 0x0000000041d2df58 in ?? ()
#31 0x00007f7fa2af44a8 in ?? ()
#32 0x00007f7fa2a82e28 in ?? ()
#33 0x00007f7fa2af4268 in ?? ()
#34 0x0000000041c9d784 in ?? ()
#35 0x0000000000000000 in ?? ()

Thread 7 (Thread 0x7f7f9acfe700 (LWP 59292)):
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007f7fa33c3672 in _L_lock_953 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007f7fa33c34da in __GI___pthread_mutex_lock (mutex=0x7f7f840e02f0) at ../nptl/pthread_mutex_lock.c:114
#3  0x000000000058b03d in ?? ()
#4  0x0000000041d6f4bb in ?? ()
#5  0x00007f7fa283d690 in ?? ()
#6  0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7f7f9b24f700 (LWP 59293)):
#0  0x00007f7fa33c8ee9 in __libc_waitpid (pid=59298, stat_loc=0x7f7f9b24dd4c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:40
#1  0x000000000049d689 in ?? ()
#2  <signal handler called>
#3  0x00007f7fa302acc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#4  0x00007f7fa302e0d8 in __GI_abort () at abort.c:89
#5  0x000000000062ac69 in ?? ()
#6  0x000000000062ae77 in ?? ()
#7  0x000000000062afc6 in ?? ()
#8  0x000000000058aaae in ?? ()
#9  0x00000000005843d3 in ?? ()
#10 0x0000000000624666 in ?? ()
#11 0x00007f7fa33c1182 in start_thread (arg=0x7f7f9b24f700) at pthread_create.c:312
#12 0x00007f7fa30ee47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 5 (Thread 0x7f7f98c59700 (LWP 59294)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
#1  0x00007f7f99e272a3 in ?? () from /home/borrrden/Development/couchbase-lite-net/src/Couchbase.Lite.Net45.Tests/bin/Debug/libCBForest-Interop.so
#2  0x00007f7fa33c1182 in start_thread (arg=0x7f7f98c59700) at pthread_create.c:312
#3  0x00007f7fa30ee47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 4 (Thread 0x7f7f9b04e700 (LWP 59295)):
#0  __clock_nanosleep (clock_id=1, flags=1, req=0x7f7f9b04dd80, rem=0xffffffffffffffff) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:49
#1  0x00000000006122c8 in ?? ()
#2  0x0000000000588244 in ?? ()
#3  0x00000000005843d3 in ?? ()
#4  0x0000000000624666 in ?? ()
#5  0x00007f7fa33c1182 in start_thread (arg=0x7f7f9b04e700) at pthread_create.c:312
#6  0x00007f7fa30ee47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 3 (Thread 0x7f7f83fff700 (LWP 59296)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
#1  0x000000000061b982 in ?? ()
#2  0x0000000000586b58 in ?? ()
#3  0x00000000005843d3 in ?? ()
#4  0x0000000000624666 in ?? ()
#5  0x00007f7fa33c1182 in start_thread (arg=0x7f7f83fff700) at pthread_create.c:312
#6  0x00007f7fa30ee47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 2 (Thread 0x7f7f83dfe700 (LWP 59297)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
#1  0x000000000061b982 in ?? ()
#2  0x0000000000586b58 in ?? ()
#3  0x00000000005843d3 in ?? ()
#4  0x0000000000624666 in ?? ()
#5  0x00007f7fa33c1182 in start_thread (arg=0x7f7f83dfe700) at pthread_create.c:312
#6  0x00007f7fa30ee47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 1 (Thread 0x7f7fa3ee67c0 (LWP 59286)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00000000005fef47 in ?? ()
#2  0x000000000061101b in ?? ()
#3  0x00000000005855d8 in ?? ()
#4  0x0000000041d2ee51 in ?? ()
#5  0x00007f7f9b4af030 in ?? ()
#6  0x00007f7fa2a82e10 in ?? ()
#7  0x00007f7fa2a829a0 in ?? ()
#8  0x0000000000000001 in ?? ()
#9  0x00007f7fa2af45a0 in ?? ()
#10 0x00007ffc06700e70 in ?? ()
#11 0x0000000041d2eba4 in ?? ()
#12 0x00007ffc06700da0 in ?? ()
#13 0x00007ffc06700c30 in ?? ()
#14 0x0000000041d2eb9f in ?? ()
#15 0x00007f7fa2af4688 in ?? ()
#16 0x0000000041d2ea6d in ?? ()
#17 0x00007f7f9b4af030 in ?? ()
#18 0x0000000041d2ea14 in ?? ()
#19 0x00007f7f9b4af030 in ?? ()
#20 0x0000000041d2e824 in ?? ()
#21 0x00007f7fa2a82e10 in ?? ()
#22 0x0000000041d2dd6e in ?? ()
#23 0x00007f7f9b40ef80 in ?? ()
#24 0x00007f7fa2a82e10 in ?? ()
#25 0x00007f7fa2a7e8c8 in ?? ()
#26 0x0000000000000001 in ?? ()
#27 0x0000000000000000 in ?? ()

=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================
Comment 1 Jim Borden 2016-03-24 03:59:18 UTC
Nevermind about it being new in 4.2.3, for some reason I didn't see it the first time around in the 4.2.2 source.
Comment 2 Jim Borden 2016-03-25 02:21:20 UTC
Note: 4.0.5.1 is not affected by this.  The refactor to one method from several for thread events seems to have caused this.
Comment 3 Jim Borden 2016-03-25 21:32:48 UTC
Just for some more information, I build the test project (Couchbase.Lite.Net45.Tests) in the NET 4.5 solution (Couchbase.Lite.Net45.sln in the root directory) in Xamarin Studio (that requires following some instructions in the README.txt located in src/StorageEngines/ForestDB/CBForest/CSharp/prebuilt) in Release mode, and either simply called nunit-console Couchbase.Lite.Net45.Tests.dll or ran a test fixture from the unit test pad in Xamarin Studio.  The exact commit that I was using was 71cb2ff3b879c11dc8e59eee737b284a99a0ad0a.
Comment 4 Jim Borden 2016-03-25 21:53:04 UTC
Debug mode will work as well, according to the log output that I posted myself xD
Comment 5 Jim Borden 2016-03-25 21:53:24 UTC
Work as in "trigger the problem"
Comment 6 Andi McClure 2016-04-05 22:05:20 UTC
Hi Jim-- I'm interested in trying to reproduce this locally (preferably with nunit-console), I'm curious if it would be possible to make a smaller repro without having to build and run all of couchbase. Do you know *which* of the NUnit tests is failing, and which tests ran before it? I think there's a flag for having NUnit print which tests it's running (-verbose ?), if you run that do you see it always failing on the same test? Thanks
Comment 7 Jim Borden 2016-04-05 22:23:56 UTC
I'd have a hard time narrowing it down to a smaller case because it doesn't show any traces through any of my code.  I think the best I could try to do is prebuild the test cases.  I can only imagine it is something that happens when the thread pool gets worked too hard (out of threads?).  There is not really a particular test that triggers it.  In fact if I run the test that fails alone it passes every time.  It only fails after a few tests are run in a row.  In this case, running the ApiTest fixture it will do this:

TestAllDocuments -> Pass
TestAPIManager -> Pass
TestAsyncViewQuery -> Crash

Perhaps if I understood what the assertion was trying to tell me I might understand better.
Comment 8 Ludovic Henry 2017-02-08 20:20:01 UTC
Jim, could you please notify us if you still witness this crash, and provide us with a repro in this case? Thank you.
Comment 9 Jim Borden 2017-02-09 06:54:34 UTC
I haven't seen this in a while.  The NUnitLite test runner (2.x version) still crashes quite often for Android but I've just given up and accepted that it's a total piece of crap.  The XUnit runner is much more reliable and what I will be using going forward after our major version bump.
Comment 10 Ludovic Henry 2017-02-16 16:01:06 UTC
As Jim cannot reproduce the bug with latest version, I am closing the bug. Please do not hesitate to reopen if you witness it again. Thank you!