Bug 23423 - SIGABRT and crash in System.Diagnostics.Process
Summary: SIGABRT and crash in System.Diagnostics.Process
Status: RESOLVED FIXED
Alias: None
Product: Runtime
Classification: Mono
Component: io-layer ()
Version: 3.2.x
Hardware: PC Linux
: --- normal
Target Milestone: ---
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2014-09-26 15:22 UTC by SN
Modified: 2015-01-16 12:53 UTC (History)
6 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
F# program that reproduces crash almost all of the time at will. (636 bytes, application/fsharp-script)
2014-09-26 15:22 UTC, SN
Details
output of program that shows stacktrace for all threads (94.68 KB, application/octet-stream)
2014-09-26 15:25 UTC, SN
Details
A workaround is provided in this attachment (489 bytes, application/fsharp-script)
2014-10-01 12:00 UTC, SN
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description SN 2014-09-26 15:22:55 UTC
Created attachment 8210 [details]
F# program that reproduces crash almost all of the time at will.

Attachment monobug.fsx is the tiniest snippet that I was able to write which 
reproduces an issue that I've encountered in mono 3.2.8 and 3.8.0 as well.
On linux such as ubuntu 14.04 run

$ cli --gc=sgen /usr/lib/cli/fsharp/fsi.exe monobug.fsx

This results in the stack trace below, full trace will be an attachment to this ticket.
The env variable MONO_DISABLE_SHM has no effect on the outcome.
F# is version 3.0 but I believe that this is not relevant to this bug report.

--- stack trace -----
* Assertion at ../../mono/io-layer/handles-private.h:201, condition `_WAPI_SHARED_HANDLE(_wapi_handle_type (handle))' not met

Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Diagnostics.Process.GetProcess_internal (int) <0xffffffff>
  at System.Diagnostics.Process.GetProcessById (int) <0x00017>
  at System.Diagnostics.Process.GetProcesses () <0x000ab>
  at FSI_0012.crashit () <0x0005b>
  at FSI_0012/vals@20-2.Invoke (Microsoft.FSharp.Core.Unit) <0x00017>
  at Microsoft.FSharp.Control.AsyncBuilderImpl/callA@780<bool, Microsoft.FSharp.Core.Unit>.Invoke (Microsoft.FSharp.Control.AsyncParams`1<bool>) <0x00142>
  at Microsoft.FSharp.Control.AsyncBuilderImpl/queueAsync@702<bool>.Invoke (Microsoft.FSharp.Core.Unit) <0x000b2>
  at <StartupCode$FSharp-Core>.$Control.loop@429-40 (Microsoft.FSharp.Control.Trampoline,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Control.FakeUnitValue>) <0x00032>
  at Microsoft.FSharp.Control.Trampoline.ExecuteAction (Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Control.FakeUnitValue>) <0x00067>
  at Microsoft.FSharp.Control.TrampolineHolder.Protect (Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Control.FakeUnitValue>) <0x00093>
  at <StartupCode$FSharp-Core>.$Control/-ctor@487-1.Invoke (object) <0x0004b>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <0xffffffff>
[... snipped ...]
Comment 1 SN 2014-09-26 15:25:13 UTC
Created attachment 8211 [details]
output of program that shows stacktrace for all threads

I don't believe that you'll need to look at this attachment because after running the program you should have a similar looking log file in just 1 or 2 tries.
Comment 2 SN 2014-09-26 18:04:02 UTC
Based on my debug session it looks like a handle of type WAPI_HANDLE_UNUSED is received contrary to the expectation that it is a SHARED_HANDLE, i.e. one of 
WAPI_HANDLE_PROCESS, WAPI_HANDLE_NAMEDMUTEX, WAPI_HANDLE_NAMEDSEM, WAPI_HANDLE_NAMEDEVENT.

------------debug session--------------------
Breakpoint 1, signal_process_if_gone (handle=0x4c5) at processes.c:1309
(gdb) p idx
$2 = 1221
(gdb) p _wapi_private_handles[idx/256][idx%256]
$3 = {type = WAPI_HANDLE_UNUSED, ref = 1, signalled = 0, .... }
pretty much all other values are zero, so they were left out
Comment 3 SN 2014-09-26 18:21:25 UTC
Strangely I get
(gdb) p _wapi_private_handles[999/256][999%256]
Cannot access memory at address 0x81f0
(gdb) p _wapi_private_handles[1000/256][1000%256]
Cannot access memory at address 0x8280

yet mapped memory was accessed for idx=1221 above
Comment 4 SN 2014-09-26 18:39:18 UTC
Further debugging reveals data corruption.
Unfortunately this means that there is no simple solution.
Until this is fixed System.Diagnostics.Process cannot be used in production code for anything but the most trivial applications.

-----------more debug data----------
 wapi_handle_ref: Attempting to ref unused handle 0x467
_wapi_handle_unref_full: Attempting to unref unused handle 0x467
_wapi_handle_ref: Attempting to ref unused handle 0x460
_wapi_handle_unref_full: Attempting to unref unused handle 0x460
*** Error in `mono': free(): invalid next size (fast): 0x00007f3de0013b60 ***
Comment 5 SN 2014-10-01 12:00:40 UTC
Created attachment 8268 [details]
A workaround is provided in this attachment

Note that proc.Close() is essential. Without it the bug continues to happen
Comment 6 SN 2014-10-01 14:55:49 UTC
In reference to the attached workaround, if there is a possibility that code that operates on 'proc' can throw an exception then replace the let binding with a use binding, as in

use proc = Process.Start(myexe)

which obviates the need to call proc.Close() explicitly in addition to disposing of proc if an exception is thrown.
Comment 7 Zoltan Varga 2014-10-07 11:09:00 UTC
Probably a dup of:
https://bugzilla.xamarin.com/show_bug.cgi?id=19140
Comment 8 SN 2014-10-07 12:08:23 UTC
A couple days ago I ran the test code pasted into bug 19140 on x86_64 multiple times without getting a crash, not even once. I tested on mono 3.2.8 and 3.8.0
Comment 9 Zoltan Varga 2014-10-14 17:15:48 UTC
So the problem seems to be in _wapi_search_handle (), the second loop calls the callback function without any lock, so the handle can become invalid while the function is working with it.
Comment 10 Zoltan Varga 2014-10-15 15:02:45 UTC
Fixed in mono master bf87077ac90946229f356cc2f6eded8d77189da3.
Comment 11 Zoltan Varga 2014-11-29 20:42:54 UTC
*** Bug 21623 has been marked as a duplicate of this bug. ***
Comment 12 xamarin 2015-01-16 11:37:06 UTC
This crash still happens to me on 3.12.0 release:

Error destroying handle 0x4ee mutex due to 16

Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Threading.InternalThread.Thread_free_internal (System.Threading.InternalThread,intptr) <0xffffffff>
  at System.Threading.InternalThread.Finalize () <0x0001b>
  at (wrapper runtime-invoke) object.runtime_invoke_virtual_void__this__ (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

        mono() [0x4accac]
        /lib/x86_64-linux-gnu/libpthread.so.0(+0xf8d0) [0x7f44a38ce8d0]
        /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37) [0x7f44a354c077]
        /lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x7f44a354d458]
        mono() [0x6232f9]
        mono() [0x623507]
        mono() [0x6235b2]
        mono() [0x5f6474]
        mono() [0x5f7bb4]
        mono() [0x57f2ee]
        [0x40b2ee81]
Comment 13 Zoltan Varga 2015-01-16 12:41:16 UTC
That could be a completely different problem. Also, this bug is not fixed in 3.12.
Comment 14 xamarin 2015-01-16 12:44:08 UTC
Release notes for 3.12.0 say it is fixed "Make process handles non-shared. Fixes #23423."
Comment 15 Zoltan Varga 2015-01-16 12:48:45 UTC
Sorry, this bug is indeed fixed in 3.12. You might be running into a separate problem. Do you have a reproducible testcase for this ?
Comment 16 xamarin 2015-01-16 12:53:59 UTC
I've had one for previous version, but that test case doesn't seem to crash in 3.12. Test case was here: https://bugzilla.xamarin.com/show_bug.cgi?id=21623#c12

Sadly I don't have a reproducible case for the crash in 3.12 (it's still related to Thread_free_internal, and crashes with same message).