Bug 7965 - Regression in 3.0 (System.Threading.Tasks): InvalidOperationException "The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled. IsTerminating is set to True"
Summary: Regression in 3.0 (System.Threading.Tasks): InvalidOperationException "The un...
Status: RESOLVED FIXED
Alias: None
Product: Class Libraries
Classification: Mono
Component: mscorlib ()
Version: unspecified
Hardware: PC Linux
: --- normal
Target Milestone: Untriaged
Assignee: Jérémie Laval
URL:
Depends on:
Blocks:
 
Reported: 2012-10-24 08:52 UTC by Andres G. Aragoneses
Modified: 2012-11-16 07:15 UTC (History)
3 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Andres G. Aragoneses 2012-10-24 08:52:18 UTC
We have detected our AcceptanceTests failing if we upgrade to Mono 3.0.

I still haven't reduced this to a simple testcase, but thought that putting here the stacktrace early on may help somehow. BTW, we are not using any async APIs or C#5.

[The regression is somewhere between the commit 551ca84 (labeled as "2.11.4", but not necessarily the same tag) and the version based on the tarball found in http://origin-download.mono-project.com/sources/mono/mono-3.0.0.tar.bz2.

Stacktrace is (caputed by xsp4):

Listening on port: 8000 (non-secure)
Hit Return to stop the server.
Handling exception type InvalidOperationException
Message is The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
IsTerminating is set to True
System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
  at System.Threading.Tasks.TaskCompletionSource`1[System.Object].ThrowInvalidException () [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskCompletionSource`1[System.Object].SetException (IEnumerable`1 exceptions) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskCompletionSource`1[System.Object].SetException (System.Exception exception) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskFactory`1[System.Object].InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1 tcs, System.Func`2 endMethod, IAsyncResult l) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskFactory`1+<FromAsyncBeginEnd>c__AnonStorey2D`3[System.Object,System.Byte[],System.Int32,System.Int32].<>m__33 (IAsyncResult l) [0x00000] in <filename unknown>:0 
  at System.Net.WebAsyncResult.CB (System.Object unused) [0x00000] in <filename unknown>:0 
Handling exception type InvalidOperationException
Message is The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
IsTerminating is set to True
System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (IEnumerable`1 exceptions) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (System.Exception exception) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskFactory`1[System.Int32].InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1 tcs, System.Func`2 endMethod, IAsyncResult l) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskFactory`1+<FromAsyncBeginEnd>c__AnonStorey2D`3[System.Int32,System.Byte[],System.Int32,System.Int32].<>m__33 (IAsyncResult l) [0x00000] in <filename unknown>:0 
  at System.Net.WebAsyncResult.CB (System.Object unused) [0x00000] in <filename unknown>:0 
Handling exception type InvalidOperationException
Message is The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
IsTerminating is set to True
System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (IEnumerable`1 exceptions) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (System.Exception exception) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskFactory`1[System.Int32].InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1 tcs, System.Func`2 endMethod, IAsyncResult l) [0x00000] in <filename unknown>:0 
  at System.Threading.Tasks.TaskFactory`1+<FromAsyncBeginEnd>c__AnonStorey2D`3[System.Int32,System.Byte[],System.Int32,System.Int32].<>m__33 (IAsyncResult l) [0x00000] in <filename unknown>:0 
  at System.Net.WebAsyncResult.CB (System.Object unused) [0x00000] in <filename unknown>:0 


PS: I recommend you to create the "3.0" version in Bugzilla :)
Comment 1 Andres G. Aragoneses 2012-10-24 09:10:13 UTC
(In reply to comment #0)
> BTW, we are not using any async APIs or C#5.

Forgot to say, this a ServiceStack-based solution, which may use async APIs.
Comment 2 Andres G. Aragoneses 2012-10-30 12:23:49 UTC
Got a stacktrace now with line numbers:

[0xb4d0cb70:] EXCEPTION handling: System.ArgumentNullException: null handle
Parameter name: waitHandles

"Threadpool worker" tid=0x0xb4d0cb70 this=0x0x569a0 thread handle 0x41f state : not waiting owns ()
  at System.Threading.WaitHandle.CheckArray (System.Threading.WaitHandle[],bool) [0x00085] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading/WaitHandle.cs:82
  at System.Threading.WaitHandle.WaitAny (System.Threading.WaitHandle[],System.TimeSpan,bool) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading/WaitHandle.cs:215
  at System.Threading.RegisteredWaitHandle.Wait (object) [0x00019] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading/RegisteredWaitHandle.cs:74
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <IL 0x00052, 0xffffffff>
[0xb4d0cb70:] EXCEPTION handling: System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.

"Threadpool worker" tid=0x0xb4d0cb70 this=0x0x569a0 thread handle 0x41f state : not waiting owns ()
  at System.Threading.Tasks.TaskCompletionSource`1<int>.SetResult (int) [0x0000c] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:86
  at System.Threading.Tasks.TaskFactory`1<int>.InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1<int>,System.Func`2<System.IAsyncResult, int>,System.IAsyncResult) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:461
  at System.Threading.Tasks.TaskFactory`1/<FromAsyncBeginEnd>c__AnonStorey2D`3<int, byte[], int, int>.<>m__33 (System.IAsyncResult) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:443
  at System.Net.WebAsyncResult.CB (object) [0x00000] in /home/dev/mono-3.0.0/mcs/class/System/System.Net/WebAsyncResult.cs:148
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <IL 0x00052, 0xffffffff>
[0xb4d0cb70:] EXCEPTION handling: System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.

"Threadpool worker" tid=0x0xb4d0cb70 this=0x0x569a0 thread handle 0x41f state : not waiting owns ()
  at System.Threading.Tasks.TaskCompletionSource`1<int>.SetException (System.Collections.Generic.IEnumerable`1<System.Exception>) [0x0000c] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:80
  at System.Threading.Tasks.TaskCompletionSource`1<int>.SetException (System.Exception) [0x00011] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:74
  at System.Threading.Tasks.TaskFactory`1<int>.InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1<int>,System.Func`2<System.IAsyncResult, int>,System.IAsyncResult) [0x0001f] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:465
  at System.Threading.Tasks.TaskFactory`1/<FromAsyncBeginEnd>c__AnonStorey2D`3<int, byte[], int, int>.<>m__33 (System.IAsyncResult) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:443
  at System.Net.WebAsyncResult.CB (object) [0x00000] in /home/dev/mono-3.0.0/mcs/class/System/System.Net/WebAsyncResult.cs:148
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <IL 0x00052, 0xffffffff>
Handling exception type InvalidOperationException
Message is The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
IsTerminating is set to True
System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (IEnumerable`1 exceptions) [0x0000c] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:80 
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (System.Exception exception) [0x00011] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:74 
  at System.Threading.Tasks.TaskFactory`1[System.Int32].InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1 tcs, System.Func`2 endMethod, IAsyncResult l) [0x0001f] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:465 
  at System.Threading.Tasks.TaskFactory`1+<FromAsyncBeginEnd>c__AnonStorey2D`3[System.Int32,System.Byte[],System.Int32,System.Int32].<>m__33 (IAsyncResult l) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:443 
  at System.Net.WebAsyncResult.CB (System.Object unused) [0x00000] in /home/dev/mono-3.0.0/mcs/class/System/System.Net/WebAsyncResult.cs:148 
[0xb4d0cb70:] EXCEPTION handling: System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.

"Threadpool worker" tid=0x0xb4d0cb70 this=0x0x569a0 thread handle 0x41f state : not waiting owns ()
  at System.Threading.Tasks.TaskCompletionSource`1<int>.SetResult (int) [0x0000c] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:86
  at System.Threading.Tasks.TaskFactory`1<int>.InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1<int>,System.Func`2<System.IAsyncResult, int>,System.IAsyncResult) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:461
  at System.Threading.Tasks.TaskFactory`1/<FromAsyncBeginEnd>c__AnonStorey2D`3<int, byte[], int, int>.<>m__33 (System.IAsyncResult) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:443
  at System.Net.WebAsyncResult.CB (object) [0x00000] in /home/dev/mono-3.0.0/mcs/class/System/System.Net/WebAsyncResult.cs:148
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <IL 0x00052, 0xffffffff>
[0xb4d0cb70:] EXCEPTION handling: System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.

"Threadpool worker" tid=0x0xb4d0cb70 this=0x0x569a0 thread handle 0x41f state : not waiting owns ()
  at System.Threading.Tasks.TaskCompletionSource`1<int>.SetException (System.Collections.Generic.IEnumerable`1<System.Exception>) [0x0000c] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:80
  at System.Threading.Tasks.TaskCompletionSource`1<int>.SetException (System.Exception) [0x00011] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:74
  at System.Threading.Tasks.TaskFactory`1<int>.InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1<int>,System.Func`2<System.IAsyncResult, int>,System.IAsyncResult) [0x0001f] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:465
  at System.Threading.Tasks.TaskFactory`1/<FromAsyncBeginEnd>c__AnonStorey2D`3<int, byte[], int, int>.<>m__33 (System.IAsyncResult) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:443
  at System.Net.WebAsyncResult.CB (object) [0x00000] in /home/dev/mono-3.0.0/mcs/class/System/System.Net/WebAsyncResult.cs:148
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <IL 0x00052, 0xffffffff>
Handling exception type InvalidOperationException
Message is The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
IsTerminating is set to True
System.InvalidOperationException: The underlying Task is already in one of the three final states: RanToCompletion, Faulted, or Canceled.
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (IEnumerable`1 exceptions) [0x0000c] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:80 
  at System.Threading.Tasks.TaskCompletionSource`1[System.Int32].SetException (System.Exception exception) [0x00011] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskCompletionSource.cs:74 
  at System.Threading.Tasks.TaskFactory`1[System.Int32].InnerInvoke (System.Threading.Tasks.TaskCompletionSource`1 tcs, System.Func`2 endMethod, IAsyncResult l) [0x0001f] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:465 
  at System.Threading.Tasks.TaskFactory`1+<FromAsyncBeginEnd>c__AnonStorey2D`3[System.Int32,System.Byte[],System.Int32,System.Int32].<>m__33 (IAsyncResult l) [0x00000] in /home/dev/mono-3.0.0/mcs/class/corlib/System.Threading.Tasks/TaskFactory_T.cs:443 
  at System.Net.WebAsyncResult.CB (System.Object unused) [0x00000] in /home/dev/mono-3.0.0/mcs/class/System/System.Net/WebAsyncResult.cs:148 


(This is running the 3.0 tarball)
Comment 3 Jérémie Laval 2012-11-16 06:40:28 UTC
The regression might be in WebAsyncResult itself but in the meantime I have committed a safeguard in 94f551d46357c4a71f0108169ac65a3615cda686 (master) that should workaround the issue.
Comment 4 Andres G. Aragoneses 2012-11-16 07:03:11 UTC
Actually, I saw these 3 commits which happened before yours (https://github.com/mono/mono/commit/94f551d4) which are System.Threading.Taskish related:

https://github.com/mono/mono/commit/7c3708d2

https://github.com/mono/mono/commit/e3bd9da8

https://github.com/mono/mono/commit/0cf33093

And I thought they could be related to this. I then tested master branch 2 days after them (after the most recent one), and I didn't get the exception here anymore. But it didn't solve my problem completely (somehow it deadlocks).

So as I also recently saw a commit about a deadlock from Rodrigo: 
https://github.com/mono/mono/commit/47dd377f8587b466475ac5a6cf548f49b9394d0d

I was wondering if that could fix it, but it didn't.

So I'll try master with your fix and report back. Thanks very much.
Comment 5 Jérémie Laval 2012-11-16 07:15:46 UTC
Yeah, my commit is probably not going to help with your deadlock issue. Could you log another bug for that deadlock and assign it to Rodrigo? Thanks.