Bug 12236 - Use of FromAsync results in growing memory use until OoM
Summary: Use of FromAsync results in growing memory use until OoM
Status: RESOLVED INVALID
Alias: None
Product: Class Libraries
Classification: Mono
Component: System ()
Version: master
Hardware: PC Mac OS
: High normal
Target Milestone: Untriaged
Assignee: marcos.henrich
URL:
Depends on:
Blocks:
 
Reported: 2013-05-14 12:41 UTC by Neale Ferguson
Modified: 2014-08-11 14:35 UTC (History)
6 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED INVALID

Description Neale Ferguson 2013-05-14 12:41:46 UTC
I’ve been working with Task.Factory.FromAsync() methods and have been experiencing severe memory leakage. I’ve used the profiler and it shows that a lot of objects just seem to be hanging around after use:

Heap shot 140 at 98.591 secs: size: 220177584, object count: 2803125, class count: 98, roots: 666
         Bytes      Count  Average Class name
      25049168     142325      175 System.Threading.Tasks.Task<System.Int32> (bytes: +398816, count: +2266)
            1 root references (1 pinning)
            142324 references from: System.Threading.Tasks.Task
            142305 references from: System.Threading.Tasks.TaskCompletionSource<System.Int32>
            98309 references from: task_test.Task3Test.<Run>c__AnonStorey1
      25049024     142324      176 System.Threading.Tasks.Task (bytes: +398816, count: +2266)
            142304 references from: System.Threading.Tasks.TaskContinuation
      17078880     142324      120 System.Action<System.Threading.Tasks.Task<System.Int32>> (bytes: +271920, count: +2266)
            142324 references from: System.Threading.Tasks.TaskActionInvoker.ActionTaskInvoke<System.Int32>
      17076600     142305      120 System.Runtime.Remoting.Messaging.MonoMethodMessage (bytes: +271680, count: +2264)
            1 root references (1 pinning)
            142304 references from: System.MonoAsyncCall
      17076584     142305      119 System.AsyncCallback (bytes: +271920, count: +2266)
            1 root references (1 pinning)
            142304 references from: System.MonoAsyncCall
      17076584     142305      119 System.Func<System.Int32> (bytes: +271920, count: +2266)
            1 root references (1 pinning)
            142305 references from: System.Func<System.IAsyncResult,System.Int32>
            142304 references from: System.Runtime.Remoting.Messaging.AsyncResult
            1 references from: System.Func<System.AsyncCallback,System.Object,System.IAsyncResult>
      17076584     142305      119 System.Func<System.IAsyncResult,System.Int32> (bytes: +271920, count: +2266)
            1 root references (1 pinning)
            142305 references from: System.Threading.Tasks.TaskFactory.<FromAsyncBeginEnd>c__AnonStorey3A<System.Int32>
      17076480     142304      120 System.Runtime.Remoting.Messaging.AsyncResult (bytes: +271800, count: +2265)
            98461 references from: System.Object[]

I’m trying to work out what type of things may/may not be occurring that prevent the gc from recognizing the object is no longer in use. FromAsync returns a Task object which is obtained from TaskCompletionSource which has a class variable “source” that holds the value of the Task it in turn gets from the new Task invocation.

Here's the test case. It also includes a case using StartNew() where there is no explosion in memory use. The initial Test3Task below did not use the ContinueWith but to see if it was something we weren't cleaning up we put it in (to no effect). [And no, the listening variable used below is not used - there were plans to make the test more intelligent but a do forever was just as good.]

using System;
using System.Threading;
using System.Threading.Tasks;

namespace task_test
{
    class MainClass
    {
            public static void Main (string[] args)
            {
                    // Test3 - Leaky
                    var t = new Task3Test();

                    // Test4 - Doesn't leak
                    // var t = new Task4Test();

                    t.Run();

            }
    }

    public class BaseTask
    {
            public int GetRandomInt(int top)
            {
                    Random random = new Random();

                    return random.Next(1,top);
            }
    }

    public class FibArgs
    {
            public byte[] data;
            public int n;
    }

    public class Fib
    {
            public int Calculate(FibArgs args)
            {
                    int n = args.n;

                    int a = 0;
                    int b = 1;
                    // In N steps compute Fibonacci sequence iteratively.
                    for (int i = 0; i < n; i++)
                    {
                            int temp = a;
                            a = b;
                            b = temp + b;
                    }
                    Console.WriteLine("ThreadId: {2}, fib({0}) = {1}", n, a, Thread.CurrentThread.GetHashCode());
                    return a;
            }
    }

    public class Task3Test : BaseTask
    {
            public void Run()
            {
                    bool listening = true;
                    long i = 0;
                    while (listening)
                    {
                            i++;

                            Func<int> fun = () => {
                                    int n = GetRandomInt(100);
                                    Fib f = new Fib();
                                    FibArgs args = new FibArgs();
                                    args.n = n;

                                    return f.Calculate(args);
                            };

                            var t = Task<int>.Factory.FromAsync(fun.BeginInvoke, fun.EndInvoke, null);
                            t.ContinueWith( x => { 
                                                    if (x.IsCompleted) {
                                                            x.Dispose();
                                                            x = null;
                                                    }
                                            }
                                    );
                    }
            }
    }

    public class Task4Test : BaseTask
    {
            public void Run()
            {
                    bool listening = true;
                    long i = 0;
                    while (listening)
                    {
                            int n = GetRandomInt(100);
                            Fib f = new Fib();
                            FibArgs args = new FibArgs();
                            args.n = n;

                            Task.Factory.StartNew(() => f.Calculate(args), TaskCreationOptions.LongRunning)
                                    .ContinueWith(x => {
                                            if(x.IsFaulted)
                                            { 
                                                    Console.WriteLine("OOPS, error!!!");
                                                    x.Exception.Handle(_ => true); //just an example, you'll want to handle properly

                                            }
                                            else if(x.IsCompleted)
                                            {
                                                    Console.WriteLine("Cleaning up task {0}", x.Id);
                                                    x.Dispose();
                                            }
                                    }
                            );
                    }
            }

    }
}

These symptoms only seem to affect x86_64 as when I run on s390x I have no problems with Boehm or sgen. However, if I leave the WriteLine in the Calculate method I do see exponential memory consumption on s390x as well (the heapshot reports are very very different hen running with and without that statement). Removal of that statement from x86_64 has no effect - it grows regardless.

The symptoms that the above test case exhibits are also experienced on an application that only creates a few tasks per second.
Comment 1 Miguel de Icaza [MSFT] 2013-06-20 20:52:07 UTC
Jeremie, could you eyeball this?

Any info you can provide would be useful.   I am asking Martin to look at this.
Comment 2 Nikita Tsukanov 2013-12-24 05:28:54 UTC
	if (i > 1000000)
		listening = false;
	}
Thread.Sleep (2000);
while(true)
{
	GC.Collect (10, GCCollectionMode.Forced);
	Thread.Sleep (1000);
}
Thread.Sleep (-1);

Added this at the end of the while block in Test3. As I can see it consumes about 1.3 Gb and after a while releases it going back to 44MB RES. So there is no memory leak, runtime just can't keep up with the speed you are creating new tasks, so they stay scheduled forever.

The weird thing is that a single task consumes about 1MB RAM.
Comment 3 Nikita Tsukanov 2013-12-24 05:50:00 UTC
Oh, I've miscalculated. It's not 1MB per task, it's 1KB per task which is acceptable amount.
Comment 4 marcos.henrich 2014-08-11 06:45:08 UTC
This is not a GC issue but a TaskScheduler issue. 
Tasks can be created at a faster pace than they are completed.

I wrote a small program than can shows us the problem:
https://gist.github.com/esdrubal/9cca12fc760abbe01743

Whereas .NET has rarely more than 3000 running tasks, Mono task count diverges.

It is worse when the tasks take more time to complete (eg: doing Console.WriteLine)
Comment 5 Rodrigo Kumpera 2014-08-11 14:35:24 UTC
This is not a bug. The same behavior can be observed on .net.

The issue is that you're queueing tasks faster than the system can process them.