Bug 18083 - Memory Leak
Summary: Memory Leak
Status: RESOLVED FIXED
Alias: None
Product: Runtime
Classification: Mono
Component: GC ()
Version: unspecified
Hardware: PC Linux
: --- normal
Target Milestone: ---
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2014-02-28 07:57 UTC by Alexandre Faria
Modified: 2014-03-05 21:17 UTC (History)
4 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Alexandre Faria 2014-02-28 07:57:58 UTC
For some time now, I have been detecting major memory leaks in mono, recently it got worse, making the execution time of production code increase in a kind of exponential way, ending up crashing because it couldn't allocate more memory.
An example (sgen) was that partial was about 30m and combined 1h30m.

A few months ago, the same production code would finish a lot faster without crashing, I'm guessing back then I could be using Boehm.
Even with Boehm I suspected the same problem was present, although on a less severe scale. As partial executions were always faster, sometimes a lot faster.

The problem is to find a simple test case that exhibits the problem fast so that you can work on it.


Recently I was able to find a test case that would exhaust all the memory and move into swap, when it wasn't supposed to: 300 with consume about 4GB of ram.

using System;

public class Example
{
  public static void Main()
  {
    for(int i=0; i<1000; i++)
    {
      System.Console.WriteLine("\n\nIteration " + i);
      AppDomain ad = AppDomain.CreateDomain("ChildDomain");
      AppDomain.Unload(ad);
    }
  }
}




Mono JIT compiler version 3.4.0 (master/e98625a Qui Fev 27 19:05:39 WET 2014)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	LLVM:          yes(3.4svn-mono-mono/e656cac)
	GC:            sgen

Ubuntu 13.10
Comment 1 Alexandre Faria 2014-02-28 07:58:32 UTC
PS: Very important, you will have to run that test case with --llvm.
Comment 2 Mark Probst 2014-02-28 17:12:12 UTC
I cannot reproduce this.  This program (on SGen, with or without --llvm) exhibits totally stable memory usage on Linux/AMD64 for me, and finishes in less than 3 minutes.

Any particulars that might be important for reproduction?
Comment 3 Alexandre Faria 2014-02-28 17:29:46 UTC
I don't think so, I just use the flags --with-large-heap=yes --enable-llvm.

I use a ubuntu 13.10 i940 12GB.
Comment 4 Alexandre Faria 2014-03-01 14:11:28 UTC
I just recompiled mono again from git, this time only with --enable-llvm and I can still reproduce the problem executing with --llvm.

Then I recompiled mono again, this time only with --enable-loadedllvm and I can still reproduce the problem executing with --llvm.

This happens on Ubuntu 13.10:
Mono JIT compiler version 3.4.0 (master/eef1f28 Sáb Mar  1 18:36:37 WET 2014)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	LLVM:          yes(3.4svn-mono-mono/e656cac)
	GC:            sgen

On mono version from Ubuntu 13.10 I can't reproduce the problem, and each iteration is a lot faster.

Mono JIT compiler version 2.10.8.1 (Debian 2.10.8.1-5ubuntu2)
Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	LLVM:          supported, not enabled.
	GC:            sgen

How can I provide more useful info?
Comment 5 Mark Probst 2014-03-03 15:09:43 UTC
I can reproduce it now.  You're right, using LLVM leads to unbounded memory growth.

Zoltan, could you take a look at this?
Comment 6 Zoltan Varga 2014-03-03 21:03:30 UTC
Fixed most of the leaks, the memory usage is still increasing with the testcase, but much more slowly.
Comment 7 Alexandre Faria 2014-03-04 11:35:28 UTC
Yes, its a huge improvement, from about 4G for 300, to about 350M for 1000.

On my machine, 1000 takes about 55m to complete, is that normal?
I understand llvm is heavier.
Comment 8 Zoltan Varga 2014-03-04 11:56:49 UTC
LLVM is an optimizing compiler, and every time an appdomain is unloaded, all the compiled code in the appdomain is thrown out,  and needs to be recompiled.
Also, you might need to compile an optimized version of lllvm by configuring it as:
./configure --enable-optimized=yes --enable-assertions=no
Comment 9 Alexandre Faria 2014-03-04 13:43:10 UTC
Yes, I do understand that, and in this case its useful as it allows to uncover memory leaks.

Thanks for the additional flags.

The reasoning behind my question is that the sample isn't loading additional assemblies, so its optimizing assemblies that were already optimized in the main AppDomain.
So in theory it makes sense not to optimize again, although I understand that llvm, some optimization or isolation requirements may prevent that, that's why I asked.
Comment 10 Zoltan Varga 2014-03-04 14:10:32 UTC
Appdomains require some remoting code which lives in the child domain, so creating an appdomain compiles a lot of code in that appdomain.
Comment 11 Zoltan Varga 2014-03-05 17:39:20 UTC
Fixed a bunch of small leaks, memory usage still seems to slowly increase, but that might be normal.
Comment 12 Alexandre Faria 2014-03-05 19:55:46 UTC
Thanks!

I have complex code that in about 8m leaks about 1GB, is there an easy way to uncover what it is leaking?
Comment 13 Zoltan Varga 2014-03-05 20:01:57 UTC
Not really, it could be a leak in the application, or in the runtime. For application level leaks, you can try
the mono logging profiler, see 'man mprof-report' for more info.
Comment 14 Alexandre Faria 2014-03-05 21:17:15 UTC
Thanks!