Bug 461 - sgen gc doesn't actually multiprocess
Summary: sgen gc doesn't actually multiprocess
Status: RESOLVED FIXED
Alias: None
Product: Runtime
Classification: Mono
Component: GC ()
Version: unspecified
Hardware: PC Linux
: --- major
Target Milestone: ---
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2011-08-25 19:41 UTC by dennis
Modified: 2011-11-23 18:10 UTC (History)
2 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description dennis 2011-08-25 19:41:17 UTC
I have a multiprocess app which when run under the boehm gc runs on the expected number of processors. When run under sgen it only uses a single processor. 

I can't run this with large data sets under boehm because of heap allocation errors (Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS). I am not sure if there is something I have stuffed up in my code but I don;'t think so. My dictionaries were concurrentdictionaries, but I changed them - made no diff.

I also noticed that when reading the files into memory, it is much slower under sgen than boehm.


//
        Dictionary<string, List<transcript>> transcripts = new Dictionary<string, List<transcript>>();
        Dictionary<string, List<transcript>> qryRef = new Dictionary<string, List<transcript>>();

//Code excerpt
ParallelOptions po = new ParallelOptions() { MaxDegreeOfParallelism = NoProc };

            Parallel.ForEach(transcripts, tsRefList =>
            {
                //Need to check if there is a CDS Reference. No point continuing on this chromosome if there isn't. 
                if (!qryRef.ContainsKey(tsRefList.Key) && !WriteErrorMessage(String.Format("No reference entry for {0}.", tsRefList.Key))) return;
                Console.Error.WriteLine("Processing " + tsRefList.Key);

                int i = 0;
                List<transcript> tsRefOrdered = tsRefList.Value.OrderBy(x => x.chromStart).ToList();
                List<transcript> tsQryOrdered = qryRef[tsRefList.Key].OrderBy(x => x.chromStart).ToList();
                List<transcript> wipTSList = new List<transcript>();

                //Create CDS for overlapping entries. Lists are ordered so work our way through
                foreach (transcript tsRef in tsRefOrdered)
                {
                    try
                    {
                        //We store the list of transcripts overlapping each transcript in a temporary list
                        //due to variances in the target transcript list end position. Check the previous iteration and
                        //add only those transcripts that do not finish earlier on the chrom to the current
                        List<transcript> wipTS_tempList = new List<transcript>();
                        foreach (transcript wipTS in wipTSList)
                            if (wipTS.chromEnd > tsRef.chromStart) wipTS_tempList.Add(wipTS);

                        // Add any new transcripts from the reference until beyond the end of the query
                        while (i < tsQryOrdered.Count && tsQryOrdered[i].chromStart <= tsRef.chromEnd)
                        {
                            wipTS_tempList.Add(tsQryOrdered[i]);
                            i++;
                        }
                        wipTSList = wipTS_tempList.ToList(); //Copy the list

                        //At this point wipTSList should contain a list of all refTranscripts overlapping inTranscripts (+ some beyond)
                        //Work through them and output query transcripts meeting overlap criteria, and output stats if required

                        int olap = 0;
                        foreach (transcript tsQry in wipTSList)
                        {
                            Tuple<int, int> olapSet = tsRef.getTranscriptOverlap(tsQry, useStrand);
                            if (olapSet == null) continue;
                            olap += (olapSet.Item2 * tsQry.score);
                        }

                        double ntcover = (double)olap / (double)tsRef.mRNASize;
                        tsRef.attributes.Add(ntcover.ToString("n2"));

                        Console.WriteLine(tsRef.ToString(true));
                    }
                    catch (Exception ex)
                    {
                        Console.Error.WriteLine(String.Format("Error processing query ID {0}{2}{2}{1}", tsRef.name, ex.Message, Environment.NewLine));
                    }
                }
            });
Comment 1 Rodrigo Kumpera 2011-08-26 07:38:24 UTC
Did you enable the parallel collector? To do so set the MONO_GC_PARAMS to "major=marksweep-par".
Comment 2 dennis 2011-09-01 02:27:00 UTC
I thought this would work, but alas, as soon as I set the environment variable, it fails with the following. It is compiled


* Assertion at sgen-workers.c:236, condition `num_workers <= sizeof (workers_primes) / sizeof (workers_primes [0])' not met

        mono-sgen() [0x492960]
        /lib64/libpthread.so.0(+0xf5d0) [0x2aaaab33f5d0]
        /lib64/libc.so.6(gsignal+0x35) [0x2aaaab580945]
        /lib64/libc.so.6(abort+0x181) [0x2aaaab581f21]
        mono-sgen() [0x5f629e]
        mono-sgen() [0x5f6476]
        mono-sgen() [0x5930c5]
        mono-sgen() [0x5993fc]
        mono-sgen() [0x513d6f]
        mono-sgen() [0x4199e1]
        mono-sgen(mono_main+0x367) [0x46fbc7]
        /lib64/libc.so.6(__libc_start_main+0xe6) [0x2aaaab56cbc6]
        mono-sgen() [0x4142f9]

Debug info from gdb:

Missing separate debuginfo for /lib64/libm.so.6
Try: zypper install -C "debuginfo(build-id)=365e4d2c812908177265c8223f222a1665fe1035"
Missing separate debuginfo for /lib64/librt.so.1
Try: zypper install -C "debuginfo(build-id)=81a3a96c7c0bc95cb4aa5b29702689cf324a7fcd"
Missing separate debuginfo for /lib64/libdl.so.2
Try: zypper install -C "debuginfo(build-id)=fbb2084bce1dd7adc76d9ef90319cf89b601db94"
Missing separate debuginfo for /lib64/libpthread.so.0
Try: zypper install -C "debuginfo(build-id)=1f39dc121cb7220d95ee27e00f745c2832346070"
Missing separate debuginfo for /lib64/libc.so.6
Try: zypper install -C "debuginfo(build-id)=609a6e344dfdc723c6c768cae37c3d91373a731b"
Missing separate debuginfo for /lib64/ld-linux-x86-64.so.2
Try: zypper install -C "debuginfo(build-id)=31d4dbbe49e777eb2dceeb2b6cb722d01a7b35dc"
Missing separate debuginfo for /lib64/libgcc_s.so.1
Try: zypper install -C "debuginfo(build-id)=3f06bcfc74f9b01780d68e89b8dce403bef9b2e3"
Mono support loaded.
[Thread debugging using libthread_db enabled]
0x00002aaaab33e560 in __read_nocancel () from /lib64/libpthread.so.0
* 1 Thread 0x2aaaab8acba0 (LWP 23500)  0x00002aaaab33e560 in __read_nocancel () from /lib64/libpthread.so.0

Thread 1 (Thread 0x2aaaab8acba0 (LWP 23500)):
#0  0x00002aaaab33e560 in __read_nocancel () from /lib64/libpthread.so.0
#1  0x0000000000492adf in mono_handle_native_sigsegv (signal=<value optimized out>, ctx=<value optimized out>) at mini-exceptions.c:2207
#2  <signal handler called>
#3  0x00002aaaab580945 in raise () from /lib64/libc.so.6
#4  0x00002aaaab581f21 in abort () from /lib64/libc.so.6
#5  0x00000000005f629e in monoeg_g_logv (log_domain=0x0, log_level=G_LOG_LEVEL_ERROR, format=0x60ad18 "* Assertion at %s:%d, condition `%s' not met\n", args=
    0x7fffffffd1b0) at goutput.c:134
#6  0x00000000005f6476 in monoeg_assertion_message (format=0x60ad18 "* Assertion at %s:%d, condition `%s' not met\n") at goutput.c:154
#7  0x00000000005930c5 in workers_init (num_workers=16) at sgen-workers.c:236
#8  0x00000000005993fc in mono_gc_base_init () at sgen-gc.c:7056
#9  0x0000000000513d6f in mono_init_internal (filename=0x7fffffffdbd8 "/home/uqdgasco/bin/scripts/pantsNJacket.exe", exe_filename=
    0x7fffffffdbd8 "/home/uqdgasco/bin/scripts/pantsNJacket.exe", runtime_version=0x0) at domain.c:1285
#10 0x00000000004199e1 in mini_init (filename=0x7fffffffdbd8 "/home/uqdgasco/bin/scripts/pantsNJacket.exe", runtime_version=0x0) at mini.c:6320
#11 0x000000000046fbc7 in mono_main (argc=8, argv=0x7fffffffd6e8) at driver.c:1746
#12 0x00002aaaab56cbc6 in __libc_start_main () from /lib64/libc.so.6
#13 0x00000000004142f9 in _start () at ../sysdeps/x86_64/elf/start.S:113

=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
Comment 3 dennis 2011-09-01 19:26:24 UTC
More information.

Setting major=marksweep-par causes the exception even on a simple hello world using mono-sgen.

Cheers
D
Comment 4 Rodrigo Kumpera 2011-11-23 18:10:37 UTC
Very complex samples now work with marksweep-par.

If you're having issues, reopen this bug and post a working sample that shows your issue, a code snippet is of no use to figure out what's wrong.