Bug 10121 - Parallel.ForEach
Summary: Parallel.ForEach
Status: RESOLVED FIXED
Alias: None
Product: Class Libraries
Classification: Mono
Component: System ()
Version: unspecified
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: Untriaged
Assignee: Jérémie Laval
URL:
Depends on:
Blocks:
 
Reported: 2013-02-07 06:25 UTC by ian
Modified: 2015-03-17 02:43 UTC (History)
4 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description ian 2013-02-07 06:25:06 UTC
In .Net we can use the .Net4 and TPL classes to implement a queue with multiple consumers using a BlockCollection<T> and Parallel.ForEach to create multiple consuming enumerables.

It's very simple and works amazingly well:

     Parallel.ForEach(_authenticatedCommandQueue.GetConsumingEnumerable(), options, methodToExecute);

The problem is that this doesn't not work in MONO - single thread calls to GetConsumingEnumerable work fine, but the Parallel.ForEach simple results in NO consumers.

This is a big shame and means we may have to create a lot of MONO only code in an area that's not trivial. Is this a known bug?

It's on Mono 3.0.x

Thanks
ian
Comment 1 ian 2013-02-19 12:22:07 UTC
Huddle are fully licensed Xamarin.Mac users with a quick response time on our issues. As a result I've created the attached MonoDevelop proj file that demonstrates our issue.

It is a simple producer/consumers using a BlockingCollection, with multiple consumers via the TPL.

The exact same code works as expected with .Net. However nothing happens (no consumers are generated) within Mono.

I've enclosed the code below. Here's the output:

Output - VisualStudio
=======================================================================================================================
Start
LOG:STARTED multi consume
Done-multi
Consuming - A
Consuming - B
Consuming - C

Output - MonoDevelop
=======================================================================================================================
Start
LOG:STARTED multi consume
Done-multi


Source
=======================================================================================================================
using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;

namespace BlockCollectionParallelConsumerTest
{
    class MainClass
    {
        public static void Main(string[] args)
        {
            var cp = new ProducerConsumer();
            Console.WriteLine("Start");
            cp.StartParallelQueue();
            cp.AddToQueue("A");
            cp.AddToQueue("B");
            cp.AddToQueue("C");
            Console.WriteLine("Done-multi");
            Console.ReadLine();
        }
    }
    public class ProducerConsumer
    {
        private BlockingCollection<string> queue;
        private Task _taskToConsumeQueue;
        private CancellationToken _taskCancelToken;

        public void StartParallelQueue()
        {
            queue = new BlockingCollection<string>();
            _taskCancelToken = new CancellationTokenSource().Token;
            _taskToConsumeQueue = Task.Factory.StartNew(ConsumeQueueItems, _taskCancelToken);
            WriteLog("STARTED multi consume");
        }
        private void ConsumeQueueItems()
        {
            var options = new ParallelOptions { MaxDegreeOfParallelism = 10 };
            Parallel.ForEach(queue.GetConsumingEnumerable(), options, ConsumeAnItem);
        }
        private void ConsumeAnItem(string message)
        {
            Console.WriteLine("Consuming - " + message);
        }
        public void AddToQueue(string toAdd)
        {
            queue.Add(toAdd);
        }
        private void WriteLog(string p)
        {
            Console.WriteLine("LOG:" + p);
        }
    }
}
Comment 2 Jérémie Laval 2013-02-19 14:12:30 UTC
Hey,

So this problem is by design and illustrates a different behavior between Microsoft .NET and us.

Essentially, our implementation of Parallel.ForEach will try to accumulate elements from the given enumerable (5 for each worker) and only then starts to process them.

In your case that limit is never reached because you stop adding element and you don't close the BlockingCollection with CompleteAdding

For that specific test case, either adding more elements or calling CompleteAdding after you have finished adding A, B, C fixes the issue.

Couple of workarounds:

  - Change the original code to call CompleteAdding if no more elements are expected
  - Use a foreach loop (inside a Parallel.Invoke for instance to replicate Parallel.ForEach) to have unbuffered access
  - Take the Mono code and change the buffer limit[0]

Hope that helps.

[0]  https://github.com/mono/mono/blob/master/mcs/class/corlib/System.Threading.Tasks/Parallel.cs#L349 

PS: Priority support needs to go through the e-mail address as listed in your Xamarin account - https://store.xamarin.com/account/Products/
Comment 3 Marek Safar 2015-03-17 02:43:48 UTC
Fixed in Mono 4.0