Bug 39279 - Thread pool sizes are incorrect on linux when processor affinity is set, which complicates running mono applications in docker containers
Summary: Thread pool sizes are incorrect on linux when processor affinity is set, whic...
Status: RESOLVED FIXED
Alias: None
Product: Runtime
Classification: Mono
Component: General ()
Version: 4.2.0 (C6)
Hardware: PC Linux
: --- normal
Target Milestone: ---
Assignee: Ludovic Henry
URL:
Depends on:
Blocks:
 
Reported: 2016-03-02 00:48 UTC by Ben Burns
Modified: 2016-03-06 21:58 UTC (History)
3 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Ben Burns 2016-03-02 00:48:03 UTC
When running a mono application as a cgroup task which is limited to a particular cpuset (such as when the application is running under a docker container which was created using --cpuset-cpus argument), mono will create its various threadpools sized as though it has access to all CPU cores.

This complicates running in a multi-tenant dockerized environment, as per above, docker uses cgroups to set processor affinity for each container. For more information about this, see [1] below.

This cause of this problem appears to be that when running on linux, mono appears to set thread pool sizes based on the number of system processors. At present this count is determined either by parsing /proc/stat, or via the glibc sysconf extension _SC_NPROCESSORS_ONLN. When running as a cgroup task limited to a specific cpuset (as is the case with docker), or when running with processor affinity set, these values still report the full core count of the host machine.

It seems likely that mono should instead account for processor affinity when determining these thread sizes. It's arguable that mono may also wish to consider CPU affinity when reporting System.Environment.Processors, as I'm not sure that any other runtime call would exist to signal to code within the VM that it is running on a reduced set of processors.

1: https://github.com/docker/docker/issues/20770
Comment 1 Ben Burns 2016-03-02 00:58:15 UTC
er, that should have read System.Environment.ProcessorCount :-/
Comment 2 Ludovic Henry 2016-03-04 14:08:40 UTC
Hi Ben,

Here is a PR to fix this issue: https://github.com/mono/mono/pull/2722. Could you please review it, and check that it fixes your issue, as I don't have any docker container set up or anything.

Thank you,
Ludovic
Comment 3 Ben Burns 2016-03-06 21:58:33 UTC
Thanks heaps for the quick turn-around, Ludovic! At first glance it looks like it should work. Since it appears to be merged into master I'll see if I can't fire up a test container running our application which has been experiencing this issue.

I'll likely only be able to get to this tomorrow, however. Either way I'll keep you posted.