Bug 24118 - [regression] Simple cross-AppDomain call is more than 100 times slower in Mono
Summary: [regression] Simple cross-AppDomain call is more than 100 times slower in Mono
Status: RESOLVED FIXED
Alias: None
Product: Runtime
Classification: Mono
Component: Remoting ()
Version: 3.8.0
Hardware: PC Linux
: --- normal
Target Milestone: ---
Assignee: Alexander Kyte
URL:
Depends on:
Blocks:
 
Reported: 2014-10-28 17:16 UTC by Arseny Vakhrushev
Modified: 2015-03-24 10:58 UTC (History)
5 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Test-case (557 bytes, text/plain)
2014-10-28 17:16 UTC, Arseny Vakhrushev
Details
straced test-case of 10 thousand calls to Test.Do() (399.42 KB, text/plain)
2015-03-18 16:55 UTC, Arseny Vakhrushev
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Arseny Vakhrushev 2014-10-28 17:16:09 UTC
Created attachment 8535 [details]
Test-case

I use AppDomains for better isolation between host and guest code in my application. Somewhere in the past (I believe 3.4 or earlier), I observed pretty poor performance of simple calls between two domains within the same application on OS X. It wasn't my target platform, so I didn't pay much attention since on Linux, things worked alright. I thought it might be due to particular OS X implementation specifics.

But lately, the same AppDomain performance test started to show the same poor results on Linux which became critical. The same test case works alright on Windows. I tried both virtual and physical environments - results are the same.

Output I get:

OS X:

$ mono TestRemoting.exe
Rate: 13992.95/sec

$ mono --version
Mono JIT compiler version 3.8.0

Linux:

$ mono TestRemoting.exe
Rate: 13422.53/sec
$ mono --version
Mono JIT compiler version 3.10.0


Windows 8.1:

E:\devel\tmp>TestRemoting.exe
Rate: 1647150.68/sec


The results on Windows look pretty much adequate. The numbers on Mono look like the runtime hits some thread synchronization primitive which shouldn't be probably involved in such a scenario.
Comment 1 Zoltan Varga 2014-10-29 18:58:18 UTC
This is a regression caused by the changes to ExecutionContext in 4b4ddb66b315759d030a10153f47d5b17dc03af8. In previous revisions, calling
CallContext.CreateLogicalCallContext (false) would return false, causing RemotingServices:SerializeCallData () to skip any serialization, and simply return null.
Comment 2 Zoltan Varga 2014-11-03 15:00:39 UTC
Marek, could you look at this ?
Comment 3 Arseny Vakhrushev 2014-12-17 00:22:20 UTC
I wonder if there is any progress or time estimates with this bug. Thanks!
Comment 4 Alexander Kyte 2015-03-16 14:44:19 UTC
After changing CreateLogicalCallContext to return false in the same situation that it would before 4b4ddb66b315759d030a10153f47d5b17dc03af8, I have noticed a negligible performance increase. It still serializes, it just doesn't use a wrapper object. 

I can make the code not serialize and simply return null, but I'm not sure that this preserves the intended semantics of the API.
Comment 5 Arseny Vakhrushev 2015-03-18 16:53:18 UTC
Ok, I may be way out of tune here, but it seems to me that no feasible amount of extra serialization or other "expensive" work could lead to such an astonishing 100 times slowdown. I mean it would be something deliberately and expressively "heavy" in that case.

If you run the provided test-case under the 'strace' utility, you will see a whole bunch of syscalls being made while making cross-appdomain calls. You will see TONS of:

cheap - gettimeofday, brk, mmap, munmap
expensive - stat("/etc/localtime"), futex
threading - tgkill

The presence of 'tgkill' and 'futex' may indicate that there is some threading involved which is potentially what causes the lag. This was my first guess when I filed the bug.

Please see the log attached. To generate it, I enclosed 10 thousand Test.Do() calls in two Console.WriteLine statements. You can find the beginning and the end by searching for '>>>>>>' and '<<<<<<' respectively.
Comment 6 Arseny Vakhrushev 2015-03-18 16:55:03 UTC
Created attachment 10402 [details]
straced test-case of 10 thousand calls to Test.Do()
Comment 7 Alexander Kyte 2015-03-18 17:18:39 UTC
https://github.com/mono/mono/pull/1632

If you apply this commit yourself, you'll see that the speedup is indeed significant.
Comment 8 Arseny Vakhrushev 2015-03-18 17:24:21 UTC
Looks good. I am looking forward to it in an upcoming release. Hopefully, my input can help identify all other accompanying issues as well.
Comment 9 Alexander Kyte 2015-03-24 10:58:13 UTC
I'm closing this bug.