Bug 53685 - "System.IO.IOException: Too many open files" causes "The Xamarin license file is invalid" and various other symptoms after using Xamarin Studio for a while on certain projects (up to 4-5 times a week)
Summary: "System.IO.IOException: Too many open files" causes "The Xamarin license file...
Status: RESOLVED FIXED
Alias: None
Product: Class Libraries
Classification: Mono
Component: System ()
Version: master
Hardware: PC Mac OS
: --- major
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2017-03-21 21:15 UTC by Brendan Zagaeski (Xamarin Team, assistant)
Modified: 2017-09-11 11:03 UTC (History)
5 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Brendan Zagaeski (Xamarin Team, assistant) 2017-03-21 21:15:58 UTC
"System.IO.IOException: Too many open files" causes "The Xamarin license file is invalid" and various other symptoms after using Xamarin Studio for a while on certain projects (up to 4-5 times a week)




## Note: not yet replicated locally on my machine

This report is currently only surfacing information from the original end user: I have not attempted to replicate the issue locally by stress testing Xamarin Studio (for example with a large project).  In part the hope of this report is that the attached information about open file descriptors might hint at a code path in Xamarin Studio that is leaking file descriptors.  (Or alternately, perhaps the attached information will be sufficient to suggest that the issue has a good chance of being resolved already in a more recent version of Xamarin Studio or Visual Studio for Mac.)




## Usage conditions related to the symptom

- The symptom happens after the user has spent some time (roughly 1 day or less) using Xamarin Studio for usual IDE activities (editing, building, debugging, etc.) on a Xamarin.iOS project.

- Perhaps of interest, the issue does not seem to have appeared for the user's coworkers so far.  It is not yet clear what might be unique to the user's environment.  As one small test, the user tried removing and reinstalling Xamarin Studio, but that did not change the behavior.

- The complete Ide*.log file and output from `lsof` will be shared with the Xamarin team non-publicly in a follow-up comment.




## Excerpt of the top of the call stack from the Ide*.log file

> MSBuild project could not be evaluated
> System.IO.IOException: Too many open files
>   at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x00335] in /private/tmp/source-mono-4.6.0/bockbuild-mono-4.6.0-branch/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/corlib/System.IO/FileStream.cs:290 
>   at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in /private/tmp/source-mono-4.6.0/bockbuild-mono-4.6.0-branch/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/corlib/System.IO/FileStream.cs:97 
>   at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
>   at System.IO.File.OpenRead (System.String path) [0x00000] in /private/tmp/source-mono-4.6.0/bockbuild-mono-4.6.0-branch/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/corlib/System.IO/File.cs:357 
>   at MonoDevelop.Projects.MSBuild.FileUtil.GetTextFormatInfo (System.String file) [0x00008] in /Users/builder/data/lanes/3509/7fabf929/source/monodevelop/main/src/core/MonoDevelop.Core/MonoDevelop.Projects.MSBuild/FileUtil.cs:41 
>   at MonoDevelop.Projects.MSBuild.MSBuildProject.Load (System.String file, MonoDevelop.Projects.MSBuild.MSBuildXmlReader reader) [0x00019] in /Users/builder/data/lanes/3509/7fabf929/source/monodevelop/main/src/core/MonoDevelop.Core/MonoDevelop.Projects.MSBuild/MSBuildProject.cs:212 



## Preliminary observations about the output from `lsof`

Command used:
lsof -p $( pgrep XamarinStudio ) > "$HOME/Desktop/XamarinStudioOpenFiles.txt"


There are 766 open file names with the pattern:
> /private/var/folders/*/*/T/mono.anonmap*
There are 179 open files that have no file name (it seems):
> ->(none)



## Environment information for the user (from Ide*.log)

Xamarin Studio 6.1.3 (build 19)
Mono 4.6.2 (mono-4.6.0-branch/ac9e222) (64-bit)
Mac OS 10.11.6

Xamarin.iOS 10.3.1.7
Comment 4 Brendan Zagaeski (Xamarin Team, assistant) 2017-03-21 21:51:51 UTC
## Possible next steps for the original user (while waiting on further feedback from the Xamarin engineering team about whether the information collected so far does suggest a particular problematic code path in Xamarin Studio)

- Just to be thorough, if they get a chance, the user can reply back on this bug briefly the next time they see the issue in the latest version of Xamarin Studio 6.2 to record that it is indeed still a problem in that version.

- It might be interesting to check whether running Xamarin Studio under a new clean user account on the same Mac stops the problem.  This idea comes from the observation that the other coworkers have not seen the issue.  If a new clean user account on the same Mac does not hit the problem, then we can work on pinning down the preferences or other stored files in the original account that are responsible the difference in behavior.

Thanks!
Comment 5 Marius Ungureanu 2017-03-23 15:54:37 UTC
The only piece of code which allocates files via mmap is actually Roslyn for the typesystem documents.

Based on my analysis, there are two possibilities on the plethora of mmap'ed files being created by XS:
1) The solution is indeed huge enough to create those many memory mapped files, although that would not explain the lack of filename
2) We keep a strong reference to the documents somewhere (not sure where) that would prevent the files from getting GC'd.
3) Mono bug explained below

Although 2) can be an issue, I'd rather point out that the bug below is a better explanation.

https://github.com/mono/mono/blob/0bcbe39b148bb498742fc68416f8293ccd350fb6/mcs/class/System.Core/System.IO.MemoryMappedFiles/MemoryMappedFile.cs#L113
https://github.com/mono/mono/blob/0bcbe39b148bb498742fc68416f8293ccd350fb6/mcs/class/referencesource/System.Core/System/IO/MemoryMappedFiles/MemoryMappedFile.cs#L30

Although both are IDisposable, .NET uses a different way of handling the safe handle, instead of having an opaque IntPtr, it has a SafeMemoryMappedFileHandle which handles release of the mmap'ed file even on finalization.

Therefore, I conclude this is a mono bug, exposed by roslyn itself.
Comment 6 Zoltan Varga 2017-03-23 17:43:06 UTC
https://github.com/mono/mono/pull/4584
Comment 7 Marek Safar 2017-09-11 11:03:22 UTC
Closing as the PR was merged