Bug 6619 - ReadContentAsBase64 causes OutOfMemoryException with large SOAP responses
Summary: ReadContentAsBase64 causes OutOfMemoryException with large SOAP responses
Status: RESOLVED FIXED
Alias: None
Product: Class Libraries
Classification: Mono
Component: System.XML ()
Version: unspecified
Hardware: PC Linux
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2012-08-21 07:10 UTC by xamarinbugzilla
Modified: 2015-03-17 06:32 UTC (History)
4 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Test case (2.45 KB, text/plain)
2012-08-21 07:11 UTC, xamarinbugzilla
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description xamarinbugzilla 2012-08-21 07:10:37 UTC
I attached a test program which simply creates a large XML file where the content is base64 encoded binary data.
Then it reads it using XmlReader. It uses ReadContentAsBase64 in small nice chunks of 4kb.

With Microsoft's .Net Framework this works without consuming much RAM.

With Mono the whole XML-node with all content is loaded into an internal variable.

This crashes my program on Linux/Mono if it receives large responses via SOAP.

The high memory usage happens inside
private void ReadText (bool notWhitespace)
where one can clearly see that every character is read, inspected for entities etc. and appended to a StringBuilder.

This happens until < is encountered.

Then later the base64 decoder gets stuff from this buffer (or worse: First the StringBuilder is converted to a string via the property Value of the XmlReader).

Is there any chance that this might be improved to read the contents internally only in chunks?

The ReadContentAsBase64 would have to grab chunks of base64 encoded characters from the XmlReader which would have to read it from the base stream.

I guess that this would also easily affect MonoTouch programms because mobile phones are especially restricted for RAM and need efficient, streaming implementations.

Would such a design change inside the stable System.XML code even be possible?

Am I overlooking some possible way to read such files without using O(n) bytes of RAM?

Does anybody know about tiny, lightweight implementations of simple XML in C# that I could use to read those files?

Thank you very much for looking into this!
Comment 1 xamarinbugzilla 2012-08-21 07:11:11 UTC
Created attachment 2382 [details]
Test case
Comment 2 Atsushi Eno 2015-03-17 06:32:34 UTC
We imported referencesource for XmlReader and should not be responsible on this kind of problem (should be UPSTREAM if it is still problematic).