Bug 1741 - Error in downloading file stream from web service (Memory leakage)
Summary: Error in downloading file stream from web service (Memory leakage)
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: Mono runtime / AOT Compiler ()
Version: 1.9.2
Hardware: PC Windows
: Highest normal
Target Milestone: ---
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2011-10-27 19:56 UTC by Nirban Dutta
Modified: 2011-12-01 14:12 UTC (History)
9 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Stack Trace of error from Dalvik debug monitor (4.09 KB, text/plain)
2011-10-27 19:56 UTC, Nirban Dutta
Details
Stacktrace of wcf call failure. (11.54 KB, application/octet-stream)
2011-10-30 19:41 UTC, Nirban Dutta
Details
Demo Test Project to test the WCF call crash (727.97 KB, application/x-zip-compressed)
2011-11-01 01:45 UTC, Nirban Dutta
Details
Test Project with Certificate Validation (730.11 KB, application/x-zip-compressed)
2011-11-04 00:42 UTC, Nirban Dutta
Details
Login successful - Message (20.44 KB, image/png)
2011-11-04 00:43 UTC, Nirban Dutta
Details
Screenshot for Login successful - Message (160.16 KB, image/pjpeg)
2011-11-04 00:44 UTC, Nirban Dutta
Details
Demo Application using Web Reference and WebClient (2.31 MB, application/x-zip-compressed)
2011-11-07 01:53 UTC, Nirban Dutta
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Nirban Dutta 2011-10-27 19:56:14 UTC
Created attachment 784 [details]
Stack Trace of error from Dalvik debug monitor

We are downloading lot of image files  by calling web method once for each file in loop. We receive the file as stream of bytes.
When tested, for a download of about 100 files,  after few 20~30 files the web service call fails and the app freezes and then crashes, at any random file (not specific to any file).

File sizes are normally 0.5MB to 4 MB,  and we have already set timeout etc, as given below:

         binding.SendTimeout = new TimeSpan(0, 20, 0);
         binding.MaxReceivedMessageSize = 2147483647;
         binding.MaxBufferSize = 2147483647;
         svc.Proxy = new ProjectCentreServiceClient(binding, endpoint);
Comment 1 Nirban Dutta 2011-10-27 19:57:36 UTC
I have tested it on Mono for Android 1.2 as well as 1.9.2, it crashes in both.
Comment 2 Nirban Dutta 2011-10-28 01:38:51 UTC
Hi,

While downloading many files, the app crashed every time, but in few occations(1 out of 20), it crashes with the following error:

10-28 16:31:09.693: INFO/mono(5136): Stacktrace:
10-28 16:31:09.693: INFO/mono(5136):   at Mono.Security.Protocol.Tls.RecordProtocol.ReadStandardRecordBuffer (System.IO.Stream) <0x0013f>
10-28 16:31:09.693: INFO/mono(5136):   at Mono.Security.Protocol.Tls.RecordProtocol.ReadRecordBuffer (int,System.IO.Stream) <0x00067>
10-28 16:31:09.693: INFO/mono(5136):   at Mono.Security.Protocol.Tls.RecordProtocol.InternalReceiveRecordCallback (System.IAsyncResult) <0x00123>
10-28 16:31:09.693: INFO/mono(5136):   at System.IO.Stream.BeginRead (byte[],int,int,System.AsyncCallback,object) <0x00113>
10-28 16:31:09.693: INFO/mono(5136):   at Mono.Security.Protocol.Tls.RecordProtocol.BeginReceiveRecord (System.IO.Stream,System.AsyncCallback,object) <0x0015b>
10-28 16:31:09.693: INFO/mono(5136):   at Mono.Security.Protocol.Tls.RecordProtocol.ReceiveRecord (System.IO.Stream) <0x00023>
10-28 16:31:09.693: INFO/mono(5136):   at Mono.Security.Protocol.Tls.SslStreamBase.InternalReadCallback (System.IAsyncResult) <0x00293>
10-28 16:31:09.693: INFO/mono(5136):   at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <0xffffffff>


In all other occations, it crashes with "System.Text.StringBuilder.InternalEnsureCapacity (int) <0x000b7>" as per the attachment.
Comment 3 Nirban Dutta 2011-10-30 19:41:36 UTC
Created attachment 794 [details]
Stacktrace of wcf call failure.

Stacktrace of wcf call failure.
While downloading (200 ~750 files) by calling wcf method GetAttachment in a loop
the application crashes after downloading 36 files.
Comment 4 Nirban Dutta 2011-10-30 19:57:11 UTC
Hi Willem Meints,

I can not provide you exception details as per your request, as this one could not be caught in exception handler. In this specific case, the application freezes without throwing any exception.
I have implemented the exception handling as follows;

1) while calling wcf service

        public AttachmentSyncResponse GetAttachmentFile(RequestItem requestItem, int attachmentId, bool isThumbnail)
        {
            try
            {
                Log.Info("FileCrashInfoND", "Going to download file - " + attachmentId.ToString() + ", isThumbnail - " + isThumbnail.ToString());
                return _serviceClient.GetAttachmentFile(requestItem, attachmentId, isThumbnail);
            }
            catch (Exception ex)
            {
                Log.Info("FileCrashInfoND", "Exception in calling wcf - " + ex.Message.Substring(0,10));
                RaiseSyncException("ServiceManager/GetAttachmentFile", ex, "");
            }
            return null;
        }

2) Also on Activity

a) AndroidEnvironment.UnhandledExceptionRaiser += new EventHandler<RaiseThrowableEventArgs>(AndroidEnvironment_UnhandledExceptionRaiser);

        void AndroidEnvironment_UnhandledExceptionRaiser(object sender, RaiseThrowableEventArgs e)
        {
            Log.Info("FileCrashInfoND", "Exception in calling wcf - " + e.Exception.ToString());
        }

b) Impleted our own exception handler

 PCExceptionRaiser.Current.PCExceptionThrown += OnPCExceptionThrown;

  public void OnPCExceptionThrown(object sender, ParameterEventArgs<PCExceptionHelper> e)
        {
           .....
}

As, you can see in the attached, while downloading (200 ~750 files) by calling wcf method GetAttachmentFile(...) in a loop the application crashes after downloading 36 files, but it does does not throw any exception. I tried to catch exception also in debugging, but no success.

Please let me know, if you need any other information, but we need to get this sorted soon, as our whole application depends on this file sync.
Comment 5 Jonathan Pryor 2011-10-31 16:08:25 UTC
Could you please attach a project that demonstrate the problem? This would make reproducing the issue much easier and faster.

Otherwise, it sounds like https://bugzilla.novell.com/show_bug.cgi?id=648862#c9 in which not all streams are disposed of in a timely fashion.

Thanks,
 - Jon
Comment 6 Nirban Dutta 2011-11-01 01:45:52 UTC
Created attachment 798 [details]
Demo Test Project to test the WCF call crash

Hi,
I have attached a test project, when you run this,

1) First click login button,

2) After login is successful, click Get Files button: This will download files and attachments, you can see the Info messages in Dalvik. After downloading some files, the application will crash, the crash stacktrace is bit different from the actual application, that is because we do lot of different activities in between like saving file to storage, updating sqlite db etc.
But the error looks similar.

If you don't see the error try one more time, the crash will happen almost every time.

In the actual application, I have disposed the objects returened by web service after using it (after saving file/update db) mostly by assigning null.

Please analyse and let me know how to go ahead to fix this.
Comment 7 Nirban Dutta 2011-11-01 18:08:24 UTC
Hi Jon,
Could you please work on this and let us know the progress.

Regards,

Nirban
Comment 8 Nirban Dutta 2011-11-03 18:04:19 UTC
Hi Mono Team/Jon,

I have not heard from you about the status of this bug.

We have our Product delivery date of 11th Nov,2011 and this bug is stopping us from going ahead for the full test cycle.

In any case, we need this to issue to be fixed by 8th Nov,2011, to be able to run our full test cycle and deliver the product on time.

If we cannot deliver on time, there is a financial impact on us, and that's why this issue is so critical.

Please take special effort to resolve this and let us know the progress.

Regards,

Nirban Dutta
Comment 9 Atsushi Eno 2011-11-03 20:35:50 UTC
While we'll have a look for bugfixing, it's not appropriate to expect any bugs fixed in very limited timeframe. You should not depend on unsupported alpha component (which is explicitly documented). You should stop using WCF (either partly or completely) and instead use direct file download via raw HTTP which should be very possible.
Comment 10 Atsushi Eno 2011-11-04 00:30:10 UTC
The sample Login never succeeds.
Comment 11 Nirban Dutta 2011-11-04 00:36:20 UTC
Hi,

This may be because of root certificate, but it works on all our devices.

Can you please add this at the end of the function (public static ProjectCentreProxy ConfigureProxy())

 System.Net.ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, errors) =>
            {
                return true;
            };


Cheers,

Nirban
Comment 12 Nirban Dutta 2011-11-04 00:42:33 UTC
Created attachment 817 [details]
Test Project with Certificate Validation

I am attaching the Test Project with Certificate Validation.

Regards,

Nirban
Comment 13 Nirban Dutta 2011-11-04 00:43:42 UTC
Created attachment 818 [details]
Login successful - Message

Screenshot for Login successful - Message
Comment 14 Nirban Dutta 2011-11-04 00:44:05 UTC
Created attachment 819 [details]
Screenshot for Login successful - Message

Screenshot for Login successful - Message
Comment 15 Atsushi Eno 2011-11-04 01:34:52 UTC
OK thanks, login is now successful, and the results were inconsistent:

1) the same TLS protocol error as in comment #2, which looks like HTTPS-level error.

2) something like runtime crash.

I/MonoTest(14597): Going to download file - 154139305, isThumbnail - True
I/mono    (14597): Stacktrace:
I/mono    (14597): 

3) ANR without any further outputs.

4) GC crash when I inserted GC.Collect() at Activity1.cs line 207 (i.e. inside the download loop).

I/mono    (10483): Stacktrace:
I/mono    (10483):   at System.GC.Collect () [0x00000] in 

None of them looks like WCF issue.

I also suspected that the client code is eating too much resources in very short time and inserted some Thread.Sleep() but it didn't fix the issue.

What should be tried is, to *not* use WCF and communicate with the equivalent HTTP messages and see if the problem still occurs. I'm kind of sure it will.
Comment 16 Atsushi Eno 2011-11-04 01:39:17 UTC
It still tells you why you had better avoid WCF - it hides the true underlying issues. It makes investigating the issues hard (no single person can do that). And no one can help WCF issues now.
This kind of simple file downloading can be done with really simple HTTP(S) message exchanges. It should be doable within short days.
Comment 17 Nirban Dutta 2011-11-04 01:41:11 UTC
Hi Alsushi,

We know the WCF has no issue because the same WCF is consumed by iPhone App, WP7 app, and also desktop app, and its live for long time.

The issue is with Android App, probably it is going out of memery and Mono Garbage collector is unable to clean up.

What do you suggest to resolve this, any special handling of memory?
Can you fix this issue in Mono Code?

Regards,

Nirban
Comment 18 Atsushi Eno 2011-11-04 01:49:40 UTC
What I can suggest in current implementation approach is, still, to create a raw set of messages to "simulate" communication with the server and see if/how it causes the problem. There are individual hackers who are familiar with MfA socket stack, HTTP (maybe it's not in TLS implementation issue),  GC issues, and the runtime issues in general, and they will shed some lights when the problem gets simplified. I'm sure that most of them thought it is *WCF* issue and didn't get involved.
Comment 19 Nirban Dutta 2011-11-07 01:53:53 UTC
Created attachment 834 [details]
Demo Application using Web Reference and WebClient

Hi Alsushi/Mono Team,

I have attached this Demo application which demonstarates connecting to web service by:
1) Using Web Reference
2) Using Web Client 

In both cases, the download of file streams fails after some 50 or 100 request.

Please look into this issue and suggest some fix.
Comment 20 James Clancey 2011-11-08 18:33:44 UTC
Hello Nirban,

Is it possible for you to send us a copy of the webconfig for the WCF project?  Also could you please attach a code sample where you configure the endpoint?  I have a feeling there is an issue with the settings which is causing it to time out.  If you have a small test case that would work best.

Thanks,

James Clancey
Comment 21 Nirban Dutta 2011-11-08 23:48:58 UTC
Hi James,

It is purely a client side issue, based on the fact that the same web service is comsumed by our desktop and WinPhone7 live applications(without any failure) and the sequence of downloading 100s of files are the same.

And also, if the server times out, we should be able to catch that exception in client side.

I have provided 2 demo application, you can do your testing on those and analyse why the client(android app) is crashing (purely a memory management issue in android application).

We have also set a timeout of 20 mins in the 1st demo project and a single request takes fraction of a minute.

Regards,

Nirban Dutta
Comment 22 Nick Randolph 2011-11-09 19:33:22 UTC
James,

The second code samples provided by Nirban clearly illustrate the issue we're experiencing both using WebClient and an ASMX web reference to call the service. There should be no reason that this should fail without a trapable exception after a certain number of calls. 

This is causing this project some severe issues as it starts into a testing phase and will potentially cause a delay in shipping this product. Whilst we would like to use WCF we understand that this is not a supported feature. However, I would assume that the use of WebClient is supported and hence we request that a solution to this issue be provided as quickly as reasonably possible.

Thanks in advance.

Nick
Comment 23 Jonathan Pryor 2011-12-01 14:12:23 UTC
I'm testing the attachments on what will be the next release.

Attachment 798 [details] (MonoTestProject.zip) runs for me w/o error, even after repeatedly smashing the "Get Files" button.

Likewise with Attachment 818 [details] (MonoAndroidApplication1.zip), if I repeatedly mash on the button, the app runs as expected.

Thus, I believe that this will be fixed in the next release.