Bug 3100 - Error With System.Net.WebClient.UploadProgressChanged Event
Summary: Error With System.Net.WebClient.UploadProgressChanged Event
Status: RESOLVED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: Xamarin.iOS.dll ()
Version: 5.2
Hardware: Macintosh Other
: --- normal
Target Milestone: Untriaged
Assignee: Martin Baulig
URL:
Depends on:
Blocks:
 
Reported: 2012-01-26 18:50 UTC by Anthony Greco
Modified: 2016-02-08 17:40 UTC (History)
8 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Anthony Greco 2012-01-26 18:50:54 UTC
When using assigning UploadProgressChanged onto a object of System.Net.WebClient, the event is never fired.

To duplicate this, run:
client = new System.Net.WebClient();
client.UploadProgressChanged += UploadProgressChanged;
client.UploadFileCompleted += HandleClientUploadFileCompleted;
client.UploadFileAsync(new Uri("url to download"), writeToFolder + "package.zip");

You will notice that HandleClientUploadFileComplete fires when complete, yet UploadProgressChanged never does.
Comment 3 Gonzalo Paniagua Javier 2012-03-14 09:49:27 UTC
I will take a look.
Comment 4 Anthony Greco 2012-03-21 15:07:17 UTC
any chance of making any progress on this?  I have an app that has to upload a 20mb db file used by about 50 people and it's really causing issues not being able to show progress.  I'd prefer to not have to re-invent the week and do this with pure HTTP protocol just for the progress.

I appreciate you looking into this
Comment 5 Gonzalo Paniagua Javier 2012-03-21 16:08:43 UTC
Give me a few hours and it will be fixed (after that, it will take about 1 week until the fix shows up in the beta channel. A few days later it will be in stable).
Comment 6 Anthony Greco 2012-03-21 22:26:51 UTC
Thank you so much.  I will be sure to turn my settings to beta and check daily for that.
Comment 7 Gonzalo Paniagua Javier 2012-03-22 00:28:38 UTC
Fixed in master/30dcaff and mono-2-10/6b3c555.
Comment 8 Anthony Greco 2012-03-22 10:57:19 UTC
Thank you so much, I will check it out later today. I appreciate you doing this so quickly!
Comment 9 Anthony Greco 2012-04-18 02:20:48 UTC
The fix finally hit beta however it still has a major bug. While the event is now firing, it appears to be doing it as it ads to the outgoing queue, not actually as it sends out.  An simple test is to attach an UploadProgressChanged to any large upload and run the following inside the method:

string a = System.DateTime.Now.ToString ("hh:MM:ss:FFFFF") + ": " + e.BytesSent.ToString() + " of " + e.TotalBytesToSend.ToString() + " : " + (e.TotalBytesToSend-e.BytesSent).ToString();
Console.WriteLine (a);

You will notice that on a very large file, all of the bytes get reported one after another extremely quickly, in less than a second. After it hits 100%, it stops reporting even though it may not even have sent 1%.  Because of this it definitely appears to be reporting when bytes get read from file, not as they are sent to the clien (and thus reporting what progress is left).

Below is a copy of the above showing all of my file was reported in under a second, though it takes over 60 seconds to send.  I can also verify this by watching wireshark send the data after this runs until the HandleClientUploadFileCompleted is fired.


This was tested in the latest beta release of Mono/MonoTouch:
Mono 2.10.9
Monotouch 5.2.11
----------------

01:04:46:02057: 0 of 4062727 : 4062727
01:04:46:02112: 40960 of 4062727 : 4021767
01:04:46:02131: 81920 of 4062727 : 3980807
01:04:46:02147: 122880 of 4062727 : 3939847
01:04:46:02174: 163840 of 4062727 : 3898887
01:04:46:02186: 204800 of 4062727 : 3857927
01:04:46:02198: 245760 of 4062727 : 3816967
01:04:46:02258: 286720 of 4062727 : 3776007
01:04:46:02271: 327680 of 4062727 : 3735047
01:04:46:02283: 368640 of 4062727 : 3694087
01:04:46:02294: 409600 of 4062727 : 3653127
01:04:46:02307: 450560 of 4062727 : 3612167
01:04:46:02319: 491520 of 4062727 : 3571207
01:04:46:02329: 532480 of 4062727 : 3530247
01:04:46:02388: 573440 of 4062727 : 3489287
01:04:46:02409: 614400 of 4062727 : 3448327
01:04:46:02424: 655360 of 4062727 : 3407367
01:04:46:02439: 696320 of 4062727 : 3366407
01:04:46:02455: 737280 of 4062727 : 3325447
01:04:46:02469: 778240 of 4062727 : 3284487
01:04:46:02483: 819200 of 4062727 : 3243527
01:04:46:02497: 860160 of 4062727 : 3202567
01:04:46:02511: 901120 of 4062727 : 3161607
01:04:46:02525: 942080 of 4062727 : 3120647
01:04:46:02543: 983040 of 4062727 : 3079687
01:04:46:02557: 1024000 of 4062727 : 3038727
01:04:46:02571: 1064960 of 4062727 : 2997767
01:04:46:02664: 1105920 of 4062727 : 2956807
01:04:46:02686: 1146880 of 4062727 : 2915847
01:04:46:027: 1187840 of 4062727 : 2874887
01:04:46:02731: 1228800 of 4062727 : 2833927
01:04:46:02745: 1269760 of 4062727 : 2792967
01:04:46:02759: 1310720 of 4062727 : 2752007
01:04:46:02772: 1351680 of 4062727 : 2711047
01:04:46:02785: 1392640 of 4062727 : 2670087
01:04:46:02798: 1433600 of 4062727 : 2629127
01:04:46:02812: 1474560 of 4062727 : 2588167
01:04:46:02826: 1515520 of 4062727 : 2547207
01:04:46:02839: 1556480 of 4062727 : 2506247
01:04:46:02853: 1597440 of 4062727 : 2465287
01:04:46:0287: 1638400 of 4062727 : 2424327
01:04:46:02887: 1679360 of 4062727 : 2383367
01:04:46:02905: 1720320 of 4062727 : 2342407
01:04:46:02928: 1761280 of 4062727 : 2301447
01:04:46:02948: 1802240 of 4062727 : 2260487
01:04:46:02968: 1843200 of 4062727 : 2219527
01:04:46:02989: 1884160 of 4062727 : 2178567
01:04:46:03008: 1925120 of 4062727 : 2137607
01:04:46:03028: 1966080 of 4062727 : 2096647
01:04:46:03047: 2007040 of 4062727 : 2055687
01:04:46:03069: 2048000 of 4062727 : 2014727
01:04:46:03092: 2088960 of 4062727 : 1973767
01:04:46:0312: 2129920 of 4062727 : 1932807
01:04:46:03278: 2170880 of 4062727 : 1891847
01:04:46:03306: 2211840 of 4062727 : 1850887
01:04:46:03325: 2252800 of 4062727 : 1809927
01:04:46:03344: 2293760 of 4062727 : 1768967
01:04:46:03362: 2334720 of 4062727 : 1728007
01:04:46:03383: 2375680 of 4062727 : 1687047
01:04:46:03401: 2416640 of 4062727 : 1646087
01:04:46:03419: 2457600 of 4062727 : 1605127
01:04:46:03436: 2498560 of 4062727 : 1564167
01:04:46:03457: 2539520 of 4062727 : 1523207
01:04:46:03474: 2580480 of 4062727 : 1482247
01:04:46:03492: 2621440 of 4062727 : 1441287
01:04:46:03511: 2662400 of 4062727 : 1400327
01:04:46:03528: 2703360 of 4062727 : 1359367
01:04:46:03546: 2744320 of 4062727 : 1318407
01:04:46:03564: 2785280 of 4062727 : 1277447
01:04:46:03582: 2826240 of 4062727 : 1236487
01:04:46:03599: 2867200 of 4062727 : 1195527
01:04:46:03617: 2908160 of 4062727 : 1154567
01:04:46:03634: 2949120 of 4062727 : 1113607
01:04:46:03652: 2990080 of 4062727 : 1072647
01:04:46:03669: 3031040 of 4062727 : 1031687
01:04:46:03687: 3072000 of 4062727 : 990727
01:04:46:03705: 3112960 of 4062727 : 949767
01:04:46:03723: 3153920 of 4062727 : 908807
01:04:46:0374: 3194880 of 4062727 : 867847
01:04:46:03758: 3235840 of 4062727 : 826887
01:04:46:03775: 3276800 of 4062727 : 785927
01:04:46:03794: 3317760 of 4062727 : 744967
01:04:46:03812: 3358720 of 4062727 : 704007
01:04:46:0383: 3399680 of 4062727 : 663047
01:04:46:03851: 3440640 of 4062727 : 622087
01:04:46:03868: 3481600 of 4062727 : 581127
01:04:46:03886: 3522560 of 4062727 : 540167
01:04:46:03909: 3563520 of 4062727 : 499207
01:04:46:03931: 3604480 of 4062727 : 458247
01:04:46:03951: 3645440 of 4062727 : 417287
01:04:46:0397: 3686400 of 4062727 : 376327
01:04:46:03989: 3727360 of 4062727 : 335367
01:04:46:04009: 3768320 of 4062727 : 294407
01:04:46:04033: 3809280 of 4062727 : 253447
01:04:46:04053: 3850240 of 4062727 : 212487
01:04:46:04073: 3891200 of 4062727 : 171527
01:04:46:04093: 3932160 of 4062727 : 130567
01:04:46:04112: 3973120 of 4062727 : 89607
01:04:46:04132: 4014080 of 4062727 : 48647
01:04:46:04151: 4055040 of 4062727 : 7687
01:04:46:0416: 4062727 of 4062727 : 0
Comment 10 Gonzalo Paniagua Javier 2012-04-19 16:47:19 UTC
I can hook this up to the point where we write to the socket. But it is very likely that you see a similar behavior because the kernel itself will also buffer the data. For anything that fits in a kernel buffer, the time taken will be similar and you would still get the callbacks called very fast.

Try creating a class that derives from WebClient and override the GetWebRequest() method so that it returns a HttpWebRequest with AllowWriteStreamBuffering = false and see if there is any difference.
Comment 11 Keith Maurino 2012-10-25 09:14:56 UTC
Just trying to find out if there is any further progress on this bug? Looks like it was attempted to be fixed a number of months ago, but nothing since then. Any information would be greatly appreciated.
Comment 12 Rodja Trappe 2013-03-26 23:03:42 UTC
The bug is still present in Xamarin.Android Version: 4.6.0 (Business Edition). The suggested workaround from http://stackoverflow.com/questions/12119766/uploading-large-files-with-monodroid is not helping much, because the content length computation may be difficult.
Comment 13 Martin Baulig 2013-06-26 06:29:27 UTC
See #7 and #10.
Comment 14 Alek Slater 2014-02-22 04:57:44 UTC
The bug is still present in MonoTouch 7.0.6
Comment 15 Martin Baulig 2014-02-24 08:54:31 UTC
Quick look at the code confirms that we're actually firing the event.  So Gonzalo actually fixed that in the commit that he mentioned.

And we're obviously not doing anything about the "bug" described in comment 9.  Closing as FEATURE / WONTFIX.
Comment 16 Alek Slater 2014-02-24 09:08:23 UTC
I'm still experiencing this issue, so I completely disagree with the fact that this is fixed, but I'm using client.UploadValuesAsync instead of client.UploadFileAsync

client.UploadValuesCompleted+=completionHandler;
client.UploadProgressChanged+=progressHandler;
client.UploadValuesAsync(url, "POST", values);

Should I raise this as a separate issue? I mean, this bug title does describe the issue as : Error With System.Net.WebClient.UploadProgressChanged Event which technically is still the issue I'm experiencing.

In other words, the UploadValuesCompleted fires without any issues,
UploadProgressChanged is never triggered.
Comment 17 Martin Baulig 2014-02-24 09:34:40 UTC
Oops, you're right about UploadValuesAsync() - and the same thing applies to UploadDataAsync().  I only checked UploadFileAsync(), which was mentioned earlier in this bug report.

Both UploadValuesAsync() and UploadDataAsync() send the entire data in one single chunk.  UploadFileAsync() reads the input file in 4096 byte chunks and calls UploadProgressChanged each time.

We could split the data into smaller chunks, but all you'd get from that is a notification that the data was passed on to the stream.  Not sure whether that'd be very useful at all, probably depends on the size of the kernel's socket write queue and the average data size that's being sent.
Comment 18 Alek Slater 2014-02-24 09:41:42 UTC
I don't mind a workaround if anything works, but I've tried quite a few different things without any luck.
I've had to fake the progress for now, but I'm sure the client will have issues with it.
Comment 19 Anthony Greco 2014-02-24 12:17:37 UTC
I considered this basically a "is never going to be fixed" situation, so it's at least nice to see people interested in it again. Thank god my client accepted the "It's going to be fixed in a few weeks" then never brought it up again, instead of having to fake it.
Comment 20 Martin Baulig 2014-02-24 12:42:06 UTC
My apologies for the misunderstanding.  I should have checked the other functions first, not just one of them - even when the bug report only mentioned one.
Comment 21 Alek Slater 2014-02-24 13:20:23 UTC
No worries, strictly speaking perhaps I should have opened another bug report, just thought it would make sense to keep them together.
Comment 22 Anthony Greco 2014-02-24 14:07:47 UTC
Martin: So should we assume there will not be any progress in the near future on getting a true % sent, as apposed to just passed to the buffer?
Comment 23 Martin Baulig 2014-02-24 15:10:00 UTC
A "true % sent" will never happen, there is just no way of getting this information.  You could disable write stream buffering - but we can't do that by default until we've figured out why this affects performance so terribly; I saw over 50% performance losses on the tests that I've run so far.

With write stream buffering enabled, the stream always seems to take at least the first ~25mb before it starts blocking.  It varies a bit based on which buffer sizes I use on the server side to read things.

I varied with different write/read chunk sizes and random delay on the server side and came to the conclusion that it won't help unless you're sending so much data that using these APIs may not be the best choice anyways; ie. I wouldn't want to allocate some huge >25mb byte array then send it in a single chunk on a mobile app.

WebClient is just a thin wrapper around (Http)WebRequest anyways.  This is basically the entire send loop, with the changes that I tried:

				using (Stream stream = request.GetRequestStream ()) {
					int offset = 0;
					while (offset < contentLength) {
						var size = Math.Min (contentLength - offset, socketBufferSize);
						stream.Write (data, offset, size);

						offset += size;
						int percent = 0;
						if (contentLength > 0)
							percent = (int) ((long)offset * 100 / contentLength);
						var args = new UploadProgressChangedEventArgs (0, 0, offset, contentLength, percent, userToken);
						OnUploadProgressChanged (args);
					}
				}
Comment 24 Anthony Greco 2014-02-24 15:34:07 UTC
Gotcha. I do appreciate looking into this for us though! 

Thanks for digging deeper into the issue.
Comment 25 Rolf Bjarne Kvinge [MSFT] 2016-02-08 17:40:15 UTC
We're using the reference sources for Microsoft for WebClient now, so this should be fixed.