Bug 24611 - We get System.Net.WebException when using the FtpWebRequest with Chinese character in the URL
Summary: We get System.Net.WebException when using the FtpWebRequest with Chinese char...
Status: VERIFIED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: BCL Class Libraries ()
Version: 4.18.1
Hardware: PC Windows
: Highest normal
Target Milestone: 5.1
Assignee: Marek Habersack
URL:
Depends on:
Blocks:
 
Reported: 2014-11-18 09:19 UTC by Prashant Cholachagudda
Modified: 2015-04-21 10:15 UTC (History)
5 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
VERIFIED FIXED

Description Prashant Cholachagudda 2014-11-18 09:19:55 UTC
When we us the FtpWebRequest with Chinese characters in the url Xamarin.Android throws the System.Net.WebException, while same code on work on the console application


Stacktrace:
11-18 19:41:46.034 E/mono    ( 2862): System.Net.WebException: Server returned an error: 550-The filename, directory name, or volume label syntax is incorrect. 
11-18 19:41:46.034 E/mono    ( 2862):  Win32 error:   The filename, directory name, or volume label syntax is incorrect. 
11-18 19:41:46.034 E/mono    ( 2862):  Error details: File system returned an error.
11-18 19:41:46.034 E/mono    ( 2862): 550 End
11-18 19:41:46.034 E/mono    ( 2862):   at System.Net.FtpWebRequest.OpenDataConnection () [0x00000] in <filename unknown>:0 
11-18 19:41:46.034 E/mono    ( 2862):   at System.Net.FtpWebRequest.DownloadData () [0x00000] in <filename unknown>:0 
11-18 19:41:46.034 E/mono    ( 2862):   at System.Net.FtpWebRequest.ProcessMethod () [0x00000] in <filename unknown>:0 
11-18 19:41:46.034 E/mono    ( 2862):   at System.Net.FtpWebRequest.ProcessRequest () [0x00000] in <filename unknown>:0 

Sample code:

    var request = (FtpWebRequest)WebRequest.Create(new Uri("ftp://localhost/漢語.txt"));
    request.Method = WebRequestMethods.Ftp.DownloadFile;

    var response = (FtpWebResponse)request.GetResponse();
    Console.WriteLine(response);
Comment 2 Sadik Ali 2014-11-25 01:45:17 UTC
I have checked this issue and able to reproduce it with the help of bug description and getting same as mentioned in bug.

Screencast: http://www.screencast.com/t/SRtfUs9ZtV

Ide log: https://gist.github.com/Shruti360/465afab075dfd7162de4

Build output: https://gist.github.com/Shruti360/6c50f2cc39780b24a2d0

Environment Info:
Microsoft Visual Studio Professional 2013
Version 12.0.30723.00 Update 3
Microsoft .NET Framework
Version 4.5.53344

Installed Version: Professional

Xamarin   3.8.150.0 (10cfd178d55287f09c85f5a1e604dfe20889a40f)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin.Android   4.20.0.28 (ba9bbbdd44cfdc4bf485e8885bd2ad24fba525f7)
Visual Studio plugin to enable development for Xamarin.Android.

Xamarin.iOS   8.4.0.0 (840a925103a0bf4a856507f13d5eaee3c1579c2f)
Visual Studio extension to enable development for Xamarin.iOS.
Comment 4 Jonathan Pryor 2014-12-15 11:23:25 UTC
> same code on work on the console application

Which FTP server are you testing against?

This may be FTP server specific, as a console application DOES NOT retrieve a Chinese filename from OS X Server.app ftpd FTP daemon:

> System.Net.WebException: Server returned an error: 550 ??.txt: No such file or directory.
>   at System.Net.FtpWebRequest.OpenDataConnection () [0x00000] in <filename unknown>:0 
>   at System.Net.FtpWebRequest.DownloadData () [0x00000] in <filename unknown>:0 
>   at System.Net.FtpWebRequest.ProcessMethod () [0x00000] in <filename unknown>:0 
>   at System.Net.FtpWebRequest.ProcessRequest () [0x00000] in <filename unknown>:0 

This despite the fact that if I list the FTP directory contents, I *do* see the Chinese filename:

	var request = (FtpWebRequest)WebRequest.Create(
			new Uri("ftp://SERVER/Default/"));
	request.Method = WebRequestMethods.Ftp.ListDirectory;

	var response = (FtpWebResponse) request.GetResponse();
	Console.WriteLine(response);
	string r = null;
	using (var s = new StreamReader (response.GetResponseStream ())) {
		r = s.ReadToEnd ();
		Console.WriteLine (r);
	}

which prints:

> default.html.de
> default.html.en
> default.html.es
> default.html.fr
> default.html.it
> default.html.ja
> default.html.ko
> default.html.nl
> default.html.zh-CN
> default.html.zh-TW
> favicon.ico
> hello.txt
> info.php
> NetBoot
> Server.png
> 漢語.txt

Perhaps some kind of encoding issue?
Comment 5 Jonathan Pryor 2014-12-15 11:25:25 UTC
> same code on work on the console application

I suspect the "console application" referenced is running under .NET, *not* Mono. It would be helpful/appreciated if someone could test a console app *running under Mono* against this FTP server.
Comment 6 Jonathan Pryor 2014-12-15 12:11:09 UTC
Meanwhile, I'm not sure WTF is going on. This sounds like an encoding issue, but I'm not sure it's an encoding issue.

Firstly, when we list the directory what *bytes* do we get?

	var request = (FtpWebRequest) WebRequest.Create(
			new Uri("ftp://Ritsuko.local/Default/"));
	request.Method = WebRequestMethods.Ftp.ListDirectory;
	
	var response = (FtpWebResponse) request.GetResponse();
	Console.WriteLine(response);
	string r = null;
	using (var s = response.GetResponseStream ()) {
		var buf = new byte [16];
		int n;
		while ((n = s.Read (buf, 0, buf.Length)) > 0) {
			Console.Write (string.Join (" ",buf.Take (n)
					.Select (b => b.ToString ("x2"))));
			Console.Write (" | ");
			Console.WriteLine (string.Join ("", buf.Take (n)
					.Select (b => char.IsLetterOrDigit ((char) b) ? (char) b : '.')));
		}
	}

> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 64 65 0a | default.html.de.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 65 6e 0a | default.html.en.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 65 73 0a | default.html.es.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 66 72 0a | default.html.fr.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 69 74 0a | default.html.it.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 6a 61 0a | default.html.ja.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 6b 6f 0a | default.html.ko.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 6e 6c 0a | default.html.nl.
> 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e 7a 68 2d | default.html.zh.
> 43 4e 0a 64 65 66 61 75 6c 74 2e 68 74 6d 6c 2e | CN.default.html.
> 7a 68 2d 54 57 0a 66 61 76 69 63 6f 6e 2e 69 63 | zh.TW.favicon.ic
> 6f 0a 68 65 6c 6c 6f 2e 74 78 74 0a 69 6e 66 6f | o.hello.txt.info
> 2e 70 68 70 0a 4e 65 74 42 6f 6f 74 0a 53 65 72 | .php.NetBoot.Ser
> 76 65 72 2e 70 6e 67 0a e6 bc a2 e8 aa 9e 2e 74 | ver.png.æ..èª..t
> 78 74 0a | xt.

What's interesting is that the ListDirectories response is a UTF-8: the byte sequence:

> e6 bc a2 e8 aa 9e 2e 74 78 74 | æ..èª..txt

is:

> csharp> var utf8b = new byte[]{0xe6, 0xbc, 0xa2, 0xe8, 0xaa, 0x9e, 0x2e, 0x74, 0x78, 0x74};
> csharp> System.Text.Encoding.UTF8.GetString(utf8b);
> "漢語.txt"

So it looks like, ftpd is returning UTF-8 encoded data.
Comment 7 Jonathan Pryor 2014-12-15 12:19:43 UTC
One quick grep later, and we find:

https://github.com/mono/mono/blob/48e112d6/mcs/class/System/System.Net/FtpWebRequest.cs#L1083
	cmd = Encoding.ASCII.GetBytes (commandString);

Apparently Mono's command is sent as ASCII, not UTF-8!

Changing it locally to use Encoding.UTF8 allows me to grab the file.
Comment 8 Jonathan Pryor 2014-12-15 12:21:54 UTC
@grendel: As per Comment #7, FtpWebRequest is broken. The fix appears to be a one-line change (yay?), *but* we also need unit tests for this.
Comment 9 Marek Habersack 2014-12-15 14:45:40 UTC
@jonp Alas, it's not as easy as using the UTF-8 encoding. It depends on what encoding is used/expected by the daemon. Investigating this.
Comment 10 Marek Habersack 2014-12-15 17:39:55 UTC
Fixed in mono/master, commit c2dce69079a070c4b9ed6344821498ee84dcdf49

Tests will be committed at some later date.
Comment 11 Ram Chandra 2015-04-21 10:15:28 UTC
I have checked this issue and I am able to reproduce this issue on following builds:

XVS:
Microsoft Visual Studio Professional 2013
Version 12.0.30723.00 Update 3
Microsoft .NET Framework
Version 4.5.51641
Installed Version: Professional
Xamarin   3.7.248.0 (8ca7d11db8a6f874c6cd2de6d9ca0f511867ce91)
Xamarin.Android   4.18.1.3 (5474129af31e9d3a86cb7482c7c5c7a30ad315f1)
Xamarin.iOS   8.4.0.0 (209abebbd8f1a292d042420edb45fa5fbd3f017b)

XS: 5.5.2 (build 8)

Screencast: http://www.screencast.com/t/Wv94CkeB6

To verify this issue I have checked this issue with following build on both O SX and windows environment:

XVS:
Microsoft Visual Studio Professional 2013
Version 12.0.30723.00 Update 3
Microsoft .NET Framework
Version 4.5.51641
Installed Version: Professional
Xamarin   3.11.446.0 (8a39e05)
Xamarin.Android   5.1.0.115 (d23da369e436488f38c8ab8fe8a9ae7d9ea5256b)
Xamarin.iOS   8.10.0.0 (7741cc495ab0baf04ff0405d0604bc27f0ecae2e)

XS: 5.9 (build 431)

Screencast: http://www.screencast.com/t/TAhfHP2xqn8Z

Observation: I am not getting any error or exception when I try to use the FtpWebRequest with Chinese characters in the url Xamarin.Android.

This issue has been fixed. Hence, I am closing this issue.