Bug 17128 - Seek on Filestream passed 2GB raises exception
Summary: Seek on Filestream passed 2GB raises exception
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: BCL Class Libraries ()
Version: 4.10.1
Hardware: PC Windows
: Lowest normal
Target Milestone: ---
Assignee: Zoltan Varga
URL:
Depends on:
Blocks:
 
Reported: 2014-01-08 14:58 UTC by juanjo.a
Modified: 2014-01-10 16:46 UTC (History)
2 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description juanjo.a 2014-01-08 14:58:20 UTC
Doing a myFilestream.Seek(position, SeekOrigin.Begin) with a position larger than 2 GB raises the exception "System.IO.IOException: Win32 IO returned 25"

Also opening a filestream for append crashes with the same exception if the file is longer than 2 GB

It happens on both, emulator and real devices. This bug makes files larger than 2 GB ununsable with Xamarin unless they are readed or writted sequentially.

Test case 1:
        byte[] buffer = new byte[1024 * 1024]; // 1MB

        FileStream fstream = new FileStream("testFile.bin", FileMode.Create);

        for (int i = 0; i < 2500; i++)
        {
            fstream.Write(buffer, 0, buffer.Length);
            Console.WriteLine((i + 1).ToString() + " MB");
        }

        fstream.Close();

        FileStream fstream2 = new FileStream("testFile.bin", FileMode.Open);
        fstream2.Seek(0, SeekOrigin.End); // System.IO.IOException: Win32 IO returned 25 raised here

        for (int i = 2500; i < 3000; i++)
        {
            fstream2.Write(buffer, 0, buffer.Length);
            Console.WriteLine((i + 1).ToString() + " MB");
        }

        fstream2.Close();

Test case 2:

        byte[] buffer = new byte[1024 * 1024]; // 1MB

        FileStream fstream = new FileStream("testFile.bin", FileMode.Create);

        for (int i = 0; i < 2500; i++)
        {
            fstream.Write(buffer, 0, buffer.Length);
            Console.WriteLine((i + 1).ToString() + " MB");
        }

        fstream.Close();

        FileStream fstream2 = new FileStream("testFile.bin", FileMode.Append); //this line raises exception: System.IO.IOException: Win32 IO returned 25

        for (int i = 2500; i < 3000; i++)
        {
            fstream2.Write(buffer, 0, buffer.Length);
            Console.WriteLine((i + 1).ToString() + " MB");
        }

        fstream2.Close();
Comment 1 Jonathan Pryor 2014-01-08 21:33:56 UTC
The problem is that Mono's I/O layer does two things:

 1. Mono uses lseek(2) for file seeking.
 2. Mono uses a compile-time feature check to enable large file support, a'la
    http://users.suse.com/~aj/linux_lfs.html
> Compile your programs with "gcc -D_FILE_OFFSET_BITS=64".
> This forces all file access calls to use the 64 bit variants.
> Several types change also, e.g. off_t becomes off64_t.

(I believe it's also possible to explicitly use the 64-bit API variants, e.g. directly use lseek64(2) instead of lseek(2) + _FILE_OFFSET_BITS=64. Mono does not do so.)

The "problem" is that Mono's configure check (#2, above) fails on Android, because the Android NDK doesn't provide this functionality.

I've filed an NDK bug requesting this support:

  https://code.google.com/p/android/issues/detail?id=64613
Comment 2 juanjo.a 2014-01-09 05:06:50 UTC
In my opinion the bug is in the Mono's I/O layer for Android that is not fully compatible with the NDK. The NDK can be used with large files without any problems. The goal should be to make Mono compatible with Android and not Android compatible with Mono. You are requesting a change in the NDK funcionality to make it compatible with Mono, that has no sense to me.

Also, even if Android changes the NDK for future versions, all the current versions of Android would still fail, and that would make that the use of large files on Xamarin-Android unusable practically forever.
Comment 3 Jonathan Pryor 2014-01-09 10:47:04 UTC
I'm not asking the NDK to be compatible with Mono. I'm asking for the NDK to follow the established glibc standard which LOTS of FOSS projects follow, and the NDK not following the pattern prevents ALL of them from easily getting large file support.

For example, ffmpeg (mentioned in the Android bug) does the same thing with large file support, and consequently lacks large file support on Android.

See also: http://www.unix.org/version2/whatsnew/lfs20mar.html#3.3
Comment 4 Jonathan Pryor 2014-01-10 16:46:32 UTC
mono/427d2420 and monodroid/67ce82db enable support for explicitly using lseek64() for file seeking.

I also quickly glanced at other uses of off_t in io-layer, and file truncation came up:

https://github.com/mono/mono/blob/master/mono/io-layer/io.c#L619

Unfortunately, Android does NOT provide truncate64() or ftruncate64() functions, meaning file truncation for file sizes > 2GB is not possible.