Bug 223 - cant unzip a valid zip using System.IO.Packaging.Package
Summary: cant unzip a valid zip using System.IO.Packaging.Package
Status: RESOLVED FIXED
Alias: None
Product: Class Libraries
Classification: Mono
Component: WindowsBase ()
Version: 2.10.x
Hardware: PC Windows
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2011-08-08 06:49 UTC by Simon Cropp
Modified: 2011-08-18 07:31 UTC (History)
6 users (show)

Tags: windowsbase.dll package zip
Is this bug a regression?: ---
Last known good build:


Attachments
a VS solution that builds the code. (382.68 KB, application/zip)
2011-08-08 06:49 UTC, Simon Cropp
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on GitHub or Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Simon Cropp 2011-08-08 06:49:29 UTC
Created attachment 84 [details]
a VS solution that builds the code.

Component should be "WindowsBase" but I could not find that.
Using mono 2.10.3

Essentially my code is this

    var package = Package.Open("Package.zip");
            foreach (var part in package.GetParts())
            {
                var filePath = targetDir + part.Uri.OriginalString;
                Directory.CreateDirectory(Path.GetDirectoryName(filePath));
                using (var stream = part.GetStream())
                {
                    using (var output = File.OpenWrite(filePath))
                    {
                        stream.CopyTo(output);
                    }
                }
            }

It seems to die on package.GetParts.
This code works in standard .net

See attached solution.
So when I run "mono MonoPackageBug.exe" I get "mono.exe has stopped working".
Comment 1 Alan McGovern 2011-08-12 13:48:46 UTC
The testcase works flawlessly when run on MacOS with mono 2.10.3. Can you confirm that this testcase is the correct one to demonstrate the issue? Also, are you running on windows? If so, which version? It's possible you've hit a platform specific issue that doesn't manifest on MacOS.
Comment 2 Alex Corrado [MSFT] 2011-08-12 15:45:22 UTC
The testcase crashes under Windows 7 with Mono 2.10.3. Here's a stacktrace from WinDbg:

Microsoft (R) Windows Debugger Version 6.11.0001.404 X86
Copyright (c) Microsoft Corporation. All rights reserved.

*** wait with pending attach
Symbol search path is: *** Invalid ***
****************************************************************************
* Symbol loading may be unreliable without a symbol search path.           *
* Use .symfix to have the debugger choose a symbol path.                   *
* After setting your symbol path, use .reload to refresh symbol locations. *
****************************************************************************
Executable search path is: 
ModLoad: 00400000 0041b000   C:\Program Files\Mono-2.10.3\bin\mono.exe
ModLoad: 76ed0000 7700c000   C:\Windows\SYSTEM32\ntdll.dll
ModLoad: 75820000 758f4000   C:\Windows\system32\kernel32.dll
ModLoad: 750d0000 7511a000   C:\Windows\system32\KERNELBASE.dll
ModLoad: 659c0000 65c63000   C:\Program Files\Mono-2.10.3\bin\mono-2.0.dll
ModLoad: 75ab0000 75b50000   C:\Windows\system32\ADVAPI32.DLL
ModLoad: 76ac0000 76b6c000   C:\Windows\system32\msvcrt.dll
ModLoad: 77030000 77049000   C:\Windows\SYSTEM32\sechost.dll
ModLoad: 75630000 756d1000   C:\Windows\system32\RPCRT4.dll
ModLoad: 74a60000 74a9c000   C:\Windows\system32\MSWSOCK.DLL
ModLoad: 754f0000 755b9000   C:\Windows\system32\user32.dll
ModLoad: 77050000 7709e000   C:\Windows\system32\GDI32.dll
ModLoad: 754e0000 754ea000   C:\Windows\system32\LPK.dll
ModLoad: 75320000 753bd000   C:\Windows\system32\USP10.dll
ModLoad: 756e0000 75715000   C:\Windows\system32\WS2_32.dll
ModLoad: 75aa0000 75aa6000   C:\Windows\system32\NSI.dll
ModLoad: 76960000 76abc000   C:\Windows\system32\OLE32.dll
ModLoad: 76c40000 76ccf000   C:\Windows\system32\OLEAUT32.DLL
ModLoad: 75450000 75455000   C:\Windows\system32\PSAPI.DLL
ModLoad: 75ce0000 7692a000   C:\Windows\system32\SHELL32.DLL
ModLoad: 755c0000 75617000   C:\Windows\system32\SHLWAPI.dll
ModLoad: 74520000 74529000   C:\Windows\system32\VERSION.dll
ModLoad: 734d0000 73502000   C:\Windows\system32\WINMM.DLL
ModLoad: 77010000 7702f000   C:\Windows\system32\IMM32.DLL
ModLoad: 76b70000 76c3c000   C:\Windows\system32\MSCTF.dll
ModLoad: 74f70000 74f7c000   C:\Windows\system32\CRYPTBASE.dll
(89c.dc0): Break instruction exception - code 80000003 (first chance)
eax=7ffdc000 ebx=00000000 ecx=00000000 edx=76f6f125 esi=00000000 edi=00000000
eip=76f040f0 esp=0252ff5c ebp=0252ff88 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\SYSTEM32\ntdll.dll - 
ntdll!DbgBreakPoint:
76f040f0 cc              int     3
0:002> g
ModLoad: 68a40000 68be7000   C:\Program Files\Mono-2.10.3\bin\MonoPosixHelper.DLL
ModLoad: 5a4c0000 5a4d3000   C:\Program Files\Mono-2.10.3\bin\zlib1.dll
(89c.b48): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000000 ebx=0211d238 ecx=76ac98da edx=00303128 esi=000003ee edi=00000001
eip=00000000 esp=0022f7f0 ebp=0022f8c8 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00210246
00000000 ??              ???
0:000> |* ~* kp

.  0  Id: 89c.b48 Suspend: 1 Teb: 7ffde000 Unfrozen
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Mono-2.10.3\bin\MonoPosixHelper.DLL - 
ChildEBP RetAddr  
WARNING: Frame IP not in any known module. Following frames may be wrong.
0022f7ec 68a464da 0x0
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Mono-2.10.3\bin\mono-2.0.dll - 
0022f918 65b39f43 MonoPosixHelper!unzOpen2+0xfa
0022f9a8 01b81498 mono_2_0!mono_thread_internal_current+0x13
0022f9c8 01b80f98 0x1b81498
0022fa18 01b80f10 0x1b80f98
0022fa38 01b80b3e 0x1b80f10
0022fb78 65b5f36c 0x1b80b3e
0022fb88 65b5f438 mono_2_0!mono_error_raise_exception+0xe5c
0022fbb8 659cb1dc mono_2_0!mono_error_raise_exception+0xf28
0022fbf8 65b0f9a2 mono_2_0!mono_jit_compile_method+0x10c
0022fc18 65b13702 mono_2_0!mono_runtime_invoke+0x42
0022fc48 65a2aa02 mono_2_0!mono_runtime_exec_main+0xd2
*** ERROR: Module load completed but symbols could not be loaded for C:\Program Files\Mono-2.10.3\bin\mono.exe
0022fed8 00401474 mono_2_0!mono_main+0x1662
0022ff28 004010bb mono+0x1474
0022ff68 004012a8 mono+0x10bb
0022ff88 75873c45 mono+0x12a8
0022ff94 76f337f5 kernel32!BaseThreadInitThunk+0x12
0022ffd4 76f337c8 ntdll!RtlInitializeExceptionChain+0xef
0022ffec 00000000 ntdll!RtlInitializeExceptionChain+0xc2

   1  Id: 89c.9e4 Suspend: 1 Teb: 7ffdd000 Unfrozen
ChildEBP RetAddr  
WARNING: Stack unwind information not available. Following frames may be wrong.
01e8fea0 7586baf3 ntdll!KiFastSystemCallRet
01e8feb8 65b5cd32 kernel32!WaitForSingleObjectEx+0x43
01e8fed8 65b5cd91 mono_2_0!mono_sem_timedwait+0x32
01e8fef8 65abfa30 mono_2_0!mono_sem_wait+0x21
01e8ff28 65b3de8a mono_2_0!mono_gc_cleanup+0x1b0
01e8ff68 65b79812 mono_2_0!mono_thread_set_execution_context+0x14a
01e8ff88 75873c45 mono_2_0!GC_push_all_stack+0x6678
01e8ff94 76f337f5 kernel32!BaseThreadInitThunk+0x12
01e8ffd4 76f337c8 ntdll!RtlInitializeExceptionChain+0xef
01e8ffec 00000000 ntdll!RtlInitializeExceptionChain+0xc2
Comment 3 Simon Cropp 2011-08-14 18:35:05 UTC
Just an FYI as to why I want IO.Packaging working:
I am trying to get nuget working on mono.
Comment 4 Alan McGovern 2011-08-14 20:08:26 UTC
It's a simple problem. We just need to specify that mono uses the 'cdecl' calling convention on all platforms when p/invoking one of our native libraries. Once that's done everything will just work. When I get the fix tested and committed I can supply you with a recompiled assembly which you can use for testing purposes. You'd just have to overwrite the old one with the new one.
Comment 5 Simon Cropp 2011-08-14 20:37:14 UTC
Alan

Thanks so much for looking into this.

No rush but do you have a time-line. The reason I ask is depending on how long the fix will take I may shell out to an unzip process in the short term.
Comment 6 Alan McGovern 2011-08-16 08:53:21 UTC
I changed the P/Invoke declarations in the packaging code to enforce the Cdecl calling convention yet the issue still persists. Does anyone have any idea what else I might have to do?
Comment 7 Alan McGovern 2011-08-16 14:00:14 UTC
I have the issue boiled down to something which makes no sense :) I added logging to all C# stream functions which are passed to minizip to be invoked. Everything is invoked properly with apparently the correct arguments. The sequence of invocations is:

C# p/invokes unzOpen2 function:
minizip invokes the open_file delegate

minizip now calls another native function called unzlocal_SearchCentralDir and it does:

minizip invokes the 'seek' delegate to go to the end of the file, C# does so.
minizip invokes the 'tell' delegate to check the position, C# does so
minizip invokes the 'seek' delegate to go 1028 bytes from the end, C# does so.
minizio invokes the 'read' delegate to read 1028 bytes, C# blows up with a memory access exception when it tries to write the 1028th byte of data to minizips buffer.

This makes no sense. minizip allocates a buffer of hardcoded size 1028 and then calls 'read' and asks for 1028 bytes. When C# tries to write the 1028th it explodes. I have no idea how to diagnose this further :(

Any and all ideas welcome. The only way I think i can advance this further would be if i was able to set up a build environment for mono on windows and had a stepping debugger to see if somehow the pointer to the buffer is becoming invalid when passed from native to managed code.
Comment 8 Paolo Molaro 2011-08-17 05:49:27 UTC
Alan, it's not clear what change you did to enforce the cdecl call convention: since the callbacks are represented as delegates, you need to set a special attribute on the delegate, not on the P/Invoke declaration, like this:
  [UnmanagedFunctionPointerAttribute (CallingConvention.Cdecl)]
  delegate ...;

Did you do that?
ZlibFileFuncDef also has an InPtr opaque with a MarshalAs (UnmanagedType.FunctionPtr) attribute, which makes no sense (look like a cut&paste error).
The whole API binding is not correct for win64, even if that's not the immediate cause for this bug which seems to be on win32, it should be fixed, or you should throw a NotSupported exception on win64 to avoid data corruption. To fix it, our minizip implementation needs to have the long/ulong data types in the C side changed to the proper 64 bits integer and the C# interface will then use long/ulong.
Comment 9 Alan McGovern 2011-08-17 07:46:46 UTC
This bug has been fixed in git and will be part of the next release. I think we're releasing in the next couple of days so just keep an eye out for mono 2.10.5 or higher. If you want a patched build in the meantime, please send me an email. I don't want to attach a binary to the bug report as people with unrelated issues may try downloading it in a few months time and end up breaking their system.
Comment 10 Simon Cropp 2011-08-17 09:36:20 UTC
Alan

If you could send me a patched binary that would be great. This way I can validate the fix.
Comment 11 Simon Cropp 2011-08-18 07:31:28 UTC
Alan

I can confirm your patched assembly fixes my issue.

Thank all for the fast turn around