Bug 7859 - Huge memory allocation during zeroblobs initialization
Summary: Huge memory allocation during zeroblobs initialization
Status: RESOLVED UPSTREAM
Alias: None
Product: iOS
Classification: Xamarin
Component: Xamarin.iOS.dll ()
Version: 5.3.x
Hardware: PC Mac OS
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2012-10-16 03:23 UTC by Alex Anikin
Modified: 2012-10-16 10:18 UTC (History)
2 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Example (6.24 KB, application/zip)
2012-10-16 03:24 UTC, Alex Anikin
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED UPSTREAM

Description Alex Anikin 2012-10-16 03:23:53 UTC
We've found a hot issue in Sqlite database implementation and need urgent
technical help.

Our application crashes during operations with large blobs in database.

Attached sample demonstrates huge memory allocation during zeroblobs
initialization. In first case no additional memory is allocated, but in
ZeroblobFail method approx 100Mb of memory is allocated for a quick period
of zeroblob creation.

Looks like the issue depends on table fields creation order that is not OK
for any database.

Here is quote:
http://www.sqlite.org/c3ref/bind_blob.html
«The sqlite3_bind_zeroblob() routine binds a BLOB of length N that is
filled with zeroes. A zeroblob uses a fixed amount of memory (just an
integer to hold its size) while it is being processed. Zeroblobs are
intended to serve as placeholders for BLOBs whose content is later written
using incremental BLOB I/O <http://www.sqlite.org/c3ref/blob_open.html>routines»

Waiting for your quick answer.
Comment 1 Alex Anikin 2012-10-16 03:24:17 UTC
Created attachment 2749 [details]
Example
Comment 2 Rolf Bjarne Kvinge [MSFT] 2012-10-16 04:58:52 UTC
This looks like an sqlite bug in iOS, i.e. not something we can fix in MonoTouch.

Here is the stack trace for the memory allocation: http://screencast.com/t/fqb0Z6faaJA (I increased the blob 10x to make it show up more easily).

Note that it's inside sqlite3_step, not inside MonoTouch / the base class libraries, so I don't think there is much MonoTouch can do here to fix/work around it automatically.
Comment 3 Alex Anikin 2012-10-16 05:45:46 UTC
Thank you for your answer.
What should we do with that? Write to Apple?
Comment 4 Rolf Bjarne Kvinge [MSFT] 2012-10-16 05:56:45 UTC
Yes, I'd try contacting Apple.

At the same time you should probably try finding a workaround, since it will undoubtedly take some time until Apple fixes this (if they do) and the fix reaches user devices.
Comment 5 Alex Anikin 2012-10-16 10:18:16 UTC
We have an answer from sqlite creator Richard Hipp:
In order for zeroblobs to work as above (using a fixed amount of memory no matter how big they are) all zeroblobs must be at the end of the row.  In other words, the columns of the table that are receiving the zeroblobs must be the last columns in the table.  If any non-zero content follows the zeroblob, then the zeroblob is expanded into a literal sequence of zero bytes, meaning memory must be allocated for the entire zeroblob.

Hope this will help someone in future.