Bug 33366 - Embedded SQLite library not being used
Summary: Embedded SQLite library not being used
Status: RESOLVED ANSWERED
Alias: None
Product: Xamarin.Mac
Classification: Desktop
Component: Other ()
Version: Master
Hardware: PC Mac OS
: Normal normal
Target Milestone: ---
Assignee: Chris Hamons
URL:
Depends on:
Blocks:
 
Reported: 2015-08-25 13:43 UTC by Cody Beyer (MSFT)
Modified: 2015-08-25 18:26 UTC (History)
3 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
TestCase (1.16 MB, application/zip)
2015-08-25 13:43 UTC, Cody Beyer (MSFT)
Details
LibrarySource (1.16 MB, application/zip)
2015-08-25 16:59 UTC, Nick Berardi
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED ANSWERED

Description Cody Beyer (MSFT) 2015-08-25 13:43:54 UTC
Created attachment 12634 [details]
TestCase

### Description

The following app throws an exception that lists the version of SQLite that is being used by the app. The version it throws is 3008005. However, it shouldn’t be picking it up, because the library has an embedded sqlite3.a and we are referencing the internal library using __Internal, so the external library shouldn’t be getting picked up.

### Test Case

Attached

### Steps to Reproduce

1. Download test case
2. Deploy

### Expected Results

The exception should throw listing the version number of the embedded lib

### Actual Results

The exception lists the version of SQLite on the system

### Versions

=== Xamarin Studio ===

Version 5.10 (build 646)
Installation UUID: 8915c17c-ec61-4ec3-8f89-0683d6363abf
Runtime:
	Mono 4.0.3 ((detached/d6946b4)
	GTK+ 2.24.23 (Dark theme)

	Package version: 400030020

=== Xamarin.Profiler ===

Not Installed

=== Apple Developer Tools ===

Xcode 6.4 (7720)
Build 6E35b

=== Xamarin.iOS ===

Version: 8.10.4.46 (Business Edition)
Hash: 2c66d2f
Branch: master
Build date: 2015-08-04 13:52:25-0400

=== Xamarin.Mac ===

Version: 2.0.2.111 (Business Edition)

=== Xamarin.Android ===

Version: 5.1.5.3 (Business Edition)
Android SDK: /Users/codybeyer/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		4.0.3 (API level 15)
		4.4   (API level 19)
		5.0   (API level 21)
		5.1   (API level 22)

SDK Tools Version: 24.3.3
SDK Platform Tools Version: 22.0.0
SDK Build Tools Version: 22.0.1

Java SDK: /usr
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

=== Xamarin Android Player ===

Version: 0.4.4
Location: /Applications/Xamarin Android Player.app

=== Build Information ===

Release ID: 510000646
Git revision: 1857a62c2e3e5bda6d0ec70806f63b10bb31816f
Build date: 2015-08-25 10:11:13-04
Xamarin addins: 42db5671a9e03cea6fc8d6302c294835a203116e
Build lane: monodevelop-lion-master

=== Operating System ===

Mac OS X 10.10.5
Darwin BeyerStudios.router 14.5.0 Darwin Kernel Version 14.5.0
    Wed Jul 29 02:26:53 PDT 2015
    root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64
Comment 1 Chris Hamons 2015-08-25 14:34:03 UTC
If you look at the build log, mmp is being passed no library to embed nor is there anything in the csproj to suggest this. 

Please explain what behavior you believe is incorrect, not just that it isn't working as you'd expect.
Comment 2 Nick Berardi 2015-08-25 15:30:35 UTC
The library is already embedded in the SQLiteRaw dll.  As I understand it you don't need to use mmp in this case.  

The library in question is included in the TestCase, you should be able to inspect it and find that everything is in working order.
Comment 4 Chris Hamons 2015-08-25 16:27:07 UTC
So are you sure that this case even works on Mac? From their source code:

https://github.com/ericsink/SQLitePCL.raw/blob/master/src/cs/sqlite3_pinvoke.cs#L1210

	// TODO can the code below be adapted to cope with Mono on Mac or Linux?

        // Copyright (c) Microsoft Open Technologies, Inc. All rights reserved.
        // Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information.
        // https://github.com/aspnet/DataCommon.SQLite/blob/dev/src/Microsoft.Data.SQLite/Utilities/NativeLibraryLoader.cs

        [DllImport("kernel32")]
        private static extern IntPtr LoadLibraryEx(string lpFileName, IntPtr hFile, uint dwFlags);

        private static bool TryLoadFromDirectory(string dllName, string baseDirectory)
        {
}


There is no LoadLibraryEx on OS X. Am I missing something?
Comment 5 Nick Berardi 2015-08-25 16:50:59 UTC
That is an old comment, if you load up the library you will see that that code is used no where in the compiled library.  That is used in the Windows compiled version. 

If this was still referencing kernal32, the library wouldn't run and it wouldn't give the OS's version back.
Comment 6 Nick Berardi 2015-08-25 16:59:20 UTC
Created attachment 12639 [details]
LibrarySource

The source code to compile SQLitePCL.raw
Comment 7 Nick Berardi 2015-08-25 16:59:46 UTC
I have attached the library source so that you can compile the library for yourself and see the problem.
Comment 8 Chris Hamons 2015-08-25 18:26:01 UTC
So the tl;dr; is:

Xamarin.Mac does not support linkwith attribute currently, so you'll need to pull in the static library via these additional mmp arguments:

--link_flags "-Wl,-force_load,/Users/donblas/packaged_sqlite3.a"

This means you'll need to keep the static library along with the native library.

LinkWith handles this for you on iOS. We hope to have support in Xamarin.Mac for it in the future.

The long answer:

After getting the source for the library in question, I was able to confirm what the disassembly via monodis suggested, the library was using __Internal correctly but was expecting the LinkWith attribute to pull in the library in question. Once i pulled in the library, individual calls such as:

		[DllImport("__Internal", CallingConvention = CallingConvention.Cdecl)]
		public static extern int sqlite3_libversion_number();
		
               Console.WriteLine (sqlite3_libversion_number ());

would work if I put them in the main module. However, the library would continue failing. More debugging determines that unless we added -force_load, a limitation in our linker would cause those static library symbols to be stripped unless referenced in the main assembly. A similar thing happens on iOS (https://developer.xamarin.com/guides/ios/advanced_topics/native_interop/). 

The reason for the mysterious "second" sql library to be pulled in can be found with a MONO_DEBUG_LEVEL trace:

MONO_LOG_LEVEL=debug /Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MacOS/test

gives:

2015-08-25 16:40:35.361 test[7271:8444922] info: DllImport attempting to load: 'libsqlite3.0.dylib'.
2015-08-25 16:40:35.361 test[7271:8444922] info: DllImport error loading library '/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib': 'dlopen(/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib, 9): image not found'.
2015-08-25 16:40:35.361 test[7271:8444922] info: DllImport error loading library '/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib': 'dlopen(/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib, 9): image not found'.
2015-08-25 16:40:35.361 test[7271:8444922] info: DllImport error loading library '/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib.so': 'dlopen(/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib.so, 9): image not found'.
2015-08-25 16:40:35.361 test[7271:8444922] info: DllImport error loading library '/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib.bundle': 'dlopen(/Users/donblas/Downloads/test/test/bin/Debug/test.app/Contents/MonoBundle/libsqlite3.0.dylib.bundle, 9): image not found'.
2015-08-25 16:40:35.361 test[7271:8444922] info: DllImport loaded library 'libsqlite3.0.dylib'.
2015-08-25 16:40:35.361 test[7271:8444922] info: DllImport searching in: 'libsqlite3.0.dylib' ('libsqlite3.0.dylib').
2015-08-25 16:40:35.362 test[7271:8444922] info: Searching for 'sqlite3_libversion_number'.
2015-08-25 16:40:35.362 test[7271:8444922] info: Probing 'sqlite3_libversion_number'.
2015-08-25 16:40:35.362 test[7271:8444922] info: Found as 'sqlite3_libversion_number'.
3008005

And it appears we resolve to one of these files (I did not track which ones):

/Library/Frameworks/Mono.framework

./Versions/3.12.1/lib/libsqlite3.0.8.6.dylib
./Versions/3.12.1/lib/libsqlite3.0.dylib
./Versions/3.12.1/lib/libsqlite3.dylib
./Versions/4.0.3/lib/libsqlite3.0.8.6.dylib
./Versions/4.0.3/lib/libsqlite3.0.dylib
./Versions/4.0.3/lib/libsqlite3.dylib
./Versions/4.2.0/lib/libsqlite3.0.8.6.dylib
./Versions/4.2.0/lib/libsqlite3.0.dylib
./Versions/4.2.0/lib/libsqlite3.dylib