Bug 58150 - MLModel.GetPrediction hard-crashes app
Summary: MLModel.GetPrediction hard-crashes app
Status: RESOLVED INVALID
Alias: None
Product: iOS
Classification: Xamarin
Component: Xamarin.iOS.dll ()
Version: XI 10.99 (xcode9)
Hardware: PC Mac OS
: Normal normal
Target Milestone: Xcode9
Assignee: Alex Soto [MSFT]
URL:
Depends on:
Blocks:
 
Reported: 2017-07-13 23:37 UTC by Larry O'Brien
Modified: 2017-07-20 01:59 UTC (History)
3 users (show)

Tags:
Is this bug a regression?: No
Last known good build:


Attachments
Xcode SqueezeNet CoreML sample (works) (6.57 MB, application/zip)
2017-07-13 23:37 UTC, Larry O'Brien
Details
Xamarin CoreML project (crashes) (4.46 MB, application/zip)
2017-07-13 23:38 UTC, Larry O'Brien
Details
Xamarin Test Project V2 (4.47 MB, application/zip)
2017-07-14 21:41 UTC, Vincent Dondain [MSFT]
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED INVALID

Description Larry O'Brien 2017-07-13 23:37:21 UTC
Created attachment 23544 [details]
Xcode SqueezeNet CoreML sample (works)

Attempting to core MLModel.GetPrediction() causes the app to disappear with no exception and no relevant message in Console. 

I have attached a Swift project that uses the SqueezeNet model provided by Apple to classify an image. I have also attached what I think is an equivalent VS/Mac project. 

When the Xamarin project executes line 106 in SqueezeNet.cs (`var outFeatures = mdl.GetPrediction(inputs, out err);`) the app crashes without either throwing an exception or any useful information going to Console. 

To see the equivalent line in the Swift code, open the Xcode project, go to the Project Navigator, and select the SqueezeNet.mlmodel file. In the main editor pane, you'll see "Model class" and "SqueezeNet (Swift generated source code)". Within that, line 61 ( var outFeatures = mdl.GetPrediction(inputs, out err); ) is the equivalent call.  

(I copied SqueezeNet.mlmodelc from ~/Library/Developer/DerivedData/{proj}/Build/Products/Debug-iphoneos/MKK_Swift0.app/)
Comment 1 Larry O'Brien 2017-07-13 23:38:05 UTC
Created attachment 23545 [details]
Xamarin CoreML project (crashes)
Comment 2 Vincent Dondain [MSFT] 2017-07-14 21:40:41 UTC
I can confirm the issue with the Xamarin project. Now we haven't looked into compiling .mlmodel files from an MSBuild point of view (that could be it).

Search for "CoreMLModelCompile" in https://gist.github.com/VincentDondain/04aa1ef249e59a67156d6c9245704933.

Also your Xamarin project did't build so I re-uploaded it and your swift project crashes at runtime on the UIImage.
Comment 3 Vincent Dondain [MSFT] 2017-07-14 21:41:04 UTC
Created attachment 23576 [details]
Xamarin Test Project V2
Comment 5 Larry O'Brien 2017-07-17 23:13:00 UTC
The problem may be in the manner in which I specified the `IMLFeatureProvider` classes. By switching to a `MLDictionaryFeatureProvider` things work fine:

	public string Prediction(CVPixelBuffer input)
	{
		var imageValue = MLFeatureValue.FromPixelBuffer(input);
		var inputs = new NSDictionary<NSString, NSObject>(new NSString("image"), imageValue);

		var inputFp = new MLDictionaryFeatureProvider(inputs, out var error);
		var outFeatures = this.Model.GetPrediction(inputFp, out var err2);
		var label = outFeatures.GetFeatureValue("classLabel").StringValue;
		return label;
	}
Comment 6 Sebastien Pouliot 2017-07-19 13:29:39 UTC
@Alex can you have a look ? thanks!
Comment 7 Alex Soto [MSFT] 2017-07-20 01:59:04 UTC
The issue is inside SqueezeNetInputs class

> public class SqueezeNetInputs : NSObject, IMLFeatureProvider {
> ...
> // The below property causes infinite recursion if you remove this
> // the sample does not crash
> public IntPtr Handle => this.Handle;
> ...
> }

Cheers!