Bug 32005 - Can't automate Xamarin.NUnitLite TestSuiteActivity
Summary: Can't automate Xamarin.NUnitLite TestSuiteActivity
Status: RESOLVED ANSWERED
Alias: None
Product: Android
Classification: Xamarin
Component: Mono runtime / AOT Compiler ()
Version: 5.1
Hardware: PC Mac OS
: --- normal
Target Milestone: ---
Assignee: Jonathan Pryor
URL:
Depends on:
Blocks:
 
Reported: 2015-07-16 05:45 UTC by Antonio
Modified: 2016-08-24 13:30 UTC (History)
2 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED ANSWERED

Description Antonio 2015-07-16 05:45:23 UTC
Hello,

I'm trying to automate the launching of a Xamarin.Android.NUnitLite application so i can complete my CI tasks.

I'm adding intents to the MainActivity generated from this kind of project that inherits from TestSuiteActivity by:

using System.Reflection;

using Android.App;
using Android.OS;
using Xamarin.Android.NUnitLite;

namespace Namespace.Test
{
	[Activity (Label = "Namespace.Test", MainLauncher = true)]
	public class MainActivity : TestSuiteActivity
	{
		protected override void OnCreate(Bundle bundle)
		{
			AddTest (Assembly.GetExecutingAssembly ());

			Intent.PutExtra ("automated", true);
			Intent.PutExtra ("remote", true);
			Intent.PutExtra ("hostName", "http://portquiz.net");
			Intent.PutExtra ("hostPort", 8080);
			base.OnCreate (bundle);
		}

		public override void Finish ()
		{
			System.Environment.Exit (0);
		}
	}
}

Now... if i make hostPort a string on purpose to fail that part of the operation, the app will launch and run the tests automatically. If i make a valid port and host like in the example above the app fails to launch with trying to show a Toast from a place it can't (probably):

[monodroid-gc] GREF GC Threshold: 46080
[NUnitLite] NUnit automated tests loaded.
[NUnitLite] NUnit automated tests started
[Adreno-EGL] <qeglDrvAPI_eglInitialize:316>: EGL 1.4 QUALCOMM build:  (CL4169980)
[Adreno-EGL] OpenGL ES Shader Compiler Version: 17.01.10.SPL
[Adreno-EGL] Build Date: 12/25/13 Wed
[Adreno-EGL] Local Branch: 
[Adreno-EGL] Remote Branch: 
[Adreno-EGL] Local Patches: 
[Adreno-EGL] Reconstruct Branch: 
[16/07/2015 10:14:44] Sending 'Run Everything' results to http://portquiz.net:8080
[libc] [NET] getaddrinfo+,hn 19(0x687474703a2f2f),sn(0),family 10,flags 2
[libc] [NET] getaddrinfo-, 1
[libc] [NET] getaddrinfo_proxy+
[libc] [NET] getaddrinfo_proxy-,TEMP_FAILURE_RETRY
[mono] 
[mono] Unhandled Exception:
[mono] Java.Lang.Exception: Exception of type 'Java.Lang.Exception' was thrown.
[mono]   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
[mono]   at Android.Runtime.JNIEnv.CallStaticObjectMethod (IntPtr jclass, IntPtr jmethod, Android.Runtime.JValue* parms) [0x00000] in <filename unknown>:0 
[mono]   at Android.Widget.Toast.MakeText (Android.Content.Context context, ICharSequence text, ToastLength duration) [0x00000] in <filename unknown>:0 
[mono]   at Android.Widget.Toast.MakeText (Android.Content.Context context, System.String text, ToastLength duration) [0x00000] in <filename unknown>:0 
[mono]   at Xamarin.Android.NUnitLite.AndroidRunner.OpenWriter (System.String message, Android.Content.Context activity) [0x00000] in <filename unknown>:0 
[mono]   at Xamarin.Android.NUnitLite.AndroidRunner.Run (NUnit.Framework.Internal.Test test, Android.Content.Context context) [0x00000] in <filename unknown>:0 
[mono]   at Xamarin.Android.NUnitLite.TestSuiteActivity+<OnCreate>c__AnonStorey1.<>m__3 (System.Object ) [0x00000] in <filename unknown>:0 
[mono]   --- End of managed exception stack trace ---
[mono] java.lang.RuntimeException: Can't create handler inside thread that has not called Looper.prepare()
[mono] 	at android.os.Handler.<init>(Handler.java:200)
[mono] 	at android.os.Handler.<init>(Handler.java:114)
[mono] 	at android.widget.Toast$TN.<init>(Toast.java:327)
[mono] 	at android.widget.Toast.<init>(Toast.java:92)
[mono] 	at android.widget.Toast.makeText(Toast.java:241)
[mono] 	at dalvik.system.NativeStart.run(Native Method)
[mono] 

Thanks for your support.

Antonio
Comment 1 Jonathan Pryor 2015-07-16 14:35:52 UTC
> I'm trying to automate the launching of a Xamarin.Android.NUnitLite application
> so i can complete my CI tasks.
> 
> I'm adding intents to the MainActivity generated from this kind of project that
> inherits from TestSuiteActivity by:

Don't.

(This is something that we dearly need to write better documentation for.)

The problem with your approach is that calling Environment.Exit() will cause Android to re-start your Activity (or did last I checked?). It's also not a great solution because there's no way for the test runner to provide feedback to your CI server.

In short, using Activities is painful and doesn't integrate or work well.

There is a better solution: inherit from TestSuiteInstrumentation:

https://kb.xamarin.com/customer/portal/articles/1649768-how-do-i-automate-an-android-unit-test-project-

By using `adb shell am instrument -w` (as in the above URL), the invocation will block until all the tests have run, and then the results of all the tests will be printed to the shell, where your CI server can parse the output to determine pass/fail status.
Comment 2 Antonio 2015-07-16 15:30:07 UTC
Thanks Jonathan.

Its nice to speak to you again.

I ended up doing a very raw basic for the CI to check it by not using the "remote"(and therefore hostname and port), adb launching the app with "automated" run and then parsing the adb logcat of the test runner.

Obviously this is a temporary solution and will definitely try your recommendation tomorrow.

Thanks for you support and i hope you are well.

Antonio
Comment 3 Vishwas 2016-08-24 13:30:27 UTC
Hi,

I have followed the approach mentioned in the link given above. 

I am able to automate test cases through the Xamarin.Android.NUnitLite.TestSuiteInstrumentation  and ADB commands,
however many of my test cases which are successful through NUnitLite GUI are failing. My observation is Creating Activity , WebView are failing and Forms.Context is returning null which is making test cases fail.


How can we access Forms.Context and Create Activity or Webview for this automated instrumentation test casess to be successful.

Thanks for you support.