Bug 16621 - TestRunner Does Not Execute TestFixtureSetUp/TearDown When Running Individual Tests
Summary: TestRunner Does Not Execute TestFixtureSetUp/TearDown When Running Individual...
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: General ()
Version: 4.10.0.x
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: ---
Assignee: Atsushi Eno
URL:
Depends on:
Blocks:
 
Reported: 2013-12-05 12:24 UTC by Jon Goldberger [MSFT]
Modified: 2013-12-13 11:02 UTC (History)
3 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Comment 1 Jon Goldberger [MSFT] 2013-12-05 12:25:18 UTC
I have this issue:

https://bugzilla.xamarin.com/show_bug.cgi?id=13390

As far as I can tell, this has not been fixed for Android. 

Version information follows:

=== Xamarin Studio ===

Version 4.2.1 (build 1)
Installation UUID: f1732b88-1ebe-448d-89a1-fa075f56d19b
Runtime:
Mono 3.2.5 ((no/964e8f0)
GTK+ 2.24.20 theme: Raleigh
GTK# (2.12.0.0)
Package version: 302050000

=== Apple Developer Tools ===

Xcode 5.0.1 (3335.23)
Build 5A2053

=== Xamarin.Mac ===

Xamarin.Mac: Not Installed

=== Xamarin.iOS ===

Version: 7.0.4.209 (Enterprise Edition)
Hash: 23a0827
Branch:
Build date: 2013-11-11 16:04:00-0500

=== Xamarin.Android ===

Version: 4.10.1 (Enterprise Edition)
Android SDK: /Users/davidbarrett/Library/Developer/android-sdk-mac_x86
Supported Android versions:
2.1 (API level 7)
2.2 (API level 8)
2.3 (API level 10)
3.1 (API level 12)
4.0 (API level 14)
4.0.3 (API level 15)
4.1 (API level 16)
4.2 (API level 17)
4.3 (API level 18)
4.4 (API level 19)
Java SDK: /usr
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)

=== Build Information ===

Release ID: 402010001
Git revision: 844a84fe0aa0cb5f986d4e3c4807a51487d07845
Build date: 2013-11-13 22:12:16+0000
Xamarin addins: 97e44e4863da6c479427794457637e75b3d22600

=== Operating System ===

Mac OS X 10.8.5
Darwin davidbarrett-mac.local 12.5.0 Darwin Kernel Version 12.5.0
Sun Sep 29 13:33:47 PDT 2013
root:xnu-2050.48.12~1/RELEASE_X86_64 x86_64
Comment 2 Atsushi Eno 2013-12-11 07:34:34 UTC
Have you asked about the issue to NUnitLite developers? If this is the expected behavior, it should rather be NUnitLite itself but not Touch.Unit or Xamarin.Android.NUnitLite that should "fix" the issue. Otherwise it brings *inconsistency* between uses of NUnitLite in other environments and that in Xamarin.

According to the committed changes, it seems that the fix is rather "hacked" in Touch.Unit. Upgrading to NUnitLite 1.0.0 did not fix anything on Android.

I think NUnitLite developers rather defined the behavior of [TestFixtureSetup] and [TestFixtureTearDown] to work like that, because their implementation explictly creates their "OneTimeSetup" internal test runner commands in TestSuite.
http://bazaar.launchpad.net/~nunitlite-dev/nunitlite/trunk/view/head:/src/framework/Internal/Tests/TestSuite.cs#L180
Comment 3 Sebastien Pouliot 2013-12-11 09:09:38 UTC
[copy of my comment on bug #13390]

> Have you asked about the issue to NUnitLite developers? 

No I took the lazy approach and tried it myself.

* There's no "official" NUnitLite runner GUI, except the one we provided with
XI and XA, that allow you to run a single test. 

* The console runner (for the full nunit) does allow "single test execution". 

> it should rather be NUnitLite itself

No. The fact is that Touch.Unit, not NUnitLite, support this "single test
execution" feature so I don't expect the library to take care of this for me.
If you copied Touch.Unit "hacked" feature then you're in the same situation.

> Otherwise it brings *inconsistency*

Here's how NUnit behave for consistency purpose:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NUnit.Framework;

namespace nunit3test {
    [TestFixture]
    public class Class1 {

        bool setup;

        [TestFixtureSetUp]
        public void Setup ()
        {
            setup = true;
        }

        [TestFixtureTearDown]
        public void TearDown ()
        {
            Assert.False (setup);
        }

        [Test]
        public void Test1 ()
        {
            Assert.True (setup);
            setup = false;
        }

        [Test]
        public void Test2 ()
        {
            Assert.True (setup);
            setup = false;
        }
    }
}

compile (w/csc) and execute like this:

> nunit-console.exe -framework=net-4.5 Class1.dll -test=nunit3test.Class1.Test1

and you'll get only one test has executed with no failure.

> Selected test(s):
>    nunit3test.Class1.Test1
>
> Test Run Summary -
>    Overall result: Passed
>    Tests run: 1, Errors: 0, Failures: 0, Inconclusive: 0
>      Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
>         Time: 0 seconds

TestResult.xml will show that Test1 passed successfully - which can only
happens if the Setup method was called when a single test was selected.
Comment 4 Atsushi Eno 2013-12-13 08:35:03 UTC
This feature is now implemented. Thanks Sebastien for the input.

[master e063674]
Comment 5 David Barrett 2013-12-13 11:02:35 UTC
Thank you, Atsushi!  I've been following this bug and am anxious to have this fixed so I don't have to hack up my tests to conditionally run setup routines if they haven't already.