Bug 36452 - Visual Studio hangs when running (or pausing while running) xUnit tests via the Test Runner while debugging (on device only).
Summary: Visual Studio hangs when running (or pausing while running) xUnit tests via t...
Status: CONFIRMED
Alias: None
Product: Visual Studio Extensions
Classification: Xamarin
Component: Test Cloud ()
Version: 4.0.0 (C6)
Hardware: PC Mac OS
: Normal normal
Target Milestone: Future Cycle
Assignee: Bugzilla
URL:
Depends on:
Blocks:
 
Reported: 2015-12-02 01:48 UTC by Jon Goldberger [MSFT]
Modified: 2016-12-27 22:14 UTC (History)
5 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Test Project (1.14 MB, application/zip)
2015-12-02 01:48 UTC, Jon Goldberger [MSFT]
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report for Bug 36452 on Developer Community if you have new information to add and do not yet see a matching new report.

If the latest results still closely match this report, you can use the original description:

  • Export the original title and description: Developer Community HTML
  • Copy the title and description into the new report. Adjust them to be up-to-date if needed.
  • Add your new information.
Related Links:
Status:
CONFIRMED

Description Jon Goldberger [MSFT] 2015-12-02 01:48:43 UTC
Created attachment 14067 [details]
Test Project

## Description

Visual Studio hangs when running (or pausing while running) xUnit tests via the Test Runner while debugging (on device only). This does not occur 100% of the time, but the vast majority of the time. 

## Steps to reproduce.

1. Open the attached test project in Visual Studio. (Tested on VS 2015 running on Windows 10)

2. Set the startup project to WorkoutWotch.UI.TestHarness.iOS and select the Debug | iPhone configuration.

3. Deploy the WorkoutWotch.UI.TestHarness.iOS project to an iOS device (Tested on iPad Mini and iPad Air 2 running iOS 9.1)

4. When the Test Runner UI appears, tap "Run Everything"

Expected result: All tests will run... some may fail, but all should run. 

Actual result: Tests do not complete and Visual Studio freezes. 

5. Additionally try hitting "Pause" button in Visual Studio while the tests are running, so repeat steps 3 and 4 and then:

6. Hit "Pause" in Visual Studio. 

Expected result: Code where execution paused will open in document pane and "Continue" button will be available to continue execution of tests. 

Actual result: File with code does not open and "Continue" button is greyed out. Pressing "Stop" at this point in Visual Studio causes Visual Studio to freeze and need to be force quit. 

## Notes:

There is nothing notable in the Application Output. 

I got better results using the "Run all" button on the page that lists all the tests individually. 

Also running many subsets of the full suite of tests allowed normal, expected operation, e.g.:

If I run only the tests in WorkoutWotch.UnitTests.Models.Events, WorkoutWotch.UnitTests.Models.EventMatchers, and WorkoutWotch.UnitTests.Utility, using "Run Everything" seemed to worked consistently. 

If I run only the tests in ExerciseViewModelFixture, using "Run Everything" seemed to worked consistently. 

If I run only the tests in ExerciseProgramViewModelFixture, using "Run Everything" seemed to worked consistently. 

If I run only the tests in ExerciseProgramsViewModelFixture, using "Run Everything" seemed to worked consistently. 

If I run only the tests in ExerciseViewModelFixture, ExerciseProgramViewModelFixture, and ExerciseProgramsViewModelFixture, using "Run Everything" seemed to worked consistently. 

Also "Run Everything" seems to work with all tests when disconnected from the debugger (but using a Debug build). 

It also worked with the debugger connected at least once (without pausing the tests). 

I was even able to have all the tests run using "Run Everything" and pausing and continuing the Debug session at least once. 


## Version info:

Microsoft Visual Studio Professional 2015
Version 14.0.23107.0 D14REL
Microsoft .NET Framework
Version 4.6.00079

Installed Version: Professional

Visual Basic 2015   00322-40000-00000-AA850
Microsoft Visual Basic 2015

Visual C# 2015   00322-40000-00000-AA850
Microsoft Visual C# 2015

Visual C++ 2015   00322-40000-00000-AA850
Microsoft Visual C++ 2015

Visual F# 2015 RC   00322-40000-00000-AA850
Microsoft Visual F# 2015 RC

Windows Phone SDK 8.0 - ENU   00322-40000-00000-AA850
Windows Phone SDK 8.0 - ENU

Application Insights Tools for Visual Studio Package   1.0
Application Insights Tools for Visual Studio

ASP.NET and Web Tools   14.0.20626.0
ASP.NET and Web Tools

ASP.NET Web Frameworks and Tools 2013   5.2.30624.0
For additional information, visit http://www.asp.net/

Common Azure Tools   1.5
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

GenerateUnitTest   1.0
Generates unit test code for methods in classes under test.

GitHub.VisualStudio   1.0
A Visual Studio Extension that brings the GitHub Flow into Visual Studio.

Microsoft Azure Mobile Services Tools   1.4
Microsoft Azure Mobile Services Tools

NuGet Package Manager   3.2.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

PreEmptive Analytics Visualizer   1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

SQL Server Data Tools   14.0.50616.0
Microsoft SQL Server Data Tools

Visual Studio Tools for Universal Windows Apps   14.0.23309.00 d14oob
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.

Xamarin   4.0.0.1689 (f860fe4)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin.Android   6.0.0.34 (3efa14c)
Visual Studio plugin to enable development for Xamarin.Android.

Xamarin.iOS   9.2.1.51 (3c0ec35)
Visual Studio extension to enable development for Xamarin.iOS.

=== Xamarin Studio ===

Version 5.10 (build 871)
Installation UUID: 964c531b-d928-456b-a9ae-e1f82266b360
Runtime:
	Mono 4.2.1 (explicit/6dd2d0d)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 402010102

=== Xamarin.Profiler ===

Version: 0.22.0.0
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Apple Developer Tools ===

Xcode 7.1.1 (9081)
Build 7B1005

=== Xamarin.iOS ===

Version: 9.2.1.51 (Business Edition)
Hash: 3c0ec35
Branch: master
Build date: 2015-11-12 13:05:39-0500

=== Xamarin.Android ===

Version: 6.0.0.34 (Business Edition)
Android SDK: /Users/apple/Library/Developer/Xamarin/android-sdk-mac_x86
	Supported Android versions:
		4.0.3 (API level 15)
		4.1   (API level 16)
		4.2   (API level 17)
		4.3   (API level 18)
		4.4   (API level 19)
		5.0   (API level 21)
		5.1   (API level 22)
		6.0   (API level 23)

SDK Tools Version: 24.4.1
SDK Platform Tools Version: 23.0.1
SDK Build Tools Version: 23.0.2

Java SDK: /usr
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

=== Xamarin Android Player ===

Version: 0.6.5
Location: /Applications/Xamarin Android Player.app

=== Xamarin.Mac ===

Version: 2.4.0.109 (Business Edition)

=== Build Information ===

Release ID: 510000871
Git revision: 4e9c5abb5ffdae12ba02ac49da83f8b2011dbb88
Build date: 2015-11-12 06:02:54-05
Xamarin addins: 55007ed0e56436f385d8e26394a45be563abc7e8
Build lane: monodevelop-lion-cycle6

=== Operating System ===

Mac OS X 10.11.1
Darwin Jons-iMac.local 15.0.0 Darwin Kernel Version 15.0.0
    Sat Sep 19 15:53:46 PDT 2015
    root:xnu-3247.10.11~1/RELEASE_X86_64 x86_64
Comment 1 Jon Goldberger [MSFT] 2015-12-02 01:50:29 UTC
Other bugs that might be relevant:
bug #36078
bug #36087
Comment 2 Joaquin Jares 2016-12-20 13:41:59 UTC
Moving to Test Cloud