Bug 34737 - "Run in Test Cloud" Results in an Upload Failed Error
Summary: "Run in Test Cloud" Results in an Upload Failed Error
Status: VERIFIED FIXED
Alias: None
Product: Visual Studio Extensions
Classification: Xamarin
Component: Test Cloud ()
Version: 4.0.0 (C6)
Hardware: PC Mac OS
: Highest normal
Target Milestone: 4.0.0 (C6)
Assignee: Akhilesh kumar
URL:
Depends on:
Blocks:
 
Reported: 2015-10-09 18:03 UTC by Jimmy [MSFT]
Modified: 2015-11-26 10:39 UTC (History)
6 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Repro project (18.65 KB, application/zip)
2015-10-09 18:03 UTC, Jimmy [MSFT]
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
VERIFIED FIXED

Description Jimmy [MSFT] 2015-10-09 18:03:52 UTC
Created attachment 13265 [details]
Repro project

### Overview
Trying to submit tests with “Run in TestCloud” fails to upload the IPA because it tries to upload an IPA with the wrong name.

IPA file names have their build string from Info.plist appended at the end, but VS ignores the build string and always tries to upload an IPA with “-.1.0” in its name. This results in an error:

> INFORMATION: test-cloud.exe asubmit "\\psf\Home\Documents\Visual Studio 2015\Projects\TestCloudUpload\TestCloudUpload\bin\iPhone\Debug\TestCloudUpload-1.0.ipa" --assembly-dir "\\psf\Home\Documents\Visual Studio 2015\Projects\TestCloudUpload\AppTests\bin\Debug" --no-browser
> INFORMATION: Apk / ipa file doesn't exist: \\psf\Home\Documents\Visual Studio 2015\Projects\TestCloudUpload\TestCloudUpload\bin\iPhone\Debug\TestCloudUpload-1.0.ipa

This isn't an issue for newly created iOS projects since they are given the "1.0" build string by default, but once a user changes that manually it causes future submissions to fail.


### Steps to Reproduce
1. Open the attached solution
2. Build the iOS project and UITest project
3. Right click the UITest project and select “Run in TestCloud…”


### Actual Results 
The upload will fail because the IPA’s filename is actually “TestCloudUpload-5.0.ipa” since that’s the build string provided in Info.plist.


### Expected Results 
The IPA is submitted to XTC successfully.


### Environment Info
I tested this with stable and alpha channel builds and was able to reproduce it on both.


Microsoft Visual Studio Professional 2015
Version 14.0.23107.0 D14REL
Microsoft .NET Framework
Version 4.6.00081

Installed Version: Professional

Visual Basic 2015   00322-40000-00000-AA771
Microsoft Visual Basic 2015

Visual C# 2015   00322-40000-00000-AA771
Microsoft Visual C# 2015

Visual C++ 2015   00322-40000-00000-AA771
Microsoft Visual C++ 2015

Windows Phone SDK 8.0 - ENU   00322-40000-00000-AA771
Windows Phone SDK 8.0 - ENU

Application Insights Tools for Visual Studio Package   1.0
Application Insights Tools for Visual Studio

ASP.NET and Web Tools   14.0.20626.0
ASP.NET and Web Tools

ASP.NET Web Frameworks and Tools 2013   5.2.30624.0
For additional information, visit http://www.asp.net/

Common Azure Tools   1.5
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

GenerateUnitTest   1.0
Generates unit test code for methods in classes under test.

Microsoft Azure Mobile Services Tools   1.4
Microsoft Azure Mobile Services Tools

NuGet Package Manager   3.0.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

PreEmptive Analytics Visualizer   1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

SQL Server Data Tools   14.0.50616.0
Microsoft SQL Server Data Tools

Xamarin   4.0.0.1566 (5abb0de)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin.Android   6.0.0.11 (f94dc5b)
Visual Studio plugin to enable development for Xamarin.Android.

Xamarin.iOS   9.2.0.84 (b5396c2)
Visual Studio extension to enable development for Xamarin.iOS.
Comment 4 Joaquin Jares 2015-10-13 13:01:57 UTC
Fixed via https://github.com/xamarin/XamarinVS/pull/4302 (master)
Comment 5 Jimmy [MSFT] 2015-10-13 14:14:23 UTC
Current workarounds:

- Keep the Info.plist build string at "1.0"

- Use the command line tool to upload the tests to XTC: http://developer.xamarin.com/guides/testcloud/uitest/working-with/submitting-tests-at-command-line/
Comment 6 Akhilesh kumar 2015-10-21 15:02:30 UTC
I have checked this issue with attached test sample and steps provided in bug description.

I am able to reproduce this issue with XVS 4.0.0.1566 + XI 9.2.0.85.

http://screencast.com/t/WBfVs1quBkF

However, this issue is working fine with XVS 99.0.0.427 + XI 9.3.0.178, I am successfully able to “Run in TestCloud…” AppTests.


Screencast: http://screencast.com/t/b12ovhdogvk


So I am closing this issue.

Microsoft Visual Studio Ultimate 2013
Version 12.0.40629.00 Update 5
Microsoft .NET Framework
Version 4.6.00081

Xamarin   99.0.0.427 (4b209da)
Xamarin.Android   6.0.99.87 (839c5d4)
Xamarin.iOS   9.3.0.166 (aa23e77)
Comment 7 xamarin-release-manager 2015-10-21 17:53:56 UTC
Fixed in version 4.0.0.1614 (cycle6)

Author: Jose Gallardo
Commit: 53c16c478f7a39e272ee3b6bd1e89bf189436809 (xamarin/XamarinVS)
Comment 8 Akhilesh kumar 2015-10-23 10:04:00 UTC
I am able to reproduce this issue with latest C6 builds (XVS 4.0.0.1616 + XI 9.2.0.100). 

Screencast: http://screencast.com/t/NXwfW73XVAWr

Environment info:
Microsoft Visual Studio Enterprise 2015
Version 14.0.23107.0 D14REL
Microsoft .NET Framework
Version 4.6.00081

Xamarin   4.0.0.1616 (3a8391b)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin.Android   6.0.0.21 (503bdcb)
Visual Studio plugin to enable development for Xamarin.Android.

Xamarin.iOS   9.2.0.96 (2e83e2e)
Visual Studio extension to enable development for Xamarin.iOS.

***********************************

Runtime:
 Mono 4.2.1 (explicit/c706a85)
 GTK+ 2.24.23 (Raleigh theme)

 Package version: 402010081

=== Apple Developer Tools ===

Xcode 7.0 (8227)
Build 7A220

=== Xamarin.iOS ===

Version: 9.2.0.100 (Business Edition)
Hash: c1ac250
Branch: master
Build date: 2015-10-22 19:33:04-0400

Mac OS X 10.10.5
Darwin Apples-iMac.local 14.5.0 Darwin Kernel Version 14.5.0
    Wed Jul 29 02:26:53 PDT 2015
    root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64
Comment 9 Enzo Heredia 2015-10-23 14:45:52 UTC
It worked OK for me. Same XVS build 4.0.0.1616

1 - Downloaded attached sample
2 - build both ios and ui project
3 - click on run on test cloud

Then the ipa file was uploaded suceesfully.

Akhilesh can you try building both and the run in test cloud?
Comment 10 Akhilesh kumar 2015-10-23 15:58:42 UTC
Thanks Enzo, I have rechecked this by downloading fresh attached sample and clean and rebuild both project, I am also able to run in test cloud.

Screencast: http://screencast.com/t/PfA9npZO

Thanks!
Comment 11 Akhilesh kumar 2015-10-26 11:16:59 UTC
I have also checked this issue with iOS 9.1-C5SR5 builds (XVS 3.11.1587 + XI 9.1.0.18) I am successfully able to run attached project to "Run in Test Cloud".

Microsoft Visual Studio Ultimate 2013
Version 12.0.40629.00 Update 5
Microsoft .NET Framework
Version 4.6.00081

Xamarin   3.11.1587.0 (6be1e88)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin.Android   5.1.7.12 (53fce3730830417896a42f365a5ba35f1ee58d9d)
Visual Studio plugin to enable development for Xamarin.Android.

Xamarin.Forms Intellisense   1.0
Provides intellisense for Xamarin.Forms in the XML editor.

Xamarin.iOS   9.1.0.0 (a49a79c0e30fc5f9acf323008a9e44be2f0c1bd2)
Visual Studio extension to enable development for Xamarin.iOS.

Xamarin.iOS Unified Migration   1.0
Automated migration for Xamarin iOS Classic projects to Unified

Xamarin.TestCloud.Integration   1.0
Early preview of Xamarin Test Cloud integration


***********************

XI 9.1.0.18
Xcode 7.1 Stable
Comment 12 xamarin-release-manager 2015-11-26 10:39:45 UTC
Fixed in version 4.0.1.1 (cycle6)

Author: Jose Gallardo
Commit: 1b588eb781ec329dd7d6c522da0651d410add926 (xamarin/XamarinVS)