Bug 56496 - Location of linked files in build directory on Mac build host has changed
Summary: Location of linked files in build directory on Mac build host has changed
Status: RESOLVED FIXED
Alias: None
Product: Visual Studio Extensions
Classification: Xamarin
Component: iOS ()
Version: 4.5.0 (15.2)
Hardware: PC Windows
: --- normal
Target Milestone: ---
Assignee: Adrian Alonso
URL:
Depends on:
Blocks:
 
Reported: 2017-05-16 12:27 UTC by Rich Zwaap
Modified: 2017-05-26 22:12 UTC (History)
7 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Visual Studio solution containing repro case (7.73 KB, application/x-zip-compressed)
2017-05-16 12:27 UTC, Rich Zwaap
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Rich Zwaap 2017-05-16 12:27:45 UTC
Created attachment 22199 [details]
Visual Studio solution containing repro case

At the Cycle 15.2 release, the location of linked files on the Mac build host has changed.  To repro from attachment:

1.) In Visual Studio, open LinkedFileTestApp solution included in the attached zip
2.) Build the solution
3.) Navigate to the build directory on the Mac build host.  This should be ~/Library/Caches/Xamarin/mtbs/builds/LinkedFileTestApp/<iOS App GUID>.

The file LinkedFile.txt is present in the bin/<Platform>/<Configuration> (e.g. bin/iPhoneSimulator/Debug) within the build directory.  This differs from the location the file is copied to within the Mac build directory from Cycle 7 through 15.1 (ever since bug 38398 was fixed).  Those earlier releases placed the file at the root of the build directory.   Note the change in behavior only seems to apply when the file is included with Build Action set to "None." 

This change is breaking for logic that relies on this location.  This has, for instance, broken the build of any project referencing our SDK (ArcGIS Runtime for .NET), as the SDK relies on this location for the linking of native libraries.
Comment 1 Daniel Cazzulino 2017-05-16 18:10:09 UTC
Hi Rich, 
thanks for letting us know your scenario. 

In "plain" MSBuild/VisualStudio builds, nothing is copied outside of the OutDir/OutputPath unless you explicitly do so. What we did was remove ad-hoc hacks we had and instead rely on the OOB MSBuild behavior. 

I don't recall the previous behavior you're describing being documented or supported explicitly, but you can trivially bring back the ad-hoc behavior by providing a targets that copies additional stuff to whatever locations you need on the Mac. This is how we're doing it on our new common targets:

  <Target Name="CopyFilesToMacOutputDirectory" DependsOnTargets="_ProcessSymbols" AfterTargets="CopyFilesToOutputDirectory" Condition="'$(IsMacEnabled)' == 'true'">

    <ItemGroup>
      <_CopyFilesToBuildServer Include="@(ReferenceCopyLocalPaths)" Condition="'@(ReferenceCopyLocalPaths)' != '' And '%(Filename)' != 'mscorlib'">
        <TargetPath>%(DestinationSubDirectory)%(Filename)%(Extension)</TargetPath>
      </_CopyFilesToBuildServer>
      <_CopyFilesToBuildServer Include="@(DebugSymbolsProjectOutputGroupOutput -> '%(FinalOutputPath)')">
        <TargetPath>%(Filename)%(Extension)</TargetPath>
      </_CopyFilesToBuildServer>
      <_CopyFilesToBuildServer Include="@(_SourceItemsToCopyToOutputDirectory);@(_SourceItemsToCopyToOutputDirectoryAlways)" />
      <_CopyFilesToBuildServer>
        <TargetPath>$(DeviceSpecificOutputPath)%(TargetPath)</TargetPath>
      </_CopyFilesToBuildServer>
    </ItemGroup>

    <CopyFilesToBuildServer SessionId="$(BuildSessionId)" Files="@(_CopyFilesToBuildServer)" />

  </Target>

(this is from the $(MSBuildExtensionsPath)Xamarin\Xamarin.Apple.Sdk.targets). 

As you can see, you can build up whatever target paths you need, and you could even leverage this built-in target by adding your like:

<Target Name="CopyAdditionalFiles" BeforeTargets="CopyFilesToMacOutputDirectory" Condition="'$(IsMacEnabled)' == 'true'">
    <ItemGroup>
      <_CopyFilesToBuildServer Include="@(OneLevelUp)">
        <TargetPath>..\%(Filename)%(Extension)</TargetPath>
      </_CopyFilesToBuildServer>
    </ItemGroup>
</Target>

As an example. @(OneLevelUp) would contain whatever needs to be copied to $(DeviceSpecificOutputPath)..\

Let me know if this works out for you.

Thanks!
Comment 2 Rich Zwaap 2017-05-17 04:11:23 UTC
Hi Daniel,

Thanks for the quick response.  In short, I'm not sure what you've suggested is going to work for us.  Please consider the following:

1.) The pre-15.2 file copy behavior was added by Xamarin directly as a result of an issue that we logged - see bug 38398.  Although the behavior may not have been documented, the previous interaction around this clearly demonstrates an expectation on our part for this to work, and cooperation from Xamarin to make that so.  The implication is clear that we would be relying on this behavior, and Xamarin offered no objection to this.  We would have naturally been open to alternative workflows at that time, but instead a fix was provided that established the pre-15.2 behavior.

2.) We have a released, cross-platform SDK that is currently in use by many of our customers (if you're curious, see here - https://developers.arcgis.com/net/).  This SDK naturally includes a Xamarin iOS component.  Because of this change in behavior, any customers that upgrade to Cycle 15.2 will find that they can no longer compile Xamarin iOS applications that reference our product.  Of course, an inability to compile applications caused by merely taking a reference to a third party component is about as serious of an issue as an API provider can encounter.

3.) The fact that our Xamarin iOS components cannot be used with the 15.2 release puts our users in an impossible position WRT supportability.  Xamarin requires that users "upgrade to the latest stable content to remain in a supported state" (see https://developer.xamarin.com/releases/current/#Current_Stable_Channel_Releases), but if users of our Xamarin iOS components do so, their applications will no longer build.

4.) Our release cadence is not as fast as Xamarin's.  So even if we were to alter how our Xamarin iOS product's build process works to fix this, and assuming our customers were able to take up the fix as soon as it was released (which is unrealistic, of course), there would still be a substantial population of users that are either stuck on 15.1 or left without our components in the meantime.

5.) The particular fix you've suggested certainly looks like it relies on build components (i.e. _CopyFilesToBuildServer) that are not "documented or supported explicitly."  It seems rather inconsistent that using undocumented capabilities would be the recommended course of action when doing so has been offered as a justification for the behavior change at issue.

The bottom line is that this change in file copy behavior is impacting our users severely.  Although the copy behavior is not documented, the fact that it was established in a fix for an issue that we had logged would certainly seem to imply that it could be relied upon.  What we really need is for that behavior to be restored.  Please reconsider.
Comment 3 Daniel Cazzulino 2017-05-17 13:46:27 UTC
Hi Rich, 
thanks for the thorough explanation of why this is important for you and your customers. Indeed, we should fix it and we will, ASAP.

I sort of regret now that we didn't spend a bit more time looking at the original proposal in the linked bug you submitted and ended up implementing a solution that is ad-hoc and not idiomatic to how the remote build typically works (we never copy source as-is to the mac, and we'll have to copy everything twice now :(). But I agree that we can't "fix it" by breaking your customers.

We'll have news on how you can get this fix soon.

Thanks in advance, and I'm truly sorry for the inconvenience this has caused. We should be able to resolve it quickly.

Thanks for you patience.

/kzu
Comment 4 xamarin-release-manager 2017-05-17 14:54:10 UTC
Fixed in version 4.6.0.95 (master)

Author: Adrian Alonso
Commit: fc391f150e0538096a5eea040f1fa36b26b348a3 (xamarin/XamarinVS)
Comment 5 Rich Zwaap 2017-05-18 02:06:43 UTC
Great, thanks for understanding and being accommodating.
Comment 6 vga 2017-05-18 13:49:54 UTC
This should be out in the next servicing release of VS 15.2.
Comment 7 Rich Zwaap 2017-05-18 15:14:37 UTC
That's great, thank you.  Do you have any guess as to how soon that might be?  Just so we can give our users an idea as to when they might expect a fix.
Comment 8 Rich Zwaap 2017-05-25 15:03:19 UTC
I've confirmed that this is fixed in the 15.2.2 update (VS extension version 4.5.0.475).  Thanks again.
Comment 9 Daniel Cazzulino 2017-05-26 22:12:07 UTC
Good to hear Rich!

Thanks again for you patience,

/kzu