Bug 57087 - Xamarin.Android "Clean" target does not remove old .mdb files for library projects after Xamarin 15.2 release (Mono 5.0), leading to out-of-date debugging symbols for Fast Deployment mode
Summary: Xamarin.Android "Clean" target does not remove old .mdb files for library pro...
Status: RESOLVED FIXED
Alias: None
Product: Android
Classification: Xamarin
Component: MSBuild ()
Version: 7.3 (15.2)
Hardware: PC All
: --- normal
Target Milestone: 15.3
Assignee: dean.ellis
URL:
Depends on:
Blocks:
 
Reported: 2017-06-02 03:40 UTC by Brendan Zagaeski (Xamarin Team, assistant)
Modified: 2017-06-06 14:20 UTC (History)
1 user (show)

Tags:
Is this bug a regression?: Yes
Last known good build: Xamarin 15.1 release


Attachments
Test case (11.90 KB, application/zip)
2017-06-02 03:40 UTC, Brendan Zagaeski (Xamarin Team, assistant)
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Brendan Zagaeski (Xamarin Team, assistant) 2017-06-02 03:40:07 UTC
Created attachment 22636 [details]
Test case

Xamarin.Android "Clean" target does not remove old .mdb files for library projects after Xamarin 15.2 release (Mono 5.0), leading to out-of-date debugging symbols for Fast Deployment mode

The noticeable usage problem is that Fast Deployment mode also preferentially deploys only the outdated .mdb files rather than the freshly generated portable .pdb files.  This leads to out-of-date debugging symbols (and nonfunctional breakpoints) in that configuration.  The stale .mdb files also get packaged into the .apk when Fast Deployment is disabled, but at least in that case it seems that the portable .pdb files are _also_ packaged, and the debugger (maybe?) preferentially picks those over the .mdb files.  If the Xamarin.Android team likes, other little adjustments for the Fast Deployment behavior or the general inclusion of redundant .mdb files can also be made, but this bug can stay focused on any changes for the "Clean" target.  Separate quick bug reports can be filed to track changes to those other behaviors.  If in the end the team decides that those other changes will supersede any changes to the Clean target, then this bug can be marked as a duplicate of one of those other bug reports.  Thanks!




## Workaround

Manually delete the "bin" and "obj" folders for any projects that have lingering .mdb files.




## Example "real world" usage scenario to replicate on Mac


1. Set the active Mono version to 4.8.1 and the active Xamarin.Android version to 7.2.0-7.


2. Build the attached test case in the Debug configuration:

xbuild /t:Build /p:Configuration=Debug BlankAppAndroid1.sln


3. Switch the active Mono version to 5.0 or higher and the active Xamarin.Android version to 7.3 or higher.


4. Clean the test case in the Debug configuration:
msbuild /t:Clean /p:Configuration=Debug BlankAppAndroid1.sln




## BAD Results


- Notice that PortableClassLibrary1/bin/Debug/PortableClassLibrary1.dll.mdb was not deleted by the Clean target at step 4.


- If you now build the project again, you will notice that the old .mdb file gets re-copied into the output folder for the application project.

> $ find . -name "*.mdb" | while read FILE; do ls -l "$FILE"; done 
> -rw-r--r--  1 macuser  wheel  621 Jun  1 22:35 ./BlankAppAndroid1/bin/Debug/PortableClassLibrary1.dll.mdb
> -rw-r--r--  1 macuser  wheel  621 Jun  1 22:29 ./BlankAppAndroid1/obj/Debug/android/assets/PortableClassLibrary1.dll.mdb
> -rw-r--r--  1 macuser  wheel  1084 Jun  1 22:29 ./BlankAppAndroid1/obj/Debug/BlankAppAndroid1.dll.mdb
> -rw-r--r--  1 macuser  wheel  621 Jun  1 22:35 ./BlankAppAndroid1/obj/Debug/linksrc/PortableClassLibrary1.dll.mdb
> -rw-r--r--  1 macuser  wheel  621 Jun  1 22:29 ./PortableClassLibrary1/bin/Debug/PortableClassLibrary1.dll.mdb
> -rw-r--r--  1 macuser  wheel  621 Jun  1 22:29 ./PortableClassLibrary1/obj/Debug/PortableClassLibrary1.dll.mdb

- This also leads to out-of-date debugging symbols when using the Fast Deployment mode.  For example, I tried inserting a new line `var w = 10;` at line 9 in MyClass.cs, setting a breakpoint on line 10, and then debugging the app in Visual Studio for Mac.  The breakpoint was not hit.


- Only the old .mdb debugging symbols are deployed when using Fast Deployment mode.  One interesting excerpt from the diagnostic MSBuild output might be from the "GetFilesThatExist" task:

>     [Output] FilesThatExist (TaskId:154)
>       /Users/Shared/Projects/BlankAppAndroid1/PortableClassLibrary1/bin/Debug/PortableClassLibrary1.dll.mdb (TaskId:154)
>   Output Item(s): _ResolvedMdbFiles=/Users/Shared/Projects/BlankAppAndroid1/PortableClassLibrary1/bin/Debug/PortableClassLibrary1.dll.mdb (TaskId:154)

- You can also see that only the old .mdb debugging symbols are being deployed by looking in the .__override__ directory:

> $ adb shell ls /sdcard/Android/data/com.example.blankappandroid1/files/.__override__/
> BlankAppAndroid1.dll
> BlankAppAndroid1.pdb
> PortableClassLibrary1.dll
> PortableClassLibrary1.dll.mdb
> dexes
> lib
> resources



## Additional example real-world scenario usage scenario on Windows

On Windows, if you build the library dependency and then switch the DebugType from `full` to `portable` (all using the Xamarin 15.2 release), you will hit the same out-of-date symbols problem because the `full` debug symbols were converted to .mdb files that will never get cleaned up automatically after the switch the `portable` symbols.

(Depending on the fix for the Mac scenario, this alternate non-upgrade scenario might warrant an additional or slightly different fix.  In that case, a follow-up bug report can be filed to track the secondary fixes.  Thanks!)




## Additional environment info (brief)

Android SDK Tools Version: 25.2.5
Android SDK Platform Tools Version: 25.0.3
Android SDK Build Tools Version: 25

Java JDK 8u65 (1.8.0_65), 64-bit

macOS 10.12.4



### Devices

Google Android API 23 emulator (x86)
Comment 2 dean.ellis 2017-06-02 11:16:20 UTC
Hmm..

We don't control the PCL targets, so we cannot clean the .mdb file in that directory.
That is part of the msbuild/runtime system. 

We might be able to add code to only use .mdb if a portable pdb does not exist
Comment 3 Brendan Zagaeski (Xamarin Team, assistant) 2017-06-02 14:23:12 UTC
> We don't control the PCL targets

Just to double-check quickly, maybe it could be sufficient to delete the .mdb files from the library projects when the Android app project is cleaned?  That might be roughly "symmetrical" with how those files originally get generated?  But no worries if that doesn't actually make sense or help in this scenario.
Comment 4 dean.ellis 2017-06-02 15:21:30 UTC
MSBuild will prevent us from messing with files which are outside of the current "project" directories. So even if we wrote the code, I don't believe msbuild will let us delete those files. This would need to be handled by the mono PCL .targets file. 

As a work around I uploaded https://github.com/xamarin/xamarin-android/pull/627

The idea being we favour ppdb over mdb files. So if we have a ppdb we will use that and NOT copy the .mdb over into the app directory.
Comment 5 dean.ellis 2017-06-02 15:24:38 UTC
Plus Clean does not call ResolveAssemblies, so we have not idea what the dependencies of the project are at the point the Clean target is called.
Comment 6 dean.ellis 2017-06-02 15:25:56 UTC
I will see if I can do this from our targets though.. there might just be a nice entry point I can hook into.
Comment 7 dean.ellis 2017-06-06 14:20:53 UTC
The copying of the .mdb files has been fixed in 

xamarin-android/master/2f89b364

I'm going to mark this as fixed. 
I would suggest if we want the PCL targets to clean up old mob files we raise a new bug for the runtime team to deal with that issue.