Bug 29757 - Resource overriding with aapt is FUBAR
Summary: Resource overriding with aapt is FUBAR
Status: IN_PROGRESS
Alias: None
Product: Android
Classification: Xamarin
Component: MSBuild ()
Version: 5.1
Hardware: PC Mac OS
: Highest critical
Target Milestone: master
Assignee: dean.ellis
URL:
Depends on:
Blocks:
 
Reported: 2015-05-05 12:13 UTC by Jonathan Pryor
Modified: 2017-03-21 20:21 UTC (History)
7 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Example project with android library and app to show resource overriding (37.15 KB, application/octet-stream)
2015-05-11 07:27 UTC, Martin Kuckert
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report for Bug 29757 on Developer Community or GitHub if you have new information to add and do not yet see a matching new report.

If the latest results still closely match this report, you can use the original description:

  • Export the original title and description: Developer Community HTML or GitHub Markdown
  • Copy the title and description into the new report. Adjust them to be up-to-date if needed.
  • Add your new information.

In special cases on GitHub you might also want the comments: GitHub Markdown with public comments

Related Links:
Status:
IN_PROGRESS

Description Jonathan Pryor 2015-05-05 12:13:09 UTC
We have two conflicting requirements:

 1. Resources located within the App project MUST be able to override
    identically-named resource "sources" from Library projects.

    See Bug #29172 for repro.

 2. App resources should be checked for duplicates.

These don't sound conflicting, but they are, because aapt.

(2) was fixed in monodroid/290dad05 (test in monodroid/7d99748a) by changing the `aapt -S DIR` ordering so that the App resource dir was *last*, which allowed resources to be checked for duplicates (and generate an appropriate error).

The problem is that fixing (2) broke (1), because in order for (1) to work the App resource dir must be specified *first*, not last, and various other efforts to "square this circle" came up empty (e.g. Bug #29172 Comment #11).

The (hopeful) fix, then, is to (kinda/sorta) do what gradle does [0]:

> the new build system introduces a new merging mechanism that is run
> ahead of aapt and generates a single, merged, resources folder that is
> fed to aapt.

As it happens™, we *already* do some of this for our resource caching system.

Extend our resource caching infrastructure so that we do (kinda) as gradle does: merge ALL the Android resources, applying a "precedence" to each App or Library project, ensuring that App resources "override" identically-named Library resource files. This should allow us to continue supporting (1) while also supporting (2).

[0]: http://tools.android.com/tech-docs/new-build-system/resource-merging
Comment 1 Martin Kuckert 2015-05-11 07:27:25 UTC
This bug is breaking an existing and well-known logic of android: You can override library resources in your application. The previous version (Xamarin 3.9.547.0, Xamarin.Android 4.20.2.1) is behaving like that. I'll attach an example project. The current release is breaking this behavior, rendering a number of our libraries useless.
Comment 2 Martin Kuckert 2015-05-11 07:27:58 UTC
Created attachment 11149 [details]
Example project with android library and app to show resource overriding
Comment 3 Jonathan Pryor 2015-05-11 15:48:08 UTC
@mk: I think you're somewhat confused; *This* bug (Bug #29757) is about supporting both "normal" "Application resources override Library resources" behavior, *and* checking App resources for errors (duplicate names, etc.).

> This bug is breaking an existing and well-known logic of android

That breakage is in fact Bug #29172, which was fixed in Bug #29172 Comment #12, and will be in a forthcoming beta release.
Comment 4 Martin Kuckert 2015-05-12 02:40:55 UTC
Ah, i see. I'm not authorized to access bug #29172, so I wasn't aware of that situation. Thank you anyway.
Comment 5 Rodja Trappe 2015-05-20 23:56:24 UTC
Can the gradle-like merging be implemented and released in a hart beat? Otherwise I strongly suggest to roll back the conflicting requirement "App resources should be checked for duplicates" in the mean time. With the current monodroid version it's not possible to build apps as expected and hence can't be used for public app releases.
Comment 6 Jonathan Pryor 2015-05-21 13:33:07 UTC
@Rodja:
> I strongly suggest to roll back the conflicting requirement
> "App resources should be checked for duplicates" in the mean time...

This was in fact done, and is part of the 5.1.2 release (currently in Beta, I believe).
Comment 7 cfraz89 2015-05-25 21:53:39 UTC
Did the rollback in 5.1.2 supposedly fix the issue? 5.1.2 still seems to have many issues with resource compilation (Many broken builds with null references, and missing resources)
Comment 8 Jonathan Pryor 2015-05-26 17:04:44 UTC
@cfraz89:
> Did the rollback in 5.1.2 supposedly fix the issue?

It *should* have, at least for the testing I'm aware of...

> 5.1.2 still seems to have many issues with resource compilation
> (Many broken builds with null references, and missing resources)

Please file these as additional bugs and please include the stack traces of the errors. You may need to generate diagnostic build output in order to obtain the stack traces.

Thanks!
 - Jon
Comment 9 Jonathan Pryor 2015-08-17 15:29:29 UTC
Fixed in monodroid/5030043b.
Comment 10 Arpit Jha 2015-10-09 02:25:13 UTC
@JonP
I have checked this issue with  mono-android-6.0.0-9_79b448ed03b67e956ab7904a3a43c05421c88c55 and observed that issue is still exist with the help of attached projecthttps://bugzilla.xamarin.com/show_bug.cgi?id=29172#c1
Screencast:  http://www.screencast.com/t/pqzWcACe
Build Error of Bug 29172 : https://gist.github.com/Arpit360/efe463c8c9f439f69107

I am not getting issue using attached sample in https://bugzilla.xamarin.com/show_bug.cgi?id=29757#c2  and observed that I am able to build and deploy application successfully.

Screencast: http://www.screencast.com/t/wvVGyJse

Please let me know if I have to check anything else

Environment Info:
=== Xamarin Studio ===

Version 5.10 (build 813)
Installation UUID: 3d25a767-a003-4a7d-9f5e-e57987cf6cf0
Runtime:
	Mono 4.2.1 (explicit/29c1622)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 402010067

=== Xamarin.Profiler ===

Version: 0.23.35.0
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Apple Developer Tools ===

Xcode 7.0 (8227)
Build 7A220

=== Xamarin.iOS ===

Version: 9.2.0.85 (Business Edition)
Hash: 7bcf0da
Branch: master
Build date: 2015-10-05 17:54:02-0400

=== Xamarin.Android ===

Version: 6.0.0.9 (Business Edition)
Android SDK: /Users/mac360_xamarin/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		2.3    (API level 10)
		4.0.3  (API level 15)
		4.1    (API level 16)
		4.4    (API level 19)
		4.4.87 (API level 20)
		5.0    (API level 21)
		5.1    (API level 22)
		6.0    (API level 23)

SDK Tools Version: 24.4
SDK Platform Tools Version: 23.0.1
SDK Build Tools Version: 23

Java SDK: /usr
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

=== Xamarin Android Player ===

Version: 0.6.1
Location: /Applications/Xamarin Android Player.app

=== Xamarin.Mac ===

Version: 2.4.0.79 (Business Edition)

=== Build Information ===

Release ID: 510000813
Git revision: 388e41428fb8f2910833c87fec0e7edaccd4f436
Build date: 2015-10-05 12:26:38-04
Xamarin addins: f21b254b56b36e417daee6da8b0300076f286ef1
Build lane: monodevelop-lion-cycle6

=== Operating System ===

Mac OS X 10.10.5
Darwin mac360-xamarins-Mac-mini.local 14.5.0 Darwin Kernel Version 14.5.0
    Wed Jul 29 02:26:53 PDT 2015
    root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64
Comment 11 Jonathan Pryor 2015-10-09 09:39:39 UTC
@Arpit: The fix was reverted for cycle6 because the fix had introduced too many other bugs.

We'll try to get this working for cycle7.
Comment 12 Martin Kuckert 2015-10-09 09:59:16 UTC
@JonP: Are you saying that Xamarin is constantly changing the behavior of their tools? And each developer should follow your zigzag strategy?

We're using a lot of libraries internally which provide shared resources. Each app is overriding some or all of that resources with customized artwork, strings or layouts. The tool chain and as a result the whole Xamarin ecosystem is totally broken, if one version is behaving correctly and the other is not. And with behaving correctly i mean following the behavior of native Android tools, http://tools.android.com/tech-docs/new-build-system/resource-merging
The tools are expected to replace the resources of a library, if the application provides own assets. It's expected because that's the typical android way to do that.

What is Xamarins strategy about this behavioral changes of core components? You can't break this core principals. You're costing us a lot of many in terms of license costs and are adding a lot of many in terms of wasted developer time following those superfluous changes.

This revert will mean for us to stick with the current version (pre-cycle6) and wait for getting the fix re-applied - with all negative consequences as no binding for the latest OS SDKs etc.
Comment 13 Jonathan Pryor 2015-10-09 11:39:49 UTC
> Are you saying that Xamarin is constantly changing the behavior of their tools?

Within reason?

> And each developer should follow your zigzag strategy?

Zigzag?

We're trying to fix bugs, which may necessitate changing behavior, while at the same time preserving existing code. Sometimes we're good at fixing bugs without breaking things, sometimes not, and sometimes it takes awhile to even know which is happening.

In the case of THIS bug, behavior in cycle6 (6.0, after 6.0.0.4) will be identical to that of cycle5 (5.1.6, the current stable). If you're on the alpha, sorry for the zig-zagging, but if you're on stable, you shouldn't be observing any zig zags. You shouldn't see any change here at all.

> And with behaving correctly i mean following the behavior of native Android
> tools, http://tools.android.com/tech-docs/new-build-system/resource-merging
> The tools are expected to replace the resources of a library, if the
> application provides own assets. 

That should be the case in cycle5, and should continue to be the case in cycle6.

What won't be checked, what isn't fixed, what THIS bug is about, is if the APP project contains *duplicate* resources, e.g. *two* "app_name" string resources, THAT WILL NOT BE AN ERROR. (It should be.) It will instead work, which is wrong. (I have no idea which copy will actually be used, which is why this behavior is wrong.)

> This revert will mean for us to stick with the current version (pre-cycle6)

Huh? Cycle6 behavior should now be the same as Cycle5 behavior. Meaning you shouldn't be observing ANY behavioral change, unless you were using one of the cycle6 5.2 alpha releases (which, again, were never stable).

The whole reason the fix from Comment #9 was reverted is because the fix broke too much during testing! The fix was never released to stable. The fix, thus, is in-progress. (With luck, we can get it sorted out for cycle7.)

Is this not what you want? A stable, consistent, experience? Not breaking your projects?

Is this not what we're doing (if you stick to stable)?

Now, if you're on alpha -- and thank you if you are! -- then you need to realize that alpha releases do NOT get full QA passes (that's what makes them alphas!), and thus may inadvertently break things without our realizing it. When this happens, please file a bug so that we can fix or revert before it hits stable, but Alpha is known to be unstable -- that's the *point* -- so I'm not sure I buy your zigzag argument there.
Comment 14 Martin Kuckert 2015-10-19 02:48:29 UTC
OK, I see. Seems like I've misunderstood the "revert"-part around cycle 6.

If the latest release behaves correctly (resources in an App override those of a Library) and all future versions of the Xamarin Android platform will behave the same, than please forget my comment and my kind of unfriendly word choice.