Bug 24287 - Including satellite assemblies for translations does not work
Summary: Including satellite assemblies for translations does not work
Status: RESOLVED INVALID
Alias: None
Product: Android
Classification: Xamarin
Component: MSBuild ()
Version: 4.18.0
Hardware: PC Mac OS
: Normal normal
Target Milestone: ---
Assignee: Jonathan Pryor
URL:
Depends on:
Blocks:
 
Reported: 2014-11-05 13:11 UTC by John Miller [MSFT]
Modified: 2017-08-08 17:16 UTC (History)
7 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Test Case (26.21 KB, application/zip)
2014-11-06 14:35 UTC, John Miller [MSFT]
Details
Incomplete test case: includes "broken" console app (34.96 KB, application/zip)
2014-11-20 15:52 UTC, Brendan Zagaeski (Xamarin Team, assistant)
Details
Fixed sample with japanese + resource translated below folder (51.89 KB, application/x-zip-compressed)
2014-11-21 01:49 UTC, floriang
Details
batch to add satellite assemblies + sign apk (761.46 KB, application/x-zip-compressed)
2014-11-21 04:20 UTC, floriang
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED INVALID

Description John Miller [MSFT] 2014-11-05 13:11:03 UTC
**Overview:**

   When including compiled assemblies (like from Alchemy Catalyst) in an Android project, the satellite assemblies are not copied into the APK so translations do not work. 

**Steps to Reproduce:**

   Try including a dll resource for translations into a project, in a subfolder like fr-FR. 

**Additional Information:**

   Discussed with Jon P. 

The MSBuild system will need to be investigated to see how it handles satellite assemblies, and use the appropriate build action/etc. to ensure that the assemblies are included in the app.

The only way I've seen/tested satellite assemblies is as .resx/etc. files *within* the build system. The use of actual assemblies is not something seen before.

The current workaround would be to directly include the .resx files in the Android project and let our build system create the satellite assemblies.
Comment 1 John Miller [MSFT] 2014-11-06 14:35:36 UTC
Created attachment 8644 [details]
Test Case
Comment 2 Parmendra Kumar 2014-11-07 09:26:53 UTC
I have checked this issue, I have Observed that the translator is not working.
The Output window show error to loading .dll file.
Could you Please check the Screencast and Output Log and Let me know If I have missed anything?

Screeencast:http://www.screencast.com/t/RAoaHxuIytZP
Application Output: https://gist.github.com/Parmendrak/ecec8f31337b20376745

Environment Info:
=== Xamarin Studio ===

Version 5.5.3 (build 6)
Installation UUID: 1a096c6f-0678-402e-89b2-a2c10f7e80e4
Runtime:
	Mono 3.10.0 ((detached/e204655)
	GTK+ 2.24.23 (Raleigh theme)

	Package version: 310000023

=== Apple Developer Tools ===

Xcode 6.1 (6602)
Build 6A1052c

=== Xamarin.iOS ===

Version: 8.4.0.16 (Business Edition)
Hash: 80e9ff7
Branch: 
Build date: 2014-10-22 15:09:12-0400

=== Xamarin.Mac ===

Version: 1.10.0.18 (Business Edition)

=== Xamarin.Android ===

Version: 4.18.1 (Business Edition)
Android SDK: /Users/360_macmini/Desktop/android-sdk-macosx
	Supported Android versions:
		2.1    (API level 7)
		2.2    (API level 8)
		2.3    (API level 10)
		3.1    (API level 12)
		3.2    (API level 13)
		4.0    (API level 14)
		4.0.3  (API level 15)
		4.1    (API level 16)
		4.2    (API level 17)
		4.3    (API level 18)
		4.4    (API level 19)
		4.4.87 (API level 20)
Java SDK: /usr
java version "1.7.0_65"
Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

=== Build Information ===

Release ID: 505030006
Git revision: fbe3e9453daf6a3bb9a9709ed22bec35f7c9056b
Build date: 2014-10-23 13:08:38-04
Xamarin addins: e44add2b39de4dd57c0742bb2e620dfad84c64c6

=== Operating System ===

Mac OS X 10.9.4
Darwin ShrutiMac.local 13.3.0 Darwin Kernel Version 13.3.0
    Tue Jun  3 21:27:35 PDT 2014
    root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64
Comment 4 floriang 2014-11-18 21:41:48 UTC
Hello, our development is blocked by this bug.
What is the status of this item?
Do you know when it will be fixed?
On our side we need a fix until next week.
Tell me if we can help you work faster on this.
Thank you.
Comment 5 Cody Beyer (MSFT) 2014-11-18 22:13:08 UTC
It appears that it was recreated, so I am confirming this
Comment 6 Brendan Zagaeski (Xamarin Team, assistant) 2014-11-20 15:52:15 UTC
Created attachment 8823 [details]
Incomplete test case: includes "broken" console app

I did some further investigation on this bug today, and I do not yet understand the correct behavior for Windows C# apps.

This attachment modifies the original test app to include a desktop C# console app. The console app requests the value of the "MY_STRING" resource for the "fr-fr" culture and displays the result. *But* the app currently always returns the non-localized string. If I add an additional `strings.fr-FR.resx` file that includes a "MY_STRING" resource, then the app correctly displays the localized string from that new file.


## Steps to reproduce

Build and run the "ConsoleApplication1" project in Visual Studio on Windows.


## Result

The program outputs the following line:
> MyResources string in English.


## Expected result

The program outputs a different string stored in MyResources.resources.dll:
> C'est une string en Français



## Steps to produce _localized_ output

1. Change the Build Action of the `strings.fr-FR.resx` file from "None" to "Embedded Resource".

2. (Optional) Change the Build Action of `MyResources.resources.dll` from "Content" to "None".

3. Build and run the "ConsoleApplication1" project.


## Results

The program outputs the following line:
> Bonjour.


## Version information

Microsoft Visual Studio Professional 2013
Version 12.0.30723.00 Update 3
Microsoft .NET Framework
Version 4.5.51641

Xamarin   3.8.145.0 (bd95bb97b6f8c7dfd6486b6ae3d967149022c95e)
Xamarin.Android   4.20.0.26 (4f8f6db2eeda44dab7d4c42eb381725b3035662d)
Xamarin.iOS   8.4.0.0 (84277d4b8fb363af32626a5263977f03227a1fdf)
Comment 9 floriang 2014-11-21 00:04:10 UTC
PS: interestingly, when you extract the resource file from 
MyResources.resources.dll
You will get 
MyResources.strings.resw under fr/ directory:

fr/MyResources.strings.resw

This corresponds to the fixed namespace.
I don't know how this DLL was generated in the first place, but it doesn't seem to include the "Folder" hierarchy.

Florian
Comment 10 floriang 2014-11-21 00:21:23 UTC
PPS:
By doing this change, I was able to display the translated string in french on android, in release build.
So it looks like it's starting to work.
The question here is why the translated assembly has the wrong namespace, and why it doesn't work with our real application.
We will investigate more, but do you think that there is an issue in Xamarin.Android now? (on my side I think it's likely to be an issue with the namespaces)

Florian
Comment 11 floriang 2014-11-21 01:30:10 UTC
Hello again,
At the moment I am still not able to make translation work in our application release mode.
I verified our assemblies seem to have the correct path components.
I will try to generate the proper assembly for the sample (with Folder path component included), to see if it works.
Florian
Comment 12 floriang 2014-11-21 01:49:26 UTC
Created attachment 8826 [details]
Fixed sample with japanese + resource translated below folder
Comment 13 floriang 2014-11-21 01:53:02 UTC
Comment on attachment 8826 [details]
Fixed sample with japanese + resource translated below folder

By re-generating properly the satellite assembly for Japanese, I am able to display it properly using the sample attached (french was not re-generated so culture fr won't work, only culture ja-JP).
Then, using the language settings of android, if current culture is set to japanese the assembly is used properly.

At the moment I think the issue comes our translated assemblies.
For example I noticed that they were using culture ja, not ja-JP. I'm not sure but it looks like an issue to me.
What do you think?
Comment 14 floriang 2014-11-21 03:02:00 UTC
Now I could confirm that by adding a properly translated DLL inside our project (with build action=None, copy to output folder: if newer), I am able to display translation properly.
Then I wanted to try batch adding those satellite assemblies by hand after the build using android package tools instead of having them inside our solution, but at the moment I didn't success.
I keep you posted.
Comment 15 floriang 2014-11-21 04:20:22 UTC
Created attachment 8827 [details]
batch to add satellite assemblies + sign apk

By removing the ja-JP folder containing the satellite assembly from the project, and using aapt to add the assembly by hand, I can not get the translation string!
What am I missing?
We have a lot of DLLs to translate, and it would be much more convenient for us to manage them all outside our main solution instead of spreading them in all projects.
Can you check the way I include the assembly in the apk, and tell me if anything is wrong?
THank you.
Comment 16 Jonathan Pryor 2014-11-21 15:47:47 UTC
I do not understand the actual problem, nor how to repro the problem, nor what the problem even looks like.

What's the problem?

@Parmendra: This message is *not* a problem:

> [Mono] AOT module '/storage/emulated/0/Android/data/I18nResourcesBug.I18nResourcesBug/files/.__override__/MyResources.dll.so' not found: dlopen failed: library "/data/data/I18nResourcesBug.I18nResourcesBug/lib//storage/emulated/0/Android/data/I18nResourcesBug.I18nResourcesBug/files/.__override__/MyResources.dll.so" not found

If you see AOT in the message, IGNORE IT. AOT is not currently supported in Xamarin.Android.

This is also a warning, not an error; were it an error, it would actually say "error", possibly followed by a process exit. Neither of which happens here.
Comment 18 Jonathan Pryor 2014-11-21 16:16:50 UTC
As I understand things, the original problem was that the application was using satellite assemblies to store translations without using the "normal" .resx mechanism.

Aside: To find satellite assemblies, we use the <ResolveAssemblyReference/> MSBuild task with FindSatellites="True" and read the SatelliteFiles output parameter. This in turn *requires* that satellite assemblies be findable via <ResolveAssemblyReference/>...which doesn't actually document *how* it does things.

http://msdn.microsoft.com/en-us/library/9ad3f294.aspx

What is the normal .resx mechanism?

Consider an app uses a Build action of EmbeddedResource for a .resx file, as is done in Attachment #3808 [details] in Bug #5037.

When the project is built, the @(EmbeddedResource) files are processed by the <GenerateResources/> task (in Microsoft.Common.targets), which are used to fill the @(ManifestResourceWithNoCulture) and @(ManifestResourceWithCulture) groups. These groups are then used by the GenerateSatelliteAssemblies target to generate the .resources.dll file.

For example:

    MyResources/strings.fr-CA.resx ->
    obj/Release/MyResources.strings.fr-CA.resources ->
    obj/Release/fr-CA/MyResources.resources.dll

When packaging the project, the <ResolveAssemblyReference/> task is used to resolve all referenced assemblies, and thus should pull in the generated satellite assemblies. This in turn allows the satellite assemblies to be embedded into the .apk, into the appropriate location, because (as part of assembly resolution) the satellite assemblies were (indirectly) referenced by assemblies within @(ReferencePath) (set by the ResolveAssemblyReferences target in Microsoft.Common.targets).
Comment 19 Jonathan Pryor 2014-11-21 16:20:06 UTC
If you don't want to use .resx files, and instead want to include separately built resource/satellite assemblies, you need to "fit in" with MSBuild; it needs to be done such that @(ReferencePath), @(ResolvedUserAssemblies), and <ResolveAssemblyReference/> in concert are able to find them.

(@(ResolvedUserAssemblies) is a Xamarin item group, which comes from taking all assemblies in @(ReferencePath), plus the project's output file, and scanning them *all* with Cecil to find the complete closure of all referenced assemblies. @(ResolvedUserAssemblies) are assemblies in this set that aren't from the Xamarin.Android SDK.)
Comment 20 Jonathan Pryor 2014-11-21 16:49:05 UTC
Comment #18 and Comment #18 are what I *think* is being referred to here. Unfortunately, I'm not sure I can confirm or deny anything.

Consider Attachment #8826 [details] and the ResxDllsAsContent.zip!MyResources/MyResources.csproj project file. It appears to mention a satellite assembly, but it uses a Build action of @(None):

  <ItemGroup>
    <None Include="ja-JP\MyResources.resources.dll">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
    <Content Include="MyResources.resources.dll" />
  </ItemGroup>

Note: @(Content) is NOT SUPPORTED in Xamarin.Android. Do not use it.

Unsurprisingly, if we build the ResxDllsAsContent.zip!I18nResourcesBug/I18nResourcesBug.csproj project, there is no assemblies/ja-JP/MyResources.resources.dll entry:

    $ unzip -l bin/Release/*-Signed.apk | grep ja-JP
    # No matches.

However, if we change the Build action on ja-JP\MyResources.resources.dll to something that will cause it to be found by <ResolveAssemblyReference/>, like @(ManifestResourceWithCulture):

  <ItemGroup>
    <ManifestResourceWithCulture Include="ja-JP\MyResources.resources.dll">
      <Culture>ja-JP</Culture>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </ManifestResourceWithCulture>
  </ItemGroup>

THEN it IS included:

    $ unzip -l bin/Release/*-Signed.apk | grep ja-JP
         6144  11-21-14 16:46   assemblies/ja-JP/MyResources.resources.dll
Comment 21 floriang 2014-11-24 21:29:45 UTC
Hi Jon,
I don't understand how you managed to build with the tag "ManifestResourceWithCulture". When I try, I get the following error:

1>C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(2800,5): warning MSB9006: ManifestResourceWithCulture item type is deprecated. Emit EmbeddedResource items instead, with metadata WithCulture='true', Type='Resx', and optional LogicalName.
1>ALINK : error AL1019: アセンブリを作成中にメタデータが失敗しました -- 指定されたパスが見つかりません。

Can you upload a sample project?
Comment 22 Jonathan Pryor 2014-11-24 22:51:18 UTC
I built on OS X/xbuild, not Windows/MSBuild, so it may be an xbuild bug/"feature".

Unfortunately, I don't think @(EmbeddedResource) will work for you, because that will take a file and embed into the $(OutputPath)\$(AssemblyName).dll assembly, not add as a new assembly.

More exploration would be necessary. Hopefully Comment #18, Comment #19, and Comment #20 can provide enough information to point you in the right direction.

You should also read the Microsoft.Common.targets file to see how things are converted.
Comment 23 floriang 2014-11-24 23:05:27 UTC
Hello Jonathan,
Ok, I tried many things and I don't think it's going to work with VS2012.
Then, I still don't understand why the assemblies are ignored when I add them using the android aapt tools as I did in the batch of my previous attachment.
In this case I see that they are inside the resulting apk, at the proper path.
Do you have an idea?
Comment 24 Jonathan Pryor 2014-11-24 23:31:36 UTC
@floriang; when adding assemblies to the .apk, they MUST be added as UNCOMPRESSED a files (`aapt -0`), otherwise they can't be loaded. 

Don't firget to zipalign the .apk after signing.
Comment 25 floriang 2014-11-25 00:11:06 UTC
Hi Jonathan,
>> when adding assemblies to the .apk, they MUST be added as
>> UNCOMPRESSED a files (`aapt -0`), otherwise they can't be loaded. 
I think this was the issue!!!
It seems to be working now with the sample project.
I will try with our application and give you a feedback, but I'm fairly confident that it is solved now.
Comment 26 floriang 2014-11-25 00:24:27 UTC
>> I will try with our application and give you a feedback, but I'm fairly
>> confident that it is solved now.
Yep, it works!
So people should be careful not to compress the satellite assemblies if they add them to the apk afterwards in the build process.
Thank you very much to all of you for your support!
Comment 27 Jon Douglas [MSFT] 2017-08-08 17:16:37 UTC
Thank you for taking the time to submit this report. After reviewing the description of this bug, we believe it no longer affects the current version of Xamarin.Android. If you are still experiencing the issue after updating your packages, please reopen this report with an attached reproduction.