Difference between revisions of "Build DMD for Android"

From D Wiki
Jump to: navigation, search
(Note updated age and add warning)
 
(19 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This is a work in progress: Android/x86 is mostly done, going to work on Android/ARM nextAlmost all the druntime/phobos unit tests pass on Android/x86, plus a sample C/OpenGLES 1.0 purely native app from the NDK has been ported to D.
+
Note: these patches and instructions have not been updated for a couple years, as all work has moved to [[Build LDC for Android|ldc and Android/ARM, which passes the entire D test suite]]As such, these instructions are somewhat outdated and are only for reference.
 +
 
 +
All druntime and phobos tests from 2.070 git master pass on Android/x86, I'll try the dmd test suite someday.  A C/OpenGLES 1.0 purely native sample app from the NDK has been ported to D.  I list work that still needs to be done at the end.
  
 
==Prerequisites==
 
==Prerequisites==
  
* 32-bit linux/x86 host on which to build dmd
+
* linux host on which to build dmd
** A virtual machine like VirtualBox/VMware will work fine, but you should have at least 512 MB of memory allocated and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
+
** A virtual machine like VirtualBox/VMware will do fine, but you should have at least 512 MB of memory allocated and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
 
* C++ compiler and toolchain, to build dmd
 
* C++ compiler and toolchain, to build dmd
 +
* A pre-built D compiler for linux, needed because the D frontend has been translated to D.
 
* dmd/druntime/phobos source
 
* dmd/druntime/phobos source
** Get druntime and phobos from git, as they have Android-specific changes that are not in the release branches yet.
+
** Get the source from git, as the following Android patches have only been tested on the master branch of each repo.
* Android native toolchain, [http://developer.android.com/tools/sdk/ndk/index.html the 32-bit NDK] and [http://developer.android.com/sdk/index.html SDK]
+
* Android native toolchain, [http://developer.android.com/tools/sdk/ndk/index.html the NDK] and optionally [http://developer.android.com/sdk/index.html the SDK]
** The "SDK Tools only" version of the SDK is enough, if you don't plan on using their IDE integration.  I will only write about using the command-line tools.  The SDK requires JDK 6 and Ant 1.8 or later, follow their instructions to make sure it's installed right.
+
** You don't need the SDK if you only want to build command-line binaries, the SDK is only needed to package and sign GUI apps.  The "SDK Tools only" version of the SDK is enough, if you don't plan on using their IDE integration.  I will only write about using the command-line tools.  The SDK requires JDK 7: follow their instructions to make sure it's installed right.
 
* Android/x86, whether a device or VM
 
* Android/x86, whether a device or VM
** The SDK comes with an Android/x86 emulator and Intel puts out one of their own.  I use [http://www.android-x86.org/download the builds put out by the android-x86 project] and recommend the deprecated 4.3 build, as it's the least buggy one I've dealt with.  I run the iso in a VM with 512 MBs of memory without installing to disk.
+
** The SDK comes with an emulator and Intel puts out their own.  I use [http://www.android-x86.org/download the builds put out by the android-x86 project] and recommend the deprecated 4.3 build, as it's the least buggy one I've dealt with.  I run the iso in a VM with 512 MBs of memory without installing to disk.
  
 
===To run Android/x86 in a VM===
 
===To run Android/x86 in a VM===
  
Download the Android-x86 iso and install as normal: it'll put you through a configuration process, but you can skip every step.  Go to Settings->Security and disable the Verify Apps option, or it'll ask you about that every time you install a test app.  Alt-F1 will take you to a root shell, which you'll want to do any time you're not using Android/x86 for a couple minutes or the screen will eventually go black and the VM will freeze up.  Alt-F7 takes you back to the normal GUI screen.
+
Download the Android-x86 iso and install as normal.  It'll put you through a configuration process, but you can skip every step.  Go to Settings->Security and disable the Verify Apps option, or it'll ask you about that every time you install a test app.  Alt-F1 will take you to a root shell, which you'll want to do any time you're not using Android/x86 for a couple minutes, or the screen will eventually go black and the VM will freeze up.  Alt-F7 takes you back to the normal GUI screen.
  
 
==Build D for Android/x86==
 
==Build D for Android/x86==
  
Download [http://164.138.25.188/dmd/packed_tls_for_elf.patch the patch for dmd], apply it, and [http://wiki.dlang.org/Building_DMD build normally]:
+
Download [https://patch-diff.githubusercontent.com/raw/D-Programming-Language/dmd/pull/3643.patch the patch for dmd], apply it, and [[Building DMD|build normally]]:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
 
cd dmd
 
cd dmd
git apply packed_tls_for_elf.patch
+
git apply 3643.patch
 
make -f posix.mak -j5
 
make -f posix.mak -j5
 
</syntaxhighlight>
 
</syntaxhighlight>
  
This patch adds TLS support for Android/x86, more info can be found on [https://github.com/D-Programming-Language/dmd/pull/3643 the pull request].
+
This patch adds emulated TLS for Android/x86, more info can be found on [https://github.com/D-Programming-Language/dmd/pull/3643 the pull request].
  
Assuming druntime and phobos are in the same directory as dmd, download and apply [http://164.138.25.188/dmd/druntime_build.patch the patch for druntime] and [http://164.138.25.188/dmd/phobos_build.patch the patch for phobos], set the NDK environment variable to the path of wherever you installed the NDK, and build each in turn with PIC enabled:
+
Assuming druntime and phobos are in the same directory as dmd, download and apply [https://gist.github.com/joakim-noah/f2f22dfcd2ef30880f14 the patch for druntime] and [https://gist.github.com/joakim-noah/5d399fdcd5e484d6aaa2 the patch for phobos], then set the NDK environment variable to the path of wherever you installed the NDK:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
 
cd ../druntime
 
cd ../druntime
 
export NDK=/path/to/your/android-ndk-r10
 
export NDK=/path/to/your/android-ndk-r10
git apply druntime_build.patch
+
git apply druntime_android
make -f posix.mak PIC=1
+
make -f posix.mak
  
 
cd ../phobos
 
cd ../phobos
git apply phobos_build.patch
+
git apply phobos_android
make -f posix.mak PIC=1
+
make -f posix.mak
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Both patches avoid building druntime/phobos as a shared library and set the appropriate C compiler path and flags.  The druntime patch also adds an import I missed before.
+
Both patches avoid building druntime/phobos as a shared library and set the appropriate C compiler and linker path and flags, along with a couple small fixes for Android that haven't been upstreamed yet.  The remaining changes are for the unit tests and are addressed below.
  
==Build a sample Android app==
+
==Build an Android sample app==
  
 
I've put up [https://github.com/joakim-noah/android an android repository which contains several translated headers and a sample C app translated to D].  Clone it into the same directory as dmd/druntime/phobos:
 
I've put up [https://github.com/joakim-noah/android an android repository which contains several translated headers and a sample C app translated to D].  Clone it into the same directory as dmd/druntime/phobos:
Line 56: Line 59:
 
First, let's see how it's done by default by compiling the C version.  Go to the sample native-activity app in my android repo, which is almost copied verbatim from the NDK/samples.  The only differences are that it also contains a translation of the given jni/main.c source to jni/main.d, changes one line in jni/Application.mk to build for x86 by default, and includes a dmd.conf for building with dmd.
 
First, let's see how it's done by default by compiling the C version.  Go to the sample native-activity app in my android repo, which is almost copied verbatim from the NDK/samples.  The only differences are that it also contains a translation of the given jni/main.c source to jni/main.d, changes one line in jni/Application.mk to build for x86 by default, and includes a dmd.conf for building with dmd.
  
Run the following commands to compile the C source into a debug app that you can install on Android, [http://developer.android.com/tools/sdk/ndk/index.html#Samples these commands are taken from the NDK instructions (scroll down to "Exploring the native-activity Sample Application")]:
+
Run the following commands to compile the C source into a debug app that you can install on Android, [http://developer.android.com/ndk/guides/index.html these commands are taken from the NDK instructions]:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
Line 66: Line 69:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
/home/joakim/android-ndk-r10/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang
+
/opt/ndk/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang -MMD -MP -MF
-MMD -MP -MF ./obj/local/x86/objs/native-activity/main.o.d -gcc-toolchain  
+
./obj/local/x86/objs/native-activity/main.o.d -gcc-toolchain
/home/joakim/android-ndk-r10/toolchains/x86-4.8/prebuilt/linux-x86 -target
+
/opt/ndk/toolchains/x86-4.8/prebuilt/linux-x86 -target i686-none-linux-android
i686-none-linux-android -ffunction-sections -funwind-tables -fstack-protector
+
-ffunction-sections -funwind-tables -fstack-protector -fPIC
-fPIC -no-canonical-prefixes -O2 -g -DNDEBUG -fomit-frame-pointer -fstrict-aliasing  
+
-Wno-invalid-command-line-argument -Wno-unused-command-line-argument
-I/home/joakim/android-ndk-r10/sources/android/native_app_glue -Ijni -DANDROID
+
-no-canonical-prefixes -O2 -g -DNDEBUG -fomit-frame-pointer -fstrict-aliasing
-Wa,--noexecstack -Wformat -Werror=format-security
+
-I/opt/ndk/sources/android/native_app_glue -Ijni -DANDROID -Wa,--noexecstack
-I/home/joakim/android-ndk-r10/platforms/android-9/arch-x86/usr/include  
+
-Wformat -Werror=format-security -I/opt/ndk/platforms/android-9/arch-x86/usr/include
 
-c jni/main.c -o ./obj/local/x86/objs/native-activity/main.o
 
-c jni/main.c -o ./obj/local/x86/objs/native-activity/main.o
 
</syntaxhighlight>
 
</syntaxhighlight>
Line 82: Line 85:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
/home/joakim/android-ndk-r10/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang++
+
/opt/ndk/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang++
 
-Wl,-soname,libnative-activity.so -shared
 
-Wl,-soname,libnative-activity.so -shared
--sysroot=/home/joakim/android-ndk-r10/platforms/android-9/arch-x86
+
--sysroot=/opt/ndk/platforms/android-9/arch-x86
 
./obj/local/x86/objs/native-activity/main.o
 
./obj/local/x86/objs/native-activity/main.o
./obj/local/x86/libandroid_native_app_glue.a -lgcc -gcc-toolchain
+
./obj/local/x86/libandroid_native_app_glue.a -lgcc -gcc-toolchain
/home/joakim/android-ndk-r10/toolchains/x86-4.8/prebuilt/linux-x86 -target  
+
/opt/ndk/toolchains/x86-4.8/prebuilt/linux-x86 -target i686-none-linux-android
i686-none-linux-android -no-canonical-prefixes -Wl,--no-undefined
+
-no-canonical-prefixes -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
-Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now
+
-Wl,-z,now -L/opt/ndk/platforms/android-9/arch-x86/usr/lib -llog -landroid
-L/home/joakim/android-ndk-r10/platforms/android-9/arch-x86/usr/lib
+
-lEGL -lGLESv1_CM -llog -lc -lm -o ./obj/local/x86/libnative-activity.so
-llog -landroid -lEGL -lGLESv1_CM -llog -lc -lm
 
-o ./obj/local/x86/libnative-activity.so
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 98: Line 99:
  
 
Getting back to building the C app, set the SDK environment variable for the path to your SDK and run the commands to package an Android app/apk:
 
Getting back to building the C app, set the SDK environment variable for the path to your SDK and run the commands to package an Android app/apk:
 +
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
 
export SDK=/path/to/your/android-sdk-linux
 
export SDK=/path/to/your/android-sdk-linux
Line 104: Line 106:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Now you'll push the final app, bin/NativeActivity-debug.apk, to your Android/x86 environment.  Connecting to Android will vary based on which Android/x86 you're using: I'll show you how I do it for Android/x86 installed in a VM, using [http://www.android-x86.org/documents/debug-howto the first method shown in the Android-x86 docs].  Hit Alt-F1 inside the VM to go to the root shell and type "netcfg" to get its IP address: let's say it's 192.168.0.1.  Hit Alt-F7 to go back to the UI, then go to Settings->Apps.  It should be empty.
+
Now you'll push the final app, NativeActivity-debug.apk, to your Android/x86 environment.  Connecting to Android will vary based on which Android/x86 you're using: I'll show you how I do it for Android/x86 installed in a VM, using [http://www.android-x86.org/documents/debug-howto the first method shown in the Android-x86 docs].  Hit Alt-F1 inside the VM to go to the root shell and type "netcfg" to get its IP address: let's say it's 192.168.0.1.  Hit Alt-F7 to go back to the UI, then go to Settings->Apps.  It should be empty.
  
Going back to the linux/x86 host:
+
Going back to the linux host:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
Line 114: Line 116:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
The NativeActivity app should show up in the Settings->Apps list in Android/x86.  Go to the app launcher and click on NativeActivity to run it.  Move the mouse and you should see a bunch of colors continuously flashing on the screen.  Looking at the log dump in linux/x86, you'll see some numbers from the accelerometer, which are being reported by the app.
+
The NativeActivity app should show up in the Settings->Apps list in Android/x86.  Go to the app launcher and click on NativeActivity to run it.  Move the mouse and you should see a bunch of colors continuously flashing on the screen.  Looking at the log dump in the linux host, you'll see some numbers from the accelerometer, which are being reported by the app.
  
 
===Build the translated D app===
 
===Build the translated D app===
Line 138: Line 140:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:232: error: undefined reference to 'rt_init'
+
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:232:
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:234: error: undefined reference to 'rt_term'
+
error: undefined reference to 'rt_init'
 +
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:234:
 +
error: undefined reference to 'rt_term'
 
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
 
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
 
make: *** [obj/local/x86/libnative-activity.so] Error 1
 
make: *** [obj/local/x86/libnative-activity.so] Error 1
Line 147: Line 151:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
../../../dmd/src/dmd -fPIC -I../.. -ofobj/local/x86/objs/native-activity/main.o
+
../../../dmd/src/dmd -android -I../.. -ofobj/local/x86/objs/native-activity/main.o
 
-c jni/main.d ../../android/sensor.d
 
-c jni/main.d ../../android/sensor.d
  
$NDK/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang -Wl,-soname,libnative-activity.so
+
$NDK/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang
-shared --sysroot=$NDK/platforms/android-9/arch-x86
+
-Wl,-soname,libnative-activity.so -shared
 +
--sysroot=$NDK/platforms/android-9/arch-x86
 
./obj/local/x86/objs/native-activity/main.o
 
./obj/local/x86/objs/native-activity/main.o
 
./obj/local/x86/libandroid_native_app_glue.a -lgcc  -gcc-toolchain  
 
./obj/local/x86/libandroid_native_app_glue.a -lgcc  -gcc-toolchain  
Line 162: Line 167:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
android/sensor.d is included in the first command to work around [https://issues.dlang.org/show_bug.cgi?id=12238 a dmd bug with unions declared in a separate file].  Note the final linker command is a lightly modified version of the one issued when linking the C shared library above, with phobos added to the mix. -fuse-ld=bfd was added to force the use of the ld.bfd linker instead of the gold linker, because ld.bfd works better with the Android TLS patch.
+
android/sensor.d is included in the first command to avoid [https://issues.dlang.org/show_bug.cgi?id=12238 a dmd bug with unions declared in a separate file].  Note the final linker command is a lightly modified version of the one issued when linking the C shared library above, with phobos added to the mix. -fuse-ld=bfd was added to force the use of the ld.bfd linker instead of the gold linker, because only ld.bfd works with the Android/x86 TLS patch.
  
 
Finally, you can build an apk and install to the Android/x86 VM as before:
 
Finally, you can build an apk and install to the Android/x86 VM as before:
Line 173: Line 178:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Run the app as before and you should see the same results with the D version.
+
Run the D app and you should see the same results.
  
 
==Build a command-line executable==
 
==Build a command-line executable==
  
This might be useful if you want to run your unit tests on the command-line, as I'll show with the druntime/phobos unit tests.
+
This might be useful if you want to run your unit tests on the command-line, as I'll show with the druntime/phobos unit tests later.
  
Let's try building one of the sample apps that come with dmd:
+
Let's try building one of the sample files that come with dmd:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
../../../dmd/src/dmd -fPIC -c ../../../dmd/samples/sieve.d
+
../../../dmd/src/dmd -android -c ../../../dmd/samples/sieve.d
  
$NDK/toolchains/llvm-3.3/prebuilt/linux-x86/bin/clang
+
$NDK/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang
 
-Wl,-z,nocopyreloc --sysroot=$NDK/platforms/android-9/arch-x86 -lgcc  
 
-Wl,-z,nocopyreloc --sysroot=$NDK/platforms/android-9/arch-x86 -lgcc  
 
-gcc-toolchain $NDK/toolchains/x86-4.8/prebuilt/linux-x86 -target
 
-gcc-toolchain $NDK/toolchains/x86-4.8/prebuilt/linux-x86 -target
Line 195: Line 200:
 
The linker command was extracted by building the sample test-libstdc++ executable from the NDK and removing a handful of flags that weren't needed.
 
The linker command was extracted by building the sample test-libstdc++ executable from the NDK and removing a handful of flags that weren't needed.
  
Now we can push it to Android, there are some quirks however.  Android doesn't allow you to run executables from the /sdcard partition, but you can only push apps to the sdcard.  I work around these restrictions by pushing to the sdcard and then copying it as root to the /data partition.  There are other ways to fix this, including remounting the partitions differently, which I haven't messed with.
+
Now we can push it to Android: there are some quirks however.  Android doesn't allow you to run executables from the /sdcard partition, but you can only push files to the sdcard.  I route around these restrictions by pushing to the sdcard and then copying the executable as root to the /data partition.
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
Line 204: Line 209:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
cp /sdcard/sieve data
+
mkdir /data/test
chmod 755 data/sieve
+
cd /data/test
./data/sieve
+
cp /sdcard/sieve .
 +
chmod 755 sieve
 +
./sieve
 
</syntaxhighlight>
 
</syntaxhighlight>
  
You'll see the following output:
+
You'll see the following output if everything went well:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
Line 216: Line 223:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Next, how to build and run the unit tests for druntime and phobos.
+
==Run the druntime and phobos unit tests==
 +
 
 +
Go back to the linux host and build the druntime and phobos unit tests:
 +
 
 +
<syntaxhighlight lang=bash>
 +
cd ../../../druntime
 +
make -f posix.mak unittest
 +
</syntaxhighlight>
 +
 
 +
If everything went right, you should see the following error, as the tests were compiled but won't be run on the linux host:
 +
 
 +
<syntaxhighlight lang=bash>
 +
make: obj/32/test_runner: Command not found
 +
posix.mak:234: recipe for target 'obj/32/object_' failed
 +
make: *** [obj/32/object_] Error 127
 +
</syntaxhighlight>
 +
 
 +
The patch for phobos adds support for extracting timezone data from Android's concatenated tzdata file to std.datetime, includes some workarounds for Android, and makes a few function call and formatting changes because "long double" and "double" are both 64-bit on Android:
 +
 
 +
<syntaxhighlight lang=bash>
 +
cd ../phobos
 +
make -f posix.mak unittest BUILD=release
 +
</syntaxhighlight>
 +
 
 +
Again, it will error out as the resulting test_runner binary won't run on the linux host:
 +
 
 +
<syntaxhighlight lang=bash>
 +
make: generated/linux/release/32/unittest/test_runner: Command not found
 +
posix.mak:384: recipe for target 'unittest/std/array.run' failed
 +
make: *** [unittest/std/array.run] Error 127
 +
</syntaxhighlight>
 +
 
 +
Download these lists of tests for [https://gist.github.com/joakim-noah/15420d6a933727a96aba druntime] and [https://gist.github.com/joakim-noah/b471e303a43c2ba2d055 phobos] and push them and the test runners to the Android VM:
 +
 
 +
<syntaxhighlight lang=bash>
 +
$SDK/platform-tools/adb push ../druntime/test_runner /sdcard/druntime_test_runner
 +
$SDK/platform-tools/adb push druntime_tests /sdcard/
 +
$SDK/platform-tools/adb push phobos_tests /sdcard/
 +
$SDK/platform-tools/adb push test_runner /sdcard/
 +
</syntaxhighlight>
 +
 
 +
Switch to the root shell in the Android VM and run the tests:
 +
 
 +
<syntaxhighlight lang=bash>
 +
cp /sdcard/druntime_test_runner test_runner
 +
cp /sdcard/druntime_tests /sdcard/phobos_tests .
 +
chmod 755 test_runner
 +
source druntime_tests > druntime_tests.log
 +
 
 +
cp /sdcard/test_runner .
 +
mkdir std
 +
echo test > std/string.d
 +
source phobos_tests > phobos_tests.log
 +
</syntaxhighlight>
 +
 
 +
I get the following results for [https://gist.github.com/joakim-noah/6784f95718f5554ad577 druntime] and [https://gist.github.com/joakim-noah/7f8cf4adf45d8c781569 phobos]: all modules pass their tests.
 +
 
 +
==Directions for future work==
 +
 
 +
* I want to try running the dmd test suite on Android/x86.
 +
 
 +
[[Category: DMD Compiler]]
 +
[[Category: Android]]

Latest revision as of 06:11, 10 January 2017

Note: these patches and instructions have not been updated for a couple years, as all work has moved to ldc and Android/ARM, which passes the entire D test suite. As such, these instructions are somewhat outdated and are only for reference.

All druntime and phobos tests from 2.070 git master pass on Android/x86, I'll try the dmd test suite someday. A C/OpenGLES 1.0 purely native sample app from the NDK has been ported to D. I list work that still needs to be done at the end.

Prerequisites

  • linux host on which to build dmd
    • A virtual machine like VirtualBox/VMware will do fine, but you should have at least 512 MB of memory allocated and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
  • C++ compiler and toolchain, to build dmd
  • A pre-built D compiler for linux, needed because the D frontend has been translated to D.
  • dmd/druntime/phobos source
    • Get the source from git, as the following Android patches have only been tested on the master branch of each repo.
  • Android native toolchain, the NDK and optionally the SDK
    • You don't need the SDK if you only want to build command-line binaries, the SDK is only needed to package and sign GUI apps. The "SDK Tools only" version of the SDK is enough, if you don't plan on using their IDE integration. I will only write about using the command-line tools. The SDK requires JDK 7: follow their instructions to make sure it's installed right.
  • Android/x86, whether a device or VM
    • The SDK comes with an emulator and Intel puts out their own. I use the builds put out by the android-x86 project and recommend the deprecated 4.3 build, as it's the least buggy one I've dealt with. I run the iso in a VM with 512 MBs of memory without installing to disk.

To run Android/x86 in a VM

Download the Android-x86 iso and install as normal. It'll put you through a configuration process, but you can skip every step. Go to Settings->Security and disable the Verify Apps option, or it'll ask you about that every time you install a test app. Alt-F1 will take you to a root shell, which you'll want to do any time you're not using Android/x86 for a couple minutes, or the screen will eventually go black and the VM will freeze up. Alt-F7 takes you back to the normal GUI screen.

Build D for Android/x86

Download the patch for dmd, apply it, and build normally:

cd dmd
git apply 3643.patch
make -f posix.mak -j5

This patch adds emulated TLS for Android/x86, more info can be found on the pull request.

Assuming druntime and phobos are in the same directory as dmd, download and apply the patch for druntime and the patch for phobos, then set the NDK environment variable to the path of wherever you installed the NDK:

cd ../druntime
export NDK=/path/to/your/android-ndk-r10
git apply druntime_android
make -f posix.mak

cd ../phobos
git apply phobos_android
make -f posix.mak

Both patches avoid building druntime/phobos as a shared library and set the appropriate C compiler and linker path and flags, along with a couple small fixes for Android that haven't been upstreamed yet. The remaining changes are for the unit tests and are addressed below.

Build an Android sample app

I've put up an android repository which contains several translated headers and a sample C app translated to D. Clone it into the same directory as dmd/druntime/phobos:

cd ..
git clone https://github.com/joakim-noah/android.git

Default build of the C sample app

First, let's see how it's done by default by compiling the C version. Go to the sample native-activity app in my android repo, which is almost copied verbatim from the NDK/samples. The only differences are that it also contains a translation of the given jni/main.c source to jni/main.d, changes one line in jni/Application.mk to build for x86 by default, and includes a dmd.conf for building with dmd.

Run the following commands to compile the C source into a debug app that you can install on Android, these commands are taken from the NDK instructions:

cd android/samples/native-activity
NDK_TOOLCHAIN_VERSION=clang $NDK/ndk-build V=1

Looking at the output, it compiles jni/main.c, a small wrapper library called android_native_app_glue, and links everything together into a shared library. Let's look at the command that compiles jni/main.c:

/opt/ndk/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang -MMD -MP -MF
./obj/local/x86/objs/native-activity/main.o.d -gcc-toolchain
/opt/ndk/toolchains/x86-4.8/prebuilt/linux-x86 -target i686-none-linux-android
-ffunction-sections -funwind-tables -fstack-protector -fPIC
-Wno-invalid-command-line-argument -Wno-unused-command-line-argument
-no-canonical-prefixes -O2 -g -DNDEBUG -fomit-frame-pointer -fstrict-aliasing
-I/opt/ndk/sources/android/native_app_glue -Ijni -DANDROID -Wa,--noexecstack
-Wformat -Werror=format-security -I/opt/ndk/platforms/android-9/arch-x86/usr/include
-c jni/main.c -o ./obj/local/x86/objs/native-activity/main.o

This is where I extracted the C compiler path and flags for the previous druntime/phobos patches, leaving out the dependency file (-MMD -MP -MF).

This command links the shared library that gets packaged into the native app:

/opt/ndk/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang++
-Wl,-soname,libnative-activity.so -shared
--sysroot=/opt/ndk/platforms/android-9/arch-x86
./obj/local/x86/objs/native-activity/main.o
./obj/local/x86/libandroid_native_app_glue.a -lgcc -gcc-toolchain
/opt/ndk/toolchains/x86-4.8/prebuilt/linux-x86 -target i686-none-linux-android
-no-canonical-prefixes -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
-Wl,-z,now -L/opt/ndk/platforms/android-9/arch-x86/usr/lib -llog -landroid
-lEGL -lGLESv1_CM -llog -lc -lm -o ./obj/local/x86/libnative-activity.so

You'll use a modified version of the above command to link your D source later.

Getting back to building the C app, set the SDK environment variable for the path to your SDK and run the commands to package an Android app/apk:

export SDK=/path/to/your/android-sdk-linux
$SDK/tools/android update project -p . -s --target 1
ant debug

Now you'll push the final app, NativeActivity-debug.apk, to your Android/x86 environment. Connecting to Android will vary based on which Android/x86 you're using: I'll show you how I do it for Android/x86 installed in a VM, using the first method shown in the Android-x86 docs. Hit Alt-F1 inside the VM to go to the root shell and type "netcfg" to get its IP address: let's say it's 192.168.0.1. Hit Alt-F7 to go back to the UI, then go to Settings->Apps. It should be empty.

Going back to the linux host:

$SDK/platform-tools/adb connect 192.168.0.1:5555
$SDK/platform-tools/adb install bin/NativeActivity-debug.apk
$SDK/platform-tools/adb logcat native-activity *:S

The NativeActivity app should show up in the Settings->Apps list in Android/x86. Go to the app launcher and click on NativeActivity to run it. Move the mouse and you should see a bunch of colors continuously flashing on the screen. Looking at the log dump in the linux host, you'll see some numbers from the accelerometer, which are being reported by the app.

Build the translated D app

I've translated jni/main.c to a D version, jni/main.d. First, you'll need to recompile the android_native_app_glue library so that it calls a couple functions necessary for a D shared library. Hit ctrl-c to get out of the Android log and open $NDK/sources/android/native_app_glue/android_native_app_glue.c in an editor. Find the android_main function and add the following rt_init()/rt_term() calls before and after it:

    rt_init();
    android_main(android_app);
    rt_term();

Clean up and compile as before:

$NDK/ndk-build clean
NDK_TOOLCHAIN_VERSION=clang $NDK/ndk-build V=1

You should see the following linker error, as the linker can't find the rt_init and rt_term functions you just added:

/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:232:
error: undefined reference to 'rt_init'
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:234:
error: undefined reference to 'rt_term'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [obj/local/x86/libnative-activity.so] Error 1

That's fine, now let's build and link the D source in instead:

../../../dmd/src/dmd -android -I../.. -ofobj/local/x86/objs/native-activity/main.o
-c jni/main.d ../../android/sensor.d

$NDK/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang
-Wl,-soname,libnative-activity.so -shared
--sysroot=$NDK/platforms/android-9/arch-x86
./obj/local/x86/objs/native-activity/main.o
./obj/local/x86/libandroid_native_app_glue.a -lgcc  -gcc-toolchain 
$NDK/toolchains/x86-4.8/prebuilt/linux-x86 -target i686-none-linux-android
-no-canonical-prefixes  -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
-Wl,-z,now  -L$NDK/platforms/android-9/arch-x86/usr/lib -llog -landroid
-lEGL -lGLESv1_CM -llog -lc -lm -fuse-ld=bfd
-L../../../phobos/generated/linux/release/32 -l:libphobos2.a 
-o ./libs/x86/libnative-activity.so

android/sensor.d is included in the first command to avoid a dmd bug with unions declared in a separate file. Note the final linker command is a lightly modified version of the one issued when linking the C shared library above, with phobos added to the mix. -fuse-ld=bfd was added to force the use of the ld.bfd linker instead of the gold linker, because only ld.bfd works with the Android/x86 TLS patch.

Finally, you can build an apk and install to the Android/x86 VM as before:

ant debug
$SDK/platform-tools/adb uninstall com.example.native_activity
$SDK/platform-tools/adb install bin/NativeActivity-debug.apk
$SDK/platform-tools/adb logcat native-activity *:S

Run the D app and you should see the same results.

Build a command-line executable

This might be useful if you want to run your unit tests on the command-line, as I'll show with the druntime/phobos unit tests later.

Let's try building one of the sample files that come with dmd:

../../../dmd/src/dmd -android -c ../../../dmd/samples/sieve.d

$NDK/toolchains/llvm-3.5/prebuilt/linux-x86/bin/clang
-Wl,-z,nocopyreloc --sysroot=$NDK/platforms/android-9/arch-x86 -lgcc 
-gcc-toolchain $NDK/toolchains/x86-4.8/prebuilt/linux-x86 -target
i686-none-linux-android -no-canonical-prefixes -fuse-ld=bfd 
-Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now
-Wl,--export-dynamic -lc -lm sieve.o
-L../../../phobos/generated/linux/release/32 -l:libphobos2.a -o ./sieve

The linker command was extracted by building the sample test-libstdc++ executable from the NDK and removing a handful of flags that weren't needed.

Now we can push it to Android: there are some quirks however. Android doesn't allow you to run executables from the /sdcard partition, but you can only push files to the sdcard. I route around these restrictions by pushing to the sdcard and then copying the executable as root to the /data partition.

$SDK/platform-tools/adb push sieve /sdcard/

Switch to the Android VM and hit Alt-F1 to get a root shell:

mkdir /data/test
cd /data/test
cp /sdcard/sieve .
chmod 755 sieve
./sieve

You'll see the following output if everything went well:

10 iterations
1899 primes

Run the druntime and phobos unit tests

Go back to the linux host and build the druntime and phobos unit tests:

cd ../../../druntime
make -f posix.mak unittest

If everything went right, you should see the following error, as the tests were compiled but won't be run on the linux host:

make: obj/32/test_runner: Command not found
posix.mak:234: recipe for target 'obj/32/object_' failed
make: *** [obj/32/object_] Error 127

The patch for phobos adds support for extracting timezone data from Android's concatenated tzdata file to std.datetime, includes some workarounds for Android, and makes a few function call and formatting changes because "long double" and "double" are both 64-bit on Android:

cd ../phobos
make -f posix.mak unittest BUILD=release

Again, it will error out as the resulting test_runner binary won't run on the linux host:

make: generated/linux/release/32/unittest/test_runner: Command not found
posix.mak:384: recipe for target 'unittest/std/array.run' failed
make: *** [unittest/std/array.run] Error 127

Download these lists of tests for druntime and phobos and push them and the test runners to the Android VM:

$SDK/platform-tools/adb push ../druntime/test_runner /sdcard/druntime_test_runner
$SDK/platform-tools/adb push druntime_tests /sdcard/
$SDK/platform-tools/adb push phobos_tests /sdcard/
$SDK/platform-tools/adb push test_runner /sdcard/

Switch to the root shell in the Android VM and run the tests:

cp /sdcard/druntime_test_runner test_runner
cp /sdcard/druntime_tests /sdcard/phobos_tests .
chmod 755 test_runner
source druntime_tests > druntime_tests.log

cp /sdcard/test_runner .
mkdir std
echo test > std/string.d
source phobos_tests > phobos_tests.log

I get the following results for druntime and phobos: all modules pass their tests.

Directions for future work

  • I want to try running the dmd test suite on Android/x86.