View source for Build LDC for Android

This page will show you how to build a ldc cross-compiler for Android/ARM on linux, along with how to build and run both the druntime/phobos tests and an Android D app using the cross-compiler.

All of the standard library's unit tests and the full compiler testsuite passes on Android/ARM.  Two of the native OpenGL sample apps from the Android NDK have been ported to D, I'll port more soon.  Remaining work to be done is listed last.

==Prerequisites==

* linux host, where you'll build and run ldc
** You can use a virtual machine like VirtualBox/VMware, with at least 512 MB of memory and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
* C++ compiler and toolchain, to build a slightly patched llvm and parts of ldc
* A pre-built D compiler for linux, needed because the ldc frontend is written in D.
** You can get [http://dlang.org/download the official dmd release for linux] or [https://github.com/ldc-developers/ldc/releases one of the ldc releases for linux].  Many distros also have D compiler packages.
* Common development tools, such as CMake and git, and ldc uses libconfig++
* ldc/druntime/phobos source
** Get the source using git, as these Android patches were tested on the release-1.0.0 branch of each repo.
* llvm 3.8 source, either from the official release or git 
* Android native toolchain, [http://developer.android.com/ndk/index.html the NDK] and optionally [http://developer.android.com/sdk/index.html the SDK]
** The SDK is necessary if you want to package a GUI app; the NDK is enough if you just want to build a command-line binary, such as a test runner.  If you get the SDK, all that's needed is the "SDK Tools only" version, as long as you don't plan on using their IDE integration.  I will only write about using the command-line tools.  The SDK requires JDK 7: follow their instructions to make sure it's installed right.
* Android/ARM, whether a device or emulator
** The SDK comes with an emulator.  I use actual hardware, so that's what I'll discuss.

==Compile llvm==

Get the source for llvm, either [http://llvm.org/releases/download.html#3.8.0 the latest official 3.8.0 release] or [https://github.com/llvm-mirror/llvm a git repository, like this llvm mirror].  [https://gist.github.com/joakim-noah/1fb23fba1ba5b7e87e1a Download the patch for llvm], apply it, and then [http://llvm.org/docs/GettingStarted.html#getting-started-quickly-a-summary build llvm as you would normally], with the ARM target:

<syntaxhighlight lang=bash>
curl -O http://llvm.org/releases/3.8.0/llvm-3.8.0.src.tar.xz
tar xvf llvm-3.8.0.src.tar.xz
cd llvm-3.8.0.src/
curl -O https://gist.githubusercontent.com/joakim-noah/1fb23fba1ba5b7e87e1a/raw/ff54ecbe824b5f45669ea3a86f136ded16b1dd91/android_tls
git apply android_tls

mkdir build
cd build/
cmake .. -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=ARM
make -j5
</syntaxhighlight>

==Build ldc for Android/ARM==

Clone the ldc repository, check out the release-1.0.0 branch, [https://gist.github.com/joakim-noah/eff6d4ccca7975f32c3c35eb85f29554 apply the Android patch], set the DMD environment variable to the path of your pre-built D compiler, set the NDK environment variable to the location of your NDK and NDK_ARCH to its architecture, either x86 or x86_64, and [[Building LDC from source|build ldc as usual]]:

<syntaxhighlight lang=bash>
cd ../../
git clone --recursive https://github.com/ldc-developers/ldc.git
cd ldc/
git checkout -b ddmd origin/release-1.0.0
git submodule update
curl -O https://gist.githubusercontent.com/joakim-noah/eff6d4ccca7975f32c3c35eb85f29554/raw/3214799889e576e1b750b1164663c922e9408412/ldc_1.0.0_android_arm
git apply ldc_1.0.0_android_arm

mkdir build
cd build/
export DMD=/path/to/your/dmd2/linux/bin64/dmd
export NDK=/path/to/your/android-ndk-r12
export NDK_ARCH=x86_64
cmake .. -DLLVM_CONFIG=../../llvm-3.8.0.src/build/bin/llvm-config
make ldc2 -j5
</syntaxhighlight>

Download and apply [https://gist.github.com/joakim-noah/069036835dd92cf648a8219bfe4d68c3 the patch for druntime] and [https://gist.github.com/joakim-noah/17c5c37c32609dec218ba6031658a2c9 the patch for phobos] before building them:

<syntaxhighlight lang=bash>
cd ../runtime/druntime/
curl -O https://gist.githubusercontent.com/joakim-noah/069036835dd92cf648a8219bfe4d68c3/raw/9f42b65b0a17c9a15edc800bcdf644de99ffa72c/druntime_1.0.0_ldc_arm
git apply druntime_1.0.0_ldc_arm

cd ../phobos/
curl -O https://gist.githubusercontent.com/joakim-noah/17c5c37c32609dec218ba6031658a2c9/raw/689d1fc66cd0661a93d0eee55809569aeda1a0f2/phobos_1.0.0_ldc_arm
git apply phobos_1.0.0_ldc_arm

cd ../../build/
make druntime-ldc phobos2-ldc -j5
</syntaxhighlight>

More info about the Android/ARM patches can be [https://github.com/joakim-noah/android/releases/tag/ddmd found with their release].

==Build a command-line executable==

Now that we have a D cross-compiler and cross-compiled the standard library for Android/ARM, let's try building a small program, the classic Sieve of Eratosthenes single-core benchmark:

<syntaxhighlight lang=bash>
./bin/ldc2 -mtriple=armv7-none-linux-androideabi
           -c ../tests/d2/dmd-testsuite/runnable/sieve.d

$NDK/toolchains/llvm/prebuilt/linux-$NDK_ARCH/bin/clang -Wl,-z,nocopyreloc
--sysroot=$NDK/platforms/android-9/arch-arm -lgcc
-gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-$NDK_ARCH
-target armv7-none-linux-androideabi -no-canonical-prefixes -fuse-ld=bfd
-Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
-Wl,-z,now -fPIE -pie -mthumb -Wl,--export-dynamic -lc -lm sieve.o lib/libphobos2-ldc.a
lib/libdruntime-ldc.a -o sieve
</syntaxhighlight>

The compiler and linker flags were taken from [http://wiki.dlang.org/Build_DMD_for_Android#Default_build_of_the_C_sample_app the output from running a NDK sample app's build scripts in verbose mode].

Run this program on an Android device or emulator.  I've solely run on actual Android devices, with either [https://play.google.com/store/apps/details?id=com.termux&hl=en a terminal app] or [https://play.google.com/store/apps/details?id=berserker.android.apps.sshdroid&hl=en an SSH server app].  Once you have either of those apps installed, copy the sieve program to the device, go to the app's local directory by typing 'cd' at its command-line, copy the program there, and run it:

<syntaxhighlight lang=bash>
cd
cp /sdcard/sieve .
./sieve foobar
</syntaxhighlight>

The program requires an argument, which is ignored.  If it runs correctly, you'll see the following output, saying it ran 10 times and found 1899 primes in the first 8191 integers:

<syntaxhighlight lang=bash>
10 iterations
1899 primes
</syntaxhighlight>

==Run the druntime and phobos unit tests==

Go back to the linux host and build the tests for druntime and phobos (don't add the -j5 flag to build in parallel unless you have GBs of memory available, as compiling some of the phobos modules' tests takes a fair amount of RAM):

<syntaxhighlight lang=bash>
make test-runner
</syntaxhighlight>

Copy the test-runner and [https://github.com/joakim-noah/android/releases/download/ddmd/test.list this list of druntime and phobos modules] to your device and run it.  I use the SSH server app on a random port, here's what I'd do (replace 192.168.35.7 with the IP address of your device and 20345 with the port you configured for the SSH service):

<syntaxhighlight lang=bash>
scp -P20345 test.list runtime/test-runner jo@192.168.35.7:
ssh -p20345 jo@192.168.35.7
./test-runner
</syntaxhighlight>

The tests take about 40 seconds to run on my dual Cortex-A15 device: all should pass.  Two passing test blocks in rt.lifetime were disabled, as they cause problems for subsequent tests.  One module, core.sync.semaphore, is not included in the list of modules, [https://github.com/D-Programming-Language/druntime/pull/784#issuecomment-42777328 because sem_destroy works differently in bionic], which has been amended to match in the latest Android.  Two test blocks in std.net.curl may fail, depending on what version of curl they're run against.

==Build a sample OpenGL Android app ported to D==

Clone [https://github.com/joakim-noah/android my android repository], which contains several headers and a C/OpenGL app from the NDK, translated to D:

<syntaxhighlight lang=bash>
cd ../../
git clone https://github.com/joakim-noah/android.git
</syntaxhighlight>

You can find more info about building using the NDK in [[Build DMD for Android#Build_an_Android_sample_app|my earlier instructions for Android/x86]].  This is just the essence, redone for ARM.  You will build a pure native apk without any Java source, ie pure D along with the basic C glue/wrapper that comes with the NDK.

First, you need to edit the C wrapper from the NDK, so that it initializes the D runtime properly for a shared library.  Go to the native-activity sample app and copy android_native_app_glue.h and android_native_app_glue.c from the NDK:

<syntaxhighlight lang=bash>
cd android/samples/native-activity/
cp $NDK/sources/android/native_app_glue/android_native_app_glue.* .
</syntaxhighlight>

Open android_native_app_glue.c in an editor and find the function android_main, then insert rt_init() and rt_term() around it, so it looks like this:

<syntaxhighlight lang=C>

    rt_init();
    android_main(android_app);
    rt_term();

</syntaxhighlight>

Compile the D source and the file you just modified, then link them into a shared library and place it in the directory that the SDK expects:

<syntaxhighlight lang=bash>
../../../ldc/build/bin/ldc2 -mtriple=armv7-none-linux-androideabi
                            -I../../ -c jni/main.d

../../../ldc/build/bin/ldc2 -mtriple=armv7-none-linux-androideabi
                            -I../../ -c ../../android/sensor.d

$NDK/toolchains/llvm/prebuilt/linux-$NDK_ARCH/bin/clang
-gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-$NDK_ARCH
-fpic -ffunction-sections -funwind-tables -fstack-protector-strong
-Wno-invalid-command-line-argument -Wno-unused-command-line-argument
-no-canonical-prefixes -fno-integrated-as -target armv7-none-linux-androideabi
-march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Os -g -DNDEBUG
-fomit-frame-pointer -fno-strict-aliasing -I$NDK/sources/android/native_app_glue
-DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
-I$NDK/platforms/android-9/arch-arm/usr/include
-c ./android_native_app_glue.c -o ./android_native_app_glue.o

mkdir -p libs/armeabi-v7a/

$NDK/toolchains/llvm/prebuilt/linux-$NDK_ARCH/bin/clang -Wl,-soname,libnative-activity.so
-shared --sysroot=$NDK/platforms/android-9/arch-arm main.o sensor.o
../../../ldc/build/lib/libphobos2-ldc.a ../../../ldc/build/lib/libdruntime-ldc.a
android_native_app_glue.o -lgcc
-gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-$NDK_ARCH
-no-canonical-prefixes -fuse-ld=bfd -target armv7-none-linux-androideabi
-Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now
-mthumb -L$NDK/platforms/android-9/arch-arm/usr/lib -llog -landroid -lEGL -lGLESv1_CM
-lc -lm -o libs/armeabi-v7a/libnative-activity.so
</syntaxhighlight>

Package the app as the SDK directs.  I use the older Ant approach, which is being deprecated, replace it with the Gradle command from a newer SDK if needed.  For Ant, set the path to your SDK, then run these commands:

<syntaxhighlight lang=bash>
export SDK=/path/to/your/android-sdk-linux
$SDK/tools/android update project -p . -s --target 1
ant debug
</syntaxhighlight>

Transfer the resulting bin/NativeActivity-debug.apk to your device, go to Settings->Security and allow installation of apps from unknown sources, ie outside the Play Store, then install it.  Go to your app folder and run the app named NativeActivity: it'll show a black screen and start flashing a bunch of colors upon a touch.

==Run the druntime and phobos unit tests in an apk==

Copy and edit the C wrapper and create the libs/armeabi-v7a/ directory as shown in the last section, then download and apply [https://gist.github.com/joakim-noah/8ba3cd4958266f357295 the small patch to have the sample app invoke the test runner], and rebuild:

<syntaxhighlight lang=bash>
curl -O https://gist.githubusercontent.com/joakim-noah/8ba3cd4958266f357295/raw/a52fcf1e63715f8b1bd3527afaa85872087b0f30/native_ldc_arm
git apply native_ldc_arm

cd ../../../ldc/build/
make test-runner-apk
</syntaxhighlight>

This assumes that the ldc and android repositories are in the same directory, as shown in these instructions.  If not, modify ANDROID_DIR in the CMake build script to use the path you want.

Finally, package the test runner apk:

<syntaxhighlight lang=bash>
cd ../../android/samples/native-activity/
ant debug
</syntaxhighlight>

Transfer the resulting bin/NativeActivity-debug.apk to your device, and install it as before.  Also, copy [https://github.com/joakim-noah/android/releases/download/ddmd/test.list the list of modules to test] to the /sdcard/ directory.  The app will append its results to /sdcard/test.log, so if you happen to have a file with that name, move it.

This time, it should show a black screen for about a minute, while all the tests run.  A touch after that and it should start flashing a bunch of colors.  If not, look at the output in /sdcard/test.log and check if the app hung after any particular tested module.  You can remove that module from test.list and try running again.

==Directions for future work==

* Port the NDK support libraries and more of its sample apps to D, including apps that require using JNI, ie interfacing with Java.

* Two modules, core.thread and std.parallelism, have tests that cause the test runner to hang when run from inside an apk as opposed to on the command line.  Trying to suspend a thread from another thread, either directly by calling thread_suspendAll() or indirectly when the GC runs a full collect on a multi-threaded app, fails, because pthread_kill doesn't return and hangs the calling thread.  It appears that this is related to using SIGUSR1/2 for suspending and resuming threads: simply switching the two signals works around this issue for now.

* You may notice that I added an empty main function in the D translation of the C sample app: that's a hack to build a shared library.  Some of the linux shared library support in druntime's rt.sections_elf_shared may eventually be integrated with Android to get rid of that.

* The android_native_app_glue C wrapper can probably be replaced by D startup code, so that no C wrapper is necessary.

* Now that we can write D code for Android, it'll make building easier if the D cross-compilers are integrated with a build tool, like [https://github.com/atilaneves/reggae reggae] or [http://jasonwhite.github.io/button/ Button].

[[Category:LDC]]
[[Category: Android]]