Difference between revisions of "Build LDC for Android"

From D Wiki
Jump to: navigation, search
(Add instructions to build test runner apk)
(Small tweaks to prerequisites)
(24 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page will show you how to build a ldc cross-compiler for Android/ARM on linux, along with how to build and run both the druntime/phobos tests and an Android D app using the cross-compiler.
+
This page will show you how to build an ldc cross-compiler for Android/ARM on linux or Windows 10 (by using the new bash on linux subsystem), along with how to build and run the standard library's tests using the cross-compiler, both as a command-line binary and as a GUI Android app.  [https://github.com/joakim-noah/android/releases Prebuilt native and cross-compilers are available here].
  
Almost all the druntime/phobos unit tests pass on Android/ARM.  One of the native OpenGL sample apps from the Android NDK has been ported to D, I'll port some more soon.  Remaining work to be done is listed last.
+
All of the standard library's unit tests and the full compiler testsuite passes on Android/ARM.  Remaining work to be done is listed last.
 
 
You can also try out [[Build DMD for Android|dmd for Android/x86]].
 
  
 
==Prerequisites==
 
==Prerequisites==
  
* linux host, where you'll build and run ldc
+
* linux/x64 shell, where you'll build and run ldc
 
** You can use a virtual machine like VirtualBox/VMware, with at least 512 MB of memory and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
 
** You can use a virtual machine like VirtualBox/VMware, with at least 512 MB of memory and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
* C++ compiler and toolchain, to build ldc
+
** Windows 10: You can alternately use Bash on Windows (the Windows Subsystem for Linux), [[#Notes_for_Bash_on_Ubuntu_on_Windows|see full steps below]]
* Common development tools, such as CMake and git, and ldc uses libconfig++
+
* C++ compiler and toolchain, to build a slightly patched llvm and parts of ldc
 +
* A pre-built D compiler for linux, as the ldc frontend is written in D.
 +
** You can get [http://dlang.org/download the official dmd release for linux] or [https://github.com/ldc-developers/ldc/releases one of the ldc releases for linux].  Many distros also have D compiler packages.
 +
* Common development tools, such as CMake, Make, and git
 
* ldc/druntime/phobos source
 
* ldc/druntime/phobos source
** Get the source using git, as these Android patches were tested on the master branch of each repo.
+
** Get the source using git, as these Android patches were tested on the release-1.3.x branch
* llvm 3.6 source, either from the official release or git
+
* llvm 4.0.1 source, lightly tweaked for ldc
** llvm 3.7 or later will work too, but you'll have to modify the small llvm patch so it still applies.
+
* Android native toolchain, [https://developer.android.com/ndk/index.html the NDK] and optionally [https://developer.android.com/studio/index.html the SDK]
* Android native toolchain, [http://developer.android.com/ndk/index.html the NDK] and optionally [http://developer.android.com/sdk/index.html the SDK]
+
** The SDK is only needed if you want to package a GUI app; the NDK is enough if you just want to build a command-line binary, such as a test runner.
** The SDK is necessary if you want to package a GUI app; the NDK is enough if you just want to build a command-line binary, such as a test runner.  If you get the SDK, all that's needed is the "SDK Tools only" version, as long as you don't plan on using their IDE integration.  I will only write about using the command-line tools.  The SDK requires JDK 7: follow their instructions to make sure it's installed right.
 
 
* Android/ARM, whether a device or emulator
 
* Android/ARM, whether a device or emulator
 
** The SDK comes with an emulator.  I use actual hardware, so that's what I'll discuss.
 
** The SDK comes with an emulator.  I use actual hardware, so that's what I'll discuss.
 +
 +
===Notes for Bash on Ubuntu on Windows===
 +
* Necessary packages
 +
<syntaxhighlight lang=bash>
 +
sudo apt-get install build-essential
 +
sudo apt-get install git
 +
sudo apt-get install cmake
 +
sudo apt-get install unzip
 +
sudo apt-get install libconfig-dev
 +
</syntaxhighlight>
 +
 +
* DMD Compiler
 +
<syntaxhighlight lang=bash>
 +
cd ~
 +
curl -L -O http://downloads.dlang.org/releases/2.x/2.075.1/dmd_2.075.1-0_amd64.deb
 +
sudo dpkg -i dmd_2.075.1-0_amd64.deb
 +
</syntaxhighlight>
 +
 +
* Android Native Development Kit
 +
<syntaxhighlight lang=bash>
 +
sudo mkdir -p /opt/android-sdk/ndk-bundle
 +
curl -L -O https://dl.google.com/android/repository/android-ndk-r15c-linux-x86_64.zip
 +
sudo unzip android-ndk-r15c-linux-x86_64.zip 'android-ndk-r15c/*' -d /opt/android-sdk/ndk-bundle
 +
export NDK=/opt/android-sdk/ndk-bundle/android-ndk-r15c
 +
</syntaxhighlight>
 +
 +
As Windows Subsystem for Linux does not support USB, you have to install Android SDK and Ant on your Windows system and execute the commands "android" and "ant" from your DOS console.
  
 
==Compile llvm==
 
==Compile llvm==
  
Get the source for llvm, either [http://llvm.org/releases/download.html#3.6.2 the last official 3.6.2 release] or [https://android.googlesource.com/toolchain/llvm/ a git repository like the official Android llvm], which has some modifications but shouldn't really change much[https://gist.github.com/joakim-noah/1fb23fba1ba5b7e87e1a Download the patch for llvm], apply it, and then [http://llvm.org/docs/GettingStarted.html#getting-started-quickly-a-summary build llvm as you would normally], with the ARM target:
+
Get the source for llvm, either [https://github.com/ldc-developers/llvm/releases/tag/ldc-v4.0.1 this lightly tweaked llvm 4.0.1 source tarfile] or [https://github.com/ldc-developers/llvm the git repository for that tweaked branch].  [http://llvm.org/docs/GettingStarted.html#getting-started-quickly-a-summary Build llvm as you would normally], with the ARM, AArch64, and X86 targets, for Android/ARM as the default target, and leaving out tools and utilities you don't need:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
curl -O http://llvm.org/releases/3.6.2/llvm-3.6.2.src.tar.xz
+
curl -L -O https://github.com/ldc-developers/llvm/releases/download/ldc-v4.0.1/llvm-4.0.1.src.tar.xz
tar xvf llvm-3.6.2.src.tar.xz
+
tar xvf llvm-4.0.1.src.tar.xz
cd llvm-3.6.2.src/
+
cd llvm-4.0.1.src/
curl -O https://gist.githubusercontent.com/joakim-noah/1fb23fba1ba5b7e87e1a/raw/edb8005d8e972b2c258cd4699e6ad1b8315a8af7/android_tls
 
git apply android_tls
 
  
 
mkdir build
 
mkdir build
 
cd build/
 
cd build/
cmake .. -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=ARM
+
cmake .. -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD="ARM;AArch64;X86" -DLLVM_DEFAULT_TARGET_TRIPLE=armv7-none-linux-android -DLLVM_BUILD_TOOLS=OFF -DLLVM_BUILD_UTILS=OFF
make -j5
+
make all llvm-config -j5
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
==Build ldc for Android/ARM==
 
==Build ldc for Android/ARM==
  
Clone the ldc repository, [https://gist.github.com/joakim-noah/63693ead3aa62216e1d9 apply the Android patch], set the NDK environment variable to the location of your NDK and NDK_ARCH to its architecture, either x86 or x86_64, and [[Building LDC from source|build ldc as usual]]:
+
Clone the ldc repository, [https://gist.github.com/joakim-noah/d74af3cf1355492557a9c56ef1bf2636 apply the Android patch], set the DMD and NDK environment variables to the paths of your pre-built D compiler and NDK install, and [[Building LDC from source|build ldc as usual]]:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
Line 45: Line 70:
 
git clone --recursive https://github.com/ldc-developers/ldc.git
 
git clone --recursive https://github.com/ldc-developers/ldc.git
 
cd ldc/
 
cd ldc/
 +
git checkout -b ldc13 origin/release-1.3.x
 
git submodule update
 
git submodule update
curl -O https://gist.githubusercontent.com/joakim-noah/63693ead3aa62216e1d9/raw/b89d77d66a80206b4dd3d78bb10d83a7e368f3d4/ldc_android_arm
+
curl -O https://gist.githubusercontent.com/joakim-noah/d74af3cf1355492557a9c56ef1bf2636/raw/a0ac30302b5d7cc47049fb5ef88a87ec9afab824/ldc_1.3_android_arm
git apply ldc_android_arm
+
git apply ldc_1.3_android_arm
  
 
mkdir build
 
mkdir build
 
cd build/
 
cd build/
export NDK=/path/to/your/android-ndk-r10e
+
export DMD=/path/to/your/dmd2/linux/bin64/dmd
export NDK_ARCH=x86
+
export NDK=/path/to/your/android-ndk-r15b
cmake .. -DLLVM_CONFIG=../../llvm-3.6.2.src/build/bin/llvm-config
+
cmake .. -DLLVM_CONFIG=../../llvm-4.0.1.src/build/bin/llvm-config
 
make ldc2 -j5
 
make ldc2 -j5
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Download and apply [https://gist.github.com/joakim-noah/d936d6a339426ad1fac3 the patch for druntime] and [https://gist.github.com/joakim-noah/5c03801fa6c59b1e90df the patch for phobos] before building them:
+
Download and apply [https://gist.github.com/joakim-noah/09a59bdd6e04dd2f3d3ad6e15573ad97 a small patch for the standard library, phobos], before building the D runtime and stdlib:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
cd ../runtime/druntime/
+
cd ../runtime/phobos/
curl -O https://gist.githubusercontent.com/joakim-noah/d936d6a339426ad1fac3/raw/9486de62c72a64111e079bda9d7a7b58b6729909/druntime_ldc_arm
+
curl -O https://gist.githubusercontent.com/joakim-noah/09a59bdd6e04dd2f3d3ad6e15573ad97/raw/ba5bd391f1e9b669599c4d4939542512400ac934/phobos_1.3_ldc_arm
git apply druntime_ldc_arm
+
git apply phobos_1.3_ldc_arm
 
 
cd ../phobos/
 
curl -O https://gist.githubusercontent.com/joakim-noah/5c03801fa6c59b1e90df/raw/8ef824365d5dcc46fb47fabf7681e5425d81cc32/phobos_ldc_arm
 
git apply phobos_ldc_arm
 
  
 
cd ../../build/
 
cd ../../build/
Line 72: Line 94:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
More info about the Android/ARM patches can be [https://github.com/joakim-noah/android/releases/tag/polish found with their release].
+
More info about the Android/ARM patches can be [https://github.com/joakim-noah/android/releases/tag/tea found with their release].
  
==Build a command-line executable==
+
You can now try [[Build D for Android#Build_a_command-line_executable|cross-compiling a sample program as shown here]].  You will need a terminal emulator Android app and to set the path to the LDC cross-compiler and libraries you just built:
 
 
Now that we have a D cross-compiler and cross-compiled the standard library for Android/ARM, let's try building a small program, the classic Sieve of Eratosthenes single-core benchmark:
 
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
./bin/ldc2 -mtriple=armv7-none-linux-androideabi -relocation-model=pic
+
export LDC=/path/to/your/ldc/build
          -c ../tests/d2/dmd-testsuite/runnable/sieve.d
 
 
 
$NDK/toolchains/llvm-3.6/prebuilt/linux-$NDK_ARCH/bin/clang -Wl,-z,nocopyreloc
 
--sysroot=$NDK/platforms/android-9/arch-arm -lgcc
 
-gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-$NDK_ARCH
 
-target armv7-none-linux-androideabi -no-canonical-prefixes -fuse-ld=bfd
 
-Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
 
-Wl,-z,now -mthumb -Wl,--export-dynamic -lc -lm sieve.o lib/libphobos2-ldc.a
 
lib/libdruntime-ldc.a -o sieve
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
The compiler and linker flags were taken from [http://wiki.dlang.org/Build_DMD_for_Android#Default_build_of_the_C_sample_app the output from running a NDK sample app's build scripts in verbose mode].
+
==Run the druntime and phobos unit tests==
  
Now we run this program on an Android device or emulator.  I've solely run on actual Android devices, with either [https://play.google.com/store/apps/details?id=com.termux&hl=en a terminal app] or [https://play.google.com/store/apps/details?id=berserker.android.apps.sshdroid&hl=en an SSH server app].  Once you have either of those apps installed, copy the sieve program to the device, go to the app's local directory by typing 'cd' at its command-line, copy the program there, and run it:
+
You can build the druntime and phobos unit tests and run the command-line test runner binaries on Android (don't add the -j5 flag to build in parallel unless you have gigabytes of memory available, as compiling some of the phobos modules' tests takes a fair amount of RAM):
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
cd
+
make druntime-test-runner phobos2-test-runner
cp /sdcard/sieve .
 
./sieve foobar
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
The program requires an argument, which is ignored. If it runs correctly, you'll see the following output, saying it ran 10 times and found 1899 primes in the first 8191 integers:
+
Copy the test runners to your device and run them.  Assuming you have an SSH server set up on the computer where you're building with the linux shell and its IP address is 192.168.35.7, you can scp the binaries into [https://play.google.com/store/apps/details?id=com.termux&hl=en the Termux Android app] with these commands and run the tests:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
10 iterations
+
apt install openssh
1899 primes
+
scp jo@192.168.35.7:"/path/to/your/ldc/build/runtime/{druntime,phobos2}-test-runner" .
 +
./druntime-test-runner
 +
./phobos2-testrunner
 
</syntaxhighlight>
 
</syntaxhighlight>
  
==Run the druntime and phobos unit tests==
+
The tests take about 25 seconds to run on my quad-core tablet: all should pass.  One module, core.sync.semaphore, will fail for any Android older than 6.0, [https://github.com/D-Programming-Language/druntime/pull/784#issuecomment-42777328 because sem_destroy used to work differently in bionic].  You can also run the tests for specified modules by passing their names to the test runner:
 
 
Go back to the linux host and build the tests for druntime and phobos (don't add the -j5 flag to build in parallel unless you have GBs of memory available, as compiling some of the phobos modules' tests takes a fair amount of RAM):
 
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
make test-runner
+
./druntime-test-runner core.thread core.sync.semaphore
 +
./phobos2-testrunner std.datetime std.random
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Copy the test-runner and [https://github.com/joakim-noah/android/releases/download/polish/test.list this list of druntime and phobos modules] to your device and run it.  I use the SSH server app on a random port, here's what I'd do (replace 192.168.35.7 with the IP address of your device and 20345 with the port you configured for the SSH service):
+
==Run the druntime and phobos unit tests in an apk==
 
 
<syntaxhighlight lang=bash>
 
scp -P 20345 test.list runtime/test-runner jo@192.168.35.7:
 
ssh -p20345 jo@192.168.35.7
 
./test-runner
 
</syntaxhighlight>
 
 
 
The tests take about 40 seconds to run on my dual Cortex-A15 device: all of them pass.  A handful of tests across four modules were disabled, either because they fail or, in the case of rt.lifetime, pass but cause problems for subsequent tests.  One module, core.sync.semaphore, is not included in the list of modules, [https://github.com/D-Programming-Language/druntime/pull/784#issuecomment-42777328 because sem_destroy works differently in bionic] and triggers a segfault on the next GC run after its tests pass, which doesn't matter because that test assumes sem_destroy works in a certain way.
 
 
 
==Build a sample OpenGL Android app ported to D==
 
  
Clone [https://github.com/joakim-noah/android my android repository], which contains several headers and a C/OpenGL app from the NDK, translated to D:
+
You can also run the tests as part of a GUI app, ie an apk, which is a slightly different runtime environment.  First, you can try [[Build D for Android#Build_a_sample_OpenGL_ES_1.0_GUI_app_ported_to_D|cross-compiling a sample GUI app from the NDK that has been translated from C to D, as shown here]].  That simple OpenGLES 1.0 GUI app can be modified to run all the tests, which is what we'll do next.  Clone my Android repo, if you haven't already, go to the native-activity sample app, and create the output directory the SDK expects:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
 
cd ../../
 
cd ../../
 
git clone https://github.com/joakim-noah/android.git
 
git clone https://github.com/joakim-noah/android.git
</syntaxhighlight>
 
  
You can find more info about building using the NDK in [[Build DMD for Android#Build_an_Android_sample_app|my earlier instructions for Android/x86]].  This is just the essence, redone for ARM.  You will build a pure native apk without any Java source, ie pure D along with the basic C glue/wrapper that comes with the NDK.
 
 
First, you need to edit the C wrapper from the NDK, so that it initializes the D runtime properly for a shared library.  Go to the sample app and copy android_native_app_glue.h and android_native_app_glue.c from the NDK:
 
 
<syntaxhighlight lang=bash>
 
 
cd android/samples/native-activity/
 
cd android/samples/native-activity/
cp $NDK/sources/android/native_app_glue/android_native_app_glue.* .
 
</syntaxhighlight>
 
 
Open android_native_app_glue.c in an editor and find the function android_main, then insert rt_init() and rt_term() around it, so it looks like this:
 
 
<syntaxhighlight lang=C>
 
 
    rt_init();
 
    android_main(android_app);
 
    rt_term();
 
 
</syntaxhighlight>
 
 
Compile the D source and the file you just modified, then link them into a shared library and place it in the directory that the SDK expects:
 
 
<syntaxhighlight lang=bash>
 
../../../ldc/build/bin/ldc2 -mtriple=armv7-none-linux-androideabi
 
                            -relocation-model=pic -I../../ -c jni/main.d
 
 
../../../ldc/build/bin/ldc2 -mtriple=armv7-none-linux-androideabi
 
                            -relocation-model=pic -I../../ -c ../../android/sensor.d
 
 
$NDK/toolchains/llvm-3.6/prebuilt/linux-$NDK_ARCH/bin/clang
 
-gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-$NDK_ARCH
 
-fpic -ffunction-sections -funwind-tables -fstack-protector-strong
 
-Wno-invalid-command-line-argument -Wno-unused-command-line-argument
 
-no-canonical-prefixes -fno-integrated-as -target armv7-none-linux-androideabi
 
-march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Os -g -DNDEBUG
 
-fomit-frame-pointer -fno-strict-aliasing -I$NDK/sources/android/native_app_glue
 
-DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
 
-I$NDK/platforms/android-9/arch-arm/usr/include
 
-c ./android_native_app_glue.c -o ./android_native_app_glue.o
 
 
 
mkdir -p libs/armeabi-v7a/
 
mkdir -p libs/armeabi-v7a/
 
$NDK/toolchains/llvm-3.6/prebuilt/linux-$NDK_ARCH/bin/clang -Wl,-soname,libnative-activity.so
 
-shared --sysroot=$NDK/platforms/android-9/arch-arm main.o sensor.o
 
../../../ldc/build/lib/libphobos2-ldc.a ../../../ldc/build/lib/libdruntime-ldc.a
 
android_native_app_glue.o -lgcc
 
-gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-$NDK_ARCH
 
-no-canonical-prefixes -fuse-ld=bfd -target armv7-none-linux-androideabi
 
-Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now
 
-mthumb -L$NDK/platforms/android-9/arch-arm/usr/lib -llog -landroid -lEGL -lGLESv1_CM
 
-lc -lm -o libs/armeabi-v7a/libnative-activity.so
 
</syntaxhighlight>
 
 
Package the app as the SDK directs.  I use the older Ant approach, which is being deprecated, replace it with the Gradle command from a newer SDK if needed.  Set the path to your SDK, then package the apk using these commands:
 
 
<syntaxhighlight lang=bash>
 
export SDK=/path/to/your/android-sdk-linux
 
$SDK/tools/android update project -p . -s --target 1
 
ant debug
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Transfer the resulting bin/NativeActivity-debug.apk to your device, go to Settings->Security and allow installation of apps from unknown sources, ie outside the Play Store, then install it.  Go to your app folder and run the app named NativeActivity: it'll show a black screen and start flashing a bunch of colors upon a touch.
+
Download and apply [https://gist.github.com/joakim-noah/8ba3cd4958266f357295 a small patch to have the sample app invoke the test runner] and [https://gist.github.com/joakim-noah/348edc378d47fb90e32708be19286a2e a patch for the test runner in druntime], then build the tests into a shared library this time:
 
 
==Run the druntime and phobos unit tests in an apk==
 
 
 
Copy and edit the C wrapper and create the libs/armeabi-v7a/ directory as shown in the last section, then download and apply [https://gist.github.com/joakim-noah/8ba3cd4958266f357295 the small patch to have the sample app invoke the test runner], and rebuild:
 
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
Line 205: Line 144:
 
git apply native_ldc_arm
 
git apply native_ldc_arm
  
cd ../../../ldc/build/
+
cd ../../../ldc/runtime/druntime/
 +
curl -O https://gist.githubusercontent.com/joakim-noah/348edc378d47fb90e32708be19286a2e/raw/2b473ff45ff4abc68852ebb1868354b8528026e0/druntime_1.3_ldc_arm
 +
git apply druntime_1.3_ldc_arm
 +
 
 +
cd ../../build/
 
make test-runner-apk
 
make test-runner-apk
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
This assumes that the ldc and android repositories are in the same directory, as shown in these instructions.  If not, modify ANDROID_DIR in the CMake build script to use the path you want.
  
 
Finally, package the test runner apk:
 
Finally, package the test runner apk:
Line 213: Line 158:
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
 
cd ../../android/samples/native-activity/
 
cd ../../android/samples/native-activity/
 +
export SDK=/path/to/your/android-sdk-linux
 +
$SDK/tools/android update project -p . -s --target 1
 
ant debug
 
ant debug
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Transfer the resulting bin/NativeActivity-debug.apk to your device, and install it as before.  Also, copy [https://github.com/joakim-noah/android/releases/download/polish/test.list the list of modules to test] to the /sdcard/ directory.  The app will append its results to /sdcard/test.log, so if you happen to have a file with that name, move it.
+
Transfer the resulting bin/NativeActivity-debug.apk to your device and install it.  Also, copy [https://github.com/joakim-noah/android/releases/download/tea/test.list the list of modules to test] to the /sdcard/ directory.  The app will append its results to /sdcard/test.log, so if you happen to have a file with that name, move it.
  
This time, it should show a black screen for about a minute, while all the tests run.  A touch after that and it should start flashing a bunch of colors.  If not, look at the output in /sdcard/test.log and check if the app hung after any particular tested module.  You can remove that module from test.list and try running again.
+
The app should show a black screen for about a minute, while all the tests run.  A touch after that and it should start flashing a bunch of colors.  If not, look at the output in /sdcard/test.log and check if the app hung after any particular tested module.  You can remove that module from test.list and try running again.
  
 
==Directions for future work==
 
==Directions for future work==
 +
* You may notice that I added an empty main function in the D translation of the C sample app: that's a hack to build a shared library.  The linux shared library support in druntime's rt.sections_elf_shared may eventually be integrated with Android to get rid of that.
 +
 +
* Now that we can write D code for Android, it'll make building easier if the D cross-compilers are integrated with a build tool, like [https://github.com/dlang/dub dub], [https://github.com/atilaneves/reggae reggae], or [http://jasonwhite.github.io/button Button].
  
* Coming soon
 
  
 
[[Category:LDC]]
 
[[Category:LDC]]
 +
[[Category: Android]]

Revision as of 13:50, 26 August 2017

This page will show you how to build an ldc cross-compiler for Android/ARM on linux or Windows 10 (by using the new bash on linux subsystem), along with how to build and run the standard library's tests using the cross-compiler, both as a command-line binary and as a GUI Android app. Prebuilt native and cross-compilers are available here.

All of the standard library's unit tests and the full compiler testsuite passes on Android/ARM. Remaining work to be done is listed last.

Prerequisites

  • linux/x64 shell, where you'll build and run ldc
    • You can use a virtual machine like VirtualBox/VMware, with at least 512 MB of memory and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
    • Windows 10: You can alternately use Bash on Windows (the Windows Subsystem for Linux), see full steps below
  • C++ compiler and toolchain, to build a slightly patched llvm and parts of ldc
  • A pre-built D compiler for linux, as the ldc frontend is written in D.
  • Common development tools, such as CMake, Make, and git
  • ldc/druntime/phobos source
    • Get the source using git, as these Android patches were tested on the release-1.3.x branch
  • llvm 4.0.1 source, lightly tweaked for ldc
  • Android native toolchain, the NDK and optionally the SDK
    • The SDK is only needed if you want to package a GUI app; the NDK is enough if you just want to build a command-line binary, such as a test runner.
  • Android/ARM, whether a device or emulator
    • The SDK comes with an emulator. I use actual hardware, so that's what I'll discuss.

Notes for Bash on Ubuntu on Windows

  • Necessary packages
sudo apt-get install build-essential
sudo apt-get install git
sudo apt-get install cmake
sudo apt-get install unzip
sudo apt-get install libconfig-dev
  • DMD Compiler
cd ~
curl -L -O http://downloads.dlang.org/releases/2.x/2.075.1/dmd_2.075.1-0_amd64.deb
sudo dpkg -i dmd_2.075.1-0_amd64.deb
  • Android Native Development Kit
sudo mkdir -p /opt/android-sdk/ndk-bundle
curl -L -O https://dl.google.com/android/repository/android-ndk-r15c-linux-x86_64.zip
sudo unzip android-ndk-r15c-linux-x86_64.zip 'android-ndk-r15c/*' -d /opt/android-sdk/ndk-bundle
export NDK=/opt/android-sdk/ndk-bundle/android-ndk-r15c

As Windows Subsystem for Linux does not support USB, you have to install Android SDK and Ant on your Windows system and execute the commands "android" and "ant" from your DOS console.

Compile llvm

Get the source for llvm, either this lightly tweaked llvm 4.0.1 source tarfile or the git repository for that tweaked branch. Build llvm as you would normally, with the ARM, AArch64, and X86 targets, for Android/ARM as the default target, and leaving out tools and utilities you don't need:

curl -L -O https://github.com/ldc-developers/llvm/releases/download/ldc-v4.0.1/llvm-4.0.1.src.tar.xz
tar xvf llvm-4.0.1.src.tar.xz
cd llvm-4.0.1.src/

mkdir build
cd build/
cmake .. -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD="ARM;AArch64;X86" -DLLVM_DEFAULT_TARGET_TRIPLE=armv7-none-linux-android -DLLVM_BUILD_TOOLS=OFF -DLLVM_BUILD_UTILS=OFF
make all llvm-config -j5

Build ldc for Android/ARM

Clone the ldc repository, apply the Android patch, set the DMD and NDK environment variables to the paths of your pre-built D compiler and NDK install, and build ldc as usual:

cd ../../
git clone --recursive https://github.com/ldc-developers/ldc.git
cd ldc/
git checkout -b ldc13 origin/release-1.3.x
git submodule update
curl -O https://gist.githubusercontent.com/joakim-noah/d74af3cf1355492557a9c56ef1bf2636/raw/a0ac30302b5d7cc47049fb5ef88a87ec9afab824/ldc_1.3_android_arm
git apply ldc_1.3_android_arm

mkdir build
cd build/
export DMD=/path/to/your/dmd2/linux/bin64/dmd
export NDK=/path/to/your/android-ndk-r15b
cmake .. -DLLVM_CONFIG=../../llvm-4.0.1.src/build/bin/llvm-config
make ldc2 -j5

Download and apply a small patch for the standard library, phobos, before building the D runtime and stdlib:

cd ../runtime/phobos/
curl -O https://gist.githubusercontent.com/joakim-noah/09a59bdd6e04dd2f3d3ad6e15573ad97/raw/ba5bd391f1e9b669599c4d4939542512400ac934/phobos_1.3_ldc_arm
git apply phobos_1.3_ldc_arm

cd ../../build/
make druntime-ldc phobos2-ldc -j5

More info about the Android/ARM patches can be found with their release.

You can now try cross-compiling a sample program as shown here. You will need a terminal emulator Android app and to set the path to the LDC cross-compiler and libraries you just built:

export LDC=/path/to/your/ldc/build

Run the druntime and phobos unit tests

You can build the druntime and phobos unit tests and run the command-line test runner binaries on Android (don't add the -j5 flag to build in parallel unless you have gigabytes of memory available, as compiling some of the phobos modules' tests takes a fair amount of RAM):

make druntime-test-runner phobos2-test-runner

Copy the test runners to your device and run them. Assuming you have an SSH server set up on the computer where you're building with the linux shell and its IP address is 192.168.35.7, you can scp the binaries into the Termux Android app with these commands and run the tests:

apt install openssh
scp jo@192.168.35.7:"/path/to/your/ldc/build/runtime/{druntime,phobos2}-test-runner" .
./druntime-test-runner
./phobos2-testrunner

The tests take about 25 seconds to run on my quad-core tablet: all should pass. One module, core.sync.semaphore, will fail for any Android older than 6.0, because sem_destroy used to work differently in bionic. You can also run the tests for specified modules by passing their names to the test runner:

./druntime-test-runner core.thread core.sync.semaphore
./phobos2-testrunner std.datetime std.random

Run the druntime and phobos unit tests in an apk

You can also run the tests as part of a GUI app, ie an apk, which is a slightly different runtime environment. First, you can try cross-compiling a sample GUI app from the NDK that has been translated from C to D, as shown here. That simple OpenGLES 1.0 GUI app can be modified to run all the tests, which is what we'll do next. Clone my Android repo, if you haven't already, go to the native-activity sample app, and create the output directory the SDK expects:

cd ../../
git clone https://github.com/joakim-noah/android.git

cd android/samples/native-activity/
mkdir -p libs/armeabi-v7a/

Download and apply a small patch to have the sample app invoke the test runner and a patch for the test runner in druntime, then build the tests into a shared library this time:

curl -O https://gist.githubusercontent.com/joakim-noah/8ba3cd4958266f357295/raw/a52fcf1e63715f8b1bd3527afaa85872087b0f30/native_ldc_arm
git apply native_ldc_arm

cd ../../../ldc/runtime/druntime/
curl -O https://gist.githubusercontent.com/joakim-noah/348edc378d47fb90e32708be19286a2e/raw/2b473ff45ff4abc68852ebb1868354b8528026e0/druntime_1.3_ldc_arm
git apply druntime_1.3_ldc_arm

cd ../../build/
make test-runner-apk

This assumes that the ldc and android repositories are in the same directory, as shown in these instructions. If not, modify ANDROID_DIR in the CMake build script to use the path you want.

Finally, package the test runner apk:

cd ../../android/samples/native-activity/
export SDK=/path/to/your/android-sdk-linux
$SDK/tools/android update project -p . -s --target 1
ant debug

Transfer the resulting bin/NativeActivity-debug.apk to your device and install it. Also, copy the list of modules to test to the /sdcard/ directory. The app will append its results to /sdcard/test.log, so if you happen to have a file with that name, move it.

The app should show a black screen for about a minute, while all the tests run. A touch after that and it should start flashing a bunch of colors. If not, look at the output in /sdcard/test.log and check if the app hung after any particular tested module. You can remove that module from test.list and try running again.

Directions for future work

  • You may notice that I added an empty main function in the D translation of the C sample app: that's a hack to build a shared library. The linux shared library support in druntime's rt.sections_elf_shared may eventually be integrated with Android to get rid of that.
  • Now that we can write D code for Android, it'll make building easier if the D cross-compilers are integrated with a build tool, like dub, reggae, or Button.