Difference between revisions of "Build DMD for Android"

From D Wiki
Jump to: navigation, search
(How to build druntime and phobos for Android/x86)
Line 1: Line 1:
This is a work in progress: Android/x86 is mostly done, going to work on Android/ARM next.  Almost all the druntime/phobos unit tests pass on Android/x86 and a sample C/OpenGLES 1.0 purely native app from the NDK has been ported to D.
+
This is a work in progress: Android/x86 is mostly done, going to work on Android/ARM next.  Almost all the druntime/phobos unit tests pass on Android/x86, plus a sample C/OpenGLES 1.0 purely native app from the NDK has been ported to D.
  
 
==Prerequisites==
 
==Prerequisites==
Line 9: Line 9:
 
** Get druntime and phobos from git, as they have Android-specific changes that are not in the release branches yet.
 
** Get druntime and phobos from git, as they have Android-specific changes that are not in the release branches yet.
 
* Android native toolchain, [http://developer.android.com/tools/sdk/ndk/index.html the 32-bit NDK] and [http://developer.android.com/sdk/index.html SDK]
 
* Android native toolchain, [http://developer.android.com/tools/sdk/ndk/index.html the 32-bit NDK] and [http://developer.android.com/sdk/index.html SDK]
** The "SDK Tools only" version of the SDK is enough, if you don't plan on using their IDE integration.  I will only write about using the command-line tools.
+
** The "SDK Tools only" version of the SDK is enough, if you don't plan on using their IDE integration.  I will only write about using the command-line tools.  The SDK requires JDK 6 and Ant 1.8, follow their instructions to make sure it's installed right.
 
* Android/x86, whether a device or VM
 
* Android/x86, whether a device or VM
 
** The SDK comes with an Android/x86 emulator and Intel puts out one of their own.  I use [http://www.android-x86.org/download the builds put out by the android-x86 project] and recommend the deprecated 4.3 build, as it's the least buggy one I've dealt with.
 
** The SDK comes with an Android/x86 emulator and Intel puts out one of their own.  I use [http://www.android-x86.org/download the builds put out by the android-x86 project] and recommend the deprecated 4.3 build, as it's the least buggy one I've dealt with.
Line 19: Line 19:
 
==Build D for Android/x86==
 
==Build D for Android/x86==
  
Download [http://164.138.25.188/dmd/packed_tls_for_elf.patch the patch for DMD], apply it, and [http://wiki.dlang.org/Building_DMD build normally]:
+
Download [http://164.138.25.188/dmd/packed_tls_for_elf.patch the patch for dmd], apply it, and [http://wiki.dlang.org/Building_DMD build normally]:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
Line 27: Line 27:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
This patch adds TLS support for Android/x86 to DMD, more info can be found on [https://github.com/D-Programming-Language/dmd/pull/3643 the pull request for DMD].
+
This patch adds TLS support for Android/x86, more info can be found on [https://github.com/D-Programming-Language/dmd/pull/3643 the pull request].
  
 
Assuming druntime and phobos are in the same directory as dmd, download and apply [http://164.138.25.188/dmd/druntime_build.patch the patch for druntime] and [http://164.138.25.188/dmd/phobos_build.patch the one for phobos], set the NDK environment variable to the path of wherever you installed the NDK, and build each in turn:
 
Assuming druntime and phobos are in the same directory as dmd, download and apply [http://164.138.25.188/dmd/druntime_build.patch the patch for druntime] and [http://164.138.25.188/dmd/phobos_build.patch the one for phobos], set the NDK environment variable to the path of wherever you installed the NDK, and build each in turn:
Line 42: Line 42:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Both patches avoid building druntime/phobos as a shared library and set the appropriate C compiler and flags.  The druntime patch also adds an import I missed before.
+
Both patches avoid building druntime/phobos as a shared library and set the appropriate C compiler path and flags.  The druntime patch also adds an import I missed before.
  
More to come on how I extracted those C compiler/flags NDK commands and how to build a sample app.
+
==Build a sample Android app==
 +
 
 +
I've put up [https://github.com/joakim-noah/android an android repository which contains several translated headers and a sample C app translated to D].  Clone it into the same directory as dmd/druntime/phobos:
 +
 
 +
<syntaxhighlight lang=bash>
 +
cd ..
 +
git clone https://github.com/joakim-noah/android.git
 +
</syntaxhighlight>
 +
 
 +
===Default build of the provided C app===
 +
First, let's see how it's done by default by compiling the C version.  Go to the sample native-activity app in my android repo, which is almost copied verbatim from the NDK/samples.  The only differences are that it also contains a translation of the given jni/main.c source to jni/main.d, changes one line in jni/Application.mk to build for x86 by default, and includes a dmd.conf for building with dmd.
 +
 
 +
Run the following commands to compile the C source into a debug app that you can install on Android, [http://developer.android.com/tools/sdk/ndk/index.html#Samples these commands are taken from the NDK instructions (scroll down to "Exploring the native-activity Sample Application")]:
 +
 
 +
<syntaxhighlight lang=bash>
 +
cd android/samples/native-activity
 +
NDK_TOOLCHAIN_VERSION=clang $NDK/ndk-build V=1
 +
</syntaxhighlight>
 +
 
 +
Looking at the output, it compiles jni/main.c, a smaller wrapper library called android_native_app_glue, and links everything together into a shared library.  Let's look at the command that compiles jni/main.c:
 +
 
 +
<syntaxhighlight lang=bash>
 +
/home/joakim/android-ndk-r10/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang
 +
-MMD -MP -MF ./obj/local/x86/objs/native-activity/main.o.d -gcc-toolchain
 +
/home/joakim/android-ndk-r10/toolchains/x86-4.8/prebuilt/linux-x86 -target
 +
i686-none-linux-android -ffunction-sections -funwind-tables -fstack-protector
 +
-fPIC -no-canonical-prefixes -O2 -g -DNDEBUG -fomit-frame-pointer -fstrict-aliasing
 +
-I/home/joakim/android-ndk-r10/sources/android/native_app_glue -Ijni -DANDROID
 +
-Wa,--noexecstack -Wformat -Werror=format-security
 +
-I/home/joakim/android-ndk-r10/platforms/android-9/arch-x86/usr/include
 +
-c jni/main.c -o ./obj/local/x86/objs/native-activity/main.o
 +
</syntaxhighlight>
 +
 
 +
This is where I extracted the C compiler path and flags for the previous druntime/phobos patches, leaving out the dependency file generation (-MMD -MP -MF).
 +
 
 +
This command links the shared library that gets packaged into the native app:
 +
 
 +
<syntaxhighlight lang=bash>
 +
/home/joakim/android-ndk-r10/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang++
 +
-Wl,-soname,libnative-activity.so -shared
 +
--sysroot=/home/joakim/android-ndk-r10/platforms/android-9/arch-x86
 +
./obj/local/x86/objs/native-activity/main.o
 +
./obj/local/x86/libandroid_native_app_glue.a -lgcc  -gcc-toolchain
 +
/home/joakim/android-ndk-r10/toolchains/x86-4.8/prebuilt/linux-x86 -target
 +
i686-none-linux-android -no-canonical-prefixes  -Wl,--no-undefined
 +
-Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now
 +
-L/home/joakim/android-ndk-r10/platforms/android-9/arch-x86/usr/lib
 +
-llog -landroid -lEGL -lGLESv1_CM -llog -lc -lm
 +
-o ./obj/local/x86/libnative-activity.so
 +
</syntaxhighlight>
 +
 
 +
You'll use a modified version of the above command to link your D source later.
 +
 
 +
Getting back to building the C app:
 +
<syntaxhighlight lang=bash>
 +
export SDK=/path/to/your/android-sdk-linux
 +
$SDK/tools/android update project -p . -s --target 1
 +
ant debug
 +
</syntaxhighlight>
 +
 
 +
Now you'll push the final app, bin/NativeActivity-debug.apk, to your Android/x86 environment.  Connecting to Android will vary based on which Android/x86 you're using: I'll show how I do it for Android/x86 installed in a VM, using [http://www.android-x86.org/documents/debug-howto the first method shown in the Android-x86 docs].  Hit Alt-F1 inside the VM to go to the root shell and type "netcfg" to get its IP address, say it's 192.168.0.1.  Hit Alt-F7 to go back to the UI, then go to Settings->Apps: it should be empty.
 +
 
 +
Going back to the linux/x86 host:
 +
 
 +
<syntaxhighlight lang=bash>
 +
$SDK/platform-tools/adb connect 192.168.0.1:5555
 +
$SDK/platform-tools/adb install bin/NativeActivity-debug.apk
 +
$SDK/platform-tools/adb logcat native-activity *:S
 +
</syntaxhighlight>
 +
 
 +
The NativeActivity app should show up in the Settings->Apps list in Android/x86.  Go to the app launcher and click on NativeActivity to run it.  Move the mouse and you should see a bunch of colors continuously flashing on the screen.  Looking at the log dump in linux/x86, you'll see some numbers from the accelerometer, which are being reported by the app.
 +
 
 +
===Build the translated D app===
 +
 
 +
I've translated jni/main.c to a D version, jni/main.d.  First, you'll need to recompile the android_native_app_glue library so that it calls a couple functions necessary for a D shared library.  Hit ctrl-c to get out of the Android log and open $NDK/sources/android/native_app_glue/android_native_app_glue.c in an editor.  Find the android_main function and add the following rt_init()/rt_term() calls before and after it:
 +
 
 +
<syntaxhighlight lang=D>
 +
 
 +
    rt_init();
 +
    android_main(android_app);
 +
    rt_term();
 +
 
 +
</syntaxhighlight>
 +
 
 +
Clean up and compile as before:
 +
 
 +
<syntaxhighlight lang=bash>
 +
$NDK/ndk-build clean
 +
NDK_TOOLCHAIN_VERSION=clang $NDK/ndk-build V=1
 +
</syntaxhighlight>
 +
 
 +
You should see the following linker error, as the linker can't find the rt_init and rt_term functions you just added:
 +
 
 +
<syntaxhighlight lang=bash>
 +
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:232: error: undefined reference to 'rt_init'
 +
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:234: error: undefined reference to 'rt_term'
 +
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
 +
make: *** [obj/local/x86/libnative-activity.so] Error 1
 +
</syntaxhighlight>
 +
 
 +
That's fine, now let's build and link the D source in instead:
 +
 
 +
<syntaxhighlight lang=bash>
 +
../../../dmd/src/dmd -fPIC -I../.. -ofobj/local/x86/objs/native-activity/main.o
 +
-c jni/main.d ../../android/sensor.d
 +
 
 +
mkdir -p libs/x86
 +
 
 +
$NDK/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang -Wl,-soname,libnative-activity.so
 +
-shared --sysroot=$NDK/platforms/android-9/arch-x86
 +
./obj/local/x86/objs/native-activity/main.o
 +
./obj/local/x86/libandroid_native_app_glue.a -lgcc  -gcc-toolchain
 +
$NDK/toolchains/x86-4.8/prebuilt/linux-x86 -target i686-none-linux-android
 +
-no-canonical-prefixes  -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
 +
-Wl,-z,now  -L$NDK/platforms/android-9/arch-x86/usr/lib -llog -landroid
 +
-lEGL -lGLESv1_CM -llog -lc -lm -fuse-ld=bfd
 +
-L../../../phobos/generated/linux/release/32 -l:libphobos2.a
 +
-o ./libs/x86/libnative-activity.so
 +
</syntaxhighlight>
 +
 
 +
android/sensor.d is included in the first command to work around [https://issues.dlang.org/show_bug.cgi?id=12238 a dmd bug with unions declared in a separate file].  I had to create the libs/x86 directory since the android build scripts just failed and didn't create it.  Note the final linker command is a lightly modified version of the one issued when linking the C shared library above, with phobos added to the mix. -fuse-ld=bfd was added to force the use of the ld.bfd linker instead of the gold linker, because ld.bfd works better with the Android TLS patch.
 +
 
 +
Finally, we can build an apk and install to the Android/x86 VM as before:
 +
 
 +
<syntaxhighlight lang=bash>
 +
ant debug
 +
$SDK/platform-tools/adb uninstall com.example.native_activity
 +
$SDK/platform-tools/adb install bin/NativeActivity-debug.apk
 +
$SDK/platform-tools/adb logcat native-activity *:S
 +
</syntaxhighlight>
 +
 
 +
Run the app as before and you should see the same results with the D version.
 +
 
 +
That's all for now, more coming later.

Revision as of 18:02, 17 August 2014

This is a work in progress: Android/x86 is mostly done, going to work on Android/ARM next. Almost all the druntime/phobos unit tests pass on Android/x86, plus a sample C/OpenGLES 1.0 purely native app from the NDK has been ported to D.

Prerequisites

  • 32-bit linux/x86 host on which to build dmd
    • A virtual machine like VirtualBox/VMware will work fine, but you should have at least 512 MB of memory allocated and 1 GB of swap, particularly if building the phobos unit tests, and 10 GB of disk space.
  • C++ compiler and toolchain, to build dmd
  • dmd/druntime/phobos source
    • Get druntime and phobos from git, as they have Android-specific changes that are not in the release branches yet.
  • Android native toolchain, the 32-bit NDK and SDK
    • The "SDK Tools only" version of the SDK is enough, if you don't plan on using their IDE integration. I will only write about using the command-line tools. The SDK requires JDK 6 and Ant 1.8, follow their instructions to make sure it's installed right.
  • Android/x86, whether a device or VM

To install Android/x86 in a VM

Download the Android-x86 iso and install as normal: it'll put you through a configuration process, but you can skip every step. Go to Settings->Security and disable the Verify Apps option, or it'll ask you about that every time you install a test app. Alt-F1 will take you to a root shell, which you'll want to do any time you're not using Android/x86 for a couple minutes or the screen will eventually go black and the VM will freeze up. Alt-F7 takes you back to the normal GUI screen.

Build D for Android/x86

Download the patch for dmd, apply it, and build normally:

cd dmd
git apply packed_tls_for_elf.patch
make -f posix.mak -j5

This patch adds TLS support for Android/x86, more info can be found on the pull request.

Assuming druntime and phobos are in the same directory as dmd, download and apply the patch for druntime and the one for phobos, set the NDK environment variable to the path of wherever you installed the NDK, and build each in turn:

cd ../druntime
export NDK=/path/to/your/android-ndk-r10
git apply druntime_build.patch
make -f posix.mak PIC=1

cd ../phobos
git apply phobos_build.patch
make -f posix.mak PIC=1

Both patches avoid building druntime/phobos as a shared library and set the appropriate C compiler path and flags. The druntime patch also adds an import I missed before.

Build a sample Android app

I've put up an android repository which contains several translated headers and a sample C app translated to D. Clone it into the same directory as dmd/druntime/phobos:

cd ..
git clone https://github.com/joakim-noah/android.git

Default build of the provided C app

First, let's see how it's done by default by compiling the C version. Go to the sample native-activity app in my android repo, which is almost copied verbatim from the NDK/samples. The only differences are that it also contains a translation of the given jni/main.c source to jni/main.d, changes one line in jni/Application.mk to build for x86 by default, and includes a dmd.conf for building with dmd.

Run the following commands to compile the C source into a debug app that you can install on Android, these commands are taken from the NDK instructions (scroll down to "Exploring the native-activity Sample Application"):

cd android/samples/native-activity
NDK_TOOLCHAIN_VERSION=clang $NDK/ndk-build V=1

Looking at the output, it compiles jni/main.c, a smaller wrapper library called android_native_app_glue, and links everything together into a shared library. Let's look at the command that compiles jni/main.c:

/home/joakim/android-ndk-r10/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang
-MMD -MP -MF ./obj/local/x86/objs/native-activity/main.o.d -gcc-toolchain 
/home/joakim/android-ndk-r10/toolchains/x86-4.8/prebuilt/linux-x86 -target
i686-none-linux-android -ffunction-sections -funwind-tables -fstack-protector
-fPIC -no-canonical-prefixes -O2 -g -DNDEBUG -fomit-frame-pointer -fstrict-aliasing 
-I/home/joakim/android-ndk-r10/sources/android/native_app_glue -Ijni -DANDROID
-Wa,--noexecstack -Wformat -Werror=format-security
-I/home/joakim/android-ndk-r10/platforms/android-9/arch-x86/usr/include 
-c jni/main.c -o ./obj/local/x86/objs/native-activity/main.o

This is where I extracted the C compiler path and flags for the previous druntime/phobos patches, leaving out the dependency file generation (-MMD -MP -MF).

This command links the shared library that gets packaged into the native app:

/home/joakim/android-ndk-r10/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang++
-Wl,-soname,libnative-activity.so -shared
--sysroot=/home/joakim/android-ndk-r10/platforms/android-9/arch-x86
./obj/local/x86/objs/native-activity/main.o
./obj/local/x86/libandroid_native_app_glue.a -lgcc  -gcc-toolchain
/home/joakim/android-ndk-r10/toolchains/x86-4.8/prebuilt/linux-x86 -target 
i686-none-linux-android -no-canonical-prefixes  -Wl,--no-undefined
-Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now
-L/home/joakim/android-ndk-r10/platforms/android-9/arch-x86/usr/lib
-llog -landroid -lEGL -lGLESv1_CM -llog -lc -lm
-o ./obj/local/x86/libnative-activity.so

You'll use a modified version of the above command to link your D source later.

Getting back to building the C app:

export SDK=/path/to/your/android-sdk-linux
$SDK/tools/android update project -p . -s --target 1
ant debug

Now you'll push the final app, bin/NativeActivity-debug.apk, to your Android/x86 environment. Connecting to Android will vary based on which Android/x86 you're using: I'll show how I do it for Android/x86 installed in a VM, using the first method shown in the Android-x86 docs. Hit Alt-F1 inside the VM to go to the root shell and type "netcfg" to get its IP address, say it's 192.168.0.1. Hit Alt-F7 to go back to the UI, then go to Settings->Apps: it should be empty.

Going back to the linux/x86 host:

$SDK/platform-tools/adb connect 192.168.0.1:5555
$SDK/platform-tools/adb install bin/NativeActivity-debug.apk
$SDK/platform-tools/adb logcat native-activity *:S

The NativeActivity app should show up in the Settings->Apps list in Android/x86. Go to the app launcher and click on NativeActivity to run it. Move the mouse and you should see a bunch of colors continuously flashing on the screen. Looking at the log dump in linux/x86, you'll see some numbers from the accelerometer, which are being reported by the app.

Build the translated D app

I've translated jni/main.c to a D version, jni/main.d. First, you'll need to recompile the android_native_app_glue library so that it calls a couple functions necessary for a D shared library. Hit ctrl-c to get out of the Android log and open $NDK/sources/android/native_app_glue/android_native_app_glue.c in an editor. Find the android_main function and add the following rt_init()/rt_term() calls before and after it:

    rt_init();
    android_main(android_app);
    rt_term();

Clean up and compile as before:

$NDK/ndk-build clean
NDK_TOOLCHAIN_VERSION=clang $NDK/ndk-build V=1

You should see the following linker error, as the linker can't find the rt_init and rt_term functions you just added:

/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:232: error: undefined reference to 'rt_init'
/home/joakim/android-ndk-r10/sources/android/native_app_glue/android_native_app_glue.c:234: error: undefined reference to 'rt_term'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [obj/local/x86/libnative-activity.so] Error 1

That's fine, now let's build and link the D source in instead:

../../../dmd/src/dmd -fPIC -I../.. -ofobj/local/x86/objs/native-activity/main.o
-c jni/main.d ../../android/sensor.d

mkdir -p libs/x86

$NDK/toolchains/llvm-3.4/prebuilt/linux-x86/bin/clang -Wl,-soname,libnative-activity.so
-shared --sysroot=$NDK/platforms/android-9/arch-x86
./obj/local/x86/objs/native-activity/main.o
./obj/local/x86/libandroid_native_app_glue.a -lgcc  -gcc-toolchain 
$NDK/toolchains/x86-4.8/prebuilt/linux-x86 -target i686-none-linux-android
-no-canonical-prefixes  -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
-Wl,-z,now  -L$NDK/platforms/android-9/arch-x86/usr/lib -llog -landroid
-lEGL -lGLESv1_CM -llog -lc -lm -fuse-ld=bfd
-L../../../phobos/generated/linux/release/32 -l:libphobos2.a 
-o ./libs/x86/libnative-activity.so

android/sensor.d is included in the first command to work around a dmd bug with unions declared in a separate file. I had to create the libs/x86 directory since the android build scripts just failed and didn't create it. Note the final linker command is a lightly modified version of the one issued when linking the C shared library above, with phobos added to the mix. -fuse-ld=bfd was added to force the use of the ld.bfd linker instead of the gold linker, because ld.bfd works better with the Android TLS patch.

Finally, we can build an apk and install to the Android/x86 VM as before:

ant debug
$SDK/platform-tools/adb uninstall com.example.native_activity
$SDK/platform-tools/adb install bin/NativeActivity-debug.apk
$SDK/platform-tools/adb logcat native-activity *:S

Run the app as before and you should see the same results with the D version.

That's all for now, more coming later.