Difference between revisions of "Build D for Android"

From D Wiki
Jump to: navigation, search
(Extract build instructions into its own page)
 
m (Adapt to LDC v1.38 and native ELF TLS)
 
(55 intermediate revisions by 7 users not shown)
Line 1: Line 1:
These instructions show you how to build D command-line executables and OpenGL ES GUI apps for Android, by using the [https://github.com/joakim-noah/android/releases native and cross-compilers available here].  There are separate steps for cross-compilation, ie building apps on a linux/x64 PC and running on Android/ARM, versus native compilation, building and running on your Android/ARM device itself.
+
These instructions show you how to build D command-line executables and OpenGL ES GUI apps for Android, either by using a desktop D compiler for Windows, Mac, or Linux, or a native Android compiler, all available [https://github.com/ldc-developers/ldc/releases here].  There are separate steps for cross-compilation, ie building apps on a Windows/Linux PC or Mac and running the app on Android, versus native compilation, both building and running on your Android device itself.
  
Since you cannot install the Android SDK on Android, I end by showing how to package a GUI Android app, a zip file called an .apk, from scratch, by using the tools available in the Termux app for Android, a terminal emulator app and OSS package manager/repository for Android devices.
+
Since you cannot install the Android SDK on Android, I end by showing how to package a GUI Android app, a zip file called an .apk, from scratch, by using the tools available in the Termux app for Android, a terminal emulator app and open-source package manager/repository for Android devices.
  
 
==Prerequisites==
 
==Prerequisites==
 
===Cross-compilation===
 
===Cross-compilation===
* linux/x64 shell, where you'll run the ldc cross-compiler, called the linux host
+
* A command shell on your host PC/Mac, where you'll run the LDC D compiler
** You can use a virtual machine like VirtualBox/VMware on Windows or Mac, with your favorite linux distro installed.
+
** Either a DOS command prompt or Powershell should work on Windows.
** Windows 10: You can alternately use Bash on Ubuntu on Windows (the Windows Subsystem for Linux).
+
** Any shell should work on Mac and Linux, typical commands for the <tt>bash</tt> shell are shown.
* Android native toolchain, [https://developer.android.com/ndk/index.html the NDK] and optionally [https://developer.android.com/studio/index.html the SDK]
+
* As of LDC v1.38, version r26 of the Android [https://developer.android.com/ndk/ NDK] and optionally the [https://developer.android.com/studio SDK]. Other versions might work too, but ideally, you'd stick with the NDK used for the official LDC Android packages.
** The SDK is necessary if you want to package a GUI app; the NDK is enough if you just want to build a command-line binary.
+
** The SDK is necessary to package a GUI app; the NDK is enough if you just want to build a command-line binary.
* Android/ARM, whether a device or emulator, to run your D code
+
* A recent LDC compiler for your host platform
 +
* Android, whether a device or emulator, to run your D code
 +
** LDC v1.38 has switched to native ELF TLS (thread-local storage), which was introduced in API level 29 (Android v10).
 
** The SDK comes with an emulator.  I use actual hardware, so that's what I'll discuss.
 
** The SDK comes with an emulator.  I use actual hardware, so that's what I'll discuss.
** If using a device, you need some way to transfer the app over.  There are several ways to do this, here are a few I've tried:
+
** When using a device, you need some way to transfer the app over.  There are several ways to do this, here are a few I've tried:
# Install an ssh server app on your device and scp the app over.  Alternately, set up an ssh server on your linux host, and use an ssh/scp client on Android to get the app.  This is what I do, by using the OpenSSH client in Termux.  
+
# Install an ssh server app on your Android device and scp the app over.  Alternately, set up an ssh server on your host PC/Mac, and use an ssh/scp client on Android to get the app.  This is what I do, by using the OpenSSH package in Termux.  
 
# Host the app in a web server and get it by using your Android browser or a downloader app.
 
# Host the app in a web server and get it by using your Android browser or a downloader app.
 
# [https://developer.android.com/studio/command-line/adb.html Setup the Android Debug Bridge (adb) on your device] and use the SDK tools to push your files over.
 
# [https://developer.android.com/studio/command-line/adb.html Setup the Android Debug Bridge (adb) on your device] and use the SDK tools to push your files over.
  
 
===Native compilation===
 
===Native compilation===
 +
* Android Version 10 ("Q"), as LDC compiler support does not exist for Android Version 9 ("Pie") and earlier.
 
* Termux for Android, available in [https://play.google.com/store/apps/details?id=com.termux&hl=en the official Play Store], [https://www.apkmirror.com/apk/fredrik-fornwall/termux/ APKMirror], or [https://f-droid.org/packages/com.termux/ F-Droid]
 
* Termux for Android, available in [https://play.google.com/store/apps/details?id=com.termux&hl=en the official Play Store], [https://www.apkmirror.com/apk/fredrik-fornwall/termux/ APKMirror], or [https://f-droid.org/packages/com.termux/ F-Droid]
 +
* LDC for Termux: <code>apt install ldc</code>  With Android version 9 and earlier, the install will fail.
  
==Setup==
+
==Cross-compilation setup==
  
Once you've got a linux/x64 shell setup or the Termux app installed, get the ldc compiler for Android and the NDK for cross-compilation.
+
Once you have LDC and have unzipped the Android NDK, it's time to set up LDC for the desired Android target(s). See [[Cross-compiling with LDC]] for the general guide; I present two examples for a quick summary:
  
===Cross-compilation===
+
* Targeting 32-bit Android/ARMv7-A on a Win64 host:
 +
*# Download the prebuilt <tt>android-armv7a</tt> [https://github.com/ldc-developers/ldc/releases/ package from GitHub] matching the version of your LDC.
 +
*# Extract the <tt>lib</tt> directory into your LDC installation directory and rename it, e.g., to <tt>lib-android_armv7a</tt>.
 +
*# Open <tt><LDC install dir>\etc\ldc2.conf</tt> in a text editor and append a section like this, adapting lib and NDK paths as needed:
 +
<pre>
 +
"armv7a-.*-linux-android":
 +
{
 +
    switches = [
 +
        "-defaultlib=phobos2-ldc,druntime-ldc",
 +
        "-gcc=C:/LDC/android-ndk-r26d/toolchains/llvm/prebuilt/windows-x86_64/bin/armv7a-linux-androideabi29-clang.cmd",
 +
    ];
 +
    lib-dirs = [
 +
        "%%ldcbinarypath%%/../lib-android_armv7a",
 +
    ];
 +
};
 +
</pre>
 +
* Targeting 64-bit Android/AArch64 on a Linux host:
 +
*# Download the prebuilt <tt>android-aarch64</tt> [https://github.com/ldc-developers/ldc/releases/ package from GitHub] matching the version of your LDC.
 +
*# Extract the <tt>lib</tt> directory into your LDC installation directory and rename it, e.g., to <tt>lib-android_aarch64</tt>.
 +
*# Open <tt><LDC install dir>/etc/ldc2.conf</tt> in a text editor and append a section like this, adapting lib and NDK paths as needed:
 +
<pre>
 +
"aarch64-.*-linux-android":
 +
{
 +
    switches = [
 +
        "-defaultlib=phobos2-ldc,druntime-ldc",
 +
        "-gcc=/home/me/android-ndk-r26d/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android30-clang",
 +
    ];
 +
    lib-dirs = [
 +
        "%%ldcbinarypath%%/../lib-android_aarch64",
 +
    ];
 +
};
 +
</pre>
 +
 
 +
The prebuilt Android packages also include the corresponding x86 simulator libraries, so 32/64-bit x86 Android simulator targets can be set up the same way.
 +
 
 +
==Build a command-line executable==
 +
 
 +
Now that we have a D compiler setup for (one or more) Android targets, let's try building a small program, [https://github.com/dlang/dmd/blob/master/samples/sieve.d the classic Sieve of Eratosthenes single-core benchmark], which finds all prime numbers up to a number you choose. Install the curl package in Termux if you're natively compiling, <tt>apt install curl</tt>.
  
Make sure curl is available, or use the equivalent wget command.  You will need tar to unpack ldc and unzip for the NDK.  I show the command to install unzip for Ubuntu, use the right package manager for your distro.
 
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
sudo apt-get install unzip
+
# Load this link in your browser and download the file otherwise
 +
curl -L -O https://raw.githubusercontent.com/dlang/dmd/master/samples/sieve.d
  
curl -L -O https://dl.google.com/android/repository/android-ndk-r15c-linux-x86_64.zip
+
# Cross-compile & -link to ARMv7-A (on any host)
unzip android-ndk-r15c-linux-x86_64.zip
+
ldc2 -mtriple=armv7a--linux-androideabi29 sieve.d
export NDK=/path/to/your/android-ndk-r15c
 
  
curl -L -O https://github.com/joakim-noah/android/releases/download/tea/ldc2-android-arm-1.3.1-beta2-linux-x86_64.tar.xz
+
# Cross-compile & -link to AArch64 (on any host)
tar xf ldc2-android-arm-1.3.1-beta2-linux-x86_64.tar.xz
+
ldc2 -mtriple=aarch64--linux-android29 sieve.d
export LDC=/path/to/your/ldc2-android-arm-1.3.1-beta2-linux-x86_64
+
 
 +
# Compile & link natively in Termux
 +
ldc2 sieve.d
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Make sure the NDK and LDC variables are set to the full path where they are located.
+
===Cross-compilation===
  
===Native compilation===
+
Copy this <tt>sieve</tt> program onto an Android device or emulator and set its permissions with the <tt>chmod</tt> command. Here's how I do it in Termux, with an ssh server running on the host PC/Mac with IP address 192.168.1.37:
  
You need the clang compiler and the linker it automatically installs, as ldc tries to use the local C compiler for linking.
 
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
apt install clang curl
+
apt install openssh
curl -L -O https://github.com/joakim-noah/android/releases/download/tea/ldc_1.3.0_arm.deb
+
cd
dpkg -i ldc_1.3.0_arm.deb
+
scp jo@192.168.1.37:sieve .
 +
chmod 700 sieve
 
</syntaxhighlight>
 
</syntaxhighlight>
  
==Build a command-line executable==
+
===Run the sieve program===
 +
The <tt>sieve</tt> program will tell you how many prime numbers there are in the first n integers, a limit you can specify.  Run this command to find how many primes there are in the first million integers:
  
Now that we have a D compiler for Android, let's try building a small program, [https://github.com/dlang/dmd/blob/master/samples/sieve.d the classic Sieve of Eratosthenes single-core benchmark], which finds all prime numbers up to a number you choose.
+
<syntaxhighlight lang=bash>
 +
./sieve 1000000
 +
</syntaxhighlight>
  
===Cross-compilation===
+
If you built <tt>sieve</tt> successfully, it should return
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
curl -L -O https://raw.githubusercontent.com/dlang/dmd/master/samples/sieve.d
+
78498 primes
 +
</syntaxhighlight>
  
$LDC/bin/ldc2 -c sieve.d
+
==Build a sample OpenGL ES 1.0 GUI app ported to D==
  
$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -Wl,-z,nocopyreloc
+
Clone the [https://github.com/Diewi/android android repository] or download [https://github.com/Diewi/android/releases its source in a zip file], which contains several headers and sample OpenGL apps from the NDK translated to D:
--sysroot=$NDK/platforms/android-16/arch-arm -lgcc
 
-gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64
 
-target armv7-none-linux-androideabi -no-canonical-prefixes -fuse-ld=bfd
 
-Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro
 
-Wl,-z,now -fPIE -pie -Wl,--export-dynamic -lc -lm sieve.o
 
$LDC/lib/libphobos2-ldc.a $LDC/lib/libdruntime-ldc.a -o sieve
 
</syntaxhighlight>
 
  
Copy this sieve program onto an Android device or emulator. Here's how I do it in Termux, with an ssh server running on the linux host:
+
<syntaxhighlight lang=bash>
 +
sudo apt-get install git # In Termux, apt install git
 +
git clone https://github.com/Diewi/android.git
 +
cd android
  
<syntaxhighlight lang=bash>
+
# Alternatively, without git:
cd
+
curl -L -O https://github.com/Diewi/android/archive/build.zip
scp jo@my.server.com:sieve .
+
unzip build.zip
 +
cd android-build
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===Native compilation===
+
Then build the Native Activity app, which is written completely in D. D code for an apk must be compiled to a shared library, which the Android runtime will call:
 +
 
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
curl -L -O https://raw.githubusercontent.com/dlang/dmd/master/samples/sieve.d
+
cd samples/native-activity
  
ldc2 sieve.d
+
ldc2 -I../.. jni/main.d ../../android/sensor.d ../../android_native_app_glue.d \
 +
    -shared -of=libs/arm64-v8a/libnative-activity.so \ # or `libs/armeabi-v7a/...` for 32-bit ARM
 +
    -L-soname -Llibnative-activity.so \
 +
    -mtriple=aarch64--linux-android29 # only for cross-compilation; use `armv7a--linux-androideabi29` for 32-bit ARM
 +
    # possibly needed: -L-llog -L-landroid -L-lEGL -L-lGLESv1_CM
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===Run the sieve program===
+
===Cross-compilation===
The sieve program will tell you how many prime numbers there are in the first n integers, a limit you can specifyRun this command to find how many primes there are in the first million integers:
+
 
 +
====Ant approach====
 +
Finally, package the app as the SDK directs: at this point, it's just like building a regular Android appI document the older Ant approach, which is deprecated, replace it with the Gradle command from a newer SDK.  With Ant on Mac or Linux, set the path to your SDK, then run these commands:
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
./sieve 1000000
+
export SDK=/path/to/your/android-sdk
 +
$SDK/tools/android update project -p . -s --target 1
 +
ant debug
 
</syntaxhighlight>
 
</syntaxhighlight>
  
If you built sieve successfully, it should return
+
====Android Studio approach====
 +
 
 +
Enter on your <application name>/app/src/main and create a folder called jniLibs, this folder is <b>extremely important</b>, it is the default folder to put your shared libraries to be imported together with your .apk. If you wish to use other name for it, you will need to change your gradle file. For actually putting your libraries inside that folder, you will actually need to make directories for the target architectures, so, create inside it:
 +
 
 +
* armeabi-v7a (For that, it is commonly used ldc2(version)-android-armv7a/lib
 +
* arm64-v8a (This is our target right now, ldc2(version)-android-aarch64/lib)
 +
* x86 (It is the lib32 for armv7a -> ldc2(version)-android-armv7a/lib686
 +
* x86_64 (It is the lib32 for the aarch64 -> ldc2(version)-android-aarch64/lib-x86_64 For reference, check ndk abi guide from official android site: [https://developer.android.com/ndk/guides/abis Android ABI Guide]
 +
 
 +
After creating those folders, you can actually move your shared library inside one of them, just click on run and you're good to go.
 +
 
 +
---
 +
 
 +
Transfer the resulting <tt>bin/NativeActivity-debug.apk</tt> to your Android device, again shown here by using <tt>scp</tt> from the Termux app.
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
78498 primes
+
scp jo@192.168.1.37:android/samples/native-activity/bin/NativeActivity-debug.apk /sdcard/Download/
 
</syntaxhighlight>
 
</syntaxhighlight>
  
==Build a sample OpenGL ES GUI app ported to D==
+
===Native compilation===
 +
 
 +
[[#Package_an_Android_app_from_scratch_on_your_Android_device|Follow the instructions below to package this native shared library into an Android apk]].
 +
 
 +
===Install and run the sample GUI app===
 +
Go to <tt>Settings->Security</tt> on your Android device and allow installation of apps from unknown sources, ie from outside the Play Store, then go to <tt>/sdcard/Download</tt> in your file manager and choose the <tt>NativeActivity-debug</tt> apk to install it.  Open the app after installing or go to your app folder and run the app named <tt>NativeActivity</tt>: it'll show a black screen initially, then flash a bunch of colors when the screen is touched.
 +
 
 +
==Build a sample OpenGL ES 2.0 GUI app mostly written in D, with some Java==
  
Clone [https://github.com/joakim-noah/android my android repository], which contains several headers and sample OpenGL apps from the NDK translated to D, and build the Native Activity app, which is written completely in D.
+
This D app has not been ported to 64-bit Android/ARM yet, only 32-bit ARM compilation will work for now:
 +
 
 +
<syntaxhighlight lang=bash>
 +
cd samples/Teapot
 +
 
 +
ldc2 -I../.. -Ijni -Jjni \
 +
    ../../ndk_helper/GLContext.d \
 +
    ../../ndk_helper/JNIHelper.d \
 +
    ../../ndk_helper/gestureDetector.d \
 +
    ../../ndk_helper/perfMonitor.d \
 +
    ../../ndk_helper/shader.d \
 +
    ../../ndk_helper/tapCamera.d \
 +
    jni/TeapotNativeActivity.d \
 +
    jni/TeapotRenderer.d \
 +
    ../../android/sensor.d \
 +
    ../../android_native_app_glue.d \
 +
    -shared -of=libs/armeabi-v7a/libTeapotNativeActivity.so \
 +
    -L-soname -LlibTeapotNativeActivity.so \
 +
    -mtriple=armv7a--linux-androideabi29 # only for cross-compilation
 +
    # possibly needed: -L-llog -L-landroid -L-lEGL -L-lGLESv2
 +
</syntaxhighlight>
  
 
===Cross-compilation===
 
===Cross-compilation===
 +
Package this shared library into an apk by using the SDK, as you would normally, and try installing and running it on your device.
  
After cloning my Android repo, go to the sample app, compile the D source, then link the objects into a shared library and place it in the directory that the SDK expects:
+
===Native compilation===
 +
 
 +
Install the right Eclipse Java compiler package for your device (the ecj4.6 package if you're running Android 5 or 6), the Android dex tool, and other packages needed to build an Android apk. Generate any Java files needed, compile and dex them, then package everything up into an apk and sign it.
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
sudo apt-get install git
+
apt install ecj dx aapt apksigner
 +
 
 +
aapt package  -M ./AndroidManifest.xml -I $PREFIX/share/java/android-21.jar -J src/ -S res -m
 +
 
 +
ecj-21 -d ./obj -sourcepath src $(find src -type f -name "*.java")
 +
 
 +
dx --dex --output=./classes.dex ./obj/
 +
 
 +
aapt package  -M ./AndroidManifest.xml -S res -A assets -F teapot.apk
 +
 
 +
aapt add teapot.apk classes.dex lib/armeabi-v7a/libTeapotNativeActivity.so
 +
 
 +
apksigner debug.ks teapot.apk teapot-signed.apk
 +
</syntaxhighlight>
 +
 
 +
Finally, move <tt>teapot-signed.apk</tt> into a public directory, from which you can install and run it.
  
git clone https://github.com/joakim-noah/android.git
+
==Examples and useful repositories==
 +
* [https://github.com/adamdruppe/d_android D Android from adamdruppe] - You can find android-setup.d which will download the pre-built Android runtimes for you and set up ldc2.conf;
  
cd android/samples/native-activity/
+
* [https://github.com/MrcSnm/D-SDL-Android-Project D SDL Android from MrcSnm|Hipreme] - Provides an Android template project for working with [http://libsdl.org SDL2];
  
$LDC/bin/ldc2 -I../../ -c jni/main.d
+
* [https://github.com/MrcSnm/D-Lang-on-Android D Lang on Android from MrcSnm|Hipreme] - Additional documentation about setting up a D project for [https://developer.android.com/studio Android Studio].
  
$LDC/bin/ldc2 -I../../ -c ../../android/sensor.d
+
==Initializing druntime in shared libraries==
  
$LDC/bin/ldc2 -I../../ -c ../../android_native_app_glue.d
+
If building a shared library and not a D command-line executable, you must initialize and exit the D runtime by calling <tt>rt_init()</tt> and <tt>rt_term()</tt> before and after all D code is run, [https://github.com/Diewi/android/blob/4fbdbb1344725a593d8df1e008ba371c0e694a11/android_native_app_glue.d#L554 as has been done in the default Android wrapper] ([[Runtime internals|<tt>rt_init</tt>/<tt>rt_term</tt> are automatically inserted and run for a D executable]]).
  
mkdir -p libs/armeabi-v7a/
+
==Package an Android app from scratch on your Android device==
  
$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -Wl,-soname,libnative-activity.so
+
Install <tt>aapt</tt>, the Android Asset Packaging Tool, and <tt>apksigner</tt>, a tool to create a hashed manifest and sign your apps.
-shared --sysroot=$NDK/platforms/android-16/arch-arm main.o sensor.o
+
 
android_native_app_glue.o $LDC/lib/libphobos2-ldc.a $LDC/lib/libdruntime-ldc.a
+
<syntaxhighlight lang=bash>
-lgcc -gcc-toolchain $NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64
+
apt install aapt apksigner
-no-canonical-prefixes -fuse-ld=bfd -target armv7-none-linux-androideabi
 
-Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now
 
-llog -landroid -lEGL -lGLESv1_CM -lc -lm -o libs/armeabi-v7a/libnative-activity.so
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Finally, package the app as the SDK directs.  I document the older Ant approach, which is deprecated, replace it with the Gradle command from a newer SDK. With Ant, set the path to your SDK, then run these commands:
+
I'll demonstrate with the <tt>NativeActivity</tt> app built above.
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
export SDK=/path/to/your/android-sdk-linux
+
cd samples/native-activity
$SDK/tools/android update project -p . -s --target 1
+
aapt package -M AndroidManifest.xml -S res -F NativeActivity-debug-unsigned.apk
ant debug
+
APK_DIR=armeabi-v7a # or `arm64-v8a` for 64-bit ARM
 +
aapt add NativeActivity-debug-unsigned.apk libs/$APK_DIR/libnative-activity.so
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Transfer the resulting bin/NativeActivity-debug.apk to your device.
+
This simple app only requires three files, <tt>AndroidManifest.xml</tt>, <tt>resources.arsc</tt>, and <tt>libs/$APK_DIR/libnative-activity.so</tt>, which you can check with the following <tt>aapt</tt> command.
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
cd /sdcard/Download/
+
aapt list NativeActivity-debug-unsigned.apk
scp jo@my.server.com:android/samples/native-activity/bin/NativeActivity-debug.apk .
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===Native compilation===
+
Now let's generate a hashed manifest, just like a Java jar file, and sign the app.  If you have your own Java Keystore already, just supply it to <tt>apksigner</tt>.  If not, <tt>apksigner</tt> will generate a self-signed Keystore file, which we name <tt>debug.ks</tt> below, which is good enough to sign and install debug apps on your own Android device.
 +
 
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
apt install git
+
apksigner debug.ks NativeActivity-debug-unsigned.apk NativeActivity-debug.apk
 +
</syntaxhighlight>
  
git clone https://github.com/joakim-noah/android.git
+
You should see three additional files in the apk, if you list its contents using the command above.  At this point, [[#Install_and_run_the_sample_GUI_app|you can install and run the signed app on your own device]].  If you modify the app, you'll need to build the manifest and sign it again: make sure you use the <tt>debug.ks</tt> you created before or Android won't allow you to reinstall the same app with a newly generated key, unless you first uninstall the app.
  
cd android/samples/native-activity/
+
===Sign your app using a certificate and OpenSSL===
  
ldc2 -I../../ -c jni/main.d
+
Unfortunately, [https://github.com/fornwall/apksigner/issues/1 apksigner only supports Java Keystore files for signing right now] and I don't know how to build one from scratch, so if you don't have a keystore and want to release your app to an app store, you'll have to use OpenSSL to sign the app.
  
ldc2 -I../../ -c ../../android/sensor.d
+
For a valid certificate for the final release, [https://developer.android.com/studio/publish/app-signing.html there's plenty of information online on how to generate one]. I'll just show how to create a self-signed certificate for debugging purposes.
  
ldc2 -I../../ -c ../../android_native_app_glue.d
+
First, install the OpenSSL package in Termux.  Then, this OpenSSL command will generate a self-signed debug certificate, <tt>apk.cert</tt>, and a 2048-bit RSA private key, <tt>key.pem</tt>, which isn't encrypted with a password. It will ask you for some signing info, for which I've shown what's used by the debug certificate in the Android SDK, but it doesn't matter what you enter, as it's ignored:
  
mkdir -p libs/armeabi-v7a/
+
<syntaxhighlight lang=bash>
 +
apt install openssl-tool
  
$PREFIX/bin/clang -Wl,-soname,libnative-activity.so -shared main.o sensor.o
+
openssl req -x509 -nodes -newkey rsa:2048 -keyout key.pem -out apk.cert
android_native_app_glue.o $PREFIX/lib/libphobos2-ldc.a $PREFIX/lib/libdruntime-ldc.a
+
 
-lgcc -no-canonical-prefixes -target armv7-none-linux-androideabi -Wl,--fix-cortex-a8
+
....................................+++
-Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -llog -landroid -lEGL
+
writing new private key to 'key.pem'
-lGLESv1_CM -lc -lm -o libs/armeabi-v7a/libnative-activity.so
+
-----
 +
You are about to be asked to enter information that will be incorporated
 +
into your certificate request.
 +
What you are about to enter is what is called a Distinguished Name or a DN.
 +
There are quite a few fields but you can leave some blank
 +
For some fields there will be a default value,
 +
If you enter '.', the field will be left blank.
 +
-----
 +
Country Name (2 letter code) [AU]:US
 +
State or Province Name (full name) [Some-State]:.
 +
Locality Name (eg, city) []:
 +
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Android
 +
Organizational Unit Name (eg, section) []:
 +
Common Name (e.g. server FQDN or YOUR name) []:Android Debug
 +
Email Address []:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Follow the instructions below to package this native shared library into an Android apk.
+
Now that we have a certificate- self-signed in this case, use your actual release certificate if you want to release the app- and private key, we use them to sign the app.  Since the apk is just a zip file, unzip it into a directory and use OpenSSL to generate a new signature file, <tt>CERT.RSA</tt>, then update the apk with the new signature, and copy the apk to a public user directory from which you can install it:
  
===Install and run the sample GUI app===
+
<syntaxhighlight lang=bash>
Go to Settings->Security and allow installation of apps from unknown sources, ie from outside the Play Store, then go to /sdcard/Download in your file manager and choose the Native Activity apk to install it.  Open the app after installing or go to your app folder and run the app named NativeActivity: it'll show a black screen initially, then flash a bunch of colors when the screen is touched.
+
mkdir unpack
 +
cd unpack/
 +
unzip ../NativeActivity-debug.apk
 +
 
 +
cd META-INF/
 +
openssl smime -sign -md sha1 -binary -noattr -in CERT.SF -out CERT.RSA -outform der -inkey ../../key.pem -signer ../../apk.cert
 +
 
 +
cd ..
 +
aapt remove ../NativeActivity-debug.apk META-INF/CERT.RSA
 +
aapt add ../NativeActivity-debug.apk META-INF/CERT.RSA
 +
 
 +
cd ..
 +
cp NativeActivity-debug.apk /sdcard/Download/
 +
</syntaxhighlight>
 +
 
 +
The OpenSSL commands to generate a certificate and sign the apk were taken [http://qistoph.blogspot.com/2012/01/manual-verify-pkcs7-signed-data-with.html from this 2012 blog post], you can follow it further to see what the signature consists of and verify it for yourself[https://nelenkov.blogspot.com/2013/04/android-code-signing.html This 2013 blog post was critical for me to understand how apk signing works], I used to run all those commands by hand until the <tt>apksigner</tt> package was added to the Termux package repo.
  
==Package an Android app from scratch on your Android device==
+
==Directions for future work==
  
Will fill in this info next...
+
* Fix [https://github.com/ldc-developers/ldc/issues/2153 the remaining stdlib incompatibilities on 64-bit ARM].
  
[[Category: Android]]
+
[[Category: Android]] [[Category: LDC]]

Latest revision as of 20:14, 11 May 2024

These instructions show you how to build D command-line executables and OpenGL ES GUI apps for Android, either by using a desktop D compiler for Windows, Mac, or Linux, or a native Android compiler, all available here. There are separate steps for cross-compilation, ie building apps on a Windows/Linux PC or Mac and running the app on Android, versus native compilation, both building and running on your Android device itself.

Since you cannot install the Android SDK on Android, I end by showing how to package a GUI Android app, a zip file called an .apk, from scratch, by using the tools available in the Termux app for Android, a terminal emulator app and open-source package manager/repository for Android devices.

Prerequisites

Cross-compilation

  • A command shell on your host PC/Mac, where you'll run the LDC D compiler
    • Either a DOS command prompt or Powershell should work on Windows.
    • Any shell should work on Mac and Linux, typical commands for the bash shell are shown.
  • As of LDC v1.38, version r26 of the Android NDK and optionally the SDK. Other versions might work too, but ideally, you'd stick with the NDK used for the official LDC Android packages.
    • The SDK is necessary to package a GUI app; the NDK is enough if you just want to build a command-line binary.
  • A recent LDC compiler for your host platform
  • Android, whether a device or emulator, to run your D code
    • LDC v1.38 has switched to native ELF TLS (thread-local storage), which was introduced in API level 29 (Android v10).
    • The SDK comes with an emulator. I use actual hardware, so that's what I'll discuss.
    • When using a device, you need some way to transfer the app over. There are several ways to do this, here are a few I've tried:
  1. Install an ssh server app on your Android device and scp the app over. Alternately, set up an ssh server on your host PC/Mac, and use an ssh/scp client on Android to get the app. This is what I do, by using the OpenSSH package in Termux.
  2. Host the app in a web server and get it by using your Android browser or a downloader app.
  3. Setup the Android Debug Bridge (adb) on your device and use the SDK tools to push your files over.

Native compilation

  • Android Version 10 ("Q"), as LDC compiler support does not exist for Android Version 9 ("Pie") and earlier.
  • Termux for Android, available in the official Play Store, APKMirror, or F-Droid
  • LDC for Termux: apt install ldc With Android version 9 and earlier, the install will fail.

Cross-compilation setup

Once you have LDC and have unzipped the Android NDK, it's time to set up LDC for the desired Android target(s). See Cross-compiling with LDC for the general guide; I present two examples for a quick summary:

  • Targeting 32-bit Android/ARMv7-A on a Win64 host:
    1. Download the prebuilt android-armv7a package from GitHub matching the version of your LDC.
    2. Extract the lib directory into your LDC installation directory and rename it, e.g., to lib-android_armv7a.
    3. Open <LDC install dir>\etc\ldc2.conf in a text editor and append a section like this, adapting lib and NDK paths as needed:
"armv7a-.*-linux-android":
{
    switches = [
        "-defaultlib=phobos2-ldc,druntime-ldc",
        "-gcc=C:/LDC/android-ndk-r26d/toolchains/llvm/prebuilt/windows-x86_64/bin/armv7a-linux-androideabi29-clang.cmd",
    ];
    lib-dirs = [
        "%%ldcbinarypath%%/../lib-android_armv7a",
    ];
};
  • Targeting 64-bit Android/AArch64 on a Linux host:
    1. Download the prebuilt android-aarch64 package from GitHub matching the version of your LDC.
    2. Extract the lib directory into your LDC installation directory and rename it, e.g., to lib-android_aarch64.
    3. Open <LDC install dir>/etc/ldc2.conf in a text editor and append a section like this, adapting lib and NDK paths as needed:
"aarch64-.*-linux-android":
{
    switches = [
        "-defaultlib=phobos2-ldc,druntime-ldc",
        "-gcc=/home/me/android-ndk-r26d/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android30-clang",
    ];
    lib-dirs = [
        "%%ldcbinarypath%%/../lib-android_aarch64",
    ];
};

The prebuilt Android packages also include the corresponding x86 simulator libraries, so 32/64-bit x86 Android simulator targets can be set up the same way.

Build a command-line executable

Now that we have a D compiler setup for (one or more) Android targets, let's try building a small program, the classic Sieve of Eratosthenes single-core benchmark, which finds all prime numbers up to a number you choose. Install the curl package in Termux if you're natively compiling, apt install curl.

# Load this link in your browser and download the file otherwise 
curl -L -O https://raw.githubusercontent.com/dlang/dmd/master/samples/sieve.d

# Cross-compile & -link to ARMv7-A (on any host)
ldc2 -mtriple=armv7a--linux-androideabi29 sieve.d

# Cross-compile & -link to AArch64 (on any host)
ldc2 -mtriple=aarch64--linux-android29 sieve.d

# Compile & link natively in Termux
ldc2 sieve.d

Cross-compilation

Copy this sieve program onto an Android device or emulator and set its permissions with the chmod command. Here's how I do it in Termux, with an ssh server running on the host PC/Mac with IP address 192.168.1.37:

apt install openssh
cd
scp jo@192.168.1.37:sieve .
chmod 700 sieve

Run the sieve program

The sieve program will tell you how many prime numbers there are in the first n integers, a limit you can specify. Run this command to find how many primes there are in the first million integers:

./sieve 1000000

If you built sieve successfully, it should return

78498 primes

Build a sample OpenGL ES 1.0 GUI app ported to D

Clone the android repository or download its source in a zip file, which contains several headers and sample OpenGL apps from the NDK translated to D:

sudo apt-get install git # In Termux, apt install git
git clone https://github.com/Diewi/android.git
cd android

# Alternatively, without git:
curl -L -O https://github.com/Diewi/android/archive/build.zip
unzip build.zip
cd android-build

Then build the Native Activity app, which is written completely in D. D code for an apk must be compiled to a shared library, which the Android runtime will call:

cd samples/native-activity

ldc2 -I../.. jni/main.d ../../android/sensor.d ../../android_native_app_glue.d \
     -shared -of=libs/arm64-v8a/libnative-activity.so \ # or `libs/armeabi-v7a/...` for 32-bit ARM
     -L-soname -Llibnative-activity.so \
     -mtriple=aarch64--linux-android29 # only for cross-compilation; use `armv7a--linux-androideabi29` for 32-bit ARM
     # possibly needed: -L-llog -L-landroid -L-lEGL -L-lGLESv1_CM

Cross-compilation

Ant approach

Finally, package the app as the SDK directs: at this point, it's just like building a regular Android app. I document the older Ant approach, which is deprecated, replace it with the Gradle command from a newer SDK. With Ant on Mac or Linux, set the path to your SDK, then run these commands:

export SDK=/path/to/your/android-sdk
$SDK/tools/android update project -p . -s --target 1
ant debug

Android Studio approach

Enter on your <application name>/app/src/main and create a folder called jniLibs, this folder is extremely important, it is the default folder to put your shared libraries to be imported together with your .apk. If you wish to use other name for it, you will need to change your gradle file. For actually putting your libraries inside that folder, you will actually need to make directories for the target architectures, so, create inside it:

  • armeabi-v7a (For that, it is commonly used ldc2(version)-android-armv7a/lib
  • arm64-v8a (This is our target right now, ldc2(version)-android-aarch64/lib)
  • x86 (It is the lib32 for armv7a -> ldc2(version)-android-armv7a/lib686
  • x86_64 (It is the lib32 for the aarch64 -> ldc2(version)-android-aarch64/lib-x86_64 For reference, check ndk abi guide from official android site: Android ABI Guide

After creating those folders, you can actually move your shared library inside one of them, just click on run and you're good to go.

---

Transfer the resulting bin/NativeActivity-debug.apk to your Android device, again shown here by using scp from the Termux app.

scp jo@192.168.1.37:android/samples/native-activity/bin/NativeActivity-debug.apk /sdcard/Download/

Native compilation

Follow the instructions below to package this native shared library into an Android apk.

Install and run the sample GUI app

Go to Settings->Security on your Android device and allow installation of apps from unknown sources, ie from outside the Play Store, then go to /sdcard/Download in your file manager and choose the NativeActivity-debug apk to install it. Open the app after installing or go to your app folder and run the app named NativeActivity: it'll show a black screen initially, then flash a bunch of colors when the screen is touched.

Build a sample OpenGL ES 2.0 GUI app mostly written in D, with some Java

This D app has not been ported to 64-bit Android/ARM yet, only 32-bit ARM compilation will work for now:

cd samples/Teapot

ldc2 -I../.. -Ijni -Jjni \
     ../../ndk_helper/GLContext.d \
     ../../ndk_helper/JNIHelper.d \
     ../../ndk_helper/gestureDetector.d \
     ../../ndk_helper/perfMonitor.d \
     ../../ndk_helper/shader.d \
     ../../ndk_helper/tapCamera.d \
     jni/TeapotNativeActivity.d \
     jni/TeapotRenderer.d \
     ../../android/sensor.d \
     ../../android_native_app_glue.d \
     -shared -of=libs/armeabi-v7a/libTeapotNativeActivity.so \
     -L-soname -LlibTeapotNativeActivity.so \
     -mtriple=armv7a--linux-androideabi29 # only for cross-compilation
     # possibly needed: -L-llog -L-landroid -L-lEGL -L-lGLESv2

Cross-compilation

Package this shared library into an apk by using the SDK, as you would normally, and try installing and running it on your device.

Native compilation

Install the right Eclipse Java compiler package for your device (the ecj4.6 package if you're running Android 5 or 6), the Android dex tool, and other packages needed to build an Android apk. Generate any Java files needed, compile and dex them, then package everything up into an apk and sign it.

apt install ecj dx aapt apksigner

aapt package  -M ./AndroidManifest.xml -I $PREFIX/share/java/android-21.jar -J src/ -S res -m

ecj-21 -d ./obj -sourcepath src $(find src -type f -name "*.java")

dx --dex --output=./classes.dex ./obj/

aapt package  -M ./AndroidManifest.xml -S res -A assets -F teapot.apk

aapt add teapot.apk classes.dex lib/armeabi-v7a/libTeapotNativeActivity.so

apksigner debug.ks teapot.apk teapot-signed.apk

Finally, move teapot-signed.apk into a public directory, from which you can install and run it.

Examples and useful repositories

  • D Android from adamdruppe - You can find android-setup.d which will download the pre-built Android runtimes for you and set up ldc2.conf;

Initializing druntime in shared libraries

If building a shared library and not a D command-line executable, you must initialize and exit the D runtime by calling rt_init() and rt_term() before and after all D code is run, as has been done in the default Android wrapper (rt_init/rt_term are automatically inserted and run for a D executable).

Package an Android app from scratch on your Android device

Install aapt, the Android Asset Packaging Tool, and apksigner, a tool to create a hashed manifest and sign your apps.

apt install aapt apksigner

I'll demonstrate with the NativeActivity app built above.

cd samples/native-activity
aapt package -M AndroidManifest.xml -S res -F NativeActivity-debug-unsigned.apk
APK_DIR=armeabi-v7a # or `arm64-v8a` for 64-bit ARM
aapt add NativeActivity-debug-unsigned.apk libs/$APK_DIR/libnative-activity.so

This simple app only requires three files, AndroidManifest.xml, resources.arsc, and libs/$APK_DIR/libnative-activity.so, which you can check with the following aapt command.

aapt list NativeActivity-debug-unsigned.apk

Now let's generate a hashed manifest, just like a Java jar file, and sign the app. If you have your own Java Keystore already, just supply it to apksigner. If not, apksigner will generate a self-signed Keystore file, which we name debug.ks below, which is good enough to sign and install debug apps on your own Android device.

apksigner debug.ks NativeActivity-debug-unsigned.apk NativeActivity-debug.apk

You should see three additional files in the apk, if you list its contents using the command above. At this point, you can install and run the signed app on your own device. If you modify the app, you'll need to build the manifest and sign it again: make sure you use the debug.ks you created before or Android won't allow you to reinstall the same app with a newly generated key, unless you first uninstall the app.

Sign your app using a certificate and OpenSSL

Unfortunately, apksigner only supports Java Keystore files for signing right now and I don't know how to build one from scratch, so if you don't have a keystore and want to release your app to an app store, you'll have to use OpenSSL to sign the app.

For a valid certificate for the final release, there's plenty of information online on how to generate one. I'll just show how to create a self-signed certificate for debugging purposes.

First, install the OpenSSL package in Termux. Then, this OpenSSL command will generate a self-signed debug certificate, apk.cert, and a 2048-bit RSA private key, key.pem, which isn't encrypted with a password. It will ask you for some signing info, for which I've shown what's used by the debug certificate in the Android SDK, but it doesn't matter what you enter, as it's ignored:

apt install openssl-tool

openssl req -x509 -nodes -newkey rsa:2048 -keyout key.pem -out apk.cert

....................................+++
writing new private key to 'key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:.
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Android
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:Android Debug
Email Address []:

Now that we have a certificate- self-signed in this case, use your actual release certificate if you want to release the app- and private key, we use them to sign the app. Since the apk is just a zip file, unzip it into a directory and use OpenSSL to generate a new signature file, CERT.RSA, then update the apk with the new signature, and copy the apk to a public user directory from which you can install it:

mkdir unpack
cd unpack/
unzip ../NativeActivity-debug.apk

cd META-INF/
openssl smime -sign -md sha1 -binary -noattr -in CERT.SF -out CERT.RSA -outform der -inkey ../../key.pem -signer ../../apk.cert

cd ..
aapt remove ../NativeActivity-debug.apk META-INF/CERT.RSA
aapt add ../NativeActivity-debug.apk META-INF/CERT.RSA

cd ..
cp NativeActivity-debug.apk /sdcard/Download/

The OpenSSL commands to generate a certificate and sign the apk were taken from this 2012 blog post, you can follow it further to see what the signature consists of and verify it for yourself. This 2013 blog post was critical for me to understand how apk signing works, I used to run all those commands by hand until the apksigner package was added to the Termux package repo.

Directions for future work