D on esp32/esp8266(llvm-xtensa+ldc) and how to get started

The Rust community has proved that Rust is running on ESP32. I just looked at that tutorial and then had an idea: is D's betterC running on ESP32? hmm, it should be! For those who don't know what ESP32 is:

  ESP32 is a series of low-cost, low-power consumption WiFi/BLE soc chips with up to a 240Mhz 32bit Xtensa CPU, 520KB RAM and SPI nor flash support, and it's a star in the IoT device market and has powerful open embedded software ecosystem supporting C, C++, MicroPython, Lua and even Javascript.

I took the Rust ESP32 tutorial's idea with building llvm+clang for xtensa and referenced the component integration with ESP-IDF(esp32 offical C SDK), and finally got a success.

Since 2019 the esp toolchain source has changed a lot and it's much easier to compile the ldc compiler against it.

I put the build script and some example projects in a git repository at: https://github.com/Reavershark/esp-dlang.

The rest of this page discusses those files in more detail.

Build script

Note that you might have to temporarily uninstall llvm, clang and ldc from your system first before running this script:

#!/usr/bin/env bash

set -e

function gcl
{
    git clone --recurse-submodules --depth 1 $@
}

[[ -d llvm-source ]] && rm -rf llvm-source
[[ -d llvm-build ]] && rm -rf llvm-build
gcl --branch "esp-15.0.0-20230404" "https://github.com/espressif/llvm-project.git" llvm-source
cmake \
    -S llvm-source/llvm \
    -B llvm-build \
    -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="Xtensa" \
    -DLLVM_ENABLE_PROJECTS="clang" \
    -DCMAKE_INSTALL_PREFIX=/opt/llvm-xtensa \
    -DCMAKE_C_COMPILER=gcc \
    -DCMAKE_CXX_COMPILER=g++ \
    -DCMAKE_ASM_COMPILER=gcc
cmake --build llvm-build
sudo cmake --install llvm-build

[[ -d ldc-source ]] && rm -rf ldc-source
[[ -d ldc-build ]] && rm -rf ldc-build
gcl --branch "v1.32.2" "https://github.com/ldc-developers/ldc.git" ldc-source
cmake \
    -S ldc-source \
    -B ldc-build \
    -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=/opt/ldc-xtensa \
    -DLLVM_ROOT_DIR=/opt/llvm-xtensa \
    -DCMAKE_C_COMPILER=gcc \
    -DCMAKE_CXX_COMPILER=g++ \
    -DCMAKE_ASM_COMPILER=gcc
cmake --build ldc-build
sudo cmake --install ldc-build

Espressif is already at llvm 16, while ldc just started using llvm 15 in the latest release at this time, so I let the script use espressifs latest llvm 15 fork and ldc version 1.32.2.

The first block (right after the gcl function) downloads the espressif fork of llvm. Using the sources in this repository, the llvm backend and the clang frontend are compiled and installed in /opt/llvm-xtensa.

The second block downloads ldc and compiles it against the llvm backend we just built. It will be installed in /opt/ldc-xtensa.

ldc2.conf

Finally, you will need to edit the /opt/ldc-xtensa/etc/ldc2.conf file so it looks like this:

default:
{
    // default switches injected before all explicit command-line switches
    switches = [
        "-mtriple=xtensa-esp32-elf",
        "-mcpu=esp32",
        "-gcc=/opt/llvm-xtensa/bin/clang",
        "-betterC",
        "-linkonce-templates",
        "-dip1000",
        "-boundscheck=off",
    ];
    // default switches appended after all explicit command-line switches
    post-switches = [
        "-I/opt/ldc-xtensa/include/d",
    ];
    // default directories to be searched for libraries when linking
    lib-dirs = [
        "/opt/ldc-xtensa/lib",
    ];
    // default rpath when linking against the shared default libs
    rpath = "/opt/ldc-xtensa/lib";
};

Testing

Write some D betterC code:

void d_func()
{
    int[4] arr;
    auto c = arr.length;
}

And build the D betterC code like this:

$ /opt/ldc-xtensa/bin/ldc2 -c --of=test.o test.d

$ file test.o
test.o: ELF 32-bit LSB relocatable, Tensilica Xtensa, version 1 (SYSV), not stripped

$ readelf -s  test.o
Symbol table '.symtab' contains 4 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 00000000     0 FILE    LOCAL  DEFAULT  ABS test.d
     2: 00000000     0 SECTION LOCAL  DEFAULT    4 
     3: 00000000    24 FUNC    GLOBAL DEFAULT    4 _D4test6d_funcFZv

Integrating with IDF

ESP IDF is an sdk for building applications in C and flashing them on ESP devices. It uses a component-based system on top of FreeRTOS.

See the example project at: https://github.com/Reavershark/esp-dlang/tree/master/examples/idf-project.

You will need to install esp-idf locally before you can use it.

The example project looks like this:

.
├── build.sh
├── CMakeLists.txt
└── main
    ├── CMakeLists.txt
    ├── dub.json
    ├── dummy.c
    └── source
        └── main.d

The entry point we have to provide to FreeRTOS is an app_main C function. This is defined in main.d:

module main;

extern(C) int printf(scope const char* fmt, scope const ...);

extern(C) void app_main()
{
    int i = 0;
    printf("Hello, dlang says: i = %d\r\n", i);
}

The CMakeLists.txt in the project root just gives the project a name and defines additional components besides main.

The CMakeLists.txt in the main component folder is just a component with a single empty C file that links against libmain.a. This file is built with dub build in the build.sh script.

You could add more C or D components to the project by creating new directories, adding them in the root CMakeLists.txt, and for D components, adding them to the build script (for dir in "main" "comp2" "comp3"...).

Running on bare metal

ToDo: Example for building and flashing without IDF/FreeRTOS.