Difference between revisions of "Building LDC from source"

From D Wiki
Jump to: navigation, search
m (Emphasize dedicated Windows build page)
(LLVM: Revise, focusing more on non-contributors while providing more details for contributors)
Line 30: Line 30:
 
=== LLVM ===
 
=== LLVM ===
  
Many Linux distributions already provide recent binary LLVM packages, sometimes in the form of user-curated package repositories (PPA, …). If a recent LLVM package is available, you might prefer to use it, as LLVM is a rather big project to build. Only the Android target requires building [https://github.com/ldc-developers/llvm/releases our lightly tweaked version of LLVM], which is what we'll use here. There are also pre-built binary tarballs of our tweaked LLVM at that link, but they don't always work in other build environments, so we lay out the steps below in case you can't use them.
+
The minimum supported LLVM version as of October 2019 is 3.9. Many Linux distributions already provide recent LLVM dev packages, sometimes in the form of user-curated package repositories (PPA, …). If a recent LLVM package is available, you might prefer to use it, as LLVM is a rather big project to build, e.g., via <code>apt-get install llvm-dev libclang-common-<matching LLVM version>-dev</code>. Only the Android target requires building [https://github.com/ldc-developers/llvm/releases our lightly tweaked version of LLVM], which is what we'll use here. There are also pre-built binary tarballs of our tweaked LLVM at that link, but they don't always work in other build environments, so we lay out the steps below in case you can't use them.
  
==== Building LLVM from source manually ====
+
==== Building LLVM from source ====
 
 
We try to keep LDC up-to-date with LLVM trunk, but the latest official release is recommended for the least amount of trouble (with LLVM trunk, you will have to recompile LLVM often). Download a [https://github.com/ldc-developers/llvm/releases lightly tweaked source tarball], extract the archive, configure and build:
 
  
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
curl -L -O https://github.com/ldc-developers/llvm/releases/download/ldc-v8.0.0/llvm-8.0.0.src.tar.xz
+
# Non-Apple platforms: install binutils-dev package required to generate the ld.gold LTO linker plugin
tar xf llvm-8.0.0.src.tar.xz
+
apt-get install binutils-dev
cd llvm-8.0.0.src
 
mkdir build && cd build
 
  
# remove `-G Ninja` to use Make instead
+
# Download & extract source tarball of our lightly tweaked LLVM
 +
curl -OL https://github.com/ldc-developers/llvm/releases/download/ldc-v9.0.0/llvm-9.0.0.src.tar.xz
 +
tar -xf llvm-9.0.0.src.tar.xz
  
cmake -G Ninja .. \
+
# Generate Ninja build files; remove `-G Ninja` to use Make instead
   -DCMAKE_BUILD_TYPE=RelWithDebInfo \
+
mkdir build-llvm && cd build-llvm # using a fresh new build dir is highly recommended whenever re-invoking CMake
   -DLLVM_ENABLE_ASSERTIONS=ON \
+
cmake -G Ninja ../llvm-9.0.0.src \
 +
   -DCMAKE_BUILD_TYPE=Release \
 +
   -DCMAKE_INSTALL_PREFIX=$PWD/../install-llvm \
 +
  -DLLVM_BINUTILS_INCDIR=/usr/include \ # non-Apple only: location of binutils-dev's plugin.h
 +
  -DLLVM_TARGETS_TO_BUILD='AArch64;ARM;Mips;MSP430;NVPTX;PowerPC;RISCV;WebAssembly;X86' \ # defaults to `all` if not specified
 
   -DCOMPILER_RT_INCLUDE_TESTS=OFF \
 
   -DCOMPILER_RT_INCLUDE_TESTS=OFF \
   -DLLVM_BINUTILS_INCDIR=/usr/include \
+
   -DLLVM_INCLUDE_TESTS=OFF
  -DCMAKE_INSTALL_PREFIX={/my/install/dir}
 
ninja
 
ninja install      # This will install LLVM on the {/my/install/dir} directory specified in the -DCMAKE_INSTALL_PREFIX above.
 
  
cd ../..
+
# Build and install (to directory specified as CMAKE_INSTALL_PREFIX above)
 +
ninja install # or `make install`
 +
cd ..
 
</syntaxhighlight>
 
</syntaxhighlight>
  
If you are planning to work on LDC itself, another option is to install a debug build of LLVM instead by using <tt>-DCMAKE_BUILD_TYPE=Debug</tt>. Warning: This leads to a ''heavy'' slowdown!
+
Check out the [http://llvm.org/docs/CMake.html#llvm-specific-variables LLVM CMake page] for more CMake variables.
So, the one used above is <tt>-DCMAKE_BUILD_TYPE=RelWithDebInfo</tt>. This results in less heavy build but with adequate debug info.
 
 
 
If you are building natively in Termux for Android, you'll want to add <tt>-DLLVM_DEFAULT_TARGET_TRIPLE=armv7-none-linux-android</tt>, because CMake cannot detect the Android platform yet.
 
 
 
==== Additional LLVM CMake Options and hints ====
 
 
 
The following additional CMake configuration options are available. It builds LLVM in release mode without debug symbols, which might save some spac eon disk:
 
* -DCMAKE_BUILD_TYPE=Release
 
  
The supported LLVM targets can be selected with the following options. The defaults are to
+
If you are planning to work on LDC itself, we recommend <tt>-DCMAKE_BUILD_TYPE=RelWithDebInfo</tt> for the (huge!) LLVM debuginfos and <tt>-DLLVM_ENABLE_ASSERTIONS=ON</tt>. LLVM's assertions mode also controls LDC's assertions (in both C++ and D parts of LDC). A debug build of LLVM (<tt>-DCMAKE_BUILD_TYPE=Debug</tt>) leads to a ''heavy'' slowdown of LDC's compilation speed. LLVM's <tt>CMAKE_BUILD_TYPE</tt> also controls how the C++ parts of LDC are compiled (LDC inherits the C++ compiler flags used to build LLVM).
build all official targets. Release 9 of LLVM adds RIC-V to the official targets. It must be
 
given as experimental target for all previous version, if needed.
 
* -DLLVM_TARGETS_TO_BUILD="AArch64;ARM;Mips;MSP430;NVPTX;PowerPC;X86;{other targets...}"
 
* -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="RISCV;WebAssembly;{other targets...}"
 
  
The default path for CMAKE_INSTALL_PREFIX is "/usr/local/". This path is usually part of the default search paths for includes, libs and binaries. As long as no other LLVM binaries are installed to the system, this might a suitable location. Otherwise keep in mind, that installing
+
If you are building natively in Termux for Android, you'll want to specify a proper default triple like <tt>-DLLVM_DEFAULT_TARGET_TRIPLE=armv7a-unknown-linux-androideabi</tt>.
different LLVM binaries in parallel might cause problems, as libs and binaries could be messed
 
up in the compile and link runs, causing cryptic errors and problems. You might need to change search paths for libs etc. to avoid these problems. It is recommended to use some custom location
 
for installation, and to use it for the LDC build. See instructions below.
 
  
 
== Building LDC from source manually ==
 
== Building LDC from source manually ==

Revision as of 16:22, 6 October 2019

This page shows you how to build and install LDC on most Posix-like systems such as linux, macOS, BSD, or Android. For building LDC on Windows, please see its dedicated page.

Advice

It is hard for us to keep these wiki pages up-to-date. If you run into trouble, have a look at the build scripts for our Continuous Integration platforms: the Azure Pipelines scripts for Ubuntu Linux and macOS and Windows are always up-to-date with the latest build setup.

Prerequisites

  • Git (for fetching the source code, if not using a tarball)
  • a C++ toolchain (GCC, Clang, …)
  • a D compiler (LDC/DMD/GDC)
    • If there's no suitable prebuilt D compiler for your platform: LDC 0.17 is the last version that does not need a D compiler to be built. Thus for bootstrapping, you can first build 0.17, and then use that to build newer compiler versions. Our testing infrastructure explicitly tests that new LDC versions can be built with 0.17 (on 64-bit systems). The git branch is called ltsmaster or you can get the source for the latest 0.17 release. Note that it doesn't support the latest LLVM versions.
  • CMake 3.8+
  • Ninja or Make (Ninja is highly recommended as it builds in parallel by default and doesn't suffer from concurrency issues wrt. CMake custom commands)
  • Python
  • libcurl for Phobos' std.net.curl (e.g., libcurl4 on recent Ubuntu)
  • zlib-dev (e.g., zlib1g-dev on Ubuntu)
  • For 0.17 ltsmaster:
    • libconfig++: get the libconfig-devel or libconfig-dev package for some Linux distributions. On OSX, sudo port install libconfig-hr; on Android with the Termux app, pkg install libconfig-dev.
    • libcurl-dev (e.g., libcurl4-gnutls-dev on Ubuntu)
  • For the tests: gdb, unzip, zip and tzdata
    • For BSD: bash and GNU make

On Ubuntu 18.04, this amounts to:

apt-get install git-core g++ ldc cmake ninja-build zlib1g-dev libcurl4 \
                gdb unzip zip tzdata # only required for the tests

LLVM

The minimum supported LLVM version as of October 2019 is 3.9. Many Linux distributions already provide recent LLVM dev packages, sometimes in the form of user-curated package repositories (PPA, …). If a recent LLVM package is available, you might prefer to use it, as LLVM is a rather big project to build, e.g., via apt-get install llvm-dev libclang-common-<matching LLVM version>-dev. Only the Android target requires building our lightly tweaked version of LLVM, which is what we'll use here. There are also pre-built binary tarballs of our tweaked LLVM at that link, but they don't always work in other build environments, so we lay out the steps below in case you can't use them.

Building LLVM from source

# Non-Apple platforms: install binutils-dev package required to generate the ld.gold LTO linker plugin
apt-get install binutils-dev

# Download & extract source tarball of our lightly tweaked LLVM
curl -OL https://github.com/ldc-developers/llvm/releases/download/ldc-v9.0.0/llvm-9.0.0.src.tar.xz
tar -xf llvm-9.0.0.src.tar.xz

# Generate Ninja build files; remove `-G Ninja` to use Make instead
mkdir build-llvm && cd build-llvm # using a fresh new build dir is highly recommended whenever re-invoking CMake
cmake -G Ninja ../llvm-9.0.0.src \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=$PWD/../install-llvm \
  -DLLVM_BINUTILS_INCDIR=/usr/include \ # non-Apple only: location of binutils-dev's plugin.h
  -DLLVM_TARGETS_TO_BUILD='AArch64;ARM;Mips;MSP430;NVPTX;PowerPC;RISCV;WebAssembly;X86' \ # defaults to `all` if not specified
  -DCOMPILER_RT_INCLUDE_TESTS=OFF \
  -DLLVM_INCLUDE_TESTS=OFF

# Build and install (to directory specified as CMAKE_INSTALL_PREFIX above)
ninja install # or `make install`
cd ..

Check out the LLVM CMake page for more CMake variables.

If you are planning to work on LDC itself, we recommend -DCMAKE_BUILD_TYPE=RelWithDebInfo for the (huge!) LLVM debuginfos and -DLLVM_ENABLE_ASSERTIONS=ON. LLVM's assertions mode also controls LDC's assertions (in both C++ and D parts of LDC). A debug build of LLVM (-DCMAKE_BUILD_TYPE=Debug) leads to a heavy slowdown of LDC's compilation speed. LLVM's CMAKE_BUILD_TYPE also controls how the C++ parts of LDC are compiled (LDC inherits the C++ compiler flags used to build LLVM).

If you are building natively in Termux for Android, you'll want to specify a proper default triple like -DLLVM_DEFAULT_TARGET_TRIPLE=armv7a-unknown-linux-androideabi.

Building LDC from source manually

Now that you're ready to build and install LDC from source, clone the LDC GitHub repository or get one of our official source releases:

$ git clone --recursive https://github.com/ldc-developers/ldc.git

If you're behind a company firewall and cloning of the submodules fails, first configure git to use a different protocol, ex https:

$ git config --global url."https://github".insteadOf git://github

If you already have the git repo, don’t forget to make sure your submodules are up to date by running git submodule update --init.

Run the following commands to configure and build ldc and its runtime libraries (see the list of useful CMake switches below):

cd ldc

# Make a working directory for the build (name/path arbitrary).
mkdir build && cd build

# If host D compiler is not on path, explicitly specify dmd/ldmd2
# (not required for ltsmaster/0.17.x).
export DMD=/path/to/your/dmd2/bin/dmd

# 1) Run CMake, giving path to top-level source directory. (Remove
# -G Ninja to use default generator, i.e. make.)
# 2) {/LLVM/install/dir} is the directory where you installed LLVM if you followed the instructions above
# (i.e. the path you set the -DCMAKE_INSTALL_PREFIX equal to).
cmake -G Ninja -DLDC_DYNAMIC_COMPILE=False -DLLVM_ROOT_DIR={/LLVM/install/dir} ..

# Build and install LDC. Use -j<n> to limit parallelism if running out of memory.
ninja
sudo ninja install

The last step is optional; instead of installing it to the system, you can also choose to run LDC from the bin/ directory in your CMake working tree.

You can specify installation location for LDC too. Set again the CMAKE_INSTALL_PREFIX accordingly.

If you need the experimental JIT functionality, you should set -DLDC_DYNAMIC_COMPILE=True. In the average case it has caused problems to the users hence it is set to False in the cmdline above.

If you're planning to work on LDC, you can set -DCMAKE_BUILD_TYPE=RelWithDebInfo here too.

Note: When you issue ninja install, that installs LDC on your system and puts the binary e.g. in /local/bin/ldc. When you issue ninja, that generates a ldc2 binary in build/bin. You can config CMake with -DCMAKE_BUILD_TYPE=Release and issue ninja install to install LDC once. Then, you can delete the build directory, and repeat the steps above but now use -DCMAKE_BUILD_TYPE=RelWithDebInfo. That way, while working on LDC, with ninja, you'll be getting a debug binary in build/bin but have a release version in your system.

If you want to target Android and are building ldc 1.4 or later, add -DLDC_TARGET_PRESET=Android-arm to the CMake config.

If cross-compiling the runtime libraries, you'll need to specify the C cross-compiler before running CMake

$ export CC=/home/david/android-ndk-r17b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang

and pass any C, D, or linker flags you need to CMake:

-DRT_CFLAGS="-target armv7-none-linux-gnueabihf -Os" -DD_FLAGS="-w;-mtriple=armv7-none-linux-gnueabihf" -DLD_FLAGS="-target armv7-none-linux-gnueabihf -fpie -pie"

Useful CMake variables

  • LIB_SUFFIX: Some Linux distributions, such as Fedora, expect 64 bit libraries in /usr/lib64 instead of /usr/lib. In this case, the installation directory can be adjusted using -DLIB_SUFFIX=64.
  • CMAKE_INSTALL_PREFIX: The installation prefix, /usr/local by default (e.g. -DCMAKE_INSTALL_PREFIX=/opt/ldc).
  • INCLUDE_INSTALL_DIR: The location the D modules for druntime and Phobos are installed to.
  • RUNTIME_DIR, PHOBOS2_DIR: By default, druntime and Phobos are expected in runtime/ as git submodules. Should circumstances require it, these paths can be changed by setting the variables accordingly.
  • LLVM_ROOT_DIR and LLVM_CONFIG: Allows you to specify the LLVM instance to use. LLVM_CONFIG specifies the path and name of the llvm-config binary to use. By default, it is assumed to be ${LLVM_ROOT_DIR}/bin/llvm-config, otherwise it is searched for on default system paths. EDIT: https://github.com/ldc-developers/ldc/issues/1928#issuecomment-268421779 suggests we should just use `ccmake -DLLVM_ROOT_DIR=$homebrew_D/ ..` on ubuntu 14.04 even if /usr/bin/llvm-config-3.8 is available
  • LIBCONFIG_LIBRARY and LIBCONFIG_INCLUDE_DIR: Only for 0.17 ltsmaster, these variables can be used to specify the location of the libconfig++ library files and the path to the corresponding header files. NOTE: on error Could NOT find LibConfig (missing: LIBCONFIG_INCLUDE_DIR LIBCONFIG_LIBRARY) and using brew, use for eg: CMAKE_PREFIX_PATH=`brew --prefix` cmake .. [see https://github.com/ldc-developers/ldc/issues/952] or use `sudo apt-get install libconfig++`
  • D_COMPILER: path to prebuilt D compiler, needed for anything newer than 0.17 ltsmaster
  • BUILD_LTO_LIBS: Set this to 'ON' to build phobos and druntime with LTO. Available on MacOS and Linux starting with LDC 1.9.0. Include D_FLAGS='-w;-flto=thin' to enable ThinLTO (so pass -DBUILD_LTO_LIBS=ON -DD_FLAGS='-w;-flto=thin' to cmake). In LDC 1.12.0 ThinLTO will be included automatically, so the D_FLAGS variable won't be necessary. LDC 1.12.0 will also support Win64 (PR 2774).

NOTE: see https://github.com/Linuxbrew/homebrew-core/blob/master/Formula/ldc.rb for brew's install

Tips

The Makefiles generated by CMake respect the DESTDIR variable for the install target. It is prepended to all the file installation targets. This can be useful for building packages:

$ make install DESTDIR=<your root directory>