Difference between revisions of "Building LDC from source"

From D Wiki
Jump to: navigation, search
(Building LDC from source manually: Revise, incl. removal of obsolete instructions)
m (More useful CMake variables: re-format table)
(11 intermediate revisions by 2 users not shown)
Line 9: Line 9:
 
* a C++ toolchain (GCC, Clang, …)
 
* a C++ toolchain (GCC, Clang, …)
 
* a D compiler (LDC/DMD/GDC)
 
* a D compiler (LDC/DMD/GDC)
** If there's no suitable prebuilt D compiler for your platform: LDC 0.17 is the last version that does not need a D compiler to be built. Thus for bootstrapping, you can first build 0.17, and then use that to build newer compiler versions. Our testing infrastructure explicitly tests that new LDC versions can be built with 0.17 (on 64-bit systems). The git branch is called <tt>ltsmaster</tt> or [https://github.com/ldc-developers/ldc/releases you can get the source for the latest 0.17 release]. Note that it doesn't support the latest LLVM versions.
+
** If there's no suitable prebuilt D compiler for your platform: LDC 0.17 is the last version that does not need a D compiler to be built. Thus for bootstrapping, you can first build 0.17, and then use that to build newer compiler versions. Our testing infrastructure explicitly tests that new LDC versions can be built with 0.17 (on 64-bit systems). The git branch is called <tt>ltsmaster</tt> or [https://github.com/ldc-developers/ldc/releases/tag/v0.17.6 you can get the source for the latest 0.17 release]. Note that it doesn't support the latest LLVM versions.
 
* CMake 3.8+
 
* CMake 3.8+
 
* [https://github.com/ninja-build/ninja/releases Ninja] or Make (Ninja is highly recommended as it builds in parallel by default and doesn't suffer from concurrency issues wrt. CMake custom commands)
 
* [https://github.com/ninja-build/ninja/releases Ninja] or Make (Ninja is highly recommended as it builds in parallel by default and doesn't suffer from concurrency issues wrt. CMake custom commands)
Line 16: Line 16:
 
* zlib-dev (e.g., <tt>zlib1g-dev</tt> on Ubuntu)
 
* zlib-dev (e.g., <tt>zlib1g-dev</tt> on Ubuntu)
 
* For 0.17 ltsmaster:
 
* For 0.17 ltsmaster:
** libconfig++: get the <tt>libconfig-devel</tt> or <tt>libconfig-dev</tt> package for some Linux distributions. On OSX, <tt>sudo port install libconfig-hr</tt>; on Android with the Termux app, <tt>pkg install libconfig-dev</tt>.
+
** libconfig++: get the <tt>libconfig-devel</tt> or <tt>libconfig-dev</tt> package for some Linux distributions. On OSX, <tt>sudo port install libconfig-hr</tt>; on Android with the Termux app, <tt>apt install libconfig-dev</tt>.
 
** libcurl-dev (e.g., <tt>libcurl4-gnutls-dev</tt> on Ubuntu)
 
** libcurl-dev (e.g., <tt>libcurl4-gnutls-dev</tt> on Ubuntu)
 
* For the tests: gdb, unzip, zip and tzdata
 
* For the tests: gdb, unzip, zip and tzdata
Line 30: Line 30:
 
=== LLVM ===
 
=== LLVM ===
  
The minimum supported LLVM version as of October 2019 is 3.9. Many Linux distributions already provide recent LLVM dev packages, sometimes in the form of user-curated package repositories (PPA, …). If a recent LLVM package is available, you might prefer to use it, as LLVM is a rather big project to build, e.g., via <code>apt-get install llvm-dev libclang-common-<matching LLVM version>-dev</code>. Only the Android target requires building [https://github.com/ldc-developers/llvm/releases our lightly tweaked version of LLVM], which is what we'll use here. There are also pre-built binary tarballs of our tweaked LLVM at that link, but they don't always work in other build environments, so we lay out the steps below in case you can't use them.
+
The minimum supported LLVM version as of October 2019 is 3.9. Many Linux distributions already provide recent LLVM dev packages, sometimes in the form of user-curated package repositories (PPA, …). If a recent LLVM package is available, you might prefer to use it, as LLVM is a rather big project to build, e.g., via <code>apt-get install llvm-dev libclang-common-<matching LLVM version>-dev</code> (the LLVM compiler-rt libraries aka <tt>libclang-common-*-dev</tt> are optional but recommended). Only the Android target requires building [https://github.com/ldc-developers/llvm-project/releases our lightly tweaked version of LLVM], which is what we'll use here. There are also pre-built binary tarballs of our tweaked LLVM at that link, but they don't always work in other build environments, so we lay out the steps below in case you can't use them.
  
 
==== Building LLVM from source ====
 
==== Building LLVM from source ====
Line 39: Line 39:
  
 
# Download & extract source tarball of our lightly tweaked LLVM
 
# Download & extract source tarball of our lightly tweaked LLVM
curl -OL https://github.com/ldc-developers/llvm/releases/download/ldc-v9.0.0/llvm-9.0.0.src.tar.xz
+
curl -OL https://github.com/ldc-developers/llvm-project/releases/download/ldc-v9.0.1/llvm-9.0.1.src.tar.xz
tar -xf llvm-9.0.0.src.tar.xz
+
tar -xf llvm-9.0.1.src.tar.xz
  
 
# Generate Ninja build files; remove `-G Ninja` to use Make instead
 
# Generate Ninja build files; remove `-G Ninja` to use Make instead
 
mkdir build-llvm && cd build-llvm # using a fresh new build dir is highly recommended whenever re-invoking CMake
 
mkdir build-llvm && cd build-llvm # using a fresh new build dir is highly recommended whenever re-invoking CMake
cmake -G Ninja ../llvm-9.0.0.src \
+
cmake -G Ninja ../llvm-9.0.1.src \
 
   -DCMAKE_BUILD_TYPE=Release \
 
   -DCMAKE_BUILD_TYPE=Release \
 
   -DCMAKE_INSTALL_PREFIX=$PWD/../install-llvm \
 
   -DCMAKE_INSTALL_PREFIX=$PWD/../install-llvm \
Line 67: Line 67:
 
Now that you're ready to build LDC from source, clone the [https://github.com/ldc-developers/ldc LDC GitHub repository] or get [https://github.com/ldc-developers/ldc/releases one of our official source releases]:
 
Now that you're ready to build LDC from source, clone the [https://github.com/ldc-developers/ldc LDC GitHub repository] or get [https://github.com/ldc-developers/ldc/releases one of our official source releases]:
  
  $ git clone --recursive https://github.com/ldc-developers/ldc.git
+
  git clone --recursive https://github.com/ldc-developers/ldc.git
  
 
If you already have the git repo, don't forget to make sure your submodules are up to date by running <tt>git submodule update --init</tt>.
 
If you already have the git repo, don't forget to make sure your submodules are up to date by running <tt>git submodule update --init</tt>.
Line 75: Line 75:
 
<syntaxhighlight lang=bash>
 
<syntaxhighlight lang=bash>
 
# Generate Ninja build files
 
# Generate Ninja build files
mkdir build && cd build # using a fresh new build dir is highly recommended whenever re-invoking CMake
+
mkdir build-ldc && cd build-ldc # using a fresh new build dir is highly recommended whenever re-invoking CMake
 
cmake -G Ninja ../ldc \
 
cmake -G Ninja ../ldc \
 
   -DCMAKE_BUILD_TYPE=Release \
 
   -DCMAKE_BUILD_TYPE=Release \
Line 90: Line 90:
 
If you are planning to work on LDC, we recommend <tt>-DCMAKE_BUILD_TYPE=RelWithDebInfo</tt>.
 
If you are planning to work on LDC, we recommend <tt>-DCMAKE_BUILD_TYPE=RelWithDebInfo</tt>.
  
If you want to target Android and are building ldc 1.4 or later, add <tt>-DLDC_TARGET_PRESET=Android-arm</tt> to the CMake config.
+
If you are building natively in Termux for Android, add <tt>-DLDC_TARGET_PRESET=Android-arm</tt> to the CMake config.
 +
 
 +
'''Note''': When installing LDC, existing Phobos/druntime includes (*.d and *.di files) aren't removed, only overwritten. This can cause conflicts when installing a new LDC version into an older LDC installation directory, e.g., when a <tt>std/datetime.d</tt> module is replaced by a <tt>std/datetime/*.d</tt> package. Cleaning up the installation dir or using a new one is recommended.
  
 
=== More useful CMake variables ===
 
=== More useful CMake variables ===
Line 96: Line 98:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
| '''LLVM_ROOT_DIR''' and '''LLVM_CONFIG''' || Allows you to specify the LLVM instance to use. LLVM_CONFIG specifies the path and name of the <tt>llvm-config</tt> binary to use. By default, it is assumed to be <tt>${LLVM_ROOT_DIR}/bin/llvm-config</tt>, otherwise it is searched for on default system paths.
+
! LLVM_ROOT_DIR or<br>LLVM_CONFIG
 +
| Allows you to specify the LLVM instance to use. LLVM_CONFIG specifies the path and name of the <tt>llvm-config</tt> binary to use. By default, it is assumed to be <tt>${LLVM_ROOT_DIR}/bin/llvm-config</tt>, otherwise it is searched for on default system paths.
 +
|-
 +
! BUILD_LTO_LIBS
 +
| Set this to <tt>ON</tt> to build an additional LTO-able set of Phobos and druntime (to be selected via <tt>-defaultlib=phobos2-ldc-lto,druntime-ldc-lto</tt>).
 +
|-
 +
! LDC_INSTALL_LTOPLUGIN
 +
| Set this to <tt>ON</tt> to include the LLVM <tt>ld.gold</tt> LTO linker plugin in the LDC installation, if found.
 
|-
 
|-
| '''BUILD_LTO_LIBS''' || Set this to <tt>ON</tt> to build an additional LTO-able set of Phobos and druntime (to be selected via <tt>-defaultlib=phobos2-ldc-lto,druntime-ldc-lto</tt>).
+
! LDC_INSTALL_LLVM_RUNTIME_LIBS
 +
| Set this to <tt>ON</tt> to include the LLVM compiler-rt libraries in the LDC installation, if found.
 
|-
 
|-
| '''LDC_INSTALL_LTOPLUGIN''' || Set this to <tt>ON</tt> to include the LLVM <tt>ld.gold</tt> LTO linker plugin in the LDC installation, if found.
+
! MULTILIB
 +
| Set this to <tt>ON</tt> to build the 32-bit libraries too. You'll need a multilib C toolchain, e.g., <tt>apt-get install g++-multilib</tt>.
 
|-
 
|-
| '''LDC_INSTALL_LLVM_RUNTIME_LIBS''' || Set this to <tt>ON</tt> to include the LLVM compiler-rt libraries in the LDC installation, if found.
+
! BUILD_SHARED_LIBS
 +
| Build only shared libraries (<tt>ON</tt>), only static libraries (<tt>OFF</tt>), or both (<tt>BOTH</tt>). Defaults to <tt>BOTH</tt> if shared libraries are supported on your platform, and <tt>OFF</tt> if they aren't.
 
|-
 
|-
| '''MULTILIB''' || Set this to <tt>ON</tt> to build the 32-bit libraries too. You'll need a multilib C toolchain, e.g., <tt>apt-get install g++-multilib</tt>.
+
! LIB_SUFFIX
 +
| Some Linux distributions, such as Fedora, expect 64 bit libraries in <tt>/usr/lib64</tt> instead of <tt>/usr/lib</tt>. In this case, the installation directory can be adjusted using <tt>-DLIB_SUFFIX=64</tt>.
 
|-
 
|-
| '''BUILD_SHARED_LIBS''' || Build only shared libraries (<tt>ON</tt>), only static libraries (<tt>OFF</tt>), or both (<tt>BOTH</tt>). Defaults to <tt>BOTH</tt> if shared libraries are supported on your platform, and <tt>OFF</tt> if they aren't.
+
! INCLUDE_INSTALL_DIR
 +
| The location the D modules for druntime and Phobos are installed to. Defaults to <tt>${CMAKE_INSTALL_PREFIX}/include/d</tt>.
 
|-
 
|-
| '''LIB_SUFFIX''' || Some Linux distributions, such as Fedora, expect 64 bit libraries in <tt>/usr/lib64</tt> instead of <tt>/usr/lib</tt>. In this case, the installation directory can be adjusted using <tt>-DLIB_SUFFIX=64</tt>.
+
! LDC_WITH_LLD
 +
| Set this to <tt>OFF</tt> to disable the LLD linker integration even if the LLD headers are found, e.g., when getting errors about conflicting command-line options when running LDC.
 
|-
 
|-
| '''INCLUDE_INSTALL_DIR''' || The location the D modules for druntime and Phobos are installed to. Defaults to <tt>${CMAKE_INSTALL_PREFIX}/include/d</tt>.
+
! LDC_DYNAMIC_COMPILE
 +
| Set this to <tt>OFF</tt> (before v1.21: <tt>False</tt>) to disable support for dynamic compilation aka JIT. Try this when getting strange CMake errors.
 
|-
 
|-
| '''LIBCONFIG_LIBRARY''' and '''LIBCONFIG_INCLUDE_DIR''' || Only for 0.17 ltsmaster, these variables can be used to specify the location of the libconfig++ library files and the path to the corresponding header files. NOTE: on error <tt>Could NOT find LibConfig (missing: LIBCONFIG_INCLUDE_DIR LIBCONFIG_LIBRARY)</tt> and using brew, use for eg: <tt>CMAKE_PREFIX_PATH=`brew --prefix` cmake …</tt> [see https://github.com/ldc-developers/ldc/issues/952].
+
! LIBCONFIG_LIBRARY and LIBCONFIG_INCLUDE_DIR
 +
| '''Only for 0.17 ltsmaster''', these variables can be used to specify the location of the libconfig++ library files and the path to the corresponding header files. NOTE: on error <tt>Could NOT find LibConfig (missing: LIBCONFIG_INCLUDE_DIR LIBCONFIG_LIBRARY)</tt> and using brew, use for eg: <tt>CMAKE_PREFIX_PATH=`brew --prefix` cmake …</tt> [see https://github.com/ldc-developers/ldc/issues/952].
 
|-
 
|-
 
|}
 
|}

Revision as of 04:02, 9 May 2020

This page shows you how to build and install LDC on most Posix-like systems such as linux, macOS, BSD, or Android. For building LDC on Windows, please see its dedicated page.

Advice

It is hard for us to keep these wiki pages up-to-date. If you run into trouble, have a look at the build scripts for our Continuous Integration platforms: the Azure Pipelines scripts for Ubuntu Linux and macOS and Windows are always up-to-date with the latest build setup.

Prerequisites

  • Git (for fetching the source code, if not using a tarball)
  • a C++ toolchain (GCC, Clang, …)
  • a D compiler (LDC/DMD/GDC)
    • If there's no suitable prebuilt D compiler for your platform: LDC 0.17 is the last version that does not need a D compiler to be built. Thus for bootstrapping, you can first build 0.17, and then use that to build newer compiler versions. Our testing infrastructure explicitly tests that new LDC versions can be built with 0.17 (on 64-bit systems). The git branch is called ltsmaster or you can get the source for the latest 0.17 release. Note that it doesn't support the latest LLVM versions.
  • CMake 3.8+
  • Ninja or Make (Ninja is highly recommended as it builds in parallel by default and doesn't suffer from concurrency issues wrt. CMake custom commands)
  • Python
  • libcurl for Phobos' std.net.curl (e.g., libcurl4 on recent Ubuntu)
  • zlib-dev (e.g., zlib1g-dev on Ubuntu)
  • For 0.17 ltsmaster:
    • libconfig++: get the libconfig-devel or libconfig-dev package for some Linux distributions. On OSX, sudo port install libconfig-hr; on Android with the Termux app, apt install libconfig-dev.
    • libcurl-dev (e.g., libcurl4-gnutls-dev on Ubuntu)
  • For the tests: gdb, unzip, zip and tzdata
    • For BSD: bash and GNU make

On Ubuntu 18.04, this amounts to:

apt-get install git-core g++ ldc cmake ninja-build zlib1g-dev libcurl4 \
                gdb unzip zip tzdata # only required for the tests

LLVM

The minimum supported LLVM version as of October 2019 is 3.9. Many Linux distributions already provide recent LLVM dev packages, sometimes in the form of user-curated package repositories (PPA, …). If a recent LLVM package is available, you might prefer to use it, as LLVM is a rather big project to build, e.g., via apt-get install llvm-dev libclang-common-<matching LLVM version>-dev (the LLVM compiler-rt libraries aka libclang-common-*-dev are optional but recommended). Only the Android target requires building our lightly tweaked version of LLVM, which is what we'll use here. There are also pre-built binary tarballs of our tweaked LLVM at that link, but they don't always work in other build environments, so we lay out the steps below in case you can't use them.

Building LLVM from source

# Non-Apple platforms: install binutils-dev package required to generate the ld.gold LTO linker plugin
apt-get install binutils-dev

# Download & extract source tarball of our lightly tweaked LLVM
curl -OL https://github.com/ldc-developers/llvm-project/releases/download/ldc-v9.0.1/llvm-9.0.1.src.tar.xz
tar -xf llvm-9.0.1.src.tar.xz

# Generate Ninja build files; remove `-G Ninja` to use Make instead
mkdir build-llvm && cd build-llvm # using a fresh new build dir is highly recommended whenever re-invoking CMake
cmake -G Ninja ../llvm-9.0.1.src \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=$PWD/../install-llvm \
  -DLLVM_BINUTILS_INCDIR=/usr/include \ # non-Apple only: location of binutils-dev's plugin.h
  -DLLVM_TARGETS_TO_BUILD='AArch64;ARM;Mips;MSP430;NVPTX;PowerPC;RISCV;WebAssembly;X86' \ # defaults to `all` if not specified
  -DCOMPILER_RT_INCLUDE_TESTS=OFF \
  -DLLVM_INCLUDE_TESTS=OFF

# Build and install (to directory specified as CMAKE_INSTALL_PREFIX above)
ninja install # or `make -j$(nproc) install`
cd ..

Check out the LLVM CMake page for more CMake variables.

If you are planning to work on LDC itself, we recommend -DCMAKE_BUILD_TYPE=RelWithDebInfo for the (huge!) LLVM debuginfos and -DLLVM_ENABLE_ASSERTIONS=ON. LLVM's assertions mode also controls LDC's assertions (in both C++ and D parts of LDC). A debug build of LLVM (-DCMAKE_BUILD_TYPE=Debug) leads to a heavy slowdown of LDC's compilation speed. LLVM's CMAKE_BUILD_TYPE also controls how the C++ parts of LDC are compiled (LDC inherits the C++ compiler flags used to build LLVM).

If you are building natively in Termux for Android, you'll want to specify a proper default triple like -DLLVM_DEFAULT_TARGET_TRIPLE=armv7a-unknown-linux-androideabi.

Building LDC from source

Now that you're ready to build LDC from source, clone the LDC GitHub repository or get one of our official source releases:

git clone --recursive https://github.com/ldc-developers/ldc.git

If you already have the git repo, don't forget to make sure your submodules are up to date by running git submodule update --init.

Run the following commands to configure and build LDC and its default libraries:

# Generate Ninja build files
mkdir build-ldc && cd build-ldc # using a fresh new build dir is highly recommended whenever re-invoking CMake
cmake -G Ninja ../ldc \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=$PWD/../install-ldc \
  -DLLVM_ROOT_DIR=$PWD/../install-llvm \                  # not needed if using a distro LLVM package
  -DD_COMPILER=$PWD/../ldc2-1.17.0-linux-x86_64/bin/ldmd2 # not needed if host D compiler (ldmd2/dmd/gdmd) is found in PATH or if building 0.17 ltsmaster

# Build; use -j<N> to limit parallelism if running out of memory. The binaries end up in bin/.
ninja
# Optional: install LDC to the CMAKE_INSTALL_PREFIX directory above
ninja install

If you are planning to work on LDC, we recommend -DCMAKE_BUILD_TYPE=RelWithDebInfo.

If you are building natively in Termux for Android, add -DLDC_TARGET_PRESET=Android-arm to the CMake config.

Note: When installing LDC, existing Phobos/druntime includes (*.d and *.di files) aren't removed, only overwritten. This can cause conflicts when installing a new LDC version into an older LDC installation directory, e.g., when a std/datetime.d module is replaced by a std/datetime/*.d package. Cleaning up the installation dir or using a new one is recommended.

More useful CMake variables

LLVM_ROOT_DIR or
LLVM_CONFIG
Allows you to specify the LLVM instance to use. LLVM_CONFIG specifies the path and name of the llvm-config binary to use. By default, it is assumed to be ${LLVM_ROOT_DIR}/bin/llvm-config, otherwise it is searched for on default system paths.
BUILD_LTO_LIBS Set this to ON to build an additional LTO-able set of Phobos and druntime (to be selected via -defaultlib=phobos2-ldc-lto,druntime-ldc-lto).
LDC_INSTALL_LTOPLUGIN Set this to ON to include the LLVM ld.gold LTO linker plugin in the LDC installation, if found.
LDC_INSTALL_LLVM_RUNTIME_LIBS Set this to ON to include the LLVM compiler-rt libraries in the LDC installation, if found.
MULTILIB Set this to ON to build the 32-bit libraries too. You'll need a multilib C toolchain, e.g., apt-get install g++-multilib.
BUILD_SHARED_LIBS Build only shared libraries (ON), only static libraries (OFF), or both (BOTH). Defaults to BOTH if shared libraries are supported on your platform, and OFF if they aren't.
LIB_SUFFIX Some Linux distributions, such as Fedora, expect 64 bit libraries in /usr/lib64 instead of /usr/lib. In this case, the installation directory can be adjusted using -DLIB_SUFFIX=64.
INCLUDE_INSTALL_DIR The location the D modules for druntime and Phobos are installed to. Defaults to ${CMAKE_INSTALL_PREFIX}/include/d.
LDC_WITH_LLD Set this to OFF to disable the LLD linker integration even if the LLD headers are found, e.g., when getting errors about conflicting command-line options when running LDC.
LDC_DYNAMIC_COMPILE Set this to OFF (before v1.21: False) to disable support for dynamic compilation aka JIT. Try this when getting strange CMake errors.
LIBCONFIG_LIBRARY and LIBCONFIG_INCLUDE_DIR Only for 0.17 ltsmaster, these variables can be used to specify the location of the libconfig++ library files and the path to the corresponding header files. NOTE: on error Could NOT find LibConfig (missing: LIBCONFIG_INCLUDE_DIR LIBCONFIG_LIBRARY) and using brew, use for eg: CMAKE_PREFIX_PATH=`brew --prefix` cmake … [see https://github.com/ldc-developers/ldc/issues/952].

NOTE: see https://github.com/Linuxbrew/homebrew-core/blob/master/Formula/ldc.rb for brew's install

Tips

The Makefiles generated by CMake respect the DESTDIR variable for the install target. It is prepended to all the file installation targets. This can be useful for building packages:

$ make install DESTDIR=<your root directory>