Difference between revisions of "Building and hacking LDC on Windows using MSVC"

From D Wiki
Jump to: navigation, search
m (Running the druntime/Phobos unit tests and druntime standalone tests)
(Update, mainly wrt. LLVM build instructions)
Line 6: Line 6:
  
 
== Advice ==
 
== Advice ==
It is hard for us to keep these wiki pages up-to-date. If you run into trouble, have a look at the build scripts for our [http://wiki.dlang.org/LDC_contributor%27s_guide#Continuous_Integration Continuous Integration] platforms: the Azure Pipelines scripts for [https://github.com/ldc-developers/ldc/blob/master/.azure-pipelines/posix.yml Ubuntu Linux and macOS] and [https://github.com/ldc-developers/ldc/blob/master/.azure-pipelines/windows.yml Windows] are always up-to-date with the latest build setup.
+
It is hard for us to keep these wiki pages up-to-date. If you run into trouble, have a look at the build scripts for our [http://wiki.dlang.org/LDC_contributor%27s_guide#Continuous_Integration Continuous Integration] platforms: the [https://github.com/ldc-developers/ldc/tree/master/.github GitHub Actions scripts] are always up-to-date with the latest build setup.
  
 
== Building LDC ==
 
== Building LDC ==
Line 14: Line 14:
 
* Windows, of course!
 
* Windows, of course!
 
* [https://www.visualstudio.com/downloads/ Visual Studio or stand-alone Build Tools] ≥ 2017. Make sure to install the Visual C++ toolchain.
 
* [https://www.visualstudio.com/downloads/ Visual Studio or stand-alone Build Tools] ≥ 2017. Make sure to install the Visual C++ toolchain.
* A [https://github.com/ldc-developers/ldc/releases/ D compiler] (the <tt>ltsmaster</tt> git branch does not need a D compiler to build). DMD as host compiler isn't supported on Windows, make sure to use LDC.
+
* A [https://github.com/ldc-developers/ldc/releases/ D compiler]. DMD as host compiler isn't supported on Windows, make sure to use LDC.
 
* A recent [https://git-scm.com/download/win git] installation
 
* A recent [https://git-scm.com/download/win git] installation
 
* [https://www.python.org/downloads/ Python] (I use [https://github.com/winpython/winpython/releases Winpython])
 
* [https://www.python.org/downloads/ Python] (I use [https://github.com/winpython/winpython/releases Winpython])
 
* [https://github.com/Kitware/CMake/releases CMake &ge; 3.8]
 
* [https://github.com/Kitware/CMake/releases CMake &ge; 3.8]
 
* [https://github.com/ninja-build/ninja/releases Ninja], a neat little and fast build system
 
* [https://github.com/ninja-build/ninja/releases Ninja], a neat little and fast build system
* For running the testsuite: GNU make ([https://dl.dropboxusercontent.com/s/4y36f5ydgrk4p5g/make-4.2.1.7z?dl=0 prebuilt v4.2.1])
+
* For running the testsuite: GNU make ([https://github.com/dlang/dmd/releases/download/nightly/gnumake-4.4-win64.zip prebuilt v4.4.0])
  
 
=== Shell environment ===
 
=== Shell environment ===
  
 
I use a little batch file to set up my LDC build environment. It's located in the root of my LDC environment: <tt>C:\LDC\shell.cmd</tt>.
 
I use a little batch file to set up my LDC build environment. It's located in the root of my LDC environment: <tt>C:\LDC\shell.cmd</tt>.
It sets up the <tt>PATH</tt> environment variable (I've installed the portable tools into <tt>C:\LDC\Tools</tt>) and then spawns a new <tt>x64 Native Tools Command Prompt for VS 2019</tt> with a below-normal process priority, so that my system stays responsive while building.
+
It sets up the <tt>PATH</tt> environment variable (I've installed the portable tools into <tt>C:\LDC\Tools</tt>) and then spawns a new <tt>x64 Native Tools Command Prompt for VS 2022</tt> with a below-normal process priority, so that my system stays responsive while building.
 
I use a shortcut on my desktop to this batch file. Please adjust it to your needs (and note that <tt>%~dp0</tt> is the directory containing the script, i.e., <tt>C:\LDC\</tt>).
 
I use a shortcut on my desktop to this batch file. Please adjust it to your needs (and note that <tt>%~dp0</tt> is the directory containing the script, i.e., <tt>C:\LDC\</tt>).
  
 
<pre>
 
<pre>
 
@echo off
 
@echo off
set PATH=%~dp0LDC-x64\bin;%~dp0LLVM-x64\bin;%~dp0Tools\Ninja 1.10.2;%~dp0Tools\make-4.2.1;C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin;%~dp0Tools\cmake-3.19.4-win64-x64\bin;%~dp0Tools\Winpython64-3.7.4.0Zero\python-3.7.4.amd64;%PATH%
+
set PATH=%~dp0LDC-x64\bin;%~dp0LLVM-x64\bin;%~dp0Tools\Ninja 1.10.2;%~dp0Tools\make-4.4.0;C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin;%~dp0Tools\cmake-3.20.3-windows-x86_64\bin;%~dp0Tools\WPy64-3940\python-3.9.4.amd64;%PATH%
 
:: set D host compiler
 
:: set D host compiler
set DMD=%~dp0ldc2-1.24.0-windows-multilib\bin\ldmd2.exe
+
set DMD=%~dp0ldc2-1.39.0-windows-multilib\bin\ldmd2.exe
:: set a few environment variables for dmd-testsuite
 
set OS=windows
 
set MODEL=64
 
 
if not exist "%TERM%" set TERM=msys
 
if not exist "%TERM%" set TERM=msys
start /belownormal cmd.exe /k "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\Tools\VsDevCmd.bat" -arch=x64
+
start /belownormal cmd.exe /k "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\Tools\VsDevCmd.bat" -arch=x64
 
</pre>
 
</pre>
  
Use <tt>-arch=x86</tt> in the last line and <tt>set MODEL=32</tt> if you want to build a 32-bit LDC.
+
Use <tt>-arch=x86</tt> in the last line if you want to build a 32-bit LDC.
  
 
Open a shell by executing the batch file.
 
Open a shell by executing the batch file.
Line 58: Line 55:
  
 
<ul>
 
<ul>
<li>Download an LLVM source tarball (<tt>llvm-x.y.z.src.tar.xz</tt>). We maintain an [https://github.com/ldc-developers/llvm-project/releases LLVM fork] with minimal modifications; [http://releases.llvm.org/download.html vanilla LLVM] can be used as well.</li>
+
<li>Download an LLVM source tarball (<tt>llvm-x.y.z.src.tar.xz</tt>). We maintain an [https://github.com/ldc-developers/llvm-project/releases LLVM fork] with minimal modifications; [https://github.com/llvm/llvm-project/releases vanilla LLVM] can be used as well (but please prefer our fork for max Windows support).</li>
<li>Extract it into <tt>C:\LDC</tt> (e.g., with [http://www.7-zip.org/ 7-Zip]), so that you end up with the source tree in e.g. <tt>C:\LDC\llvm-11.0.1.src</tt>.</li>
+
<li>Extract it into <tt>C:\LDC</tt> (e.g., with [https://7-zip.org/ 7-Zip]), so that you end up with the source tree in e.g. <tt>C:\LDC\llvm-19.1.3.src</tt>.</li>
 
<li><code>cd C:\LDC</code></li>
 
<li><code>cd C:\LDC</code></li>
 
<li>Create a build directory: <code>md build-llvm-x64</code></li>
 
<li>Create a build directory: <code>md build-llvm-x64</code></li>
 
<li>Change into it: <code>cd build-llvm-x64</code></li>
 
<li>Change into it: <code>cd build-llvm-x64</code></li>
 
<li><p>Use a command like this (in one line) to create the Ninja build files:</p>
 
<li><p>Use a command like this (in one line) to create the Ninja build files:</p>
<pre>cmake -G Ninja ..\llvm-11.0.1.src
+
<pre>cmake -G Ninja ..\llvm-19.1.3.src\llvm
 +
      -DCMAKE_BUILD_TYPE=Release
 
       -DCMAKE_INSTALL_PREFIX=&quot;C:\LDC\LLVM-x64&quot;
 
       -DCMAKE_INSTALL_PREFIX=&quot;C:\LDC\LLVM-x64&quot;
       -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_USE_CRT_RELWITHDEBINFO=MT
+
       -DLLVM_ENABLE_PROJECTS=&quot;compiler-rt;lld&quot;
 +
      -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded
 
       -DLLVM_ENABLE_ASSERTIONS=ON
 
       -DLLVM_ENABLE_ASSERTIONS=ON
      -DLLVM_TARGETS_TO_BUILD=AArch64;ARM;Mips;MSP430;NVPTX;PowerPC;RISCV;WebAssembly;X86
 
      -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-windows-msvc
 
 
       -DCOMPILER_RT_INCLUDE_TESTS=OFF
 
       -DCOMPILER_RT_INCLUDE_TESTS=OFF
       -DLLVM_INCLUDE_TESTS=OFF</pre>
+
       -DLLVM_INCLUDE_TESTS=OFF
<p>Use <code>-DCMAKE_BUILD_TYPE=Debug</code> to build a debug version (and probably use <code>-DLLVM_USE_CRT_DEBUG=MTd</code> then). The [http://llvm.org/docs/CMake.html#llvm-specific-variables LLVM page on CMake] documents other variables you can change. For a 32-bit build, use <code>-DLLVM_DEFAULT_TARGET_TRIPLE=i686-pc-windows-msvc</code>.</p></li>
+
      -DLLVM_INCLUDE_BENCHMARKS=OFF</pre>
 +
<p>Check out the [https://llvm.org/docs/CMake.html#options-and-variables LLVM CMake page] for more CMake variables.</p></li>
 
<li>Build LLVM: <code>ninja</code></li>
 
<li>Build LLVM: <code>ninja</code></li>
 
<li>Install it (to <tt>C:\LDC\LLVM-x64</tt>): <code>ninja install</code></li>
 
<li>Install it (to <tt>C:\LDC\LLVM-x64</tt>): <code>ninja install</code></li>
<li>Note that you'll need to keep the <tt>C:\LDC\build-llvm-x64</tt> directory in order to keep the LLVM debuginfos (.pdb files) available.</li>
+
<li>If enabling (huge!) LLVM debuginfos (e.g., via <tt>-DCMAKE_BUILD_TYPE=RelWithDebInfo</tt>), please note that you'll need to keep the <tt>C:\LDC\build-llvm-x64</tt> directory in order to keep the .pdb files available.</li>
 
</ul>
 
</ul>
  
Line 87: Line 85:
 
<li><p>Use a command like this (in one line) to create the Ninja build files:</p>
 
<li><p>Use a command like this (in one line) to create the Ninja build files:</p>
 
<pre>cmake -G Ninja ..\ldc
 
<pre>cmake -G Ninja ..\ldc
 +
      -DCMAKE_BUILD_TYPE=RelWithDebInfo
 
       -DCMAKE_INSTALL_PREFIX=&quot;C:\LDC\LDC-x64&quot;
 
       -DCMAKE_INSTALL_PREFIX=&quot;C:\LDC\LDC-x64&quot;
      -DCMAKE_BUILD_TYPE=RelWithDebInfo
 
 
       -DLLVM_ROOT_DIR=&quot;C:/LDC/LLVM-x64&quot;</pre>
 
       -DLLVM_ROOT_DIR=&quot;C:/LDC/LLVM-x64&quot;</pre>
 
<p>You can specify the D host compiler explicitly by adding <code>-DD_COMPILER=c:\path\to\ldmd2.exe</code>.</p>
 
<p>You can specify the D host compiler explicitly by adding <code>-DD_COMPILER=c:\path\to\ldmd2.exe</code>.</p>
Line 129: Line 127:
 
=== Running the dmd-testsuite tests ===
 
=== Running the dmd-testsuite tests ===
  
* If you haven't already, set the <tt>OS</tt> and <tt>MODEL</tt> environment variables:
 
** <code>set OS=windows</code> (since v1.17, previously <code>win{32,64}</code>)
 
** <code>set MODEL={32,64}</code>
 
 
* <code>cd C:\LDC\build-ldc-x64</code>
 
* <code>cd C:\LDC\build-ldc-x64</code>
 
* <code>ctest -V -R dmd-testsuite</code>
 
* <code>ctest -V -R dmd-testsuite</code>
** Debug only: <code>ctest -V -R "build-run-dmd-testsuite|dmd-testsuite-debug"</code>
 
** Release only: <code>ctest -V -R dmd-testsuite -E -debug</code>
 
  
 
== Developing/debugging LDC/LLVM with Visual Studio ==
 
== Developing/debugging LDC/LLVM with Visual Studio ==

Revision as of 17:10, 12 December 2024

Windows MSVC x86/x64 are first class targets for LDC. This page documents how to build, test and hack LDC2 on Windows.

LDC binaries

If you just want to download the very latest LDC binaries, head over to the GitHub CI release.

Advice

It is hard for us to keep these wiki pages up-to-date. If you run into trouble, have a look at the build scripts for our Continuous Integration platforms: the GitHub Actions scripts are always up-to-date with the latest build setup.

Building LDC

Required software

Shell environment

I use a little batch file to set up my LDC build environment. It's located in the root of my LDC environment: C:\LDC\shell.cmd. It sets up the PATH environment variable (I've installed the portable tools into C:\LDC\Tools) and then spawns a new x64 Native Tools Command Prompt for VS 2022 with a below-normal process priority, so that my system stays responsive while building. I use a shortcut on my desktop to this batch file. Please adjust it to your needs (and note that %~dp0 is the directory containing the script, i.e., C:\LDC\).

@echo off
set PATH=%~dp0LDC-x64\bin;%~dp0LLVM-x64\bin;%~dp0Tools\Ninja 1.10.2;%~dp0Tools\make-4.4.0;C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin;%~dp0Tools\cmake-3.20.3-windows-x86_64\bin;%~dp0Tools\WPy64-3940\python-3.9.4.amd64;%PATH%
:: set D host compiler
set DMD=%~dp0ldc2-1.39.0-windows-multilib\bin\ldmd2.exe
if not exist "%TERM%" set TERM=msys
start /belownormal cmd.exe /k "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\Tools\VsDevCmd.bat" -arch=x64

Use -arch=x86 in the last line if you want to build a 32-bit LDC.

Open a shell by executing the batch file.

  • Running cl should display the banner from the MS compiler.
  • Running link /version should display the banner from the MS linker.
  • Running git --version should display the banner from git.
  • Running python --version should display the banner from Python.
  • Running cmake --version should display the banner from CMake.
  • Running ninja --version should display the Ninja version.
  • For the tests:
    • Running make --version should display the banner from GNU make.
    • Starting a bash shell via bash and then invoking link /version should still display the MS linker banner. If it doesn't, make sure that git's usr\bin directory is listed before its bin directory in PATH.

Building LLVM

Building LLVM takes quite a while, so you may want to download a prebuilt version from here. We use these prebuilt packages for CI and the official release packages. Just extract the archive into C:\LDC\LLVM-x64 and then skip this section. In case you encounter errors wrt. unsupported command-line options or linking errors when building LDC, you probably use a different MSVC/clang toolchain version and will need to build LLVM yourself.

  • Download an LLVM source tarball (llvm-x.y.z.src.tar.xz). We maintain an LLVM fork with minimal modifications; vanilla LLVM can be used as well (but please prefer our fork for max Windows support).
  • Extract it into C:\LDC (e.g., with 7-Zip), so that you end up with the source tree in e.g. C:\LDC\llvm-19.1.3.src.
  • cd C:\LDC
  • Create a build directory: md build-llvm-x64
  • Change into it: cd build-llvm-x64
  • Use a command like this (in one line) to create the Ninja build files:

    cmake -G Ninja ..\llvm-19.1.3.src\llvm
          -DCMAKE_BUILD_TYPE=Release
          -DCMAKE_INSTALL_PREFIX="C:\LDC\LLVM-x64"
          -DLLVM_ENABLE_PROJECTS="compiler-rt;lld"
          -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded
          -DLLVM_ENABLE_ASSERTIONS=ON
          -DCOMPILER_RT_INCLUDE_TESTS=OFF
          -DLLVM_INCLUDE_TESTS=OFF
          -DLLVM_INCLUDE_BENCHMARKS=OFF

    Check out the LLVM CMake page for more CMake variables.

  • Build LLVM: ninja
  • Install it (to C:\LDC\LLVM-x64): ninja install
  • If enabling (huge!) LLVM debuginfos (e.g., via -DCMAKE_BUILD_TYPE=RelWithDebInfo), please note that you'll need to keep the C:\LDC\build-llvm-x64 directory in order to keep the .pdb files available.

Building LDC

  • cd C:\LDC
  • git clone --recursive https://github.com/ldc-developers/ldc.git
  • md build-ldc-x64
  • cd build-ldc-x64
  • Use a command like this (in one line) to create the Ninja build files:

    cmake -G Ninja ..\ldc
          -DCMAKE_BUILD_TYPE=RelWithDebInfo
          -DCMAKE_INSTALL_PREFIX="C:\LDC\LDC-x64"
          -DLLVM_ROOT_DIR="C:/LDC/LLVM-x64"

    You can specify the D host compiler explicitly by adding -DD_COMPILER=c:\path\to\ldmd2.exe.

  • Build LDC and the default libraries: ninja. The binaries end up in C:\LDC\build-ldc-x64\bin.
  • If you want to install it (to C:\LDC\LDC-x64): ninja install

Tests

For troubleshooting, be sure to examine the file C:\LDC\build-ldc-x64\Testing\Temporary\LastTest.log after a ctest invocation.

Running the LDC D unit tests

  • cd C:\LDC\build-ldc-x64
  • ctest --output-on-failure -R ldc2-unittest

Running the LIT (LLVM Integrated Tester) tests

You'll need to have lit installed for Python:

  • Upgrade pip: python -m pip install -U pip
  • Install lit: python -m pip install lit
  • Make sure it works: python -c "import lit.main; lit.main.main();" --version .

To run the tests from your build dir you can do:

  • cd C:\LDC\build-ldc-x64
  • ctest -V -R lit-tests

or you can go to the tests folder inside your build dir, and run the runlit.py script:

  • cd C:\LDC\build-ldc-x64\tests
  • python runlit.py -v .

The second way is convenient for running individual tests:

  • python runlit.py -v codegen/align.d

Running the druntime/Phobos unit tests and druntime integration tests

  • Make sure the 64/32-bit libcurl.dll (bundled with LDC) can be found in a PATH directory. Verify by running where libcurl.dll.
  • cd C:\LDC\build-ldc-x64
  • Build the unit tests: ninja all-test-runners
  • Run the tests, excluding LDC unit tests, LIT tests and dmd-testsuite: ctest --output-on-failure -E "dmd-testsuite|lit-tests|ldc2-unittest"

Running the dmd-testsuite tests

  • cd C:\LDC\build-ldc-x64
  • ctest -V -R dmd-testsuite

Developing/debugging LDC/LLVM with Visual Studio

Be sure to use Rainer's great Visual D plugin for VS.

  • cd C:\LDC
  • md vs-ldc-x64
  • cd vs-ldc-x64
  • Use the CMake command from the Build LDC section, but use the VS generator instead of Ninja this time: cmake -G "Visual Studio 15 Win64" ...

This creates the VS 2017 solution C:\LDC\vs-ldc-x64\ldc.sln. A Visual Studio solution for LLVM can be created the same way.

I don't recommend building LDC/LLVM with VS directly; I only use VS for development/debugging and build in the shell with Ninja.



Windows MSVC