LDC contributor's guide

This page is a work-in-progress collection of tips for LDC contributors and development guidelines.

Continuous Integration

The LDC project uses multiple CI services, see the badges in README.md. Azure Pipelines and Shippable take care of fully automated package generation + deployment to GitHub release.

Any efforts to get regular builds+tests on other platforms would be very much appreciated (especially non-x86 platforms, like ARM, PowerPC, ...).

Style guidelines

Generally, use your good taste and try to use a style similar to the surrounding code. Many parts of the LLVM Coding Standards are directly applicable to LDC as well.

Formatting, etc.

• Use clang-format with the LLVM style for formatting.
• Includes are placed at the top of the file, sorted lexicographically and grouped in the following order:
  1. Main include corresponding to the .cpp file, if any.
  2. DMD and other LDC includes.
  3. LLVM/libconfig includes.
  4. System includes.

APIs and idioms

• Avoid making changes to the front end sources (dmd directory) as much as reasonably possible. If modifications are necessary, be sure to enclose the deviations from the upstream source #if IN_LLVM blocks, and keep the original source code around for easier comparison. Add a comment referring to the relevant issue on our tracker, or the upstream DMD commit in case of a straight backport. This makes it much easier to resolve any merge conflicts when folding in new frontend releases.
• Use llvm_unreachable("message") instead of assert(0 && "message") (from llvm/Support/ErrorHandling.h). Not only might it lead to slightly better codegen, it also prevents »variable might be used uninitialized« and similar compiler warnings.

Test suite

Tests are very important to prevent regressions. Upon fixing a bug, a test case should be added so that the bug stays fixed and doesn't come back by accident. Upon adding a new feature, extensive tests should be added that test all functionalities of that feature.

• Bug fixing often starts from a minimal test case, which is a good basis for the eventual test case that is added to the testsuite.
• Instead of creating tests after feature development, consider adding tests during feature development: it is a good way to solidify the feature implementation because often you will discover tricky cases that you didn't think of and that may steer the direction of development. The "Lit-based" testsuite makes it easy to run individual tests (run python runlit.py -v [testname] in your build/tests/ directory) so there is no excuse! ;-)

LDC is tested by the LDC Lit-based testsuite and DMD's dmd-testsuite with LDC specific modifications and additions. The dmd-testsuite is put in the tests/d2/ directory. The Lit-based testsuite is found in the subdirectories of tests/, excluding the d2 subfolder. You can check the previously mentioned wiki page dedicated to the Lit-based test-suite for more info.

To run the whole DMD test-suite, you need to be in the /build directory of LDC and issue:

$ DMD_TESTSUITE_MAKE_ARGS=-j$(nproc) ctest -V -R dmd-testsuite

where $(nproc) is the number of processes to run in parallel. This will run the whole suite.

Running a specific test is a little more complicated than the Lit-based suite. Basically, you should figure out what the specific test you want needs. For example, tests in /compilable should be able to be compiled. Hence, you can test them by just compiling them (possibly with -c). Tests in /runnable should be able to run successfully and so you can compile (and run them) with -run. Some tests need extra work which is usually commented at the top of the file.

In all cases, you should use your LDC built binary for compilation.

Merging a new DMD release

Updating druntime and Phobos

For each of the 2 submodules:

• Make sure you have both ldc and dlang git remotes. Otherwise add via git remote add dlang https://github.com/dlang/druntime.git.
• Sync: git fetch --all
• Create a new branch from current ldc head, e.g.: git checkout -b ldc-merge-2.079 origin/ldc
• Merge the upstream tag, e.g.: git merge v2.079.0. Resolve any merge conflicts and make sure to include the conflicting files list in the merge commit message.
• When confident, push it to the official repo: git push -u origin ldc-merge-2.079

Updating the dmd-testsuite repository

The dmd-testsuite repository is derived from the upstream DMD repository by the update-dmd-testsuite.sh script in the ldc-scripts repository. Just run it without arguments (on Posix, not recommended on Windows), and it should update the repository with the latest commits/tags (given that you have Git installed and an SSH key configured for github.com). The changes should apply to the remote repository in a fast-forward fashion, never force a push.

After this is done, just merge the v2.<xyz> tag into a new ldc-merge-2.<xyz> branch derived from current ldc (see above), manually resolving any merge conflicts as necessary (most changes should definitely be upstreamed).

Updating the front-end

There are basically 2 approaches to this: either re-applying the LDC-specific mods onto the new upstream files, or applying the upstream patch onto the current LDC front-end. I tend to the latter with a workflow like this:

• Generate the DMD patch from the previous front-end version to the next one, e.g., in the DMD src tree: git diff v2.078.3..v2.079.0 > ../2.079.0.diff
• Edit the .diff file manually, preferrably with an editor supporting that format (collapsible sections per file diff etc.):
  • Remove obsolete file diffs. As of 2.079, we only care about files in the res, src/dmd and src/dmd/root directories, other directories can be ignored, incl. src/dmd/backend.
  • DMD's top-level res directory maps to LDC's top-level one, unlike the front-end code (src/dmd vs. dmd). If there are diffs in the res dir, apply them separately and remove the diffs from the edited .diff file.
  • Start virtually applying the edited patch onto LDC, e.g., in the ldc src tree: patch -p 2 -u --dry-run --merge -i ../2.079.0-patched.diff
  • It's most likely going to complain about missing files. That's because we don't use all files in the src/dmd directory. You can edit the .diff file in parallel while skipping the missing files, e.g., by looking up the diff for each missing file. If it's an incremental diff, LDC most likely still doesn't need that file, and you can remove the diff. If it's a full diff, i.e., a newly added file, the new front-end may require it, so keep it in mind.
• When you feel ready, it's time to remove the --dry-run switch from the patch command, thereby really applying your edited patch.
• Now on to the (potentially) hard work - fixing the merge conflicts (I usually grep for <<<<). This can be especially annoying if a code block with LDC-specifics (version (IN_LLVM)) has been removed or, worse, moved to another file. So if a conflict is pretty big & messy, it's often easier to check the LDC-specific diff for the old front-end file, and re-applying that onto the new file. If a block with LDC specifics has been removed, be sure to check whether it's just been moved/refactored, as the LDC specifics will need to be moved too. I usually look for a seemingly distinct code piece in that block and grep for it in LDC's dmd subdir.
• Update the front-end version (and probably LDC version too) in CMakeLists.txt (DMDFE_*).
• Update the git submodule hashes to the respective ldc-merge-2.079 heads.
• The first goal should be getting LDC to compile. So try building with ninja ldc2 and fix all errors along the way.
• Then try getting druntime and Phobos to build with that new LDC: ninja all
  • In case of nasty crashes, a likely culprit is a divergence of the D code with the C++ headers, as the latter are only minimally covered by DMD CI. Unfortunately, mismatching struct and vtable layouts can be hard to track down.
• As soon as LDC can build druntime and Phobos, feel free to commit and push to a merge-2.079 branch in the official LDC repo. Time to let CI do some testing on a multitude of platforms!
• From now on, it's all about investigating and fixing failures, implementing new upstream features, disabling unfitting tests etc.
• Special care needs to be taken for DMD's src/dmd/mars.d, containing its main(); LDC uses substantial parts too. Among other things, new DMD command line options show up there. Make sure to add them to LDMD's help output when/if implementing them.

Releasing a new version of LDC

There is a separate page describing the LDC release process: How to release LDC.