Starting as a Contributor

This page contains a set of instructions that get anyone from having nothing D-related on their machine, to a full-blown development rig for the reference D compiler `dmd` and one or more of its paraphernalia: core runtime (aka druntime), standard library (aka phobos), website, and tools.

0. Prerequisites

This tutorial is written for a Posix system (Linux, OSX, FreeBSD etc) and assumes you have make, g++, libcurl4-openssl-dev, and git up and running on your system, as well as a working github account. To install the appropriate code on e.g. Ubuntu:

sudo apt-get install git make g++ libcurl4-openssl-dev

To build the 32-bit phobos on a 64-bit machine, libc6-dev-i386 and libcurl4-gnutls-dev:i386 are also needed:

sudo apt-get install libc6-dev-i386 libcurl4-gnutls-dev:i386

Other versions and variations of libcurl may work as well.

1. Fetch dmd from github

Let's start by getting the current development (master) branch of dmd from github. Assume the root directory for everything D-related is ~/code (replace appropriately). This is easily done by running at a command prompt:

cd ~/code
git clone https://github.com/D-Programming-Language/dmd

After this step completes successfully, the directory ~/code/dmd should be up and filled with good stuff.

2. Bootstrap dmd

This step is interesting because in order to build dmd, dmd is necessary. Fortunately, the steps of downloading and using a preexisting dmd compiler are automated. All you need to do is run this command:

cd ~/code/dmd
make -f posix.mak AUTO_BOOTSTRAP=1

That's going to take a while. To make it faster, passing -j8 accelerates things by running eight processes in parallel. The build produces the compiler binary ~/code/dmd/src/dmd.

To make dmd builds faster in the future, you need to obviate the need for bootstrapping. Install dmd from the download page or simply put the freshly built dmd binary in a place accessible through $PATH (a popular choice is ~/bin).

3. Fetch and build druntime

druntime is the core runtime library for D, needed for building most every D program, including the standard library itself. So it's the next step in the progression (note that it requires a properly built dmd, so make sure steps 1 and 2 have completed successfully). To fetch and build druntime, issue these commands:

cd ~/code
git clone https://github.com/D-Programming-Language/druntime
cd druntime
make -f posix.mak

All that should go pretty fast. The somewhat anticlimactic result of the build is a library called libdruntime.a situated in an OS-dependent directory such as ~/code/druntime/generated/linux/release/64/. Make sure it's there.

4. Fetch and build phobos

Most D programs use D's standard library phobos. To get and build it, make sure you first fetch and build the latest dmd and druntime. Then:

cd ~/code
git clone https://github.com/D-Programming-Language/phobos
cd phobos
make -f posix.mak

The build produces (with similar anticlimacticity) static and shared libraries such as ~/code/phobos/generated/linux/release/64/libphobos2.a and ~/code/phobos/generated/linux/release/64/libphobos2.so.

4.1. Unittest phobos

If you want to work on phobos itself, you need to run unittests—either for the full library, a package, or a module. To unittest the entire library:

make -j16 -f posix.mak unittest

Adjust the parameter passed to -j depending on your machine (beefier machines support larger parameters). This command unittests phobos in both debug and release mode. To only test one of them, add BUILD=debug or BUILD=release to the command line, for example:

make -j16 -f posix.mak BUILD=debug unittest

Specifying BUILD makes unittesting faster so it is recommended in iterative development. Just make sure both debug and release builds are tested before e.g. submitting a pull request.

While changing one specific package or module, it's useful to be able to only unittest that particular entity. The following two commands only unittest (in debug mode) the std.algorithm package and the std.conv module, respectively:

make -j16 -f posix.mak BUILD=debug std/algorithm.test
make -j16 -f posix.mak BUILD=debug std/conv.test

Several modules, packages, or mix thereof may be specified for testing in the same command line. For example, this command also tests (and also in debug mode) the std.algorithm package and the std.conv module, with better parallelism:

make -j16 -f posix.mak BUILD=debug std/algorithm.test std/conv.test

Also, it's important to test for trailing whitespace in your code before issuing a pull request, as the automated testing will fail if you have any:

make -j16 -f posix.mak checkwhitespace

5. Fetch and build dlang.org

This step is optional but highly recommended. Significant changes to phobos' documentation require that the site (which includes automatically generated phobos documentation) builds successfully. The following will build all documentation in all forms (see the note below about building just the html documentation):

cd ~/code
git clone https://github.com/D-Programming-Language/dlang.org
cd dlang.org
make -f posix.mak

All of dmd, druntime, and phobos are needed for the site to build. Note that one of the first lines output during the make run looks like this:

LATEST=2.069.1 <-- place in the command line to skip network traffic.

That's advice worth heeding because fetching LATEST automatically involves network traffic, which adds time to the build. So for future builds use this:

make -f posix.mak LATEST=2.069.1

Of course, parallelizing with -j improves speed as well.

The build produces the entire site at ~/code/dlang.org/web. To informally test it, open the appropriate HTML documents in that directory. Note that the currently released phobos has documentation in ~/code/dlang.org/web/phobos, whereas the current (fresh) build of phobos has documentation in ~/code/dlang.org/web/phobos-prerelease. So, for example, if you change the embedded documentation in ~/code/phobos/std/conv.d, the changes are visible in ~/code/dlang.org/web/phobos-prerelease/std_conv.html. (The build process replaces the slashes in submodules with underscores.)

Note that the above steps will build all of the documentation in all forms, including Kindle builds and various other things that may require installing additional tools, and may download/build old versions of DMD. To prevent that, you can add the html option:

make -f posix.mak LATEST=2.069.1 html

6. Ancillary stuff: dconf.org, dub, dub-registry, installer, tools, and visuald

These ancillary repositories are of somewhat specific interest. Their installation mimics that of the repositories described above. If you get to the point where you need to work on one of these, chances are you're already versed in what needs doing. If not, ask away.

Typical Contributor Workflow

There are many ways to use git and github to contribute. Here's a typical one.

First, fork the github repository or repositories you'd like to contribute to (dmd, druntime, phobos etc) by navigating to their respective pages on github.com and clicking "Fork". Then, set up your local git repository to reflect that. For example, consider you want to contribute to phobos and have forked it. Then run these commands:

cd ~/code/phobos
git remote add myfork https://github.com/username/phobos.git
git remote update

(Replace username with your actual github user name.) This adds the "myfork" repository and makes sure everything is synchronized between your local copy and the remote repositories. Then, it's best to work in branches as shown below:

git checkout -b awesome-new-feature
# ... get some good work done here ...
git commit -am "Awesome new feature ..."
git push -f myfork

With this, your work is in your github fork of the phobos (or whichever) repository. After that, visit your fork on github.com, which looks like https://github.com/username/phobos/tree/awesome-new-feature. Then, github detects the new code and offers assistance to create a pull request with just a couple of clicks.

Contributing FAQ

Someone asked me to squash my commits, what does that mean?

After recieving feedback on your PR, it's common for it to have lots of commits that don't add much by being separate. For example, consider the following git history on a PR:

   commit [ffffff] Added new function: foobar

   commit [aaaaaa] Spelling error fix in foobar docs

   commit [cccccc] Clarified Docs for foobar

Nothing is gained from having these as three separate commits as they are all focused on one feature. Instead, they should be one commit so the history looks like this

commit [333333] Added new function: foobar

while still retaining all of your changes. In order to perform this, please consult this tutorial.

A file that I made a change on was modified by a different merged PR, and now my PR can't be merged, now what?

What you need to do is rebase your git branch on the master branch. What this does is rewrite the history of your git branch to make it seem like it was merged off of the head of master rather than the older commit where you actually branched. This will include the new commits in your PR so your PR no longer conflicts. See this tutorial for more details.

More Information

You're all set! More information about contributing on a regular basis can be found here.