Contributing to dlang.org

This document is aimed at people who want to contribute to the dlang.org website. It includes a brief tutorial in the technologies used and step-by-step instructions for typical tasks.

Installing

We assume you already have a dmd rig up and running. If not, follow the steps at Starting as a Contributor to get it running. The directory structure we'll assume in the following is:

~/
  d/
    dmd/
    druntime/
    phobos/

With this setup, let's proceed to downloading the source of dlang.org as follows:

cd ~/d
git clone https://github.com/D-Programming-Language/dlang.org

After this, the dlang.org directory will end up parallel to dmd, druntime, and phobos. Let's build the site (for now without the standard library documentation) by using the following command:

cd ~/d/dlang.org
make -j32 -f posix.mak html

The html target instructs make to only build the site. By default (if you specify no target), make also builds the core runtime and standard library documentation, both for the latest release and for the current code residing on your machine. That may get pretty involved, so let's leave it for later. For now, let's inspect the result of the build, all of which goes in the dlang.org/web directory. On Linux, for example, the command sensible-browser opens your default browser with a given file or address so we can use it as such:

sensible-browser ~/d/dlang.org/web/index.html

At this point if all went well a nicely-formatted HTML file pops up featuring a local replica of the dlang.org homepage. Congratulations!

Editing Content

Browsing through ~/d/dlang.org/ reveals that most files have the .dd extension. Those files are in Ddoc format; in order to work on dlang.org a basic understanding of the Ddoc format is needed.

At its core, Ddoc is a pure macro expansion system. In this context "pure" means the macro language has no relationship to other file format; all the expansion engine does is take in macro definitions and then munch through text and expand macros as they come along. A few macros are predefined to values that make HTML generation easy, so in that sense a slight affinity with HTML does exist; however, those macros can be trivially redefined to any other purpose. Also, Ddoc recognizes sections marked in a particular way as D source code. There are a few subtleties inherent to macro processors (e.g. how recursion works or in what order nested macros are expanded), but aside from those Ddoc is deceptively simple and very flexible. Exploiting the favorable relationship between Ddoc's simplicity and power is key to using it effectively.

Ddoc source files have the following structure:

Ddoc
Text with embedded macros such as $(MACRO1) and $(MACRO2) goes here.
Macros:
  MACRO1=definition1
  MACRO2=definition2

That is, a Ddoc file consists of the actual word "Ddoc" followed by a newline, then followed by the actual text of the document, followed by a line containing "Macros:", followed by macro definitions of the form NAME=value. The "Macros:" section is optional (as is the indentation of the macro definitions; it's present here for aesthetic reasons only). You might have guessed already that the syntax $(MACRONAME) expands the macro called MACRONAME into whatever text was ascribed to it in the "Macros:" section. Let's actually test that by saving the following text into a file called e.g. test.dd:

Ddoc
Text with embedded macros such as $(MACRO1) and $(MACRO2) goes here.
Macros:
  MACRO1=definition1
  MACRO2=definition2
  DDOC_COMMENT=
  DDOC=$(BODY)

The last two macro definitions seem to come out of nowhere and deserve some explanation, which this document will provide soon. For now let's "build" this file like this:

~/d/dmd/src/dmd test.dd
cat test.html

(You may of course just type dmd if it's in your $PATH) The produced file, by default carrying the .html extension, contains:

Text with embedded macros such as definition1 and definition2 goes here.

So the macro names got expanded to the text in their respective definitions. Sweet!

The Ddoc Expansion Process

Time to clear the air about the mysterious DDOC_COMMENT= and DDOC=$(BODY) definitions. When dmd processes a .ddoc file, it doesn't immediately expand the text; the process goes as follows:

• accumulate the text (sans the opening Ddoc\n) in memory and put it in a variable called BODY;
• when processing reaches the \nMacros:\n section, read and memorize the macro definitions underneath;
• expand and output $(DDOC_COMMENT Generated by Ddoc from filename.dd);
• expand and output the macro DDOC.

The default values of DDOC_COMMENT and DDOC are geared toward building simple HTML files. Indeed, if we remove the macro definitions from the test.dd file and rebuild, the resulting test.html contains:

<html><head>
        <META http-equiv="content-type" content="text/html; charset=utf-8">
        <title>test</title>
        </head><body>
        <h1>test</h1>
        <!-- Generated by Ddoc from test.dd -->

Text with embedded macros such as definition1 and definition2 goes here.

        <hr><small>Page generated by <a href="http://dlang.org/ddoc.html">Ddoc</a>. </small>
        </body></html>

which is a serviceable albeit bland HTML document.

Macros with Parameters

Macros are key to the power of Ddoc for at least two reasons. First, they shorten and simplify the document by allowing you to replace clumsy typesetting directives such as <span class="important">Attention!</span> with $(ATTENTION). Second, they elevate the level of the document you're writing by leveraging the traditional "extra level of indirection". Depending on how you define ATTENTION, you get to format $(ATTENTION) in various formats. For example, for HTML you'd use:

ATTENTION=<span class="important">Attention!</span>

If you don't care to define css classes etc. you may want to just go sloppy:

ATTENTION=<font color="red">Attention!</font>

To format the document as plain text, just write:

ATTENTION=Attention!

And if you want to output LaTeX, write something like:

ATTENTION={\color{red}Attention!}

In short, you get to expand Ddoc documents into many other formats by using macros and combining the documents with appropriately-defined macro batteries. (This document will discuss soon how to effect such combinations.)

For now, let's note that ATTENTION is not quite a sterling example of flexibility; it allows us to render "Attention!" in a special way, but often the need is to render various other words and phrases as attention-attracting text. So what's needed is a macro taking the text to render as a parameter. Here's how to do so in Ddoc.

In a macro definition, certain constructs access arguments as follows:

• $1, $2, $3, ..., $9 expand to the first, second, third, ..., ninth argument;
• $+ expands to all arguments except for the first;
• $0 simply expands to all macro arguments.

To pass arguments to a macro, insert a space after the macro name, then pass the arguments separated them by commas, then close the paren: for example, $(MYMACRO how are you doing, Jeff?) passes the arguments "how are you doing" and "Jeff?" to a macro called MYMACRO.

There are a couple of subtleties related to passing and expanding parameters. The first whitespace after the macro name is "munched", i.e. it just disappears from the expansion. But if there's any whitespace character following the first one, it will be considered part of the first argument. An example will make this clear. Consider the macro definition TEST=|$1|. Then, the invocation

$(TEST abc)

produces

|abc|

whereas the invocation (note the extra space)

$(TEST  abc)

produces (note the extra space before the letters):

| abc|

The whitespace after comma (if present) in multiple argument lists is handled with a similar logic. The first whitespace character immediately after the comma is considered "aesthetic" and not present in the expanded text. However, if the source inserts more than one whitespace character after comma, the extra ones are considered part of the argument. By means of example, consider the macro definition TEST=|$1|$2|. Then we have the following expansions:

$(TEST abc,xyz)

produces

|abc|xyz|

then the invocation (note the extra space)

$(TEST abc, xyz)

produces no change in output:

|abc|xyz|

However, the expansion (there are THREE spaces after the comma):

$(TEST abc,   xyz)

produces the output:

|abc| xyz|

which inserts TWO spaces before "xyz". One has been munched, the other two copied.

Ddoc "understands" parentheses and likes them paired. If a comma occurs within parenthesized text, it won't be considered a macro argument separator. For example, in the invocation $(MYMACRO abc, (xyz, tuv)), MYMACRO receives two arguments, not three. Nested parentheses work as expected, too; only top-level commas within the macro are considered argument separators.

Whitespace in macro definition

In the "Macros:" section, any number of whitespace characters preceding or following the = sign are ignored. That makes it problematic to define macros that start with a space. To remedy that, define this macro:

SPACE = $(SPACE) $(SPACE)

The recursive expansion (which will be explained in detail below) expands to nothing while the macro itself is being expanded; so the final expansion will be a space flanked by two empty strings, which is exactly what we needed. Now it's easy to use space to e.g. indent some text:

INDENT = $(SPACE)$(SPACE)$(SPACE)$(SPACE)$0

Special characters: "(", ")", "$", ","

As shown above, the characters "(", ")", "$", and "," have special meaning to Ddoc. Sometimes it's necessary to "escape" them, i.e. make them part of the produced output without them being interpreted by Ddoc. To achieve that, the following macros are highly useful:

ARGS = $0
COMMA = ,
COMMENT = 
DOLLAR = $
LPAREN = (
RPAREN = )
TAIL = $+