Compiling and linking with DMD on Windows
Contents
- 1 Compiling and linking a simple D project
- 2 Linking manually
- 3 Optlink's Syntax and Switches
- 4 Static Libraries and Import Paths
- 5 Passing search directories for static library files to Optlink
- 6 Getting Symbolic Information
- 7 Calling Conventions
- 8 Writing and Using D Interface Files
- 9 More information about Linkers
Compiling and linking a simple D project
Make sure the directory path to DMD2 and Optlink is listed in your PATH environment variable. This directory is usually C:\DMD\dmd2\windows\bin on Windows.
> dmd Digital Mars D Compiler v2.051 > link OPTLINK (R) for Win32 Release 8.00.8
Invoking dmd and optlink is shown above. If you get similar results, you're good to go. If you don't know what environment variable are, read the Environment Variables Guide. Be aware that on Windows you might have to log out and log in again before any changes to the PATH variable are seen throughout the entire system.
Our test project consists of two files:
project.d:
module project;
import foo;
void main()
{
callFoo("main", 10);
}
foo.d:
module foo;
import std.stdio;
void callFoo(string arg, int num)
{
writefln("Called from %s with number %s.", arg, num);
}
There are multiple ways of compiling and linking the two modules. The straightforward way is to list all files to be compiled and linked on the command line when invoking DMD:
dmd project.d foo.d
This will produce an executable with the name of the first module provided on the command line ("project.exe" on Windows). Behind-the-scenes DMD compiles the two modules and links them together to form one object file ("project.obj" on Windows). To create the executable DMD will link the project.obj file with the Phobos standard library (\DMD\dmd2\windows\lib\phobos.lib on Windows, or DMD\dmd2\linux\lib\libphobos2.a on linux) and any OS-specific libraries which the executable needs to succesfully run.
Linking manually
DMD uses the -c switch to skip the linking process and only compile the .d files:
dmd -c project.d foo.d
This will produce one object file per each D module:
project.obj foo.obj
Now we need to link these two object files with the Phobos standard library and any essential Operating System libraries. There are two ways of doing this.
The easy way is to let DMD call the linker automatically:
dmd project.obj foo.obj
The reason this works is because DMD parses its config file which specifies the directory paths of the Phobos and OS libraries in the LIB environment variable. Optlink will look for this variable in the linking process.
The hard way of linking is invoking the linker manually. This implies learning a thing or two about Optlink's syntax and its switches.
We know our project depends on the Phobos library, but how do we figure out which OS libraries we need to link in? A quick way to find that out is to invoke DMD with the -v (verbose) switch:
dmd -v project.obj foo.obj binary D:\DMD\dmd2\windows\bin\dmd.exe version v2.051 config D:\DMD\dmd2\windows\bin\sc.ini D:\DMD\dmd2\windows\bin\link.exe project+foo,,,user32+kernel32/noi;
Note that we're calling DMD on object files, not .d files. Invoking DMD with .d files would compile the files and produce several pages of information due to the verbose switch. Using only .obj files DMD will know all it has to do is invoke the linker on the files.
The last line shows how the linker was invoked. Optlink's syntax and switches are explained in the next section. To try your hand at linking on your own, delete any existing object files and executables, and use DMD to compile, Optlink to link:
dmd -c project.d foo.d link.exe project+foo,,,user32+kernel32/noi;
Optlink's Syntax and Switches
Optlink has the following command-line form:
link obj[,out[,map[,lib[,def[,res]]]]]
Object files can be separated with either a blank space or a + sign. Their file extensions don't have to be provided. The full documentation on Optlink's syntax is provided here.
Optlink also has numerous switches you can use. To list all of them, invoke the linker with the /h (help) switch:
link /h
Optlink's switches are described in detail here.
Now we're ready to decipher the linker command from the previous section:
link.exe project+foo,,,user32+kernel32/noi;
The first argument to Optlink is project+foo. These are two object files, separated by the + sign. Optlink automatically appends file extensions to any object or library names if you don't provide them yourself. The second and third arguments are not needed, and remain blank. The fourth argument are the libraries Optlink will link with. user32+kernel32 will expand to user32.lib and kernel32.lib. These are located in \DMD\dmd2\windows\lib\ by default.
You'll notice that we didn't specify where the two library files (user32.lib and kernel32.lib) are located . This is because Optlink parses a special configuration file located at \DMD\dmd2\windows\bin\sc.ini. When parsed, this config file modifies the LIB environment variable and adds a couple of paths, specifically it adds the path to Phobos and some system libraries. Optlink then reads the LIB environment variable for any paths. These will be used to find any library files that were specified without an absolute path.
The /noi switch forces the linker to not ignore the casing of identifiers and symbols found in object and library files.
The semicolon at the end of the invocation can be ignored.
Static Libraries and Import Paths
When your project grows large you might start experiencing slow compilation times. A common cause of these slowdowns is the constant re-compilation of modules which haven't really changed since the last time they were compiled. These problems can be solved several ways, one of which is separating relevant modules to their own static libraries. Other solutions can be to use a good build system, but this is beyond the scope of this tutorial.
Our project might have a collection of modules which belong to the libfoo library, and a front-end called driver.d. Here's what the directory structure might look like:
C:\Project\libs\libfoo\foo.d C:\Project\libs\libfoo\bar.d C:\Project\driver.d
Here are the file contents:
foo.d:
module libfoo.foo;
import std.stdio : writeln;
struct Rectangle
{
int width, height;
void display()
{
writefln("w: %s, h: %s", width, height);
}
}
bar.d:
module libfoo.bar;
import libfoo.foo;
Rectangle getRectangle()
{
return Rectangle(80, 40);
}
driver.d:
module driver;
import std.stdio;
import libfoo.bar;
void main()
{
auto rect = getRectangle();
rect.display();
}
We'll build a single .lib file out of all the modules belonging to the libfoo package. To do this, we pass the -lib switch when building libfoo's modules with DMD:
cd libs\libfoo dmd -lib -oflibfoo.lib foo.d bar.d
The -of switch is used to set a custom output filename, in this case libfoo.lib. One nice consequence of having a .lib file is that we no longer have to specify all the .d files belonging to libfoo when invoking DMD. Instead, we can use the -I import switch.
The import switch specifies a directory DMD will search for when it needs to find modules specified in import statements in your module files. We can use multiple import switches. In our driver.d file, we have these two import statements:
import std.stdio; import libfoo.bar;
The first one tells DMD that the stdio module belongs to the std package. A package is equivalent to a directory name, so stdio should be found in the std directory.
Before compilation DMD parses the sc.ini config file, located in \DMD\dmd2\windows\bin\. The config file modifies the DFLAGS environment variable, and adds import switches with the paths to the Phobos and Druntime source code directories. The location of the Phobos source code is:
\DMD\dmd2\src\phobos
Inside this directory you will find the std subdirectory.
We won't be changing our sc.ini file, instead we'll provide the import switch on the command line. Since our package name is libfoo, we have to specify the directory it is contained within for the import switch. If you accidentally specify the wrong directory, you might get an error such as the following:
dmd driver.d -I%cd%\ driver.d(3): Error: module bar is in file 'libfoo\bar.d' which cannot be read import path[0] = D:\Project\ import path[1] = D:\DMD\dmd2\windows\bin\..\..\src\phobos import path[2] = D:\DMD\dmd2\windows\bin\..\..\src\druntime\import
(%cd% expands to the current directory before calling dmd).
The libfoo directory is located in the libs subdirectory of our project, which explains the errors. The correct command is:
dmd driver.d -I%cd%\libs\
Unfortunately, now we're getting linker errors:
OPTLINK (R) for Win32 Release 8.00.8 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html driver.obj(driver) Error 42: Symbol Undefined _D6libfoo3bar12__ModuleInfoZ driver.obj(driver) Error 42: Symbol Undefined _D6libfoo3bar12getRectangleFZS6libfoo3foo9Rectangle driver.obj(driver) Error 42: Symbol Undefined _D6libfoo3foo9Rectangle7displayMFZv --- errorlevel 3
The fix is to specify the prebuilt .lib file on the command line:
dmd driver.d %cd%\libs\libfoo\libfoo.lib -I%cd%\libs\
The reason you have to manually specify the .lib file is because DMD will not compile the modules found in packages specified by the import switch. For that reason, the import switch is useful when we want to avoid recompilation of modules that are already built. Hence, using the import switch can considerably speed up the build process.
When you download 3rd party D static libraries, you will often find an import subdirectory within these libraries. This directory is normally used with the -I switch in your project. Some libraries don't come with this directory, in this case you can probably use their root directory with the import switch. A library might also come with some build scripts or instructions on how to create one or more .lib static library files.
Once you're familiar with these concepts, all you have to do to take advantage of a 3rd party static library is:
- Add the path to the import directory of the library by using the -I switch
- Link with the required .lib files of the library when invoking DMD
Passing search directories for static library files to Optlink
To specify a search directory for static library files, use the -L+ switch followed by the relative or absolute directory path. If you have this project directory:
C:\Project\main.d C:\Project\lib\mylib.lib
where main.d depends on the mylib library, you can compile via:
dmd -L+.\lib\ driver.d mylib.lib
The -L+ switch needs to be followed by a valid directory specifier syntax. You cannot specify -L+lib or -L+lib\, but -L+.\lib\, -L+%cd%\lib\ or another relative or absolute directory such as -L+..\..\ will be accepted.
Note: The path must be followed by a backslash. E.g. -L+lib may not work, -L+lib\ should be used instead.
Note: Using '-L+' will not work if you use it together with the -lib switch. This is normal since you do not need to specify any other static libraries when you're building a new static library.
Getting Symbolic Information
The DMC compiler comes with a useful utility called libunres. You can download DMC freely from here (DMC should be listed in the third table). Once you have the compiler installed, you will find libunres inside the \dm\bin folder. You might want to add \dm\bin to your PATH, or alternatively copy libunres.exe to your DMD\bin directory.
Using libunres is straightforward. Calling it on any object or library file will list most of its symbols. If we run it on the foo.obj file from the first section, it will print out a large list of symbol names, some of which are displayed here:
libunres foo.obj Unresolved externals: _D11TypeInfo_Aa6__initZ _D12TypeInfo_Aya6__initZ _D14TypeInfo_Const6__vtblZ _D15TypeInfo_Struct6__vtblZ _D3std3utf12isValidDcharFNaNbNfwZb ...
Being able to see the symbol names is important when you're trying to fix common linking errors. By examining the symbol table you might discover that you've accidentally forgot to add a calling convention specifier in your code. Another common error is specifying the wrong calling convention. This often happens when linking with WinAPI libraries, since some functions need the extern(Windows) specifier, while others might need the extern(C) specifier. Calling conventions are discussed in the next section.
You can call libunres without any arguments to get a list of switches.
Here's a sample call with the -d switch:
libunres -d foo.obj _D3foo12__ModuleInfoZ _D3foo7callFooFAyaiZv _D3std4conv16__T5parseTkTAyaZ5parseFKAyaZk4signxi
These are D2 mangled names. There are various reasons why D mangles names, compared to a typical ANSI C compiler which usually doesn't mangle its symbol names (although a C compiler might add an underscore before a symbol name).
You can read more about why mangling is necessary and how it works here.
Calling Conventions
A calling convention specifies how a function receives parameters and how it returns a result. Specifically, it specifies whether the parameters are stored in registers or the call stack, in which order the parameters are passed in, and who is responsible for the cleanup code (the caller or the callee). You can read more about calling conventions here.
On the Windows platform there are at least 3 calling conventions that you should be aware of: C, D and Windows.
Here's an example project with two files that have functions with various calling conventions:
foo.d:
module foo;
extern(C)
{
int sum(int x, int y)
{
return x + y;
}
}
struct Rectangle
{
float width, height;
}
extern(Windows) Rectangle getRectangle(float size)
{
return Rectangle(size*2, size);
}
main.d:
module main;
extern(C) int sum(int x, int y);
struct Rectangle
{
float width, height;
}
extern(Windows) Rectangle getRectangle(float size);
void main()
{
assert(sum(5, 5) == 10);
auto rect = getRectangle(25);
assert(rect.width == 50);
assert(rect.height == 25);
}
Notice that we did not import the foo module in our main module. Instead, we provide function prototypes with no body definitions. We did have to define the Rectangle structure again, otherwise we wouldn't be able to link to the getRectangle function since it returns an instance of a Rectangle structure. Without knowing what kind of type a function returns, the linker can't do its job properly. In D, the calling convention is set by using the linkage attribute extern( ident ). We can set the linkage on a function by function basis, or we can wrap multiple functions under a single calling convention block. We can even set a linkage attribute for an entire module. More about the linkage attribute syntax can be found here.
Once we have the prototypes and the calling convention correctly set, we can compile and link the two modules:
dmd -c foo.d dmd -c main.d dmd -fmain.exe main.obj foo.obj
Note that it isn't necessary to separately compile the two modules, we can let DMD do all the work at once:
dmd main.d foo.d
If you run the resulting executable, all the asserts should pass.
Of course, it doesn't make much sense to create functions with C linkage types in D if they're only going to be used in D programs. The real use case of C and Windows linkage types is being able to link to object or static library files created by other C compilers. It's also useful in cases where you want to build a D object or library that has an entry point function written in C. This would allow applications written in other languages to take advantage of the functionality of your D code. Functions that require the Windows linkage types are most often used with Microsoft Windows DLLs, these provide access to the Windows API.
Writing and Using D Interface Files
What if we didn't have the source code to the implementation of a D static library? Let's see what happens if we try to create a function prototype with the same name as a function in another D module, and try to link the two together:
foo.d:
module foo;
extern(D)
{
string getString()
{
return "I'm just a D string.";
}
}
main.d:
module main;
import std.stdio;
extern(D) string getString();
void main()
{
writeln(getString());
}
Notice we don't import the foo module in the main module.
We attempt to compile and link:
dmd -c main.d dmd -c foo.d dmd -ofmain.exe main.obj foo.obj OPTLINK (R) for Win32 Release 8.00.8 main.obj(main) Error 42: Symbol Undefined _D4main9getStringFZAya
But we get a linker error. Here's where libunres comes in handy. We can call libunres on both modules and compare their symbol names. By default libunres will show unresolved externals, but using the -d switch shows the symbols defined inside an object file:
libunres main.obj _D4main9getStringFZAya ... libunres -d foo.obj _D3foo9getStringFZAya ...
The getString function encodes the module name it belongs to in its mangled name. The two mangled names don't match, hence the linker error.
So if we can't arbitrarily link D functions without knowing which module they belong to, how could one distribute static libraries with no implementation source code?
The answer to that are D interface files. A D Interface file (.di extension) is similar to a C header file, however it is not part of the D language. It is exclusively a feature of the D compiler. Another difference compared to C header files is that D Interface files can be generated automatically by the compiler, so there is never a need to write them by hand.
Let's say we are writing such a library, called libfoo. Here are two modules from our library:
libfoo\foo.d:
module libfoo.foo;
string getFooString()
{
return "A string from libfoo.foo";
}
libfoo\bar.d:
module libfoo.bar;
string getBarString()
{
return "A string from libfoo.bar";
}
We want to compile and link these two modules into a single library file. We also want to generate the interface files. Ideally we would like to put the interface files to a separate import directory to be easily used in applications. We'll distribute the import directory with the resulting .lib file. Here's the full command that does just that:
dmd -oflibfoo.lib -lib -H -Hd"%cd%"\import\libfoo\ bar.d foo.d"
-oflibfoo.lib will name the output file as 'libfoo.lib'.
-lib will create a static library instead of an executable.
-H will generate D interface files.
-Hd will output the D interface files in the specified directory. %cd% expands to the current directory, while the double quotes are used in case our directory has any spaces in its name.
Now let's say we're the user who wants to statically link with libfoo. We're handed the libfoo.lib file and the libfoo directory which has the import directory, which holds the D interface files. We copy these inside our project directory where we have a simple main.d driver file. Here's what our project structure might look like:
main.d libfoo.lib import\libfoo\foo.di import\libfoo\bar.di
And our main file: main.d:
module main;
import std.stdio;
import libfoo.foo;
import libfoo.bar;
void main()
{
writeln(getFooString());
writeln(getBarString());
}
To compile and link with the library, we invoke DMD from within the same directory as main.d with the command:
dmd -I"%cd%"\import\ main.d libfoo.lib
-I is used to set the interface search path, as explained in the section on import files. Since we might use other libraries in our project, we've made a single import directory. Inside, we have the interface files set in separate directories, depending on which library they belong to. Remember that we had to specify the library file (libfoo.lib) on the command line.
The project should compile and link, and you can then invoke the application:
main.exe A string from libfoo.foo A string from libfoo.bar
A similar approach to using interface files is with libraries that are distributed as DLLs. These are discussed in the DLL tutorial (to be done).
More information about Linkers
Beginner's Guide to Linkers (Uses C as example code, but it's still relevant.)
