GDC/Test Suite

GDC

Running the testsuite

Execute this command in the directory where you've run configure (see GDC/Installation) to run the D test suite:

make check-d

Note: You must have built the gdc compiler using make before running the tests.

The summary and log will be saved in gcc/testsuite/gdc/ as gdc.sum and gdc.log.

XML output

To obtain xml output for the summary, use this command:

make check-d RUNTESTFLAGS="--xml"

Passing options to gdc

To pass an option to gdc which is used in every test run, use this:

make check-d RUNTESTFLAGS="--target_board=unix/-fno-section-anchors"

The unix part must always be included. Flags are then separated by /. For example, to pass -O3 and -fno-section-anchors, use this:

make check-d RUNTESTFLAGS="--target_board=unix/-fno-section-anchors/O3"

To run the testsuite against multiple targets, if testing multilib builds, you'd specify each target as in the following test flags:

make check-d RUNTESTFLAGS="--target_board=unix\{,-m32,-mx32\}"

In this example, the testsuite is ran three times. First using the default target (in this case, -m64), then it runs testsuite generating code for the targets specified (-m32 and -mx32).

Adjusting the timeouts

The test suite always enforces a certain timeout when compiling a testcase with gdc. If you see UNRESOLVED: lines in your build log with a program timed out warning your machine is too slow and you have to increase the default timeouts.

To change the timeout, edit /usr/share/dejagnu/config/unix.exp and add this line at the end:

set_board_info gcc,timeout 1200

where 1200 is the new timeout in seconds.

Running unit tests

You first have to change the directory to the libdruntime build directory:

cd objdir/${target_triplet}/libphobos/libdruntime/

where ${target_triplet} is something like armv6l-unknown-linux-gnueabihf or x86_64-unknown-linux.

Then build the unittests:

make unittest

and run them:

./unittest | tee unittest.log

Now the phobos tests:

cd ../src
make unittest
./unittest | tee unittest.log

Testing Cross-compilers

Initial setup

In order to test a cross compiler we'll need a remote system which can run the binaries generated by the cross compiler. The remote system needs to have working ssh and scp access. Also make sure that the user that'll execute the tests on the remote system can log in via ssh without entering a password (https://wiki.archlinux.org/index.php/SSH_Keys).

Edit or create ~/.dejagnurc and add this line:

lappend boards_dir "~/.dejagnu.d"

This makes sure that dejagnu will find our new board file.

Then create the board file with this content at: ~/.dejagnu.d/raspberryhf.exp

# Board definition file for board "raspberryhf"

#Load default unix configuration as a base configuration
load_generic_config "unix"

# Network address of board
set_board_info hostname raspberry

# How to log into this board via ssh and copy files via scp.
# Ideally, you'll set it up to not need a password to log in via ssh
# (see e.g. http://www-csli.stanford.edu/semlab/muri/system/howto/ssh.html).
set_board_info username gdc

set_board_info shell_prompt    "$ " 
# For DejaGnu 1.4.3 and above; DejaGnu 1.4.2.x (Debian 3.0) ignores these settings!
set_board_info rsh_prog /usr/bin/ssh
set_board_info rcp_prog /usr/bin/scp

#timeout used to compile and run the tests
set_board_info gcc,timeout 1200
#Other timeouts
set_board_info timeout 1200

Replace raspberry with the remote hostname and gdc with the username of the remote user used to execute the tests. The shell_prompt should be set to the shell prompt your remote user sees when logging in. Note: The shell promp should be a constant value, so if your shell prompt shows the current directory, hostname or something similar it should be reconfigured.

Note: dejagnu expects the ssh server to run on port 22. If your ssh server is on a different port you could for example add a local port redirect:

# Redirect local port 22 to remote-host.org:22 (if your ssh server is reachable on port 22 on the local interface).
# 42640 is the port used to connect to the ssh server.
sudo ssh -fNg -L 22:127.0.0.1:22 jpf@remote-host.org -p 42640

Troubleshooting

Timout hacking

The timeout specified in the raspberryhf.exp board file unfortunately only applies to the compiler. The timeout used to transfer files, delete the files and run the programs can't be adjusted AFAIK. It possible to overwrite the default value though: Open /usr/share/dejagnu/remote.exp search for "set timeout 300" in the function "remote_exec" and replace the value.

This sets all timeout values, but in some cases it looks as it doesn't have an effect: Although the log clearly shows a timeout of 1800 programs may timeout after a few seconds....

Download of ... to raspberryhf failed.

Check if you can upload files to the remote system:

scp hello.txt gdc@raspberry:/tmp/

Running the tests

We can now run the tests:

make check-d RUNTESTFLAGS="--target_board=raspberryhf"

You should see this output:

Test Run By jpf on Wed Dec 11 18:01:48 2013
Target is arm-unknown-linux-gnueabihf
Host   is x86_64-build_unknown-linux-gnu

		=== gdc tests ===

Schedule of variations:
    raspberryhf

Running target raspberryhf
Using ~/.dejagnu.d/raspberryhf.exp as board description file for target.
Running /home/jpf/Dokumente/d/cttest/.build/src/gcc-custom/gcc/testsuite/gdc.test/d_do_test.exp ...

Note: The test files are not being stripped before they're transported using ssh, so the files are actually quite big (~11MB) and cause a lots of traffic. Only use this test method over internet if you have fast internet access....

See also: http://kegel.com/crosstool/current/doc/dejagnu-remote-howto.html

More information

GCC test suite documentation