Difference between revisions of "Continuous Integration"
ZombineDev (talk | contribs) (Update DUB version) |
WebFreak001 (talk | contribs) (Add GitHub actions using setup-dlang) |
||
(9 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
Continuously testing your project is ensures a conflict-free shipping. | Continuously testing your project is ensures a conflict-free shipping. | ||
− | For all the listed Continuous Integration | + | For all the listed Continuous Integration providers, you will need a free account |
to enable building. | to enable building. | ||
Line 6: | Line 6: | ||
Travis has built-in support for D and building your project with D is as easy as | Travis has built-in support for D and building your project with D is as easy as | ||
− | adding to your <code>.travis.yml</code> the compilers to be tested : | + | adding to your <code>.travis.yml</code> file, the compilers to be tested : |
<syntaxhighlight lang="yaml"> | <syntaxhighlight lang="yaml"> | ||
Line 23: | Line 23: | ||
- ldc-1.0.0 | - ldc-1.0.0 | ||
</syntaxhighlight> | </syntaxhighlight> | ||
+ | |||
+ | A full list of supported compiler versions along with the corresponding DMD frontend versions is available at https://semitwist.com/travis-d-compilers. | ||
Travis also offers | Travis also offers | ||
Line 50: | Line 52: | ||
script: | script: | ||
- dub test --arch "$ARCH" | - dub test --arch "$ARCH" | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | === Enabling core dump === | ||
+ | |||
+ | You can enable a core dump on case of test failing with this : | ||
+ | |||
+ | <syntaxhighlight lang="yaml"> | ||
+ | addons: | ||
+ | apt: | ||
+ | update: true | ||
+ | packages: | ||
+ | - gdb | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <syntaxhighlight lang="yaml"> | ||
+ | # Enable core dumps | ||
+ | before_script: | ||
+ | - ulimit -c unlimited -S | ||
+ | - sudo mkdir /cores/ && sudo chmod 777 /cores/ | ||
+ | - echo "/cores/%E.%p" | sudo tee /proc/sys/kernel/core_pattern | ||
+ | |||
+ | after_failure: | ||
+ | - ./scripts/core-dump.sh | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | And adding "core-dump.sh" script to the repository, on "scripts" folder : | ||
+ | <syntaxhighlight lang="bash"> | ||
+ | #!/bin/bash | ||
+ | |||
+ | set -veo pipefail | ||
+ | |||
+ | for file in /cores/*; do | ||
+ | echo "Core file: $file" | ||
+ | EXECUTABLE_FILE=$(echo "$file" | sed 's/\/cores\///' | sed -E 's/\.[0-9]+$//' | tr '!' '/') | ||
+ | echo "Executable: $EXECUTABLE_FILE" | ||
+ | gdb -c "$file" "$EXECUTABLE_FILE" -ex 'set print pretty on' -ex "thread apply all bt" -ex "set pagination 0" -ex 'info files' -ex 'p $_siginfo._sifields._sigfault.si_addr' -ex 'info locals' -ex 'info frame' -ex 'info args' -ex 'p *sym' -batch | ||
+ | done | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Line 76: | Line 115: | ||
environment: | environment: | ||
matrix: | matrix: | ||
− | + | - DC: dmd | |
− | + | DVersion: stable | |
− | + | arch: x64 | |
− | + | - DC: dmd | |
− | + | DVersion: stable | |
− | + | arch: x86 | |
− | + | - DC: ldc | |
− | + | DVersion: stable | |
− | + | arch: x86 | |
− | + | - DC: ldc | |
− | + | DVersion: stable | |
− | + | arch: x64 | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
skip_tags: true | skip_tags: true | ||
Line 102: | Line 135: | ||
install: | install: | ||
+ | - ps: function ResolveLatestDMD | ||
+ | { | ||
+ | $version = $env:DVersion; | ||
+ | if($version -eq "stable") { | ||
+ | $latest = (Invoke-WebRequest "http://downloads.dlang.org/releases/LATEST").toString(); | ||
+ | $url = "http://downloads.dlang.org/releases/2.x/$($latest)/dmd.$($latest).windows.7z"; | ||
+ | }elseif($version -eq "beta") { | ||
+ | $latest = (Invoke-WebRequest "http://downloads.dlang.org/pre-releases/LATEST").toString(); | ||
+ | $latestVersion = $latest.split("-")[0].split("~")[0]; | ||
+ | $url = "http://downloads.dlang.org/pre-releases/2.x/$($latestVersion)/dmd.$($latest).windows.7z"; | ||
+ | }else { | ||
+ | $url = "http://downloads.dlang.org/releases/2.x/$($version)/dmd.$($version).windows.7z"; | ||
+ | } | ||
+ | if($env:arch -eq "x64"){ | ||
+ | $env:PATH += ";C:\dmd2\windows\bin64;"; | ||
+ | } | ||
+ | $env:PATH += ";C:\dmd2\windows\bin;"; | ||
+ | return $url; | ||
+ | } | ||
+ | - ps: function ResolveLatestLDC | ||
+ | { | ||
+ | $version = $env:DVersion; | ||
+ | $arch = $env:arch; | ||
+ | if($version -eq "stable") { | ||
+ | $latest = (Invoke-WebRequest "https://ldc-developers.github.io/LATEST").toString().replace("`n","").replace("`r",""); | ||
+ | $url = "https://github.com/ldc-developers/ldc/releases/download/v$($latest)/ldc2-$($latest)-windows-$($arch).7z"; | ||
+ | }elseif($version -eq "beta") { | ||
+ | $latest = (Invoke-WebRequest "https://ldc-developers.github.io/LATEST_BETA").toString().replace("`n","").replace("`r",""); | ||
+ | $url = "https://github.com/ldc-developers/ldc/releases/download/v$($latest)/ldc2-$($latest)-windows-$($arch).7z"; | ||
+ | } else { | ||
+ | $latest = $version; | ||
+ | $url = "https://github.com/ldc-developers/ldc/releases/download/v$($version)/ldc2-$($version)-windows-$($arch).7z"; | ||
+ | } | ||
+ | $env:PATH += ";C:\ldc2-$($latest)-windows-$($arch)\bin"; | ||
+ | $env:DC = "ldc2"; | ||
+ | return $url; | ||
+ | } | ||
- ps: function SetUpDCompiler | - ps: function SetUpDCompiler | ||
{ | { | ||
+ | $env:toolchain = "msvc"; | ||
if($env:DC -eq "dmd"){ | if($env:DC -eq "dmd"){ | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
echo "downloading ..."; | echo "downloading ..."; | ||
− | $ | + | $url = ResolveLatestDMD; |
− | $ | + | echo $url; |
− | Invoke-WebRequest | + | Invoke-WebRequest $url -OutFile "c:\dmd.7z"; |
echo "finished."; | echo "finished."; | ||
pushd c:\\; | pushd c:\\; | ||
Line 122: | Line 187: | ||
elseif($env:DC -eq "ldc"){ | elseif($env:DC -eq "ldc"){ | ||
echo "downloading ..."; | echo "downloading ..."; | ||
− | + | $url = ResolveLatestLDC; | |
− | + | echo $url; | |
− | + | Invoke-WebRequest $url -OutFile "c:\ldc.zip"; | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | Invoke-WebRequest | ||
echo "finished."; | echo "finished."; | ||
pushd c:\\; | pushd c:\\; | ||
Line 138: | Line 197: | ||
} | } | ||
- ps: SetUpDCompiler | - ps: SetUpDCompiler | ||
− | + | ||
− | |||
− | |||
− | |||
before_build: | before_build: | ||
+ | |||
- ps: if($env:arch -eq "x86"){ | - ps: if($env:arch -eq "x86"){ | ||
$env:compilersetupargs = "x86"; | $env:compilersetupargs = "x86"; | ||
$env:Darch = "x86"; | $env:Darch = "x86"; | ||
− | + | $env:DConf = "m32"; | |
− | elseif($env:arch -eq "x64"){ | + | }elseif($env:arch -eq "x64"){ |
$env:compilersetupargs = "amd64"; | $env:compilersetupargs = "amd64"; | ||
$env:Darch = "x86_64"; | $env:Darch = "x86_64"; | ||
+ | $env:DConf = "m64"; | ||
} | } | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
- ps: $env:compilersetup = "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall"; | - ps: $env:compilersetup = "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall"; | ||
- '"%compilersetup%" %compilersetupargs%' | - '"%compilersetup%" %compilersetupargs%' | ||
Line 170: | Line 220: | ||
- echo %Darch% | - echo %Darch% | ||
- echo %DC% | - echo %DC% | ||
+ | - echo %DMD% | ||
- echo %PATH% | - echo %PATH% | ||
− | - '%DC% -- | + | - dub --version |
+ | - '%DC% --help' | ||
- dub test --arch=%Darch% --compiler=%DC% | - dub test --arch=%Darch% --compiler=%DC% | ||
</syntaxhighlight> | </syntaxhighlight> | ||
+ | Like in Travis, you can use a specify version, setting '''DVersion''', and you can choose between the 32bit or 64bit compiler with '''arch''' : | ||
+ | |||
+ | <syntaxhighlight lang="yaml"> | ||
+ | matrix: | ||
+ | - DC: dmd | ||
+ | DVersion: 2.087.1 | ||
+ | arch: x86 | ||
+ | </syntaxhighlight> | ||
If you have troubles with a specific version or compiler on Windows, you might want to allow failure in your build matrix until you get them fixed. For example to allow failure with ldc, you would add the following: | If you have troubles with a specific version or compiler on Windows, you might want to allow failure in your build matrix until you get them fixed. For example to allow failure with ldc, you would add the following: | ||
Line 184: | Line 244: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | For a up-to-date AppVeyor setup, you can also always head to [https://github.com/libmir/mir/blob/master/appveyor.yml Mir]. | + | For a up-to-date AppVeyor setup, you can also always head to [https://github.com/Abscissa/AppVeyor-D/blob/master/appveyor.yml AppVeyor-D], or [https://github.com/dlang/dub/blob/master/appveyor.yml DUB] or [https://github.com/libmir/mir/blob/master/appveyor.yml Mir]. |
+ | |||
+ | |||
+ | == Gitlab CI == | ||
+ | |||
+ | Gitlab own CI service requires to use a docker image to test D your project. See [https://gitlab.com/basile.b/dlang-ci minimal example] | ||
+ | |||
+ | [[Category:Building]] | ||
+ | |||
+ | |||
+ | == GitHub Actions (Windows, Linux, OSX) == | ||
+ | |||
+ | Using the [https://github.com/marketplace/actions/d-compiler-installation setup-dlang] action it is possible to trivially use GitHub Actions with D, all you need to do is add a D installation step to the workflow yaml file: | ||
+ | |||
+ | <syntaxhighlight lang="yaml"> | ||
+ | steps: | ||
+ | - uses: actions/checkout@v2 # checkout respository | ||
+ | - uses: dlang-community/setup-dlang@v1 # install & configure a D compiler | ||
+ | with: | ||
+ | compiler: dmd-latest # specify latest dmd | ||
+ | - name: Run tests # use D as usual | ||
+ | run: dub test | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | With GitHub Actions it is also possible to test on multiple platforms and compilers without repeating your workflow definitions multiple times. Here is a full workflow example: | ||
+ | |||
+ | <syntaxhighlight lang="yaml"> | ||
+ | name: Run all D Tests | ||
+ | on: [push, pull_request] | ||
+ | |||
+ | jobs: | ||
+ | test: | ||
+ | name: Dub Tests | ||
+ | strategy: | ||
+ | matrix: | ||
+ | os: [ubuntu-latest, windows-latest, macOS-latest] | ||
+ | dc: [dmd-latest, ldc-latest, dmd-2.085.0, ldc-1.17.0] | ||
+ | exclude: | ||
+ | - { os: macOS-latest, dc: dmd-2.085.0 } | ||
+ | |||
+ | runs-on: ${{ matrix.os }} | ||
+ | steps: | ||
+ | - uses: actions/checkout@v2 | ||
+ | |||
+ | - name: Install D compiler | ||
+ | uses: dlang-community/setup-dlang@v1 | ||
+ | with: | ||
+ | compiler: ${{ matrix.dc }} | ||
+ | |||
+ | - name: Run tests | ||
+ | run: dub -q test | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | The above example test 11 possible combinations: the latest <code>dmd</code> and latest <code>ldc</code> on all three platforms, <code>ldc-1.17.0</code> on all three platforms, and <code>dmd-2.085.0</code> on <code>ubuntu-latest</code> and <code>windows-latest</code>. | ||
+ | |||
+ | Valid version examples are: | ||
+ | |||
+ | * <code>dmd-latest</code> | ||
+ | * <code>ldc-latest</code> | ||
+ | * <code>dmd-beta</code> | ||
+ | * <code>ldc-beta</code> | ||
+ | * <code>dmd-2.088.0</code> | ||
+ | * <code>dmd-2.088.0-beta.2</code> | ||
+ | * <code>ldc-1.17.0</code> | ||
+ | * <code>ldc-1.18.0-beta1</code> | ||
+ | * <code>dmd-master</code> | ||
+ | * <code>ldc-master</code> |
Latest revision as of 12:41, 16 July 2020
Continuously testing your project is ensures a conflict-free shipping. For all the listed Continuous Integration providers, you will need a free account to enable building.
Contents
Travis
Travis has built-in support for D and building your project with D is as easy as
adding to your .travis.yml
file, the compilers to be tested :
# latest dmd, gdc and ldc
d:
- dmd
- gdc
- ldc
You can also choose specific versions:
d:
- dmd-2.071.0
- ldc-1.0.0
A full list of supported compiler versions along with the corresponding DMD frontend versions is available at https://semitwist.com/travis-d-compilers.
Travis also offers OS X build environment, which you can enable by adding:
os:
- linux
- osx
More information is also available at Travis's documentation.
Testing 32-bit
All Travis containers use 64-bit, so if you want to test 32-bit is a bit tricky. One common trick is to use Travis build matrix feature to individually toggle extra builds as Mac OS X is x86-64 only since 10.7.
env:
- ARCH="x86_64"
matrix:
include:
- {os: linux, d: dmd-2.071.0, env: ARCH="x86", addons: {apt: {packages: [[gcc-multilib]]}}}
- {os: linux, d: ldc-1.0.0, env: ARCH="x86", addons: {apt: {packages: [[gcc-multilib]]}}}
script:
- dub test --arch "$ARCH"
Enabling core dump
You can enable a core dump on case of test failing with this :
addons:
apt:
update: true
packages:
- gdb
# Enable core dumps
before_script:
- ulimit -c unlimited -S
- sudo mkdir /cores/ && sudo chmod 777 /cores/
- echo "/cores/%E.%p" | sudo tee /proc/sys/kernel/core_pattern
after_failure:
- ./scripts/core-dump.sh
And adding "core-dump.sh" script to the repository, on "scripts" folder :
#!/bin/bash
set -veo pipefail
for file in /cores/*; do
echo "Core file: $file"
EXECUTABLE_FILE=$(echo "$file" | sed 's/\/cores\///' | sed -E 's/\.[0-9]+$//' | tr '!' '/')
echo "Executable: $EXECUTABLE_FILE"
gdb -c "$file" "$EXECUTABLE_FILE" -ex 'set print pretty on' -ex "thread apply all bt" -ex "set pagination 0" -ex 'info files' -ex 'p $_siginfo._sifields._sigfault.si_addr' -ex 'info locals' -ex 'info frame' -ex 'info args' -ex 'p *sym' -batch
done
CircleCi
CircleCi doesn't support D out-of-the-box yet, but it is easy to install dmd
and
dub
with a single line in your `circle.yml`:
dependencies:
override:
- source "$(curl -fsS --retry 3 https://dlang.org/install.sh | bash -s dmd --activate)"
test:
override:
- dub test
AppVeyor (Windows)
Testing your project on Windows is currently a bit more complicated, but we are working on simplifying this integration. At the moment the best way to go is to extend your setup from this `appveyor.yml`:
platform: x64
environment:
matrix:
- DC: dmd
DVersion: stable
arch: x64
- DC: dmd
DVersion: stable
arch: x86
- DC: ldc
DVersion: stable
arch: x86
- DC: ldc
DVersion: stable
arch: x64
skip_tags: true
branches:
only:
- master
- stable
install:
- ps: function ResolveLatestDMD
{
$version = $env:DVersion;
if($version -eq "stable") {
$latest = (Invoke-WebRequest "http://downloads.dlang.org/releases/LATEST").toString();
$url = "http://downloads.dlang.org/releases/2.x/$($latest)/dmd.$($latest).windows.7z";
}elseif($version -eq "beta") {
$latest = (Invoke-WebRequest "http://downloads.dlang.org/pre-releases/LATEST").toString();
$latestVersion = $latest.split("-")[0].split("~")[0];
$url = "http://downloads.dlang.org/pre-releases/2.x/$($latestVersion)/dmd.$($latest).windows.7z";
}else {
$url = "http://downloads.dlang.org/releases/2.x/$($version)/dmd.$($version).windows.7z";
}
if($env:arch -eq "x64"){
$env:PATH += ";C:\dmd2\windows\bin64;";
}
$env:PATH += ";C:\dmd2\windows\bin;";
return $url;
}
- ps: function ResolveLatestLDC
{
$version = $env:DVersion;
$arch = $env:arch;
if($version -eq "stable") {
$latest = (Invoke-WebRequest "https://ldc-developers.github.io/LATEST").toString().replace("`n","").replace("`r","");
$url = "https://github.com/ldc-developers/ldc/releases/download/v$($latest)/ldc2-$($latest)-windows-$($arch).7z";
}elseif($version -eq "beta") {
$latest = (Invoke-WebRequest "https://ldc-developers.github.io/LATEST_BETA").toString().replace("`n","").replace("`r","");
$url = "https://github.com/ldc-developers/ldc/releases/download/v$($latest)/ldc2-$($latest)-windows-$($arch).7z";
} else {
$latest = $version;
$url = "https://github.com/ldc-developers/ldc/releases/download/v$($version)/ldc2-$($version)-windows-$($arch).7z";
}
$env:PATH += ";C:\ldc2-$($latest)-windows-$($arch)\bin";
$env:DC = "ldc2";
return $url;
}
- ps: function SetUpDCompiler
{
$env:toolchain = "msvc";
if($env:DC -eq "dmd"){
echo "downloading ...";
$url = ResolveLatestDMD;
echo $url;
Invoke-WebRequest $url -OutFile "c:\dmd.7z";
echo "finished.";
pushd c:\\;
7z x dmd.7z > $null;
popd;
}
elseif($env:DC -eq "ldc"){
echo "downloading ...";
$url = ResolveLatestLDC;
echo $url;
Invoke-WebRequest $url -OutFile "c:\ldc.zip";
echo "finished.";
pushd c:\\;
7z x ldc.zip > $null;
popd;
}
}
- ps: SetUpDCompiler
before_build:
- ps: if($env:arch -eq "x86"){
$env:compilersetupargs = "x86";
$env:Darch = "x86";
$env:DConf = "m32";
}elseif($env:arch -eq "x64"){
$env:compilersetupargs = "amd64";
$env:Darch = "x86_64";
$env:DConf = "m64";
}
- ps: $env:compilersetup = "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall";
- '"%compilersetup%" %compilersetupargs%'
build_script:
- echo dummy build script - dont remove me
test_script:
- echo %PLATFORM%
- echo %Darch%
- echo %DC%
- echo %DMD%
- echo %PATH%
- dub --version
- '%DC% --help'
- dub test --arch=%Darch% --compiler=%DC%
Like in Travis, you can use a specify version, setting DVersion, and you can choose between the 32bit or 64bit compiler with arch :
matrix:
- DC: dmd
DVersion: 2.087.1
arch: x86
If you have troubles with a specific version or compiler on Windows, you might want to allow failure in your build matrix until you get them fixed. For example to allow failure with ldc, you would add the following:
matrix:
allow_failures:
- DC: ldc
For a up-to-date AppVeyor setup, you can also always head to AppVeyor-D, or DUB or Mir.
Gitlab CI
Gitlab own CI service requires to use a docker image to test D your project. See minimal example
GitHub Actions (Windows, Linux, OSX)
Using the setup-dlang action it is possible to trivially use GitHub Actions with D, all you need to do is add a D installation step to the workflow yaml file:
steps:
- uses: actions/checkout@v2 # checkout respository
- uses: dlang-community/setup-dlang@v1 # install & configure a D compiler
with:
compiler: dmd-latest # specify latest dmd
- name: Run tests # use D as usual
run: dub test
With GitHub Actions it is also possible to test on multiple platforms and compilers without repeating your workflow definitions multiple times. Here is a full workflow example:
name: Run all D Tests
on: [push, pull_request]
jobs:
test:
name: Dub Tests
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
dc: [dmd-latest, ldc-latest, dmd-2.085.0, ldc-1.17.0]
exclude:
- { os: macOS-latest, dc: dmd-2.085.0 }
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2
- name: Install D compiler
uses: dlang-community/setup-dlang@v1
with:
compiler: ${{ matrix.dc }}
- name: Run tests
run: dub -q test
The above example test 11 possible combinations: the latest dmd
and latest ldc
on all three platforms, ldc-1.17.0
on all three platforms, and dmd-2.085.0
on ubuntu-latest
and windows-latest
.
Valid version examples are:
dmd-latest
ldc-latest
dmd-beta
ldc-beta
dmd-2.088.0
dmd-2.088.0-beta.2
ldc-1.17.0
ldc-1.18.0-beta1
dmd-master
ldc-master