DMD Release Building

Release Engineers

Martin_Nowak

Requirements

• a linux (maybe osx) host
• virtualbox
• vagrant (https://www.vagrantup.com/)
• a Windows 7 license
• an OSX (currently 10.8) license
• a gpg key that is part of our keyring (http://dlang.org/gpg_keys.html)
• an S3 account for http://downloads.dlang.org
• an ssh account for http://ftp.digitalmars.com
• aws client tool

Setup boxes

• See the build script for how to setup the vagrant boxes.

https://github.com/D-Programming-Language/installer/blob/a01d9092a5409e507e1ea1c734f6336b8bfcd2f6/create_dmd_release/build_all.d#L14

Build steps

• Make sure dmd/VERSION and dlang.org/VERSION contains the to be build version
  • dmd/VERSION contains the NEXT version number (up to and including the release), usually should get updated right after a release
  • dlang.org/VERSION contains the LAST version number (including the release and following), should get updated with a release

 cd dmd
 git checkout stable
 echo 2.072.1 > VERSION
 git add VERSION
 git commit -m 'update VERSION file'
 git push upstream stable

 cd dlang.org
 git checkout stable
 echo 2.072.1 > VERSION
 git add VERSION
 git commit -m 'update VERSION file'
 git push upstream stable

• Combining the changelog
  • copy the changelog.dd parts from the dmd/druntime/phobos stable repos to a new dlang.org/changelog/2.068.1.dd
  • use tools/changed to get the bugzilla changelog list
  • Use dlang.org/changelog/update_nav.sh to update prev/next navigation in changelogs
  • Update dlang.org/win32.mak

• Test building the stable branch
  • cd installer/create_dmd_release
  • rdmd build_all v2.070.2 stable
  • possibly fix build script, repos, or update patches

• Tag a new version

Push the tags to your private forks (origin) first to test whether the release builds cleanly. Tags on upstream must not be changed ever. To build from your private forks you need to change the base url in cloneSources.

 for proj in dmd druntime phobos tools dlang.org installer; do
   pushd $proj
   git fetch upstream
   git tag --sign --message='v2.071.2-b4' v2.071.2-b4 upstream/stable
   git push origin v2.071.2-b4
   popd
 done

• Build binaries
  • cd installer/create_dmd_release
  • rm -rf build
  • rdmd build_all v2.070.2 v2.071.2-b4

• Sign binaries

 for file in build/*; do
   gpg2 -b $file
 done

• Perform additional tests
• Upload binaries
  • rsync --progress -av --chmod=ug=rw build/ martin@digitalmars.com:/var/pub/digitalmars.com/
  • Upload to releases or pre-releases
    • aws --profile ddo s3 sync build/ s3://downloads.dlang.org/pre-releases/$(date '+%Y')/ --acl public-read --cache-control max-age=604800
    • aws --profile ddo s3 sync build/ s3://downloads.dlang.org/releases/$(date '+%Y')/ --acl public-read --cache-control max-age=604800
    • Make sure to check for errors during upload and probably repeat

• If everything went right until here, push the tags to upstream

 for proj in dmd druntime phobos tools dlang.org installer; do
   git -C $proj push upstream v2.068.1-b1
 done

• Update downloads.dlang.org
  • cd downloads.dlang.org (https://github.com/braddr/downloads.dlang.org)
  • ls create_dmd_release/build >> make-links
  • edit 'make-links' and add symlinks for all the new files
  • run ./make-links
  • build make -C src
  • and run ./src/build-gen-index s3_index generate to regenerate the download pages to ./ddo
  • aws --profile ddo s3 sync ./ddo/ s3://downloads.dlang.org/ --acl public-read --cache-control max-age=604800
  • commit and push changes to downloads.dlang.org

• Update LATEST_BETA or LATEST on ftp server (for compatibility with older scripts)
  • echo -n 2.071.2-b4 | aws --profile ddo s3 cp - s3://downloads.dlang.org/pre-releases/LATEST --acl public-read
  • echo -n 2.071.2 | aws --profile ddo s3 cp - s3://downloads.dlang.org/releases/LATEST --acl public-read
  • ssh martin@digitalmars.com 'echo -n 2.071.2-b4 > /var/pub/digitalmars.com/LATEST_BETA'
  • ssh martin@digitalmars.com 'echo -n 2.071.2 > /var/pub/digitalmars.com/LATEST'
  • Regenerate downloads.dlang.org index.html if LATEST* was added for the first time

• Merge stable branches into master

 cd dmd # repeat for every repo
 git fetch upstream
 git checkout -b merge_stable upstream/master
 git merge upstream/stable
 # fix any conflicts
 git hub pull new -b master

• Rebuild and sync dlang.org
• Use a tool like Check-My-Links to check the download pages
• Write an announcement to digitalmars.D.announce and/or dmd-beta.
• Update the version in D's Wikipedia article
• Publish announcements to Twitter, Reddit, HackerNews, Slashdot (make noise!)
• Update DMD version in this D wiki

Draft for DUB release building

 DUB_VERSION=v1.1.1
 cd dub
 git checkout stable
 
 # (... edit change log with the final release date ...)
 
 echo "module dub.version_;" > source/dub/version_.d
 echo "enum dubVersion = \"$DUB_VERSION\";" >> source/dub/version_.d
 git add CHANGELOG.md source/dub/version_.d
 git commit -m "Update change log and version file."
 # should be a proper tag with message, possibly signed
 git tag $DUB_VERSION --sign --message="$DUB_VERSION"
 
 ./build.sh
 
 # if everything went right:
 git push upstream $DUB_VERSION
 
 # (... copy binary to release archive ...)

Note: downloads on code.dlang.org are automatically built once the new version tag gets pushed.