Development and Release Process

If you're an experienced git user, skip to the summary.

Definitions

Version numbers

We follow the traditional dmd versioning: 2.061. Tags have a v prepended, so the tag for version 2.061.1 is v2.061.1. Versions where the third part is bigger than zero are called Minor Releases. Versions where the third part is equal zero are Major Releases. Minor Releases will only contain regression fixes, Major Releases will contain bug fixes and new features.

The third digit can be omitted for Major Releases: 2.061 is the same as 2.061.0.

Meta: The dmd contributors should choose one style for all offical statements: Tags, packages, Notes on the download page. Either always write out the .0 part or always omit it.

Regression fix vs. Bug fix

A regression fix for release X is a fix for a bug which was newly introduced in release X. We distinguish between regression fixes and bug fixes to avoid introducing more regressions in minor releases. A regression fix especially should not make code invalid which was valid before the regression occurred. As an example: If we have releases 2.061 and 2.061.1 then 2.061.1 should always be "better" than 2.061. It should be tried heavily not to introduce new regressions in minor releases. Code which compiled with 2.061 should still compile in 2.061.1 (there's one rare exception: If a regression allowed code to work in 2.061 which did not work in 2.060 and which is not supposed to work, it can be disabled again in 2.061.1).

So a regression fix is a bug fix which should not have side-effects, doesn't introduce new features and fixes a bug which only recently appeared.

Long standing bugs are especially not regressions! (Or technically probably a regression in a very old release, but as we support only 1 major release we can treat those like bugs. We only care about new regressions)

Small enhancement vs. Big feature

We later suggest different workflows for those two enhancements. Those are easily distinguishable: If you implement the feature by yourself and don't need help / someone to participate / extended review and the code changes are reasonably small, it's a small enhancement. Otherwise it's a big feature.

The workflow for big features allows other contributors to participate in that feature, make special test releases for that feature, etc. If you want / need those, use the Big Feature workflow.

Stable

When is a release stable and therefore ready to be released?

The workflow already guarantees that when a release can be made there has been a feature freeze for a long time. So only bug fixes and regression fixes where allowed for some time which should already stabilize the release. But we should add some more criteria:

• All bugs marked as critical in bugzilla have been fixed
  • For practical reasons we allow exceptions from this rule for the first releases after this workflow has been established
• There are no newly introduced (known) regressions
  • This is a hard rule, there are no exceptions to it. Have a regression? Don't release!
• Later a yet to be determined percentage of non-critical bug will need to be fixed (or new features introduced)
  • This is to make sure a new release actually provides something new and prevent us from releasing too often. Currently this is not an issue.

Apart from these criteria, the final decision is made by the release manager.

Version / Release branch

This wiki page defines version branches, other projects often call those branches release branches. This wiki page will call them version branches for now, but a release branch and a version branch is exactly the same thing.

Don'ts

• do not cherry pick from one official branch into another
• do not rebase official branches
• do not rewrite git history on official branches
• do not merge master into a version branch (The only time when commits go from master to a version branch is when the version branch is being created)
• do not merge feature branches into version branches. Feature branches are always merged into master.

Release Schedule

<x> = time in weeks
vxxx = Tag for version xxx (Marks the release)
rest = branches
\ = merge
------------------------------------------------------------------------------------ master branch. development is never stopped
            \                                             \          
             \                                             ------------------------- 2.063 release branch
              \                                            |        |  <2>   |
               \                                     v2.063.0-b1    |        |
                \                                                   |        |
                 \                                              v2.063.0   v2.063.1
                  \          
                   ----------------------------------------------------------------- 2.062 release branch
                   |         |  <2>    |    <2>   |
           v2.062.0-b1   v2.062.0 v2.062.1   v2.062.2

A new release is made approximately every 2 months. Releases are stabilized in the version branch and beta releases are tagged as part of version.

Once the initial major release on the version branch is done, only regression fixes will be pushed to the version branch, then a minor release is published every 2 weeks from that version branch as long as it's supported (see #Preparing a new minor release).

Feature Previews: A feature branch may need it's own beta release before being merged into master. These are unplanned, announced on the NG, and highly volatile.

As an example we would see these releases for things like 64bit support and User Defined Attributes.

Support times

Currently only the most recent major release is supported. (This might change in the future)

As an example: version 2.061 is released. 2.061 will get updates until 2.062 is released. Then 2.062 will get updates until 2.063 is released etc.

Only supported releases receive minor update releases.

Branching model

Official branches

The following branches will be available on the official (upstream) dmd, druntime and phobos repositories:

• feature branches: New features, in general, are not developed directly on the master branch, but in a dedicated feature branch. An example for such a branch would be an UDA branch for user defined attributes. Contributors can collaborate on the feature by opening pull request targeting that feature branch. When the feature's implementation is more-or-less complete, the feature branch will be merged into the master branch. The master branch is also sporadically merged into the feature branches to get regression fixes and bug fixes and to make sure the feature branch can be merged into master later on without problems.

--------------------------------------------------------\ feature64BitWindows
   /                          /            /             \
  /                          /            /               \
 /                          /            /                 \
------------------------------------------------------------------------------------ master branch. development is never stopped

• master: The feature branches are merged into this branch if they are considered complete. So the master branch will contain new features and may have breaking changes. Regression fixes and bug fixes may be merged from version branches into master.

--------------------------------------------------------\ feature64BitWindows
                                                         \
-------------------\  featureUDA                          \
                    \                                      \
------------------------------------------------------------------------------------ master branch. development is never stopped
                                     /
                                    /
------------------------------------------------------------------------------------ 2.063 branch
                                /
                             fixEvilRegression

• version branches: Once we determine it's time to do a new release, a version branch is made by branching from the master branch. A version branch will not receive any new features, only regression and important bug fixes. To determine to which version branch regression fixes should go, see #Regression fix. Version branches are used to support the version for an extended period of time to enable updated minor releases which fix regressions.

Suggested local branches

How user repositories are managed is up to the end user, but we propose these branches:

• feature branches: Tracking upstream feature branches. You only need a feature branch if you want to work on that feature. To work on a feature, make changes to the feature branch, push to your repository and file a pull request on github targeting the feature branch

• master: Tracking the upstream master branch. New feature branches should be branched from this branch.

• version branches: Tracking the upstream version branches. If you want to push a regression fix to an older release. (See #Regression fix)

• bugfix/small feature branches: based on master. Used to develop bugfixes and small features (see Bug fix)

Git workflow

Local repository setup

All instructions on this page assume the following setup for the local repository. Experienced git users of course may use different setups.

First, create a fork of the official repository (if not already done) on github. Then follow these instructions:

#We use your own fork repository, not the official one!!!!
git clone git://github.com/yourname/dmd.git
#Now add the official repository as a remote
git remote add upstream git://github.com/D-Programming-Language/dmd.git
git remote update

#sync / create branches on origin
git push origin upstream/master:refs/heads/master

#Repeat this for every release & feature branch you want to use
git push origin upstream/2.061:refs/heads/2.061

#Now checkout / create local branches
#Repeat for every release & feature branch you want to use
git checkout -b 2.061 origin/2.061

The above instructions have to be executed only once. But if a new feature/release branch is available and you want to use it, you have to repeat this part:

git remote update

#Repeat this for every release & feature branch you want to use
git push origin upstream/2.061:refs/heads/2.061

#Repeat for every release & feature branch you want to use
git checkout -b 2.061 origin/2.061

Note: Before creating new branches, or doing similar stuff you should always do this:

git remote update

The instructions on this page always explicitly mention when you have to use this command.

Working on code updates

What type of update?

To determine to which branch an update should be made, we have to consider the type of the change:

• It's a new, big feature: This will need to get a feature branch. See #New feature / Big enhancement
• It's a small enhancement: This will go to master. See #Small enhancement
• It's a bug fix: This will usually go to master. See Bug fix. If the bug fix must be part of a specific release it may also be merged into a version branch.
• It's a regression fix: This will need to go to the oldest supported version branch which is affected by the regression. See #Regression fix

New feature / Big enhancement

We need to create a new branch based on master. As a normal user you can't create feature branches on the official repository though.

Step-By-Step: The one implementing the feature has to do this:

#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base feature on
git pull origin master #Fetch changes from origin
git pull upstream master #Fetch changes from upstream

git branch coolFeature
#Make your changes
git commit -m "Implement cool feature"
#Push to your fork on github
git push origin coolFeature

#Now request in the digitalmars.D newsgroup that someone creates a coolFeature feature branch on the official repository.

#Now go to github and file a pull request for coolFeature. The pull request must target the new official feature branch (here also called coolFeature)!

If someone requests a new feature branch the dmd staff has to do this:

#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base features on
git pull upstream master #Fetch changes from upstream

git branch coolFeature #Use a useful name
git push upstream coolFeature

Small enhancement

We merge directly into master.

Step-By-Step: The one implementing the enhancement has to do this:

#We use your own fork repository, not the official one!!!!
#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base feature on
git branch coolEnhancement
#Make your changes
git commit -m "Implement cool enhancement"
#Push to your fork on github
git push origin coolEnhancement

#Now go to github and file a pull request for coolEnhancement. The pull request must target the master branch!

Bug fix (not a regression!)

If this bug fix should be part of a specific release, follow the instructions for regression fixes: #Regression fix

We need to create a local branch based on master. Then file a pull request on github targeting master.

Step-By-Step:

#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base bugfix on
git pull origin master
git pull upstream master

git branch bugFixNNN
#Make your changes
git commit -m "Implement bugfix #NNN"
#Push to your fork on github
git push origin bugFixNNN

#Now go to github and file a pull request for bugFixNNN. The pull request must target the master branch!

Regression fix

The regression fix must be pushed to the oldest, still supported version branch affected by the regression. To determine if a version is still supported, see #Support times. For now only the last release is supported.

Note: Regression fixes are always based on an version branch, never on master!

Step-By-Step: The one fixing the regression has to do this:

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.062 #This is the branch we base our fix on
git pull origin 2.062
git pull upstream 2.062

git branch fixRegressionNNN
#Edit and fix the regression
git commit -m "Fix regression #nnn"
#Push to your fork on github
git push origin fixRegressionNNN
#Now go to github and file a pull request for fixRegressionNNN. The pull request must target our base branch, e.g. 2.062!

The dmd staff now has to merge that pull request. The regression fix will be applied to the 2.062 branch. The dmd staff now also has to make sure this fix is also applied to all newer release branches (as we only support one release this is never the case right now) and to master.

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.062 #This is the branch the fix was based on
git pull upstream 2.062 #Fetch changes from upstream

#Repeat for all newer release branches (not necessary as long as only latest release is supported)
#git checkout 2.063
#git pull upstream 2.063 #Fetch remote changes from upstream first
#git merge 2.062 #Now merge 2.062 branch into newer branch
#git push upstream 2.063 #Now publish those changes

#We have to repeat that for master
git checkout master
git pull upstream master #Fetch remote changes from upstream first
git merge 2.062 #Now merge 2.062 branch into master branch
git push upstream master #Now publish those changes

Preparing a new major release

As described in #Release Schedule every 2 months a new major release will be made based on a version branch.

To start the release process (This is always done after a major release has been made) create a new version branch for the next release:

#Make sure you've read "Local repository setup"
git remote update

git checkout master
git pull upstream master

git branch 2.063
git push upstream 2.063

Once a beta release is made, the dmd staff adds a tag on the version branch!

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.063
git pull upstream 2.063

git tag v2.063-b1 #b1=>first beta
git push upstream v2.063-b1

For the final release, create a new "v2.063" tag just as for betas.

Preparing a new minor release

As described in #Release Schedule every 2 weeks a new minor release will be prepared for all supported releases.

To put further regression fixes into this release version follow the descriptions in #Regression fix

Once the release is ready, the dmd staff adds a tag!

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.063 #checkout the base branch where a new release is made
git pull upstream 2.063

git tag v2.063.1 #.1=>first minor update
git push upstream v2.063.1

Merging a feature branch

If a feature's implementation is sufficiently complete and has been tested well enough, the feature branch can be merged into master.

DMD staff:

#Make sure you've read "Local repository setup"
git remote update

git checkout featureXXX #checkout the feature branch
git pull upstream featureXXX

git checkout master
git pull upstream master
git merge featureXXX #if merge fails, do not fix it yourself. contact the maintainer of the feature branch 
git push upstream master

Short Summary

Contributors

• Regression fixes go into version branches
• Bug fixes usually go into the master branch. If the bugfix should be included in a specific release, it's pushed to a version branch.
• Big new features are developed in feature branches and merged into master
• Small enhancements can go directly into master

DMD staff

• Merge regression fixes into version branch asap.
• Merge regressions fixes from version to master branch immediately.
• Merge bug fixes and small features into master regularly.
• Merge feature branches into master on demand.

Rationale

Staging

This is the common branch model used by many projects. While an additional staging branch could be useful in some corner cases it would complicate the process.

Regression vs bug fix

we need to differentiate between bug fixes that do not introduce new bugs vs bug fixes that may introduce new bugs *or* break existing code for the current release version. We also *have* to prevent new features and structural changes (destabilizers) from leaking into a stable release.

http://forum.dlang.org/post/dticrdtkkqdatdnhtzrc@forum.dlang.org

• The regression/bug differentiation is therefore a very strict model. It's needed for dmd as every bug fix has a probability of breaking user code and therefore we should keep the changes as few as possible. In dmd we also have the issue that we have long-standing bugs: Bugs that were not introduced, but are differences between dmd and the specification. We don't want to get any of these fixed in minor releases. For druntime and phobos there are few such bugs: Every bug has been introduced in some version. The risk of a bug fix causing regressions also seems to be lower for phobos and druntime, so we could allow bug fixes in release branches for phobos/druntime after the process has been well established.

Regression fixes in release branches

Now why we fix regressions in the release branches: Let's say we have dmd 2.061, 2.062, 2.062.1. A regression was introduced in 2.062 and my code that compiled fine in 2.061 doesn't work in 2.062 anymore. But then the regression fix is pushed to the 2.062 release branch, a minor release is made and 2.062.1 successfully compiles that code again.

If the regression already occured in 2.061 should it be fixed in 2.062? This is a different question, but that situation can't happen as long as we only support 1 release and only fix newly introduced regressions.

http://forum.dlang.org/post/kc90k0$2jue$1@digitalmars.com

Stable

• http://forum.dlang.org/post/fbtxyjvpqqrxskptcvgk@forum.dlang.org
• http://forum.dlang.org/post/noycjddhpobnnxkqsytx@forum.dlang.org

Simplification

Simplified Release Process Proposal

Release version numbering

• When we start shipping shared libraries of druntime and phobos we might have to adjust our versioning scheme as those dictate special rules. (Think of ABI compatibility between minor releases, max 3 numbers for a version, etc)

rebasing

I guess we should have a hard stance on not rebasing any of the official branches. (this especially includes feature branches where this might seem to be useful, but probably is still a bad idea)