Difference between revisions of "Development and Release Process"

From D Wiki
Jump to: navigation, search
(Kill staging. Please see related post in the newsgroup)
Line 51: Line 51:
 
* do not ''rebase'' official branches
 
* do not ''rebase'' official branches
 
* do not rewrite git history on official branches
 
* do not rewrite git history on official branches
* do not merge <span style="color:#458B00">master</span> into a <span style="color:#CD2626">version</span> branch
+
* do not merge <span style="color:#458B00">master</span> into a <span style="color:#CD2626">version</span> branch (The only time when commits go from <span style="color:#458B00">master</span> to a <span style="color:#CD2626">version</span> branch is when the <span style="color:#CD2626">version</span> branch is being created)
* do not merge <span style="color:#00008B">staging</span> into a <span style="color:#CD2626">version</span> branch (The only time when commits go from <span style="color:#00008B">staging</span> to a <span style="color:#CD2626">version</span> branch is when the <span style="color:#CD2626">version</span> branch is being created)
+
* do not merge <span style="color:#EEAD0E">feature</span> branches into <span style="color:#CD2626">version</span> branches. <span style="color:#EEAD0E">Feature</span> branches are always merged into <span style="color:#458B00">master</span>.
* do not merge <span style="color:#EEAD0E">feature</span> branches into <span style="color:#00008B">staging</span> or <span style="color:#CD2626">version</span> branch. <span style="color:#EEAD0E">Feature</span> branches are always merged into <span style="color:#458B00">master</span>.
 
* do not merge <span style="color:#00008B">staging</span> into a <span style="color:#EEAD0E">feature</span> branch. Only merge <span style="color:#00008B">staging</span> into <span style="color:#458B00">master</span>, the <span style="color:#458B00">master</span> into the <span style="color:#EEAD0E">feature</span> branch
 
  
 
==Release Schedule==
 
==Release Schedule==
Line 61: Line 59:
 
vxxx = Tag for version xxx (Marks the release)
 
vxxx = Tag for version xxx (Marks the release)
 
rest = branches
 
rest = branches
\ = merge (merges from version to staging/master not shown)
+
\ = merge
 
------------------------------------------------------------------------------------ master branch. development is never stopped
 
------------------------------------------------------------------------------------ master branch. development is never stopped
                            \                   <8>                \
+
            \                                             \        
------------------------------------------------------------------------------------ staging branch
+
            \                                            ------------------------- 2.063 release branch
            |               |                             |       |
+
               \                                            |       | <2>  |
        v2.062.0-b1        |                        2.063.0-b1    |
+
              \                                    v2.063.0-b1    |       |
                            |                                      |
+
                \                                                  |       |
                            |                   <8>                ---------------- 2.063 release branch
+
                \                                              v2.063.0  v2.063.1
                            |                                      |  <2>  |
+
                  \         
                            |                                  v2.063.0  v2.063.1
+
                  ----------------------------------------------------------------- 2.062 release branch
                            |
+
                  |        |  <2>    |    <2>  |
                            -------------------------------------------------------- 2.062 release branch
+
          v2.062.0-b1  v2.062.0 v2.062.1  v2.062.2
                            |  <2>    |    <2>  |
 
                        v2.062.0 v2.062.1  v2.062.2
 
 
</pre>
 
</pre>
  
A new release is made approximately every 2 months. Releases are stabilized in the <span style="color:#00008B">staging</span> branch and beta releases are tagged as part of <span style="color:#00008B">staging</span>. When a release is ready a new <span style="color:#CD2626">version</span> branch is created from <span style="color:#00008B">staging</span>. The first commit of this <span style="color:#CD2626">version</span> branch is tagged as the release (v2.0xx). Immediately after creating the <span style="color:#CD2626">version</span> branch <span style="color:#458B00">master</span> is merged into <span style="color:#00008B">staging</span>. Note: <span style="color:#458B00">master</span> is only ever merged into <span style="color:#00008B">staging</span> immediately after a major release. Then the <span style="color:#00008B">staging</span> branch is stabilizing again until the next release is made.
+
A new release is made approximately every 2 months. Releases are stabilized in the <span style="color:#CD2626">version</span> branch and beta releases are tagged as part of <span style="color:#CD2626">version</span>.
 
   
 
   
Once the initial release of the <span style="color:#CD2626">version</span> branch is done, only regression fixes will be pushed to the <span style="color:#CD2626">version</span> branch, then a regression fix update is published every 2 weeks from that <span style="color:#CD2626">version</span> branch as long as it's supported (see [[#Preparing a new minor release]]).  
+
Once the initial major release on the <span style="color:#CD2626">version</span> branch is done, only regression fixes will be pushed to the <span style="color:#CD2626">version</span> branch, then a minor release is published every 2 weeks from that <span style="color:#CD2626">version</span> branch as long as it's supported (see [[#Preparing a new minor release]]).  
  
  
Line 99: Line 95:
 
The following branches will be available on the official (upstream) [https://github.com/D-Programming-Language/dmd dmd], [https://github.com/D-Programming-Language/druntime druntime] and [https://github.com/D-Programming-Language/phobos phobos] repositories:
 
The following branches will be available on the official (upstream) [https://github.com/D-Programming-Language/dmd dmd], [https://github.com/D-Programming-Language/druntime druntime] and [https://github.com/D-Programming-Language/phobos phobos] repositories:
  
* '''<span style="color:#EEAD0E">feature branches</span>''': New features, in general, are '''not''' developed directly on the <span style="color:#458B00">master</span> branch, but in a dedicated <span style="color:#EEAD0E">feature</span> branch. An example for such a branch would be an ''UDA'' branch for user defined attributes. Contributors can collaborate on the feature by opening pull request targeting that <span style="color:#EEAD0E">feature</span> branch. When the feature's implementation is more-or-less complete, the <span style="color:#EEAD0E">feature</span> branch will be merged into the <span style="color:#458B00">master</span> branch. The <span style="color:#458B00">master</span>  branch is also sporadically merged into the '''<span style="color:#EEAD0E">feature</span>''' branches to get regression fixes, bug fixes and to make sure the '''<span style="color:#EEAD0E">feature</span>''' branch can be merged into <span style="color:#458B00">master</span> later on without problems.
+
* '''<span style="color:#EEAD0E">feature branches</span>''': New features, in general, are '''not''' developed directly on the <span style="color:#458B00">master</span> branch, but in a dedicated <span style="color:#EEAD0E">feature</span> branch. An example for such a branch would be an ''UDA'' branch for user defined attributes. Contributors can collaborate on the feature by opening pull request targeting that <span style="color:#EEAD0E">feature</span> branch. When the feature's implementation is more-or-less complete, the <span style="color:#EEAD0E">feature</span> branch will be merged into the <span style="color:#458B00">master</span> branch. The <span style="color:#458B00">master</span>  branch is also sporadically merged into the '''<span style="color:#EEAD0E">feature</span>''' branches to get regression fixes and bug fixes and to make sure the '''<span style="color:#EEAD0E">feature</span>''' branch can be merged into <span style="color:#458B00">master</span> later on without problems.
 
<pre>
 
<pre>
  
Line 107: Line 103:
 
  /                          /            /                \
 
  /                          /            /                \
 
------------------------------------------------------------------------------------ master branch. development is never stopped
 
------------------------------------------------------------------------------------ master branch. development is never stopped
                                      /
 
                                    /
 
------------------------------------------------------------------------------------ staging branch
 
                                /
 
                            fix123
 
 
</pre>
 
</pre>
  
* '''<span style="color:#458B00">master</span>''': The <span style="color:#EEAD0E">feature</span> branches are merged into this branch if they are considered complete. So the <span style="color:#458B00">master</span>  branch will contain new features and may have breaking changes. Regression fixes  and bug fixes will be merged from <span style="color:#00008B">staging</span> into <span style="color:#458B00">master</span>.
+
* '''<span style="color:#458B00">master</span>''': The <span style="color:#EEAD0E">feature</span> branches are merged into this branch if they are considered complete. So the <span style="color:#458B00">master</span>  branch will contain new features and may have breaking changes. Regression fixes  and bug fixes may be merged from <span style="color:#CD2626">version</span> branches into <span style="color:#458B00">master</span>.
 
<pre>
 
<pre>
  
Line 122: Line 113:
 
                     \                                      \
 
                     \                                      \
 
------------------------------------------------------------------------------------ master branch. development is never stopped
 
------------------------------------------------------------------------------------ master branch. development is never stopped
                            \        /         <8>                \
+
                                    /
                              \      /                               \
+
                                    /
------------------------------------------------------------------------------------ staging branch
+
------------------------------------------------------------------------------------ 2.063 branch
                                /
 
                            fix123
 
</pre>
 
 
 
* '''<span style="color:#00008B">staging</span>''': Bug fixes will be pushed directly into <span style="color:#00008B">staging</span> if they are not regression fixes. To determine where a regression fix should go, see [[#Regression fix]]. <span style="color:#00008B">staging</span> usually only gets bugfix updates, but after every major release, new features are merged from master. Regression fixes will be merged from version branches into <span style="color:#00008B">staging</span>.
 
<pre>
 
------------------------------------------------------------------------------------ master branch. development is never stopped
 
                            \                  <8>                \
 
------------------------------------------------------------------------------------ staging branch
 
  /        /                      /    /          /
 
fix123  fix234                fix987  /        fix999
 
                                      /
 
                              ------------------------------------------------------ 2.0.62 version branch
 
 
                                 /
 
                                 /
                             regression1
+
                             fixEvilRegression
 
</pre>
 
</pre>
  
* '''<span style="color:#CD2626">version branches</span>''': Once we determine it's time to do a new release, a <span style="color:#CD2626">version</span>  branch is made by branching from the <span style="color:#00008B">staging</span>  branch. A version branch will not receive any new features, only regression fixes. To determine to which <span style="color:#CD2626">version</span>  branch regression fixes should go, see [[#Regression fix]]. <span style="color:#CD2626">Version</span>  branches are used to support the version for an extended period of time to enable updated minor releases which fix regressions.
+
* '''<span style="color:#CD2626">version branches</span>''': Once we determine it's time to do a new release, a <span style="color:#CD2626">version</span>  branch is made by branching from the <span style="color:#458B00">master</span>  branch. A version branch will not receive any new features, only regression and important bug fixes. To determine to which <span style="color:#CD2626">version</span>  branch regression fixes should go, see [[#Regression fix]]. <span style="color:#CD2626">Version</span>  branches are used to support the version for an extended period of time to enable updated minor releases which fix regressions.
  
 
===Suggested local branches===
 
===Suggested local branches===
How user repositories are managed is up to the end user, but we propose this branches:
+
How user repositories are managed is up to the end user, but we propose these branches:
 
* '''<span style="color:#EEAD0E">feature branches</span>''': Tracking upstream <span style="color:#EEAD0E">feature</span> branches. You only need a <span style="color:#EEAD0E">feature</span> branch if you want to work on that feature. To work on a feature, make changes to the <span style="color:#EEAD0E">feature</span> branch, push to your repository and file a pull request on github '''targeting the <span style="color:#EEAD0E">feature</span> branch'''
 
* '''<span style="color:#EEAD0E">feature branches</span>''': Tracking upstream <span style="color:#EEAD0E">feature</span> branches. You only need a <span style="color:#EEAD0E">feature</span> branch if you want to work on that feature. To work on a feature, make changes to the <span style="color:#EEAD0E">feature</span> branch, push to your repository and file a pull request on github '''targeting the <span style="color:#EEAD0E">feature</span> branch'''
  
Line 152: Line 130:
 
* '''<span style="color:#CD2626">version branches</span>''': Tracking the upstream <span style="color:#CD2626">version</span>  branches. If you want to push a regression  fix to an older release. (See [[#Regression fix]])
 
* '''<span style="color:#CD2626">version branches</span>''': Tracking the upstream <span style="color:#CD2626">version</span>  branches. If you want to push a regression  fix to an older release. (See [[#Regression fix]])
  
* '''bugfix branches''': based on <span style="color:#00008B">staging</span>. Used to develop bugfixes (see [[#Bug fix (not a regression!)|Bug fix]])
+
* '''bugfix/small feature branches''': based on <span style="color:#458B00">master</span>. Used to develop bugfixes and small features (see [[#Bug fix (not a regression!)|Bug fix]])
  
 
==Git workflow==
 
==Git workflow==
Line 170: Line 148:
 
#sync / create branches on origin
 
#sync / create branches on origin
 
git push origin upstream/master:refs/heads/master
 
git push origin upstream/master:refs/heads/master
git push origin upstream/staging:refs/heads/staging
 
  
 
#Repeat this for every release & feature branch you want to use
 
#Repeat this for every release & feature branch you want to use
Line 176: Line 153:
  
 
#Now checkout / create local branches
 
#Now checkout / create local branches
git checkout -b staging origin/staging
 
 
 
#Repeat for every release & feature branch you want to use
 
#Repeat for every release & feature branch you want to use
 
git checkout -b 2.061 origin/2.061
 
git checkout -b 2.061 origin/2.061
Line 194: Line 169:
  
 
Note:
 
Note:
Before creating new branches, etc you should always do this:
+
Before creating new branches, or doing similar stuff you should always do this:
 
<source lang="bash">
 
<source lang="bash">
 
git remote update
 
git remote update
Line 207: Line 182:
 
* It's a new, big feature: This will need to get a <span style="color:#EEAD0E">feature</span> branch. See [[#New feature / Big enhancement]]
 
* It's a new, big feature: This will need to get a <span style="color:#EEAD0E">feature</span> branch. See [[#New feature / Big enhancement]]
 
* It's a small enhancement: This will go to <span style="color:#458B00">master</span>. See [[#Small enhancement]]
 
* It's a small enhancement: This will go to <span style="color:#458B00">master</span>. See [[#Small enhancement]]
* It's a bug fix: This will go straight to <span style="color:#00008B">staging</span>. See [[#Bug fix (not a regression!) | Bug fix]]
+
* It's a bug fix: This will usually go to <span style="color:#458B00">master</span>. See [[#Bug fix (not a regression!) | Bug fix]]. If the bug fix must be part of a specific release it may also be merged into a <span style="color:#CD2626">version</span> branch.
 
* It's a regression fix: This will need to go to the oldest '''supported''' <span style="color:#CD2626">version</span> branch which is affected by the regression. See [[#Regression fix]]
 
* It's a regression fix: This will need to go to the oldest '''supported''' <span style="color:#CD2626">version</span> branch which is affected by the regression. See [[#Regression fix]]
  
Line 267: Line 242:
  
 
====Bug fix (not a regression!)====
 
====Bug fix (not a regression!)====
We need to create a local branch based on <span style="color:#00008B">staging</span>. Then file a pull request on github targeting <span style="color:#00008B">staging</span>.
+
If this bug fix should be part of a specific release, follow the instructions for regression fixes: [[#Regression fix]]
 +
 
 +
We need to create a local branch based on <span style="color:#458B00">master</span>. Then file a pull request on github targeting <span style="color:#458B00">master</span>.
  
 
Step-By-Step:
 
Step-By-Step:
Line 274: Line 251:
 
git remote update
 
git remote update
  
git checkout staging #This is the branch we base bugfix on
+
git checkout master #This is the branch we base bugfix on
git pull origin staging
+
git pull origin master
git pull upstream staging
+
git pull upstream master
  
 
git branch bugFixNNN
 
git branch bugFixNNN
Line 284: Line 261:
 
git push origin bugFixNNN
 
git push origin bugFixNNN
  
#Now go to github and file a pull request for bugFixNNN. The pull request must target the staging branch!
+
#Now go to github and file a pull request for bugFixNNN. The pull request must target the master branch!
</source>
 
 
 
To make sure that bug fixes are also available in the master development version and in the feature test branches, the dmd staff should merge <span style="color:#00008B">staging</span> into <span style="color:#458B00">master</span>. This is not urgent and it's not necessary to do this after every single bug fix, it can be done sporadically.
 
 
 
DMD staff:
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout staging
 
git pull upstream staging
 
 
 
git checkout master
 
git pull upstream master
 
 
 
git merge staging
 
 
 
git push upstream master
 
 
</source>
 
</source>
  
 
====Regression fix====
 
====Regression fix====
The regression fix must be pushed to the oldest, still supported <span style="color:#CD2626">version</span> branch affected by the regression. To determine if a <span style="color:#CD2626">version</span> is still supported, see [[#Support times]].
+
The regression fix must be pushed to the oldest, still supported <span style="color:#CD2626">version</span> branch affected by the regression. To determine if a <span style="color:#CD2626">version</span> is still supported, see [[#Support times]]. For now only the last release is supported.
 
 
As an example (this doesn't match with [[#Support times]] as we need a more complicated setup to describe the concept):
 
<span style="color:#CD2626">2.063</span> is the most recent release and supported.
 
<span style="color:#CD2626">2.062</span> is the previous release and supported.
 
<span style="color:#CD2626">2.061</span> is no longer supported.
 
An regression is found that affects 2.061, 2.062, 2.063, <span style="color:#00008B">staging</span> and <span style="color:#458B00">master</span>.
 
 
 
2.061 is no longer supported, so the oldest supported and affected release is 2.062. Therefore the regression fix has to be made on the <span style="color:#CD2626">2.062</span> release branch.
 
  
'''Note''': Regression fixes are always based on an <span style="color:#CD2626">version</span> branch, never on <span style="color:#00008B">staging</span> / <span style="color:#458B00">master</span>!
+
'''Note''': Regression fixes are always based on an <span style="color:#CD2626">version</span> branch, never on <span style="color:#458B00">master</span>!
  
 
Step-By-Step:
 
Step-By-Step:
Line 336: Line 287:
 
</source>
 
</source>
  
The dmd staff now has to merge that pull request. The regression fix will be applied to the <span style="color:#CD2626">2.062</span> branch. The dmd staff now also has to make sure this fix is also applied to all newer release branches (<span style="color:#CD2626">2.063</span> in our example) and to <span style="color:#00008B">staging</span> and <span style="color:#458B00">master</span>.
+
The dmd staff now has to merge that pull request. The regression fix will be applied to the <span style="color:#CD2626">2.062</span> branch. The dmd staff now also has to make sure this fix is also applied to all newer release branches (as we only support one release this is never the case right now) and to <span style="color:#458B00">master</span>.
  
 
<source lang="bash">
 
<source lang="bash">
Line 345: Line 296:
 
git pull upstream 2.062 #Fetch changes from upstream
 
git pull upstream 2.062 #Fetch changes from upstream
  
#Repeat for all newer release branches
+
#Repeat for all newer release branches (not necessary as long as only latest release is supported)
git checkout 2.063
+
#git checkout 2.063
git pull upstream 2.063 #Fetch remote changes from upstream first
+
#git pull upstream 2.063 #Fetch remote changes from upstream first
git merge 2.062 #Now merge 2.062 branch into newer branch
+
#git merge 2.062 #Now merge 2.062 branch into newer branch
git push upstream 2.063 #Now publish those changes
+
#git push upstream 2.063 #Now publish those changes
 
 
#We have to repeat that for staging
 
git checkout staging
 
git pull upstream staging #Fetch remote changes from upstream first
 
git merge 2.062 #Now merge 2.062 branch into staging branch
 
git push upstream staging #Now publish those changes
 
  
 
#We have to repeat that for master
 
#We have to repeat that for master
 
git checkout master
 
git checkout master
 
git pull upstream master #Fetch remote changes from upstream first
 
git pull upstream master #Fetch remote changes from upstream first
git merge staging #Now merge staging branch into master branch
+
git merge 2.062 #Now merge 2.062 branch into master branch
 
git push upstream master #Now publish those changes
 
git push upstream master #Now publish those changes
 
</source>
 
</source>
  
 
===Preparing a new major release===
 
===Preparing a new major release===
As described in [[#Release Schedule]] every 2 months a new major <span style="color:#CD2626">release</span> will be made based on <span style="color:#00008B">staging</span>.
+
As described in [[#Release Schedule]] every 2 months a new major <span style="color:#CD2626">release</span> will be made based on a <span style="color:#CD2626">version</span> branch.
  
To start the release process (This is always done after a major release has been made) merge <span style="color:#458B00">master</span> into <span style="color:#00008B">staging</span>:
+
To start the release process (This is always done after a major release has been made) create a new <span style="color:#CD2626">version</span> branch for the next release:
 
<source lang="bash">
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
#Make sure you've read "Local repository setup"
Line 375: Line 320:
 
git pull upstream master
 
git pull upstream master
  
git checkout staging
+
git branch 2.063
git pull upstream staging
+
git push upstream 2.063
 
 
git merge master
 
git push upstream staging
 
 
</source>
 
</source>
  
Once a beta release is made, the dmd staff adds a tag on <span style="color:#00008B">staging</span>!
+
Once a beta release is made, the dmd staff adds a tag on the <span style="color:#CD2626">version</span> branch!
 
<source lang="bash">
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
#Make sure you've read "Local repository setup"
 
git remote update
 
git remote update
  
git checkout staging
+
git checkout 2.063
git pull upstream staging
+
git pull upstream 2.063
  
 
git tag v2.063-b1 #b1=>first beta
 
git tag v2.063-b1 #b1=>first beta
Line 394: Line 336:
 
</source>
 
</source>
  
For the final release, a new version branch is created:
+
For the final release, create a new "v2.063" tag just as for betas.
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout staging
 
git pull upstream staging
 
 
 
git branch 2.063 #replace 2.063 with the new release number
 
git push upstream 2.063
 
 
 
#And make a tag
 
git tag v2.063
 
git push upstream v2.063
 
</source>
 
  
 
===Preparing a new minor release===
 
===Preparing a new minor release===
Line 415: Line 343:
 
To put further regression fixes into this release <span style="color:#CD2626">version</span> follow the descriptions in [[#Regression fix]]
 
To put further regression fixes into this release <span style="color:#CD2626">version</span> follow the descriptions in [[#Regression fix]]
  
Once the release is made, the dmd staff adds a tag!
+
Once the release is ready, the dmd staff adds a tag!
 
<source lang="bash">
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
#Make sure you've read "Local repository setup"
Line 428: Line 356:
  
 
===Merging a feature branch===
 
===Merging a feature branch===
If a features implementation is ''sufficiently'' complete and has been tested ''well enough'', the <span style="color:#EEAD0E">feature</span> branch can be merged into <span style="color:#458B00">master</span>.
+
If a feature's implementation is ''sufficiently'' complete and has been tested ''well enough'', the <span style="color:#EEAD0E">feature</span> branch can be merged into <span style="color:#458B00">master</span>.
  
 
DMD staff:
 
DMD staff:
Line 447: Line 375:
 
===Contributors===
 
===Contributors===
 
* Regression fixes go into <span style="color:#CD2626">version</span> branches
 
* Regression fixes go into <span style="color:#CD2626">version</span> branches
* Bug fixes go into the <span style="color:#00008B">staging</span> branch
+
* Bug fixes usually go into the <span style="color:#458B00">master</span> branch. If the bugfix should be included in a specific release, it's pushed to a <span style="color:#CD2626">version</span> branch.
 
* Big new features are developed in <span style="color:#EEAD0E">feature</span> branches and merged into <span style="color:#458B00">master</span>
 
* Big new features are developed in <span style="color:#EEAD0E">feature</span> branches and merged into <span style="color:#458B00">master</span>
 
* Small enhancements can go directly into <span style="color:#458B00">master</span>
 
* Small enhancements can go directly into <span style="color:#458B00">master</span>
Line 453: Line 381:
 
===DMD staff===
 
===DMD staff===
 
* Merge regression fixes into <span style="color:#CD2626">version</span> branch asap.
 
* Merge regression fixes into <span style="color:#CD2626">version</span> branch asap.
* Merge regressions fixes from <span style="color:#CD2626">version</span> to other <span style="color:#CD2626">version</span> and to <span style="color:#00008B">staging</span> branch immediately.
+
* Merge regressions fixes from <span style="color:#CD2626">version</span> to <span style="color:#458B00">master</span> branch immediately.
* Merge <span style="color:#00008B">staging</span> branch into <span style="color:#458B00">master</span> regularly
+
* Merge bug fixes and small features into <span style="color:#458B00">master</span> regularly.
* Merge bug fixes into <span style="color:#00008B">staging</span> regularly.
 
 
* Merge <span style="color:#EEAD0E">feature</span> branches into <span style="color:#458B00">master</span> on demand.
 
* Merge <span style="color:#EEAD0E">feature</span> branches into <span style="color:#458B00">master</span> on demand.
* Merge <span style="color:#458B00">master</span> into <span style="color:#00008B">staging</span> '''after''' every release
 
  
 
==Rationale==
 
==Rationale==
 
=== <span style="color:#00008B">Staging</span> ===
 
=== <span style="color:#00008B">Staging</span> ===
This setup has the advantage that <span style="color:#00008B">staging</span> stabilizes over a long time. <span style="color:#00008B">staging</span> get's a feature updates only after every major release is made. After that <span style="color:#00008B">staging</span> only recieves bug fixes. It also has the advance that bug fixes also have a clear target branch: With <span style="color:#00008B">staging</span> we can always target <span style="color:#00008B">staging</span>. Without <span style="color:#00008B">staging</span> we'd have to target the version branch of the currently stabilizing <span style="color:#CD2626">version</span> branch. But that branch changes over time, so we'd have to update github pull requests, etc.
+
This is the common branch model used by many projects. While an additional staging branch could be useful in some corner cases it would complicate the process.
  
 
=== Regression vs bug fix===
 
=== Regression vs bug fix===
Line 488: Line 414:
 
as we only support 1 release and only fix newly introduced regressions.
 
as we only support 1 release and only fix newly introduced regressions.
 
</blockquote>
 
</blockquote>
http://forum.dlang.org/thread/kc2l4o$244s$1@digitalmars.com?page=7#post-kc90k0:242jue:241:40digitalmars.com
+
http://forum.dlang.org/post/kc90k0$2jue$1@digitalmars.com
  
 
=== Stable ===
 
=== Stable ===
Line 495: Line 421:
  
 
==Possible optimization==
 
==Possible optimization==
===Time spans===
 
* If it turns out that <span style="color:#00008B">staging</span> does not get reasonably stable to create the release immediately from <span style="color:#00008B">staging</span> we could introduce a time span (e.g 3 weeks) between branching the new <span style="color:#CD2626">version</span> branch and releasing the version / setting the tag. In this timespan only regression fixes would be merged into the <span style="color:#CD2626">version</span> branch. After that time span the release will be tagged on the <span style="color:#CD2626">version</span> branch
 
** additional betas can be released from the version branch during that time span
 
<pre>
 
<x> = time in weeks
 
vxxx = Tag for version xxx (Marks the release)
 
rest = branches
 
\ = merge (merges from version to staging/master not shown)
 
---------------------------------------------------------------------------------------------- master branch. development is never stopped
 
                            \                  <8>                \
 
---------------------------------------------------------------------------------------------- staging branch
 
            |              |                              |        |
 
      v2.062.0-b1          |                        v2.063.0-b1  |
 
                            |                                      |    <3>      .
 
                            |                    <8>                -------------------------- 2.063 release branch
 
                            |                                          |        |  <2>  |
 
                            |                                          rc1    v2.063.0  v2.063.1
 
                            |    <3>      .
 
                            ------------------------------------------------------------------ 2.062 release branch
 
                                |        |  <2>    |    <2>  |
 
                          v2.062.0-rc1 v2.062.0 v2.062.1  v2.062.2
 
</pre>
 
  
 
===Release version numbering===
 
===Release version numbering===

Revision as of 18:45, 25 February 2013

If you're an experienced git user, skip to the summary.

Definitions

Version numbers

We follow the traditional dmd versioning: 2.061. Tags have a v prepended, so the tag for version 2.061.1 is v2.061.1. Versions where the third part is bigger than zero are called Minor Releases. Versions where the third part is equal zero are Major Releases. Minor Releases will only contain regression fixes, Major Releases will contain bug fixes and new features.

The third digit can be omitted for Major Releases: 2.061 is the same as 2.061.0.

Meta: The dmd contributors should choose one style for all offical statements: Tags, packages, Notes on the download page. Either always write out the .0 part or always omit it.

Regression fix vs. Bug fix

A regression fix for release X is a fix for a bug which was newly introduced in release X. We distinguish between regression fixes and bug fixes to avoid introducing more regressions in minor releases. A regression fix especially should not make code invalid which was valid before the regression occurred. As an example: If we have releases 2.061 and 2.061.1 then 2.061.1 should always be "better" than 2.061. It should be tried heavily not to introduce new regressions in minor releases. Code which compiled with 2.061 should still compile in 2.061.1 (there's one rare exception: If a regression allowed code to work in 2.061 which did not work in 2.060 and which is not supposed to work, it can be disabled again in 2.061.1).

So a regression fix is a bug fix which should not have side-effects, doesn't introduce new features and fixes a bug which only recently appeared.

Long standing bugs are especially not regressions! (Or technically probably a regression in a very old release, but as we support only 1 major release we can treat those like bugs. We only care about new regressions)

Small enhancement vs. Big feature

We later suggest different workflows for those two enhancements. Those are easily distinguishable: If you implement the feature by yourself and don't need help / someone to participate / extended review and the code changes are reasonably small, it's a small enhancement. Otherwise it's a big feature.

The workflow for big features allows other contributors to participate in that feature, make special test releases for that feature, etc. If you want / need those, use the Big Feature workflow.

Stable

When is a release stable and therefore ready to be released?

The workflow already guarantees that when a release can be made there has been a feature freeze for a long time. So only bug fixes and regression fixes where allowed for some time which should already stabilize the release. But we should add some more criteria:

  • All bugs marked as critical in bugzilla have been fixed
    • For practical reasons we allow exceptions from this rule for the first releases after this workflow has been established
  • There are no newly introduced (known) regressions
    • This is a hard rule, there are no exceptions to it. Have a regression? Don't release!
  • Later a yet to be determined percentage of non-critical bug will need to be fixed (or new features introduced)
    • This is to make sure a new release actually provides something new and prevent us from releasing too often. Currently this is not an issue.

Apart from these criteria, the final decision is made by the release manager.

Version / Release branch

This wiki page defines version branches, other projects often call those branches release branches. This wiki page will call them version branches for now, but a release branch and a version branch is exactly the same thing.

Don'ts

  • do not cherry pick from one official branch into another
  • do not rebase official branches
  • do not rewrite git history on official branches
  • do not merge master into a version branch (The only time when commits go from master to a version branch is when the version branch is being created)
  • do not merge feature branches into version branches. Feature branches are always merged into master.

Release Schedule

<x> = time in weeks
vxxx = Tag for version xxx (Marks the release)
rest = branches
\ = merge
------------------------------------------------------------------------------------ master branch. development is never stopped
            \                                             \          
             \                                             ------------------------- 2.063 release branch
              \                                            |        |  <2>   |
               \                                     v2.063.0-b1    |        |
                \                                                   |        |
                 \                                              v2.063.0   v2.063.1
                  \          
                   ----------------------------------------------------------------- 2.062 release branch
                   |         |  <2>    |    <2>   |
           v2.062.0-b1   v2.062.0 v2.062.1   v2.062.2

A new release is made approximately every 2 months. Releases are stabilized in the version branch and beta releases are tagged as part of version.

Once the initial major release on the version branch is done, only regression fixes will be pushed to the version branch, then a minor release is published every 2 weeks from that version branch as long as it's supported (see #Preparing a new minor release).


Feature Previews: A feature branch may need it's own beta release before being merged into master. These are unplanned, announced on the NG, and highly volatile.

As an example we would see these releases for things like 64bit support and User Defined Attributes.

Support times

Currently only the most recent major release is supported. (This might change in the future)

As an example: version 2.061 is released. 2.061 will get updates until 2.062 is released. Then 2.062 will get updates until 2.063 is released etc.

Only supported releases receive minor update releases.

Branching model

Official branches

The following branches will be available on the official (upstream) dmd, druntime and phobos repositories:

  • feature branches: New features, in general, are not developed directly on the master branch, but in a dedicated feature branch. An example for such a branch would be an UDA branch for user defined attributes. Contributors can collaborate on the feature by opening pull request targeting that feature branch. When the feature's implementation is more-or-less complete, the feature branch will be merged into the master branch. The master branch is also sporadically merged into the feature branches to get regression fixes and bug fixes and to make sure the feature branch can be merged into master later on without problems.

--------------------------------------------------------\ feature64BitWindows
   /                          /            /             \
  /                          /            /               \
 /                          /            /                 \
------------------------------------------------------------------------------------ master branch. development is never stopped
  • master: The feature branches are merged into this branch if they are considered complete. So the master branch will contain new features and may have breaking changes. Regression fixes and bug fixes may be merged from version branches into master.

--------------------------------------------------------\ feature64BitWindows
                                                         \
-------------------\  featureUDA                          \
                    \                                      \
------------------------------------------------------------------------------------ master branch. development is never stopped
                                     /
                                    /
------------------------------------------------------------------------------------ 2.063 branch
                                /
                             fixEvilRegression
  • version branches: Once we determine it's time to do a new release, a version branch is made by branching from the master branch. A version branch will not receive any new features, only regression and important bug fixes. To determine to which version branch regression fixes should go, see #Regression fix. Version branches are used to support the version for an extended period of time to enable updated minor releases which fix regressions.

Suggested local branches

How user repositories are managed is up to the end user, but we propose these branches:

  • feature branches: Tracking upstream feature branches. You only need a feature branch if you want to work on that feature. To work on a feature, make changes to the feature branch, push to your repository and file a pull request on github targeting the feature branch
  • master: Tracking the upstream master branch. New feature branches should be branched from this branch.
  • version branches: Tracking the upstream version branches. If you want to push a regression fix to an older release. (See #Regression fix)
  • bugfix/small feature branches: based on master. Used to develop bugfixes and small features (see Bug fix)

Git workflow

Local repository setup

All instructions on this page assume the following setup for the local repository. Experienced git users of course may use different setups.

First, create a fork of the official repository (if not already done) on github. Then follow these instructions:

#We use your own fork repository, not the official one!!!!
git clone git://github.com/yourname/dmd.git
#Now add the official repository as a remote
git remote add upstream git://github.com/D-Programming-Language/dmd.git
git remote update

#sync / create branches on origin
git push origin upstream/master:refs/heads/master

#Repeat this for every release & feature branch you want to use
git push origin upstream/2.061:refs/heads/2.061

#Now checkout / create local branches
#Repeat for every release & feature branch you want to use
git checkout -b 2.061 origin/2.061

The above instructions have to be executed only once. But if a new feature/release branch is available and you want to use it, you have to repeat this part:

git remote update

#Repeat this for every release & feature branch you want to use
git push origin upstream/2.061:refs/heads/2.061

#Repeat for every release & feature branch you want to use
git checkout -b 2.061 origin/2.061

Note: Before creating new branches, or doing similar stuff you should always do this:

git remote update

The instructions on this page always explicitly mention when you have to use this command.

Working on code updates

What type of update?

To determine to which branch an update should be made, we have to consider the type of the change:

  • It's a new, big feature: This will need to get a feature branch. See #New feature / Big enhancement
  • It's a small enhancement: This will go to master. See #Small enhancement
  • It's a bug fix: This will usually go to master. See Bug fix. If the bug fix must be part of a specific release it may also be merged into a version branch.
  • It's a regression fix: This will need to go to the oldest supported version branch which is affected by the regression. See #Regression fix

New feature / Big enhancement

We need to create a new branch based on master. As a normal user you can't create feature branches on the official repository though.

Step-By-Step: The one implementing the feature has to do this:

#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base feature on
git pull origin master #Fetch changes from origin
git pull upstream master #Fetch changes from upstream

git branch coolFeature
#Make your changes
git commit -m "Implement cool feature"
#Push to your fork on github
git push origin coolFeature

#Now request in the digitalmars.D newsgroup that someone creates a coolFeature feature branch on the official repository.

#Now go to github and file a pull request for coolFeature. The pull request must target the new official feature branch (here also called coolFeature)!

If someone requests a new feature branch the dmd staff has to do this:

#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base features on
git pull upstream master #Fetch changes from upstream

git branch coolFeature #Use a useful name
git push upstream coolFeature

Small enhancement

We merge directly into master.

Step-By-Step: The one implementing the enhancement has to do this:

#We use your own fork repository, not the official one!!!!
#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base feature on
git branch coolEnhancement
#Make your changes
git commit -m "Implement cool enhancement"
#Push to your fork on github
git push origin coolEnhancement

#Now go to github and file a pull request for coolEnhancement. The pull request must target the master branch!

Bug fix (not a regression!)

If this bug fix should be part of a specific release, follow the instructions for regression fixes: #Regression fix

We need to create a local branch based on master. Then file a pull request on github targeting master.

Step-By-Step:

#Make sure you've read "Local repository setup"
git remote update

git checkout master #This is the branch we base bugfix on
git pull origin master
git pull upstream master

git branch bugFixNNN
#Make your changes
git commit -m "Implement bugfix #NNN"
#Push to your fork on github
git push origin bugFixNNN

#Now go to github and file a pull request for bugFixNNN. The pull request must target the master branch!

Regression fix

The regression fix must be pushed to the oldest, still supported version branch affected by the regression. To determine if a version is still supported, see #Support times. For now only the last release is supported.

Note: Regression fixes are always based on an version branch, never on master!

Step-By-Step: The one fixing the regression has to do this:

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.062 #This is the branch we base our fix on
git pull origin 2.062
git pull upstream 2.062

git branch fixRegressionNNN
#Edit and fix the regression
git commit -m "Fix regression #nnn"
#Push to your fork on github
git push origin fixRegressionNNN
#Now go to github and file a pull request for fixRegressionNNN. The pull request must target our base branch, e.g. 2.062!

The dmd staff now has to merge that pull request. The regression fix will be applied to the 2.062 branch. The dmd staff now also has to make sure this fix is also applied to all newer release branches (as we only support one release this is never the case right now) and to master.

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.062 #This is the branch the fix was based on
git pull upstream 2.062 #Fetch changes from upstream

#Repeat for all newer release branches (not necessary as long as only latest release is supported)
#git checkout 2.063
#git pull upstream 2.063 #Fetch remote changes from upstream first
#git merge 2.062 #Now merge 2.062 branch into newer branch
#git push upstream 2.063 #Now publish those changes

#We have to repeat that for master
git checkout master
git pull upstream master #Fetch remote changes from upstream first
git merge 2.062 #Now merge 2.062 branch into master branch
git push upstream master #Now publish those changes

Preparing a new major release

As described in #Release Schedule every 2 months a new major release will be made based on a version branch.

To start the release process (This is always done after a major release has been made) create a new version branch for the next release:

#Make sure you've read "Local repository setup"
git remote update

git checkout master
git pull upstream master

git branch 2.063
git push upstream 2.063

Once a beta release is made, the dmd staff adds a tag on the version branch!

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.063
git pull upstream 2.063

git tag v2.063-b1 #b1=>first beta
git push upstream v2.063-b1

For the final release, create a new "v2.063" tag just as for betas.

Preparing a new minor release

As described in #Release Schedule every 2 weeks a new minor release will be prepared for all supported releases.

To put further regression fixes into this release version follow the descriptions in #Regression fix

Once the release is ready, the dmd staff adds a tag!

#Make sure you've read "Local repository setup"
git remote update

git checkout 2.063 #checkout the base branch where a new release is made
git pull upstream 2.063

git tag v2.063.1 #.1=>first minor update
git push upstream v2.063.1

Merging a feature branch

If a feature's implementation is sufficiently complete and has been tested well enough, the feature branch can be merged into master.

DMD staff:

#Make sure you've read "Local repository setup"
git remote update

git checkout featureXXX #checkout the feature branch
git pull upstream featureXXX

git checkout master
git pull upstream master
git merge featureXXX #if merge fails, do not fix it yourself. contact the maintainer of the feature branch 
git push upstream master

Short Summary

Contributors

  • Regression fixes go into version branches
  • Bug fixes usually go into the master branch. If the bugfix should be included in a specific release, it's pushed to a version branch.
  • Big new features are developed in feature branches and merged into master
  • Small enhancements can go directly into master

DMD staff

  • Merge regression fixes into version branch asap.
  • Merge regressions fixes from version to master branch immediately.
  • Merge bug fixes and small features into master regularly.
  • Merge feature branches into master on demand.

Rationale

Staging

This is the common branch model used by many projects. While an additional staging branch could be useful in some corner cases it would complicate the process.

Regression vs bug fix

we need to differentiate between bug fixes that do not introduce new bugs vs bug fixes that may introduce new bugs *or* break existing code for the current release version. We also *have* to prevent new features and structural changes (destabilizers) from leaking into a stable release.

http://forum.dlang.org/post/dticrdtkkqdatdnhtzrc@forum.dlang.org

  • The regression/bug differentiation is therefore a very strict model. It's needed for dmd as every bug fix has a probability of breaking user code and therefore we should keep the changes as few as possible. In dmd we also have the issue that we have long-standing bugs: Bugs that were not introduced, but are differences between dmd and the specification. We don't want to get any of these fixed in minor releases. For druntime and phobos there are few such bugs: Every bug has been introduced in some version. The risk of a bug fix causing regressions also seems to be lower for phobos and druntime, so we could allow bug fixes in release branches for phobos/druntime after the process has been well established.

Regression fixes in release branches

Now why we fix regressions in the release branches: Let's say we have dmd 2.061, 2.062, 2.062.1. A regression was introduced in 2.062 and my code that compiled fine in 2.061 doesn't work in 2.062 anymore. But then the regression fix is pushed to the 2.062 release branch, a minor release is made and 2.062.1 successfully compiles that code again.

If the regression already occured in 2.061 should it be fixed in 2.062? This is a different question, but that situation can't happen as long as we only support 1 release and only fix newly introduced regressions.

http://forum.dlang.org/post/kc90k0$2jue$1@digitalmars.com

Stable

Possible optimization

Release version numbering

  • When we start shipping shared libraries of druntime and phobos we might have to adjust our versioning scheme as those dictate special rules. (Think of ABI compatibility between minor releases, max 3 numbers for a version, etc)

rebasing

I guess we should have a hard stance on not rebasing any of the official branches. (this especially includes feature branches where this might seem to be useful, but probably is still a bad idea)