Difference between revisions of "Development and Release Process"

From D Wiki
Jump to: navigation, search
m
m (Redirected page to DIP75)
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
<!--
+
#REDIRECT [[DIP75]]
Copy/Paste colored markup for branch names:
 
 
 
<span style="color:#EEAD0E">feature</span>
 
<span style="color:#458B00">master</span>
 
<span style="color:#CD2626">version</span>
 
<span style="color:#00008B">staging</span>
 
 
 
-->
 
'''If you're an experienced git user, skip to the [[#Short Summary|summary]].'''
 
 
 
==Definitions==
 
===Version numbers===
 
We follow the traditional dmd versioning: ''2.061''. Tags have a ''v'' prepended, so the tag for version ''2.061.1'' is ''v2.061.1''. Versions where the third part is bigger than zero are called ''Minor Releases''. Versions where the third part is equal zero are ''Major Releases''. ''Minor Releases'' will only contain regression fixes, ''Major Releases'' will contain bug fixes and new features.
 
 
 
The third digit can be omitted for ''Major Releases'': ''2.061'' is the same as ''2.061.0''.
 
 
 
Meta: The dmd contributors should choose one style for all offical statements: Tags, packages, Notes on the download page. Either '''always''' write out the ''.0'' part or '''always''' omit it.
 
 
 
===Regression fix vs. Bug fix===
 
A regression fix for release '''X''' is a fix for a bug which was newly introduced in release '''X'''. We distinguish between regression fixes and bug fixes to avoid introducing more regressions in minor releases. A regression fix especially should not make code invalid which was valid before the regression occurred. As an example: If we have releases 2.061 and 2.061.1 then 2.061.1 should always be "better" than 2.061. It should be tried heavily not to introduce new regressions in minor releases. Code which compiled with 2.061 should still compile in 2.061.1 (there's one rare exception: If a regression allowed code to work in 2.061 which did not work in 2.060 and which is not supposed to work, it can be disabled again in 2.061.1).
 
 
 
So a regression fix is a bug fix which should not have side-effects, doesn't introduce new features and fixes a bug which only recently appeared.
 
 
 
Long standing bugs are especially not regressions! (Or technically probably a regression in a very old release, but as we support only 1 major release we can treat those like bugs. We only care about new regressions)
 
 
 
===Small enhancement vs. Big feature===
 
We later suggest different workflows for those two enhancements. Those are easily distinguishable: If you implement the feature by yourself and don't need help / someone to participate / extended review and the code changes are ''reasonably'' small, it's a small enhancement. Otherwise it's a big feature.
 
 
 
The workflow for big features allows other contributors to participate in that feature, make special test releases for that feature, etc. If you want / need those, use the Big Feature workflow.
 
 
 
===Stable===
 
When is a release stable and therefore ready to be released?
 
 
 
The workflow already guarantees that when a release can be made there has been a feature freeze for a long time. So only bug fixes and regression fixes where allowed for some time which should already stabilize the release.  But we should add some more criteria:
 
 
 
* All bugs marked as critical in bugzilla have been fixed
 
** For practical reasons we allow exceptions from this rule for the first releases after this workflow has been established
 
* There are no newly introduced (known) regressions
 
** This is a hard rule, there are no exceptions to it. Have a regression? Don't release!
 
* Later a yet to be determined percentage of non-critical bug will need to be fixed (or new features introduced)
 
** This is to make sure a new release actually provides something new and prevent us from releasing too often. Currently this is not an issue.
 
 
 
Apart from these criteria, the final decision is made by the release manager.
 
 
 
=== <span style="color:#CD2626">Version</span> / <span style="color:#CD2626">Release</span> branch ===
 
This wiki page defines <span style="color:#CD2626">version</span> branches, other projects often call those branches <span style="color:#CD2626">release</span> branches. This wiki page will call them <span style="color:#CD2626">version</span> branches for now, but a <span style="color:#CD2626">release</span> branch and a <span style="color:#CD2626">version</span> branch is exactly the same thing.
 
 
 
==Don'ts==
 
* do not ''cherry pick'' from one official branch into another
 
* do not ''rebase'' official branches
 
* do not rewrite git history on official branches
 
* do not merge <span style="color:#458B00">master</span> into a <span style="color:#CD2626">version</span> branch (The only time when commits go from <span style="color:#458B00">master</span> to a <span style="color:#CD2626">version</span> branch is when the <span style="color:#CD2626">version</span> branch is being created)
 
* do not merge <span style="color:#EEAD0E">feature</span> branches into <span style="color:#CD2626">version</span> branches. <span style="color:#EEAD0E">Feature</span> branches are always merged into <span style="color:#458B00">master</span>.
 
 
 
==Release Schedule==
 
<pre>
 
<x> = time in weeks
 
vxxx = Tag for version xxx (Marks the release)
 
rest = branches
 
\ = merge
 
------------------------------------------------------------------------------------ master branch. development is never stopped
 
            \                                            \         
 
            \                                            ------------------------- 2.063 release branch
 
              \                                            |        |  <2>  |
 
              \                                    v2.063.0-b1    |        |
 
                \                                                  |        |
 
                \                                              v2.063.0  v2.063.1
 
                  \         
 
                  ----------------------------------------------------------------- 2.062 release branch
 
                  |        |  <2>    |    <2>  |
 
          v2.062.0-b1  v2.062.0 v2.062.1  v2.062.2
 
</pre>
 
 
 
A new release is made approximately every 2 months. Releases are stabilized in the <span style="color:#CD2626">version</span> branch and beta releases are tagged as part of <span style="color:#CD2626">version</span>.
 
 
Once the initial major release on the <span style="color:#CD2626">version</span> branch is done, only regression fixes will be pushed to the <span style="color:#CD2626">version</span> branch, then a minor release is published every 2 weeks from that <span style="color:#CD2626">version</span> branch as long as it's supported (see [[#Preparing a new minor release]]).
 
 
 
 
 
Feature Previews: A <span style="color:#EEAD0E">feature</span> branch may need it's own beta release before being merged into <span style="color:#458B00">master</span>. These are unplanned, announced on the NG, and highly volatile.
 
 
 
''As an example we would see these releases for things like 64bit support and User Defined Attributes.''
 
 
 
===Support times===
 
Currently only the most recent major release is supported. (This might change in the future)
 
 
 
As an example:
 
version <span style="color:#CD2626">2.061</span> is released. <span style="color:#CD2626">2.061</span> will get updates until <span style="color:#CD2626">2.062</span> is released. Then <span style="color:#CD2626">2.062</span> will get updates until <span style="color:#CD2626">2.063</span> is released etc.
 
 
 
Only supported <span style="color:#CD2626">releases</span> receive minor update releases.
 
 
 
==Branching model==
 
 
 
=== Official branches ===
 
The following branches will be available on the official (upstream) [https://github.com/D-Programming-Language/dmd dmd], [https://github.com/D-Programming-Language/druntime druntime] and [https://github.com/D-Programming-Language/phobos phobos] repositories:
 
 
 
* '''<span style="color:#EEAD0E">feature branches</span>''': New features, in general, are '''not''' developed directly on the <span style="color:#458B00">master</span> branch, but in a dedicated <span style="color:#EEAD0E">feature</span> branch. An example for such a branch would be an ''UDA'' branch for user defined attributes. Contributors can collaborate on the feature by opening pull request targeting that <span style="color:#EEAD0E">feature</span> branch. When the feature's implementation is more-or-less complete, the <span style="color:#EEAD0E">feature</span> branch will be merged into the <span style="color:#458B00">master</span> branch. The <span style="color:#458B00">master</span>  branch is also sporadically merged into the '''<span style="color:#EEAD0E">feature</span>''' branches to get regression fixes and bug fixes and to make sure the '''<span style="color:#EEAD0E">feature</span>''' branch can be merged into <span style="color:#458B00">master</span> later on without problems.
 
<pre>
 
 
 
--------------------------------------------------------\ feature64BitWindows
 
  /                          /            /            \
 
  /                          /            /              \
 
/                          /            /                \
 
------------------------------------------------------------------------------------ master branch. development is never stopped
 
</pre>
 
 
 
* '''<span style="color:#458B00">master</span>''': The <span style="color:#EEAD0E">feature</span> branches are merged into this branch if they are considered complete. So the <span style="color:#458B00">master</span>  branch will contain new features and may have breaking changes. Regression fixes  and bug fixes may be merged from <span style="color:#CD2626">version</span> branches into <span style="color:#458B00">master</span>.
 
<pre>
 
 
 
--------------------------------------------------------\ feature64BitWindows
 
                                                        \
 
-------------------\  featureUDA                          \
 
                    \                                      \
 
------------------------------------------------------------------------------------ master branch. development is never stopped
 
                                    /
 
                                    /
 
------------------------------------------------------------------------------------ 2.063 branch
 
                                /
 
                            fixEvilRegression
 
</pre>
 
 
 
* '''<span style="color:#CD2626">version branches</span>''': Once we determine it's time to do a new release, a <span style="color:#CD2626">version</span>  branch is made by branching from the <span style="color:#458B00">master</span>  branch. A version branch will not receive any new features, only regression and important bug fixes. To determine to which <span style="color:#CD2626">version</span>  branch regression fixes should go, see [[#Regression fix]]. <span style="color:#CD2626">Version</span>  branches are used to support the version for an extended period of time to enable updated minor releases which fix regressions.
 
 
 
===Suggested local branches===
 
How user repositories are managed is up to the end user, but we propose these branches:
 
* '''<span style="color:#EEAD0E">feature branches</span>''': Tracking upstream <span style="color:#EEAD0E">feature</span> branches. You only need a <span style="color:#EEAD0E">feature</span> branch if you want to work on that feature. To work on a feature, make changes to the <span style="color:#EEAD0E">feature</span> branch, push to your repository and file a pull request on github '''targeting the <span style="color:#EEAD0E">feature</span> branch'''
 
 
 
* '''<span style="color:#458B00">master</span>''': Tracking the upstream <span style="color:#458B00">master</span>  branch. New feature branches should be branched from this branch.
 
 
 
* '''<span style="color:#CD2626">version branches</span>''': Tracking the upstream <span style="color:#CD2626">version</span>  branches. If you want to push a regression  fix to an older release. (See [[#Regression fix]])
 
 
 
* '''bugfix/small feature branches''': based on <span style="color:#458B00">master</span>. Used to develop bugfixes and small features (see [[#Bug fix (not a regression!)|Bug fix]])
 
 
 
==Git workflow==
 
 
 
===Local repository setup===
 
All instructions on this page assume the following setup for the local repository. Experienced git users of course may use different setups.
 
 
 
First, create a fork of the official repository (if not already done) on github. Then follow these instructions:
 
 
 
<source lang="bash">
 
#We use your own fork repository, not the official one!!!!
 
git clone git://github.com/yourname/dmd.git
 
#Now add the official repository as a remote
 
git remote add upstream git://github.com/D-Programming-Language/dmd.git
 
git remote update
 
 
 
#sync / create branches on origin
 
git push origin upstream/master:refs/heads/master
 
 
 
#Repeat this for every release & feature branch you want to use
 
git push origin upstream/2.061:refs/heads/2.061
 
 
 
#Now checkout / create local branches
 
#Repeat for every release & feature branch you want to use
 
git checkout -b 2.061 origin/2.061
 
</source>
 
 
 
The above instructions have to be executed only once. But if a new feature/release branch is available and you want to use it, you have to repeat this part:
 
<source lang="bash">
 
git remote update
 
 
 
#Repeat this for every release & feature branch you want to use
 
git push origin upstream/2.061:refs/heads/2.061
 
 
 
#Repeat for every release & feature branch you want to use
 
git checkout -b 2.061 origin/2.061
 
</source>
 
 
 
Note:
 
Before creating new branches, or doing similar stuff you should always do this:
 
<source lang="bash">
 
git remote update
 
</source>
 
The instructions on this page always explicitly mention when you have to use this command.
 
 
 
===Working on code updates===
 
 
 
====What type of update?====
 
To determine to which branch an update should be made, we have to consider the type of the change:
 
 
 
* It's a new, big feature: This will need to get a <span style="color:#EEAD0E">feature</span> branch. See [[#New feature / Big enhancement]]
 
* It's a small enhancement: This will go to <span style="color:#458B00">master</span>. See [[#Small enhancement]]
 
* It's a bug fix: This will usually go to <span style="color:#458B00">master</span>. See [[#Bug fix (not a regression!) | Bug fix]]. If the bug fix must be part of a specific release it may also be merged into a <span style="color:#CD2626">version</span> branch.
 
* It's a regression fix: This will need to go to the oldest '''supported''' <span style="color:#CD2626">version</span> branch which is affected by the regression. See [[#Regression fix]]
 
 
 
====New feature / Big enhancement====
 
We need to create a new branch based on <span style="color:#458B00">master</span>. As a normal user you can't create <span style="color:#EEAD0E">feature</span> branches on the official repository though.
 
 
 
Step-By-Step:
 
The one implementing the feature has to do this:
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout master #This is the branch we base feature on
 
git pull origin master #Fetch changes from origin
 
git pull upstream master #Fetch changes from upstream
 
 
 
git branch coolFeature
 
#Make your changes
 
git commit -m "Implement cool feature"
 
#Push to your fork on github
 
git push origin coolFeature
 
 
 
#Now request in the digitalmars.D newsgroup that someone creates a coolFeature feature branch on the official repository.
 
 
 
#Now go to github and file a pull request for coolFeature. The pull request must target the new official feature branch (here also called coolFeature)!
 
</source>
 
 
 
If someone requests a new <span style="color:#EEAD0E">feature</span> branch the dmd staff has to do this:
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout master #This is the branch we base features on
 
git pull upstream master #Fetch changes from upstream
 
 
 
git branch coolFeature #Use a useful name
 
git push upstream coolFeature
 
</source>
 
 
 
====Small enhancement====
 
We merge directly into <span style="color:#458B00">master</span>.
 
 
 
Step-By-Step:
 
The one implementing the enhancement has to do this:
 
<source lang="bash">
 
#We use your own fork repository, not the official one!!!!
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout master #This is the branch we base feature on
 
git branch coolEnhancement
 
#Make your changes
 
git commit -m "Implement cool enhancement"
 
#Push to your fork on github
 
git push origin coolEnhancement
 
 
 
#Now go to github and file a pull request for coolEnhancement. The pull request must target the master branch!
 
</source>
 
 
 
====Bug fix (not a regression!)====
 
If this bug fix should be part of a specific release, follow the instructions for regression fixes: [[#Regression fix]]
 
 
 
We need to create a local branch based on <span style="color:#458B00">master</span>. Then file a pull request on github targeting <span style="color:#458B00">master</span>.
 
 
 
Step-By-Step:
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout master #This is the branch we base bugfix on
 
git pull origin master
 
git pull upstream master
 
 
 
git branch bugFixNNN
 
#Make your changes
 
git commit -m "Implement bugfix #NNN"
 
#Push to your fork on github
 
git push origin bugFixNNN
 
 
 
#Now go to github and file a pull request for bugFixNNN. The pull request must target the master branch!
 
</source>
 
 
 
====Regression fix====
 
The regression fix must be pushed to the oldest, still supported <span style="color:#CD2626">version</span> branch affected by the regression. To determine if a <span style="color:#CD2626">version</span> is still supported, see [[#Support times]]. For now only the last release is supported.
 
 
 
'''Note''': Regression fixes are always based on an <span style="color:#CD2626">version</span> branch, never on <span style="color:#458B00">master</span>!
 
 
 
Step-By-Step:
 
The one fixing the regression has to do this:
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout 2.062 #This is the branch we base our fix on
 
git pull origin 2.062
 
git pull upstream 2.062
 
 
 
git branch fixRegressionNNN
 
#Edit and fix the regression
 
git commit -m "Fix regression #nnn"
 
#Push to your fork on github
 
git push origin fixRegressionNNN
 
#Now go to github and file a pull request for fixRegressionNNN. The pull request must target our base branch, e.g. 2.062!
 
</source>
 
 
 
The dmd staff now has to merge that pull request. The regression fix will be applied to the <span style="color:#CD2626">2.062</span> branch. The dmd staff now also has to make sure this fix is also applied to all newer release branches (as we only support one release this is never the case right now) and to <span style="color:#458B00">master</span>.
 
 
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout 2.062 #This is the branch the fix was based on
 
git pull upstream 2.062 #Fetch changes from upstream
 
 
 
#Repeat for all newer release branches (not necessary as long as only latest release is supported)
 
#git checkout 2.063
 
#git pull upstream 2.063 #Fetch remote changes from upstream first
 
#git merge 2.062 #Now merge 2.062 branch into newer branch
 
#git push upstream 2.063 #Now publish those changes
 
 
 
#We have to repeat that for master
 
git checkout master
 
git pull upstream master #Fetch remote changes from upstream first
 
git merge 2.062 #Now merge 2.062 branch into master branch
 
git push upstream master #Now publish those changes
 
</source>
 
 
 
===Preparing a new major release===
 
As described in [[#Release Schedule]] every 2 months a new major <span style="color:#CD2626">release</span> will be made based on a <span style="color:#CD2626">version</span> branch.
 
 
 
To start the release process (This is always done after a major release has been made) create a new <span style="color:#CD2626">version</span> branch for the next release:
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout master
 
git pull upstream master
 
 
 
git branch 2.063
 
git push upstream 2.063
 
</source>
 
 
 
Once a beta release is made, the dmd staff adds a tag on the <span style="color:#CD2626">version</span> branch!
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout 2.063
 
git pull upstream 2.063
 
 
 
git tag v2.063-b1 #b1=>first beta
 
git push upstream v2.063-b1
 
</source>
 
 
 
For the final release, create a new "v2.063" tag just as for betas.
 
 
 
===Preparing a new minor release===
 
As described in [[#Release Schedule]] every 2 weeks  a new minor release will be prepared for all supported releases.
 
 
 
To put further regression fixes into this release <span style="color:#CD2626">version</span> follow the descriptions in [[#Regression fix]]
 
 
 
Once the release is ready, the dmd staff adds a tag!
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout 2.063 #checkout the base branch where a new release is made
 
git pull upstream 2.063
 
 
 
git tag v2.063.1 #.1=>first minor update
 
git push upstream v2.063.1
 
</source>
 
 
 
===Merging a feature branch===
 
If a feature's implementation is ''sufficiently'' complete and has been tested ''well enough'', the <span style="color:#EEAD0E">feature</span> branch can be merged into <span style="color:#458B00">master</span>.
 
 
 
DMD staff:
 
<source lang="bash">
 
#Make sure you've read "Local repository setup"
 
git remote update
 
 
 
git checkout featureXXX #checkout the feature branch
 
git pull upstream featureXXX
 
 
 
git checkout master
 
git pull upstream master
 
git merge featureXXX #if merge fails, do not fix it yourself. contact the maintainer of the feature branch
 
git push upstream master
 
</source>
 
 
 
==Short Summary==
 
===Contributors===
 
* Regression fixes go into <span style="color:#CD2626">version</span> branches
 
* Bug fixes usually go into the <span style="color:#458B00">master</span> branch. If the bugfix should be included in a specific release, it's pushed to a <span style="color:#CD2626">version</span> branch.
 
* Big new features are developed in <span style="color:#EEAD0E">feature</span> branches and merged into <span style="color:#458B00">master</span>
 
* Small enhancements can go directly into <span style="color:#458B00">master</span>
 
 
 
===DMD staff===
 
* Merge regression fixes into <span style="color:#CD2626">version</span> branch asap.
 
* Merge regressions fixes from <span style="color:#CD2626">version</span> to <span style="color:#458B00">master</span> branch immediately.
 
* Merge bug fixes and small features into <span style="color:#458B00">master</span> regularly.
 
* Merge <span style="color:#EEAD0E">feature</span> branches into <span style="color:#458B00">master</span> on demand.
 
 
 
==Rationale==
 
=== <span style="color:#00008B">Staging</span> ===
 
This is the common branch model used by many projects. While an additional staging branch could be useful in some corner cases it would complicate the process.
 
 
 
=== Regression vs bug fix===
 
<blockquote style="background-color: lightgrey; border: solid thin grey;">
 
we need to differentiate between bug fixes
 
that do not introduce new bugs vs bug fixes that may introduce
 
new bugs *or* break existing code for the current release
 
version. We also *have* to prevent new features and structural
 
changes (destabilizers) from leaking into a stable release.
 
</blockquote>
 
http://forum.dlang.org/post/dticrdtkkqdatdnhtzrc@forum.dlang.org
 
 
 
*The regression/bug differentiation is therefore a very strict model. It's needed for dmd as every bug fix has a probability of breaking user code and therefore we should keep the changes as few as possible. In dmd we also have the issue that we have long-standing bugs: Bugs that were not introduced, but are differences between dmd and the specification. We don't want to get any of these fixed in minor releases. For druntime and phobos there are few such bugs: Every bug has been introduced in some version. The risk of a bug fix causing regressions also seems to be lower for phobos and druntime, so we could allow bug fixes in release branches for phobos/druntime after the process has been well established.
 
 
 
=== Regression fixes in release branches ===
 
<blockquote style="background-color: lightgrey; border: solid thin grey;">
 
Now why we fix regressions in the release branches:
 
Let's say we have dmd 2.061, 2.062, 2.062.1. A regression was
 
introduced in 2.062 and my code that compiled fine in 2.061 doesn't work
 
in 2.062 anymore. But then the regression fix is pushed to the 2.062
 
release branch, a minor release is made and 2.062.1 successfully
 
compiles that code again.
 
 
 
If the regression already occured in 2.061 should it be fixed in 2.062?
 
This is a different question, but that situation can't happen as long
 
as we only support 1 release and only fix newly introduced regressions.
 
</blockquote>
 
http://forum.dlang.org/post/kc90k0$2jue$1@digitalmars.com
 
 
 
=== Stable ===
 
* http://forum.dlang.org/post/fbtxyjvpqqrxskptcvgk@forum.dlang.org
 
* http://forum.dlang.org/post/noycjddhpobnnxkqsytx@forum.dlang.org
 
 
 
==Possible optimization==
 
 
 
===Simplification===
 
[[Simplified Release Process Proposal]]
 
 
 
===Release version numbering===
 
* When we start shipping shared libraries of druntime and phobos we might have to adjust our versioning scheme as those dictate special rules. (Think of ABI compatibility between minor releases, max 3 numbers for a version, etc)
 
 
 
===rebasing===
 
I guess we should have a hard stance on not rebasing any of the official branches. (this especially includes feature branches where this might seem to be useful, but probably is still a bad idea)
 
 
 
----
 
 
 
[[Category:git]]
 
[[Category:development]]
 
[[Category:know-how]]
 

Latest revision as of 18:47, 8 September 2015

Redirect to: