www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Next focus: PROCESS

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Hello all,


Walter and I have had a long discussion following his trip to Australia. 
Following the current sprint for Win64 (which we all, I think, agree was 
long overdue and had to be done), the main area we need to work on (as 
I'm sure many would agree) is improving our process, in particular the 
approach to releasing the compiler and standard library.

Walter has delegated me to lead this effort, and agreed to obey with 
whatever process comes forward as long as it's reasonably clear and simple.

In turn, I'll be collecting thoughts and opinions in this thread, 
distilled from previous discussions. We should develop a few simple git 
scripts (such as git-start-new-feature or git-start-bugfix etc) 
supported by an equally simple piece of documentation describing how to 
start a new release, fix a bug, experiment with a new feature, and such.

(One piece that has been brought forward is 
http://nvie.com/posts/a-successful-git-branching-model/ - something to 
keep in mind.)

Please chime in with ideas and thoughts.


Thanks,

Andrei
Dec 10 2012
next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:

 Hello all,


 Walter and I have had a long discussion following his trip to 
 Australia. Following the current sprint for Win64 (which we 
 all, I think, agree was long overdue and had to be done), the 
 main area we need to work on (as I'm sure many would agree) is 
 improving our process, in particular the approach to releasing 
 the compiler and standard library.

 Walter has delegated me to lead this effort, and agreed to obey 
 with whatever process comes forward as long as it's reasonably 
 clear and simple.

 In turn, I'll be collecting thoughts and opinions in this 
 thread, distilled from previous discussions. We should develop 
 a few simple git scripts (such as git-start-new-feature or 
 git-start-bugfix etc) supported by an equally simple piece of 
 documentation describing how to start a new release, fix a bug, 
 experiment with a new feature, and such.

 (One piece that has been brought forward is 
 http://nvie.com/posts/a-successful-git-branching-model/ - 
 something to keep in mind.)


As someone who uses the git workflow described in this link every 
single day let me say I'm a big fan.  It's so well documented 
(and widely used) that it should be pretty easy for everyone to 
adopt and it provides a lot of real benefits to a project.  From 
the look of it most contributors already use feature branches for 
their pull requests so this should be a pretty simple transition.


 Please chime in with ideas and thoughts.


I like the idea of having a release manager (that changes to 
whoever volunteers for each release). It'll take some of the 
release time pressure off of Walter (at least for the times he 
isn't the release manager) and raise D's bus factor (which is 
very close to being above 1 these days) as the release process is 
documented and shared among several individuals.  I imagine the 
role of release manager being fairly laid back though (it is a 
volunteer job still).  Their responsibilities would be rolling 
the betas and actual releases and just being the point-man who 
goes through and makes sure all the pull requests that are ready 
to merge are merged and last minute regressions are acknowledged.

BA
Dec 10 2012
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:

 Hello all,
 
 Walter and I have had a long discussion following his trip to
 Australia. Following the current sprint for Win64 (which we all, I
 think, agree was long overdue and had to be done), the main area we
 need to work on (as I'm sure many would agree) is improving our
 process, in particular the approach to releasing the compiler and
 standard library.


*applause* It's about time we did this right.



 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and
 simple.


Good to know.



 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple
 git scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how
 to start a new release, fix a bug, experiment with a new feature,
 and such.


Sounds OK to me, though it seems unnecessarily complicated. What's wrong
with just doing things like:



	git branch -b brand_new_feature

	git commit -a


	git checkout develop
	git merge brand_new_feature


	git checkout develop
	git branch -b 2.062-rc1

	git commit -a


	git checkout master
	git merge 2.062-rc1

Seems to me these are all straightforward everyday git commands. The
whole selling point of git is that branching/merging is so easy that you
could do it multiple times a day within a single implementation effort.
For example, I frequently find myself doing things like this:


	git checkout master


	git checkout -b feature_X_1





	git commit -a
	git checkout -b feature_X_alt_impl





	git checkout feature_X_1




	git commit -a
	git checkout master

	git commit -a



	git checkout feature_X_1




	git checkout -b feature_X_new_idea

	git commit -a



	git checkout -b feature_X_1
	git merge feature_X_new_idea





	git branch -d feature_X_new_idea
	git branch -d feature_X_alt_impl


	vim feature.d
	git commit -a



	git checkout master
	git merge feature_X_1

This can happen several times a day when I'm doing a lot of coding.



 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something
 to keep in mind.)


[...]

Glanced at it briefly. I think the basic ideas behind it are sound.
Whether or not we should adopt all of the specifics is up for
discussion, but I think we should definitely incorporate the main ideas:
(1) have a release-ready branch, (2) a separate dev branch, and (3)
always use a separate branch for new features so that if things go wrong
or it takes too long to finish implementation, development work on other
stuff can continue without being held up by bugs in the new feature.

Furthermore, in a git environment it's perfectly normal to have multiple
"exploratory" or "sub-feature" branches within a single feature branch,
like I illustrated above in exploring how to implement feature X. These
are usually just local branches, but if the feature is complex enough,
it can warrant pushing to github so that other devs can also play around
with the alternative implementation approaches. Eventually the best
approach can be merged back to the feature branch and the others
discarded.

Personally, I find that this approach is very profitable, because you
don't have to worry that a particular breaking change may turn out to be
a dead-end; do the experiment in a sub-branch and throw it away if it
doesn't work. The cheapness of doing this allows one to explore
ambitious or risky approaches without affecting the stable "traditional"
implementation, and so opens up the possibility of discovering new and
novel ways of doing things that are usually left unexplored.


T

-- 
It said to install Windows 2000 or better, so I installed Linux instead.
Dec 10 2012
next sibling parent =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 11-12-2012 01:19, H. S. Teoh wrote:

 On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:

 Hello all,

 Walter and I have had a long discussion following his trip to
 Australia. Following the current sprint for Win64 (which we all, I
 think, agree was long overdue and had to be done), the main area we
 need to work on (as I'm sure many would agree) is improving our
 process, in particular the approach to releasing the compiler and
 standard library.


 *applause* It's about time we did this right.



 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and
 simple.


 Good to know.



 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple
 git scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how
 to start a new release, fix a bug, experiment with a new feature,
 and such.


 Sounds OK to me, though it seems unnecessarily complicated. What's wrong
 with just doing things like:



 	git branch -b brand_new_feature

 	git commit -a


 	git checkout develop
 	git merge brand_new_feature


 	git checkout develop
 	git branch -b 2.062-rc1

 	git commit -a


 	git checkout master
 	git merge 2.062-rc1


master must be the development branch. This is a pitfall the Rust guys 
walked into: People accidentally send their pull requests to the master 
branch instead of the develop branch which overall ends up being a mess 
when a reviewer doesn't notice the error before merging.

Otherwise sounds good.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Dec 10 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/10/12 7:19 PM, H. S. Teoh wrote:

 Sounds OK to me, though it seems unnecessarily complicated.


FWIW there's also this:

http://drewfradette.ca/a-simpler-successful-git-branching-model/


Andrei
Dec 10 2012
next sibling parent =?ISO-8859-1?Q?S=F6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 11.12.2012 03:02, schrieb Andrei Alexandrescu:

 On 12/10/12 7:19 PM, H. S. Teoh wrote:

 Sounds OK to me, though it seems unnecessarily complicated.


 
 FWIW there's also this:
 
 http://drewfradette.ca/a-simpler-successful-git-branching-model/
 
 
 Andrei


Just throwing this in: http://vibed.org/temp/branch-model-small.png

It avoids the pull request problem that was mentioned (pull target for a new
feature may often
erroneously be master vs. develop) and treats the release branches as the
stable branches from which
release tags are directly published. This seems pretty close to the actual mode
of development and a
single master for releases, as in the "successful git branching" models, seems
to be insufficient
anyway; continuous parallel releases of 1.x and 2.x (or 2.0.x and 2.1.x) seem
not to really fit in
there.
Dec 11 2012
prev sibling parent reply "foobar" <foo bar.com> writes:
On Tuesday, 11 December 2012 at 02:02:54 UTC, Andrei Alexandrescu
wrote:

 On 12/10/12 7:19 PM, H. S. Teoh wrote:

 Sounds OK to me, though it seems unnecessarily complicated.


 FWIW there's also this:

 http://drewfradette.ca/a-simpler-successful-git-branching-model/


 Andrei


First of all - Yay!

There are still a few open questions that need to be decided
before a suitable process can be defined.

a. Should we have more than one stable release and provide
support branches?
That means we have version X which is latest stable with all the
latest (stable) features included and we have an older Y version
that only receives bug-fixes and is intended for support (think
of Ubuntu's LTE versions). If we have such additional versions,
how many do we have, for how long?
My opinion - (based on Walter's) you can always download a
previous version of the compiler. I'd say we should _at most_
support _one_ previous stable version with critical bug fixes
only. we could always decide to support ad-hoc additional
specific versions, for example before we introduce major paradigm
shifting changes and this also depends on available manpower.

B. should we have public pre-release versions?
I think we should release additional "releases" - call them beta,
pre-release, release candidates, whatever. These are for staging
changes, allowing to field test new features and language changes
before they are made final. Also allowing users to adjust their
code-bases.

Given the above, I think the "simpler flow" model above looks
very promising and we should adopt it with minor changes:
1. Both "staging" and "release" should be "public" - the website
lists both the stable versions and the staging versions.
2. the development branch should be the master branch for easier
integration with pull requests.

The staging releases (let's call them beta) will provide previews
of upcoming and experimental features to try out without the
promise of a stable API and would be released _monthly_.

The stable releases will be every 3-4 months and will guaranty a
stable API. A user could download DMD-201212 (i.e. this month)
and play around with UDA and of course report bugs, but knowing
that this feature could change syntax or even be completely
removed thus allowing the developers to test stuff out in the
open without committing to anything.
Dec 11 2012
parent reply "Rob T" <rob ucora.com> writes:
On Tuesday, 11 December 2012 at 13:19:56 UTC, foobar wrote:

 First of all - Yay!

 There are still a few open questions that need to be decided
 before a suitable process can be defined.

 I'd say we should _at most_
 support _one_ previous stable version with critical bug fixes
 only.


I agree with that as well, although I think that after a new 
major "stable" release, the previously stable should be supported 
(at a minimum kept available for download) for some time until 
the new stable is fully stabilized and most people have been able 
to adopt it. It may be good enough to just and pick and choose if 
the previous stable should get certain bug fixes or not until the 
support period runs out.


 B. should we have public pre-release versions?


A lot of people will want to use the latest available changes for 
actual development, so the "testing" or "pre-release" branch 
should be public and kept reasonably stable, although anything 
can happen, so it's not considered "stable", just "stable enough" 
given that it may be supporting new features and improvements 
that have been selected for the next major stable release.


 I think we should release additional "releases" - call them 
 beta,
 pre-release, release candidates, whatever. These are for staging
 changes, allowing to field test new features and language 
 changes
 before they are made final. Also allowing users to adjust their
 code-bases.


I think you'll need at a minimum experimental branches for 
testing out new ideas, the main development branch witch is 
considered unstable (the master branch is probably best for this 
one as was suggested), a pre-release or testing branch that is 
used for preparing for the next major stable release, and of 
course the current stable branch which only receives bug fixes up 
until the next pre-release branch is merged into stable.

One more thing, is that we should adopt a version numbering 
system that is appropriate to indicate major, minor, and bug fix 
releases. The method of major.minor.revision can work well for 
us, but there may be alternatives that will work even better 
depending on what the process ends up being.

What I'd hate to see continuing, is a major new release going out 
with no indication in the version number that it is a major new 
release as opposed to a minor revision change. For example, the 
current DMD stable is 2.060, and the next release will be 2.061, 
but it includes brand new poorly tested features, and one of them 
is still being debated on, therefore it may be subject to change. 
The next release will be anything but a minor update and it 
should not even be considered as a stable release, it's more like 
a pre-release version for testing and for adoption by those who 
absolutely need the latest "reasonably stable" version for their 
development work.

--rt
Dec 11 2012
parent reply "foobar" <foo bar.com> writes:
On Tuesday, 11 December 2012 at 19:57:55 UTC, Rob T wrote:

 On Tuesday, 11 December 2012 at 13:19:56 UTC, foobar wrote:

 First of all - Yay!

 There are still a few open questions that need to be decided
 before a suitable process can be defined.

 I'd say we should _at most_
 support _one_ previous stable version with critical bug fixes
 only.


 I agree with that as well, although I think that after a new 
 major "stable" release, the previously stable should be 
 supported (at a minimum kept available for download) for some 
 time until the new stable is fully stabilized and most people 
 have been able to adopt it. It may be good enough to just and 
 pick and choose if the previous stable should get certain bug 
 fixes or not until the support period runs out.


By support I meant specifically _bug fixes_. You can already 
download all previous released versions from the website and in 
no way am I arguing to change that policy.
Even if we ever get to a point where we don't want to keep older 
releases which I doubt very much (each zip is only a few MBs and 
therefore insignificant on today's storage) - we could still 
easily just checkout the specific release tagged in git and just 
build that.


 B. should we have public pre-release versions?


 A lot of people will want to use the latest available changes 
 for actual development, so the "testing" or "pre-release" 
 branch should be public and kept reasonably stable, although 
 anything can happen, so it's not considered "stable", just 
 "stable enough" given that it may be supporting new features 
 and improvements that have been selected for the next major 
 stable release.


This is precisely what I addressed bellow. we have monthly build 
of our staging branch - call them monthly betas that include new 
upcoming features that are already stable enough to be released 
to the public for field testing and are tentative for the next 
actual release but until these feature actually get to a release 
they hold no guaranties and can be further modified based on the 
wider public testing - including changes in API. Once released, 
they do hold such guaranties of API stability. So these monthly 
betas will provide preview of language changes and allow people 
to prepare for future changes and also provide feedback.


 I think we should release additional "releases" - call them 
 beta,
 pre-release, release candidates, whatever. These are for 
 staging
 changes, allowing to field test new features and language 
 changes
 before they are made final. Also allowing users to adjust their
 code-bases.


 I think you'll need at a minimum experimental branches for 
 testing out new ideas, the main development branch witch is 
 considered unstable (the master branch is probably best for 
 this one as was suggested), a pre-release or testing branch 
 that is used for preparing for the next major stable release, 
 and of course the current stable branch which only receives bug 
 fixes up until the next pre-release branch is merged into 
 stable.


See comment above. the pre-release will contain new features 
already stable enough to be consumes by the general public 
_before_ we the developers are ready to a commit finally to their 
API. E.g. Walter's release of DMD with attributes that was 
already tested and working but after release people argued about 
changing its syntax from [attributes] to  (attributes).

developers can have their own experimental branches for their own 
tests and new ideas, but once a feature reaches pre-release it 
should already be tested and working and ready for public 
consumption without commiting to a final API.


 One more thing, is that we should adopt a version numbering 
 system that is appropriate to indicate major, minor, and bug 
 fix releases. The method of major.minor.revision can work well 
 for us, but there may be alternatives that will work even 
 better depending on what the process ends up being.


I really don't care about the numbering scheme and this is 
irrelevant to the topic of this discussion. We are discussing the 
PROCESS of development. How the releases are tagged is completely 
beside the point and could be named after sweet delights, cat 
names, Parks or even digits of PI. I really don't care and it 
really is _not important_.
This is one of those fundamental things that are required to 
truly understand git - versions are the _content_ (code) they 
contain and are identified by a hash of that content.
This is pure bikesheding but why not: Why not extend the 
astronomical theme to releases as well? What would you say about 
the latest Pluto release of DMD? ;)
(Yeah, I know this is already used by the eclipse project..)


 What I'd hate to see continuing, is a major new release going 
 out with no indication in the version number that it is a major 
 new release as opposed to a minor revision change. For example, 
 the current DMD stable is 2.060, and the next release will be 
 2.061, but it includes brand new poorly tested features, and 
 one of them is still being debated on, therefore it may be 
 subject to change. The next release will be anything but a 
 minor update and it should not even be considered as a stable 
 release, it's more like a pre-release version for testing and 
 for adoption by those who absolutely need the latest 
 "reasonably stable" version for their development work.

 --rt
Dec 11 2012
next sibling parent reply "Rob T" <rob ucora.com> writes:
On Tuesday, 11 December 2012 at 23:15:27 UTC, foobar wrote:

 By support I meant specifically _bug fixes_. You can already 
 download all previous released versions from the website and in 
 no way am I arguing to change that policy.
 Even if we ever get to a point where we don't want to keep 
 older releases which I doubt very much (each zip is only a few 
 MBs and therefore insignificant on today's storage) - we could 
 still easily just checkout the specific release tagged in git 
 and just build that.


That's what I meant too, I agree with you on this point.


 This is precisely what I addressed bellow. we have monthly 
 build of our staging branch - call them monthly betas that 
 include new upcoming features that are already stable enough to 
 be released to the public for field testing and are tentative 
 for the next actual release but until these feature actually 
 get to a release they hold no guaranties and can be further 
 modified based on the wider public testing - including changes 
 in API. Once released, they do hold such guaranties of API 
 stability. So these monthly betas will provide preview of 
 language changes and allow people to prepare for future changes 
 and also provide feedback.


Having a pre-release build that can be downloaded and installed 
would be very nice to have if that's what you mean. Currently, to 
get DMD 2.061 "alpha" I have to jump through hoops to get it to a 
point where it is operational, and that is not so good as it 
limits severely how many users will be able to test it out.


 See comment above. the pre-release will contain new features 
 already stable enough to be consumes by the general public 
 _before_ we the developers are ready to a commit finally to 
 their API. E.g. Walter's release of DMD with attributes that 
 was already tested and working but after release people argued 
 about changing its syntax from [attributes] to  (attributes).

 developers can have their own experimental branches for their 
 own tests and new ideas, but once a feature reaches pre-release 
 it should already be tested and working and ready for public 
 consumption without commiting to a final API.


OK, but who has already tested it and how many people have been 
able to test it and comment on it? I was thinking more along the 
lines of how Debian does it, with a 4 staged release process: 
experimental => unstable => testing (pre-release) => stable.

We could probably do away with a  common "experimental" branch, 
leaving 3 common branches.

A common "unstable branch" will allow more people to test out 
newly introduced ideas well before they become merged into 
pre-release, and that may help avoid problems as we're seeing 
with the sudden introduction of UDA's.

I am hoping that we can have a pre-release version that will be 
stable enough to be used on real-world applications, knowing that 
some things can change but not in a major way. If I don't mind 
dealing with major changes, then I would instead make use of 
what's in the "unstable" branch, but I expect mostly compiler 
devs will be using unstable, and no one will be using it on 
anything important.


 I really don't care about the numbering scheme and this is 
 irrelevant to the topic of this discussion. We are discussing 
 the PROCESS of development. How the releases are tagged is 
 completely beside the point and could be named after sweet 
 delights, cat names, Parks or even digits of PI. I really don't 
 care and it really is _not important_.
 This is one of those fundamental things that are required to 
 truly understand git - versions are the _content_ (code) they 
 contain and are identified by a hash of that content.
 This is pure bikesheding but why not: Why not extend the 
 astronomical theme to releases as well? What would you say 
 about the latest Pluto release of DMD? ;)
 (Yeah, I know this is already used by the eclipse project..)


I'm very surprised by your comment on version numbers and I hope 
you are a significant minority holder of that view.

How version numbers are assigned and what the number represents 
has to be made a defined part of the release process otherwise 
you'll end up with a meaningless random number or symbol (or 
nothing) for each new release, and the only information it will 
supply to you is "here's a new release". I don't see that as an 
improvement to the current situation, it just perpetuates the 
existing problem where users of the D compiler have no easy way 
to know if a new release is a major ground breaking one, or a 
minor one, or just a simple bug fix. In order to find out what a 
new release means in terms of the magnitude of changes, one has 
to read through a technical change log, and read through endless 
forum posts, and possibly even take a look at the commit log. Why 
not do something better than that when it's so easy to do?

For example, if I see that the stable version has been updated, I 
want to quickly know if it's a bug fix update or something more 
significant.

For mission critical applications, I'll jump on bug fix releases 
quickly, but I'll stay clear of any major releases up until it 
has had enough time to fully stabilize. I do this kind of 
filtering for all new releases of software that I use, and it's 
very easy to do when there's a version number that has meaning to 
it.

--rt
Dec 11 2012
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 12, 2012 at 02:03:56AM +0100, Rob T wrote:
[...]

 OK, but who has already tested it and how many people have been able
 to test it and comment on it? I was thinking more along the lines of
 how Debian does it, with a 4 staged release process: experimental =>
 unstable => testing (pre-release) => stable.


FYI, this is a misrepresentation of how Debian works. The actual process
is unstable => testing => stable. I.e., most new packages are uploaded
directly to unstable.

The experimental branch is for packages that aren't fully working, may
break your packaging system, contain untested code, etc.. Some stuff in
experimental is never migrated to unstable, because they're for, um,
experimenting with volatile or incomplete projects deemed unsuitable for
general use.

To apply the analogy to D development, this would be local git branches
where, say, Walter is toying around with incomplete code for a new
feature which may not even be compilable yet. The unstable branch would
be more-or-less complete, compilable code that haven't been extensively
tested yet (it more-or-less works, but may crash horribly, use at your
own risk, etc.). After a period of limited use, it gets merged into
testing, where a larger audience will begin using it. After it's been
proven in the field, then only it gets put into stable.

The experimental branch is really not needed as a formal part of the
process, since the developers can just use temporary local git branches
for that purpose without needing to push it to a public repository. So
really, the Debian process isn't really that different from what you
propose below:



 We could probably do away with a  common "experimental" branch,
 leaving 3 common branches.


[...]


T

-- 
Microsoft is to operating systems & security ... what McDonalds is to gourmet
cooking.
Dec 11 2012
prev sibling parent reply "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 01:03:59 UTC, Rob T wrote:

 On Tuesday, 11 December 2012 at 23:15:27 UTC, foobar wrote:

 By support I meant specifically _bug fixes_. You can already 
 download all previous released versions from the website and 
 in no way am I arguing to change that policy.
 Even if we ever get to a point where we don't want to keep 
 older releases which I doubt very much (each zip is only a few 
 MBs and therefore insignificant on today's storage) - we could 
 still easily just checkout the specific release tagged in git 
 and just build that.


 That's what I meant too, I agree with you on this point.


 This is precisely what I addressed bellow. we have monthly 
 build of our staging branch - call them monthly betas that 
 include new upcoming features that are already stable enough 
 to be released to the public for field testing and are 
 tentative for the next actual release but until these feature 
 actually get to a release they hold no guaranties and can be 
 further modified based on the wider public testing - including 
 changes in API. Once released, they do hold such guaranties of 
 API stability. So these monthly betas will provide preview of 
 language changes and allow people to prepare for future 
 changes and also provide feedback.


 Having a pre-release build that can be downloaded and installed 
 would be very nice to have if that's what you mean. Currently, 
 to get DMD 2.061 "alpha" I have to jump through hoops to get it 
 to a point where it is operational, and that is not so good as 
 it limits severely how many users will be able to test it out.


 See comment above. the pre-release will contain new features 
 already stable enough to be consumes by the general public 
 _before_ we the developers are ready to a commit finally to 
 their API. E.g. Walter's release of DMD with attributes that 
 was already tested and working but after release people argued 
 about changing its syntax from [attributes] to  (attributes).

 developers can have their own experimental branches for their 
 own tests and new ideas, but once a feature reaches 
 pre-release it should already be tested and working and ready 
 for public consumption without commiting to a final API.


 OK, but who has already tested it and how many people have been 
 able to test it and comment on it? I was thinking more along 
 the lines of how Debian does it, with a 4 staged release 
 process: experimental => unstable => testing (pre-release) => 
 stable.

 We could probably do away with a  common "experimental" branch, 
 leaving 3 common branches.

 A common "unstable branch" will allow more people to test out 
 newly introduced ideas well before they become merged into 
 pre-release, and that may help avoid problems as we're seeing 
 with the sudden introduction of UDA's.

 I am hoping that we can have a pre-release version that will be 
 stable enough to be used on real-world applications, knowing 
 that some things can change but not in a major way. If I don't 
 mind dealing with major changes, then I would instead make use 
 of what's in the "unstable" branch, but I expect mostly 
 compiler devs will be using unstable, and no one will be using 
 it on anything important.


 I really don't care about the numbering scheme and this is 
 irrelevant to the topic of this discussion. We are discussing 
 the PROCESS of development. How the releases are tagged is 
 completely beside the point and could be named after sweet 
 delights, cat names, Parks or even digits of PI. I really 
 don't care and it really is _not important_.
 This is one of those fundamental things that are required to 
 truly understand git - versions are the _content_ (code) they 
 contain and are identified by a hash of that content.
 This is pure bikesheding but why not: Why not extend the 
 astronomical theme to releases as well? What would you say 
 about the latest Pluto release of DMD? ;)
 (Yeah, I know this is already used by the eclipse project..)


 I'm very surprised by your comment on version numbers and I 
 hope you are a significant minority holder of that view.

 How version numbers are assigned and what the number represents 
 has to be made a defined part of the release process otherwise 
 you'll end up with a meaningless random number or symbol (or 
 nothing) for each new release, and the only information it will 
 supply to you is "here's a new release". I don't see that as an 
 improvement to the current situation, it just perpetuates the 
 existing problem where users of the D compiler have no easy way 
 to know if a new release is a major ground breaking one, or a 
 minor one, or just a simple bug fix. In order to find out what 
 a new release means in terms of the magnitude of changes, one 
 has to read through a technical change log, and read through 
 endless forum posts, and possibly even take a look at the 
 commit log. Why not do something better than that when it's so 
 easy to do?

 For example, if I see that the stable version has been updated, 
 I want to quickly know if it's a bug fix update or something 
 more significant.

 For mission critical applications, I'll jump on bug fix 
 releases quickly, but I'll stay clear of any major releases up 
 until it has had enough time to fully stabilize. I do this kind 
 of filtering for all new releases of software that I use, and 
 it's very easy to do when there's a version number that has 
 meaning to it.

 --rt


To summarize:
1. Yes, I do suggest that "beta" versions will be available to 
download and prominently displayed on the download web-page.
2. The version scheme is meaningless. Please check out 
http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
example. It's perfectly clear, you can choose what Mozilla calls 
a channel - i.e "release" or "beta".
We can put both in a two column table with high-level description 
of changes and links to more-detailed change logs.
3. Debian is a HUGE project with lots of manpower. D is not. I 
think it is enough to have two _publicly visible download 
channels_ - release and beta (testing). Those are the only two 
that should be formally defined. There is no need to define prior 
stages of development as people can just informally create their 
own local branches for their own experiments and they could also 
push them upstream to share with other developers without any 
requirement to list an executable on the official download page. 
The source code is available on github - if you wanted to you 
could pull whatever branch and build it yourself at your own risk.
4. The only official executables on the download page are for two 
channels - release and beta. You can grab the latest stable 
"release" that only gets critical bug-fixes or the monthly beta 
that also contains preview features such as the upcoming 
user-defined attributes. You can also grab previous versions of 
both release channels by clicking the "previous versions" link. 
Anything else has zero guaranties and you are required to build 
yourself from source.
Dec 12 2012
next sibling parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:

 To summarize:
 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".


I agree Firefox has a meaningless version numbering system 
because Firefox adopted a meaningless version numbering system. I 
don't understand the rational behind the decision to switch from 
the previous one that was meaningful.

What advantage do you gain by dropping a major, minor, and 
revision from your version numbering system?


 We can put both in a two column table with high-level 
 description of changes and links to more-detailed change logs.


There's no harm in doing that, but what would be most useful is 
knowing if the change is a major one or a minor one or just for 
bug fixes.


 3. Debian is a HUGE project with lots of manpower. D is not.


We all know this, but that does not mean we cannot do a lot 
better than we're doing now, and if we can get rid of the waste, 
like those endless debates that cannot be resolved with one 
branch, then there'll effectively be more manpower available.


 I think it is enough to have two _publicly visible download 
 channels_ - release and beta (testing). Those are the only two 
 that should be formally defined. There is no need to define 
 prior stages of development as people can just informally 
 create their own local branches for their own experiments and 
 they could also push them upstream to share with other 
 developers without any requirement to list an executable on the 
 official download page. The source code is available on github 
 - if you wanted to you could pull whatever branch and build it 
 yourself at your own risk.


But that's almost what we have now, and it does not work and it 
never will work.

The problem with only two official branches, is that one has to 
be for a stable version, and the other has to be for the "new 
stuff", but there's two kinds of "new stuff". One kind is for 
breaking changes, or for introducing new concepts that have not 
been well tested. and the other kind is for field testing new 
concepts that are reasonably well understood and cause no 
breaking changes.

What real-world programmers want to have, is a reasonably stable 
"beta" version that they can use for real work - this is NOT for 
playing around with, but for making things work for real. They 
want the beta because it has something that the stable does not 
yet have.

What these guys don't want, and cannot put up with for real world 
programming, is an unstable alpha, but that's what you are 
proposing we continue on with, which is exactly what we have now 
and it does not work.

In addition, once the beta/alpha is close to ready to be released 
into stable, it has to be frozen for a few months, where no new 
features are added and only bug fixes continue on, but then 
you've just lost your unstable branch!

Have a look at the thread on the introduction of UDA's to see 
what's going on, and ask yourself if a two branch system will 
solve that kind of never ending problem.


 4. The only official executables on the download page are for 
 two channels - release and beta. You can grab the latest stable 
 "release" that only gets critical bug-fixes or the monthly beta 
 that also contains preview features such as the upcoming 
 user-defined attributes. You can also grab previous versions of 
 both release channels by clicking the "previous versions" link. 
 Anything else has zero guaranties and you are required to build 
 yourself from source.


That all works fine. What I think is glaringly missing is the 
unstable branch, so we need at a minimum 3 branches.

--rt
Dec 12 2012
parent "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 18:25:10 UTC, Rob T wrote:

 On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:

 To summarize:
 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".


 I agree Firefox has a meaningless version numbering system 
 because Firefox adopted a meaningless version numbering system. 
 I don't understand the rational behind the decision to switch 
 from the previous one that was meaningful.

 What advantage do you gain by dropping a major, minor, and 
 revision from your version numbering system?


 We can put both in a two column table with high-level 
 description of changes and links to more-detailed change logs.


 There's no harm in doing that, but what would be most useful is 
 knowing if the change is a major one or a minor one or just for 
 bug fixes.


 3. Debian is a HUGE project with lots of manpower. D is not.


 We all know this, but that does not mean we cannot do a lot 
 better than we're doing now, and if we can get rid of the 
 waste, like those endless debates that cannot be resolved with 
 one branch, then there'll effectively be more manpower 
 available.


 I think it is enough to have two _publicly visible download 
 channels_ - release and beta (testing). Those are the only two 
 that should be formally defined. There is no need to define 
 prior stages of development as people can just informally 
 create their own local branches for their own experiments and 
 they could also push them upstream to share with other 
 developers without any requirement to list an executable on 
 the official download page. The source code is available on 
 github - if you wanted to you could pull whatever branch and 
 build it yourself at your own risk.


 But that's almost what we have now, and it does not work and it 
 never will work.

 The problem with only two official branches, is that one has to 
 be for a stable version, and the other has to be for the "new 
 stuff", but there's two kinds of "new stuff". One kind is for 
 breaking changes, or for introducing new concepts that have not 
 been well tested. and the other kind is for field testing new 
 concepts that are reasonably well understood and cause no 
 breaking changes.

 What real-world programmers want to have, is a reasonably 
 stable "beta" version that they can use for real work - this is 
 NOT for playing around with, but for making things work for 
 real. They want the beta because it has something that the 
 stable does not yet have.

 What these guys don't want, and cannot put up with for real 
 world programming, is an unstable alpha, but that's what you 
 are proposing we continue on with, which is exactly what we 
 have now and it does not work.

 In addition, once the beta/alpha is close to ready to be 
 released into stable, it has to be frozen for a few months, 
 where no new features are added and only bug fixes continue on, 
 but then you've just lost your unstable branch!

 Have a look at the thread on the introduction of UDA's to see 
 what's going on, and ask yourself if a two branch system will 
 solve that kind of never ending problem.


 4. The only official executables on the download page are for 
 two channels - release and beta. You can grab the latest 
 stable "release" that only gets critical bug-fixes or the 
 monthly beta that also contains preview features such as the 
 upcoming user-defined attributes. You can also grab previous 
 versions of both release channels by clicking the "previous 
 versions" link. Anything else has zero guaranties and you are 
 required to build yourself from source.


 That all works fine. What I think is glaringly missing is the 
 unstable branch, so we need at a minimum 3 branches.

 --rt


I really didn't want to get dragged into the versioning scheme 
debate as it is just bikesheding in disguise but oh well.. I'll 
ask in return, what's the added benefit of your scheme? I 
honestly see no added benefit whatsoever.
There's no intrinsic benefit compared to a changelog that lists 
"Jelly bean" after "Ice cream sandwich". How do you know what was 
added to a version? Simple, you have a "what's new" page that 
lists new major features denoted by "New feature:" tag and 
bug-fixes denoted by "Bug-Fix:" tag.
Here's an example - 
http://www.mozilla.org/en-US/firefox/17.0.1/releasenotes/

Regarding the important (IMO) part of the discussion - 2 vs. 3 
levels - I feel I haven't explained myself properly so let me 
clarify:
I was taking into account what I thought was a reasonable 
assumption that D2 is in the process of stabilization and no 
major redesigns are planed. This means the "Major" version is 2 
as in D2.
My stable releases are meant to be the "Minor" releases on the D2 
branch. These releases can introduce new backwards-compatible 
features and restricted breaking changes when absolutely required 
_over several releases_ with _a well defined migration(*) 
process_ - in other words support evolution.
Major redesigns belong under a "revolution" such as the python3 
effort. There is no point on setting time-lines for that. If and 
when we accumulate enough reasons/momentum/etc for such a major 
break we'll just create a new "major" version of D called D3 
which right now is completely off the table.

This is _not_ exactly what we have now because now new features 
are introduced right away, without public field testing (the beta 
channel). We have no defined criteria as to when a feature is 
mature enough to have it's syntax/API frozen and should be 
graduated from beta phase and integrated into stable. That is the 
point of defining the process.

(*) A well defined migration path should span several stable 
versions and define these stages:
1. Warn the user that a feature is going away/ is going to be 
changed
2. Deprecate the feature and don't accept it at compilation 
(unless -d is used)
3. remove the feature completely.
Dec 12 2012
prev sibling parent reply "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:

 To summarize:



 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".


This is a poor example as it doesn't show how the development 
team develops these versions.

If we are going to have a branch supported separate from the 
beta/dev builds then we need a way to say that this stable 
version is newer but is not as new as the beta/dev.

If we release 2.61 as a stable, we would then develop new 
features in which version? 2.62 beta 1? If so, when we release 
2.61 with a bug fix which version do we release? 2.62? 2.61 
stable 2?

You are right that version numbers have absolutely no meaning, I 
don't remember if it was you, but they are also not as important 
as the process improvements. However if we assign meaning to 
numbers there can be benefit. Mozilla basically just did away 
with the Major version. (I say we do a Linux and leave the 2 then 
realize it has no meaning and up it to 3 when we reach the age of 
the 2.0 kernel)

We should combine your beta with an added version number.

2.61.0 => bugfixes => 2.61.1
2.61.0 => newfeatures => 2.62.0 beta 1
2.62 => preparing for stabilizing => 2.62 rc1

just some thoughts.
Dec 12 2012
next sibling parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 18:33:17 UTC, Jesse Phillips 
wrote:

 If we release 2.61 as a stable, we would then develop new 
 features in which version? 2.62 beta 1? If so, when we release 
 2.61 with a bug fix which version do we release? 2.62? 2.61 
 stable 2?

 You are right that version numbers have absolutely no meaning, 
 I don't remember if it was you, but they are also not as 
 important as the process improvements. However if we assign 
 meaning to numbers there can be benefit. Mozilla basically just 
 did away with the Major version. (I say we do a Linux and leave 
 the 2 then realize it has no meaning and up it to 3 when we 
 reach the age of the 2.0 kernel)

 We should combine your beta with an added version number.


The beta foobar speaks of is actually a combined alpha+beta, and 
that'll cause lots of problems and also delay releases because 
the combined alpha+beta will have to be frozen at some point 
prior to release as the next stable (i.e., frozen from 
enhancements that are not bug fixes only). Freezing the 
alpha+beta combo is not a problem for the beta portion, but it is 
a problem for the alpha portion that has no requirement to be 
frozen. You could just branch off the beta from the alpha into a 
special frozen branch, but then what are you doing? It makes no 
sense. Also important is that fact that there will be many 
programmers who could not wait for the next stable release and 
want to use the latest beta, but there is none, there's only and 
alpha+beta combo, and that'll mean the current problem we have 
right now will not be resolved where newly introduced features 
mess up the beta portion that people want to be able to rely on 
for real-world programming. The separation of alpha and beta 
allows for a beta that is at least far more stable than an 
alpha-beta combo, and that will satisfy the people who wish to 
move past the stable release into a beta, but wish to avoid the 
alpha entirely.


 2.61.0 => bugfixes => 2.61.1
 2.61.0 => newfeatures => 2.62.0 beta 1
 2.62 => preparing for stabilizing => 2.62 rc1

 just some thoughts.


A 3 point numbering system used to differentiate between the 
stable bug fix releases and the beta version works just fine, and 
there's no need to introduce descriptive symbols, although that 
can be useful to indicate what is stable, beta or alpha.

In any event, I fully agree that the process is the most 
important item to work out first and foremost. When I mentioned 
version numbers, I simply wanted to remind people that it should 
also be on the table for change AND should be included into the 
process at some point to make it a well understood concept as 
opposed to what we have now. The response I got from Foobar was 
very surprising to me which is why I explored it further.

As was suggested in another post, I'll do a little digging to see 
what is going on with the development cycle of other programming 
languages. Best not to try and reinvent the wheel if we can. One 
thing about D is that I suspect there's a sizable faction that 
does not wish to do what other programming languages are doing, 
and that's to avoid introducing breaking changes completely. 
Personally I'd rather see flawed concepts get fixed at some point 
rather than have to live with them forever, the point being that 
we may be somewhat unique with our development process 
requirements.

--rt
Dec 12 2012
next sibling parent reply "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 19:28:54 UTC, Rob T wrote:


 2.61.0 => bugfixes => 2.61.1
 2.61.0 => newfeatures => 2.62.0 beta 1
 2.62 => preparing for stabilizing => 2.62 rc1

 just some thoughts.


 A 3 point numbering system used to differentiate between the 
 stable bug fix releases and the beta version works just fine, 
 and there's no need to introduce descriptive symbols, although 
 that can be useful to indicate what is stable, beta or alpha.


So I say let us stay away from beta/alpha because those mean 
something different for different people and we need to define 
what they mean in D.

1. Bug fix only (how important is backwards compatibility here 
[when derived from a bug])
2. Backward compatible additions
3. Vetting

This seems to be the main releases that are being talk about. For 
which you are probably talking about the ability to use a 3 point 
system.

The mapping I think people envision from this

1 & 2 => stable release
3 => dev release

[side rant]
When doing version numbering there are two main approaches I've 
seen used, maybe this has a formal name, but I'll call them front 
loaded and back loaded.

Front loaded version generally comes from having goals/milestones 
for a version. In this case all the work for that version comes 
before the release. This is generally seen pre-1.0 as that will 
be the stable release.

Back loaded is what generally comes from deciding on a new 
direction and where D currently is. In this case all the work is 
done within the version. This is generally seen in 2.0, software 
is released with the 2.xx stamp but is not fulfilling all the 
requirements needed to meet being the second version.
[/side rant]

I'd like to see what you envision for mapping these versions.

vetting.additions.bug?

Do we release a stable then increment vetting and develop under 
that (back loaded)? Or

stable.vetting.additions/bug

Do we increment stable upon release and continue increasing 
vetting as we release those for the volatile changes. (front 
loaded)
Dec 12 2012
parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
This is how I see the two approaches without wrecking havoc on 
current version numbers. I'll use [notes] to reference to the "A 
successful Git branching model" article.

---------------front loaded (2.0 still back loaded)
stable
2.61   // always release whole number as stable
2.61.1 [Seen in the master branch coming from hotfix branch]
2.61.2
dev
2.62  // develop on whole number
2.63 [not shown they only do a single stable release]
2.64 [these releases are inclusive of hotfix, feature, and 
release branch]
2.65 [first green circle]
2.66
2.67
rc
2.65rc1 // beat release into submission
2.65rc2 [this is the release branch]
----------------back loaded
stable
2.61.7 // always release in development
2.61.8 // continue work after release
2.61.9
dev
2.62 // always start development on whole
2.62.1
2.62.2
2.62.3
2.63
2.63.1
rc
2.62.3rc1 // beat release into submission
2.62.4rc2 // maybe change dev number
Dec 12 2012
prev sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 19:28:54 UTC, Rob T wrote:


 The beta foobar speaks of is actually a combined alpha+beta, 
 --rt


No. I have already addressed this point before.
Alpha is not part of the _official_ release process.

As a developer I can create my own local branch "DMD-new_feature" 
to experiment with some new idea I had. I can later on push this 
upstream and share my new cool feature with other developers to 
play with. besides other developers, users can also fetch these 
new branches and build themselves. This is done _before_ it 
matures enough to be included in beta testing.
Dec 12 2012
prev sibling parent reply "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 18:33:17 UTC, Jesse Phillips 
wrote:

 On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:

 To summarize:



 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".


 This is a poor example as it doesn't show how the development 
 team develops these versions.

 If we are going to have a branch supported separate from the 
 beta/dev builds then we need a way to say that this stable 
 version is newer but is not as new as the beta/dev.

 If we release 2.61 as a stable, we would then develop new 
 features in which version? 2.62 beta 1? If so, when we release 
 2.61 with a bug fix which version do we release? 2.62? 2.61 
 stable 2?

 You are right that version numbers have absolutely no meaning, 
 I don't remember if it was you, but they are also not as 
 important as the process improvements. However if we assign 
 meaning to numbers there can be benefit. Mozilla basically just 
 did away with the Major version. (I say we do a Linux and leave 
 the 2 then realize it has no meaning and up it to 3 when we 
 reach the age of the 2.0 kernel)

 We should combine your beta with an added version number.

 2.61.0 => bugfixes => 2.61.1
 2.61.0 => newfeatures => 2.62.0 beta 1
 2.62 => preparing for stabilizing => 2.62 rc1

 just some thoughts.


Per my answer to Rob:
D2 *is* the major version.
releases should be minor versions and largely backwards 
compatible - some evolution is allowed given some reasonable 
restrictions like a proper migration path over several releases.
Critical bug-fixes can go directly on stable releases but nothing 
else.

Other enhancements, new features, etc, go first on monthly beta 
versions and need to pass acceptance tests & community approval 
before graduating to stable. So a big feature such as "User 
defined attributes" can spend several cycles of beta testing and 
can go redesigns until everyone agrees that the final design is 
worthy of inclusion in the next stable release.
Dec 12 2012
parent reply "Rob T" <rob ucora.com> writes:
On Thursday, 13 December 2012 at 07:18:16 UTC, foobar wrote:

 Per my answer to Rob:
 D2 *is* the major version.
 releases should be minor versions and largely backwards 
 compatible - some evolution is allowed given some reasonable 
 restrictions like a proper migration path over several releases.
 Critical bug-fixes can go directly on stable releases but 
 nothing else.


I just realized that we are mixing together the version numbers 
for two entirely different things.

As you have correctly pointed out, one of the version numbers is 
for the D language specification, which is version 2.something, 
the other version is for the compiler releases, which has the 
major language specification version assigned to identify it as 
supporting version 2.something of the D language. The remaining 
numbers indicate incremental releases which may also roughly 
correspond to the minor evolutionary changes to the language 
specification, i.e., does DMD 2.061 also mean D specification 
2.061?

I think it makes good sense that the "2" is used to indicate that 
the compiler supports D major version 2 of the specification, but 
we should all be clear that is all that it represents. Therefore 
DMD 3.x will never appear unless there's a D language ver 3 
specification for it to support.

A completely separate issue that should be dealt with, is that 
the language specification's version number is not indicated 
anywhere that I could see. We just assume it's version 
2.something and we have no idea where the "something" part is 
currently at or what has changed since ver 2.0. This is no good 
because it means that we cannot indicate in the compiler release 
change log what minor version of the 2.x specification the 
compiler is actually supporting.

--rt
Dec 15 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 03:59:33 UTC, Rob T wrote:

 On Thursday, 13 December 2012 at 07:18:16 UTC, foobar wrote:

 Per my answer to Rob:
 D2 *is* the major version.
 releases should be minor versions and largely backwards 
 compatible - some evolution is allowed given some reasonable 
 restrictions like a proper migration path over several 
 releases.
 Critical bug-fixes can go directly on stable releases but 
 nothing else.


 I just realized that we are mixing together the version numbers 
 for two entirely different things.

 As you have correctly pointed out, one of the version numbers 
 is for the D language specification, which is version 
 2.something, the other version is for the compiler releases, 
 which has the major language specification version assigned to 
 identify it as supporting version 2.something of the D 
 language. The remaining numbers indicate incremental releases 
 which may also roughly correspond to the minor evolutionary 
 changes to the language specification, i.e., does DMD 2.061 
 also mean D specification 2.061?


Yes it does. If new features are introduced, then this is by 
definition a change in the specs.


 I think it makes good sense that the "2" is used to indicate 
 that the compiler supports D major version 2 of the 
 specification, but we should all be clear that is all that it 
 represents. Therefore DMD 3.x will never appear unless there's 
 a D language ver 3 specification for it to support.


A version 3 would means introducing change in the specification 
that may break a lot of code. This is type of stuff you shouldn't 
do often.


 A completely separate issue that should be dealt with, is that 
 the language specification's version number is not indicated 
 anywhere that I could see. We just assume it's version 
 2.something and we have no idea where the "something" part is 
 currently at or what has changed since ver 2.0. This is no good 
 because it means that we cannot indicate in the compiler 
 release change log what minor version of the 2.x specification 
 the compiler is actually supporting.


The compiler source code is probably what we have that look like 
the most as a spec right now.
Dec 15 2012
parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 07:39:09 UTC, deadalnix wrote:

 The compiler source code is probably what we have that look 
 like the most as a spec right now.


Unfortunately that is likely the case. I hope we can all agree 
that the specification should be managed better, but it's a much 
lesser problem at the moment than what we're trying to solve in 
here.

I think once we can establish a process for development testing 
and release, the other problem areas will stand out a lot more 
clearly for elimination.

--rt
Dec 16 2012
prev sibling parent "Marco Nembrini" <marco.nembrini.co gmail.com> writes:
 I really don't care about the numbering scheme and this is 
 irrelevant to the topic of this discussion. We are discussing 
 the PROCESS of development. How the releases are tagged is 
 completely beside the point and could be named after sweet 
 delights, cat names, Parks or even digits of PI. I really don't 
 care and it really is _not important_.
 


I agree that the numbering scheme should be chosen after the 
process is defined and not before, and numbering schemes are 
meaningless (like firefox's) _unless_ they represent some 
guarantee that the developer is willing to make. In that case the 
can be very effective imho (see http://semver.org/ for an 
example).

So if the process chosen makes guarantees about the project, 
those should be expressed by the version number. For example 
ubuntu's Server LTS guarantees 5 years of support, and the 
numbering scheme is year.month, so you know right away that 
ubuntu 8.04 is still supported right now. They could have named 
it Ubuntu Pluto and you could have looked up the end date for the 
support, but since you have to differentiate your releases 
anyway, you might as well use meaningful numbers.
Dec 12 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:

 (One piece that has been brought forward is 
 http://nvie.com/posts/a-successful-git-branching-model/ - 
 something to keep in mind.)


In this workflow, only one version of the software is produced. 
But we have an extra need : the one to combine language evolution 
(that may break) and long term support. This require some 
adjustment.

Here is how I see things :

New feature are develloped in their own branches.
When a feature is ready, it is included in the dev branch. The 
feature must be rebased on the dev branch to be included.
 From time to time, dev branch is forked. The forked branch only 
receive bug fixes. The second digit of the version is incremented 
(minor). I'll call it stabilization branch.
When the fork branch is stable enough, a tag is made and we have 
a realease. The revision number is incremented.
In the future, more bug fixes can be done in that branch and new 
tag created for new release.

No let's focus on bug fixes. As feature, they are made in their 
own branches. The branch will then be merged in any stabilization 
branch where it make sense and in dev.

master is the highest numbered stabilization branch. By doing so, 
we ensure that user that get sources have a preview of the next 
thing that will be shipped.

I would advice against maintaining a lot of stabilization 
branches as this is more work and too much of them don't bring 
any benefit. 2 to 3 of them is the right amount.

The proposal works well with github, which is a plus considering 
we are using it.
Dec 10 2012
prev sibling next sibling parent Matt Soucy <msoucy csh.rit.edu> writes:
On 12/10/2012 06:41 PM, Andrei Alexandrescu wrote:

 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple git
 scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how to
 start a new release, fix a bug, experiment with a new feature, and such.


One thing that I don't think I've seen mentioned: The person who started 
that git-flow concept created a git plugin to do exactly this for that 
workflow: https://github.com/nvie/gitflow

-Matt
Dec 10 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:

 Please chime in with ideas and thoughts.


 Thanks,

 Andrei


Andrei, as you know, a lot has already been discussed here:
http://forum.dlang.org/thread/k99coi$1q5e$1 digitalmars.com

In summary, here's where I understood things were left at ...

Instead of all of our ideas getting mixed around in multiple 
threads and eventually vanishing into the black hole, we had 
decided that we needed a fixed location to discuss and work on 
the issue, and a place where documents that describe the process 
can be developed and stored.

The poster named "1100110" started a git repo here
https://github.com/D-Programming-Language-Stable/dmd/wiki/Specification

I'm not sure it should be called "D-Programming-Language-Stable", 
it should be named something like 
"D-Programming-Language-Dev-Process", and it should only be 
concerned with developing the new development and release process 
for D.

--rt
Dec 10 2012
prev sibling next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:

 In turn, I'll be collecting thoughts and opinions in this 
 thread, distilled from previous discussions. We should develop 
 a few simple git scripts (such as git-start-new-feature or 
 git-start-bugfix etc) supported by an equally simple piece of 
 documentation describing how to start a new release, fix a bug, 
 experiment with a new feature, and such.


There may be some helpful scripts to facilitate a new workflow, 
but the examples given I feel just translate to learning a D 
specific git commands. Seriously you'll end up with a layer of 
abstraction that will make it harder for those familiar with git 
to understand and limit the ability of newer users to understand 
the state their working copy is in.

git-start-new-feature:
$ git checkout devbranch
$ git checkout -b newfeaturename

git-start-bugfix:
$ git checkout stablebranch
$ git checkout -b fixbugxxx

How these should be folded back is the harder part. For example 
the bug fix should then be into stablebranch and devbranch.

There is also the matter of pulling in changes of your stable or 
dev branch as you work, this is done with the add --rebase switch 
(and I believe opens the doors for more merge conflicts than a 
standard merge).


 (One piece that has been brought forward is 
 http://nvie.com/posts/a-successful-git-branching-model/ - 
 something to keep in mind.)


I would like to note that these simple flows do not mimic our 
world. We have three projects that need to be marked for release 
in tandem. It doesn't need address now, but it should be noted.

I think it would be cool if the idea behind workbench project 
someone started was used officially to sync dmd/phobos/dlang.org 
and provide any of these needed deployment scripts. It would add 
to the complexity but much of it is already there this would just 
make it more apparent.
Dec 11 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/12/12 1:37 AM, Jesse Phillips wrote:

 There may be some helpful scripts to facilitate a new workflow, but the
 examples given I feel just translate to learning a D specific git
 commands. Seriously you'll end up with a layer of abstraction that will
 make it harder for those familiar with git to understand and limit the
 ability of newer users to understand the state their working copy is in.


I disagree. I wrote a few git scripts for Facebook-specific workflows 
and a lot of people (including me) use them.

Beyond that we need to have a wiki page with workflows:

"To initiate a new feature"

"To initiate a bug fix"

"To initiate a library addition without formal review"

"To initiate a library addition with formal review"

...


 git-start-new-feature:
 $ git checkout devbranch
 $ git checkout -b newfeaturename

 git-start-bugfix:
 $ git checkout stablebranch
 $ git checkout -b fixbugxxx

 How these should be folded back is the harder part. For example the bug
 fix should then be into stablebranch and devbranch.


All of these will be part of the instructions and the macros. The branch 
names will be standardized.


 There is also the matter of pulling in changes of your stable or dev
 branch as you work, this is done with the add --rebase switch (and I
 believe opens the doors for more merge conflicts than a standard merge).


All of these should be subsections in the documentation.

"If your branch has conflicts"

"If your branch became really messed up"

...


 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something to
 keep in mind.)


 I would like to note that these simple flows do not mimic our world. We
 have three projects that need to be marked for release in tandem. It
 doesn't need address now, but it should be noted.

 I think it would be cool if the idea behind workbench project someone
 started was used officially to sync dmd/phobos/dlang.org and provide any
 of these needed deployment scripts. It would add to the complexity but
 much of it is already there this would just make it more apparent.


What is the workbench project?


Andrei
Dec 12 2012
next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 16:14:44 UTC, Andrei 
Alexandrescu wrote:

 I disagree. I wrote a few git scripts for Facebook-specific 
 workflows and a lot of people (including me) use them.


May have been a knee jerk reaction. Good to hear it has been used.


 git-start-new-feature:
 $ git checkout devbranch
 $ git checkout -b newfeaturename

 git-start-bugfix:
 $ git checkout stablebranch
 $ git checkout -b fixbugxxx





 All of these will be part of the instructions and the macros. 
 The branch names will be standardized.


I'm not sure I follow on the name standardizing. Doesn't that 
make it so a person can only work on one bugfix and one feature 
at a time? The name used for the dev and stable branches should 
be standardized (I don't care much what master is, but I 
generally like it as the dev branch)


 All of these should be subsections in the documentation.

 "If your branch has conflicts"

 "If your branch became really messed up"


Yep, just pointing out where I see the difficulty and scripts may 
be of more use.


 What is the workbench project?


 Andrei


Found it, by Nathan Swan

http://forum.dlang.org/post/ygwzndxbwzrxzlhuooty forum.dlang.org
Dec 12 2012
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
Let me do some proposal. If people are happy with them, I can 
start some wikification.

First here are the branch already existing :

dev : development branch. It is used to merge new features.
master : is the branch where the next version is stabilized 
(right now, 2.061). It receive only bug fix. eventually new 
features if the feature is of strategic importance.
2.60 : contains the current released version of D + bug fix.

Release process :

When branch 2.60 have enoug bug fixes to justify a new release, a 
tag is created (2.60.N+1). The amount and the importance of 
corrected bugs will dictate when the release is done?

When master is stabilized enough, a new version of D can be 
released. A new branch s created from master, named 2.61 and a 
tag 2.61.0 is created :

git checkout master
git branch 2.61
git checkout 2.61
git tag 2.61.0

Then, the dev version is merged into master and a new 
stabilization process begin :
git checkout master
git merge dev

At some point, 2.60 will not receive any more bug fix, and will 
be considered very stable, or dead, depending on one's standard 
(both are kind of the same).

I advice against maintaining a log of 2.XXX branches, as too much 
choice is as bad as no choice for the user, and it is no better 
for us. 2 is IMO the right amount.

NB: I have chosen the 3 number system for branches names, this is 
up to debate, but as it is widely used in programming languages 
it is a good base. Anyway, let's not debate the name for too 
long, as it really don't matter, and concentrate on the workflow.

On Wednesday, 12 December 2012 at 16:14:44 UTC, Andrei 
Alexandrescu wrote:

 Beyond that we need to have a wiki page with workflows:

 "To initiate a new feature"


A new feature is developed in its own branch. The reference is 
the dev branch. In git it goes as follow :

git checkout dev
git branch featureName

Development of the feature.

You can work on many feature at the same time; each one in its 
own branch. You initiate them the same way and switch from one 
another with git checkout.

Before submitting a feature, it has to be reabsed as follow :

git checkout featureName
git rebase dev

When the feature is ready to inclusion into dev, the dev can make 
a pull request.


 "To initiate a bug fix"


The same as for new feature, except that you start from the 
branch the bug is in. If the bug is in several branches, then the 
most stable one is choosen, as i is usually easier to bubble up 
the bugfix from a stable branch to dev than the other way around.


 "To initiate a library addition without formal review"

 "To initiate a library addition with formal review"

 ...


I don't really know what to do here. I guess bug fixes on the 
library can be done the same way, but something specific probably 
have to be done with new modules.


 All of these should be subsections in the documentation.

 "If your branch has conflicts"


If your branch has conflict, it is up to you to resolve them. The 
conflict will most likely occurs when rebasing. In such case, git 
status will inform you on which files are impacted. Edit the file 
to resolve the conflict and do git add file, then git rebase 
--continue .

If you don't want to resolve the conflict now, you can abort the 
rebase by doing git rebase --abort


 "If your branch became really messed up"


Well this one is difficult, because it is unclear what is wrong, 
so unclear ho to solve it.

Is that good ?
Dec 12 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/13/2012 12:13 AM, deadalnix пОшет:

 Let me do some proposal. If people are happy with them, I can start some
 wikification.

 First here are the branch already existing :

 dev : development branch. It is used to merge new features.
 master : is the branch where the next version is stabilized (right now,
 2.061). It receive only bug fix. eventually new features if the feature
 is of strategic importance.
 2.60 : contains the current released version of D + bug fix.

 Release process :

 When branch 2.60 have enoug bug fixes to justify a new release, a tag is
 created (2.60.N+1). The amount and the importance of corrected bugs will
 dictate when the release is done?

 When master is stabilized enough, a new version of D can be released. A
 new branch s created from master, named 2.61 and a tag 2.61.0 is created :

 git checkout master
 git branch 2.61
 git checkout 2.61
 git tag 2.61.0

 Then, the dev version is merged into master and a new stabilization
 process begin :
 git checkout master
 git merge dev

 At some point, 2.60 will not receive any more bug fix, and will be
 considered very stable, or dead, depending on one's standard (both are
 kind of the same).

 I advice against maintaining a log of 2.XXX branches, as too much choice
 is as bad as no choice for the user, and it is no better for us. 2 is
 IMO the right amount.

 NB: I have chosen the 3 number system for branches names, this is up to
 debate, but as it is widely used in programming languages it is a good
 base. Anyway, let's not debate the name for too long, as it really don't
 matter, and concentrate on the workflow.


So far me likes.

 On Wednesday, 12 December 2012 at 16:14:44 UTC, Andrei Alexandrescu wrote:

 Beyond that we need to have a wiki page with workflows:

 "To initiate a new feature"


 A new feature is developed in its own branch. The reference is the dev
 branch. In git it goes as follow :

 git checkout dev
 git branch featureName

 Development of the feature.

 You can work on many feature at the same time; each one in its own
 branch. You initiate them the same way and switch from one another with
 git checkout.

 Before submitting a feature, it has to be reabsed as follow :

 git checkout featureName
 git rebase dev


Just make sure your local dev is always in sync with upstream!

Or rather add an 'upstream' remote as 
github.com/D-Programming-Language/phobos.git (dmd, druntime etc.) and 
always do:

git pull --rebase upstream dev
git push --force origin featureName

before submitting pull via github.


 When the feature is ready to inclusion into dev, the dev can make a pull
 request.


 "To initiate a bug fix"


 The same as for new feature, except that you start from the branch the
 bug is in. If the bug is in several branches, then the most stable one
 is choosen, as i is usually easier to bubble up the bugfix from a stable
 branch to dev than the other way around.


 "To initiate a library addition without formal review"

 "To initiate a library addition with formal review"

 ...


 I don't really know what to do here. I guess bug fixes on the library
 can be done the same way, but something specific probably have to be
 done with new modules.


 All of these should be subsections in the documentation.

 "If your branch has conflicts"


 If your branch has conflict, it is up to you to resolve them. The
 conflict will most likely occurs when rebasing. In such case, git status
 will inform you on which files are impacted. Edit the file to resolve
 the conflict and do git add file, then git rebase --continue .

 If you don't want to resolve the conflict now, you can abort the rebase
 by doing git rebase --abort


 "If your branch became really messed up"


 Well this one is difficult, because it is unclear what is wrong, so
 unclear ho to solve it.


I find it to be 'git reset' to some early point, stash changes, fast 
forward from the upstream version to the head of current branch and 
stash pop your local changes.

But it really depends on symptoms and what the end result should be 
discard changes?, save changes elsewhere but restore to something 
stable? which branch is messed up? It's an infinite list of questions.


 Is that good ?


I think it's good.

But personally I'd expect:

* master to be what you define as dev, because e.g. GitHub puts master 
as default target branch when making pull requests. Yeah, I know it's 
their quirk that it's easy to miss. Still leaving less burden to check 
the branch label for both reviewers and pull authors is a net gain.

* And what you describe as master (it's a snapshot or a pre-release to 
me) to be named e.g. staging.

And we can as well drop the dead branch 'dev' then.
-- 
Dmitry Olshansky
Dec 13 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry Olshansky 
wrote:

 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub puts 
 master as default target branch when making pull requests. 
 Yeah, I know it's their quirk that it's easy to miss. Still 
 leaving less burden to check the branch label for both 
 reviewers and pull authors is a net gain.

 * And what you describe as master (it's a snapshot or a 
 pre-release to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.


That sound also like a good plan to me.
Dec 13 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:

 On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry Olshansky 
 wrote:

 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub 
 puts master as default target branch when making pull 
 requests. Yeah, I know it's their quirk that it's easy to 
 miss. Still leaving less burden to check the branch label for 
 both reviewers and pull authors is a net gain.

 * And what you describe as master (it's a snapshot or a 
 pre-release to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.


 That sound also like a good plan to me.


Updated to follow the idea, plus added bunch of process 
description. Feel free to comment in order to refine this.

http://wiki.dlang.org/Release_Process
Dec 13 2012
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 12:34:13AM +0100, deadalnix wrote:

 On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:


[...]

 Updated to follow the idea, plus added bunch of process description.
 Feel free to comment in order to refine this.
 
 http://wiki.dlang.org/Release_Process


I've rearranged the page a bit and added more higher-level discussion.
Please feel free to edit/correct any mistakes I may have had.


T

-- 
MSDOS = MicroSoft's Denial Of Service
Dec 13 2012
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/14/2012 3:34 AM, deadalnix пОшет:

 On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:

 On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry Olshansky wrote:

 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub puts
 master as default target branch when making pull requests. Yeah, I
 know it's their quirk that it's easy to miss. Still leaving less
 burden to check the branch label for both reviewers and pull authors
 is a net gain.

 * And what you describe as master (it's a snapshot or a pre-release
 to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.


 That sound also like a good plan to me.


 Updated to follow the idea, plus added bunch of process description.
 Feel free to comment in order to refine this.

 http://wiki.dlang.org/Release_Process


I wasn't comfortable doing speculative edits to the wiki directly so 
here a few more comments:

I think one of major goals is to be able to continue ongoing development 
while at the _same time_ preparing a release. To me number one problem 
is condensed in the statement "we are going to release do not merge 
anything but regressions" the process should sidestep this "lock-based" 
work-flow. Could be useful to add something along these line to goals 
section. (i.e. the speed and smoothness is a goal)

Second point is about merging master into staging - why not just rewrite 
it with master branch altogether after each release?
master is the branch with correct history (all new stuff is rebased on 
it) thus new staging will have that too.

-- 
Dmitry Olshansky
Dec 15 2012
next sibling parent "RenatoUtsch" <renatoutsch gmail.com> writes:
On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry Olshansky
wrote:

 12/14/2012 3:34 AM, deadalnix пОшет:

 On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:

 On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry 
 Olshansky wrote:

 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub 
 puts
 master as default target branch when making pull requests. 
 Yeah, I
 know it's their quirk that it's easy to miss. Still leaving 
 less
 burden to check the branch label for both reviewers and pull 
 authors
 is a net gain.

 * And what you describe as master (it's a snapshot or a 
 pre-release
 to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.


 That sound also like a good plan to me.


 Updated to follow the idea, plus added bunch of process 
 description.
 Feel free to comment in order to refine this.

 http://wiki.dlang.org/Release_Process


 I wasn't comfortable doing speculative edits to the wiki 
 directly so here a few more comments:

 I think one of major goals is to be able to continue ongoing 
 development while at the _same time_ preparing a release. To me 
 number one problem is condensed in the statement "we are going 
 to release do not merge anything but regressions" the process 
 should sidestep this "lock-based" work-flow. Could be useful to 
 add something along these line to goals section. (i.e. the 
 speed and smoothness is a goal)

 Second point is about merging master into staging - why not 
 just rewrite it with master branch altogether after each 
 release?
 master is the branch with correct history (all new stuff is 
 rebased on it) thus new staging will have that too.


Here what I proposed on the discussion page, what do you think?

------------------
I have come with a good idea for the release schedule. Say that
we are 3 or 4 monthes before the next LTS release. We need to
branch the staging branch in a 2.N (put the contents of the
staging branch in the 2.N branch) branch, where N is the new LTS
version. Then, no other features can be included in this 2.N
branch, only bugfixes are allowed. This period will have one RC
every month (?) with the latest fixes in the 2.N branch. After
the 3 or 4 monthes period we'll tag the 2.N.0 release. After
every 4~6 monthes, we'll release a new 2.N.x version with the
latest bugfixes in the 2.N branch, but with no additional
features (including non-breaking features here will just be more
work to the devs, I don't think it is a good idea). While these
bugfix releases are being made, every feature that stabilizes
enough in the master branch is merged into the staging branch,
where the feature shouldn't have much breaking changes (although
changes are still allowed, the branch is not frozen). Every 3
monthes, dev releases are made with these features that made
their way into the staging branch. This is done while the fixes
in the 2.N branch are made. Then, 4 monthes prior to the next LTS
release, a new 2.N+1 branch will be created with the staging
branch contents. The cycle will repeat, these 4 monthes will have
no additional features on the 2.N+1 branch, and neither on the
next 3 years. This organization, in dev and LTS releases, will
allow releases for the ones that want a stable environment to
develop (the LTS releases) and the ones that want the latest
great features from D will have a somewhat stable environment
(the dev releases, somewhat like the ones we have now, maybe a
little more stable) to use. On top of that, the staging branch
will never be frozen, so development will never stop, as someone
was saying on the forums that was a bad idea. And, when the new
LTS release is made (2 years after the older LTS), the older LTS
will be mantained for more one year, what means that each LTS
will be mantained for 3 years. What do you think? RenatoUtsch
(talk) 14:14, 15 December 2012 (CET)
------------------
http://wiki.dlang.org/Talk:Release_Process#Expanding_LTS
Dec 15 2012
prev sibling next sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:


 I think one of major goals is to be able to continue ongoing development while
at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are going to
release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could be useful to
add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)


I've been staying out of this thread for the most part, but I feel the need to
comment on this part specifically.  It's
quite common for most major projects to have a clear "we're wrapping up a
release" phase where work _is_ restricted to
bug fixing and stabilizing.  They don't stop people from working off in their
development branches (no one could
effectively impose such restrictions even if they wanted to), but they _do_
tighten down on what's allowed to be merged.

This is a forcing function that's just required.  There's a lot of issues that
otherwise won't get sufficient attention.
 If all it took was altruism then regressions would be fixed immediately, bugs
would always be fixed in highest priority
to lowest priority (assuming that could even be effectively designed), etc.

Without the 'ok, guys, focus in this smaller more critical subset of bugs'
step, release branches would be created and
never polished (or focused down to the release manager to do all the work if
he's that skilled and/or generous of his time).

There's a phrase I'm trying to remember, but it's something to the effect that
'hope isn't a recipe for success.'
Hoping that people fix regressions on release critical bugs isn't sufficient. 
Incentive and steering is required.  The
desire to ungate master branch merges is one approach that's been shown to be
successful.
Dec 15 2012
next sibling parent "Jesse Phillips" <Jessekphillips+d gmail.com> writes:
On Saturday, 15 December 2012 at 19:03:49 UTC, Brad Roberts wrote:


 This is a forcing function that's just required.


There is a focusing that needs to happen, but as you say, you 
can't really dictate where someone puts their time (for open 
source). So it is best to let the person who decided to review a 
pull to also spend the time to merge it. But strongly encourage 
everyone on the team to get the release out.

Remember we are preparing for growth and will want to be ready to 
handle all types of contributors.
Dec 15 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 15 December 2012 at 19:03:49 UTC, Brad Roberts wrote:

 On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:


 I think one of major goals is to be able to continue ongoing 
 development while at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are 
 going to release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could 
 be useful to add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)


 I've been staying out of this thread for the most part, but I 
 feel the need to comment on this part specifically.  It's
 quite common for most major projects to have a clear "we're 
 wrapping up a release" phase where work _is_ restricted to
 bug fixing and stabilizing.  They don't stop people from 
 working off in their development branches (no one could
 effectively impose such restrictions even if they wanted to), 
 but they _do_ tighten down on what's allowed to be merged.

 This is a forcing function that's just required.  There's a lot 
 of issues that otherwise won't get sufficient attention.
  If all it took was altruism then regressions would be fixed 
 immediately, bugs would always be fixed in highest priority
 to lowest priority (assuming that could even be effectively 
 designed), etc.

 Without the 'ok, guys, focus in this smaller more critical 
 subset of bugs' step, release branches would be created and
 never polished (or focused down to the release manager to do 
 all the work if he's that skilled and/or generous of his time).

 There's a phrase I'm trying to remember, but it's something to 
 the effect that 'hope isn't a recipe for success.'
 Hoping that people fix regressions on release critical bugs 
 isn't sufficient.  Incentive and steering is required.  The
 desire to ungate master branch merges is one approach that's 
 been shown to be successful.


Very good point. The final sprint is a good practice to adopt.
Dec 15 2012
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/15/2012 11:03 PM, Brad Roberts пОшет:

 On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:


 I think one of major goals is to be able to continue ongoing development while
at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are going to
release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could be useful to
add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)


 I've been staying out of this thread for the most part, but I feel the need to
comment on this part specifically.  It's
 quite common for most major projects to have a clear "we're wrapping up a
release" phase where work _is_ restricted to
 bug fixing and stabilizing.  They don't stop people from working off in their
development branches (no one could
 effectively impose such restrictions even if they wanted to), but they _do_
tighten down on what's allowed to be merged.

 This is a forcing function that's just required.  There's a lot of issues that
otherwise won't get sufficient attention.
   If all it took was altruism then regressions would be fixed immediately,
bugs would always be fixed in highest priority
 to lowest priority (assuming that could even be effectively designed), etc.


I understand the desire of focusing people attention on a priority but 
there is no denying the fact that only a small subset of folks that is 
able to (say) fix a regression in the compiler.

What I'm trying to avoid here is a situation where the compiler team 
fights with last regressions but the phobos development is frozen. Or 
the other way around. There could be a couple of these ping-pongs during 
the process. And that's what I think we had and is suboptimal.

It's part of process to have a frozen view t in a form of staging branch 
but keep the master branch going. Otherwise it's just more branches (and 
pain) for no particular purpose.


 Without the 'ok, guys, focus in this smaller more critical subset of bugs'
step, release branches would be created and
 never polished (or focused down to the release manager to do all the work if
he's that skilled and/or generous of his time).


In the grander scale it boils down to managing who does what. Basically 
there should be a release check-list that everybody knows about: a list 
of issues and takers that currently work on them (Bugzilla provides that 
and should be more heavily used). If it's more or less covered (or 
nobody else could do it) the others may go on with the development on 
master.

The main problem I see with "everybody focus on these" is that I expect 
the number of contributors to grow but their area of expertise to get be 
more and more specialized (in general). Thus IMHO it won't scale.


 There's a phrase I'm trying to remember, but it's something to the effect that
'hope isn't a recipe for success.'
 Hoping that people fix regressions on release critical bugs isn't sufficient. 
Incentive and steering is required.  The
 desire to ungate master branch merges is one approach that's been shown to be
successful.


Yes. The focus of never stopping the development is to encourage new 
contributors by providing shorter feedback cycle.

-- 
Dmitry Olshansky
Dec 15 2012
prev sibling parent "Rob T" <rob ucora.com> writes:
On Saturday, 15 December 2012 at 19:03:49 UTC, Brad Roberts wrote:

 On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:


 I think one of major goals is to be able to continue ongoing 
 development while at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are 
 going to release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could 
 be useful to add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)


 I've been staying out of this thread for the most part, but I 
 feel the need to comment on this part specifically.  It's
 quite common for most major projects to have a clear "we're 
 wrapping up a release" phase where work _is_ restricted to
 bug fixing and stabilizing.  They don't stop people from 
 working off in their development branches (no one could
 effectively impose such restrictions even if they wanted to), 
 but they _do_ tighten down on what's allowed to be merged.

 This is a forcing function that's just required.  There's a lot 
 of issues that otherwise won't get sufficient attention.
  If all it took was altruism then regressions would be fixed 
 immediately, bugs would always be fixed in highest priority
 to lowest priority (assuming that could even be effectively 
 designed), etc.

 Without the 'ok, guys, focus in this smaller more critical 
 subset of bugs' step, release branches would be created and
 never polished (or focused down to the release manager to do 
 all the work if he's that skilled and/or generous of his time).

 There's a phrase I'm trying to remember, but it's something to 
 the effect that 'hope isn't a recipe for success.'
 Hoping that people fix regressions on release critical bugs 
 isn't sufficient.  Incentive and steering is required.  The
 desire to ungate master branch merges is one approach that's 
 been shown to be successful.


I feel you've made a very important point here, and I've put up a 
section in the wiki process talk page devoted to the subject.

http://wiki.dlang.org/Talk:Release_Process#The_Path_of_Least_Resistance:_Incentives_and_Barriers

Although debating ideas in here is welcome, please be sure to 
post your ideas in the talk page, esp after there's a conclusion 
made, otherwise your excellent idea may vanish into the Ether and 
never become implemented as it should have been.

--rt
Dec 15 2012
prev sibling parent reply "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry Olshansky 
wrote:

 Second point is about merging master into staging - why not 
 just rewrite it with master branch altogether after each 
 release?
 master is the branch with correct history (all new stuff is 
 rebased on it) thus new staging will have that too.


Why you don't rewrite is because it is a public branch. Unlike 
feature branches which will basically be thrown out everyone on 
the development team will need to have staging updated. If we 
rewrite history then instead of

$ git pull staging

At random times it will be (I don't know the commands and won't 
even look it up)

It just won't be pretty.


I've made modifications to the graphic hoping to illustrate some 
thoughts.

http://i.imgur.com/rJVSg.png

This does not depict what is currently described (in terms of 
branching). But is what I've written under 
http://wiki.dlang.org/Release_Process#Release_Schedule

I see patches going into the LTS-1 (if applicable), the LTS-1 is 
then merged into the latest LTS, which is merged into any active 
staging, that is then merged into master.

The monthly release don't get bug fixes (just wait for the next 
month).

I've removed some version numbering since I don't know if we 
should have a distinct numbering for LTS and Monthly. I've 
already give some thoughts on this: 
http://forum.dlang.org/post/ydmgqmbqngwderfkljde forum.dlang.org
Dec 15 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 15 December 2012 at 20:32:42 UTC, Jesse Phillips 
wrote:

 On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry Olshansky 
 wrote:

 Second point is about merging master into staging - why not 
 just rewrite it with master branch altogether after each 
 release?
 master is the branch with correct history (all new stuff is 
 rebased on it) thus new staging will have that too.


 Why you don't rewrite is because it is a public branch. Unlike 
 feature branches which will basically be thrown out everyone on 
 the development team will need to have staging updated. If we 
 rewrite history then instead of

 $ git pull staging

 At random times it will be (I don't know the commands and won't 
 even look it up)

 It just won't be pretty.


 I've made modifications to the graphic hoping to illustrate 
 some thoughts.

 http://i.imgur.com/rJVSg.png

 This does not depict what is currently described (in terms of 
 branching). But is what I've written under 
 http://wiki.dlang.org/Release_Process#Release_Schedule

 I see patches going into the LTS-1 (if applicable), the LTS-1 
 is then merged into the latest LTS, which is merged into any 
 active staging, that is then merged into master.

 The monthly release don't get bug fixes (just wait for the next 
 month).

 I've removed some version numbering since I don't know if we 
 should have a distinct numbering for LTS and Monthly. I've 
 already give some thoughts on this: 
 http://forum.dlang.org/post/ydmgqmbqngwderfkljde forum.dlang.org


Can we drop the LTS name ? It reminds me of ubuntu, and I clearly 
hope that people promoting that idea don't plan to reproduce 
ubuntu's scheme :
  - it is not suitable for a programming language (as stated 3 
time now, so just read before why I won't repeat it).
  - ubuntu is notoriously unstable.
Dec 15 2012
next sibling parent "RenatoUtsch" <renatoutsch gmail.com> writes:
On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:

 On Saturday, 15 December 2012 at 20:32:42 UTC, Jesse Phillips 
 wrote:

 On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry 
 Olshansky wrote:

 Second point is about merging master into staging - why not 
 just rewrite it with master branch altogether after each 
 release?
 master is the branch with correct history (all new stuff is 
 rebased on it) thus new staging will have that too.


 Why you don't rewrite is because it is a public branch. Unlike 
 feature branches which will basically be thrown out everyone 
 on the development team will need to have staging updated. If 
 we rewrite history then instead of

 $ git pull staging

 At random times it will be (I don't know the commands and 
 won't even look it up)

 It just won't be pretty.


 I've made modifications to the graphic hoping to illustrate 
 some thoughts.

 http://i.imgur.com/rJVSg.png

 This does not depict what is currently described (in terms of 
 branching). But is what I've written under 
 http://wiki.dlang.org/Release_Process#Release_Schedule

 I see patches going into the LTS-1 (if applicable), the LTS-1 
 is then merged into the latest LTS, which is merged into any 
 active staging, that is then merged into master.

 The monthly release don't get bug fixes (just wait for the 
 next month).

 I've removed some version numbering since I don't know if we 
 should have a distinct numbering for LTS and Monthly. I've 
 already give some thoughts on this: 
 http://forum.dlang.org/post/ydmgqmbqngwderfkljde forum.dlang.org


 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope that people promoting that idea don't plan to 
 reproduce ubuntu's scheme :
  - it is not suitable for a programming language (as stated 3 
 time now, so just read before why I won't repeat it).
  - ubuntu is notoriously unstable.


Of course, lets just call it "stable", then. Or you have a better 
name?

Anyways, I do think that "stable" releases every 3 or more years 
and monthly or every 3 months releases are the best solution to 
the current D users.

-- Renato
Dec 15 2012
prev sibling next sibling parent reply "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope that people promoting that idea don't plan to 
 reproduce ubuntu's scheme :
  - it is not suitable for a programming language (as stated 3 
 time now, so just read before why I won't repeat it).


You don't need to repeat your self, you need to expand on your 
points. Joseph has already requested that you give specifics of 
your objection, you have explained why the situation is different 
but not what needs to be different.

Your points were specific to Debian's model, which is not 
Ubuntu's.


  - ubuntu is notoriously unstable.


I don't know anyone who uses the LTS releases. That isn't to say 
no one is, but Ubuntu is doing a lot of experimenting in their 6 
month releases.
Dec 15 2012
next sibling parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 02:03:34 UTC, Jesse Phillips wrote:

 On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope that people promoting that idea don't plan to 
 reproduce ubuntu's scheme :
 - it is not suitable for a programming language (as stated 3 
 time now, so just read before why I won't repeat it).


 You don't need to repeat your self, you need to expand on your 
 points. Joseph has already requested that you give specifics of 
 your objection, you have explained why the situation is 
 different but not what needs to be different.

 Your points were specific to Debian's model, which is not 
 Ubuntu's.


 - ubuntu is notoriously unstable.


 I don't know anyone who uses the LTS releases. That isn't to 
 say no one is, but Ubuntu is doing a lot of experimenting in 
 their 6 month releases.


I think if we focus on the end results that the Ubuntu process is 
designed to accomplish and what the Debian is designed to 
accomplish we can start to think about which model is more likely 
to produce the desired results that we wish to achieve with the D 
process.

I'll sum it up as follows:

What both systems attempt to accomplish is in conflict, Ubuntu 
attempts to create a reasonably stable distribution with more 
recent updates so that users have access to current software, but 
that means there will be more bugs and less stability.

Debian attempts to distribute a bug free stable distribution, 
however the delays in getting there mean that the software is 
often much less current than what is currently available.

The end results may be summarized as follows:

I would *never* use Ubuntu for mission critical tasks, for 
example I would not use it to power a 24/7 remote server. I may 
use it for a workstation at home, but not at work, although I may 
use it at work if I needed access to the most current software.

Why not use it for mission critical tasks? Because it's unstable. 
Why is it unstable? Because it's based on Debian's unstable 
branch.

Debian stable on the other hand, is rock solid, and works well 
for mission critical tasks, and is suitable for use in a server 
environment, however, it does not come with the most current 
software, a some packages may be a couple of versions behind 
(which can be very frustrating at times).

So to achieve stability vs instability, you have to trade away 
less mature versions of software for older more mature versions 
of software.

Debian's process is design specifically for stability, Ubuntu's 
process is designed specifically to provide the most current 
software in a reasonably stable way.

--rt
Dec 15 2012
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 02:03:34 UTC, Jesse Phillips wrote:

 On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope that people promoting that idea don't plan to 
 reproduce ubuntu's scheme :
 - it is not suitable for a programming language (as stated 3 
 time now, so just read before why I won't repeat it).


 You don't need to repeat your self, you need to expand on your 
 points. Joseph has already requested that you give specifics of 
 your objection, you have explained why the situation is 
 different but not what needs to be different.


This is completely backward, but I'll do it anyway. But first, 
let me explain why it is backward.

You are using distro's versionning system as a base of reflexion. 
But such system is made to achieve different goal than a 
programming language. I shouldn't be here explaining why this is 
wrong, you should be here explaining me why it can be applied 
anyway.

Otherwise, anyone can come with any point, whatever how stupid it 
is, and each time we have to prove that person wrong. When you 
come with something, you have to explain why it make sens, not 
the other way around.

Back to the point, and it will be the last time. A distro is a 
set of programs. The goal of the distro is to provide a set of 
programs, as up to date as possible, that integrate nicely with 
each other, and with as few bugs as possible. Some of these goals 
are in conflict, so we see different pattern emerge, with 
different tradeoff, as ubuntu and debians's processes.

 From one version to another, distro don't need backward 
compatibility. Easy migration is important, but not 
compatibility. This is why debian can switch to multiarch in its 
next version (which break a hell lot of things). You don't need 
revision of the distro because software are updated on a per 
software basis when problems are detected (package in fact, but 
it don't really matter).

This is very different from a programming language where :
- no package update is possible. The concept of package don't 
even exist in our world.
- backward compatibility is a very important point, when it isn't 
for distros.

The only goal that is coming is trying to reach some level of 
stability. Everything else is completely different.


 Your points were specific to Debian's model, which is not 
 Ubuntu's.


 - ubuntu is notoriously unstable.


 I don't know anyone who uses the LTS releases. That isn't to 
 say no one is, but Ubuntu is doing a lot of experimenting in 
 their 6 month releases.


I used to work with ubuntu. I've done a ton of things with that 
distro. It IS unstable. In fact, it is based on debian unstable, 
so it isn't really a surprise.
Dec 15 2012
next sibling parent reply "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 07:35:27 UTC, deadalnix wrote:

 The only goal that is coming is trying to reach some level of 
 stability. Everything else is completely different.


There are still some clear similarities between what Debian is 
doing and what I presume most people do in software development.

For the software I develop we have what is called a "live" 
branch, which is the code that is in active use by our customers. 
This branch corresponds to the stable branch we're trying to 
achieve, and it only gets critical bug fixes until the next major 
update which includes new features and/or major adjustments as 
well as non-critical bug fixes. We also have a testing branch, 
which includes the latest pre-release code for the next major 
update. This code is running on a VM and is tested by our 
customers (and the developers), under conditions similar to 
"live". There's also a common "dev" branch for the code which is 
in development but not yet ready for testing. This branch 
corresponds to the Dev (Master) Branch, The master branch gets 
updated from individual forks owned by each developer. 
Coordination among developers is essential to prevent 
duplications and major conflicts.

So in essence we're following a similar model to the Debian model 
of (Master Dev) which is "unstable " => Pre-release Testing => 
Stable.

It works great, and I see no way to remove any of the branches 
without seriously compromising the end result.

For example, we can't go directly from Dev to Stable, that would 
be like committing suicide. We can't use testing for dev updates, 
it's too destabilizing and we'd never get a properly tested 
stable release out unless all development updates were halted for 
a long period of time, but then the individual forks would pile 
up with major changes that would be very difficult to sort out 
when merged all back together.

We need 3 common branches, and I just don't see a way out of that.

BTW, I'm using Mercurial which has many similarities to Git. 
Unfortunately I don't know my way around Git very well at this 
time.

--rt
Dec 16 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 08:30:04 UTC, Rob T wrote:

 On Sunday, 16 December 2012 at 07:35:27 UTC, deadalnix wrote:

 The only goal that is coming is trying to reach some level of 
 stability. Everything else is completely different.


 There are still some clear similarities between what Debian is 
 doing and what I presume most people do in software development.

 For the software I develop we have what is called a "live" 
 branch, which is the code that is in active use by our 
 customers. This branch corresponds to the stable branch we're 
 trying to achieve, and it only gets critical bug fixes until 
 the next major update which includes new features and/or major 
 adjustments as well as non-critical bug fixes. We also have a 
 testing branch, which includes the latest pre-release code for 
 the next major update. This code is running on a VM and is 
 tested by our customers (and the developers), under conditions 
 similar to "live". There's also a common "dev" branch for the 
 code which is in development but not yet ready for testing. 
 This branch corresponds to the Dev (Master) Branch, The master 
 branch gets updated from individual forks owned by each 
 developer. Coordination among developers is essential to 
 prevent duplications and major conflicts.

 So in essence we're following a similar model to the Debian 
 model of (Master Dev) which is "unstable " => Pre-release 
 Testing => Stable.

 It works great, and I see no way to remove any of the branches 
 without seriously compromising the end result.

 For example, we can't go directly from Dev to Stable, that 
 would be like committing suicide. We can't use testing for dev 
 updates, it's too destabilizing and we'd never get a properly 
 tested stable release out unless all development updates were 
 halted for a long period of time, but then the individual forks 
 would pile up with major changes that would be very difficult 
 to sort out when merged all back together.

 We need 3 common branches, and I just don't see a way out of 
 that.

 BTW, I'm using Mercurial which has many similarities to Git. 
 Unfortunately I don't know my way around Git very well at this 
 time.


So distro's versioning system is good for a programming language 
because you use it successfully in your software which isn't a 
programming language (and we also don't know according to which 
goal it is successful) ?

By the way, debian testing is not what you think it is : 
http://www.debian.org/devel/testing.en.html
Dec 16 2012
parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 08:52:24 UTC, deadalnix wrote:

 So distro's versioning system is good for a programming 
 language because you use it successfully in your software which 
 isn't a programming language (and we also don't know according 
 to which goal it is successful) ?


Let's not get different things mixed together.

There is a compiler, which is software. There is a language, but 
it is not software, it's the specification that the compiler 
implements.

It does not matter how branches are versioned, what matters is 
how they move from a highly unstable state towards an 
increasingly stable state until it is released in stable form for 
the end-user to use.

I will agree with you that there are unique challenges for a 
compiler, however we're not really talking about a compiler 
specifically at this stage, we're talking about establishing a 
way of moving source code from an unstable form into a stable 
form. What the compiler needs to achieve is a totally different 
concept, which is unfortunately not well documented. There is the 
current language specification, which is improperly managed as I 
pointed out in a previous post, and there needs to be certain 
guidelines set for the compiler to follow that are not a part of 
the language specification, but we're straying way off course at 
this point - those problem areas will have to be dealt with 
later, otherwise absolutely nothing at all will get done. There's 
just too many things to fix up around here, so we have to pick 
and chose very carefully what to deal with first before moving on 
to the next problem.


 By the way, debian testing is not what you think it is : 
 http://www.debian.org/devel/testing.en.html


I think that I do know what Debian testing is. I do understand 
that the Debian distro is made up out of individual packages, but 
these packages can be generalized to what goes on when software 
is developed. For example, new features = new packages, code 
modifications = updated packages. When you distill things down to 
a process level, the payload managed by the process no longer 
matters so much.

--rt
Dec 16 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/16/2012 08:35 AM, deadalnix wrote:

 On Sunday, 16 December 2012 at 02:03:34 UTC, Jesse Phillips wrote:

 You don't need to repeat your self, you need to expand on your points. Joseph
 has already requested that you give specifics of your objection, you have
 explained why the situation is different but not what needs to be different.


 This is completely backward, but I'll do it anyway. But first, let me explain
 why it is backward.

 You are using distro's versionning system as a base of reflexion. But such
 system is made to achieve different goal than a programming language. I
 shouldn't be here explaining why this is wrong, you should be here explaining
me
 why it can be applied anyway.


And once again, you're failing to address the specifics of my proposal, simply 
saying "A distro and a programming language have different requirements."

Because while some of my proposals were inspired by some of the ways in which 
Ubuntu manages its releases, and share some of the same terminology, _they are 
not the same_.  In fact the only thing that it really shares with Ubuntu is the 
idea that the support and maintenance periods for long-term "stable" versions 
should overlap so that users who require that stable base have adequate time to 
upgrade.

That's useful for _any_ software no matter what the purpose.

So once again I'd ask you to engage with the specifics of my proposal --
because 
from what I've read so far, it seems like the _only_ source of your objections 
is the use of the words "Ubuntu" and "Long-Term Support", and that if I'd 
written the same proposal without those words, you'd have given a different 
response.
Dec 16 2012
prev sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 16 December 2012 at 07:35:27 UTC, deadalnix wrote:

 I shouldn't be here explaining why this is wrong, you should be 
 here explaining me why it can be applied anyway.



We are developing a process, specifying what it will look like.

If I'm building an airplane and say I think it should be kind of 
like a car, I'd expect your response would be.

You can't build an airplane with wheels, windshield, and a cabin 
for passengers. Cars have those and meant for driving on the 
ground with very sticked adherence to the laws of gravity.

What I expect you to say, we shouldn't be building an airplane, 
helicopters are much better at traveling and landing in tight 
places and that is what we need. Airplanes are good for when we 
need long distance.

Instead I hear, plans are good for long distance we shouldn't 
build them.


 Otherwise, anyone can come with any point, whatever how stupid 
 it is, and each time we have to prove that person wrong. When 
 you come with something, you have to explain why it make sens, 
 not the other way around.


We have explained why it makes sense, the goals are already up on 
the wiki.

So you should review those goals and explain why the plan does 
not meet those, or correct the goals. You are doing neither.


 Back to the point, and it will be the last time. A distro is a 
 set of programs. The goal of the distro is to provide a set of 
 programs, as up to date as possible, that integrate nicely with 
 each other, and with as few bugs as possible. Some of these 
 goals are in conflict, so we see different pattern emerge, with 
 different tradeoff, as ubuntu and debians's processes.


I said you don't have to repeat yourself.
Dec 16 2012
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/15/2012 09:39 PM, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I clearly hope that
 people promoting that idea don't plan to reproduce ubuntu's scheme :
   - it is not suitable for a programming language (as stated 3 time now, so
just
 read before why I won't repeat it).
   - ubuntu is notoriously unstable.


Call them "stable release cycles" if you like, which is what they are intended 
to be.  It's just that "long-term support" actually conveys quite well what 
_users_ can expect using these versions.

By the way, no one has proposed "using Ubuntu's scheme".  I made a proposal
that 
drew on a number of ideas inspired by Ubuntu's release model, including 
terminology, but the actual detail was quite different and was tailored to my 
perceived requirements of D, not an operating system.

So, once again, I'd appreciate your response to those details, rather than your 
perception that I'm proposing "using Ubuntu's scheme".
Dec 16 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:

 On 12/15/2012 09:39 PM, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's scheme :
 - it is not suitable for a programming language (as stated 3 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.


 Call them "stable release cycles" if you like, which is what they are
 intended to be.


Just one tidbit of information: I talked to Walter and we want to build 
into the process the ability to modify any particular release. (One 
possibility is to do so as part of paid support for large corporate 
users.) That means there needs to be one branch per release.

Andrei
Dec 16 2012
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/16/2012 04:05 PM, Andrei Alexandrescu wrote:

 Just one tidbit of information: I talked to Walter and we want to build into
the
 process the ability to modify any particular release. (One possibility is to do
 so as part of paid support for large corporate users.) That means there needs
to
 be one branch per release.


Can you expand on the requirements/goals there a bit?  Would be useful to have
a 
good sense of what kind of modifications you have in mind.

I don't think anything in what I've proposed (elsewhere in this thread, I'm 
afraid I haven't written it up on the wiki yet...) is incompatible with the 
notion of one branch per release.
Dec 16 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/12 10:15 AM, Joseph Rushton Wakeling wrote:

 On 12/16/2012 04:05 PM, Andrei Alexandrescu wrote:

 Just one tidbit of information: I talked to Walter and we want to
 build into the
 process the ability to modify any particular release. (One possibility
 is to do
 so as part of paid support for large corporate users.) That means
 there needs to
 be one branch per release.


 Can you expand on the requirements/goals there a bit? Would be useful to
 have a good sense of what kind of modifications you have in mind.


Right now we're using a tagging system for releases, implying releases 
are just snapshots of a continuum. But what we need is e.g. to be able 
to patch 2.065 to fix a bug for a client who can't upgrade right now. 
That means one branch per release.


 I don't think anything in what I've proposed (elsewhere in this thread,
 I'm afraid I haven't written it up on the wiki yet...) is incompatible
 with the notion of one branch per release.


I agree. Just wanted to add this to the general discussion.


Andrei
Dec 16 2012
next sibling parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 16:23:57 UTC, Andrei Alexandrescu 
wrote:

 Right now we're using a tagging system for releases, implying 
 releases are just snapshots of a continuum. But what we need is 
 e.g. to be able to patch 2.065 to fix a bug for a client who 
 can't upgrade right now. That means one branch per release.


I don't see a problem with retaining release branches for some 
time, and I think we were planning to do this as a means to 
support previous stable versions for some time period.

However, patching a downstream release while leaving upstream 
unpatched is not going to work well within the public branches, 
the flow must be from up stream to down stream, otherwise it'll 
be a confusing mess where older versions may have specific bug 
fixes but higher versions don't have the same bug fixes. In 
addition stable branches are forbidden from receiving new 
features because that will destabilize them and in addition 
upstream will be out of sync, causing a lot of confusion.

 From what I can see, part of what you are proposing will have to 
be performed off-site as a private affair between the corporate 
user and the service provider. What we can certainly provide is a 
stable branch as-is when it was released, and that branch can 
certainly receive any patches that made their way that far down 
stream, however the patches will be bug fixes only.

--rt
Dec 16 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/16/2012 05:23 PM, Andrei Alexandrescu wrote:

 Right now we're using a tagging system for releases, implying releases are just
 snapshots of a continuum. But what we need is e.g. to be able to patch 2.065 to
 fix a bug for a client who can't upgrade right now. That means one branch per
 release.


Ahh, OK.  That makes complete sense, in fact I'd assumed it was a given that 
we'd wind up with one branch per release and release-specific bugfixes, which
is 
why your remarks made me wonder if there was something more that needed
explaining.
Dec 16 2012
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, December 16, 2012 11:23:57 Andrei Alexandrescu wrote:

 Right now we're using a tagging system for releases, implying releases
 are just snapshots of a continuum. But what we need is e.g. to be able
 to patch 2.065 to fix a bug for a client who can't upgrade right now.
 That means one branch per release.


Well, you can still do that. You just create a branch from the tag when you 
need a branch. You don't have to branch from the latest. You can branch from 
any point on the tree, and tags provide a good marker for points to branch if 
you need to.

Then the official release is still clearly tagged, but you can have branches 
based off it when you need it. It also avoids some minor overhead when patching 
releases is rare. My main concern though would be that the actual release 
needs to be clearly marked separately from any patches, so it pretty much 
requires a tag regardless of what you do with branches.

What I would have expected that we do (though I haven't read through most of 
this thread or the wiki yet, so I don't know what's being proposed) would be 
to branch when we do beta, and that branch would be what would ultimately end 
up where we release from, with a tag on the commit which was the actual 
release. If further commits were made for specific clients after the release, 
then either you'd make a branch from that which was specific to that client, or 
you'd put them on the same branch, where'd they'd be after the tag for the 
release and wouldn't affect it.

- Jonathan M Davis
Dec 16 2012
parent "foobar" <foo bar.com> writes:
On Sunday, 16 December 2012 at 23:18:20 UTC, Jonathan M Davis 
wrote:

 On Sunday, December 16, 2012 11:23:57 Andrei Alexandrescu wrote:

 Right now we're using a tagging system for releases, implying 
 releases
 are just snapshots of a continuum. But what we need is e.g. to 
 be able
 to patch 2.065 to fix a bug for a client who can't upgrade 
 right now.
 That means one branch per release.


 Well, you can still do that. You just create a branch from the 
 tag when you
 need a branch. You don't have to branch from the latest. You 
 can branch from
 any point on the tree, and tags provide a good marker for 
 points to branch if
 you need to.

 Then the official release is still clearly tagged, but you can 
 have branches
 based off it when you need it. It also avoids some minor 
 overhead when patching
 releases is rare. My main concern though would be that the 
 actual release
 needs to be clearly marked separately from any patches, so it 
 pretty much
 requires a tag regardless of what you do with branches.

 What I would have expected that we do (though I haven't read 
 through most of
 this thread or the wiki yet, so I don't know what's being 
 proposed) would be
 to branch when we do beta, and that branch would be what would 
 ultimately end
 up where we release from, with a tag on the commit which was 
 the actual
 release. If further commits were made for specific clients 
 after the release,
 then either you'd make a branch from that which was specific to 
 that client, or
 you'd put them on the same branch, where'd they'd be after the 
 tag for the
 release and wouldn't affect it.

 - Jonathan M Davis


Precisely. Thank you.
Dec 17 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 16, 2012 at 10:05:58AM -0500, Andrei Alexandrescu wrote:
[...]

 Just one tidbit of information: I talked to Walter and we want to
 build into the process the ability to modify any particular release.
 (One possibility is to do so as part of paid support for large
 corporate users.) That means there needs to be one branch per
 release.


[...]

This can be done easily if we maintain each release as a separate git
branch. It does not even require modifying the current process, since as
I understand it we're already proposing to branch when making a release.
So in theory, if there's an urgent need, we can sidestep the "maintain
at most two stable releases at a time" part of the process, checkout an
older release, apply a bugfix, release another patch, etc..

If we name the git branches properly, there shouldn't be a problem with
the proliferation of branches, as it will be obvious what each branch is
for.


T

-- 
Try to keep an open mind, but not so open your brain falls out. -- theboz
Dec 16 2012
prev sibling next sibling parent reply "foobar" <foo bar.com> writes:
On Sunday, 16 December 2012 at 15:05:58 UTC, Andrei Alexandrescu 
wrote:

 On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:

 On 12/15/2012 09:39 PM, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's 
 scheme :
 - it is not suitable for a programming language (as stated 3 
 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.


 Call them "stable release cycles" if you like, which is what 
 they are
 intended to be.


 Just one tidbit of information: I talked to Walter and we want 
 to build into the process the ability to modify any particular 
 release. (One possibility is to do so as part of paid support 
 for large corporate users.) That means there needs to be one 
 branch per release.

 Andrei


I don't see why that is a requirement (having a branch per 
release).
We can still have a single stable branch with tags for releases 
and when Walter needs to provide special customizations he can 
always just branch off of the tagged release. This should be 
Walter's business and not part of the "official" community 
process.

in git terms, assuming we have a tagged release of 2.61
$ git checkout -b 2.61-partner 2.61
This branches off a new branch "2.61-partner" for the specific 
partner modification based off the contents of the "2.61" release.

Also, this kinda messes with the notion of integration. A single 
stable branch helps prevent "forgotten" bug-fixes. I.e a critical 
bug was fixed on release N, what will ensure it will be included 
in release N+1? If Release N+1 is on the same branch than the 
bug-fix is included by default and prevented the need to perform 
a manual operation (a merge) that could be forgotten.
Dec 16 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/12 11:56 AM, foobar wrote:

 I don't see why that is a requirement (having a branch per release).
 We can still have a single stable branch with tags for releases and when
 Walter needs to provide special customizations he can always just branch
 off of the tagged release.


To the extent possible, a good process should facilitate the desirable 
scenarios. Clearly this is possible, but that doesn't mean we shouldn't 
make it easy within reason.


 This should be Walter's business and not part
 of the "official" community process.


I think this is rather shortsighted. Corporate support by Walter is only 
one possible scenario, but there are many others.


 in git terms, assuming we have a tagged release of 2.61
 $ git checkout -b 2.61-partner 2.61
 This branches off a new branch "2.61-partner" for the specific partner
 modification based off the contents of the "2.61" release.

 Also, this kinda messes with the notion of integration. A single stable
 branch helps prevent "forgotten" bug-fixes. I.e a critical bug was fixed
 on release N, what will ensure it will be included in release N+1? If
 Release N+1 is on the same branch than the bug-fix is included by
 default and prevented the need to perform a manual operation (a merge)
 that could be forgotten.


The prevalent use of the feature would be that a bug in a future release 
is back-patched onto an older release.


Andrei
Dec 16 2012
next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 16 December 2012 at 18:37:48 UTC, Andrei Alexandrescu 
wrote:


 The prevalent use of the feature would be that a bug in a 
 future release is back-patched onto an older release.


 Andrei


The issue I see is we will have X releases. A bug is fixed, do we 
now apply it to all X or only Y releases?

I see it unreasonable to maintain every release. If we need to 
make a special "maintenance" release then we directly handle the 
issue being requested instead of providing every fix applicable.

Branches work when needing to handle continuous support, but tags 
are the perfect lite weight solution when you'll have to handle 
one off support. It can be done right now, but I don't know what 
version you'd release 2.57 + 1 as.
Dec 16 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-12-16 20:04, Jesse Phillips wrote:


 I see it unreasonable to maintain every release. If we need to make a
 special "maintenance" release then we directly handle the issue being
 requested instead of providing every fix applicable.


We could say that we support X number of older releases.

-- 
/Jacob Carlborg
Dec 16 2012
parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 16 December 2012 at 20:08:11 UTC, Jacob Carlborg wrote:

 On 2012-12-16 20:04, Jesse Phillips wrote:


 I see it unreasonable to maintain every release. If we need to 
 make a
 special "maintenance" release then we directly handle the 
 issue being
 requested instead of providing every fix applicable.


 We could say that we support X number of older releases.


That was my Y.

A patch should be created off the oldest supported version. If Y 
is 5 then that means checking 5 releases to decided where to 
create the patch from. The current suggestion makes Y at most 3 
and generally 2.

What is being requested is to back port a patch for customer 
using version Z, even if Z is 2.34. It is a special case and the 
driving force could be any number of reasons. The age is not 
really important, but the point is the community officially 
dropped support but someone made a compelling argument to bring 
that support back.
Dec 16 2012
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 18:37:48 UTC, Andrei Alexandrescu 
wrote:

 This should be Walter's business and not part
 of the "official" community process.


 I think this is rather shortsighted. Corporate support by 
 Walter is only one possible scenario, but there are many others.


Indeed. To avoid the proliferation of branch that need support, 
it is better to slow down the number of new feature that each 
release provide and release them by packs.

So less branch need to be maintained.


 The prevalent use of the feature would be that a bug in a 
 future release is back-patched onto an older release.


It is usually easier to go from the oldest to newest than the 
other way around.
Dec 16 2012
prev sibling parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Sunday, 16 December 2012 at 15:05:58 UTC, Andrei Alexandrescu 
wrote:

 On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:

 On 12/15/2012 09:39 PM, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's 
 scheme :
 - it is not suitable for a programming language (as stated 3 
 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.


 Call them "stable release cycles" if you like, which is what 
 they are
 intended to be.


 Just one tidbit of information: I talked to Walter and we want 
 to build into the process the ability to modify any particular 
 release. (One possibility is to do so as part of paid support 
 for large corporate users.) That means there needs to be one 
 branch per release.

 Andrei


This sounds to me like a bad idea. And indeed, I haven't heard of 
any other project doing this.

If you do so, you'll quickly stop maintaining the older branches 
(especially when the corporate users pay for their specific 
developments), forcing users to hop to newer branches, with the 
possibility of breaking changes, so it's no different than 
today's situation.

If they really want specific developments, let them have their 
own branch and not interfere with a community driven process. In 
fact, I may sound harsh and a bit extreme, but I think paying 
users should have a priority to bugfixes and that's it, not on 
the development of the language itself, as they will attempt to 
rush half baked features.

I do think a bleeding edge branch and one or two stable branches 
(let's say one per year for the last two years) is good enough. 
And one branch for the paying user, which will merge to the 
bleeding edge if its specific developments prove worthwhile. But 
as soon as there are more than one paying user, it will 
instantaneously become impractical, so that really, they should 
just be limited to bugfixes.

BTW, will they share with the community their own developments ?
Dec 16 2012
next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:

 On Sunday, 16 December 2012 at 15:05:58 UTC, Andrei 
 Alexandrescu wrote:

 On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:

 On 12/15/2012 09:39 PM, deadalnix wrote:

 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's 
 scheme :
 - it is not suitable for a programming language (as stated 3 
 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.


 Call them "stable release cycles" if you like, which is what 
 they are
 intended to be.


 Just one tidbit of information: I talked to Walter and we want 
 to build into the process the ability to modify any particular 
 release. (One possibility is to do so as part of paid support 
 for large corporate users.) That means there needs to be one 
 branch per release.

 Andrei


 This sounds to me like a bad idea. And indeed, I haven't heard 
 of any other project doing this.

 If you do so, you'll quickly stop maintaining the older 
 branches (especially when the corporate users pay for their 
 specific developments), forcing users to hop to newer branches, 
 with the possibility of breaking changes, so it's no different 
 than today's situation.

 If they really want specific developments, let them have their 
 own branch and not interfere with a community driven process. 
 In fact, I may sound harsh and a bit extreme, but I think 
 paying users should have a priority to bugfixes and that's it, 
 not on the development of the language itself, as they will 
 attempt to rush half baked features.

 I do think a bleeding edge branch and one or two stable 
 branches (let's say one per year for the last two years) is 
 good enough. And one branch for the paying user, which will 
 merge to the bleeding edge if its specific developments prove 
 worthwhile. But as soon as there are more than one paying user, 
 it will instantaneously become impractical, so that really, 
 they should just be limited to bugfixes.

 BTW, will they share with the community their own developments ?


Actually, it's pretty much what's written in the wiki.
http://wiki.dlang.org/Release_Process#Release_Schedule
Dec 16 2012
prev sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:

 This sounds to me like a bad idea. And indeed, I haven't heard 
 of any other project doing this.


Having release branches is a common practice for many open source 
projects. For example, KDE creates a branch per minor release, 
with patch releases being made off that branch. LLVM also has a 
branch for each release.

David
Dec 16 2012
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 16, 2012 at 11:43:12PM +0100, David Nadlinger wrote:

 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:

This sounds to me like a bad idea. And indeed, I haven't heard of
any other project doing this.


 
 Having release branches is a common practice for many open source
 projects. For example, KDE creates a branch per minor release, with
 patch releases being made off that branch. LLVM also has a branch
 for each release.


[...]

Not just open source projects; at my day job, we do this as well.
Development continues on the main branch, and releases are branched off
this main branch at various point. We don't release immediately after
branching, but run development in parallel for a period of time until
the code to be released is well-tested and stabilized. Then a release is
made.

After a release is made, critical bugfixes and major issues are still
being made to the release branch, and patch releases are issued
periodically from the release branch. Sometimes we go up to two
different release branches receiving bugfixes before the older release
is phased out.

Bugfixes made to the release branches are propagated to the development
branch where applicable. Usually, if the corresponding piece of code
hasn't changed much, the same bug exists in both places and generally
the same fix applies to both of them. Usually, if the code has diverged
significantly, the bug doesn't exist in the new code and the fix can
just remain in the release branch. (Sometimes two different fixes are
needed for the same bug 'cos the code has diverged a lot, but then it
can be treated as two separate bug fixes.)

Furthermore, sometimes a critical bug is found in the release branch but
the fix is already in the development branch. In that case, we backport
the fix from the dev branch, which may sometimes require significant
changes in order to account for the code divergence between the two
branches. Or if even that is insufficient, we have to make a fix
specific to the release branch.

Anyway, I'm saying all this to point out that we should not get too
worked up about the exact details of how all these things work; in
practice, things will always be far more complicated than the idealized
model of the release process that we put on paper. Let's just
concentrate on the most salient points of it -- what branches there are,
when we use them, what is necessary to port bugfixes across them, etc.,
and leave the details to the actual people doing the work. Trying to
dictate everything down to the fine points is a waste of time, because
the core devs probably aren't going to follow it anyway.


T

-- 
Claiming that your operating system is the best in the world because more
people use it is like saying McDonalds makes the best food in the world. --
Carl B. Constantine
Dec 16 2012
prev sibling next sibling parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Sunday, 16 December 2012 at 22:43:13 UTC, David Nadlinger 
wrote:

 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:

 This sounds to me like a bad idea. And indeed, I haven't heard 
 of any other project doing this.


 Having release branches is a common practice for many open 
 source projects. For example, KDE creates a branch per minor 
 release, with patch releases being made off that branch. LLVM 
 also has a branch for each release.

 David


This can only work if there are very few releases, else one has 
to patch all the branches since a bug has been discovered. If you 
release once per month, you potentially have to patch 10 or 20 
branches.
Dec 16 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 17 December 2012 at 05:29:23 UTC, SomeDude wrote:

 On Sunday, 16 December 2012 at 22:43:13 UTC, David Nadlinger 
 wrote:

 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:

 This sounds to me like a bad idea. And indeed, I haven't 
 heard of any other project doing this.


 Having release branches is a common practice for many open 
 source projects. For example, KDE creates a branch per minor 
 release, with patch releases being made off that branch. LLVM 
 also has a branch for each release.

 David


 This can only work if there are very few releases, else one has 
 to patch all the branches since a bug has been discovered. If 
 you release once per month, you potentially have to patch 10 or 
 20 branches.


This is exactly to handle this kind of grunt work that computer 
and software were invented.
Dec 16 2012
parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Monday, 17 December 2012 at 07:04:29 UTC, deadalnix wrote:


 This is exactly to handle this kind of grunt work that computer 
 and software were invented.


Yep, and why Git handles merging so well. But it doesn't know 
which branches you are merging into until you tell it. So you can 
go ahead and write the program to do that for you.
Dec 17 2012
prev sibling parent reply "foobar" <foo bar.com> writes:
On Sunday, 16 December 2012 at 22:43:13 UTC, David Nadlinger 
wrote:

 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:

 This sounds to me like a bad idea. And indeed, I haven't heard 
 of any other project doing this.


 Having release branches is a common practice for many open 
 source projects. For example, KDE creates a branch per minor 
 release, with patch releases being made off that branch. LLVM 
 also has a branch for each release.

 David


Huh?
Both LLVM and KDE are developed on *subversion* and as such their 
work-flows are not applicable. Not to mention that KDE is vastly 
different in concept and goals than a programming language.

Subversion is conceptually very different from git and its model 
imposes practical restrictions that are not relevant for git, 
mostly with regards to branches, merging, etc. Actions which are 
first class and trivial to accomplish in git. This is analogous 
to designing highways based on the speed properties of bicycles.
Dec 17 2012
parent reply "David Nadlinger" <see klickverbot.at> writes:
On Monday, 17 December 2012 at 17:31:45 UTC, foobar wrote:

 Huh?
 Both LLVM and KDE are developed on *subversion* and as such 
 their work-flows are not applicable. Not to mention that KDE is 
 vastly different in concept and goals than a programming 
 language.

 Subversion is conceptually very different from git and its 
 model imposes practical restrictions that are not relevant for 
 git, mostly with regards to branches, merging, etc. Actions 
 which are first class and trivial to accomplish in git. This is 
 analogous to designing highways based on the speed properties 
 of bicycles.


Guess what, I know that. The post by SomeDude just claimed that 
release branches in general are impractical and not used by open 
source projects, which is wrong.

David
Dec 17 2012
next sibling parent reply "foobar" <foo bar.com> writes:
On Monday, 17 December 2012 at 17:45:12 UTC, David Nadlinger 
wrote:

 On Monday, 17 December 2012 at 17:31:45 UTC, foobar wrote:

 Huh?
 Both LLVM and KDE are developed on *subversion* and as such 
 their work-flows are not applicable. Not to mention that KDE 
 is vastly different in concept and goals than a programming 
 language.

 Subversion is conceptually very different from git and its 
 model imposes practical restrictions that are not relevant for 
 git, mostly with regards to branches, merging, etc. Actions 
 which are first class and trivial to accomplish in git. This 
 is analogous to designing highways based on the speed 
 properties of bicycles.


 Guess what, I know that. The post by SomeDude just claimed that 
 release branches in general are impractical and not used by 
 open source projects, which is wrong.

 David


At least the first part in that sentence is correct - there are 
more practical work flows that are just more difficult to achieve 
in svn. The branch per release in those projects is just a 
consequence of SVN limitations. hence open source projects _that 
use git_ don't need to follow this route.

Either way, this is completely irrelevant for our purpose.
The process should be designed with DVCS in mind since we already 
settled on this [very successful] model for D. We should avoid 
designing a work-flow based on other models and limited 
experience with git.
Suggestions such as implementing specific [shell?] scripts and 
having branch-per-release brings no improvement over what we 
already have.

I personally transitioned my [previous] team from ancient systems 
(rcs, cvs, proprietary in-house crap) to git. It requires not 
only memorizing a few new command line commands but also grokking 
a different model. Those that do, use git to great effect and 
greatly increase their efficiency, others simply have a "ci" 
script that calls "git commit" and gain nothing.
At the moment we may use git commands but really we are still 
developing on mostly a subversion model. Walter used to accept 
patches and those were simply replaced by pull requests. There 
isn't any change in the mental model required to really benefit 
from a decentralized system such as git. This is what the process 
discussion is ultimately meant to fix.
Dec 17 2012
parent reply "Rob T" <rob ucora.com> writes:
On Monday, 17 December 2012 at 18:14:54 UTC, foobar wrote:

 At the moment we may use git commands but really we are still 
 developing on mostly a subversion model. Walter used to accept 
 patches and those were simply replaced by pull requests. There 
 isn't any change in the mental model required to really benefit 
 from a decentralized system such as git. This is what the 
 process discussion is ultimately meant to fix.


I think you've made a very good point. In order to make the most 
out of the way decentralized systems work vs a centralized one, 
will require an adjustment in the way people think. If we choose 
to follow the centralized model nothing will be gained when using 
a decentralized service, however centralization has its place and 
there's an obvious need to merge decentralized branches into a 
common centralize branch so that there's a common branch to use 
for testing and performing releases (etc).

I find it's great to debate ideas in here first, not the talk 
page, but any conclusions or interesting points of view should be 
posted into the wiki talk page so that it is not lost. IMO this 
one should go in the wiki if only to remind people that we have 
the flexibility of decentralized model to take advantage of.

--rt
Dec 17 2012
parent reply "foobar" <foo bar.com> writes:
On Monday, 17 December 2012 at 21:03:04 UTC, Rob T wrote:

 On Monday, 17 December 2012 at 18:14:54 UTC, foobar wrote:

 At the moment we may use git commands but really we are still 
 developing on mostly a subversion model. Walter used to accept 
 patches and those were simply replaced by pull requests. There 
 isn't any change in the mental model required to really 
 benefit from a decentralized system such as git. This is what 
 the process discussion is ultimately meant to fix.


 I think you've made a very good point. In order to make the 
 most out of the way decentralized systems work vs a centralized 
 one, will require an adjustment in the way people think. If we 
 choose to follow the centralized model nothing will be gained 
 when using a decentralized service, however centralization has 
 its place and there's an obvious need to merge decentralized 
 branches into a common centralize branch so that there's a 
 common branch to use for testing and performing releases (etc).

 I find it's great to debate ideas in here first, not the talk 
 page, but any conclusions or interesting points of view should 
 be posted into the wiki talk page so that it is not lost. IMO 
 this one should go in the wiki if only to remind people that we 
 have the flexibility of decentralized model to take advantage 
 of.

 --rt


DVCS is not about centralized vs. non centralized. This is a 
common misunderstanding which I too had when I started using git.
The actual difference is a client-server topology (CVS, SVN, etc) 
vs. P2P or perhaps "free-form" topology. By making all users 
equal with their own full copies of the repository, git and 
similar systems made the topology an aspect of the human 
work-flow instead of part of the technical design & 
implementation.

This gives you the freedom to have whatever topology you want - a 
star, a circle, whatever. For instance, Linux is developed with a 
"web of trust". the topology represents trust relationships. 
Thus, all the people Linus is pulling from directly are "core" 
developers he personally trust. They in turn trust other 
developers, and so on and so forth. Linus's version is the 
semi-official release of the Linux kernel but it is not the only 
release. For instance, Linux distributions can have their own 
repositories, and Google maintains their own repository for the 
android fork. So in fact, there are *multiple* repositories that 
represent graph roots in this "web of trust" topology.

What about D?
The current git-hub repository owned by Walter and the core devs 
(the github organization) is the official repository. *But*, we 
could also treat other compilers as roots. more over, there's no 
requirement for developers to go through the "main" github 
repository to share and sync. E.g Walter can pull directly from 
Don's repository *without* going through a formal branch on 
github. This in fact should be *the default workflow* for 
internal collaboration to reduce clutter and facilitate better 
organization.

This is why I'm arguing fiercely against having any sort of 
official alpha stage. There is no need standardizing this and it 
only mixes "private" developer only code and builds with builds 
aimed at end-users (those would be us, people writing D code and 
compiling with DMD). If you look on SVN servers, you often find 
an endless list of developer folders, "test" branches, 
"experimental" branches, etc, etc.

As a user, this is highly annoying to figure out what branches 
are meant for user consumption (releases, betas, preview for a 
new feature) and what isn't, dev X's private place to conduct 
experiments. This is only a result of the client-server topology 
imposed by svn architecture.

The official process should only standardize and focus on the 
points where integration is required. Everything else is much 
better served as being left alone. On the other hand, I think 
that we need to add more focus on pre/post stages of the actual 
coding. This means, the planning, setting priorities (aka 
roadmap, mile-stones, etc) as well as reducing regressions. I saw 
a post suggesting an automated build-bot that will run the test 
suit and build nightlies. What about DIPs, how do they integrate 
in the work-flow?
I think Andrei talked about the DIPs before but I don't thik it 
was discussed as part of this thread.
Dec 17 2012
parent reply "foobar" <foo bar.com> writes:
On Monday, 17 December 2012 at 22:02:45 UTC, foobar wrote:

 On Monday, 17 December 2012 at 21:03:04 UTC, Rob T wrote:

 On Monday, 17 December 2012 at 18:14:54 UTC, foobar wrote:

 At the moment we may use git commands but really we are still 
 developing on mostly a subversion model. Walter used to 
 accept patches and those were simply replaced by pull 
 requests. There isn't any change in the mental model required 
 to really benefit from a decentralized system such as git. 
 This is what the process discussion is ultimately meant to 
 fix.


 I think you've made a very good point. In order to make the 
 most out of the way decentralized systems work vs a 
 centralized one, will require an adjustment in the way people 
 think. If we choose to follow the centralized model nothing 
 will be gained when using a decentralized service, however 
 centralization has its place and there's an obvious need to 
 merge decentralized branches into a common centralize branch 
 so that there's a common branch to use for testing and 
 performing releases (etc).

 I find it's great to debate ideas in here first, not the talk 
 page, but any conclusions or interesting points of view should 
 be posted into the wiki talk page so that it is not lost. IMO 
 this one should go in the wiki if only to remind people that 
 we have the flexibility of decentralized model to take 
 advantage of.

 --rt


 DVCS is not about centralized vs. non centralized. This is a 
 common misunderstanding which I too had when I started using 
 git.
 The actual difference is a client-server topology (CVS, SVN, 
 etc) vs. P2P or perhaps "free-form" topology. By making all 
 users equal with their own full copies of the repository, git 
 and similar systems made the topology an aspect of the human 
 work-flow instead of part of the technical design & 
 implementation.

 This gives you the freedom to have whatever topology you want - 
 a star, a circle, whatever. For instance, Linux is developed 
 with a "web of trust". the topology represents trust 
 relationships. Thus, all the people Linus is pulling from 
 directly are "core" developers he personally trust. They in 
 turn trust other developers, and so on and so forth. Linus's 
 version is the semi-official release of the Linux kernel but it 
 is not the only release. For instance, Linux distributions can 
 have their own repositories, and Google maintains their own 
 repository for the android fork. So in fact, there are 
 *multiple* repositories that represent graph roots in this "web 
 of trust" topology.

 What about D?
 The current git-hub repository owned by Walter and the core 
 devs (the github organization) is the official repository. 
 *But*, we could also treat other compilers as roots. more over, 
 there's no requirement for developers to go through the "main" 
 github repository to share and sync. E.g Walter can pull 
 directly from Don's repository *without* going through a formal 
 branch on github. This in fact should be *the default workflow* 
 for internal collaboration to reduce clutter and facilitate 
 better organization.

 This is why I'm arguing fiercely against having any sort of 
 official alpha stage. There is no need standardizing this and 
 it only mixes "private" developer only code and builds with 
 builds aimed at end-users (those would be us, people writing D 
 code and compiling with DMD). If you look on SVN servers, you 
 often find an endless list of developer folders, "test" 
 branches, "experimental" branches, etc, etc.

 As a user, this is highly annoying to figure out what branches 
 are meant for user consumption (releases, betas, preview for a 
 new feature) and what isn't, dev X's private place to conduct 
 experiments. This is only a result of the client-server 
 topology imposed by svn architecture.

 The official process should only standardize and focus on the 
 points where integration is required. Everything else is much 
 better served as being left alone. On the other hand, I think 
 that we need to add more focus on pre/post stages of the actual 
 coding. This means, the planning, setting priorities (aka 
 roadmap, mile-stones, etc) as well as reducing regressions. I 
 saw a post suggesting an automated build-bot that will run the 
 test suit and build nightlies. What about DIPs, how do they 
 integrate in the work-flow?
 I think Andrei talked about the DIPs before but I don't thik it 
 was discussed as part of this thread.


I forgot to explain that the multiple roots is applicable to D as 
well in that we have there fully working compilers - LDC, GDC and 
DMD. They currently share the same front-end - GDC and LDC merge 
DMD's code. This situation is also sub optimal and requires 
manual work and has redundancies in the process. E.g. what if the 
LDC guys fix a bug in the "shared" front-end? Should this be 
integrated back to DMD? What if Walter comes up with a different 
fix?
Dec 17 2012
parent "Rob T" <rob ucora.com> writes:
On Monday, 17 December 2012 at 22:10:12 UTC, foobar wrote:

 I forgot to explain that the multiple roots is applicable to D 
 as well in that we have there fully working compilers - LDC, 
 GDC and DMD. They currently share the same front-end - GDC and 
 LDC merge DMD's code. This situation is also sub optimal and 
 requires manual work and has redundancies in the process. E.g. 
 what if the LDC guys fix a bug in the "shared" front-end? 
 Should this be integrated back to DMD? What if Walter comes up 
 with a different fix?


It will take me some time to read and digest all of the points 
you've made, so with that in mind I have some reading and 
thinking to do, but I am aware that LDC, GDC and possibly some 
others are poorly supported through whatever process we have 
going on at the moment, and you are raising another perfectly 
valid point that should be listed in the wiki page (if you have 
not done so already).

For example, one of our goals should be that the process will 
include specifics that are intended to support the various DMD 
front ends that are out there.

For supporting various front ends better, there's of course much 
more that can be done than adjusting the process, but as you know 
that's off topic and for later consideration, however some of it 
may have to be considered in order to fully understand what will 
eventually have to be done which may have an effect on the 
process we're conjuring up.

It would be great if the LDC and and GDC guys could toss in their 
thoughts on this subject with respect to what the process must do 
in order to support them better.

--rt
Dec 17 2012
prev sibling parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Monday, 17 December 2012 at 17:45:12 UTC, David Nadlinger 
wrote:

 On Monday, 17 December 2012 at 17:31:45 UTC, foobar wrote:

 Huh?
 Both LLVM and KDE are developed on *subversion* and as such 
 their work-flows are not applicable. Not to mention that KDE 
 is vastly different in concept and goals than a programming 
 language.

 Subversion is conceptually very different from git and its 
 model imposes practical restrictions that are not relevant for 
 git, mostly with regards to branches, merging, etc. Actions 
 which are first class and trivial to accomplish in git. This 
 is analogous to designing highways based on the speed 
 properties of bicycles.


 Guess what, I know that. The post by SomeDude just claimed that 
 release branches in general are impractical and not used by 
 open source projects, which is wrong.

 David


I was actually correct, saying that it can only work with very 
few releases, and indeed, for KDE, there are at most 2 releases 
per year and usually less.
For instance, the 4.10 is scheduled to span from october 2012 to 
july 2013, with numerous patches and corrective releases in 
between. So it's roughly the same release process as the one 
branch/year I advocated.

And it's not the same at all as creating one branch every month.
Dec 18 2012
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/18/2012 09:48 PM, SomeDude wrote:

 And it's not the same at all as creating one branch every month.


But why would you do that?  The general discussion seems to have been around
the 
idea of 1 or 2 stable releases per year, 1 every 3 months max.
Dec 18 2012
next sibling parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Tuesday, 18 December 2012 at 20:55:34 UTC, Joseph Rushton 
Wakeling wrote:

 On 12/18/2012 09:48 PM, SomeDude wrote:

 And it's not the same at all as creating one branch every 
 month.


 But why would you do that?  The general discussion seems to 
 have been around the idea of 1 or 2 stable releases per year, 1 
 every 3 months max.


THat's what I understood from Andrei's post:


 Just one tidbit of information: I talked to Walter and we want 
 to build into the process the ability to modify any particular 
 release. (One possibility is to do so as part of paid support 
 for large corporate users.) That means there needs to be one 
 branch per release.

 Andrei


Maybe I misunderstood him, I don't know.
Dec 18 2012
parent "Rob T" <rob ucora.com> writes:
On Tuesday, 18 December 2012 at 22:37:10 UTC, SomeDude wrote:

 On Tuesday, 18 December 2012 at 20:55:34 UTC, Joseph Rushton 
 Wakeling wrote:

 On 12/18/2012 09:48 PM, SomeDude wrote:

 And it's not the same at all as creating one branch every 
 month.


 But why would you do that?  The general discussion seems to 
 have been around the idea of 1 or 2 stable releases per year, 
 1 every 3 months max.


 THat's what I understood from Andrei's post:


 Just one tidbit of information: I talked to Walter and we want 
 to build into the process the ability to modify any particular 
 release. (One possibility is to do so as part of paid support 
 for large corporate users.) That means there needs to be one 
 branch per release.

 Andrei


 Maybe I misunderstood him, I don't know.


He seemed to imply that, but to me it makes little sense to 
support older versions of a stable release. The support comes 
from the latest stable bug fix update, and you normally don't 
bother propagating a bug fix any further down stream. The best 
you may do, is after a new release is made, continue supporting 
the latest older release for a limited period of time until the 
newest release is considered completely free of any major bugs.

If there's a code breaking release, I can see the previous older 
stable version being supported for a fairly long period of time.

--rt
Dec 18 2012
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 18 December 2012 at 20:55:34 UTC, Joseph Rushton 
Wakeling wrote:

 On 12/18/2012 09:48 PM, SomeDude wrote:

 And it's not the same at all as creating one branch every 
 month.


 But why would you do that?  The general discussion seems to 
 have been around the idea of 1 or 2 stable releases per year, 1 
 every 3 months max.


This is probably too slow for revision and too fast for new 
versions.

This is yet another point where the distro process don't apply to 
us : distro update a given version on a per package basis, which 
make absolute no sense for us.
Dec 18 2012
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/19/2012 01:00 AM, deadalnix wrote:

 This is probably too slow for revision and too fast for new versions.


I'm not counting minor-point bugfix updates as a "release" in this context, as
I 
doubt you'd have a separate branch for those.


 This is yet another point where the distro process don't apply to us : distro
 update a given version on a per package basis, which make absolute no sense
for us.


....?  I don't think this has any relation to anything that anyone has proposed.
Dec 18 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 01:08:28 UTC, Joseph Rushton 
Wakeling wrote:

 On 12/19/2012 01:00 AM, deadalnix wrote:

 This is probably too slow for revision and too fast for new 
 versions.


 I'm not counting minor-point bugfix updates as a "release" in 
 this context, as I doubt you'd have a separate branch for those.


I don't see how they aren't release. They are released. But they 
have branches, they are tags. Each time you package the software 
to put it in download somewhere for users to use, you do a 
release.

I you want to talk about branch or versions talk about branch or 
versions, not release please.
Dec 18 2012
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/19/2012 02:22 AM, deadalnix wrote:

 I don't see how they aren't release. They are released. But they have branches,
 they are tags. Each time you package the software to put it in download
 somewhere for users to use, you do a release.

 I you want to talk about branch or versions talk about branch or versions, not
 release please.


OK, let me rephrase it.  I doubt that it makes sense to have a separate branch 
for minor point releases (i.e. going from N.x.y --> N.x.y+1, a bugfix release).

You'd have an N.x branch and you'd tag the .y minor point releases.

I don't think this is in contradiction with Andrei's intentions.
Dec 18 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 01:38:16 UTC, Joseph Rushton 
Wakeling wrote:

 On 12/19/2012 02:22 AM, deadalnix wrote:

 I don't see how they aren't release. They are released. But 
 they have branches,
 they are tags. Each time you package the software to put it in 
 download
 somewhere for users to use, you do a release.

 I you want to talk about branch or versions talk about branch 
 or versions, not
 release please.


 OK, let me rephrase it.  I doubt that it makes sense to have a 
 separate branch for minor point releases (i.e. going from N.x.y 
 --> N.x.y+1, a bugfix release).

 You'd have an N.x branch and you'd tag the .y minor point 
 releases.

 I don't think this is in contradiction with Andrei's intentions.


No indeed. I think that this way make sense. But I'm afraid that 
releasing a new version (note version, not revision) every 6 
month or so will lead to an important number of version to 
maintain simultaneously if we want to ensure some stability.

I think the smart move here is to release new feature less often, 
but to release more at once, and at the same time go fast on bug 
fixes.

I frankly more worried about the fact that D has feature that 
don't integrate nicely with each other right now than by D not 
having enough features.
Dec 18 2012
parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 01:48:49 UTC, deadalnix wrote:

 I think the smart move here is to release new feature less 
 often, but to release more at once, and at the same time go 
 fast on bug fixes.


I'm not a a fan of time line based releases (snapshots), it makes 
no sense to me at all other than for producing a testing version, 
and even then I'm not so sure.

Why release a new stable branch every 6 mo's or whatever? Why not 
release a new stable branch when there's one worth releasing? Why 
not instead focus on releasing "revisions" (bug fixes) instead, 
up until there's enough of something new worth releasing as a 
package. New stable releases, should be major version number 
increments and they should not happen very often. Bug fixes 
should happen much more often, and if they are released as new 
branches, then the old ones should not get support, as it's 
pointless, that's what the most recent stable release branch 
provides. I can see us supporting a previous major release 
version, but not any of the minor bug fix increments that 
appeared along the way.

Perhaps there is resistance to changing from the current 
"snapshot" process which tends to produce meaningless 
buggy/breaking releases, to one that is a "feature" based release.


 I frankly more worried about the fact that D has feature that 
 don't integrate nicely with each other right now than by D not 
 having enough features.


I sure hope everyone can agree with that.

--rt
Dec 18 2012
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 06:31:04 UTC, Rob T wrote:

 On Wednesday, 19 December 2012 at 01:48:49 UTC, deadalnix wrote:

 I think the smart move here is to release new feature less 
 often, but to release more at once, and at the same time go 
 fast on bug fixes.


 I'm not a a fan of time line based releases (snapshots), it 
 makes no sense to me at all other than for producing a testing 
 version, and even then I'm not so sure.

 Why release a new stable branch every 6 mo's or whatever? Why 
 not release a new stable branch when there's one worth 
 releasing? Why not instead focus on releasing "revisions" (bug 
 fixes) instead, up until there's enough of something new worth 
 releasing as a package. New stable releases, should be major 
 version number increments and they should not happen very 
 often. Bug fixes should happen much more often, and if they are 
 released as new branches, then the old ones should not get 
 support, as it's pointless, that's what the most recent stable 
 release branch provides. I can see us supporting a previous 
 major release version, but not any of the minor bug fix 
 increments that appeared along the way.

 Perhaps there is resistance to changing from the current 
 "snapshot" process which tends to produce meaningless 
 buggy/breaking releases, to one that is a "feature" based 
 release.


I can't even say if you agree or disagree with what you quote.

If we want to understand each other, it may be nice to start 
using common vocabulary. In the planned process exposed on the 
wiki, you'll find no branch called stable (for good reasons).

Nobody talked about doing revisions in branches, it make no 
sense. Again, you should read what is written in the wiki page.

Nothing is released as new branch. I don't even understand what 
it is supposed to mean. Thing are released as 
.zip/.deb/.rpm/.dmg/whatever packages.

Finally, nobody is supposed to support fix, this is the other way 
around : fix are what you do when a supported version has a bug.
Dec 19 2012
prev sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 19 December 2012 at 06:31:04 UTC, Rob T wrote:

 Perhaps there is resistance to changing from the current 
 "snapshot" process which tends to produce meaningless 
 buggy/breaking releases, to one that is a "feature" based 
 release.


I don not see a greater correlation between snapshot releases and 
buggy/breaking any more than a feature based release being 
buggy/breaking.

To facilitate feature releases a plan for what/how many features 
will make a release. I'm not against having these, only against 
requiring it for a release.

Having it based on "important" or "enough" changes is subjective 
and the small things can be very important to someone. And even 
with just bugs, how many make for a good release/revision (we 
have way to many names that all seem to mean something different 
to everyone)?

I will agree though, if there isn't anything worth releasing, 
don't release it. But my threshold for 'worth' is much lower than 
yours.

I'd also say it has little to do with new "features" as it is 
about completing features and disruptive bugs. I'd think the 
supported/stable/lts/somethingsomething would be open to Phobos 
additions, but I'm not too concerned as things can be changed.
Dec 19 2012
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/11/2012 12:41 AM, Andrei Alexandrescu wrote:

 In turn, I'll be collecting thoughts and opinions in this thread, distilled
from
 previous discussions. We should develop a few simple git scripts (such as
 git-start-new-feature or git-start-bugfix etc) supported by an equally simple
 piece of documentation describing how to start a new release, fix a bug,
 experiment with a new feature, and such.

 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something to keep in
 mind.)

 Please chime in with ideas and thoughts.


It seems to me that while process is clearly very important, what's maybe 
necessary first is to identify what the actual results of the process are 
intended to be.  What should a 'stable version' of D actually mean, in
practice? 
  What should its lifetime be?  Should there be interim releases in-between 
stable versions which can introduce e.g. breaking features, and should those be 
treated only as beta versions or as releases in their own right?  How is that 
coupled (or decoupled) from releases of the library? etc.

I'd personally rather get those issues addressed out before talking about git 
methods, which to be honest I suspect will be similar no matter what the
release 
model.

So, with that in mind, here are some thoughts/suggestions.

People have talked about Debian's unstable => testing => stable model, but 
instead, why not consider something more akin to Ubuntu's LTS and interim
releases?

    * STABLE/Long-term support releases would be guaranteed to receive bugfixes
      and have no breaking changes for 3 years.

    * A new LTS release would be made every 2 years, so that there would be a
      1-year overlap between the current LTS release and the previous one.

    * Optionally, non-breaking new features could be included in LTS release
      updates during this timeframe, subject to strict testing.  Such new
      feature updates should only go to the latest LTS release (so, during the
      1-year overlap period, the older LTS would continue to receive bugfixes
      but not have new features added).  The process for including new features
      should probably be more liberal for Phobos than for other parts of the D
      toolchain.

    * In between LTS releases, there should be regular (3- or 6-monthly?)
      interim releases that have gone through testing and are considered
      'stable' in a working sense.  These releases _can_ make breaking changes,
      subject to strict conditions, and should be more liberal about new feature
      inclusion in general.  The idea should be that the interim releases
      converge towards the next LTS release, so the conditions for make breaking
      changes and the testing of new features both become more and more
      stringent the closer to the next LTS release you are.

    * For preference, deprecated features to be 'removed' by simply removing
      documentation and support for a feature rather than removing the feature
      itself (thinking here particularly of Phobos and Walter's observations).

The time periods involved could of course be tweaked, e.g. an LTS release might 
be supported for 5 years and be released every 3 or 4.  There could also be an 
even larger-scale cadence (10 years?) for really major redesigns.

The above reflects my personal preference that I'd rather have a language which 
is willing to continue to iron out annoying idiosyncrasies or bad design 
decisions -- but which can still provide a good stable base for development, 
making sure that there is a good overlap period between LTS releases so that 
developers have a substantial amount of time to update their codebases.
Dec 12 2012
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
Wakeling wrote:

 People have talked about Debian's unstable => testing => stable 
 model, but instead, why not consider something more akin to 
 Ubuntu's LTS and interim releases?



No, no, please. As already stated, distro release systems intend 
to solve the exact opposite problem of ours. If you want to look 
at what is done in the field, look at other programming languages.
Dec 12 2012
next sibling parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 18:47:58 UTC, deadalnix wrote:

 On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
 Wakeling wrote:

 People have talked about Debian's unstable => testing => 
 stable model, but instead, why not consider something more 
 akin to Ubuntu's LTS and interim releases?



 No, no, please. As already stated, distro release systems 
 intend to solve the exact opposite problem of ours. If you want 
 to look at what is done in the field, look at other programming 
 languages.


Point taken. Do you have any links or further info to offer on 
this line of thinking? I'd like to investigate, so a starting 
point would be nice to have if you can offer one up. Thanks!

--rt
Dec 12 2012
parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 12 December 2012 at 19:33:00 UTC, Rob T wrote:

 On Wednesday, 12 December 2012 at 18:47:58 UTC, deadalnix wrote:

 On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
 Wakeling wrote:

 People have talked about Debian's unstable => testing => 
 stable model, but instead, why not consider something more 
 akin to Ubuntu's LTS and interim releases?



 No, no, please. As already stated, distro release systems 
 intend to solve the exact opposite problem of ours. If you 
 want to look at what is done in the field, look at other 
 programming languages.


 Point taken. Do you have any links or further info to offer on 
 this line of thinking? I'd like to investigate, so a starting 
 point would be nice to have if you can offer one up. Thanks!


I did earlier, but let me repeat myself.

Distro rely on many program, that they intend to integrate.
Programming language rely on almost nothing but many programs 
rely on them.

The problems you have to solve are the exact opposite of one 
another.

I suggest considering how things are done in python, java, PHP or 
other successful languages.
Dec 12 2012
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/12/2012 07:47 PM, deadalnix wrote:

 No, no, please. As already stated, distro release systems intend to solve the
 exact opposite problem of ours.


Yes, I did already read your earlier email objecting to a "Debian-like" model. 
The thing is, it's not clear to me what actual details you're objecting to, 
apart from citing a distro as inspiration.

I'm not wedded to any detail of what I suggested, although certain parts of it 
seem common sense, but it's hard to deal with an objection that isn't backed up 
with details and counter-examples.


 If you want to look at what is done in the field, look at other programming
languages.


Could you give concrete examples to contrast with what I've proposed?
Dec 13 2012
prev sibling parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
Wakeling wrote:

 It seems to me that while process is clearly very important, 
 what's maybe necessary first is to identify what the actual 
 results of the process are intended to be.  What should a 
 'stable version' of D actually mean, in practice?
  What should its lifetime be?  Should there be interim releases 
 in-between stable versions which can introduce e.g. breaking 
 features, and should those be treated only as beta versions or 
 as releases in their own right?  How is that coupled (or 
 decoupled) from releases of the library? etc.

 I'd personally rather get those issues addressed out before 
 talking about git methods, which to be honest I suspect will be 
 similar no matter what the release model.


Agreed 100%. We're going about this completely ass backwards.

Here's my stab at defining the main points to achieve:

1) We want improved stability and predictability with compiler 
releases to prevent a new release from unexpectedly breaking 
existing D code without significant prior warnings and time to 
prepare for it.

2) Stable releases should remain stable while being maintained, 
which means "bug fixes only".

3) We don't want to cast the language in stone, so new features 
must be allowed to be introduced in a more predictable and 
planned way that allows D programmers to confidently use the new 
features in their production code.

4) We want to allow D to continue to be enhanced in a way that 
allows the D compiler developers to do what they do best, while 
not interfering with the growing D user base who wish to rely on 
relative stability and predictability.

Points 3 and 4 may seem the same, but the important difference is 
that there are two types of D users, those who use the language 
and those who develop the language and compiler (a person may do 
both of course), so the goals and requirements for each group of 
user are different and may conflict.

If I write stuff like this in here, it'll just end up 
disappearing into the black hole.

Can we agree to create a wiki page for defining what the main 
goals are, and what we wish to achieve?

Once we have consensus on what is to be achieved, we can define a 
process and test it to see if it will achieve what we want.

This is a much better approach than trying to build a process to 
achieve something that is not yet defined and agreed on.

--rt
Dec 12 2012
next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 21:17:31 UTC, Rob T wrote:


 Can we agree to create a wiki page for defining what the main 
 goals are, and what we wish to achieve?


I agree with the listed points.
Dec 12 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 12, 2012 at 10:17:30PM +0100, Rob T wrote:
[...]

 If I write stuff like this in here, it'll just end up disappearing
 into the black hole.
 
 Can we agree to create a wiki page for defining what the main goals
 are, and what we wish to achieve?
 
 Once we have consensus on what is to be achieved, we can define a
 process and test it to see if it will achieve what we want.
 
 This is a much better approach than trying to build a process to
 achieve something that is not yet defined and agreed on.


[...]

Please let's use the new wiki at wiki.dlang.org to coordinate this
effort. We don't want this discussion to vanish into the ether after 2
months, and we end up back at square one.


T

-- 
There are two ways to write error-free programs; only the third one works.
Dec 12 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 12, 2012 at 07:04:17PM +0100, Joseph Rushton Wakeling wrote:
[...]

 It seems to me that while process is clearly very important, what's
 maybe necessary first is to identify what the actual results of the
 process are intended to be.  What should a 'stable version' of D
 actually mean, in practice?  What should its lifetime be?  Should
 there be interim releases in-between stable versions which can
 introduce e.g. breaking features, and should those be treated only
 as beta versions or as releases in their own right?  How is that
 coupled (or decoupled) from releases of the library? etc.
 
 I'd personally rather get those issues addressed out before talking
 about git methods, which to be honest I suspect will be similar no
 matter what the release model.


+1. Thanks for the reminder to not lose sight of the forest for the
trees.



 So, with that in mind, here are some thoughts/suggestions.
 
 People have talked about Debian's unstable => testing => stable
 model, but instead, why not consider something more akin to Ubuntu's
 LTS and interim releases?
 
    * STABLE/Long-term support releases would be guaranteed to receive
      bugfixes and have no breaking changes for 3 years.
 
    * A new LTS release would be made every 2 years, so that there
      would be a 1-year overlap between the current LTS release and the
      previous one.


[...]

 The above reflects my personal preference that I'd rather have a
 language which is willing to continue to iron out annoying
 idiosyncrasies or bad design decisions -- but which can still
 provide a good stable base for development, making sure that there
 is a good overlap period between LTS releases so that developers
 have a substantial amount of time to update their codebases.


I like this idea!!

I think this is much better than the current choice of either breaking
existing code and annoying existing users or living with bad design
decisions forever. One year is a good amount of time to upgrade people's
codebases, so we won't be hampered by bad design decisions in the past,
but can move forward with breaking changes. Freezing the language along
with bad design decisions is what made C++ the monster that it is today.

(Of course, the actual overlap times can be tweaked, but I definitely
support the idea of being able to introduce breaking changes *and* not
immediately alienate a good proportion of the userbase.)


T

-- 
The peace of mind---from knowing that viruses which exploit Microsoft system
vulnerabilities cannot touch Linux---is priceless. -- Frustrated system
administrator.
Dec 12 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, December 12, 2012 19:04:17 Joseph Rushton Wakeling wrote:

 It seems to me that while process is clearly very important, what's maybe
 necessary first is to identify what the actual results of the process are
 intended to be.  What should a 'stable version' of D actually mean, in
 practice? What should its lifetime be?  Should there be interim releases
 in-between stable versions which can introduce e.g. breaking features, and
 should those be treated only as beta versions or as releases in their own
 right?  How is that coupled (or decoupled) from releases of the library?
 etc.
 
 I'd personally rather get those issues addressed out before talking about
 git methods, which to be honest I suspect will be similar no matter what
 the release model.


Agreed.


 So, with that in mind, here are some thoughts/suggestions.
 
 People have talked about Debian's unstable => testing => stable model, but
 instead, why not consider something more akin to Ubuntu's LTS and interim
 releases?


That makes a _lot_ more sense than the unstable => testing => stable model.

I like the idea of having an LTS release at some interval (probably 1 year) 
where that branch has bug fix releases more or less monthly. We then have a dev 
release cycle where we put out a version of the master branch every 2 or 3 
months or so, where it includes whatever new features we want to include but 
haven't been merged into the LTS branch yet. Then with the next major LTS 
release, we merge in all of the new stuff from the master branch so that it's 
essentially up to date with master (possibly modulo some newer features or 
other major changes if they weren't deemed ready yet). So, we then end up with 
the LTS/stable branch having a versioning scheme something like

2.061.1, 2.061.2, 2.061.3, 2.061.4 ... 2.062.1, 2.062.2, ...

and the master/dev branch has a versioning scheme like

2.062.dev1, 2.062.dev2, 2.062.dev3, 2.062, 2.063.dev1, ...

The releases on the master/dev branch which aren't marked with dev are then 
the new major LTS release, which the LTS/stable branch bases its next years 
worth of minor releases off of.

The exact way to name the versions probably needs some work, but the basic 
model sounds fairly straightforward and clean to me. We then have longer term 
releases for those who want stability where they still get bug fixes fairly 
frequently (possibly more frequently than now, since there won't be new 
features being added, causing more risk for releasing), and we continue to 
have dev releases more or less as we have been for those that want new 
features faster.

As for what that means for github and branching, I'd expect us to continue 
more or less as we have been for master except that newer features and other 
major changes would start out on feature branches so that they can be properly 
sorted out before being merged into master, and we'd actually branch when 
doing betas for releases and do the releases from those branches rather than 
master. Then the folks mananging the stable branch would merge in bug fixes as 
we go along, doing their minor releases whenever they feel it's appropriate.

As for the standard library and its releases, I wouldn't really expect them to 
be decoupled from the compiler's releases but rather that they would continue 
more or less as they have been. I think that it's too often the case that 
changes in them need to be made at the same time for it to really work to 
decouple them at this point.

- Jonathan M Davis
Dec 13 2012
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/13/2012 10:07 AM, Jonathan M Davis wrote:

 That makes a _lot_ more sense than the unstable => testing => stable model.

 I like the idea of having an LTS release at some interval (probably 1 year)
 where that branch has bug fix releases more or less monthly. We then have a dev
 release cycle where we put out a version of the master branch every 2 or 3
 months or so, where it includes whatever new features we want to include but
 haven't been merged into the LTS branch yet. Then with the next major LTS
 release, we merge in all of the new stuff from the master branch so that it's
 essentially up to date with master (possibly modulo some newer features or
 other major changes if they weren't deemed ready yet).


I wonder if I maybe created some confusion by conflating "LTS" and "stable" --
I 
don't think I should really have done that.

The problem that I see with what you describe is that it gives no clear 
mechanism for how breaking changes should be handled -- I don't mean the 
decision-making process of "Should we make this change?" but the means in which 
users are supported in making the shift from one LTS to another.

So, perhaps it would help if I proposed instead:

    -- A _stable_ release is what we all understand, a release that has been
       properly tested and has no outstanding bug reports at the time of first
       release, and to which only bugfixes are subsequently made

    -- LTS should refer to a _cycle_ of stable releases within which no breaking
       changes are made

    -- Successive LTS cycles should overlap so that the previous cycle is still
       receiving bugfixes for some time after the new LTS cycle begins.  In this
       way, users have a safe period in which to update their codebases to the
       new LTS.

So -- forgetting D's versioning scheme for a moment -- suppose you label a
given 
LTS cycle with N.  Then N.0, N.1, N.2, ... would be the stable releases 
containing new but non-breaking features, which might be made annually, or
every 
6 months.  N.x.0, N.x.1, N.x.2, ... would be the bugfix-only updates to those 
releases, made as often as necessary.

Once LTS cycle N+1 has begun, then you stop making N.x releases and only apply 
further bug fixes.  Likewise, bugfixes to N.x stop getting released once N.x+1 
is released.

So, if you assume a stable release annually, and that an LTS cycle lasts 3 
years, you might have:

1st year    N.0.0
                 - N.0.1   [bugfixes]
                 - N.0.2
                   ...

2nd year    N.1.0   [non-breaking new features]
                 - N.1.1
                 - N.1.2
                   ...
                                          [new LTS cycle begins, maybe with
                                           breaking changes]
3rd year    N.2.0                       N+1.0.0
                 - N.2.1                       - N+1.0.1
                 - N.2.2                       - N+1.0.2
                   ...                           ...

4th year    N+1.1.0    [entirely moved over to new LTS]
                   - N+1.1.1
                   - N+1.1.2
                     ...

5th year    N+1.2.0                     N+2.0.0 [new LTS begins]

6th year    N+2.1.0

7th year    N+2.2.0                     N+3.0.0

             ... etc.

Of course, the timeline could be altered but the point would be that for every 
LTS cycle you'd have up to a year in which to switch, and _within_ every LTS 
cycle you would receive stable updates with bugfixes (regularly) and
well-tested 
new features (annually, 6-monthly?).

That's surely more work for developers but is probably worth it in terms of 
security for users of D.  In the event that a new LTS is accompanied by no 
breaking changes, you might do away with the year's overlap to save the extra
work.

If you want to translate the above into actual D version numbers, you could do 
it in the form V.N.x.y where V is the D language version (2 for the foreseeable 
future:-), N the LTS cycle, etc.

That leaves out "dev" releases, but you might want to number them differently 
anyway -- e.g. dev-YYYYmmdd or whatever.


 As for the standard library and its releases, I wouldn't really expect them to
 be decoupled from the compiler's releases but rather that they would continue
 more or less as they have been. I think that it's too often the case that
 changes in them need to be made at the same time for it to really work to
 decouple them at this point.


My feeling here was that as it stands Phobos is still somewhat more in flux
than 
D itself, not so much in terms of breaking changes as in terms of new features 
being added.  So, I was wondering if it might be worthwhile to make 
new-Phobos-feature releases on a more regular basis at least in the short term 
while the library is still being expanded.  But I won't press for it too much.
Dec 13 2012
next sibling parent "RenatoUtsch" <renatoutsch gmail.com> writes:
On Thursday, 13 December 2012 at 15:44:25 UTC, Joseph Rushton 
Wakeling wrote:

 On 12/13/2012 10:07 AM, Jonathan M Davis wrote:

 That makes a _lot_ more sense than the unstable => testing => 
 stable model.

 I like the idea of having an LTS release at some interval 
 (probably 1 year)
 where that branch has bug fix releases more or less monthly. 
 We then have a dev
 release cycle where we put out a version of the master branch 
 every 2 or 3
 months or so, where it includes whatever new features we want 
 to include but
 haven't been merged into the LTS branch yet. Then with the 
 next major LTS
 release, we merge in all of the new stuff from the master 
 branch so that it's
 essentially up to date with master (possibly modulo some newer 
 features or
 other major changes if they weren't deemed ready yet).


 I wonder if I maybe created some confusion by conflating "LTS" 
 and "stable" -- I don't think I should really have done that.

 The problem that I see with what you describe is that it gives 
 no clear mechanism for how breaking changes should be handled 
 -- I don't mean the decision-making process of "Should we make 
 this change?" but the means in which users are supported in 
 making the shift from one LTS to another.

 So, perhaps it would help if I proposed instead:

    -- A _stable_ release is what we all understand, a release 
 that has been
       properly tested and has no outstanding bug reports at the 
 time of first
       release, and to which only bugfixes are subsequently made

    -- LTS should refer to a _cycle_ of stable releases within 
 which no breaking
       changes are made

    -- Successive LTS cycles should overlap so that the previous 
 cycle is still
       receiving bugfixes for some time after the new LTS cycle 
 begins.  In this
       way, users have a safe period in which to update their 
 codebases to the
       new LTS.

 So -- forgetting D's versioning scheme for a moment -- suppose 
 you label a given LTS cycle with N.  Then N.0, N.1, N.2, ... 
 would be the stable releases containing new but non-breaking 
 features, which might be made annually, or every 6 months.  
 N.x.0, N.x.1, N.x.2, ... would be the bugfix-only updates to 
 those releases, made as often as necessary.

 Once LTS cycle N+1 has begun, then you stop making N.x releases 
 and only apply further bug fixes.  Likewise, bugfixes to N.x 
 stop getting released once N.x+1 is released.

 So, if you assume a stable release annually, and that an LTS 
 cycle lasts 3 years, you might have:

 1st year    N.0.0
                 - N.0.1   [bugfixes]
                 - N.0.2
                   ...

 2nd year    N.1.0   [non-breaking new features]
                 - N.1.1
                 - N.1.2
                   ...
                                          [new LTS cycle begins, 
 maybe with
                                           breaking changes]
 3rd year    N.2.0                       N+1.0.0
                 - N.2.1                       - N+1.0.1
                 - N.2.2                       - N+1.0.2
                   ...                           ...

 4th year    N+1.1.0    [entirely moved over to new LTS]
                   - N+1.1.1
                   - N+1.1.2
                     ...

 5th year    N+1.2.0                     N+2.0.0 [new LTS begins]

 6th year    N+2.1.0

 7th year    N+2.2.0                     N+3.0.0

             ... etc.

 Of course, the timeline could be altered but the point would be 
 that for every LTS cycle you'd have up to a year in which to 
 switch, and _within_ every LTS cycle you would receive stable 
 updates with bugfixes (regularly) and well-tested new features 
 (annually, 6-monthly?).

 That's surely more work for developers but is probably worth it 
 in terms of security for users of D.  In the event that a new 
 LTS is accompanied by no breaking changes, you might do away 
 with the year's overlap to save the extra work.

 If you want to translate the above into actual D version 
 numbers, you could do it in the form V.N.x.y where V is the D 
 language version (2 for the foreseeable future:-), N the LTS 
 cycle, etc.

 That leaves out "dev" releases, but you might want to number 
 them differently anyway -- e.g. dev-YYYYmmdd or whatever.


 As for the standard library and its releases, I wouldn't 
 really expect them to
 be decoupled from the compiler's releases but rather that they 
 would continue
 more or less as they have been. I think that it's too often 
 the case that
 changes in them need to be made at the same time for it to 
 really work to
 decouple them at this point.


 My feeling here was that as it stands Phobos is still somewhat 
 more in flux than D itself, not so much in terms of breaking 
 changes as in terms of new features being added.  So, I was 
 wondering if it might be worthwhile to make new-Phobos-feature 
 releases on a more regular basis at least in the short term 
 while the library is still being expanded.  But I won't press 
 for it too much.


This approach looks the best to D. As we have the "problem" that 
some users want to see D more stable and other users want to see 
it evolve, to not turn into what C++ has become, this approach 
looks the best solution of all that were proposed here.

It will satisfy the users that like the language to be more 
stable for serious development, while the language can still 
evolve and the language devs will be able to do the things right.

if each LTS is supported for about 5 years, and with 1 year 
supporting the last LTS after a new one is released, D will be 
stable enough for serious development, what will attract much 
people, as the language is simply incredible, the only problem 
(at least with all people I know) is that it is not really stable.

While the LTS versions are supported, breaking changes can be 
introduced in non-LTS versions that are supported for bugfixes 
for... say 6 monthes, and non-breaking changes are merged on both 
the LTS and the non-LTS.

This will satisfy all D users... it will only require more work 
from the D team.
Dec 13 2012
prev sibling parent reply "Rob T" <rob ucora.com> writes:
On Thursday, 13 December 2012 at 15:44:25 UTC, Joseph Rushton 
Wakeling wrote:

 My feeling here was that as it stands Phobos is still somewhat 
 more in flux than D itself, not so much in terms of breaking 
 changes as in terms of new features being added.  So, I was 
 wondering if it might be worthwhile to make new-Phobos-feature 
 releases on a more regular basis at least in the short term 
 while the library is still being expanded.  But I won't press 
 for it too much.


I'll be the first to agree that Phobos should be separated from 
the compiler releases, but not entirely. I wish the std lib could 
be separated into an "essential component" and a "non-essential 
component" so that a partial decoupling could be achieved, that 
way the essential component could be directly locked into 
successive compiler releases, while leaving out the non-essential 
as a separate add-on package following it's own course of 
development.

But IMO we're straying off-topic because what we really need to 
do first, is fire up a wiki page to write down and agree on (as 
best as we can) what we want to achieve (a list of goals), and 
only after that can we continue on deciding what process will 
meet all of the agreed on goals.

Anyone proficient in wiki pages want to create a new page for 
defining the goals?

--rt
Dec 13 2012
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 13, 2012 at 07:26:18PM +0100, Rob T wrote:
[...]

 But IMO we're straying off-topic because what we really need to do
 first, is fire up a wiki page to write down and agree on (as best as
 we can) what we want to achieve (a list of goals), and only after
 that can we continue on deciding what process will meet all of the
 agreed on goals.
 
 Anyone proficient in wiki pages want to create a new page for
 defining the goals?


[...]

There's been too much talk and little action, so I've gone ahead with a
stub page for this discussion:

http://wiki.dlang.org/Proposed_new_D_development_process

Please fill in the details as you understand them, so that we have
something to start with.

Also, it will probably be best if we continue this discussion on the
talk page, so that everything is in one place.

Since each of us may have conflicting ideas about what the final process
should be, let's adopt the convention that if something on the page is
not how you understand things should be, you should discuss on the talk
page before making the change, so that we don't devolve into an edit
war.


T

-- 
Computerese Irregular Verb Conjugation: I have preferences.  You have biases. 
He/She has prejudices. -- Gene Wirchenko
Dec 13 2012
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 13 December 2012 at 18:41:47 UTC, H. S. Teoh wrote:

 On Thu, Dec 13, 2012 at 07:26:18PM +0100, Rob T wrote:
 [...]

 But IMO we're straying off-topic because what we really need 
 to do
 first, is fire up a wiki page to write down and agree on (as 
 best as
 we can) what we want to achieve (a list of goals), and only 
 after
 that can we continue on deciding what process will meet all of 
 the
 agreed on goals.
 
 Anyone proficient in wiki pages want to create a new page for
 defining the goals?


 [...]

 There's been too much talk and little action, so I've gone 
 ahead with a
 stub page for this discussion:

 http://wiki.dlang.org/Proposed_new_D_development_process

 Please fill in the details as you understand them, so that we 
 have
 something to start with.

 Also, it will probably be best if we continue this discussion 
 on the
 talk page, so that everything is in one place.

 Since each of us may have conflicting ideas about what the 
 final process
 should be, let's adopt the convention that if something on the 
 page is
 not how you understand things should be, you should discuss on 
 the talk
 page before making the change, so that we don't devolve into an 
 edit
 war.


I just started this http://wiki.dlang.org/Release_Process

Feel free to merge or whatever.
Dec 13 2012
prev sibling parent reply "Rob T" <rob ucora.com> writes:
On Thursday, 13 December 2012 at 18:41:47 UTC, H. S. Teoh wrote:

 Since each of us may have conflicting ideas about what the 
 final process
 should be, let's adopt the convention that if something on the 
 page is
 not how you understand things should be, you should discuss on 
 the talk
 page before making the change, so that we don't devolve into an 
 edit
 war.


 T


I was wondering how we'll resolve conflicts, because not everyone 
will agree on everything thats for certain. We don't really have 
a voting system in place, and there's no "benevolent dictator" 
heading the project unless Andrei has appointed himself for 
resolving conflicts, but he's not really participating much at 
this point. We're kinda leaderless and left with no mechanism for 
making the inevitable hard decisions that we'll have to make if 
this effort is to move ahead.

Any ideas?

--rt
Dec 13 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 13, 2012 at 10:36:17PM +0100, Rob T wrote:

 On Thursday, 13 December 2012 at 18:41:47 UTC, H. S. Teoh wrote:

Since each of us may have conflicting ideas about what the final
process should be, let's adopt the convention that if something on
the page is not how you understand things should be, you should
discuss on the talk page before making the change, so that we don't
devolve into an edit war.




[...]

 I was wondering how we'll resolve conflicts, because not everyone will
 agree on everything thats for certain. We don't really have a voting
 system in place, and there's no "benevolent dictator" heading the
 project unless Andrei has appointed himself for resolving conflicts,
 but he's not really participating much at this point.  We're kinda
 leaderless and left with no mechanism for making the inevitable hard
 decisions that we'll have to make if this effort is to move ahead.
 
 Any ideas?


[...]

I don't have a good answer to this, but maybe we could use Wikipedia's
method of garnering consensus before a major change is made? I don't
know how well it will work, given our relatively small numbers, though.

It would be nice if Andrei could step in and head up this process. But
then again, he's a busy man and only does D on his limited free time, so
it may be just a matter of waiting for a bit for him to pipe up.


T

-- 
Mediocrity has been pushed to extremes.
Dec 13 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-12-11 00:41, Andrei Alexandrescu wrote:

 Hello all,


 Walter and I have had a long discussion following his trip to Australia.
 Following the current sprint for Win64 (which we all, I think, agree was
 long overdue and had to be done), the main area we need to work on (as
 I'm sure many would agree) is improving our process, in particular the
 approach to releasing the compiler and standard library.

 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and simple.

 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple git
 scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how to
 start a new release, fix a bug, experiment with a new feature, and such.

 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something to
 keep in mind.)


Don't know if this has already been said but there already exists a git 
command for this workflow:

https://github.com/nvie/gitflow

It's available in MacPorts so I should assume it's available in most 
Linux package mangers as well.

-- 
/Jacob Carlborg
Dec 14 2012
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
One thing that I think would be really awesome when this is all rolling, is
attach a system that does nightly builds of the dev line every evening.
I've often had to pester Walter to produce a new build for us when a
critical bug/feature was fixed.
Dec 14 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/14/2012 4:12 PM, Manu пОшет:

 One thing that I think would be really awesome when this is all rolling,
 is attach a system that does nightly builds of the dev line every evening.
 I've often had to pester Walter to produce a new build for us when a
 critical bug/feature was fixed.


Then this could be of interest:

http://forum.dlang.org/post/mailman.2633.1355345752.5162.digitalmars-d puremagic.com

 From that I take it's basically coming soon.

-- 
Dmitry Olshansky
Dec 14 2012
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/14/12, Manu <turkeyman gmail.com> wrote:

 I've often had to pester Walter to produce a new build for us when a
 critical bug/feature was fixed.


Why waste Walter's time when building is as simple as calling make?
Dec 14 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-12-14 13:15, Andrej Mitrovic wrote:


 Why waste Walter's time when building is as simple as calling make?


Agree. And with DVM it's just one command, after all repositories are 
updated, instead of calling "make" three times.

https://github.com/jacob-carlborg/dvm

-- 
/Jacob Carlborg
Dec 14 2012
parent reply "Dan" <dbdavidson yahoo.com> writes:
On Friday, 14 December 2012 at 12:56:42 UTC, Jacob Carlborg wrote:

 On 2012-12-14 13:15, Andrej Mitrovic wrote:


 Why waste Walter's time when building is as simple as calling 
 make?


 Agree. And with DVM it's just one command, after all 
 repositories are updated, instead of calling "make" three times.

 https://github.com/jacob-carlborg/dvm


Just saw after posting. The description looks great. So, this is 
what D developers are using? Good stuff. Thanks.
Dec 14 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-12-14 14:10, Dan wrote:


 Just saw after posting. The description looks great. So, this is what D
 developers are using? Good stuff. Thanks.


At least some are using it. Not the whole community. The latest release 
has 20 downloads across Linux, Windows and Mac OS X.

https://github.com/jacob-carlborg/dvm/downloads

-- 
/Jacob Carlborg
Dec 14 2012
prev sibling parent reply "Dan" <dbdavidson yahoo.com> writes:
On Friday, 14 December 2012 at 12:15:28 UTC, Andrej Mitrovic 
wrote:

 On 12/14/12, Manu <turkeyman gmail.com> wrote:

 I've often had to pester Walter to produce a new build for us 
 when a
 critical bug/feature was fixed.


 Why waste Walter's time when building is as simple as calling 
 make?


His point was he clearly wants *not* to waste Walter's time.

In the discussions of build system one thread was about using 
Ruby. A comment was why not just use D to script it, you can do 
everything in D that you can from ruby (or shell)? I wrote a 
small script to see how cumbersome it would be to download and 
build D with that approach: one piece grabs latest source 
(phobos, runtime, dmd, tools), one builds it. 
http://pastebin.com/CC7DvZuJ

It ain't pretty and tailored to linux. I don't know how D 
developers set up their environment - a technote may exist 
somewhere? The focus in this thread is on release process which 
is critical, but an addition to wiki page on "how to get started 
as a developer on D project" might be useful. I don't think it is 
that complex to build D, but for me it was not as simple as just 
calling make because you have to get the prereqs, know things 
like phobos inludes the runtime, so build runtime before phobos, 
and if you want release and debug builds you should have separate 
folders since they build in place.
I also made a small change local to posix.mak, to be able to 
build debug.

------
-GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2
+ifneq (x,x$(DEBUG))
+	GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -g -DDEBUG=1 
-DUNITTEST $(COV)
+else
+	GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2
+endif
------

This may all be bad advice, since I only did it to see if I could 
step into associative array code to track down a bug and I don't 
know the good D dev setup for linux.

Any ideas that push toward continuous build/integration/test are 
the right direction.
Dec 14 2012
next sibling parent Manu <turkeyman gmail.com> writes:
On 14 December 2012 13:07, Dan <dbdavidson yahoo.com> wrote:


 On Friday, 14 December 2012 at 12:15:28 UTC, Andrej Mitrovic wrote:


 On 12/14/12, Manu <turkeyman gmail.com> wrote:


 I've often had to pester Walter to produce a new build for us when a
 critical bug/feature was fixed.


 Why waste Walter's time when building is as simple as calling make?


 His point was he clearly wants *not* to waste Walter's time.

 In the discussions of build system one thread was about using Ruby. A
 comment was why not just use D to script it, you can do everything in D
 that you can from ruby (or shell)? I wrote a small script to see how
 cumbersome it would be to download and build D with that approach: one
 piece grabs latest source (phobos, runtime, dmd, tools), one builds it.
 http://pastebin.com/CC7DvZuJ

 It ain't pretty and tailored to linux. I don't know how D developers set
 up their environment - a technote may exist somewhere? The focus in this
 thread is on release process which is critical, but an addition to wiki
 page on "how to get started as a developer on D project" might be useful. I
 don't think it is that complex to build D, but for me it was not as simple
 as just calling make because you have to get the prereqs, know things like
 phobos inludes the runtime, so build runtime before phobos, and if you want
 release and debug builds you should have separate folders since they build
 in place.
 I also made a small change local to posix.mak, to be able to build debug.

 ------
 -GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2
 +ifneq (x,x$(DEBUG))
 +       GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -g -DDEBUG=1
 -DUNITTEST $(COV)
 +else
 +       GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2
 +endif
 ------

 This may all be bad advice, since I only did it to see if I could step
 into associative array code to track down a bug and I don't know the good D
 dev setup for linux.

 Any ideas that push toward continuous build/integration/test are the right
 direction.


Thank you.
Dec 14 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 02:07:40PM +0100, Dan wrote:
[...]

 I don't know how D developers set up their environment - a technote
 may exist somewhere? The focus in this thread is on release process
 which is critical, but an addition to wiki page on "how to get started
 as a developer on D project" might be useful. I don't think it is that
 complex to build D, but for me it was not as simple as just calling
 make because you have to get the prereqs, know things like phobos
 inludes the runtime, so build runtime before phobos, and if you want
 release and debug builds you should have separate folders since they
 build in place.


[...]

http://wiki.dlang.org/Building_DMD


T

-- 
War doesn't prove who's right, just who's left. -- BSD Games' Fortune
Dec 14 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 06:29:50AM -0800, H. S. Teoh wrote:

 On Fri, Dec 14, 2012 at 02:07:40PM +0100, Dan wrote:
 [...]

 I don't know how D developers set up their environment - a technote
 may exist somewhere? The focus in this thread is on release process
 which is critical, but an addition to wiki page on "how to get started
 as a developer on D project" might be useful. I don't think it is that
 complex to build D, but for me it was not as simple as just calling
 make because you have to get the prereqs, know things like phobos
 inludes the runtime, so build runtime before phobos, and if you want
 release and debug builds you should have separate folders since they
 build in place.


 [...]
 
 http://wiki.dlang.org/Building_DMD


[...]

Also, a more general guide to contributing to D is here:

	http://wiki.dlang.org/Get_involved

(As this is a wiki, if you find any errors/omissions please edit and
improve the page.)


T

-- 
Study gravitation, it's a field with a lot of potential.
Dec 14 2012
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:

 Hello all,
 
 Walter and I have had a long discussion following his trip to
 Australia. Following the current sprint for Win64 (which we all, I
 think, agree was long overdue and had to be done), the main area we
 need to work on (as I'm sure many would agree) is improving our
 process, in particular the approach to releasing the compiler and
 standard library.
 
 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and
 simple.
 
 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple
 git scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how
 to start a new release, fix a bug, experiment with a new feature,
 and such.
 
 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something
 to keep in mind.)
 
 Please chime in with ideas and thoughts.


[...]

Hi Andrei,

A number of us have put up a draft of a proposed release process on the
wiki, based on some of the things discussed in this thread.

	http://wiki.dlang.org/Release_Process

It would be *very* helpful if you could chime in with some comments from
your POV as one of the core developers (in fact, it would be very good
if we can have some input from _any_ of the core devs):

	http://wiki.dlang.org/Talk:Release_Process

Right now we think we have some good ideas going, but we don't want to
go too far afield without any input from the core devs, since they are
the ones (mainly) who will be implementing this process.


T

-- 
Three out of two people have difficulties with fractions. -- Dirk Eddelbuettel
Dec 14 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/12 10:02 AM, H. S. Teoh wrote:

 A number of us have put up a draft of a proposed release process on the
 wiki, based on some of the things discussed in this thread.

 	http://wiki.dlang.org/Release_Process


Seen it, will give a thorough read today. Thanks!

Andrei
Dec 14 2012
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 10:16:45AM -0500, Andrei Alexandrescu wrote:

 On 12/14/12 10:02 AM, H. S. Teoh wrote:

A number of us have put up a draft of a proposed release process on
the wiki, based on some of the things discussed in this thread.

	http://wiki.dlang.org/Release_Process


 
 Seen it, will give a thorough read today. Thanks!


[...]

Just to follow up, any comments on what's there currently?

I fear that some of us on this thread may be over-engineering the whole
thing without input from the core devs who will actually be implementing
this process.


T

-- 
My program has no bugs! Only undocumented features...
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 1:45 PM, H. S. Teoh wrote:

 On Fri, Dec 14, 2012 at 10:16:45AM -0500, Andrei Alexandrescu wrote:

 On 12/14/12 10:02 AM, H. S. Teoh wrote:

 A number of us have put up a draft of a proposed release process on
 the wiki, based on some of the things discussed in this thread.

 	http://wiki.dlang.org/Release_Process


 Seen it, will give a thorough read today. Thanks!


 [...]

 Just to follow up, any comments on what's there currently?

 I fear that some of us on this thread may be over-engineering the whole
 thing without input from the core devs who will actually be implementing
 this process.


I was hoping for more consensus to build in this thread. Right now it 
seems there's still quite a bit of controversy about what the best way 
to go is.

Andrei
Dec 19 2012
next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
Alexandrescu wrote:

 I was hoping for more consensus to build in this thread. Right 
 now it seems there's still quite a bit of controversy about 
 what the best way to go is.

 Andrei


We need to list out all the main points of contention and pick 
away at them until there's none left

I suspect that some of it may be more of a misunderstanding than 
real contention, but no doubt a few key points are in contention.

In terms of "core devs", I know about Walter and Andrei, so who 
else is there that has yet to give us any input?

--rt
Dec 19 2012
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
Alexandrescu wrote:

 I was hoping for more consensus to build in this thread. Right 
 now it seems there's still quite a bit of controversy about 
 what the best way to go is.


I'm afraid a lot of discussions we see right now are plain 
useless because overengineered. This shouldn't prevent us from 
switching some more basics things, like the git workflow and see 
how it work.

This process need some actual practical use for the reflection to 
go forward, as it has gone way too far in the stratosphere.
Dec 19 2012
parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 19:26:48 UTC, deadalnix wrote:

 On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
 Alexandrescu wrote:

 I was hoping for more consensus to build in this thread. Right 
 now it seems there's still quite a bit of controversy about 
 what the best way to go is.


 I'm afraid a lot of discussions we see right now are plain 
 useless because overengineered. This shouldn't prevent us from 
 switching some more basics things, like the git workflow and 
 see how it work.

 This process need some actual practical use for the reflection 
 to go forward, as it has gone way too far in the stratosphere.


Yes, overengineering is pointless, and doing something basic is 
better than doing nothing at all, and knowing what will work and 
what won't and what the challenges are means that we have to try 
something out or we'll never learn anything.

There are some main points that I think everyone agrees with that 
could be implemented immediately.

Do we all agree that MASTER becomes the "Dev" branch?

Do we all agree that we need a "testing" (aka staging) branch?

Do we all agree that we need a "stable" branch?

--rt
Dec 19 2012
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:

 On Wednesday, 19 December 2012 at 19:26:48 UTC, deadalnix wrote:

 On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
 Alexandrescu wrote:

 I was hoping for more consensus to build in this thread. 
 Right now it seems there's still quite a bit of controversy 
 about what the best way to go is.


 I'm afraid a lot of discussions we see right now are plain 
 useless because overengineered. This shouldn't prevent us from 
 switching some more basics things, like the git workflow and 
 see how it work.

 This process need some actual practical use for the reflection 
 to go forward, as it has gone way too far in the stratosphere.


 Yes, overengineering is pointless, and doing something basic is 
 better than doing nothing at all, and knowing what will work 
 and what won't and what the challenges are means that we have 
 to try something out or we'll never learn anything.

 There are some main points that I think everyone agrees with 
 that could be implemented immediately.

 Do we all agree that MASTER becomes the "Dev" branch?


Yes


 Do we all agree that we need a "testing" (aka staging) branch?


Yes


 Do we all agree that we need a "stable" branch?


No. Stable isn't a boolean criteria. You'll find different degree 
of stability going from not so stable (dev version) to very 
stable (dead project).

The wiki already mention a process with a branch per version of 
the software.
Dec 19 2012
parent reply "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix wrote:

 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:


 Do we all agree that we need a "stable" branch?


 No. Stable isn't a boolean criteria. You'll find different 
 degree of stability going from not so stable (dev version) to 
 very stable (dead project).

 The wiki already mention a process with a branch per version of 
 the software.


Let's generalize this point for the sake of reaching consensus - 
we need _at least one_ "stable" branch which is separate from 
"staging". We are still conflicted as to what should be the 
maximum amount. For the record, I'm with the camp advocating at 
most a fixed amount countable on one hand. That's an O(1) with a 
very small constant as opposed to the O(n) suggestion by Andrei. 
I hope Andrei appreciates the order of efficiency here.
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 4:23 PM, foobar wrote:

 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix wrote:

 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:


 Do we all agree that we need a "stable" branch?


 No. Stable isn't a boolean criteria. You'll find different degree of
 stability going from not so stable (dev version) to very stable (dead
 project).

 The wiki already mention a process with a branch per version of the
 software.


 Let's generalize this point for the sake of reaching consensus - we need
 _at least one_ "stable" branch which is separate from "staging". We are
 still conflicted as to what should be the maximum amount. For the
 record, I'm with the camp advocating at most a fixed amount countable on
 one hand. That's an O(1) with a very small constant as opposed to the
 O(n) suggestion by Andrei. I hope Andrei appreciates the order of
 efficiency here.


I agree with one "stable" branch.

Andrei
Dec 19 2012
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:

 On 12/19/12 4:23 PM, foobar wrote:

 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix 
 wrote:

 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:


 Do we all agree that we need a "stable" branch?


 No. Stable isn't a boolean criteria. You'll find different 
 degree of
 stability going from not so stable (dev version) to very 
 stable (dead
 project).

 The wiki already mention a process with a branch per version 
 of the
 software.


 Let's generalize this point for the sake of reaching consensus 
 - we need
 _at least one_ "stable" branch which is separate from 
 "staging". We are
 still conflicted as to what should be the maximum amount. For 
 the
 record, I'm with the camp advocating at most a fixed amount 
 countable on
 one hand. That's an O(1) with a very small constant as opposed 
 to the
 O(n) suggestion by Andrei. I hope Andrei appreciates the order 
 of
 efficiency here.


 I agree with one "stable" branch.


This does conflict with the requirement you gave before about 
being able to support anything, as previous stable version cannot 
be revised.

Or does stable here mean supported ? (which means we still have 
branch per version, but only one version is supported)
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 4:40 PM, deadalnix wrote:

 On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei Alexandrescu wrote:

 On 12/19/12 4:23 PM, foobar wrote:

 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix wrote:

 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:


 Do we all agree that we need a "stable" branch?


 No. Stable isn't a boolean criteria. You'll find different degree of
 stability going from not so stable (dev version) to very stable (dead
 project).

 The wiki already mention a process with a branch per version of the
 software.


 Let's generalize this point for the sake of reaching consensus - we need
 _at least one_ "stable" branch which is separate from "staging". We are
 still conflicted as to what should be the maximum amount. For the
 record, I'm with the camp advocating at most a fixed amount countable on
 one hand. That's an O(1) with a very small constant as opposed to the
 O(n) suggestion by Andrei. I hope Andrei appreciates the order of
 efficiency here.


 I agree with one "stable" branch.


 This does conflict with the requirement you gave before about being able
 to support anything, as previous stable version cannot be revised.

 Or does stable here mean supported ? (which means we still have branch
 per version, but only one version is supported)


Walter needs to chime in about that. One possibility is to continue 
using tags for marking releases, and then branch for the few important 
releases that we want to patch.


Andrei
Dec 19 2012
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 19, 2012 at 04:48:22PM -0500, Andrei Alexandrescu wrote:

 On 12/19/12 4:40 PM, deadalnix wrote:

On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei Alexandrescu wrote:

On 12/19/12 4:23 PM, foobar wrote:






[...]

Let's generalize this point for the sake of reaching consensus - we
need _at least one_ "stable" branch which is separate from
"staging". We are still conflicted as to what should be the maximum
amount. For the record, I'm with the camp advocating at most a
fixed amount countable on one hand. That's an O(1) with a very
small constant as opposed to the O(n) suggestion by Andrei. I hope
Andrei appreciates the order of efficiency here.


I agree with one "stable" branch.


This does conflict with the requirement you gave before about being
able to support anything, as previous stable version cannot be
revised.

Or does stable here mean supported ? (which means we still have
branch per version, but only one version is supported)


 
 Walter needs to chime in about that. One possibility is to continue
 using tags for marking releases, and then branch for the few
 important releases that we want to patch.


[...]

This is a good idea, to avoid cluttering the git repo with branches.
(But then again, branches in git are cheap so I don't think this is
really that big of a deal.)


T

-- 
Just because you survived after you did it, doesn't mean it wasn't stupid!
Dec 19 2012
parent "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 21:53:04 UTC, H. S. Teoh wrote:

 On Wed, Dec 19, 2012 at 04:48:22PM -0500, Andrei Alexandrescu 
 wrote:

 On 12/19/12 4:40 PM, deadalnix wrote:

On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:

On 12/19/12 4:23 PM, foobar wrote:






 [...]

Let's generalize this point for the sake of reaching 
consensus - we
need _at least one_ "stable" branch which is separate from
"staging". We are still conflicted as to what should be the 
maximum
amount. For the record, I'm with the camp advocating at 
most a
fixed amount countable on one hand. That's an O(1) with a 
very
small constant as opposed to the O(n) suggestion by Andrei. 
I hope
Andrei appreciates the order of efficiency here.


I agree with one "stable" branch.


This does conflict with the requirement you gave before about 
being
able to support anything, as previous stable version cannot be
revised.

Or does stable here mean supported ? (which means we still 
have
branch per version, but only one version is supported)


 
 Walter needs to chime in about that. One possibility is to 
 continue
 using tags for marking releases, and then branch for the few
 important releases that we want to patch.


 [...]

 This is a good idea, to avoid cluttering the git repo with 
 branches.
 (But then again, branches in git are cheap so I don't think 
 this is
 really that big of a deal.)


 T


This has nothing to do with repository size, git optimizes that 
anyway.

This is a very good decision which I endorse (well, I suggested 
it in the first place..). Having a single stable branch helps 
integration and minimizes human error in the process.
Dec 19 2012
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei 
Alexandrescu wrote:

 Walter needs to chime in about that. One possibility is to 
 continue using tags for marking releases, and then branch for 
 the few important releases that we want to patch.


Note that what is described on the wiki distinguish new version 
(that actually bring new stuff) and revision (that contains only 
bugfixes).

Does having branches that are not used anymore is a problem ? 
They'll not make the repository much bigger because the data is 
in history anyway.
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 5:07 PM, deadalnix wrote:

 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei Alexandrescu wrote:

 Walter needs to chime in about that. One possibility is to continue
 using tags for marking releases, and then branch for the few important
 releases that we want to patch.


 Note that what is described on the wiki distinguish new version (that
 actually bring new stuff) and revision (that contains only bugfixes).

 Does having branches that are not used anymore is a problem ? They'll
 not make the repository much bigger because the data is in history anyway.


I kinda like the idea that we have tags for releases not planned for 
long term support and distinguished branches for those that we do. 
Clarifies intent a lot.

Andrei
Dec 19 2012
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 22:30:29 UTC, Andrei 
Alexandrescu wrote:

 On 12/19/12 5:07 PM, deadalnix wrote:

 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei 
 Alexandrescu wrote:

 Walter needs to chime in about that. One possibility is to 
 continue
 using tags for marking releases, and then branch for the few 
 important
 releases that we want to patch.


 Note that what is described on the wiki distinguish new 
 version (that
 actually bring new stuff) and revision (that contains only 
 bugfixes).

 Does having branches that are not used anymore is a problem ? 
 They'll
 not make the repository much bigger because the data is in 
 history anyway.


 I kinda like the idea that we have tags for releases not 
 planned for long term support and distinguished branches for 
 those that we do. Clarifies intent a lot.


Should something be released if nobody will support it ?

If it is in order to provide overview of what is coming in the 
next version, I guess the best option is to provide a snapshot of 
the staging branch on regular basis (and call it beta or 
whatever).

Reading this answer, I start to think that we have something 
pretty close in mind but fail to explain it to each other.
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 5:43 PM, deadalnix wrote:

 On Wednesday, 19 December 2012 at 22:30:29 UTC, Andrei Alexandrescu wrote:

 On 12/19/12 5:07 PM, deadalnix wrote:

 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei Alexandrescu
 wrote:

 Walter needs to chime in about that. One possibility is to continue
 using tags for marking releases, and then branch for the few important
 releases that we want to patch.


 Note that what is described on the wiki distinguish new version (that
 actually bring new stuff) and revision (that contains only bugfixes).

 Does having branches that are not used anymore is a problem ? They'll
 not make the repository much bigger because the data is in history
 anyway.


 I kinda like the idea that we have tags for releases not planned for
 long term support and distinguished branches for those that we do.
 Clarifies intent a lot.


 Should something be released if nobody will support it ?


My understanding is that's what many projects do. Supporting each minor 
release would make for a ton of work.


 If it is in order to provide overview of what is coming in the next
 version, I guess the best option is to provide a snapshot of the staging
 branch on regular basis (and call it beta or whatever).


No, I'm not talking about previews.


 Reading this answer, I start to think that we have something pretty
 close in mind but fail to explain it to each other.


That's quite possible. Again, I'm not an expert in the matter and turned 
to the community for that reason. What we're looking for is a process 
that makes things better for everyone involved without having too much 
overhead.


Andrei
Dec 19 2012
parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 23:08:59 UTC, Andrei 
Alexandrescu wrote:

 My understanding is that's what many projects do. Supporting 
 each minor release would make for a ton of work.


The whole point is to not support all minor release. Usually 
project supports one or 2 of them, that's it. Nobody win in 
supporting more.

For instance, python is supporting 2.7 and 3.3 .
PHP support 5.3 and 5.4 .
Ruby support 1.9 .
Java support 1.6 and 1.7 .
scala 2.8 and 2.9 .

As you can see, this is very common way of doing thing in the 
programming language world.

I propose to release snapshot of staging in order to allow for 
very few version branches to be created, while still alowing 
people to try the very last D, and so allow to support them for 
an extended period of time without havin to ends up in a 
maintenance hell.


 If it is in order to provide overview of what is coming in the 
 next
 version, I guess the best option is to provide a snapshot of 
 the staging
 branch on regular basis (and call it beta or whatever).


 No, I'm not talking about previews.


Then call let's call it rolling release, release candidate, 
bleeding edge D, unstable, latest, whatever may fit depending on 
how high we raise the bar in terms of stability.
Dec 19 2012
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 22:30:29 UTC, Andrei 
Alexandrescu wrote:

 On 12/19/12 5:07 PM, deadalnix wrote:

 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei 
 Alexandrescu wrote:

 Walter needs to chime in about that. One possibility is to 
 continue
 using tags for marking releases, and then branch for the few 
 important
 releases that we want to patch.


 Note that what is described on the wiki distinguish new 
 version (that
 actually bring new stuff) and revision (that contains only 
 bugfixes).

 Does having branches that are not used anymore is a problem ? 
 They'll
 not make the repository much bigger because the data is in 
 history anyway.


 I kinda like the idea that we have tags for releases not 
 planned for long term support and distinguished branches for 
 those that we do. Clarifies intent a lot.


Let me propose an updated version of branches w have, according 
to what I understand here.  Note that I avoid using the term 
stable because I'm pretty sure it is a source of confusion and 
miscommunication.


master : used as a base for development. New feature are merged 
here.
staging : used to provide a view of what the next version will 
look like. Regular snapshot of that branch are made so public can 
use the last features.
version : used to contain a version that will have a support for 
an extended period of time.

If I understand correctly, version is what you call stable : a 
branch that is detached to provide long term support.
Dec 19 2012
parent reply "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:


 master : used as a base for development. New feature are merged 
 here.
 staging : used to provide a view of what the next version will 
 look like. Regular snapshot of that branch are made so public 
 can use the last features.
 version : used to contain a version that will have a support 
 for an extended period of time.


I do not see how development in master is moved to staging based 
on this description. I'll try and be more specific.

I realize you don't like features, but we need to talk about 
Phobos additions,  property mechanics, and other highly 
disruptive bugs. I'll call it the major-change.

The major-change branch is generally developed to completion and 
may just be a pull requests from developer342's master.

 From the sound of it this request is pulled into master. We 
continue to pull many of these changes in. How do we decide they 
should be placed into staging, when we pull them into master?. If 
we wait for some 'magic time' how do we pull it into staging, 
does that mean we now cherry pick commits of master?

Another issue is it sounds like master becomes a "phantom" 
branch. At no point in time would master resemble what is 
released. I see this as a problem because it is the branch people 
are developing off of, it means nothing to test in master as 
staging has the actual state that will be released.
Dec 19 2012
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips 
wrote:

 On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:


 master : used as a base for development. New feature are 
 merged here.
 staging : used to provide a view of what the next version will 
 look like. Regular snapshot of that branch are made so public 
 can use the last features.
 version : used to contain a version that will have a support 
 for an extended period of time.


 I do not see how development in master is moved to staging 
 based on this description. I'll try and be more specific.

 I realize you don't like features, but we need to talk about 
 Phobos additions,  property mechanics, and other highly 
 disruptive bugs. I'll call it the major-change.

 The major-change branch is generally developed to completion 
 and may just be a pull requests from developer342's master.

 From the sound of it this request is pulled into master. We 
 continue to pull many of these changes in. How do we decide 
 they should be placed into staging, when we pull them into 
 master?. If we wait for some 'magic time' how do we pull it 
 into staging, does that mean we now cherry pick commits of 
 master?

 Another issue is it sounds like master becomes a "phantom" 
 branch. At no point in time would master resemble what is 
 released. I see this as a problem because it is the branch 
 people are developing off of, it means nothing to test in 
 master as staging has the actual state that will be released.


That is very true. With the given modification master and 
stagging become redundant with one another. Why not get rid of 
stagging ?
Dec 19 2012
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 20, 2012 at 05:48:13AM +0100, deadalnix wrote:

 On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips wrote:

On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:


master : used as a base for development. New feature are merged
here.
staging : used to provide a view of what the next version will
look like. Regular snapshot of that branch are made so public
can use the last features.
version : used to contain a version that will have a support for
an extended period of time.






[...]

From the sound of it this request is pulled into master. We
continue to pull many of these changes in. How do we decide they
should be placed into staging, when we pull them into master?. If
we wait for some 'magic time' how do we pull it into staging, does
that mean we now cherry pick commits of master?




In my mind, after a release, the contents of staging are updated to be
exactly the same as master. This can be done either via a merge, or
simply deleting the current staging and making a new one by branching
from master.



Another issue is it sounds like master becomes a "phantom" branch.
At no point in time would master resemble what is released. I see
this as a problem because it is the branch people are developing off
of, it means nothing to test in master as staging has the actual
state that will be released.


 
 That is very true. With the given modification master and stagging
 become redundant with one another. Why not get rid of stagging ?


No, staging does not receive new features except immediately after
release. But it will get bugfixes that do not introduce new things.

Example timeline:
- time=0: we just made a release, so staging := master.

- time=1: a crash bug is discovered in master. Walter checks in a fix.
  Fix is propagated to staging because it doesn't introduce new things,
  only fixes existing problems.

- time=2: Walter decides to merge UDA implementation into master (for
  example). Staging does NOT get UDA merged in, because UDA is new.

- time=3: more bugs are found in master. Fixes are checked in. Fixes get
  propagated to staging.

- time=4: a UDA bug is fixed. Fix is NOT propagated to staging, because
  staging doesn't have UDA (yet).

- time=5: we decide that it's time to release. So:
   - we tag a release on current staging
   - we merge master into staging, so now staging gets UDA

- time=6: Walter adds HalfFloat to master. HalfFloat is NOT propagated
  to staging, because it's new.

- time=7: users running staging find a nasty design flaw in UDA. Walter
  makes a (breaking) fix in master, and the fix is propagated to
  staging, because UDA is already in staging.

- time=8: Walter finds bugs in HalfFloat. Fixes are checked into master,
  but NOT staging.

- time=9: Walter adds more new features to master. These are NOT
  propagated to staging, because they're new.

- time=10: it is decided that UDA design is stable now, and we're ready
  for release. So:
   - we tag a new release on current staging
   - we merge master into staging, now staging gets HalfFloat.

- etc.

Hopefully this makes it clearer what we're trying to achieve.


T

-- 
 Arise, you prisoners of Windows 
 Arise, you slaves of Redmond, Wash, 
 The day and hour soon are coming 
 When all the IT folks say "Gosh!" 
 It isn't from a clever lawsuit 
 That Windowsland will finally fall, 
 But thousands writing open source code 
 Like mice who nibble through a wall.
-- The Linux-nationale by Greg Baker
Dec 19 2012
next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 05:32:30 UTC, H. S. Teoh wrote:

 On Thu, Dec 20, 2012 at 05:48:13AM +0100, deadalnix wrote:

 On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips


 In my mind, after a release, the contents of staging are 
 updated to be
 exactly the same as master. This can be done either via a 
 merge, or
 simply deleting the current staging and making a new one by 
 branching
 from master.


Yes, exactly!


 No, staging does not receive new features except immediately 
 after
 release. But it will get bugfixes that do not introduce new 
 things.

 Example timeline:
 - time=0: we just made a release, so staging := master.

 - time=1: a crash bug is discovered in master. Walter checks in 
 a fix.
   Fix is propagated to staging because it doesn't introduce new 
 things,
   only fixes existing problems.

 - time=2: Walter decides to merge UDA implementation into 
 master (for
   example). Staging does NOT get UDA merged in, because UDA is 
 new.

 - time=3: more bugs are found in master. Fixes are checked in. 
 Fixes get
   propagated to staging.

 - time=4: a UDA bug is fixed. Fix is NOT propagated to staging, 
 because
   staging doesn't have UDA (yet).

 - time=5: we decide that it's time to release. So:
    - we tag a release on current staging
    - we merge master into staging, so now staging gets UDA

 - time=6: Walter adds HalfFloat to master. HalfFloat is NOT 
 propagated
   to staging, because it's new.

 - time=7: users running staging find a nasty design flaw in 
 UDA. Walter
   makes a (breaking) fix in master, and the fix is propagated to
   staging, because UDA is already in staging.

 - time=8: Walter finds bugs in HalfFloat. Fixes are checked 
 into master,
   but NOT staging.

 - time=9: Walter adds more new features to master. These are NOT
   propagated to staging, because they're new.

 - time=10: it is decided that UDA design is stable now, and 
 we're ready
   for release. So:
    - we tag a new release on current staging
    - we merge master into staging, now staging gets HalfFloat.

 - etc.

 Hopefully this makes it clearer what we're trying to achieve.


Yes, fully agreeable with me!


 T


Thanks for writing that out so well. Excellent!

--rt
Dec 19 2012
prev sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Thursday, 20 December 2012 at 05:32:30 UTC, H. S. Teoh wrote:

On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix 
wrote:


master : used as a base for development. New feature are 
merged
here.
staging : used to provide a view of what the next version 
will
look like. Regular snapshot of that branch are made so public
can use the last features.
version : used to contain a version that will have a support 
for
an extended period of time.






 In my mind, after a release, the contents of staging are 
 updated to be
 exactly the same as master. This can be done either via a 
 merge, or
 simply deleting the current staging and making a new one by 
 branching
 from master.


I can get behind this.

deadalnix, can we please go back to calling "version" as 
"release" that is what it is and what everyone else uses I don't 
see a need make a new name because you see "release" as something 
different.
Dec 20 2012
prev sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips 
wrote:

 From the sound of it this request is pulled into master. We 
 continue to pull many of these changes in. How do we decide 
 they should be placed into staging, when we pull them into 
 master?. If we wait for some 'magic time' how do we pull it 
 into staging, does that mean we now cherry pick commits of 
 master?

 Another issue is it sounds like master becomes a "phantom" 
 branch. At no point in time would master resemble what is 
 released. I see this as a problem because it is the branch 
 people are developing off of, it means nothing to test in 
 master as staging has the actual state that will be released.


Most of the D users are not D devs, and they don't want to be 
testing highly unstable features, however some of them will want 
to use reasonably stable new features, even though there may be 
some bugs and there may be some changes. The reason for using the 
testing branch at all, is that they need something in it that's 
not available in the stable branch, so they may take a smallish 
risk and try out the testing or staging branch (need to decide on 
the terms here).

I placed a note in the wiki about the issue of making sure there 
are incentives for people to do the things you wish them to do 
for each major step in the process. If there is no incentives for 
people to test what comes out of development, few people  will 
test it, but you need plenty of testers. The carrot is that a 
staging tester gets a snapshot of what is most current that is 
also reasonably stable, ie., there has to be enough confidence 
that the code is stable enough to use for real, else the risk 
will be too high to bother with it.

I know this because I'm one of those people. I pick and choose 
stable vs
"reasonably" stable based on my needs and the risk assessment. 
You need to be quick about supporting the testing branch and 
careful about not dumping crap into it, it's a delicate balance 
yes, but you *need* testers, and there's only one good way to get 
them, you have to dangle a carrot out and hope they take the 
bait, and then you have to be prepared to help them out when 
things go wrong.

I expect very few people will take a chance on the dev branch 
other than the devs themselves and a few of the experienced 
people who know their way around the compiler, that's just the 
way things will be because it'll be too risky for the regular 
folks to bother with.

The user count will always be lowest for the dev branch, more for 
staging but you want to encourage highest use possible which 
means quick support, and the highest use will be for stable, but 
because it's stable means it's not really being tested because it 
should be mostly bug free, i.e., it's being "used" as opposed to 
being "tested".

I don't think you can afford to drop the testing branch unless 
you think there's no need for very many Average Joe testers, but 
those are the guys who will uncover a ton of unexpected bugs 
missed by the dev testers.

--rt
Dec 19 2012
prev sibling next sibling parent reply "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:

 On 12/19/12 4:23 PM, foobar wrote:

 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix 
 wrote:

 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:


 Do we all agree that we need a "stable" branch?


 No. Stable isn't a boolean criteria. You'll find different 
 degree of
 stability going from not so stable (dev version) to very 
 stable (dead
 project).

 The wiki already mention a process with a branch per version 
 of the
 software.


 Let's generalize this point for the sake of reaching consensus 
 - we need
 _at least one_ "stable" branch which is separate from 
 "staging". We are
 still conflicted as to what should be the maximum amount. For 
 the
 record, I'm with the camp advocating at most a fixed amount 
 countable on
 one hand. That's an O(1) with a very small constant as opposed 
 to the
 O(n) suggestion by Andrei. I hope Andrei appreciates the order 
 of
 efficiency here.


 I agree with one "stable" branch.

 Andrei


That's great!
What about the other high level points summarized eloquently by 
H. S. Teoh?
Any other high level points or goals the process should 
incorporate?

Personally, I think the whole pre-development stage needs to get 
looks at -
Specifically having some sort of at least high-level *binding* 
planning - road map, mile stones, todo lists. This should give 
more weight to DIPs.
DIPs at the moment have no weight whatsoever. The attributes 
showed all the major flows in the decision making process:

1. Feature idea comes up in discussion.
2. Feature was discussed heavily by the community reaching some 
design consensus.
3. Plan is abandoned/forgotten due to Walter's objections - not 
to the design but the idea itself.
4. Feature comes up again in discussion, thus returning to point 
1 above in infinite loop.

** 5. After many cycles spanning years, the community accepts 
that the feature will never be added despite a consensus and 
constant demand.
5.1 Feature is added by surprise with deviations from consensus 
design, optionally integrates purely with other parts of the 
language.
5.1.1 Feature is final!

Another prime example is the auto-ref feature which we are stuck 
with because Walter apparently understood differently from what 
you intended.

This is highly coupled with Walter, making it very hard to agree 
on a high level design and delegate its implementation to other 
developers. As far as I see, No major feature was ever 
implemented by someone other than Walter and he has final say on 
all design decisions.

I agree with Walter's motives, he wants to stabilize the language 
and by resisting to feature suggestions he sets a very high bar 
for any major change. That is a good thing. The problem is that 
it all happens in his head and not in plain sight. The community 
has no idea what his plans are, I'm not convinced that even you 
get all the details. That means no planning ahead and no 
delegation. Had we known that Walter sign off featureX but just 
doesn't have the time to implement it, than it still would be on 
the official *binding* todo list and someone else would be able 
to implement this feature. As the situation now stands, no one 
will bother doing any significant work if it has a high chance of 
being entirely dismissed by Walter.
Dec 19 2012
next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:

 On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
 Alexandrescu wrote:

 On 12/19/12 4:23 PM, foobar wrote:

 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix 
 wrote:

 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:


 Do we all agree that we need a "stable" branch?


 No. Stable isn't a boolean criteria. You'll find different 
 degree of
 stability going from not so stable (dev version) to very 
 stable (dead
 project).

 The wiki already mention a process with a branch per version 
 of the
 software.


 Let's generalize this point for the sake of reaching 
 consensus - we need
 _at least one_ "stable" branch which is separate from 
 "staging". We are
 still conflicted as to what should be the maximum amount. For 
 the
 record, I'm with the camp advocating at most a fixed amount 
 countable on
 one hand. That's an O(1) with a very small constant as 
 opposed to the
 O(n) suggestion by Andrei. I hope Andrei appreciates the 
 order of
 efficiency here.


 I agree with one "stable" branch.

 Andrei


 That's great!
 What about the other high level points summarized eloquently by 
 H. S. Teoh?
 Any other high level points or goals the process should 
 incorporate?

 Personally, I think the whole pre-development stage needs to 
 get looks at -
 Specifically having some sort of at least high-level *binding* 
 planning - road map, mile stones, todo lists. This should give 
 more weight to DIPs.
 DIPs at the moment have no weight whatsoever. The attributes 
 showed all the major flows in the decision making process:

 1. Feature idea comes up in discussion.
 2. Feature was discussed heavily by the community reaching some 
 design consensus.
 3. Plan is abandoned/forgotten due to Walter's objections - not 
 to the design but the idea itself.
 4. Feature comes up again in discussion, thus returning to 
 point 1 above in infinite loop.

 ** 5. After many cycles spanning years, the community accepts 
 that the feature will never be added despite a consensus and 
 constant demand.
 5.1 Feature is added by surprise with deviations from consensus 
 design, optionally integrates purely with other parts of the 
 language.
 5.1.1 Feature is final!

 Another prime example is the auto-ref feature which we are 
 stuck with because Walter apparently understood differently 
 from what you intended.

 This is highly coupled with Walter, making it very hard to 
 agree on a high level design and delegate its implementation to 
 other developers. As far as I see, No major feature was ever 
 implemented by someone other than Walter and he has final say 
 on all design decisions.

 I agree with Walter's motives, he wants to stabilize the 
 language and by resisting to feature suggestions he sets a very 
 high bar for any major change. That is a good thing. The 
 problem is that it all happens in his head and not in plain 
 sight. The community has no idea what his plans are, I'm not 
 convinced that even you get all the details. That means no 
 planning ahead and no delegation. Had we known that Walter sign 
 off featureX but just doesn't have the time to implement it, 
 than it still would be on the official *binding* todo list and 
 someone else would be able to implement this feature. As the 
 situation now stands, no one will bother doing any significant 
 work if it has a high chance of being entirely dismissed by 
 Walter.


The high-level steps as I see them:
1. The community discusses the merits and costs of the suggested 
feature and a consensus must be reached regarding adding such a 
feature.
2. An initial design must be provided - also with consensus. 
including this parts:
2.a. Suggested semantics of the feature - describe as precisely 
as possible
2.b. All interactions with other language features must be 
carefully examined and listed.
2.c An initial syntax agreed upon by the community.
3. Feature is implemented and after reaching some agreed upon 
level of stability is than field tested by the public.
3.1 Based on public feedback, feature design can be refined and 
bugs are fixed until consensus is reached that feature is ready 
to be finalized and merged to the next stable release.

It's vague on purpose - the exact details of how each point above 
is approved is to be decided later. The main principle is that 
approved planned features should be listed on the road map and 
point to their DIP with description of their design. Rejected 
features are also listed as DIPs with the rationale for the 
rejection, this is to avoid wasting time on repeat discussing 
them and requesting them.
Ultimate goal for all this is making the planning process as 
transparent as possible for the community.
Dec 19 2012
prev sibling parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:

 Personally, I think the whole pre-development stage needs to 
 get looks at -
 Specifically having some sort of at least high-level *binding* 
 planning - road map, mile stones, todo lists. This should give 
 more weight to DIPs.
 DIPs at the moment have no weight whatsoever. The attributes 
 showed all the major flows in the decision making process:

 1. Feature idea comes up in discussion.
 2. Feature was discussed heavily by the community reaching some 
 design consensus.
 3. Plan is abandoned/forgotten due to Walter's objections - not 
 to the design but the idea itself.
 4. Feature comes up again in discussion, thus returning to 
 point 1 above in infinite loop.


A big fix is in order there too, it's actually what should be 
fixed first but from what I've seen going on in here if we can 
get something even basic implemented for the development and 
releases that will be a *huge* step forward. Once we have some 
kind of formalized process in place and prove to everyone how 
much better things are because of it, it'll be a first by the 
looks of things, and from that experience the other big holes 
will become easier to pick out and deal with.

I think we have to keep it simple for now, focus on a dev, 
staging, release process, implement something, work out the bugs, 
get people used to having it, and prove the value, then we can 
get on with tackling the other major issues in a similar way.

There's lots of room for improvement ahead, but we need to make 
at least one significant step forward in a successful way before 
we can hope to move on to the next one.

--rt
Dec 19 2012
parent reply "foobar" <foo bar.com> writes:
On Thursday, 20 December 2012 at 05:33:27 UTC, Rob T wrote:

 On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:

 Personally, I think the whole pre-development stage needs to 
 get looks at -
 Specifically having some sort of at least high-level *binding* 
 planning - road map, mile stones, todo lists. This should give 
 more weight to DIPs.
 DIPs at the moment have no weight whatsoever. The attributes 
 showed all the major flows in the decision making process:

 1. Feature idea comes up in discussion.
 2. Feature was discussed heavily by the community reaching 
 some design consensus.
 3. Plan is abandoned/forgotten due to Walter's objections - 
 not to the design but the idea itself.
 4. Feature comes up again in discussion, thus returning to 
 point 1 above in infinite loop.


 A big fix is in order there too, it's actually what should be 
 fixed first but from what I've seen going on in here if we can 
 get something even basic implemented for the development and 
 releases that will be a *huge* step forward. Once we have some 
 kind of formalized process in place and prove to everyone how 
 much better things are because of it, it'll be a first by the 
 looks of things, and from that experience the other big holes 
 will become easier to pick out and deal with.

 I think we have to keep it simple for now, focus on a dev, 
 staging, release process, implement something, work out the 
 bugs, get people used to having it, and prove the value, then 
 we can get on with tackling the other major issues in a similar 
 way.

 There's lots of room for improvement ahead, but we need to make 
 at least one significant step forward in a successful way 
 before we can hope to move on to the next one.

 --rt


I see both as going hand-in-hand, otherwise we have chicken-egg 
problem.
We need a better process to allow more developers to contribute 
code more easily *and* we need better planning to provide 
incentive for new developer to contribute code.
Implementing only the the planning provides incentive but no 
means. Implementing only the process provides the means but not 
the incentive.

While improving git mastery levels is beneficial of itself, it 
brings no benefit to the _users_ if only Walter is still the main 
developer. We end up wasting Walter's time on adjusting to a new 
system for a hypothetical non existent system of many developers, 
when he already has a working system for the actual existing 
situation of being the sole main developer.

We need an *initial high-level* design for both parts. We can 
define and refine the details later and we probably should defer 
it to later and adjust it as we go along based on experience.
Arguing now on the exact time span of a release just wastes mind 
cycles and detracts from the important bits - reaching a 
consensus regarding the goals of the project and defining a 
general workflow that achieves those goals with minimal amount of 
overhead.
Dec 20 2012
parent "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 08:27:22 UTC, foobar wrote:

 I see both as going hand-in-hand, otherwise we have chicken-egg 
 problem.
 We need a better process to allow more developers to contribute 
 code more easily *and* we need better planning to provide 
 incentive for new developer to contribute code.
 Implementing only the the planning provides incentive but no 
 means. Implementing only the process provides the means but not 
 the incentive.

 While improving git mastery levels is beneficial of itself, it 
 brings no benefit to the _users_ if only Walter is still the 
 main developer. We end up wasting Walter's time on adjusting to 
 a new system for a hypothetical non existent system of many 
 developers, when he already has a working system for the actual 
 existing situation of being the sole main developer.

 We need an *initial high-level* design for both parts. We can 
 define and refine the details later and we probably should 
 defer it to later and adjust it as we go along based on 
 experience.
 Arguing now on the exact time span of a release just wastes 
 mind cycles and detracts from the important bits - reaching a 
 consensus regarding the goals of the project and defining a 
 general workflow that achieves those goals with minimal amount 
 of overhead.


I think what was proposed - to fast track this one, and not over 
engineer - is a good idea. That way we can at least get something 
that's far better than before, and be able to more quickly move 
on to the next big problem area which I would agree is what you 
are mentioning. There's lack of planning and how decisions are 
made is very inefficient and arbitrary, and I would also say we 
have a big problem with the specification being poorly managed, 
eg, it's not even available for download, has no process for 
revisions and releases, nothing at all - it's worse than the 
existing development process which we all agree is in need of a 
big overhaul.

Please, let's all agree to deal with that monster next, but it's 
a big problem to solve, and if we try to bundle it in with this 
one, we may not get any accomplished at all. I'd prefer the 
divide and conquer approach.

--rt
Dec 20 2012
prev sibling parent reply "Joseph Cassman" <jc7919 outlook.com> writes:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:

 I agree with one "stable" branch.

 Andrei


Just some food for thought.

In the section about the "Branching model", the wiki currently 
has a staging branch in addition to the master branch. From what 
I understand, the idea seems to be to vet a release on staging 
until it is considered production level and then marked as the 
release.

Another idea could be to keep the quality of the master branch at 
a high level so as to be able to branch into a release at any 
time, directly from master. Before feature branches are merged 
back into master, their quality is vetted so the quality of 
master is maintained.

This idea seems similar to what is used for the vibe.d project 
(http://vibed.org/temp/branch-model-small.png). My apologies if I 
misunderstood their process.

It looks like Xamarin has been using this process for a while and 
it seems to be working for them. 
http://tirania.org/blog/archive/2011/Oct-14.html

Joseph
Dec 20 2012
next sibling parent "sclytrack" <sclytrack simplicity.com> writes:
On Thursday, 20 December 2012 at 23:43:12 UTC, Joseph Cassman 
wrote:

 On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
 Alexandrescu wrote:

 I agree with one "stable" branch.

 Andrei


 Just some food for thought.

 In the section about the "Branching model", the wiki currently 
 has a staging branch in addition to the master branch. From 
 what I understand, the idea seems to be to vet a release on 
 staging until it is considered production level and then marked 
 as the release.

 Another idea could be to keep the quality of the master branch 
 at a high level so as to be able to branch into a release at 
 any time, directly from master. Before feature branches are 
 merged back into master, their quality is vetted so the quality 
 of master is maintained.


I prefer this one. It's simpler, less to memorize. Staging area 
complicates things. I propose to go for a yearly release of the 
stable branches with one year support (In the beginning). So 
development needs to be of super high quality once a year.



 This idea seems similar to what is used for the vibe.d project 
 (http://vibed.org/temp/branch-model-small.png). My apologies if 
 I misunderstood their process.

 It looks like Xamarin has been using this process for a while 
 and it seems to be working for them. 
 http://tirania.org/blog/archive/2011/Oct-14.html

 Joseph
Dec 20 2012
prev sibling parent reply "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 23:43:12 UTC, Joseph Cassman 
wrote:

 Just some food for thought.

 In the section about the "Branching model", the wiki currently 
 has a staging branch in addition to the master branch. From 
 what I understand, the idea seems to be to vet a release on 
 staging until it is considered production level and then marked 
 as the release.

 Another idea could be to keep the quality of the master branch 
 at a high level so as to be able to branch into a release at 
 any time, directly from master. Before feature branches are 
 merged back into master, their quality is vetted so the quality 
 of master is maintained.

 This idea seems similar to what is used for the vibe.d project 
 (http://vibed.org/temp/branch-model-small.png). My apologies if 
 I misunderstood their process.

 It looks like Xamarin has been using this process for a while 
 and it seems to be working for them. 
 http://tirania.org/blog/archive/2011/Oct-14.html

 Joseph


Doesn't that just turn master into staging, and turn the feature 
branches into a diluted and distributed version of master?

If there's no common development branch to work with that 
integrates the most current features together, then how will such 
a thing ever be properly tested before going into a high quality 
common branch?

We also need the ability to stop brand new poorly tested features 
from making their way into a release, so at some point a common 
pre-release branch needs to be frozen from receiving any new 
features so that it can be honed into a high quality product. If 
you use the master branch for such a thing, then no new features 
can go into it, so with master frozen, what common branch is 
available for the devs to merge their new work into?

--rt
Dec 21 2012
parent "foobar" <foo bar.com> writes:
On Friday, 21 December 2012 at 18:34:12 UTC, Rob T wrote:

 On Thursday, 20 December 2012 at 23:43:12 UTC, Joseph Cassman 
 wrote:

 Just some food for thought.

 In the section about the "Branching model", the wiki currently 
 has a staging branch in addition to the master branch. From 
 what I understand, the idea seems to be to vet a release on 
 staging until it is considered production level and then 
 marked as the release.

 Another idea could be to keep the quality of the master branch 
 at a high level so as to be able to branch into a release at 
 any time, directly from master. Before feature branches are 
 merged back into master, their quality is vetted so the 
 quality of master is maintained.

 This idea seems similar to what is used for the vibe.d project 
 (http://vibed.org/temp/branch-model-small.png). My apologies 
 if I misunderstood their process.

 It looks like Xamarin has been using this process for a while 
 and it seems to be working for them. 
 http://tirania.org/blog/archive/2011/Oct-14.html

 Joseph


 Doesn't that just turn master into staging, and turn the 
 feature branches into a diluted and distributed version of 
 master?

 If there's no common development branch to work with that 
 integrates the most current features together, then how will 
 such a thing ever be properly tested before going into a high 
 quality common branch?

 We also need the ability to stop brand new poorly tested 
 features from making their way into a release, so at some point 
 a common pre-release branch needs to be frozen from receiving 
 any new features so that it can be honed into a high quality 
 product. If you use the master branch for such a thing, then no 
 new features can go into it, so with master frozen, what common 
 branch is available for the devs to merge their new work into?

 --rt


Precisely.
I think people just don't understand the purpose of these 
additional branches. The point being - integration.

The general flow of events should be:
1. Developer has cool idea/feature which he explores on a 
separate private "feature branch"
2. During development of the feature the developer can optionally 
collaborate with other developers. This can be done either by 
pulling from other developers' repositories directly or by 
pushing to a branch on github. Either way, this is an ad hoc "my 
new feature" branch.
3. First level of integration - feature is complete and is merged 
into official first level of integration - the "dev" branch 
(consensus was to use master for that)
4. Feature can than be further refined and _integration bugs_ can 
be fixed by the general dev team.
5. When the "dev" branch is considered stable enough by the team 
(exact criteria to be defined later), the changes are merged to 
the _2nd level of integration_ - the "staging" branch. This 
allows for a wider audience to test and provide real-world 
feedback.
6. when the 2nd level of integration is complete, the changes are 
stable enough to be released and the included features finalized.

Since git provides each developer with their own private copy of 
the entire repository there is *no need* to define any official 
processes prior to initial integration. The developers are free 
to collaborate by using ad hoc branches. The only common sense 
recommendation I'd give (again, NOT part of the _official 
process_) is to use meaningful branch names if they are meant to 
be shared with other people.
Dec 21 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 19, 2012 at 02:01:57PM -0500, Andrei Alexandrescu wrote:

 On 12/19/12 1:45 PM, H. S. Teoh wrote:

On Fri, Dec 14, 2012 at 10:16:45AM -0500, Andrei Alexandrescu wrote:

On 12/14/12 10:02 AM, H. S. Teoh wrote:

A number of us have put up a draft of a proposed release process on
the wiki, based on some of the things discussed in this thread.

	http://wiki.dlang.org/Release_Process


Seen it, will give a thorough read today. Thanks!


[...]

Just to follow up, any comments on what's there currently?

I fear that some of us on this thread may be over-engineering the
whole thing without input from the core devs who will actually be
implementing this process.


 
 I was hoping for more consensus to build in this thread. Right now
 it seems there's still quite a bit of controversy about what the
 best way to go is.


[...]

I think we need more input from the core devs before any further
discussions make sense. Like deadalnix said, we've gone way overboard
with the over-engineering; can we at least get the core devs to evaluate
the most basic tenets of the proposal first?

Others can correct me if I'm wrong, but the main points as I understand
them are:

- Continue using master for development, as before.

- New features will be developed in their own dedicated branches
  (branched from master), NOT on master directly. When the new feature
  is deemed ready, by some criterion to be decided on, then it's merged
  into master.

- At some point in the process, perhaps after a release is made from
  staging (see below), the new changes in master are folded into the
  staging branch (we can argue about the exact name of this later).

- The intent of staging is to test new features and possibly breaking
  changes by a wider audience before the official release. It is to be
  used by people who want bleeding-edge features and can live with
  existing code breaking once in a while.

- Staging only receives bugfixes and no new features until it's deemed
  release-ready. It may receive breaking changes if a particular feature
  is deemed unsuitable in its present form and needs to be revised.
  (Said changes are also applied to master, of course.) This is where
  new features are refined from their initial design to their final
  form.

- Once staging is release-ready, a release branch is made from staging,
  then the latest stuff from master is folded into staging.

- A release branch is used for applying critical bugfixes and
  showstopper fixes after a release is made. It does not receive
  low-priority bug fixes, and certainly no new features and no breaking
  changes.

I deliberately left out things like version numbering and timelines,
etc., as those are just bikeshedding. The exact criteria under which
master is folded into staging, and staging is deemed release-ready, are
intentionally left vague. We need the core devs to evaluate whether
these high-level steps in the process are actually workable first,
before we even think about timelines, version numberings, and
merging/branching criteria.


T

-- 
All problems are easy in retrospect.
Dec 19 2012
prev sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
 From the discussion there appears to be more that believe that 
master should remain the development branch.

 From this, 'staging' is seen as the branch to prepare a release 
from (bug fix only).

We are currently in the 2.61 freeze, let us move this freeze to 
the staging branch now!

Any disagreement with this? Does anyone see this changing as we 
continue to work out the flow? All I can see is bikeshedding the 
name.
Dec 14 2012
parent reply "Rob T" <rob ucora.com> writes:
On Friday, 14 December 2012 at 15:36:05 UTC, Jesse Phillips wrote:

 From the discussion there appears to be more that believe that 
 master should remain the development branch.

 From this, 'staging' is seen as the branch to prepare a release 
 from (bug fix only).

 We are currently in the 2.61 freeze, let us move this freeze to 
 the staging branch now!
 Any disagreement with this? Does anyone see this changing as we 
 continue to work out the flow? All I can see is bikeshedding 
 the name.


Yes, we could use 2.61 as the first trial to get things moving 
along, even if it's not 100% ready and agreed on, it seems 
obvious that we'll want to use a "staging branch" to work out the 
bugs while in freeze - I don't think anyone is seriously 
objecting to that, right?

However here's a cautionary note: I read that Walter has specific 
motives right now concerning a timely release for a strategic D 
user that can help move D into a more prominent position, so if 
we're to support a release as smoothly as possible, I wonder if 
now is or is not the time to try this out on 2.61?

My gut says, "of course go for it", because I can't see how 
anything could get worse instead of better by doing so, however 
just mentioning that this release is an important one to 
carefully consider.

BTW, the wiki page is looking very good guys. Well done!!!

--rt
Dec 14 2012
next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Friday, 14 December 2012 at 17:17:16 UTC, Rob T wrote:

 However here's a cautionary note: I read that Walter has 
 specific motives right now concerning a timely release for a 
 strategic D user that can help move D into a more prominent 
 position, so if we're to support a release as smoothly as 
 possible, I wonder if now is or is not the time to try this out 
 on 2.61?


I thought on this and believe it is a good time to make 2.61 
stable. Especially with Walter wanting to keep the feature 
(syntax) in just for them. They can trust it will remain a 
warning for x years.

But let us do this for a shorter than planned stable. This will 
allow us to try out the new process, and be no worse off than 
before.

But let us first get this branch. Ignoring how long this release 
will last we should all be agreeable on continuing development in 
master and fixing release bugs in 'staging'
Dec 14 2012
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 06:17:15PM +0100, Rob T wrote:
[...]

 BTW, the wiki page is looking very good guys. Well done!!!


[...]

Thanks, but we do still have issues to work out, details to fill in,
etc.. We'd love to have your input too!


T

-- 
We've all heard that a million monkeys banging on a million typewriters
will eventually reproduce the entire works of Shakespeare.  Now, thanks
to the Internet, we know this is not true. -- Robert Wilensk
Dec 14 2012
parent "Rob T" <rob ucora.com> writes:
On Friday, 14 December 2012 at 19:01:23 UTC, H. S. Teoh wrote:

 On Fri, Dec 14, 2012 at 06:17:15PM +0100, Rob T wrote:
 [...]

 BTW, the wiki page is looking very good guys. Well done!!!


 [...]

 Thanks, but we do still have issues to work out, details to 
 fill in,
 etc.. We'd love to have your input too!


 T


Yes very true, long ways to go yet. I will try and continue 
contributing! To me, all things considered, this is THE most 
important issue limiting the growth and adoption of D because 
absolutely everything else depends on it running smoothly.

--rt
Dec 14 2012