www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What's wrong with stability, LTS, and not fixing bugs for

reply FeepingCreature <feepingcreature gmail.com> writes:
There's recently been discussion of how we can avoid breaking 
people's code in the process of fixing bugs. I think this 
approach is fundamentally wrongheaded. Sure, deprecations are 
good if you can get them, but it should never happen that a bug 
is unfixed because code depends on it.

I think this is to some extent an inherent problem with the D 
*forum.* In Github and other forum software, you can give 
upvotes, thumbs-up or likes to posts as a way of signalling 
agreement. The D forum, due to its NNTP heritage, does not have 
this capability. This severely hurts us in cases like the 
named-parameter DIP or string formatting DIP, where even if the 
great majority of D users are in favor of it, they will still 
have *less to say* about their favor because agreement simply has 
a smaller rhetorical surface than disagreement - you can disagree 
in many more ways about many more things than agree. As such, in 
a purely volume-driven discussion as you will see on these 
forums, disagreement will always loom large. The preview feature 
process is a small bulwark against this, by allowing people to 
gather experience with a feature *without* having first convinced 
or at any rate outlasted nearly every naysayer on the forums, 
which is why the misunderstandings about -preview=in, even from 
people like Andrei, are so worrying to me. In my opinion, D does 
*not* have too many breaking changes; D has *far too few* 
breaking changes. When I can switch from 2.090.1 to 2.094.0 and 
often only get warnings, I'm more upset than happy - what has 
gone wrong? Have we as a language reached perfection? (We have 
not.)

And no, big separations between major revisions such as LTS 
versions are not a solution, they'd be contributing to the 
problem. You think getting people to upgrade *right now* is hard? 
Just wait until they have a reasonable expectation to let 
upgrades lie for actual years. "If something is painful, do it 
more often"; this aphorism is driven by the experience that doing 
it less often makes it *more* painful. We risk ending up in the 
D1 (Python 2, Perl 5) case where people don't update at all.

But that's not my biggest objection. This is: the purpose of a 
language is the code that people write in it. Are we 
acknowledging that most code in D has already been written? Are 
we giving up on growth? Are we saying "the level of popularity 
that D has currently reached is probably at or beyond the peak of 
usage"? Because if not, if we think that most code written in D 
is still to come, then we are harming people's future experience 
with D at the expense of not harming legacy code. I think this is 
a self-fulfilling prophecy that will kill the language.
Oct 08 2020
next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
wrote:
 There's recently been discussion of how we can avoid breaking 
 people's code in the process of fixing bugs. I think this 
 approach is fundamentally wrongheaded. Sure, deprecations are 
 good if you can get them, but it should never happen that a bug 
 is unfixed because code depends on it.

 [...]
Completely agree (just to overcome the problem you alluded to that agreement is normally silent).
Oct 08 2020
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
wrote:

 I think this is to some extent an inherent problem with the D 
 *forum.* In Github and other forum software, you can give 
 upvotes, thumbs-up or likes to posts as a way of signalling 
 agreement. The D forum, due to its NNTP heritage, does not have 
 this capability.
I don't see a reason why someone couldn't contribute these features to the forum, and if you're using the NNTP interface, you just won't have access to these features.
 But that's not my biggest objection. This is: the purpose of a 
 language is the code that people write in it. Are we 
 acknowledging that most code in D has already been written? Are 
 we giving up on growth? Are we saying "the level of popularity 
 that D has currently reached is probably at or beyond the peak 
 of usage"? Because if not, if we think that most code written 
 in D is still to come, then we are harming people's future 
 experience with D at the expense of not harming legacy code. I 
 think this is a self-fulfilling prophecy that will kill the 
 language.
That's what Scott Meyers said at DConf: most D code is not written yet. Don't make me come back again. Then he did come back again. -- /Jacob Carlborg
Oct 08 2020
parent reply Fred <fred gmail.com> writes:
On Thursday, 8 October 2020 at 12:40:55 UTC, Jacob Carlborg wrote:
 On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
 wrote:
 But that's not my biggest objection. This is: the purpose of a 
 language is the code that people write in it. Are we 
 acknowledging that most code in D has already been written? 
 Are we giving up on growth? Are we saying "the level of 
 popularity that D has currently reached is probably at or 
 beyond the peak of usage"? Because if not, if we think that 
 most code written in D is still to come, then we are harming 
 people's future experience with D at the expense of not 
 harming legacy code. I think this is a self-fulfilling 
 prophecy that will kill the language.
That's what Scott Meyers said at DConf: most D code is not written yet. Don't make me come back again. Then he did come back again. -- /Jacob Carlborg
I don't really agree with this sort of mentality. If you keep breaking code then the most D code won't ever be written. Broken code that doesn't end up being fixed will stay broken. This stuns growth, especially for larger projects. Constantly fixing breaking changes is fine for smaller projects, but its a disaster for large projects.
Oct 09 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 9 October 2020 at 15:16:32 UTC, Fred wrote:
 I don't really agree with this sort of mentality. If you keep 
 breaking code then the most D code won't ever be written. 
 Broken code that doesn't end up being fixed will stay broken. 
 This stuns growth, especially for larger projects. Constantly 
 fixing breaking changes is fine for smaller projects, but its a 
 disaster for large projects.
On CPPCON they discussed what it would take to move from implicit constructors by default to explicit constructors by default. They could do it like this: year 1: deprecate unqualified constructors and issue warnings, so you have to write either "explicit" or "implicit" before all constructors. year 20: make unqualified destructors "explicit" LOL I think there is a difference between a language with millions of users and other languages though. Major versions is a good solution for most languages, especially if all changes are caught at compile-time.
Oct 09 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 9 October 2020 at 15:27:26 UTC, Ola Fosheim Grøstad 
wrote:
 On Friday, 9 October 2020 at 15:16:32 UTC, Fred wrote:
 I don't really agree with this sort of mentality. If you keep 
 breaking code then the most D code won't ever be written. 
 Broken code that doesn't end up being fixed will stay broken. 
 This stuns growth, especially for larger projects. Constantly 
 fixing breaking changes is fine for smaller projects, but its 
 a disaster for large projects.
On CPPCON they discussed what it would take to move from implicit constructors by default to explicit constructors by default. They could do it like this:
Uh, that was wrong, more like this: year 1: deprecate unqualified constructors and issue warnings, so you have to write either "explicit" or "implicit" before all constructors. year 10: make unqualified constructors illegal year 20: make unqualified constructors "explicit"
Oct 09 2020
prev sibling next sibling parent reply kinke <noone nowhere.com> writes:
I fully agree, but must add that I also don't have to maintain a 
codebase where it'd matter much. ;)

What makes matters worse in this regard is the Buildkite CI 
integration. E.g., https://github.com/dlang/dmd/pull/11387 is a 
bugfix but blocked as it breaks 5 3rd-party projects. Frankly, I 
have more important stuff to do than to fix unrelated code to get 
a fix in. Another bugfix, 
https://github.com/dlang/dmd/pull/10124, broke one Buildkite 
project; I've adapted it but still had to wait for over a year 
(incl. rebasings and solving merge conflicts of course) until a 
new tag was finally available and the fix could finally be merged.

Checking the impact of PRs on 3rd-party projects is of course 
valuable, but we need to find a solution for bugfixes breaking 
some of these.
Oct 08 2020
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2020 6:30 AM, kinke wrote:
 What makes matters worse in this regard is the Buildkite CI integration. E.g., 
 https://github.com/dlang/dmd/pull/11387 is a bugfix but blocked as it breaks 5 
 3rd-party projects.
Each 3rd party library in github needs to have a maintainer (preferably the developer of it) who agrees to keep it updated. If there is no maintainer and it breaks, remove support for it from buildkite. buildkite needs to provide the email address of the maintainer in its build log. I don't see any other practical way.
 I have more important stuff to do than to fix unrelated code to get a fix in.
Right on. We all do.
Oct 10 2020
prev sibling next sibling parent reply Guillaume Piolat <first.name guess.com> writes:
On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
wrote:
 "If something is painful, do it more often"; this aphorism is 
 driven by the experience that doing it less often makes it 
 *more* painful. We risk ending up in the D1 (Python 2, Perl 5) 
 case where people don't update at all.
One idea could be, every one or two DMDFE releases, take one -preview flag and make it the default. Else the "ideal" language will grow further apart the language as used in the field. It is a sort of debt. In the times of package manager, all you have to do when your code breaks is issue a minor tag, and dub update your dependencies. Breaking code isn't _that_ bad in that respect. _At the same time_, I will dread the day where builtin complexes are removed, in a year where a particular trendy OS vendor is giving us months of work just to make working software continue running. Churn is _also_ annoying for industrial programmers.
Oct 08 2020
parent FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 8 October 2020 at 15:43:01 UTC, Guillaume Piolat 
wrote:
 On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
 wrote:
 "If something is painful, do it more often"; this aphorism is 
 driven by the experience that doing it less often makes it 
 *more* painful. We risk ending up in the D1 (Python 2, Perl 5) 
 case where people don't update at all.
One idea could be, every one or two DMDFE releases, take one -preview flag and make it the default. Else the "ideal" language will grow further apart the language as used in the field. It is a sort of debt.
As I understand it, the process is to add a preview switch `-preview=feature`, gather experience, make a DIP, then after that is accepted and some time has passed, change the feature to be on by default and optionally reverted with `-revert=feature`. Then after some more time, the feature is removed. I broadly think this is a good process inasmuch as we can get to actually carry it out.
Oct 08 2020
prev sibling next sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
wrote:
 There's recently been discussion of how we can avoid breaking 
 people's code in the process of fixing bugs. I think this 
 approach is fundamentally wrongheaded. Sure, deprecations are 
 good if you can get them, but it should never happen that a bug 
 is unfixed because code depends on it.
I used to think that way. I don't anymore. What changed ? I ended up being on the receiving end. It's fine to "move fast and break things" as long as the people breaking things are the people who will have to fix them. The cost, or burden, is on the same people, and it's those people decision what cost they want to shoulder. But if the people suffering from the change are not the one benefitting from it, then you are just shifting the cost, and those people will eventually leave for a vendor that does not shift the maintenance cost on them. Does no one remember when every single D release was a breaking change?
 I think this is to some extent an inherent problem with the D 
 *forum.* In Github and other forum software, you can give 
 upvotes, thumbs-up or likes to posts as a way of signalling 
 agreement. The D forum, due to its NNTP heritage, does not have 
 this capability. This severely hurts us in cases like the 
 named-parameter DIP or string formatting DIP, where even if the 
 great majority of D users are in favor of it, they will still 
 have *less to say* about their favor because agreement simply 
 has a smaller rhetorical surface than disagreement - you can 
 disagree in many more ways about many more things than agree. 
 As such, in a purely volume-driven discussion as you will see 
 on these forums, disagreement will always loom large. The 
 preview feature process is a small bulwark against this, by 
 allowing people to gather experience with a feature *without* 
 having first convinced or at any rate outlasted nearly every 
 naysayer on the forums, which is why the misunderstandings 
 about -preview=in, even from people like Andrei, are so 
 worrying to me. In my opinion, D does *not* have too many 
 breaking changes; D has *far too few* breaking changes. When I 
 can switch from 2.090.1 to 2.094.0 and often only get warnings, 
 I'm more upset than happy - what has gone wrong? Have we as a 
 language reached perfection? (We have not.)
Well, your first paragraph mentions code depending on bugs, while the second mentions purely additive features. I don't think the purely additive features have been held back by an abundance of conservatism, but more by inertia, which is another problem. I however strongly agree with your feeling w.r.t. rhetorical surface of agreement vs disagreement. Another thing to keep in mind is that a minor, and very specific portion of D users are on the forums. I'm sure it's obvious to you, as I would be ready to bet than less than 10% of the D developers in your company are on the forums - as was the case with Sociomantic. Regarding v2.090.1 => v2.094.0, I'm very happy that I don't get anything else than deprecations, and sometimes warning. It means that the libraries I use, which haven't been touched for years will not randomly break. As much as I love to contribute to D, and as much as I've been involved over the years, working on D doesn't generate income for me. Building software for my company does, and D is just a means to an end. It so happen to be the best tool for the job, but if it wasn't, I'd use something else. Hence, the more things are shifted on me by breaking things, the less I can do my job, and the less D is an efficient tool.
 And no, big separations between major revisions such as LTS 
 versions are not a solution, they'd be contributing to the 
 problem. You think getting people to upgrade *right now* is 
 hard? Just wait until they have a reasonable expectation to let 
 upgrades lie for actual years. "If something is painful, do it 
 more often"; this aphorism is driven by the experience that 
 doing it less often makes it *more* painful. We risk ending up 
 in the D1 (Python 2, Perl 5) case where people don't update at 
 all.
This is an excellent case of: https://upload.wikimedia.org/wikipedia/commons/9/98/Survivorship-bias.png All languages have major revision and provide stability. Rust has editions, Go has forward compatibility, C++ and C do their own slow thing, etc... Having every upgrade break your code is a great way to ensure our users don't return.
 But that's not my biggest objection. This is: the purpose of a 
 language is the code that people write in it. Are we 
 acknowledging that most code in D has already been written? Are 
 we giving up on growth? Are we saying "the level of popularity 
 that D has currently reached is probably at or beyond the peak 
 of usage"? Because if not, if we think that most code written 
 in D is still to come, then we are harming people's future 
 experience with D at the expense of not harming legacy code. I 
 think this is a self-fulfilling prophecy that will kill the 
 language.
I don't see any hint that people are not willing to improve the language. I must admit I was a bit discouraged by the reaction I saw to `-preview=in` in the forum, as opposed to the discussions I had before / the reactions on Github, and had a mild case of "this is why we can't have nice things". Yet, the flag is still there, and looking back at what happened over the last two years, I do not see a stagnating language.
Oct 09 2020
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 9 October 2020 at 10:35:40 UTC, Mathias LANG wrote:
 As much as I love to contribute to D, and as much as I've been 
 involved over the years, working on D doesn't generate income 
 for me. Building software for my company does, and D is just a 
 means to an end.
This is the core point - we don't see our code as "in maintenance mode". We want to write new code, add new features - and nothing is more frustrating than spending days, paid for by the company, to reduce a bug and find out it was deliberately left unfixed, or that the D project left you to sink weeks of effort into a barely-maintained semi-legacy part of the language that you now have to refactor your way out of, cough std.json cough.
Oct 09 2020
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/9/2020 3:58 AM, FeepingCreature wrote:
 This is the core point - we don't see our code as "in maintenance mode". We
want 
 to write new code, add new features - and nothing is more frustrating than 
 spending days, paid for by the company, to reduce a bug and find out it was 
 deliberately left unfixed, or that the D project left you to sink weeks of 
 effort into a barely-maintained semi-legacy part of the language that you now 
 have to refactor your way out of, cough std.json cough.
std.json is still in Phobos, and still passes its unittests. I do not understand what the issue is with it - if you're using it, and it hasn't changed, why do you have to refactor out of it?
Oct 10 2020
prev sibling next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
wrote:
 And no, big separations between major revisions such as LTS 
 versions are not a solution, they'd be contributing to the 
 problem. You think getting people to upgrade *right now* is 
 hard? Just wait until they have a reasonable expectation to let 
 upgrades lie for actual years. "If something is painful, do it 
 more often"; this aphorism is driven by the experience that 
 doing it less often makes it *more* painful. We risk ending up 
 in the D1 (Python 2, Perl 5) case where people don't update at 
 all.
Actually. That is a rather good solution. BTW upgrading from Python 2 would have been easy for me if it wasn't for the fact that service providers who use Python 2 change their service when switching to Python 3. Upgrading from Python 2 to Python 3 is easy. Btw upgrading from an old C code base to C++20 is relatively easy too, but not if you want to upgrade to C++20 libraries. So yeah, major versions work pretty good, if you have access to the same infrastructure after upgrading.
Oct 09 2020
prev sibling next sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 8 October 2020 at 10:37:22 UTC, FeepingCreature 
wrote:
 Sure, deprecations are good if you can get them,
 but it should never happen that a bug is unfixed because code 
 depends on it.
I assume you're talking about https://github.com/dlang/phobos/pull/6716. Now, it took me a bit of time to figure out what it was that you were referring to, exactly. At first, I was just going from the description that you provided, which at face value, does seem quite wrong, and generally does not describe the way we do things. But, it's not just "some code" that would be broken by fixing the bug there. For a lot of projects, Vibe.d is a dependency that's just as important as the standard library. So, to put those alongside each other, would we ever release a version of D which contained a bugfix in the compiler that caused the standard library to fail to compile? No, of course not, that's absurd. And, it's no different in the Vibe.d case. The solution here is to work as a community and fix Vibe.d first, which it seems like someone tried doing, though unsuccessfully.
 This is the core point - we don't see our code as "in 
 maintenance mode". We want to write new code, add new features 
 - and nothing is more frustrating than spending days, paid for 
 by the company, to reduce a bug and find out it was 
 deliberately left unfixed, or that the D project left you to 
 sink weeks of effort into a barely-maintained semi-legacy part 
 of the language that you now have to refactor your way out of, 
 cough std.json cough.
I think it would definitely take less than a few days to fix the bug in Vibe.d and test that it was, in fact, fixed. Actually, I would like to recommend using Digger to make this task even easier: In Vibe.d's directory, you can run `digger run Vibe.d using the changes in your PR. In any case, including the full details of your grievances would help understand your position better. Otherwise, in the spirit of game theory, one would be prudent to assume that you are exaggerating or misleading through omission, as doing so would benefit your case.
 D has *far too few* breaking changes.
This attitude works if you have a finite amount of code to maintain, and can take all the time you need to do so. The problem occurs as the amount of code you need to maintain grows. An exaggerated illustration: 2010: you have 10000 lines of code, and it takes 10 hours to update. 2015: you have 100000 lines of code, and it takes 100 hours to update. 2020: you have 1000000 lines of code, and it takes 1000 hours to update. At a certain point you need to give up on writing new projects, because you no longer have time to do so. That time has been eaten up by maintaining your existing projects. Now, if you are in a situation where you maintain a fixed amount of code at a time, and are not under any time pressure that would disincentivize maintaining code quality, then I see how frequent iterations in the language would be welcome. I guess this would be the case for R&D or data science, where many programs are written, executed once, then deleted. An opposite case would be trying to maintain a large and ever-growing number of projects and services - not only is there the cost of actually maintaining the project, but also the cost of task switching, as each project has its own environment and requirements. (That's putting aside the adage that while a project is used then it will need to be maintained anyway - in my experience, a program really can be "finished", at least to some degrees where breaking language changes become a considerable nuisance.) Now, so far D has been rather moderate as far as breaking changes go, and I'm very thankful for that (I've done my share of complaining to ensure it). If breaking changes were to come as often as some people would have liked, I probably would have stopped using the language a long time ago. After all, what's the point of writing code in it if nobody will be able to run it in five years?
 And no, big separations between major revisions such as LTS 
 versions are not a solution, they'd be contributing to the 
 problem. You think getting people to upgrade *right now* is 
 hard? Just wait until they have a reasonable expectation to let 
 upgrades lie for actual years. "If something is painful, do it 
 more often"; this aphorism is driven by the experience that 
 doing it less often makes it *more* painful.
You seem to be working on the assumption that everyone writes and maintains code in the same manner than you do. This isn't true. But, regardless of that, when maintaining many projects, it is definitely favorable to bulk small updates per project, because of the high cost of task switching. So, again, what you wrote is only true for your use case. The rest of your post seems to be based on similar assumptions. Kindly consider that different users of D use D differently, and thus have different priorities and desires for how the language ought to evolve. I would hope that perhaps the tone of your future messages will take this into consideration.
 We risk ending up in the D1 (Python 2, Perl 5) case where 
 people don't update at all.
You can still run Python 2 code. I still can just install it and run it. Even as the Python foundation stopped maintaining it, third-party vendors continue to support it. You certainly can still run Perl 5 code. I don't really know the situation with newer Perl versions, but as I understand a newer language version was not in as high demand, and the majority of Perl code probably has already been written. You cannot still run D1 code, at least not easily. Even if you can get the compiler binary itself running on your system, odds are that the binaries that it will produce will not link or run. This, I think, is a silent tragedy: how much code have we rewritten in vain just because this time it's based on slightly newer technology, and how many times?
Oct 09 2020
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
The idea is we don't want to break documented behavior that people used in good 
faith, unless we *have* to. If people are exploiting a bug in the compiler, I'm 
not terribly sympathetic to that, and fixing that bug is not a problem. We
don't 
*have* to remove the old operator new and delete overloads. We can
"un-document" 
it, i.e. move its documentation to a separate section of the documentation, and 
encourage people to leave it behind.

I don't see how this takes anything away from D.
Oct 10 2020
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2020 3:37 AM, FeepingCreature wrote:
 I think this is to some extent an inherent problem with the D *forum.* In
Github 
 and other forum software, you can give upvotes, thumbs-up or likes to posts as
a 
 way of signalling agreement. The D forum, due to its NNTP heritage, does not 
 have this capability.
The D community has a large diversity of members that run the gamut of experience levels. Should a person who just compiled his first D program yesterday have a vote counting as much as, say, yours? Democracy is fine for politics, not for engineering. Would you feel safer on an airplane designed by the general public voting on how thick to make the wing spar, or by experienced aerospace engineers? I look for compelling, well-founded opinions by people who are long-time contributors with a track record of demonstrated expertise. People who have a stake in D, who care about it and will have to live with the results. (That said, I still welcome perspectives by people new to D, as they may bring expertise to the table that none of us have. Sometimes the best ideas come from outsiders.) It goes without saying that reviews consisting of nothing more than +1 are not compelling. At least give some sort of rationale.
Oct 10 2020
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2020 3:37 AM, FeepingCreature wrote:
 But that's not my biggest objection. This is: the purpose of a language is the 
 code that people write in it. Are we acknowledging that most code in D has 
 already been written? Are we giving up on growth? Are we saying "the level of 
 popularity that D has currently reached is probably at or beyond the peak of 
 usage"? Because if not, if we think that most code written in D is still to 
 come, then we are harming people's future experience with D at the expense of 
 not harming legacy code. I think this is a self-fulfilling prophecy that will 
 kill the language.
I honestly don't know where this is coming from. There are a number of proposals in the DIP queue. Stefan is working on type functions. I'm working on an alternative method to speed up template metaprogramming. Steven is outlining a Phobos std2 design. I'm working on an ownership/borrowing addition. As for the named parameter DIP, it was approved. My string formatting DIP was rejected, even though I strongly feel it is the right move for D, because nobody else seemed to want it. There is a long list of -preview switches implemented.
 When I can switch from 2.090.1 to 2.094.0 and often only get warnings, I'm 
more upset than happy I interpret that as we're doing a good job not breaking peoples' code :-/
Oct 10 2020