www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Thoughts on versioning

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Versioning Phobos would free us from maintaining backward compatibility 
with a variety of decisions that did not withstand the test of time:

- autodecoding
- liberal use of the GC
- antediluvian run-time support library API (Object, Typeinfo, ...)
- liberal throwing of exceptions
- many others

Goals of library versioning:

- avoid copy-and-paste code duplication across versions
- support mixed use of versions for incremental migration
- allow library writers to structure code in a versioning-friendly manner

Non-goals:

- migrate code that was not written for versioning in a black-box manner 
(i.e. without touching it)
- support arbitrarily radical API changes with no effort

To open the discussion, here's a simple example: say we want to port the 
mismatch function in std.algorithm.comparison to a future std2 version 
that removes autodecoding. (I'll use the two-parameters version for 
simplicity.)

Tuple!(Range1, Range2)
mismatch(alias pred = "a == b", Range1, Range2)(Range1 r1, Range2 r2)
if (isInputRange!(Range1) && isInputRange!(Range2))
{
     for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
     {
         if (!binaryFun!(pred)(r1.front, r2.front)) break;
     }
     return tuple(r1, r2);
}

This same source code would work with or without autodecoding. It all 
depends on how the primitives front and popFront are implemented. If 
pasted within an std2 version without autodecoding, it would just work.

Of course, we don't want to paste it twice, so the challenge is to have 
the source code in a single place and use it both in std, with 
auto-decoding, and std2, without autodecoding.

One naive solution would be to simply instruct mismatch to look up 
front, popFront etc. in the same namespace in which it's called, much 
like a C-style macro. This is called "dynamic lookup" and its use is 
largely discredited at least in statically-typed languages.

So there's a need to parameterize mismatch with the namespace in which 
lookup is to be done. For example:


// In std/algorithm/comparison.d:

template mismatch(alias the_std)
{
     import the_std.typecons : Tuple, tuple;
     import the_std.range.primitives : isInputRange, empty, front, popFront;
     Tuple!(Range1, Range2)
     mismatch(alias pred = "a == b", Range1, Range2)(Range1 r1, Range2 r2)
     if (isInputRange!(Range1) && isInputRange!(Range2))
     {
         for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
         {
             if (!binaryFun!(pred)(r1.front, r2.front)) break;
         }
         return tuple(r1, r2);
     }
}

Then:

// In std/algorithm/comparison.d:
alias mismatch = mismatch!std;

// In std2/algorithm/comparison.d:
alias mismatch = mismatch!std2;

Pros:

- no duplication of implementation
- freedom to pull an old name in a new version, or simply redefine it

Cons:

- one line of scaffolding per name introduced; can't do bulk without a 
bit of support compile-time reflectioncode
- does not support easy migration of directory structure - what if we 
want to reorganized the modules such that Tuple is no longer in 
std2.typecons?
- does not work within the current language

About the last point: currently the language does allow passing a 
package name as an alias, but trying to import the_std.typecons ignores 
any alias definition and opens the hardcoded path "the_std/typcons.d". 
This would need to be changed. If breaking changes are to be allowed we 
need to introduce a new syntax such as:

     import alias(the_std).typecons : Tuple, tuple;

There are many other aspects to discuss, but I'll keep this short so the 
discussion doesn't meander too much.
Oct 25 2021
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
Alternatively:

mixin template StandardAlgorithmComparison() {
     Tuple!(Range1, Range2) mismatch(alias pred = "a == b", Range1, 
Range2)(Range1 r1, Range2 r2)
     if (isInputRange!(Range1) && isInputRange!(Range2))
     {
         for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
         {
             if (!binaryFun!(pred)(r1.front, r2.front)) break;
         }
         return tuple(r1, r2);
     }
}

In std2.algorithm.comparison:

mixin StandardAlgorithmComparison;

In std.algorithm.comparison:

mixin StandardAlgorithmComparison;
Oct 25 2021
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/25/21 9:41 PM, rikki cattermole wrote:

 
 Alternatively:
 
 mixin template StandardAlgorithmComparison() {
      Tuple!(Range1, Range2) mismatch(alias pred = "a == b", Range1, 
 Range2)(Range1 r1, Range2 r2)
      if (isInputRange!(Range1) && isInputRange!(Range2))
      {
          for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
          {
              if (!binaryFun!(pred)(r1.front, r2.front)) break;
          }
          return tuple(r1, r2);
      }
 }
 
 In std2.algorithm.comparison:
 
 mixin StandardAlgorithmComparison;
 
 In std.algorithm.comparison:
 
 mixin StandardAlgorithmComparison;


Good stuff. A great plus is that it works within the existing languages. 
Other pros and cons?
Oct 25 2021
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 26/10/2021 3:04 PM, Andrei Alexandrescu wrote:

 Good stuff. A great plus is that it works within the existing languages. 
 Other pros and cons?


My concerns with this approach would be auto completion and documentation.

If the documentation engine doesn't see past the mixin template and 
include the members transparently, then that it is a bug.

If the auto completion can't see into the mixin template that is a bug 
too. But one that may not be fixable without changing to dmd-fe and that 
is something the core developers should be owning: IDE integrations.
Oct 25 2021
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/25/21 10:04 PM, Andrei Alexandrescu wrote:

 On 10/25/21 9:41 PM, rikki cattermole wrote:

 Alternatively:

 mixin template StandardAlgorithmComparison() {
      Tuple!(Range1, Range2) mismatch(alias pred = "a == b", Range1, 
 Range2)(Range1 r1, Range2 r2)
      if (isInputRange!(Range1) && isInputRange!(Range2))
      {
          for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
          {
              if (!binaryFun!(pred)(r1.front, r2.front)) break;
          }
          return tuple(r1, r2);
      }
 }

 In std2.algorithm.comparison:

 mixin StandardAlgorithmComparison;

 In std.algorithm.comparison:

 mixin StandardAlgorithmComparison;


 
 Good stuff. A great plus is that it works within the existing languages. 
 Other pros and cons?


Found a con:

- Cannot use local imports at all; the mixin must do all of its lookup 
in the top namespace.
Oct 25 2021
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 26/10/2021 3:10 PM, Andrei Alexandrescu wrote:

 Found a con:
 
 - Cannot use local imports at all; the mixin must do all of its lookup 
 in the top namespace.


This works:

void main() {
     FooBar foobar;
     foobar.zar();
}

struct FooBar {
     import std;

     mixin Zar;

     void mar() {
         writeln("ugh");
     }
}

mixin template Zar() {
     void zar() {
         writeln("hi");
         mar();
     }
}

What is not working for you?
Oct 25 2021
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/25/21 10:21 PM, rikki cattermole wrote:

 What is not working for you?


Most Phobos functions import std.something locally, as they should. 
Those imports would need to be all hoisted to top level for the mixin to 
work. Wouldn't want to lose the advantages of local imports.
Oct 25 2021
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 26/10/2021 3:29 PM, Andrei Alexandrescu wrote:

 On 10/25/21 10:21 PM, rikki cattermole wrote:

 What is not working for you?


 
 Most Phobos functions import std.something locally, as they should. 
 Those imports would need to be all hoisted to top level for the mixin to 
 work. Wouldn't want to lose the advantages of local imports.


Ah yes, importing inside a function body.

Hmm, does anyone know what the cost is of a named import if unused is? 
That could resolve this issue.
Oct 25 2021
prev sibling next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 10/25/21 6:19 PM, Andrei Alexandrescu wrote:


 template mismatch(alias the_std)
 {
      import the_std.typecons : Tuple, tuple;


[...]

 }


[...]

 About the last point: currently the language does allow passing a
 package name as an alias, but trying to import the_std.typecons ignores
 any alias definition and opens the hardcoded path "the_std/typcons.d".


Sorry to stray from the main point but I needed that missing feature in 
the past: "This is a string template parameter but I don't want to use a 
string at call site." (Similar to how non-string uses of opDispatch are 
presented as strings.)

For example, Flag!"foo" does not look good. Instead, I want to say 
Flag!foo. I just thought about the following syntaxes:

a) template mismatch(auto mixin the_std) { /* ... */ }

b) template mismatch(string mixin the_std) { /* ... */ }

c) Something like that

So, at the call site:

   mismatch!std2;

So, the_std parameter would be the string literal "std2", and it would 
be automatically mixed in like a string mixin.

I know that we cannot mixin at that granularity but maybe this is a 
useful idea.

Ali
Oct 25 2021
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/25/21 9:59 PM, Ali Çehreli wrote:

 On 10/25/21 6:19 PM, Andrei Alexandrescu wrote:
 
  > template mismatch(alias the_std)
  > {
  >      import the_std.typecons : Tuple, tuple;
 [...]
  > }
 [...]
  > About the last point: currently the language does allow passing a
  > package name as an alias, but trying to import the_std.typecons ignores
  > any alias definition and opens the hardcoded path "the_std/typcons.d".
 
 Sorry to stray from the main point but I needed that missing feature in 
 the past: "This is a string template parameter but I don't want to use a 
 string at call site." (Similar to how non-string uses of opDispatch are 
 presented as strings.)
 
 For example, Flag!"foo" does not look good. Instead, I want to say 
 Flag!foo.


Walter and I discussed this a long time ago, several times, under the 
proposal "new alias":

template Flag(new alias name) { ... }

If a template takes a "new alias" it means it takes an alias that does 
not refer to anything that currently exists. Inside the template the 
alias can be used to introduce a name or as a string.
Oct 25 2021
prev sibling next sibling parent reply zjh <fqbqrr 163.com> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 Versioning Phobos would free us from maintaining backward


can we improve this
```d
if (isInputRange!(Range1) && isInputRange!(Range2))
```
to:
```d
if isallInputRange(Range1,Range2)
```
?
`C++`'s `concept` is very grace.
Oct 25 2021
next sibling parent reply zjh <fqbqrr 163.com> writes:
On Tuesday, 26 October 2021 at 04:58:41 UTC, zjh wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei


We can use a method similar to the `C++` namespace without 
`mixing` the old code with the new code. Use an `import std2` to 
open the `std2` space. The `new code` is placed in the `std2` 
directory. The `new code` copies all the `old code` first, and 
then `modifies and tests` on it.
The old `std` code remains as it is, and it is maintained and 
modified.
There is no need to `mix` the new code with the old code, 
otherwise, The function is too ugly.
However, we can say to the user that the maintenance period of 
the `old std` is "10 years / 5 years" ,then we will delete the 
old code and only maintain the "new code" as "std". If the user 
continues to use the "old code", it will be maintained by the 
user.
Preferably, you can also optimize the import, such as `import 
std2: {algorithm, file, array}` To batch import modules. with 
this simplifiction, users are naturally willing to migrate.
Oct 25 2021
parent zjh <fqbqrr 163.com> writes:
Or, if you don't want users to worry, you should use the 
following in the `std` Code:
Make the original `std` like `version std = std1`, and the 
current `std` like `version std = std2`.
I think we should do it at the `std` level, not at the `function` 
level, or we should do it one by one like `fun!std`, what a big 
hole.
Oct 25 2021
prev sibling next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 10/25/21 9:58 PM, zjh wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu wrote:

 Versioning Phobos would free us from maintaining backward


 
 can we improve this
 ```d
 if (isInputRange!(Range1) && isInputRange!(Range2))
 ```
 to:
 ```d
 if isallInputRange(Range1,Range2)
 ```
 ?
 `C++`'s `concept` is very grace.
 
 


Easy in D but the following otherwise-attractive-short-hand-method 
imports symbols at module level (at least privately):

private import std.meta : allSatisfy;
private import std.range : isInputRange;

alias isallInputRange(T...) = allSatisfy!(isInputRange, T);

void foo(A, B)(A a, B b)
if (isallInputRange!(A, B)) {
}

unittest {
   static assert ( __traits(compiles, foo([0], [0])));
   static assert (!__traits(compiles, foo([0], 42)));
   static assert (!__traits(compiles, foo(42, [0])));
}

void main() {
}

Ali
Oct 25 2021
parent zjh <fqbqrr 163.com> writes:
On Tuesday, 26 October 2021 at 05:59:44 UTC, Ali Çehreli wrote:

 On 10/25/21 9:58 PM, zjh wrote:


Thank you for your reply:
```d
void foo(A, B)(A a, B b)
if (isallInputRange!(A, B)) {
}
```
That's good. But I hope we can be better. Like this:
```d
void foo(ConceptA A,ConceptB B)(A a, B b)
     if (ConceptAB!(A, B)) {
     ...
}
```
I hope that when `conceptAB` is not needed but 
`conceptA/conceptB` is needed, it will be added directly before 
`A/B`.
And the `C++`'s `...`  is very cool, but `manu` was defeated by 
`Walter`. Sometimes it's better to expand directly than to use 
`array`, Why doesn't `Walter` accept it?
Oct 25 2021
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 12:58 AM, zjh wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu wrote:

 Versioning Phobos would free us from maintaining backward


 
 can we improve this
 ```d
 if (isInputRange!(Range1) && isInputRange!(Range2))
 ```
 to:
 ```d
 if isallInputRange(Range1,Range2)
 ```
 ?
 `C++`'s `concept` is very grace.


There's https://github.com/dlang/phobos/pull/8303 that does something 
similar along with making mismatch variadic.
Oct 26 2021
prev sibling next sibling parent reply sarn <sarn theartofmachinery.com> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 Goals of library versioning:

 - avoid copy-and-paste code duplication across versions


This might be overengineering.  Unless new features are going to 
be backported to the original std, is there anything wrong with 
just forking std to std2?
Oct 25 2021
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 26 October 2021 at 06:03:38 UTC, sarn wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei 
 Alexandrescu wrote:

 Goals of library versioning:

 - avoid copy-and-paste code duplication across versions


 This might be overengineering.  Unless new features are going 
 to be backported to the original std, is there anything wrong 
 with just forking std to std2?


Yeah, I kind of agree there, just copy the stuff. Simple and 
effective; no crazy mixins or module aliases.
Oct 26 2021
next sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Tuesday, 26 October 2021 at 07:04:31 UTC, Sebastiaan Koppe 
wrote:

 On Tuesday, 26 October 2021 at 06:03:38 UTC, sarn wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei 
 Alexandrescu wrote:

 Goals of library versioning:

 - avoid copy-and-paste code duplication across versions


 This might be overengineering.  Unless new features are going 
 to be backported to the original std, is there anything wrong 
 with just forking std to std2?


 Yeah, I kind of agree there, just copy the stuff. Simple and 
 effective; no crazy mixins or module aliases.


Yeah, this "no copy paste" is a bad requirement that is 
stonewalling the proposal.

If the concern is about backporting bug fixes, git can handle 
that semi-automatically (and it can be fully automated in many 
cases by a PR hook script) since it knows the common ancestor.
Oct 26 2021
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 3:04 AM, Sebastiaan Koppe wrote:

 On Tuesday, 26 October 2021 at 06:03:38 UTC, sarn wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu wrote:

 Goals of library versioning:

 - avoid copy-and-paste code duplication across versions


 This might be overengineering.  Unless new features are going to be 
 backported to the original std, is there anything wrong with just 
 forking std to std2?


 
 Yeah, I kind of agree there, just copy the stuff. Simple and effective; 
 no crazy mixins or module aliases.


Indeed that's simple and would allow trivial concurrent use of the two 
libraries.

I foresee the following challenges:

* Evolution to future versions. Does copying the whole codebase and 
tweaking it scale to e.g. annual releases of Phobos? Five years from now 
we'll have essentially five copypastas. How do you maintain them?

* For maintenance across versions, Adam proposed in discord that we 
leverage git patching. You copy the codebase, master is the most recent, 
and bugfixes are cherry-picked into previous releases (which are copies 
of the git tree at defined points in time). I am unclear (as others on 
discord seem to be) about how well this will work and what to do when it 
breaks - e.g. cherry-picking fails etc. Judging by an old refactoring PR 
that I'm trying to rebase - 
https://github.com/dlang/phobos/pull/7638/files - it is nigh impossible. 
And that's just a refactoring that doesn't aim to change semantincs, 
unlike phobos evolution. I am worried this will become a huge drag that 
will discourage us from doing maintenance as much as we are not 
discouraged from doing evolution. We should aim for both.
Oct 26 2021
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 26 October 2021 at 14:41:38 UTC, Andrei Alexandrescu 
wrote:

 I foresee the following challenges:

 * Evolution to future versions. Does copying the whole codebase 
 and tweaking it scale to e.g. annual releases of Phobos? Five 
 years from now we'll have essentially five copypastas. How do 
 you maintain them?


Simple: release the new version on code.dlang.org, and don't 
merge it into Phobos proper until its API is stable and it's had 
time to fully bake. No need for annual releases, no need for 
multiple rounds of copy-pasta.

As far as I can tell all past experience should tell us that 
Phobos is a bad place to develop new code, and code.dlang.org is 
a good one. Consider `sumtype`: it began development in 2018 as a 
dub package, and was merged into Phobos in 2021 after reaching 
version 1.0. Meanwhile, `std.experimental.allocator` was started 
several years *before* `sumtype` (git history goes back to 2015), 
but still has not reached a stable, finished state.
Oct 26 2021
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 10:53 AM, Paul Backus wrote:

 On Tuesday, 26 October 2021 at 14:41:38 UTC, Andrei Alexandrescu wrote:

 I foresee the following challenges:

 * Evolution to future versions. Does copying the whole codebase and 
 tweaking it scale to e.g. annual releases of Phobos? Five years from 
 now we'll have essentially five copypastas. How do you maintain them?


 
 Simple: release the new version on code.dlang.org, and don't merge it 
 into Phobos proper until its API is stable and it's had time to fully 
 bake. No need for annual releases, no need for multiple rounds of 
 copy-pasta.
 
 As far as I can tell all past experience should tell us that Phobos is a 
 bad place to develop new code, and code.dlang.org is a good one. 
 Consider `sumtype`: it began development in 2018 as a dub package, and 
 was merged into Phobos in 2021 after reaching version 1.0. Meanwhile, 
 `std.experimental.allocator` was started several years *before* 
 `sumtype` (git history goes back to 2015), but still has not reached a 
 stable, finished state.


That definitely should be the way for any new component to be added. For 
incremental evolution of the entire library (e.g. no autodecoding, 
reduce use of the GC and exceptions), putting the code elsewhere has the 
same issues as copypasta.
Oct 26 2021
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 26 October 2021 at 14:43:19 UTC, Andrei Alexandrescu 
wrote:

 I regret I forgot to mention in the initial post that we're not 
 looking at v2. We're looking at v2, v3, v4, ... released e.g. 
 at an annual or trienal pace. It's the having a fish vs. 
 knowing how to fish problem.


That does change things. Obviously the copy/paste approach isn't 
going to work then.

Whenever people have to maintain several versions of a product 
what they often do is to implement the old version in terms of 
the new. From the outside it seems version 1 is still available 
but internally it is just version 2 + the quirks of version 1. 
That might not be possible everywhere, especially if the newer 
versions stray too much.

It is not much different from what you propose, except that it 
doesn't penalize all the code to have extra version boilerplate 
around it (mixins, alias module). That might not be possible in 
certain cases because of a too intrusive change, where you can't 
write version 1 with version 2, and you either have to copy it 
(if it is small) or wrap it in version/behavior-selection code.

You can end up in a situation where most of version 1 is 
implemented with version 2, except for a few annoying bits here 
and there. Which would also be acceptable as long as it isn't too 
much. The unanswered questions are "what is too much?", "how to 
measure/detect", "how to mitigate?".

Ultimately it depends a lot on how well you can extract generic 
code usable for both versions. If the ratio generic code to 
version-specific quirk code is favorable then it can definitely 
work.

It goes without saying, do this all in one repo please.
Oct 27 2021
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 27 October 2021 at 07:44:19 UTC, Sebastiaan Koppe 
wrote:

 Whenever people have to maintain several versions of a product 
 what they often do is to implement the old version in terms of 
 the new. From the outside it seems version 1 is still available 
 but internally it is just version 2 + the quirks of version 1. 
 That might not be possible everywhere, especially if the newer 
 versions stray too much.


This is can be problematic when you have advanced introspection. 
This could potentially ruin future language advances (more 
powerful introspection).

I don't think the community should assume that what works for 
other languages will serve D well.

Basically, maintaining multiple versions of a standard lib for a 
language that is evolving is overreaching. You get another layer 
of issues in addition to the ones you already have.
Oct 27 2021
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/27/21 3:44 AM, Sebastiaan Koppe wrote:

 On Tuesday, 26 October 2021 at 14:43:19 UTC, Andrei Alexandrescu wrote:

 I regret I forgot to mention in the initial post that we're not 
 looking at v2. We're looking at v2, v3, v4, ... released e.g. at an 
 annual or trienal pace. It's the having a fish vs. knowing how to fish 
 problem.


 
 That does change things. Obviously the copy/paste approach isn't going 
 to work then.
 
 Whenever people have to maintain several versions of a product what they 
 often do is to implement the old version in terms of the new. From the 
 outside it seems version 1 is still available but internally it is just 
 version 2 + the quirks of version 1. That might not be possible 
 everywhere, especially if the newer versions stray too much.
 
 It is not much different from what you propose, except that it doesn't 
 penalize all the code to have extra version boilerplate around it 
 (mixins, alias module). That might not be possible in certain cases 
 because of a too intrusive change, where you can't write version 1 with 
 version 2, and you either have to copy it (if it is small) or wrap it in 
 version/behavior-selection code.
 
 You can end up in a situation where most of version 1 is implemented 
 with version 2, except for a few annoying bits here and there. Which 
 would also be acceptable as long as it isn't too much. The unanswered 
 questions are "what is too much?", "how to measure/detect", "how to 
 mitigate?".
 
 Ultimately it depends a lot on how well you can extract generic code 
 usable for both versions. If the ratio generic code to version-specific 
 quirk code is favorable then it can definitely work.
 
 It goes without saying, do this all in one repo please.


Thanks. Lots of good stuff here.

I'll set out to put together a WIP PR in phobos showcasing a couple of 
artifacts migration. It will look ugly because I'll need to supplant 
language support with string mixins and other tricks.

Then you and others could improve it and/or provide alternatives.
Oct 27 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/26/21 2:03 AM, sarn wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu wrote:

 Goals of library versioning:

 - avoid copy-and-paste code duplication across versions


 This might be overengineering.  Unless new features are going to be 
 backported to the original std, is there anything wrong with just 
 forking std to std2?


After suggesting and exploring probably 4 different mechanisms, 
including template mixins (which almost worked), I have come to the same 
conclusion. Just copy and backport fixes when needed. What you end up 
with if you ever did find a mechanism is subtle differences in the 
API/code are going to be littered with version(v2) or version(v3) or 
whatnot everywhere, making the code hard to follow and hard to review. 
You end up with the situation Walter doesn't like W.R.T. `#define`s.

Those situations where the code is exactly the same are annoying to have 
to copy. But automation can help maintain all of that. It would be nice 
to focus on standardized tools that do this for you.

For cases where the original code still works exactly the same (and 
unfortunately in Phobos there are going to be few of these), then just 
public import the symbol.

-Steve
Oct 26 2021
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 26 October 2021 at 13:52:01 UTC, Steven Schveighoffer 
wrote:

 [snip]

 After suggesting and exploring probably 4 different mechanisms, 
 including template mixins (which almost worked), I have come to 
 the same conclusion. Just copy and backport fixes when needed. 
 [snip]


I think the copy and paste approach needs to flesh out the exact 
procedure that needs to happen at the start of development on 
that release. For instance, should the std folder be copy/pasted 
with the result re-named `std_v_x_xx`. This would break code if 
people stay on std. So maybe the new `std` should be `std2` with 
folders saved as `std2_v_x_xx`?

What about creating aliases for each year's release? So for 
instance, `alias std_yyyy = std_v_x_xx;`? And then increment it 
as new releases come out each year, until the final one. It would 
be nice to alias an entire module or package, but otherwise would 
need to have some automated approach to add in the aliases.
Oct 26 2021
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 1:57 PM, jmh530 wrote:

 On Tuesday, 26 October 2021 at 13:52:01 UTC, Steven Schveighoffer wrote:

 [snip]

 After suggesting and exploring probably 4 different mechanisms, 
 including template mixins (which almost worked), I have come to the 
 same conclusion. Just copy and backport fixes when needed. [snip]


 
 I think the copy and paste approach needs to flesh out the exact 
 procedure that needs to happen at the start of development on that 
 release. For instance, should the std folder be copy/pasted with the 
 result re-named `std_v_x_xx`. This would break code if people stay on 
 std. So maybe the new `std` should be `std2` with folders saved as 
 `std2_v_x_xx`?
 
 What about creating aliases for each year's release? So for instance, 
 `alias std_yyyy = std_v_x_xx;`? And then increment it as new releases 
 come out each year, until the final one. It would be nice to alias an 
 entire module or package, but otherwise would need to have some 
 automated approach to add in the aliases.


It seems to me the copypasta-and-cherry-pick approach is "well-tried, 
used by most, takes huge effort and doesn't quite work".

We have a really nice namespace isolation with D and we could leverage 
it. I would want to give that a solid shake of the stick before throwing 
our hands and forking 340 KLOC.

One more thing - forking and copying is a one way street. Once we enter 
there, it's difficult to back off. In contrast if we have another 
solution we can always change our mind and fall back to forking.
Oct 26 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 26 October 2021 at 19:53:37 UTC, Andrei Alexandrescu 
wrote:

 One more thing - forking and copying is a one way street. Once 
 we enter there, it's difficult to back off. In contrast if we 
 have another solution we can always change our mind and fall 
 back to forking.


If you fork on code.dlang.org, you can change your mind any time 
you want. The only decision that's irreversible is deciding to 
ship something as part of the `std` package in the official D 
distribution.
Oct 26 2021
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 5:30 PM, Paul Backus wrote:

 On Tuesday, 26 October 2021 at 19:53:37 UTC, Andrei Alexandrescu wrote:

 One more thing - forking and copying is a one way street. Once we 
 enter there, it's difficult to back off. In contrast if we have 
 another solution we can always change our mind and fall back to forking.


 
 If you fork on code.dlang.org, you can change your mind any time you 
 want. The only decision that's irreversible is deciding to ship 
 something as part of the `std` package in the official D distribution.


Agreed. I conflated the moment of "forking" with the moment of "releasing".

The core issue remains the same. Consider e.g. we fork there and we set 
out to eliminate autodecoding in the new version. Once we release that, 
we have two trees in two repos on our hands. Back to versioning hell.
Oct 26 2021
next sibling parent reply zjh <fqbqrr 163.com> writes:
On Tuesday, 26 October 2021 at 21:59:33 UTC, Andrei Alexandrescu 
wrote:

We can also learn Microsoft's `A` version and `W` version.
Then the `A` version code calls the `W` version one.
Here, `A` is the old version. `W` is the new version.
Oct 26 2021
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Wednesday, 27 October 2021 at 01:17:54 UTC, zjh wrote:

 On Tuesday, 26 October 2021 at 21:59:33 UTC, Andrei 
 Alexandrescu wrote:

 We can also learn Microsoft's `A` version and `W` version.
 Then the `A` version code calls the `W` version one.
 Here, `A` is the old version. `W` is the new version.


This is unrelated. The A and W API are funadementally there 
because it's a C API.
No mangling, no overloads. Different names are required.
Oct 26 2021
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 27 October 2021 at 01:23:44 UTC, Basile B. wrote:

 The A and W API are funadementally there because it's a C API.


That's irrelevant. They do actually overload it with macros.

The point is the old functions - the A ones - are kept around but 
simply forward to the new functions - the W ones - after doing 
some compatibility transformations.

That is a viable path for Phobos. The existing std.x functions 
can forward to std.v2.x as appropriate.
Oct 26 2021
next sibling parent reply bauss <jj_1337 live.dk> writes:
On Wednesday, 27 October 2021 at 01:36:24 UTC, Adam D Ruppe wrote:

 On Wednesday, 27 October 2021 at 01:23:44 UTC, Basile B. wrote:

 The A and W API are funadementally there because it's a C API.


 That's irrelevant. They do actually overload it with macros.

 The point is the old functions - the A ones - are kept around 
 but simply forward to the new functions - the W ones - after 
 doing some compatibility transformations.

 That is a viable path for Phobos. The existing std.x functions 
 can forward to std.v2.x as appropriate.


Yes and no, if it's done subtle and you don't know which 
functions change it could possibly lead to silent bugs.
Oct 26 2021
parent Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 27 October 2021 at 06:10:46 UTC, bauss wrote:

 Yes and no, if it's done subtle and you don't know which 
 functions change it could possibly lead to silent bugs.


Keep the existing unittests with the compatibility shim.

If they pass, it is good enough.
Oct 27 2021
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 9:36 PM, Adam D Ruppe wrote:

 On Wednesday, 27 October 2021 at 01:23:44 UTC, Basile B. wrote:

 The A and W API are funadementally there because it's a C API.


 
 That's irrelevant. They do actually overload it with macros.
 
 The point is the old functions - the A ones - are kept around but simply 
 forward to the new functions - the W ones - after doing some 
 compatibility transformations.
 
 That is a viable path for Phobos. The existing std.x functions can 
 forward to std.v2.x as appropriate.


It's interesting that both you and Sebastiaan mention an "inverted 
delta" of sorts whereby current std is redone to use std2.

The converse approach would be to do minimal changes on std and write 
the modified behavior in std2.

What are the comparative pros and cons of the two approaches?
Oct 27 2021
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 27 October 2021 at 13:57:50 UTC, Andrei 
Alexandrescu wrote:

 It's interesting that both you and Sebastiaan mention an 
 "inverted delta" of sorts whereby current std is redone to use 
 std2.


Well, generally new functions end up being more fundamental than 
old, so this is just naturally simpler.

In the case of the Windows functions, the Unicode functions can 
take any string the ANSI functions can, which is not true vice 
versa. So naturally, you'd be forced to write the less functional 
one in terms of the more functional one.

In the case of various Phobos modules:

* Removing autodecoding in v2 makes it more general. You can 
always write an autodecode function in terms of a plain function 
- just do a wrapper that adds the decode adapter.

Doing it the other way happens to be possible here too, using a 
byCodeUnit wrapper, but generally it is easier to add something 
in a wrapper than remove something that was already added.

* Some people hate std.socket because it uses classes. Well, if 
you build a std2.socket on top of it, those people will still be 
angry. But std2.socket could be a struct api and then the 
original std.socket class is built on top of it.

(in that case it would probably be a waste of time; just keeping 
the original std.socket unmodified - it is already a fairly thin 
layer on the underlying C functions anyway - and writing 
std2.socket right on top of the underlying C functions too is 
easy enough.)

* A generalization of std.socket: you can always build GC 
functions on top of  nogc foundations (e.g. `string 
toLower(string s) { return s.asLowerCase.array; } `). Not true 
other way around.

* If you want to trim down an interface, again, you can define 
std.v2 to be minimal then layer on stuff for compatibility on 
top. Users who stick to std.v2, importing just it and not the old 
one, they will not pay the cost of carrying these functions 
around.

But if you defined std.v2 on top of std.v1, it is impossible to 
remove functions. If a user imports std.v2, they automatically 
drag all the std.v1 baggage along too, whether they like it or 
not.

* Just as a matter of merge conflicts, making the bulk of your 
edits on the forward moving version is a more natural way to work.

The con here is maintaining compatibility with the old one is a 
bit harder since you're making edits and the old one might rely 
on it. But.... worse case you just copy/paste when the divergence 
is too much and lock in the old version letting the new one 
evolve.
Oct 27 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/27/21 10:22 AM, Adam D Ruppe wrote:


 * Removing autodecoding in v2 makes it more general. You can always 
 write an autodecode function in terms of a plain function - just do a 
 wrapper that adds the decode adapter.


I wish this were true. But there is a fundamental problem with 
autodecode -- Phobos claims, for example, that a `string` has no 
indexing, yet you can index it. `hasLength!string` is false, yet 
`someString.length` works. There are a large amount of places where you 
can use strings, that you can't use some wrapped autodecoding range. 
Want to implicitly cast a `string` to a `const(char)[]`? No problem! Try 
doing this with a custom type.

There is literally no way to reproduce the abomination known as an 
autodecoding for backwards compatibility. I tried.

And given that a major focus of v2 is going to be ridding the library of 
autodecoding, it means a huge segment of code will need to be redone. I 
was actually kind of surprised how much I had to change when I was 
trying to do it. It gets to the point where you are thinking "why are we 
trying to save this code? Let's just rewrite it".

-Steve
Oct 27 2021
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven 
Schveighoffer wrote:

 There is literally no way to reproduce the abomination known as 
 an autodecoding for backwards compatibility. I tried.


in that case realistically you can't make a v2 act sane to 
strings when built on v1 either, meaning the only choice is to 
rewrite (aka copy/paste aka fork) as needed.

I'm ok with that.
Oct 27 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Oct 27, 2021 at 03:06:10PM +0000, Adam D Ruppe via Digitalmars-d wrote:

 On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:

 There is literally no way to reproduce the abomination known as an
 autodecoding for backwards compatibility. I tried.


 
 in that case realistically you can't make a v2 act sane to strings
 when built on v1 either, meaning the only choice is to rewrite (aka
 copy/paste aka fork) as needed.
 
 I'm ok with that.


Couldn't we just do something like this?

	// Phobos v1
	auto someStringFunc(R)(R range)
	{
		static if (willAutoDecode!R) {
			// old v1 code here
		} else {
			// thin wrapper over v2 code
		}
	}


T

-- 
It is widely believed that reinventing the wheel is a waste of time; but I
disagree: without wheel reinventers, we would be still be stuck with wooden
horse-cart wheels.
Oct 27 2021
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/27/21 12:33 PM, H. S. Teoh wrote:

 On Wed, Oct 27, 2021 at 03:06:10PM +0000, Adam D Ruppe via Digitalmars-d wrote:

 On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:

 There is literally no way to reproduce the abomination known as an
 autodecoding for backwards compatibility. I tried.


 in that case realistically you can't make a v2 act sane to strings
 when built on v1 either, meaning the only choice is to rewrite (aka
 copy/paste aka fork) as needed.

 I'm ok with that.


 
 Couldn't we just do something like this?
 
 	// Phobos v1
 	auto someStringFunc(R)(R range)
 	{
 		static if (willAutoDecode!R) {
 			// old v1 code here
 		} else {
 			// thin wrapper over v2 code
 		}
 	}


I'm not seeing the point. Why wrap v2 code when you are going to have to 
maintain the v1 code anyway?

Plus, now it's a template that accepts ranges. Like it or not, 
`someStringFunc(string)` is not the same as `someStringFunc(R)(R)`.

-Steve
Oct 27 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Oct 27, 2021 at 01:04:25PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:

 On 10/27/21 12:33 PM, H. S. Teoh wrote:

 On Wed, Oct 27, 2021 at 03:06:10PM +0000, Adam D Ruppe via Digitalmars-d wrote:

 On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:

 There is literally no way to reproduce the abomination known as
 an autodecoding for backwards compatibility. I tried.


 
 in that case realistically you can't make a v2 act sane to strings
 when built on v1 either, meaning the only choice is to rewrite
 (aka copy/paste aka fork) as needed.
 
 I'm ok with that.


 
 Couldn't we just do something like this?
 
 	// Phobos v1
 	auto someStringFunc(R)(R range)
 	{
 		static if (willAutoDecode!R) {
 			// old v1 code here
 		} else {
 			// thin wrapper over v2 code
 		}
 	}


 
 I'm not seeing the point. Why wrap v2 code when you are going to have
 to maintain the v1 code anyway?


I thought the you guys were arguing for making Phobos v1 a thin wrapper
that forwards to v2 code.  If you wanted to retain a copy of v1 code,
then this problem wouldn't come up at all?



 Plus, now it's a template that accepts ranges. Like it or not,
 `someStringFunc(string)` is not the same as `someStringFunc(R)(R)`.


[...]

Point.  So what, does that mean we're stuck with keeping v1 code around
forever?

Honestly, I think a more practical approach is to give up the
perfectionist ideal of old code compiling forever (the D community
simply does not have the resources to maintain such a thing, unlike,
say, MS Windows), and just do a best-efforts v1 emulation in v2 code,
that will mostly work but may have a few rare cases where it won't
compile, etc.. Then users just change the few bits that broke, and can
continue enjoying v2 benefits.

//

Alternatively, here's a different approach: make v1 code a dub package,
then break upstream Phobos at will to make v2.

Well OK, maybe have a transitional period where we maintain a copy of v1
code around, but with big fat warnings that within N months/years/etc.
you will need to use the v1 dub package to continue compiling the code,
or migrate to v2 now.  Once the transition period is over, we shed v1
code from upstream, but keep the dub package around for people who need
it. There will be no more new features, only non-breaking bugfixes.


T

-- 
It only takes one twig to burn down a forest.
Oct 27 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/27/21 1:16 PM, H. S. Teoh wrote:

 On Wed, Oct 27, 2021 at 01:04:25PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:

 I'm not seeing the point. Why wrap v2 code when you are going to have
 to maintain the v1 code anyway?


 
 I thought the you guys were arguing for making Phobos v1 a thin wrapper
 that forwards to v2 code.  If you wanted to retain a copy of v1 code,
 then this problem wouldn't come up at all?


The point of making a v2 that is separate from v1 is that you can 
include a library that depends on v1 in the same application as one that 
depends on v2, and all work together.

Autodecoding is one of the worst problems, but there are so many more. 
I'll note there is a strong focus in this thread on how to make this 
maintainable from a library side. But not a lot of focus on how usable 
it would be to actually depend on both libraries at once.

Note that if Autodecoding was ascribed to a *custom type* and not just a 
char array, this probably isn't even close to as big an issue.


 Plus, now it's a template that accepts ranges. Like it or not,
 `someStringFunc(string)` is not the same as `someStringFunc(R)(R)`.


 [...]
 
 Point.  So what, does that mean we're stuck with keeping v1 code around
 forever?


I'm fine with just migrating everything to v2 and if you still want v1, 
you go get the old compiler, and good luck to you.

This is what I started with 
[here](https://forum.dlang.org/post/rhbon6$16ve$1 digitalmars.com), but 
it was decided [this was 
undesirable](https://forum.dlang.org/post/rhkf7p$1sto$1 digitalmars.com), so 
I spent a bunch of time trying to figure out how to make tools that help 
with maintaining a v2/v1 split in the same code base, without success. 
Not to say that a solution isn't possible, I just couldn't find one that 
seemed reasonable. Every time I thought I had something that would work, 
I'd run into a critical problem that destroyed it.

Some things that we toyed with, but never implemented, were ways to 
parameterize imports (i.e. same source code, but treated as 2 different 
imports with the package symbols automatically migrated). This I think 
was the most promising, but it is hard to play with if you don't know 
how to augment the compiler.


 Alternatively, here's a different approach: make v1 code a dub package,
 then break upstream Phobos at will to make v2.


I think having dub packages for each major phobos version would be 
ideal. Then you select the package you depend on (and e.g. phobos v2 
could depend on v1 if you want to share code). You still need tools to 
automatically change the imports and things like that (these have to be 
in different packages).

That is, if we want to maintain all-time backwards compatibility.

-Steve
Oct 27 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 27 October 2021 at 17:48:49 UTC, Steven 
Schveighoffer wrote:

 I'll note there is a strong focus in this thread on how to make 
 this maintainable from a library side. But not a lot of focus 
 on how usable it would be to actually depend on both libraries 
 at once.


Exactly. How can one make this work without breaking 
introspection and type equality?
Oct 27 2021
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/27/21 12:33 PM, H. S. Teoh wrote:

 On Wed, Oct 27, 2021 at 03:06:10PM +0000, Adam D Ruppe via Digitalmars-d wrote:

 On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:

 There is literally no way to reproduce the abomination known as an
 autodecoding for backwards compatibility. I tried.


 in that case realistically you can't make a v2 act sane to strings
 when built on v1 either, meaning the only choice is to rewrite (aka
 copy/paste aka fork) as needed.

 I'm ok with that.


 
 Couldn't we just do something like this?
 
 	// Phobos v1
 	auto someStringFunc(R)(R range)
 	{
 		static if (willAutoDecode!R) {
 			// old v1 code here
 		} else {
 			// thin wrapper over v2 code
 		}
 	}


I think it's much easier than that. And no static if or version. I 
should have the time to put together a prototype during the weekend.
Oct 27 2021
prev sibling parent reply zjh <fqbqrr 163.com> writes:
On Wednesday, 27 October 2021 at 01:17:54 UTC, zjh wrote:

 On Tuesday, 26 October 2021 at 21:59:33 UTC, Andrei


I think we still have to `find a way` in language,
If we can implement something similar to `using std = std2021;`, 
In this way, rename `libnames` at the `package level`. In this 
way, the user's modification amount should not be too large. At 
the same time, if it is not modified, it will have no effect.
We can default that the `top-level` directory in each code base 
is the `package name`. We can switch the library version by 
setting the `alias` of the `package name`.
Oct 26 2021
parent reply zjh <fqbqrr 163.com> writes:
On Wednesday, 27 October 2021 at 01:58:07 UTC, zjh wrote:

 On Wednesday, 27 October 2021 at 01:17:54 UTC, zjh wrote:


while the old `std`, you only need to add `using std = std2021;`, 
Then change it to our forwarding code(compile time forwarding).
Through compile time forwarding, the user's code speed should not 
change.
Oct 26 2021
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 10:08 PM, zjh wrote:

 On Wednesday, 27 October 2021 at 01:58:07 UTC, zjh wrote:

 On Wednesday, 27 October 2021 at 01:17:54 UTC, zjh wrote:


 
 while the old `std`, you only need to add `using std = std2021;`, Then 
 change it to our forwarding code(compile time forwarding).
 Through compile time forwarding, the user's code speed should not change.
 


We tried a number of approaches along these lines, we couldn't make 
anything work. My learning bit has been that you must modify code to 
make it versionable.
Oct 27 2021
parent zjh <fqbqrr 163.com> writes:
On Wednesday, 27 October 2021 at 13:56:14 UTC, Andrei 
Alexandrescu wrote:

 On 10/26/21 10:08 PM, zjh wrote:


Yes, we must change the `language`. But as long as it is confined 
within a `small range`.
I think it can `satisfy` the user, because the user only needs a 
small amount of modification to obtain the incremental function, 
which is worth it for the user.
For example, `ahead of the module`, in `sc.ini`, or in 
`dub.sdl/dub.json`, We can add a version declaration. I think we 
can minimize the modification to resolve the `branch version`.
In fact, as long as the `version statement` is added to `std` and 
forwarded to like `std2`, the `user code` that depends on `std` 
may not be destroyed.
At the same time, don't be afraid of change. The "big change" 
when the number of users is small is better than the "big change" 
when the number of users is large.
However, we should strive for the understanding of the users.
Oct 27 2021
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 26 October 2021 at 21:59:33 UTC, Andrei Alexandrescu 
wrote:

 Agreed. I conflated the moment of "forking" with the moment of 
 "releasing".

 The core issue remains the same. Consider e.g. we fork there 
 and we set out to eliminate autodecoding in the new version. 
 Once we release that, we have two trees in two repos on our 
 hands. Back to versioning hell.


It seems to me like you only really end up in versioning hell if, 
after releasing v2 in Phobos (and committing to no further 
breaking changes), you go on to develop v3 (with new breaking 
changes) in the same repo. Otherwise, you can just mirror changes 
from the Phobos version to the dub version.

I think the root of the problem is, releasing N versions of the 
Phobos API means you are committing  to maintaining N versions of 
the Phobos API, more-or-less indefinitely. And with annual or 
even triennial releases, this maintenance burden will quickly 
grow beyond what the D community can handle.

As Steven Schveighoffer points out [1], having all N versions in 
the same tree does not really save you here--it gets you out of 
versioning hell, but you end up in #ifdef hell instead.

So, what if we didn't release N versions? What if we let the new 
library evolve through v2, v3, etc. on code.dlang.org, and only 
released it into Phobos when we were well and truly done with it, 
and willing to commit to supporting its API long-term?

[1] https://forum.dlang.org/post/sl9161$2ntu$1 digitalmars.com
Oct 27 2021
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/27/21 9:47 AM, Paul Backus wrote:

 On Tuesday, 26 October 2021 at 21:59:33 UTC, Andrei Alexandrescu wrote:

 Agreed. I conflated the moment of "forking" with the moment of 
 "releasing".

 The core issue remains the same. Consider e.g. we fork there and we 
 set out to eliminate autodecoding in the new version. Once we release 
 that, we have two trees in two repos on our hands. Back to versioning 
 hell.


 
 It seems to me like you only really end up in versioning hell if, after 
 releasing v2 in Phobos (and committing to no further breaking changes), 
 you go on to develop v3 (with new breaking changes) in the same repo. 
 Otherwise, you can just mirror changes from the Phobos version to the 
 dub version.


On the contrary, I see repeated mirroring of changes hell, and mindful 
departure by a mix of reuse and rewriting much easier.

At least there is evidence in the way everybody maintains versions now 
that mirroring them changes is NOT easy. It remains to be seen whether 
better language support for versioning will improve the matter; I am 
hopeful it does.


 I think the root of the problem is, releasing N versions of the Phobos 
 API means you are committing  to maintaining N versions of the Phobos 
 API, more-or-less indefinitely. And with annual or even triennial 
 releases, this maintenance burden will quickly grow beyond what the D 
 community can handle.


We can and we should establish a retirement schedule.

Minimizing effort is exactly why I'm trying to improve on the status 
quo. Do we agree that extant methods of maintaining multiple versions is 
difficult?


 As Steven Schveighoffer points out [1], having all N versions in the 
 same tree does not really save you here--it gets you out of versioning 
 hell, but you end up in #ifdef hell instead.


There is no ifdef hell. The new version of a library either redoes an 
artifact or reuses it. Really the new version is a delta over the old.


 So, what if we didn't release N versions? What if we let the new library 
 evolve through v2, v3, etc. on code.dlang.org, and only released it into 
 Phobos when we were well and truly done with it, and willing to commit 
 to supporting its API long-term?
 
 [1] https://forum.dlang.org/post/sl9161$2ntu$1 digitalmars.com


I think breaking std would be a disaster.
Oct 27 2021
prev sibling next sibling parent reply bauss <jj_1337 live.dk> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 There are many other aspects to discuss, but I'll keep this 
 short so the discussion doesn't meander too much.


The only problem I see when versioning the standard library is 
that some packages will rely on specific versions of the standard 
library and it could make it difficult to use packages that rely 
on different versions of the standard library.

In general I agree it would be a great idea, but we have to be 
careful about fragmenting things too much.
Oct 25 2021
parent reply Ogi <ogion.art gmail.com> writes:
On Tuesday, 26 October 2021 at 06:05:08 UTC, bauss wrote:

 The only problem I see when versioning the standard library is 
 that some packages will rely on specific versions of the 
 standard library and it could make it difficult to use packages 
 that rely on different versions of the standard library.


You’re saying it like it’s not a big deal but I’m afraid it’s 
going to be a huge source of pain. Imagine not being able to get 
some `DateTime`, or `Complex`, or `File` from one package and 
pass it to another because one depends on `std1` and the other 
depends on `std2`. Or imagine dealing with different versions of 
the same exception class. It appears to me that versioning the 
standard library is a bullet train straight to dependency hell.
Oct 28 2021
next sibling parent bauss <jj_1337 live.dk> writes:
On Thursday, 28 October 2021 at 11:30:18 UTC, Ogi wrote:

 On Tuesday, 26 October 2021 at 06:05:08 UTC, bauss wrote:

 The only problem I see when versioning the standard library is 
 that some packages will rely on specific versions of the 
 standard library and it could make it difficult to use 
 packages that rely on different versions of the standard 
 library.


 You’re saying it like it’s not a big deal but I’m afraid it’s 
 going to be a huge source of pain. Imagine not being able to 
 get some `DateTime`, or `Complex`, or `File` from one package 
 and pass it to another because one depends on `std1` and the 
 other depends on `std2`. Or imagine dealing with different 
 versions of the same exception class. It appears to me that 
 versioning the standard library is a bullet train straight to 
 dependency hell.


Yes you're probably right that it is a much greater issue than I 
first claimed.

I should probably have emphasized that it is a massive issue!
Oct 28 2021
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/28/21 7:30 AM, Ogi wrote:

 On Tuesday, 26 October 2021 at 06:05:08 UTC, bauss wrote:

 The only problem I see when versioning the standard library is that 
 some packages will rely on specific versions of the standard library 
 and it could make it difficult to use packages that rely on different 
 versions of the standard library.


 
 You’re saying it like it’s not a big deal but I’m afraid it’s going to 
 be a huge source of pain. Imagine not being able to get some `DateTime`, 
 or `Complex`, or `File` from one package and pass it to another because 
 one depends on `std1` and the other depends on `std2`. Or imagine 
 dealing with different versions of the same exception class. It appears 
 to me that versioning the standard library is a bullet train straight to 
 dependency hell.


Yes, this is a part that is not being considered. We have clues as to 
how hellish it will be with experience in using Windows DLLs which have 
their own independent copy of the runtime (and all the "same but not 
same" `TypeInfo` instances).

And there is the problem with `string` actually being the same type in 
both libraries but doing completely different things (autodecoding).

I'm wondering, if we did things like you do database migrations, if it 
might help (with the former problem). That is, if you pass a 
`std.stdio.File` to a function accepting a `std2.stdio.File`, the 
compiler is given a way to convert the internals/state. Something like 
that might make things palatable.

-Steve
Oct 28 2021
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 28 October 2021 at 14:12:44 UTC, Steven 
Schveighoffer wrote:

 is, if you pass a `std.stdio.File` to a function accepting a 
 `std2.stdio.File`, the compiler is given a way to convert the 
 internals/state.


Well, if there was implicit construction, you could always make a 
constructor for the one File that accepts the other and converts.

Of course, the user could also just call a transition function 
you define.

If you're willing to do a little work you can make it happen.
Oct 28 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/28/21 10:18 AM, Adam D Ruppe wrote:

 On Thursday, 28 October 2021 at 14:12:44 UTC, Steven Schveighoffer wrote:

 is, if you pass a `std.stdio.File` to a function accepting a 
 `std2.stdio.File`, the compiler is given a way to convert the 
 internals/state.


 
 Well, if there was implicit construction, you could always make a 
 constructor for the one File that accepts the other and converts.


This is going to still be a bit painful. I was thinking for more 
compiler help.

For example, if you had:

```d
import lib1; // uses Phobos 1 File
import lib2; // uses Phobos 2 File

void foo()
{
    lib2.funAcceptingFile(lib1.funProducingFile());
}
```

This is going to have an error, instead you need to do:

```d
void foo()
{
    import std2.stdio : File;
    lib2.funAcceptingFile(File(lib1.funProducingFile()));
}
```

Which isn't horrible, but could cause problems if you have both symbols. 
I also imagine the error to be comical:

Error: No overload of funAcceptingFile accepts arguments (File). 
Possible overloads:
    void funAcceptingFile(File)

If the compiler just figured out the conversion is possible on the 
original code (by being told how it can do that), then there's no need 
to manually do this conversion.


 If you're willing to do a little work you can make it happen.


Sure, with a little work, you can make anything happen. But will people 
be willing to do that work? If not, then won't we just end up with a lot 
of effort to build and maintain something nobody uses? I personally 
would rather just use the v2 exclusively, and eliminate any usage of v1.

-Steve
Oct 28 2021
parent Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 28 October 2021 at 14:30:38 UTC, Steven 
Schveighoffer wrote:

 On 10/28/21 10:18 AM, Adam D Ruppe wrote:
 if there was implicit construction

 This is going to still be a bit painful. I was thinking for 
 more compiler help.


well, if there was implicit construction, the compiler would do 
that conversion for you, c++ style.

but yeah today you can import the other thing and do the 
conversion yourself....


 I personally would rather just use the v2 exclusively, and 
 eliminate any usage of v1.


I think that ought to be the goal. You'd keep the v1 stuff around 
just as a temporary bridge to give some flexibility to schedule 
that transition, but realistically, the transition is inevitable 
unless you want to lock everything (which is always an option of 
course).
Oct 28 2021
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Thursday, 28 October 2021 at 11:30:18 UTC, Ogi wrote:

 You’re saying it like it’s not a big deal but I’m afraid it’s 
 going to be a huge source of pain. Imagine not being able to 
 get some `DateTime`, or `Complex`, or `File` from one package 
 and pass it to another because one depends on `std1` and the 
 other depends on `std2`.


The packages would not be totally locked apart:
```d
auto dateTime = package1.getDateTime;
package2.useDateTime(*cast(std2.datetime.DateTime*)&dateTime);
```

This would suck of course, but at least it works. Unless the 
`DateTime` format is different between the Phobos versions, in 
which case more cowboying is needed. Preferably wrapped in a nice 
conversion function by the user.

Hopefully this would be needed only rarely. Ranges would still 
work from one Phobos to another because the DBI interface 
presumably stays the same, albeit there would be extra template 
bloat.


 Or imagine dealing with different versions of the same 
 exception class. It appears to me that versioning the standard 
 library is a bullet train straight to dependency hell.


Fortunately exceptions are all inherited from the DRuntime class, 
so at least they will have the same base class. Assuming DRuntime 
version stays coupled with compiler's.
Oct 28 2021
parent bauss <jj_1337 live.dk> writes:
On Thursday, 28 October 2021 at 15:50:28 UTC, Dukc wrote:

 Unless the `DateTime` format is different between the Phobos 
 versions


That would most likely end up being the case.
Oct 29 2021
prev sibling next sibling parent SealabJaster <sealabjaster gmail.com> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 ...


I know this is basically a DOA suggestion, but what are your 
thoughts on embracing a package manager for the std?


Microsoft are handled via the NuGet package manager, and are 
tagged in a way that aligns to the current version of .Net.

e.g. For .Net Core 3.1 the tag is v3.1.x, for 3.2 it's v3.2.x, 
etc.

https://www.nuget.org/packages/Microsoft.Extensions.Logging/

I'm not savvy enough to be able to state pros and cons though, so 
I leave that to others >;3
Oct 25 2021
prev sibling next sibling parent reply Andrea Fontana <nospam example.org> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 [...]


Probably not the best idea but also this should work:

std.... >

version(NEW)
{
.....
}
else
{
....
}

std2...>

version = NEW
Oct 26 2021
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/26/21 3:18 AM, Andrea Fontana wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu wrote:

 [...]


 
 Probably not the best idea but also this should work:
 
 std.... >
 
 version(NEW)
 {
 .....
 }
 else
 {
 ....
 }
 
 std2...>
 
 version = NEW


This will grow with the number of releases.

I regret I forgot to mention in the initial post that we're not looking 
at v2. We're looking at v2, v3, v4, ... released e.g. at an annual or 
trienal pace. It's the having a fish vs. knowing how to fish problem.
Oct 26 2021
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 26 October 2021 at 14:43:19 UTC, Andrei Alexandrescu 
wrote:

 I regret I forgot to mention in the initial post that we're not 
 looking at v2. We're looking at v2, v3, v4, ... released e.g. 
 at an annual or trienal pace. It's the having a fish vs. 
 knowing how to fish problem.


If you expect the std lib to break at this rate, then maybe slim 
it down and have optional foundation-backed libraries instead?

What I like about C++ is that the std lib breakage up till now 
has been fairly insignificant, and that std lib expansions are 
tied to language expansions.
Oct 26 2021
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Oct 26, 2021 at 08:43:19AM -0600, Andrei Alexandrescu via Digitalmars-d
wrote:
[...]

 It's the having a fish vs. knowing how to fish problem.


Give a man a fish, and he eats once. Teach a man to fish, and he will sit
forever.

;-)


T

-- 
People walk. Computers run.
Oct 26 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/26/21 1:08 PM, H. S. Teoh wrote:

 On Tue, Oct 26, 2021 at 08:43:19AM -0600, Andrei Alexandrescu via
Digitalmars-d wrote:
 [...]

 It's the having a fish vs. knowing how to fish problem.


 
 Give a man a fish, and he eats once. Teach a man to fish, and he will sit
forever.
 
 ;-)


Build a man a fire, and he is warm for a night. Set a man on fire, and 
he is warm for the rest of his life.

-Steve
Oct 26 2021
prev sibling parent sarn <sarn theartofmachinery.com> writes:
On Tuesday, 26 October 2021 at 14:43:19 UTC, Andrei Alexandrescu 
wrote:

 I regret I forgot to mention in the initial post that we're not 
 looking at v2. We're looking at v2, v3, v4, ... released e.g. 
 at an annual or trienal pace. It's the having a fish vs. 
 knowing how to fish problem.


I guessed this might have been the motivation, but I don't think 
regular versions would make things simpler overall.  You'll still 
have compatibility problems, but it's the library users who'll 
have to worry about them instead of just the library developers.
Oct 26 2021
prev sibling next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.con> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 Versioning Phobos would free us from maintaining backward 
 compatibility with a variety of decisions that did not 
 withstand the test of time:


I think language and std lib should follow the same versioning 
scheme with the same attitude to breaking changes. Nobody wants 
to use the older outdated std lib.

Basically, if you are willing to break the std lib, then you also 
should be willing to have breaking changes at the language level.
Oct 26 2021
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 Versioning Phobos would free us from maintaining backward 
 compatibility with a variety of decisions that did not 
 withstand the test of time


I have a relatively simple LTS scheme in mind:

- We agree on a scheme for long-term support Phobos versions. For 
the sake of example, let's say we agree on 2.096 and every sixth 
minor version afterwards (2.102, 2.108 and so on).

- Any released DMD is required to compile all LTS Phobos versions 
older than the DMD. LTS Phobos versions may be patched to satisfy 
that requirement, so no need to keep around any DMD bugs or old 
features Phobos depends on. However, the LTS Phobos versions do 
not aim to support using newer language features. Phobos bug 
fixes may but are not required to be backported.

- DRuntime always uses the same version as DMD, and thus must 
also be compatible with older Phobos LTS versions.

- We change the language to allow relative imports, and convert 
Phobos to use them. This is needed to allow using two or more 
different versions of Phobos side-by-side.

- After these are in place, we may proceed with relatively 
aggressive changes in Phobos.


These do not mean that big a maintenance burden, because the LTS 
versions are only required to be kept compiling - only a fraction 
of the effort that would be needed to backport the bug fixes or 
new features. Also the LTS Phobos would act as a good real-world 
regression test for DMD and DRuntime.

However, this plan requires that Phobos must stop calling 
DRuntime internals directly - otherwise it's going to cause lots 
of trouble with this scheme. I don't think that is a problem 
though - Phobos is supposed to be just another library. Surely it 
does not need special privileges to do its job? If it needs, 
DRuntime needs a redesign.
Oct 27 2021
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 27 October 2021 at 18:01:47 UTC, Dukc wrote:

 [snip]

 - We change the language to allow relative imports, and convert 
 Phobos to use them. This is needed to allow using two or more 
 different versions of Phobos side-by-side.
 [snip]


Can you expand on what you mean by relative imports?
Oct 27 2021
parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 27 October 2021 at 18:22:32 UTC, jmh530 wrote:

 On Wednesday, 27 October 2021 at 18:01:47 UTC, Dukc wrote:

 [snip]

 - We change the language to allow relative imports, and 
 convert Phobos to use them. This is needed to allow using two 
 or more different versions of Phobos side-by-side.
 [snip]


 Can you expand on what you mean by relative imports?


Importing a package relative to current module in path. For 
example, being able to import `std.range` from 
`std.algorithm.iteration` without specifying `std`. The syntax 
might be `import package.package.range`, where `package` means 
the parent module.

The idea is the same as with relative symbolic links (for example 
`../bin/dmd`), as opposed to absolute symbolic links (say 
`/home/walter/d/bin/dmd`)
Oct 27 2021
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 27 October 2021 at 18:41:02 UTC, Dukc wrote:

 [snip]

 Importing a package relative to current module in path. For 
 example, being able to import `std.range` from 
 `std.algorithm.iteration` without specifying `std`. The syntax 
 might be `import package.package.range`, where `package` means 
 the parent module.

 The idea is the same as with relative symbolic links (for 
 example `../bin/dmd`), as opposed to absolute symbolic links 
 (say `/home/walter/d/bin/dmd`)


Yeah, I was thinking more of the second example. What tripped me 
up is that manipulating directories on a relative basis has a 
current working directory that you are relative to. So for 
instance, something like `from std.algorithm.iteration import 
../../range` made a bit more sense to me, but that seemed a bit 
messy...
Oct 27 2021
prev sibling next sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 ...


Throwing out another DOA idea but: Would a special ` version()` 
UDA manage to get anywhere?

For example

```d
 version(1)
auto splitter(...)(...){}

 version(2)
auto splitter(...)(...){}
```

And then introduce syntax to specify which version to use?

```
import std.algorithm : splitter 2; // as an example.
```

I've also been trying to think of a way where a specialised 
versioning tool could be used (kind of like a package manager, 
but purely for Phobos), but struggling to really come up with 
anything concrete.
Oct 27 2021
next sibling parent bauss <jj_1337 live.dk> writes:
On Thursday, 28 October 2021 at 04:05:09 UTC, SealabJaster wrote:

 On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei 
 Alexandrescu wrote:

 ...


 Throwing out another DOA idea but: Would a special ` version()` 
 UDA manage to get anywhere?

 For example

 ```d
  version(1)
 auto splitter(...)(...){}

  version(2)
 auto splitter(...)(...){}
 ```

 And then introduce syntax to specify which version to use?

 ```
 import std.algorithm : splitter 2; // as an example.
 ```

 I've also been trying to think of a way where a specialised 
 versioning tool could be used (kind of like a package manager, 
 but purely for Phobos), but struggling to really come up with 
 anything concrete.


That's gonna create a dependency hell of packages because some 
packages will support specific versions for specific calls and 
other packages will support different versions for the same calls.
Oct 28 2021
prev sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 28 October 2021 at 04:05:09 UTC, SealabJaster wrote:

 Throwing out another DOA idea but: Would a special ` version()` 
 UDA manage to get anywhere?

 For example

 ```d
  version(1)
 auto splitter(...)(...){}

  version(2)
 auto splitter(...)(...){}
 ```

 And then introduce syntax to specify which version to use?

 ```
 import std.algorithm : splitter 2; // as an example.



You could just as well do

auto splitter_v1.....
auto splitter_v2.........

import std.algorithm: splitter = splitter_v2

today.
Oct 28 2021
parent SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 28 October 2021 at 13:27:03 UTC, Adam D Ruppe wrote:

 You could just as well do

 auto splitter_v1.....
 auto splitter_v2.........

 import std.algorithm: splitter = splitter_v2

 today.


Aha, so you could!
Oct 28 2021
prev sibling next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
I think this version schema is fundamentally wrong and looks to 
me like a quick way to the D1 standard library mess.

Having/installing multiple std versions bundled with a 
DMD/druntime release has written disaster all over it.

Let me be clear, phobos needs work (breaking changes).
I'm not denying that.

But this feels like we are scared of repeating the D1 to D2 PR 
disaster, and do a kind of soft major version break.
Only this way we can say: "but we didn't break the api its still 
D2"

We have DMD 2.0100 coming up.
Just look at that silly number.

When I have to explain to somebody what that number means plus an 
additional std v2 thing, they have installed and compiled the 
rust hello world before I'm done.


Lets be bold!

Lets us call what would be D release 2.0100 "D" The semver is 
100.0.0

I have been using D2 since ~2009 and IMO it had enough breaking 
changes to call it version 100.0.0.
There is one number that describes D and its std.
When a symbol is no longer there in phobos, we increment the 
major version number.
When we add something to phobos we increment the minor number 
version number.
Here comes the somewhat tricky part, when dmd can not compile or 
a test now fails for the 50 most popular dub packages that worked 
with dmd N-1 we increment the version number.
We continue with the current release cycle, but the SemVer number 
is computed by a small script.
If we think a change warrants a major increment we can increment, 
but we can never not do an increment if we break something.
The changelog would list what broke/was removed.
Again, this can be done by a script.
The breakage testing for phobos, should not be clever, iff a 
public declaration is removed or changed it is a breaking change.
This is 2021 why is a human deciding what the version number is 
if the computer can do it.


And then lets have a proper vision for D, and not only small 
technical improvements.
I understand that we can't direct people to work on things in our 
community, but we can ask them.

So here it goes.



First we call D: "D the best programming language"

Obviously, that is correct in IMHO, but even if people disagree, 
they are at that point talking about D.
We lure them in with clickbait, but than doing anything, in any 
other language takes longer than doing it in D.

D should be the glue that makes the world go round.



Why is https://github.com/symmetryinvestments/autowrap not in 
phobos, and why doesn't it support, WASM, python, typescript, go, 
and haskell?

If I want to call this one function out of the python std, the 
only thing in my way must be the fact that I have forgotten how 
save a file in emacs, because I'm doing it on a friends computer 
how is using emacs.

If I have to write more than

```D
static import std.python;

auto result = 
std.python.thisOneFunction(a_bunch_of_complex_arguments);
```

we have failed.


I want to say, "ah you want to call a scikit function from D. No 
problem just right this line"
"But you have to call haskell first to get the data 
pre-processed, sure just at this line". No dub package required, 
just a D compiler and phobos.


phobos needs to have an event loop library included.
golang has go routines inside the language, we at least need 
something not much worse in phobos.


Support for yaml, toml, ini, json-schema, etc. in phobos.

When I have a json-schema, or whatever the equivalent is in yaml 
or any of the other, I want to write 
mixin(jsonSchemaBuild(import("path_to_schema_file.json"))
and have classes that represent the AST of the schema and a 
parser that turns json into those classes.
100% of people that see this feature will not care that it uses 
the GC.




Why do I have to call dmd, ldc, or gdc when I want a different 
backend to generate code. Isn't that what shared libraries are 
for?
Why can't I call dmd with --backend=llvm and compile with llvm 
instead of the dmd's backend.
Just think about this, do we really want people to understand 
that we have a D version 2.0100 with std version 2 or 3, and you 
have three compiles that support three different version of the 
language plus three different versions of the std, with all 
having different cli arguments.
No thank you, I'll just write some js and use node.js.


WASM: If I have a library or program, I need to be able to say 
--backend=XXX --outputType=WASM.
No betterC, no nogc, no excuses, it just needs to work. And 
additionally, it needs to put out a typescript file that does the 
converting of "json to D in WASM and back again" for you.
All without installing anything other than the D compiler and 
phobos.


The D compiler needs to be a file watcher daemon thing.
It is 2021 I have 64GB of memory, why is dmd still being split up 
in phases like its 1970 and all I have is a couple KB of RAM.
dmd is fast sure, but have you tried dart's or svelte's compiler.
The D compiler should be a diff compiler, were unchanged source 
is not lexed, not parsed, not semantically checked, nor ir 
generated.
When I work on phobos and press CTRL-S, the unittest's for phobos 
should start running before the keyUp event has received by my 
editor.


Why do I have to wait for assembler to be linked when I want 
unittest to run.
The D compiler daemon should execute some VM IR. This is linked 
to the --backend=XXX --outputType=SOME_VM switches from before.

Also, when the compiler stores all dependencies between functions 
and types in memory, it can also keep track of which unittest 
dependents on what code.
So when I change something that only has one unittest depending 
on it, only that test should be run.
Assuming, I didn't do anything stupid in the unittest, I would 
expect the unittest to have finished before my editor gets the 
keyUp event from pressing CTRL-S.


The compiler daemon, needs to support lsp perfectly.
I know you don't need that, because you know all the functions by 
heart and "real" programmers don't need auto complete.
But you are wrong and they do. Just look at web crowd is doing.
Developers that use this forum is not our target audience.

The features of that daemon need to be so good that they drive 
the development of the lsp protocol.
So hard indeed, that all other languages have to stub out the new 
features of the protocol with assert(false, "Impossible to 
implement") for years to come.

e.g. Before the keyUp event reaches my editor, I want the 
unittest line coverage changes to appear left of the line numbers 
in the file I just saved.


Doing a "normal" dub build will not be harder with this compiler 
daemon, but doing dub build that uses all my 8 cores without 
doing work twice will be easier.
Doing a dub build for dmd/druntime/phobos in release mode, needs 
to take less than a second.



Move everything to github.
Maybe bugzilla has features github does not, but github has 10^6 
more users than dlang's bugzilla.
At 10^2 we should have stop caring about features.
The casual user that finds a bug is not going to create an 
bugzilla account to create an issue.

Let's use milestones in github to express our aims/vision for the 
language.
E.g. the WASM milestone would link to all the issues related to 
WASM.
If "average rust user" wants to know what D's story is; simply 
look at the milestone. Easy to find, easy to contribute to.


Sorry for the rant.
Oct 28 2021
next sibling parent Robert burner Schadek <rburners gmail.com> writes:
I forgot something:



When I write a library to do X.

It has to work on the js,wasm,android,windows,linux,ios without 
every thinking about that this could be a challange.
The only thing I have to do, must be to write

--output=js,wasm,android,windows,linux,ios

anything less and we have failed.
Oct 28 2021
prev sibling next sibling parent norm <norm.rowtree gmail.com> writes:
On Thursday, 28 October 2021 at 08:50:31 UTC, Robert burner 
Schadek wrote:

 I think this version schema is fundamentally wrong and looks to 
 me like a quick way to the D1 standard library mess.

 [...]


`Developers that use this forum is not our target audience.`

+1
Oct 28 2021
prev sibling next sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 28 October 2021 at 08:50:31 UTC, Robert burner 
Schadek wrote:

 ...


While your ideal world is basically lala-land, you know, I kind 
of wished I lived in it.

I really appreciate this vision, but from experience of lurking 
this forum, I can tell you that, particular the older 
programmers, won't see much worth in a lot of your suggestions 
regarding the compiler changes (assuming we'd ever even have the 
manpower to make it possible in the first place).

I'm not going to spend much time going "but ackstually" and 
refute your points, since I'm going to be treating this as an "in 
a perfect world" post. So I'll mostly be sharing my own thoughts 
alongside yours.


 I think this version schema is fundamentally wrong and looks to 
 me like a quick way to the D1 standard library mess.

 Having/installing multiple std versions bundled with a 
 DMD/druntime release has written disaster all over it.

 Let me be clear, phobos needs work (breaking changes).
 I'm not denying that.


But we've become scared of the very word "breaking change", hence 
why baggage like ` property` are likely here to stay :(


 But this feels like we are scared of repeating the D1 to D2 PR 
 disaster, and do a kind of soft major version break.
 Only this way we can say: "but we didn't break the api its 
 still D2"

 We have DMD 2.0100 coming up.
 Just look at that silly number.

 When I have to explain to somebody what that number means plus 
 an additional std v2 thing, they have installed and compiled 
 the rust hello world before I'm done.


Coincidentally I was thinking about this today, but again, I 
doubt there's enough people who see any worth on fixing the 
versioning number to something a bit more... sane.

2.100 could be something special, instead of just a normal 
release with a nice, round number.


 Lets be bold!


Here we now enter lala-land.


 And then lets have a proper vision for D, and not only small 
 technical improvements.
 I understand that we can't direct people to work on things in 
 our community, but we can ask them.


I believe that's similar to how the vision documents went. 
"Here's a wishlist, please-maybe-possibly-if-you-want-to do it 
Community, best of luck and lots of love - from, the D foundation"


 First we call D: "D the best programming language"


**D** best programming language ;)


 Obviously, that is correct in IMHO, but even if people 
 disagree, they are at that point talking about D.
 We lure them in with clickbait, but than doing anything, in any 
 other language takes longer than doing it in D.


I'm kind of uncertain that'd work, but it's an interesting idea 
anyway.


 ...

 If I have to write more than

 ```D
 static import std.python;

 auto result = 
 std.python.thisOneFunction(a_bunch_of_complex_arguments);
 ```

 we have failed.


 ...


That's a very interesting vision... "D, the language to unite 
them all".


 phobos needs to have an event loop library included.
 golang has go routines inside the language, we at least need 
 something not much worse in phobos.


 Support for yaml, toml, ini, json-schema, etc. in phobos.

 When I have a json-schema, or whatever the equivalent is in 
 yaml or any of the other, I want to write 
 mixin(jsonSchemaBuild(import("path_to_schema_file.json"))
 and have classes that represent the AST of the schema and a 
 parser that turns json into those classes.
 100% of people that see this feature will not care that it uses 
 the GC.


"But you see, does it cover ` nogc nothrow pure const inout  safe 
-betterC` all at the same time? No? Rejected!"

or

"What do you mean it needs to use the GC for more than just 
exceptions? Haven't you heard, we don't do GC anymore because 
it's scary to the C++ programmers."




 Why do I have to call dmd, ldc, or gdc when I want a different


I understand the sentiment, but the compilers are all at 
different levels of support in terms of which version of the 
language they support.

I mean, ideally it would be nice to have a single, unified 
compiler, but I feel there's **many** obstacles in the way for 
this to actually happen.


 When I work on phobos and press CTRL-S, the unittest's for 
 phobos should start running before the keyUp event has received 
 by my editor.
 
 Why do I have to wait for assembler to be linked when I want 
 unittest to run.
 The D compiler daemon should execute some VM IR. This is linked 
 to the --backend=XXX --outputType=SOME_VM switches from before.

 Also, when the compiler stores all dependencies between 
 functions and types in memory, it can also keep track of which 
 unittest dependents on what code.
 So when I change something that only has one unittest depending 
 on it, only that test should be run.
 Assuming, I didn't do anything stupid in the unittest, I would 
 expect the unittest to have finished before my editor gets the 
 keyUp event from pressing CTRL-S.


Now *this* is ambition. Unfortunately, as I said, many here 
likely don't see any value in this. (disregarding the huge amount 
of work that'd have to go into making this possible in the first 
place that *someone* would have to volunteer to champion)


 The compiler daemon, needs to support lsp perfectly.
 I know you don't need that, because you know all the functions 
 by heart and "real" programmers don't need auto complete.
 But you are wrong and they do. Just look at web crowd is doing.


I dream of the day we get a compiler-as-a-library completion 
service, so maybe things like UFCS and templated types can be 
autocompleted <3~~

As ok as code-d is, it can be a real PITA when it's only able to 
work a quarter of the time because it's completely incapable of 
handling templates in any shape or form.


`myobj.` and get the full API + doc comments.


 Developers that use this forum is not our target audience.


^^^^^^^^^^^^^... who is our target audience exactly, because I 
couldn't actually tell you.


 Move everything to github.
 Maybe bugzilla has features github does not, but github has 
 10^6 more users than dlang's bugzilla.
 At 10^2 we should have stop caring about features.
 The casual user that finds a bug is not going to create an 
 bugzilla account to create an issue.

 Let's use milestones in github to express our aims/vision for 
 the language.
 E.g. the WASM milestone would link to all the issues related to 
 WASM.
 If "average rust user" wants to know what D's story is; simply 
 look at the milestone. Easy to find, easy to contribute to.


Wasn't there some kind of initiative at some point to port the 
bugzilla issues over to Github issues? I wonder what happened 
with that.


 Sorry for the rant.


I love these kinds of "in a perfect world" posts, because at the 
very least it can spark ideas in the direction things could go.

Rants in general can be very fun and somewhat enlightening to 
read.

If only posts like this could come from those who have the power, 
time, and motivation to organise a strong, coherent, unified 
vision. And if only we had the manpower to make our strongest 
wishes come true.

A huge restructuring, a complete rebrand, an extreme focus on 
developer tooling, language integration baked into the std. Some 
possibly exciting things, but reality generally hates this sort 
of excitement, and rewards it with a coin flip.

Thank you for this post, will be exciting to see what discussion 
this sparks though I suspect it won't be as positive a response 
as I'd hope for.
Oct 28 2021
next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
On Thursday, 28 October 2021 at 09:48:42 UTC, SealabJaster wrote:


 ...


Please read the following by no means as a personal attack.

Much of what you have written in the post above sounds defeatist 
to me.

Why try, we never gone make it to Mars?
But if we try, we might aim for Mars and make it to the Moon.

I should have added a leadership section to my post,
but at heart I'm an engineer at those things are difficult for me.

I would say though that the role of leadership in this instance is
to push for transformation, to set the long term vision, even and 
maybe
especially, if it is met with strong disagreement by parts of the 
community.
I rather would see people being annoyed by safe being the default 
and
forced GC usage, than see nobody on the forum, because nobody 
cares anymore.

The defeatist in me thinks, if D doesn't transform it will slowly 
but surely die
a quiet death.

"Change or go bust"

The optimist in me would say, that if people aren't angry with 
you for your decisions you haven't tried hard enough to do the 
right thing.
Oct 28 2021
next sibling parent SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 28 October 2021 at 10:38:20 UTC, Robert burner 
Schadek wrote:

 On Thursday, 28 October 2021 at 09:48:42 UTC, SealabJaster 
 wrote:


 ...


 Please read the following by no means as a personal attack.

 Much of what you have written in the post above sounds 
 defeatist to me.


I'm very much a pessimist, so this is pretty on point >:D No 
offense taken btw.

But from my personal perspective of D, it's "slowing down" and 
reaching a "stuck"/stagnation state.

One can only hope that the leadership will take the helm and 
provide us a proper vision going forward.
Oct 28 2021
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 28 October 2021 at 10:38:20 UTC, Robert burner 
Schadek wrote:

 I rather would see people being annoyed by safe being the 
 default and
 forced GC usage, than see nobody on the forum, because nobody 
 cares anymore.


Although I don't favour D's GC or approach to memory management I 
do agree that drawing a line in the sand is more important. 
Letting the wind take control, ending up with a fragmented eco 
system is not a good option.

But if you fully embrace higher level "semantics" then D also 
needs to set a vision that provides more of a higher level 
language (like adding better constructs for functional 
programming).

I think Perl waited too long before changing. Perl was big around 
2000, and essentially allowed Python to take over. Python made 
the right decision to make breaking changes and fix unicode 
strings, such weaknesses opens up for a take-over.

If you have weaknesses in areas where you aim to be best, well, 
you need to fix them before somebody else does it. "Wait and see" 
is usually too late (because of the inertia).
Oct 28 2021
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 28 October 2021 at 09:48:42 UTC, SealabJaster wrote:

 I really appreciate this vision, but from experience of lurking 
 this forum, I can tell you that, particular the older 
 programmers, won't see much worth in a lot of your suggestions 
 regarding the compiler changes (assuming we'd ever even have 
 the manpower to make it possible in the first place).


Yes, there is a distance between those that still use emacs/vi 
and those that expect AI-like visualized editing. And this 
distance will only increase as editors become more "intelligent" 
over time and require better static analysis from tooling. The 
distance is increasing year by year, so it isn't even about 
status quo, but where will you be in 10 years? If there is a 
noticeable distance today, then there will be a huge distance in 
10 years.

Anyway, the compiler has hit an evolutionary wall. When the 
language can no longer evolve because of internal compiler 
structure then you need to change priorities; meaning design a 
new architecture and restructure/rewrite compiler components and 
freeze language changes.

The cost of modifying the compiler will increase over time if you 
don't change the architecture. The effort spent on making a 
compiler modular does pay for itself over time.

It is basically the difference between having a long term plan 
for the architecture or just evolve iteratively without a plan as 
you go along. There are few examples of the latter ending well.



 I believe that's similar to how the vision documents went. 
 "Here's a wishlist, please-maybe-possibly-if-you-want-to do it 
 Community, best of luck and lots of love - from, the D 
 foundation"


The vision document was too much of a list of details rather than 
a vision of the end-result. You need to have a strong vision for 
the end result in order to plan and make the best priorities.



 If only posts like this could come from those who have the 
 power, time, and motivation to organise a strong, coherent, 
 unified vision. And if only we had the manpower to make our 
 strongest wishes come true.


It is difficult to get manpower without first projecting a strong 
vision.

Anyway, kudos to Robert for putting a lot of thought into his 
post.
Oct 28 2021
parent reply SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 28 October 2021 at 10:39:21 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 28 October 2021 at 09:48:42 UTC, SealabJaster 
 wrote:

 I really appreciate this vision, but from experience of 
 lurking this forum, I can tell you that, particular the older 
 programmers, won't see much worth in a lot of your suggestions 
 regarding the compiler changes (assuming we'd ever even have 
 the manpower to make it possible in the first place).


 Yes, there is a distance between those that still use emacs/vi 
 and those that expect AI-like visualized editing. And this 
 distance will only increase as editors become more 
 "intelligent" over time and require better static analysis from 
 tooling. The distance is increasing year by year, so it isn't 
 even about status quo, but where will you be in 10 years? If 
 there is a noticeable distance today, then there will be a huge 
 distance in 10 years.


I think this sort of ties into that discussion about fringe 
languages and polyglot programmers.

The less appealing to those exploring new languages, especially 
those that are mono-linguists or are only proficient in languages 
with good tooling, the less likely they are to come to D, no 
matter how much we gloat about the language.

Sure we might pick up a few old school C++ programmers, and maybe 
the odd curious Pythonista, but what access to we have to the 
newer generation(s) of programmers? a.k.a the future?

Even C has better autocomplete than D, which, granted, is due to 
its simplicity, but still it's technically easier for someone to 
get into a new C library than it is a D library because they have 
to scour source code/online documentation instead of being able 
to do the comfy thing and stay within their text editor/IDE.

I had an initial interview yesterday about a job (fingers 
crossed!) and I was like "ideally I'd use D everywhere, but it's 
not really going anywhere anytime soon, and it's ecosystem is 


This is the opposite direction we want things. We want others to 
come to us, not for us to go to others.


 Anyway, the compiler has hit an evolutionary wall. When the 
 language can no longer evolve because of internal compiler 
 structure then you need to change priorities; meaning design a 
 new architecture and restructure/rewrite compiler components 
 and freeze language changes.


I can't say too much since I haven't worked on the compiler 
enough, but it does seem similar to other complaints regarding a 
lack of compiler devs.


 It is difficult to get manpower without first projecting a 
 strong vision.


Indeed


 Anyway, kudos to Robert for putting a lot of thought into his 
 post.


Agreed, it definitely seems like this was on his mind for a while 
to come out of nowhere with a post like that.
Oct 28 2021
next sibling parent SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 28 October 2021 at 11:03:35 UTC, SealabJaster wrote:

 Even C has better autocomplete than D, which, granted, is due 
 to its simplicity, but still it's technically easier for 
 someone to get into a new C library than it is a D library 
 because they have to scour source code/online documentation 
 instead of being able to do the comfy thing and stay within 
 their text editor/IDE.


To add to this: C and C++ tooling are capable of parsing and 
evaluating macros to a decent level, yet D tooling can't even 
figure out simple templates, and often can't even figure out 
return types when you use an `auto` variable. We have plenty of 
room for growth, but I fear this is only a bandaid without a 
proper compiler-as-a-library to build with.
Oct 28 2021
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 28 October 2021 at 11:03:35 UTC, SealabJaster wrote:

 I think this sort of ties into that discussion about fringe 
 languages and polyglot programmers.


Yes, more advanced IDEs have made it possible for me to have fun 
with more languages for sure! Less cognitive/memory load. So 
advanced IDEs enable more polyglot programming.

Maybe "good programmers" in the future switch between many 
languages with little loyalty. I don't know. Anyway, if this 
becomes a trend, then it also would makes fringe languages that 
are good in particular domains more interesting.

I can't predict, but it is possible.
Oct 28 2021
prev sibling next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 28 October 2021 at 08:50:31 UTC, Robert burner 
Schadek wrote:

 I think this version schema is fundamentally wrong and looks to 
 me like a quick way to the D1 standard library mess.


The problem there was you couldn't use the two together. The 
point of this versioning thing is to try to come up with a scheme 
where you CAN use them together.
Oct 28 2021
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 28 October 2021 at 13:22:46 UTC, Adam D Ruppe wrote:

 On Thursday, 28 October 2021 at 08:50:31 UTC, Robert burner 
 Schadek wrote:

 I think this version schema is fundamentally wrong and looks 
 to me like a quick way to the D1 standard library mess.


 The problem there was you couldn't use the two together. The 
 point of this versioning thing is to try to come up with a 
 scheme where you CAN use them together.


"can" is one thing, but can you do it in a way with that is 
undetectable? Because if users can detect it, then it is likely 
to lead to incompatibility-issues. Isn't it?

Since D has the "does it compile" feature, it will be very 
difficult to prove that the design is sound and that a package, 
that makes assumptions about V1 types, is not going to be 
affected down the road.
Oct 28 2021
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 28 October 2021 at 13:34:14 UTC, Ola Fosheim Grøstad 
wrote:

 "can" is one thing, but can you do it in a way with that is 
 undetectable?


No, but I also don't think that matters. You need to pick a new 
version so then you expect things to be different.

There's always problems. You need to pick which ones you care 
about. There's two things to think about:

1) the hassle of updating with breaking changes
2) the incompatibility of diamond dependencies

A diamond dependency is when your app depends on libA which 
depends on thingv1 and also on libB which depends on thingv2.


much of a pain, version lock what you already have. But you'll 
probably lead to a diamond dependency eventually anyway as not 
everyone locks.

Let's look at some solutions.

With dub's version scheme, this is an error. When it detects 
this, dub stops building.

With the SAOC mangling scheme proposal, you're liable to have 
runtime crashes when you import one version of a type yet pass it 
to a library that needs another version - ABI mismatch beyond 
what mangles can see. The idea to add type layout to the mangle 
would turn this from a runtime crash into linker error, but it 
still fails.

Moving the version difference up to the module level is liable to 
create compile errors of the type "cannot pass X to function 
expecting argument X" (where X is v1 in one case and v2 in the 
other case). You can actually solve this with a runtime 
conversion, or making the compatibility layer check or whatever. 
It won't "just work" but since it is manageable it is my 
preference.

But then changing versions means updating imports to opt into 
it... which again i say "meh" but some people can't stand. It 
also means potential duplication which can be a hassle but again 
it is something you can manage so again "meh".

And sure maybe you can detect a change in the compatibility layer 
with reflection. Maybe some types won't convert back and forth. 
But... so be it. My view is a compatibility layer is to bridge 
the gap between the inevitable migration. Again, for an 
individual, you can just version lock. It is really only the 
diamond problem that is blocked.... and there, if they don't 
migrate eventually it will just get worse anyway, but they might 
not all migrate at the same time.

And if a lib never updates and it is hard incompatible the whole 
time? Just use a better lib.
Oct 28 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 28 October 2021 at 14:01:57 UTC, Adam D Ruppe wrote:

 On Thursday, 28 October 2021 at 13:34:14 UTC, Ola Fosheim 
 Grøstad wrote:

 "can" is one thing, but can you do it in a way with that is 
 undetectable?


 No, but I also don't think that matters. You need to pick a new 
 version so then you expect things to be different.


Hm, not sure if I follow... For this to work well I think you 
would need to have a different culture and a type system that 
focus more on abstract interfaces.

When programmers write code for a particular 
language/library/runtime/compiler they tend to make assumptions, 
that they probably shouldn't have. Like implementation specific 
aspects of how the runtime behaves or how classes are laid out 
and serialized.

Let's say you have a file cache package that assumes V1 file 
objects. It does use templated parameters so it actually accepts 
anything (duck typing) with the right interface. In the docs it 
says you have to always use objects from the standard lib. But 
there is no actual type checking that forces it to be V1, so the 
outcome could be unpredictable when handed V7 objects, as the 
package is tailored to V1 (making assumptions that are specific 
to V1).

One might claim that people should not write ugly code, but most 
applications of scale does contain ugly code.
Oct 28 2021
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 28 October 2021 at 14:30:42 UTC, Ola Fosheim Grøstad 
wrote:

 When programmers write code for a particular 
 language/library/runtime/compiler they tend to make 
 assumptions, that they probably shouldn't have.


Then there's the risk things will break. So be it.


 One might claim that people should not write ugly code, but 
 most applications of scale does contain ugly code.


They also have their own test procedures and the option to 
version-lock the system if the breakage is too much. They might 
file a bug and I'm sure upstream can consider their problems on a 
case-by-case basis if there's no other solution.

What we have now is fear leading to analysis paralysis. I say 
just do something and if there is a problem, we'll deal with it 
as it comes.

(the truth that nobody really wants to talk about is that even 
the most popular D libraries have just a handful of users anyway. 
we can just work with them as needed.)
Oct 28 2021
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 28 October 2021 at 14:53:26 UTC, Adam D Ruppe wrote:

 What we have now is fear leading to analysis paralysis. I say 
 just do something and if there is a problem, we'll deal with it 
 as it comes.


So what you are implying is that this is necessary for political 
reasons? Without this people will block desired changes to the 
standard lib/language?

So in essence, this feature is a way to suger coat breakage so 
that we can change more?

Well, if that is the political reality… then I guess it is 
necessary…
Oct 28 2021
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 28 October 2021 at 15:21:59 UTC, Ola Fosheim Grøstad 
wrote:

 So in essence, this feature


My view is that no feature is necessary.
Oct 28 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 28 October 2021 at 15:29:09 UTC, Adam D Ruppe wrote:

 On Thursday, 28 October 2021 at 15:21:59 UTC, Ola Fosheim 
 Grøstad wrote:

 So in essence, this feature


 My view is that no feature is necessary.


No language-feature, but a multi-versioned standard lib is 
necessary to get people to accept more breakage? Is that the 
political reality?
Oct 29 2021
next sibling parent bauss <jj_1337 live.dk> writes:
On Friday, 29 October 2021 at 12:52:31 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 28 October 2021 at 15:29:09 UTC, Adam D Ruppe 
 wrote:

 On Thursday, 28 October 2021 at 15:21:59 UTC, Ola Fosheim 
 Grøstad wrote:

 So in essence, this feature


 My view is that no feature is necessary.


 No language-feature, but a multi-versioned standard lib is 
 necessary to get people to accept more breakage? Is that the 
 political reality?


That's the political delusion
Oct 29 2021
prev sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Friday, 29 October 2021 at 12:52:31 UTC, Ola Fosheim Grøstad 
wrote:

 No language-feature, but a multi-versioned standard lib is 
 necessary to get people to accept more breakage? Is that the 
 political reality?


Probably. You do legitimately want an easy migration path anyway, 
and I do think removing things just to remove them is silly. The 
version thing helps with those.

But like yeah at this point I think the argument is more over how 
they do it than if they do it.
Oct 29 2021
prev sibling parent reply harakim <harakim gmail.com> writes:
On Thursday, 28 October 2021 at 14:53:26 UTC, Adam D Ruppe wrote:

 On Thursday, 28 October 2021 at 14:30:42 UTC, Ola Fosheim 
 Grøstad wrote:

 When programmers write code for a particular 
 language/library/runtime/compiler they tend to make 
 assumptions, that they probably shouldn't have.


 Then there's the risk things will break. So be it.


 One might claim that people should not write ugly code, but 
 most applications of scale does contain ugly code.


 They also have their own test procedures and the option to 
 version-lock the system if the breakage is too much. They might 
 file a bug and I'm sure upstream can consider their problems on 
 a case-by-case basis if there's no other solution.

 What we have now is fear leading to analysis paralysis. I say 
 just do something and if there is a problem, we'll deal with it 
 as it comes.

 (the truth that nobody really wants to talk about is that even 
 the most popular D libraries have just a handful of users 
 anyway. we can just work with them as needed.)


I'm going to start by saying you have always been more than 
helpful on the forum, I respect your opinions, and most of my D 
problems have historically been solved by asking you or by asking 
someone who references something you wrote. I don't have any 
claim to the direction of the D language/phobos library compared 
to you, so this is not supposed to be prescriptive, but a 
different perspective.

I think the reason the most popular D libraries have just a 
handful of users is related to the idea that the D language 
maintainers should just do something and if there is a problem, 
make affected users come to the forum and ask for help. Why 
should they do that when they can easily switch to a more stable 
language?

I posted a [post on 
reddit](https://www.reddit.com/r/d_language/comments/q74bzr/comment/hgpobuu/?utm_source=share&utm_med
um=web2x&context=3) in reply to why D is not popular. I dread coming back to my
projects and having to spend time to get them to recompile. I didn't include
this specifically in the reddit, but the libraries are pretty bad in some cases
with comments like this one, from Derelict:




It doesn't compile with the latest version of DMD and I will not 
be updating it.


Why doesn't it compile? Not even the maintainer wants to find 
out, what chance do I have?

D will never be popular until this problem is solved. If D wants 
to be a niche language with few people maintaining the entire 
ecosystem, rapid change is desirable. If D wants to become 
widespread, then it has to start viewing breaking changes as 
generally unacceptable and a failure of the ecosystem. For that 
reason, I would be very hesitant about doing something to break 
libraries that use std.

Auto-decoding seems like a corner case that was used for a time, 
but isn't used much anymore except by libraries that this forum 
more or less is willing to change. If that's true then I think 
it's worth a breaking change, but I still feel like that change 
needs to be viewed as a failure and retrospected upon.

I think the compiler could be smart enough to issue an error 
about the library change if it sees a type difference of std.X.Y 
and std2.X.Y in a parameter. Maybe a hyperlink to a document that 
explains a solution, for example: "Hey that old duration type 
you're passing in? You need to pass in a new version, but the old 
version takes the new version in one of its constructors. Here's 
an example ..."
I think it could "autobox" the new object for you automatically 
and give a warning. Then this transition would be like the Java 
1.4 to Java 1.5 change (like generics, but also like 
auto-boxing). I understand that might be too much to ask or even 
not desirable. I also think it should be end-of-life'd at some 
point.

I don't know how to deal with the issue of reference equality 
tests between std and std2 types, but I don't know how often that 
comes up, to be honest.

It might be worth saving up a bunch of these breaking changes and 
coming up with a D3 that is just one breaking change of language 
and library that D can go off of, without breaking changes, for a 
long time. Perhaps it would be best to leave auto-decoding in 
until that version is ready.

Based on the discussion, it seems like there is not really a 
coherent vision and things get added here and there. That is not 
how popular languages are. Their releases are few and far between 
and they take breaking changes very seriously. People can count 
on the language and when they learn or build something, it is 
typically learned or built for good. This is not to say 
everything about the way popular languages are maintained is 
better than D. I just think the community needs to figure out a 
way to create stability. It doesn't mean we have to give up 
experimentation. Maybe there could be an experimental D++ that 
ports back the best features to D in spaced-out increments for 
the common folk. D++ would allow breaking changes and would 
always have features D doesn't.

\<feelings>
Whoever said that D's target audience is not the people on the 
forum: I feel like I'm one of the people you're talking about, I 
just happen to be on the forum sometimes. For me, D is like a 
girlfriend with a borderline personality disorder. When I get 
real excited and try to rekindle a part of our relationship (ie 
try to compile something), it feels like she doesn't give a shit 
about me and I'm going to have to work for here love. Not exactly 
marriage material. Java might be ugly and not very helpful, but I 
know where I stand with her.

I would love to be more involved in the D community and use it 
for all my projects. I'd love to come up with some libraries and 
a microservice collection and even help develop some tools. As 
things stand, I hesitate to suggest using D at work, and I will 
shy away from committing to it in my side business, until I feel 
like it is stable.
\</feelings>
Oct 29 2021
next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Friday, 29 October 2021 at 19:21:43 UTC, harakim wrote:

 I dread coming back to my projects and having to spend time to 
 get them to recompile.


Do you have a recollection of why, specifically they would fail 
to compile when you get back to them? Language changes? Std lib 
changes? Third party libs? And how difficult it was to update 
them?

"breakage" isn't really a binary yes/no thing. Some are worse 
than others. One example I did in my minigui was misspell a word. 
I made that a deprecated alias with the message saying "it was 
misspelled, use THIS instead".

Now, worth noting, that isn't exactly breakage, since you can 
just ignore the message. I've left it like that for years now; it 
doesn't really hurt anything. (My own libs' policy is actually 
pretty strict - I almost never break anything and maintain 
compatibility with both old and new compilers, and with simple 
build systems; my "no breakage" policy even includes the specific 
compile command line. And when I do break something, I make it as 
easy as I can think of to make migration quick and painless.)

But still that's an example of a partial break that is easy to 
deal with. Should be able to update it in a matter of minutes.

And compatibility shims are easy: the deprecated alias is 
something you could too in your code. Doesn't have to be in mine. 
So you can choose your own compiler as well.

This is how I'd like the stdlib to be too. Where changes are 
needed, you make it easy. Where they're not needed, you can leave 
it alone.

Phobos needs some changes right now. That doesn't mean you have 
to delete the current code, but you also can't leave it alone out 
of fear of what might happen.


 Why doesn't it compile?


I'd like to know that too. Is it really the language's fault? And 
why couldn't you keep using the same version you had before?
Oct 29 2021
parent reply harakim <harakim gmail.com> writes:
On Friday, 29 October 2021 at 20:39:56 UTC, Adam D Ruppe wrote:

 On Friday, 29 October 2021 at 19:21:43 UTC, harakim wrote:

 I dread coming back to my projects and having to spend time to 
 get them to recompile.


 Do you have a recollection of why, specifically they would fail 
 to compile when you get back to them? Language changes? Std lib 
 changes? Third party libs? And how difficult it was to update 
 them?

 "breakage" isn't really a binary yes/no thing. Some are worse 
 than others. One example I did in my minigui was misspell a 
 word. I made that a deprecated alias with the message saying 
 "it was misspelled, use THIS instead".

 Now, worth noting, that isn't exactly breakage, since you can 
 just ignore the message. I've left it like that for years now; 
 it doesn't really hurt anything. (My own libs' policy is 
 actually pretty strict - I almost never break anything and 
 maintain compatibility with both old and new compilers, and 
 with simple build systems; my "no breakage" policy even 
 includes the specific compile command line. And when I do break 
 something, I make it as easy as I can think of to make 
 migration quick and painless.)

 But still that's an example of a partial break that is easy to 
 deal with. Should be able to update it in a matter of minutes.

 And compatibility shims are easy: the deprecated alias is 
 something you could too in your code. Doesn't have to be in 
 mine. So you can choose your own compiler as well.

 This is how I'd like the stdlib to be too. Where changes are 
 needed, you make it easy. Where they're not needed, you can 
 leave it alone.

 Phobos needs some changes right now. That doesn't mean you have 
 to delete the current code, but you also can't leave it alone 
 out of fear of what might happen.


 Why doesn't it compile?


 I'd like to know that too. Is it really the language's fault? 
 And why couldn't you keep using the same version you had before?


You have a point. I believe this was the compilation issue I had 
last which appeared to be a backwards-compatibility issue in the 
std lib:
```
auto sekunden = to!("seconds", double)(dauer);
		
return cast(long)floor(sekunden * 18);
```
It used to compile and now it doesn't. It's not a big deal. I 
know I have had more like this in the past but I have not kept 
track over the years. The only time I felt overwhelmed was when I 
switched from Tango + version 1 to Phobos + version 2, which is 
reasonable.

The point you got at was that it's mostly third party libraries I 
have an issue with. That is true. I had to re-download the 
DerelictSDL source and I did not know which version since I 
hadn't saved it on the flash drive with my project. When I got 
it, it wouldn't compile. I upgraded compilers because it used 
newer library functions (and made a few other fixes), but it 
didn't draw anything after I did that. I wasn't sure why so when 
I saw this message:




It doesn't compile with the latest version of DMD and I will not 
be updating it.


I ended up moving to a new library, bindbc.

This is just a description of the last time I pulled out an old 
project. There was:
1. An instance of a library function being added in a minor 
version, which caused me to have to upgrade to use the current 
version of the library. Even though I was previously using the 
same major version, it would not compile.
2. An instance of a library function call breaking with only 
minor version upgrades.


My argument is that we should:
1. Signify breaking changes in the version number. Code using a 
library with version x.a should compile using any version of the 
library x.b, where b>=a.

2. Breaking changes should be limited to infrequent releases. 
Multiple breaking changes should build up and get released at the 
same time with an upgrade guide of some sort.

Maybe I'm just not having the correct perspective or even 
understanding when it comes to framework and language versioning. 
I just know that I have to worry about these things in D, but not 
in other languages.
Oct 29 2021
parent reply Mike Parker <aldacron gmail.com> writes:
On Saturday, 30 October 2021 at 00:39:21 UTC, harakim wrote:


 The point you got at was that it's mostly third party libraries 
 I have an issue with. That is true. I had to re-download the 
 DerelictSDL source and I did not know which version since I 
 hadn't saved it on the flash drive with my project. When I got 
 it, it wouldn't compile. I upgraded compilers because it used 
 newer library functions (and made a few other fixes), but it 
 didn't draw anything after I did that. I wasn't sure why so 
 when I saw this message:




It doesn't compile with the latest version of DMD and I will 
not be updating it.


 I ended up moving to a new library, bindbc.



Oh, wow. I stopped maintaining Derelict years ago. That's why it 
no longer compiles. I added the note quite some time later. It 
was superseded by DerelictOrg, and that in turn was superseded by 
BindBC (which you've already discovered).

I should change that message in the Derelict repository to point 
people to bindbc. Sorry about that.
Oct 29 2021
parent harakim <harakim gmail.com> writes:
On Saturday, 30 October 2021 at 01:42:29 UTC, Mike Parker wrote:

 I should change that message in the Derelict repository to 
 point people to bindbc. Sorry about that.


It's fine. I'm not specifically pointing to this library as an 
issue. If I was used to working in a language where libraries 
become unusable with time, I would have noticed myself more 
quickly. I was more pointing out that it's harder to maintain a 
third party library in D for some reason than Java. Lots of Java 
libraries don't get maintained but they still work. The worst you 
get is a bunch of compile-time warnings.
Nov 01 2021
prev sibling parent reply Greg Strong <mageofmaple protonmail.com> writes:
On Friday, 29 October 2021 at 19:21:43 UTC, harakim wrote:

 Based on the discussion, it seems like there is not really a 
 coherent vision and things get added here and there. That is 
 not how popular languages are. Their releases are few and far 
 between and they take breaking changes very seriously.


Well, there is a bit of chicken-and-egg problem here.  Difficult 
to become popular if we don't make some breaking changes to 
address some issues, but also difficult to become popular if we 
do make breaking changes.  I also think that this thread shows a 
serious attitude toward the issue, although I can't really speak 
to the distant past...

Finally, I'd point out that popular languages do this as well.  
Python needed to transition to Python 3, and they did.  I also 
just looked in on Scala after a couple years of paying no 
attention, and apparently they just made what appears to be a 
very large changes in a transition to Scala 3 (although you could 
argue it's a stretch to call Scala a popular language...  It's a 
language I've never really used but keep looking in on from time 
to time because it is certainly an interesting language.)
Oct 29 2021
parent reply harakim <harakim gmail.com> writes:
On Friday, 29 October 2021 at 21:26:52 UTC, Greg Strong wrote:

 Well, there is a bit of chicken-and-egg problem here.  
 Difficult to become popular if we don't make some breaking 
 changes to address some issues, but also difficult to become 
 popular if we do make breaking changes.  I also think that this 
 thread shows a serious attitude toward the issue, although I 
 can't really speak to the distant past...


I am in favor of making a breaking change, even a large one, 
provided it will lead to a significantly better experience and 
more stability instead of less.

Python 3 made a serious breaking change after 20 years.
Java made some major language changes without a breaking change 
in Java 1.5.

Angular was by far the most popular web UI framework for new 
projects, but lost that edge with frequent major version releases.
Windows Vista never became more popular than Windows XP

I just want D to be in the first category rather than the last.
Oct 29 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 30 October 2021 at 01:46:18 UTC, harakim wrote:

 Angular was by far the most popular web UI framework for new 
 projects, but lost that edge


Is that true though? What framework is more used than Angular for 
larger web applications?

I agree that a D3 approach is the best option,  but that requires 
a clear direction: more high level or more low  level? D3 without 
focus would be worse...
Oct 29 2021
next sibling parent harakim <harakim gmail.com> writes:
On Saturday, 30 October 2021 at 05:27:56 UTC, Ola Fosheim Grøstad 
wrote:

 On Saturday, 30 October 2021 at 01:46:18 UTC, harakim wrote:

 Angular was by far the most popular web UI framework for new 
 projects, but lost that edge


 Is that true though? What framework is more used than Angular 
 for larger web applications?

 I agree that a D3 approach is the best option,  but that 
 requires a clear direction: more high level or more low  level? 
 D3 without focus would be worse...


I totally agree a D3 without focus would be worse. I feel like a 
lot of people are hitting the nail on the head. I read Adam's 
commentary on his site about needing a vision and there is a post 
right now about D's values. If the D community can decide on a 
direction and people think it will be successful, they will work 
to support it. The ecosystem is not actually that hard to build. 
It's mostly built.
Oct 30 2021
prev sibling next sibling parent bauss <jj_1337 live.dk> writes:
On Saturday, 30 October 2021 at 05:27:56 UTC, Ola Fosheim Grøstad 
wrote:

 Is that true though? What framework is more used than Angular 
 for larger web applications?



Vue and React.
Nov 01 2021
prev sibling parent reply harakim <harakim gmail.com> writes:
On Saturday, 30 October 2021 at 05:27:56 UTC, Ola Fosheim Grøstad 
wrote:

 On Saturday, 30 October 2021 at 01:46:18 UTC, harakim wrote:

 Angular was by far the most popular web UI framework for new 
 projects, but lost that edge


 Is that true though? What framework is more used than Angular 
 for larger web applications?

 I agree that a D3 approach is the best option,  but that 
 requires a clear direction: more high level or more low  level? 
 D3 without focus would be worse...


Based on people I know and job postings, React is the number 1 
framework. I use Vue, but I think Angular and Vue are probably 
fairly close in adoption. I see more jobs for Angular, but 
everyone I know who uses on of the two uses Vue at their job. 
Angular is certainly not as popular as it used to be and is 
declining in relative popularity all the time.

https://trends.google.com/trends/explore?date=all&geo=US&q=%2Fg%2F11c6w0ddw9,%2Fg%2F11c0vmgx5d,%2Fm%2F012l1vxv

Angular 2 was released in September of 2016.
Nov 01 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 1 November 2021 at 20:44:45 UTC, harakim wrote:

 fairly close in adoption. I see more jobs for Angular, but 
 everyone I know who uses on of the two uses Vue at their job. 
 Angular is certainly not as popular as it used to be and is 
 declining in relative popularity all the time.

 https://trends.google.com/trends/explore?date=all&geo=US&q=%2Fg%2F11c6w0ddw9,%2Fg%2F11c0vmgx5d,%2Fm%2F012l1vxv


I don't know how well search reflects usage as people search more 
when learning, but it varies a lot between countries:

https://trends.google.com/trends/explore?date=all&q=%2Fg%2F11c6w0ddw9,%2Fg%2F11c0vmgx5d,%2Fm%2F012l1vxv

Chinese developers seems to have a preference for vue.js, maybe 
better documentation?


 Angular 2 was released in September of 2016.


And now it is at Angular 12…
Nov 01 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/1/21 5:15 PM, Ola Fosheim Grøstad wrote:


 Angular 2 was released in September of 2016.


 
 And now it is at Angular 12…
 


Angular 2 was the break from AngularJS, and essentially the start of the 
downhill slide.

I've only used angular for about 2 years (sporadically, for work), and 
it's probably the worst framework of any language I've ever seen. When I 
first was looking at angular, I was looking at (what is now) AngularJS, 
and it didn't look too bad. But I never got to use it. Since Angular 2, 
it requires a compiler to build your javascript web pages, plus you are 
not writing html any more, or using the browser's features, it all has 
to go through the framework. The whole thing is extra complicated for 
the sake of being complicated.

-Steve
Nov 02 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 3 November 2021 at 02:23:05 UTC, Steven 
Schveighoffer wrote:

 Angular 2 was the break from AngularJS, and essentially the 
 start of the downhill slide.


Hm, I Angular 2+ looks like a massive improvement to me.


 I've only used angular for about 2 years (sporadically, for 
 work), and it's probably the worst framework of any language 
 I've ever seen.


I haven't looked at vue.js, but I find React to be a hack. It 
does not provide me with enough advantages and structure over 
plain TypeScript to offset the disadvantages.


 Since Angular 2, it requires a compiler to build your 
 javascript web pages, plus you are not writing html any more, 
 or using the browser's features, it all has to go through the 
 framework. The whole thing is extra complicated for the sake of 
 being complicated.


Ok, so I get your objections. Angular is most suitable for 
business-like applications like admin interfaces, where you get 
change requests from users frequently. What it does reasonable 
well is allowing you to build independent components that makes 
maintenance easier (so you work with HTML/CSS within one 
component). But you have to spend extra time on interfacing 
between components. Like, I've found that if I tailor 
data-structures to Angular then the code can be reasonably clean, 
so sometimes you are better off transforming received json into a 
datastructure tailored to the components you design. It can 
sometimes be a challenge to formulate the state of your 
web-client as rxjs streams if you want more advanced coupling of 
GUI-elements, so you have to adopt that mindset and use Angular 
for projects where it makes sense.

In my opinion, Angular becomes more attractive if you also adopt 
Material GUI. If you don't plan on maintaining the project for a 
long time, then Angular may not be worth the extra interfacing 
effort.

(I don't see the compilation requirement as a big deal. It can be 
set up to happen in the background, and I rarely use plain 
JavaScript anyway. TypeScript has taken over, and that also 
requires compilation.)
Nov 03 2021
parent SealabJaster <sealabjaster gmail.com> writes:
On Wednesday, 3 November 2021 at 07:57:15 UTC, Ola Fosheim 
Grøstad wrote:

 I haven't looked at vue.js, but I find React to be a hack. It


Vue is a joy to work with (IMO). If you go to their webpage 
https://vuejs.org/ and watch the "Why Vue" video, I think it 
explains pretty well how easy it is to use.
Nov 03 2021
prev sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
On Thursday, 28 October 2021 at 13:22:46 UTC, Adam D Ruppe wrote:

 The problem there was you couldn't use the two together. The 
 point of this versioning thing is to try to come up with a 
 scheme where you CAN use them together.


hm fair enough, but very likely still a mess in the middle run.

What about a compiler daemon library thing that you could write 
small programs for that converts the use of the deprecated 
feature/function into the up to date version.

Think small scale python 2to3 script, but programmable and 
something that actually understands the language.

That way, when you break and api or the language, you provide a 
fix script for each.
Apply them one after the other and you breaking change isn't 
technically breaking anything anymore.

So that way when you update your old program that used D 120.4.7 
to 202.7.2 you rust run the 82 batches of fix up scripts and move 
on with your life.

Did somebody said database migrations ....
Oct 28 2021
parent Ogi <ogion.art gmail.com> writes:
On Thursday, 28 October 2021 at 14:00:48 UTC, Robert burner 
Schadek wrote:

 What about a compiler daemon library thing that you could write 
 small programs for that converts the use of the deprecated 
 feature/function into the up to date version.

 Think small scale python 2to3 script, but programmable and 
 something that actually understands the language.

 That way, when you break and api or the language, you provide a 
 fix script for each.
 Apply them one after the other and you breaking change isn't 
 technically breaking anything anymore.

 So that way when you update your old program that used D 
 120.4.7 to 202.7.2 you rust run the 82 batches of fix up 
 scripts and move on with your life.


That would be cool. And it would be extra cool if it could 
compile the program after each iteration to verify that it 
actually, well, compiles.

When you think of it, this sounds like an obvious solution. Was 
something like this done before in some other programming 
language or library?
Oct 28 2021
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/28/2021 1:50 AM, Robert burner Schadek wrote:

 [...]


Thank you for a most insightful and excellent post.
Oct 29 2021
prev sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 28 October 2021 at 08:50:31 UTC, Robert burner 
Schadek wrote:

 I think this version schema is fundamentally wrong and looks to 
 me like a quick way to the D1 standard library mess.

 [...]


Don't know how I could've missed this post. Well ranted
Nov 02 2021
prev sibling parent forkit <forkit gmail.com> writes:
On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu 
wrote:

 Versioning Phobos would free us from maintaining backward 
 compatibility with a variety of decisions that did not 
 withstand the test of time:


So the question is, 'how can we best adapt the D programming 
language to change?'...

This surely is the perennial question of language development.

Your answer seems to be 'versioning'.

Fine.

Unless I hear a better solution, then (as nike would say) 'just 
do it'.

I will use it.
Nov 01 2021