www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Blog Post: Beating std::visit Without Really Trying

reply Paul Backus <snarwin gmail.com> writes:
I was curious how C++17's std::variant compared to the options we 
have in D, like Algebraic and SumType, so I did a simple 
comparison of the generated assembly for each of them. You can 
read about it at the link below. And as you can probably guess 
from the title, D comes out ahead, in the end.

https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

This is my first attempt at sharing something like this, so any 
comment or feedback is very much appreciated!
Oct 04 2019
next sibling parent reply Meta <jared771 gmail.com> writes:
On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options 
 we have in D, like Algebraic and SumType, so I did a simple 
 comparison of the generated assembly for each of them. You can 
 read about it at the link below. And as you can probably guess 
 from the title, D comes out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

 This is my first attempt at sharing something like this, so any 
 comment or feedback is very much appreciated!


I'm not sure if you're aware, but funnily enough, I also wrote an 
article[1] on std::variant vs. the D alternative that references 
Matt Kline's article on std::visit. It seems we're really making 
getting our money's worth from his article.

I really enjoyed this - I think you're right in that it comes 
down to the complexity of implementation, and I suspect that C++ 
forced the developers of std::variant to choose between a usable 
API (usable, not good) and performance.

I've been trying to communicate this major selling point of D to 
my coworkers, but it's a real uphill battle. I haven't even been 
able to convince them that built-in unit tests are a killer 
feature.

As an aside, I actively use your sumtype library and for the most 
part find it very nice to use. Thanks for the great work.

1.https://dlang.org/blog/2018/03/29/std-variant-is-everything-cool-about-d/
Oct 04 2019
parent Paul Backus <snarwin gmail.com> writes:
On Saturday, 5 October 2019 at 04:07:45 UTC, Meta wrote:

 I'm not sure if you're aware, but funnily enough, I also wrote 
 an article[1] on std::variant vs. the D alternative that 
 references Matt Kline's article on std::visit. It seems we're 
 really making getting our money's worth from his article.

 I really enjoyed this - I think you're right in that it comes 
 down to the complexity of implementation, and I suspect that 
 C++ forced the developers of std::variant to choose between a 
 usable API (usable, not good) and performance.


I remember seeing your article when it went up on the D blog. 
It's a great illustration of how things that are complex in C++ 
are often easy or even trivial in D.

Some of the commenters on reddit brought up boost.variant2 and 
mpark::variant as alternative C++ implementations that generate 
the same code as C. So it's clearly *possible* for C++ to get 
good performance on this. It's just that in C++, you have to work 
really hard for it, whereas in D, it's so easy you'd have to work 
harder to get it wrong.
Oct 05 2019
prev sibling next sibling parent reply Arun Chandrasekaran <aruncxy gmail.com> writes:
On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options 
 we have in D, like Algebraic and SumType, so I did a simple 
 comparison of the generated assembly for each of them. You can 
 read about it at the link below. And as you can probably guess 
 from the title, D comes out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

 This is my first attempt at sharing something like this, so any 
 comment or feedback is very much appreciated!


Good one. Any plans to push SumType as a replacement of Phobo's 
Algebraic?
Oct 04 2019
parent reply Seb <seb wilzba.ch> writes:
On Saturday, 5 October 2019 at 06:40:35 UTC, Arun Chandrasekaran 
wrote:

 On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options 
 we have in D, like Algebraic and SumType, so I did a simple 
 comparison of the generated assembly for each of them. You can 
 read about it at the link below. And as you can probably guess 
 from the title, D comes out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

 This is my first attempt at sharing something like this, so 
 any comment or feedback is very much appreciated!


 Good one. Any plans to push SumType as a replacement of Phobo's 
 Algebraic?


Phobos is essentially dead/frozen (feature-wise). Though if 
someone ever manages to get v2 of the ground, SumType would be 
the obvious choice.

Anyhow, currently we would have to name it differently (e.g. dts 
- https://github.com/wilzbach/dts). Maybe the upcoming SAoC 
project will change this and allow multiple versions of a library 
to co-exist in a binary.
Oct 05 2019
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Saturday, 5 October 2019 at 13:58:52 UTC, Seb wrote:

 On Saturday, 5 October 2019 at 06:40:35 UTC, Arun 
 Chandrasekaran wrote:

 Good one. Any plans to push SumType as a replacement of 
 Phobo's Algebraic?


 Phobos is essentially dead/frozen (feature-wise). Though if 
 someone ever manages to get v2 of the ground, SumType would be 
 the obvious choice.

 Anyhow, currently we would have to name it differently (e.g. 
 dts - https://github.com/wilzbach/dts). Maybe the upcoming SAoC 
 project will change this and allow multiple versions of a 
 library to co-exist in a binary.


I actually would like to submit SumType to Phobos, albeit as an 
alternative to Algebraic rather than a replacement for it. Given 
how dramatic the improvement is, I think it's worth the 
awkwardness of having two sum type implementations in the 
standard library.


[1], which (for somewhat complicated reasons [2]) makes it 
impossible for SumType to support members with  disabled and 
non-const opEquals overloads.

[1] https://issues.dlang.org/show_bug.cgi?id=19458
[2] https://github.com/pbackus/sumtype/issues/16
Oct 05 2019
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/19 7:58 AM, Seb wrote:

 On Saturday, 5 October 2019 at 06:40:35 UTC, Arun Chandrasekaran wrote:

 On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options we 
 have in D, like Algebraic and SumType, so I did a simple comparison 
 of the generated assembly for each of them. You can read about it at 
 the link below. And as you can probably guess from the title, D comes 
 out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html 


 This is my first attempt at sharing something like this, so any 
 comment or feedback is very much appreciated!


 Good one. Any plans to push SumType as a replacement of Phobo's 
 Algebraic?


 
 Phobos is essentially dead/frozen (feature-wise). Though if someone ever 
 manages to get v2 of the ground, SumType would be the obvious choice.


I think we'd do good to choose words with care. An extraordinary 
assertion, however odd and unjustified, may go by chance unchallenged 
and if repeated enough soon starts passing as the accepted truth. This 
is very damaging, regardless of how well it was intended initially, 
goodwill I have no doubt about. To set the record straight: Phobos is 
not dead and is not frozen feature-wise. It is and has always been open 
to adding new functionality, and of course to improve implementation of 
existing functionality.

Variant's design and implementation are very, very, very old. We're 
talking 15 years. It is literally the first generic design I've ever 
done in D, and at the time getting it (or anything nontrivial) to work 
was a large challenge. At that time only the most basic examples worked 
for any template-related feature. The code also predates most of the 
CTFE niceties we now take for granted, static foreach included. Not to 
mention, I'd be the first to say I engineered the thing poorly. A 
radical simplification is definitely possible and desirable.


 Anyhow, currently we would have to name it differently (e.g. dts - 
 https://github.com/wilzbach/dts). Maybe the upcoming SAoC project will 
 change this and allow multiple versions of a library to co-exist in a 
 binary.


Adding it as a new facility is a possibility, but only if the challenges 
of keeping the existing interface are too large.
Oct 05 2019
parent Tobias Pankrath <tobias pankrath.net> writes:
On Sunday, 6 October 2019 at 00:52:38 UTC, Andrei Alexandrescu 
wrote:

 Anyhow, currently we would have to name it differently (e.g. 
 dts - https://github.com/wilzbach/dts). Maybe the upcoming 
 SAoC project will change this and allow multiple versions of a 
 library to co-exist in a binary.


 Adding it as a new facility is a possibility, but only if the 
 challenges of keeping the existing interface are too large.


I just want to chime in as a returning D user, who stumbled upon 
Nullable, Typedef and Algebraic.

I think is is important to have high quality and well supported 
versions of these kinds of type constructors in the standard 
library, because to really shine support for them has to be 
pervasive. I could roll my own (or use some package from 
code.dlang.org), but one of the key features have to be that they 
'just work'. I want to std.conv.to them [1], use them in a json 
serialization library out of the box, etc, etc.

[1] https://issues.dlang.org/show_bug.cgi?id=11704
Oct 06 2019
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2019 6:58 AM, Seb wrote:

 Phobos is essentially dead/frozen (feature-wise).


I beg to disagree. A couple cases in point:

     https://github.com/dlang/phobos/pull/7211

which is a re-imagining, rethinking of hexString.

and:

     https://github.com/dlang/phobos/pull/7130
     https://github.com/dlang/phobos/pull/7144

both of which work to remove autodecode from Phobos. 7130 in particular can use 
some help with anyone who wants to help drive this forward.
Oct 05 2019
next sibling parent reply Seb <seb wilzba.ch> writes:
On Sunday, 6 October 2019 at 02:33:15 UTC, Walter Bright wrote:

 On 10/5/2019 6:58 AM, Seb wrote:

 Phobos is essentially dead/frozen (feature-wise).


 I beg to disagree. A couple cases in point:

     https://github.com/dlang/phobos/pull/7211

 which is a re-imagining, rethinking of hexString.


I'm sorry, my choice of words wasn't wise (and I certainly didn't 
mean it in a negative way). Phobos is amazing and stable, but 
exactly because of these attributes there isn't much active 
development happening. Stability implies no breaking changes 
which is **very very good** because it allows the ecosystem to 
thrive (and not break builds because Algebraic was swapped 
against SumType).

Below I have listed a summary of all announced changes in Phobos 
since the beginning of this year (six releases, 2.088 - 2.083). 
This lists only the announced changes (and there are more bug 
fixes), but in total we're looking at one new function 
(getAvailableDiskSpace), two new overloads (schwarzSort, 
unpredictableSeed) over these six releases. That's what I was 
referring to.
For comparison: in the meantime about 200 new Dub packages have 
been added [1, 2] and in total about 600 Dub packages have been 
updated [3].


2.088:
- std.array.Appender and RefAppender: use .opSlice() instead of 
data()
- ErrnoException.errno is now nothrow pure  nogc  safe
- Nullable alias get this has been deprecated
- Added the std.file.getAvailableDiskSpace functionality.
- Allow std.json to overlook trailing comma

2.087:
- Added a table of control characters in ASCII table
- Count processors via sched_getaffinity on Linux
- Add overload std.algorithm.sorting.schwartzSort!(alias 
transform, SwapStrategy ss, R)
- Phobos is now compiled with -preview=dip1000

2.086:
- Fixed comparison bug in 
std.algorithm.comparison.levenshteinDistance
- std.experimental.all has been moved to std

2.085: (no changes)

2.084:
- Add overload std.random.unpredictableSeed!UIntType

2.083: (no changes)

See more at [4].

[1] 
https://web.archive.org/web/20181229043818/http://code.dlang.org/
[2] https://code.dlang.org/
[3] 
https://code.dlang.org/?sort=updated&category=&skip=500&limit=100
[4] https://dlang.org/changelog/index.html


 and:

     https://github.com/dlang/phobos/pull/7130
     https://github.com/dlang/phobos/pull/7144

 both of which work to remove autodecode from Phobos. 7130 in 
 particular can use some help with anyone who wants to help 
 drive this forward.


I'm sorry and while I like this effort, I don't expect it to help 
because it does break the world.
AFAICT the only way to avoid breaking the world when removing 
autodecode is to ship a new/different version of Phobos 
_together_ with the current one, s.t. it can be opted-in by users.

Anyhow, I think Andrei summarized it better a few months ago:


 The time is ripe for std.v2 
 [https://forum.dlang.org/post/q7j3s0$15n7$1 digitalmars.com]


My earlier post tried to point out that SumType is an excellent 
candidate for v2.
Oct 05 2019
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 6 October 2019 at 03:47:25 UTC, Seb wrote:

 My earlier post tried to point out that SumType is an excellent 
 candidate for v2.


Sorry, Seb, but I don't get this. There's no reason to wait for a 
v2 to introduce a new SumType symbol that outperforms the old 
Variant (assuming it's not possible to just rewrite the latter 
implementation under the hood without changing behaviour).

On the contrary, the best way to do a v2 is likely to be that all 
or most of the stuff we want in it is already there and has been 
battle tested in the wild.
Oct 06 2019
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
Speaking of performance, I was intrigued by the Reddit response 
noting that Rust can go one better by eliminating the error path 
at compile time:
https://www.reddit.com/r/programming/comments/ddi5wb/comment/f2iow4u

The commenter suggests that's because Rust bakes the 
functionality into the compiler instead of making it a library 
type.  Walter,  Andrei, that might be an interesting data point 
for the question of what should have compiler vs. library support?
Oct 06 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 6 October 2019 at 07:18:37 UTC, Joseph Rushton 
Wakeling wrote:

 Speaking of performance, I was intrigued by the Reddit response 
 noting that Rust can go one better by eliminating the error 
 path at compile time:


D can eliminate error paths at compile time too, e.g. static 
assert - which can be used to create all kinds of new useful 
errors. So I am guessing this is just a case of the code needing 
a lil tweak or the compiler being conservative and putting the 
code in even though it is never supposed to happen (like final 
switch keeps an error path because you can do like 
cast(some_enum) value_not_in_enum.... and then better to have the 
assertion failure than UB.)
Oct 06 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 6 October 2019 at 14:08:07 UTC, Adam D. Ruppe wrote:

 D can eliminate error paths at compile time too, e.g. static 
 assert - which can be used to create all kinds of new useful 
 errors. So I am guessing this is just a case of the code 
 needing a lil tweak or the compiler being conservative and 
 putting the code in even though it is never supposed to happen 
 (like final switch keeps an error path because you can do like 
 cast(some_enum) value_not_in_enum.... and then better to have 
 the assertion failure than UB.)


Good to hear. I confess I was a bit mystified about why it should 
be an issue for D or why compiler vs. library implementation 
should make a difference to the ability to eliminate the error 
path (I infer from your remarks that it shouldn't, in principle).

I'm not fluent in assembly so, leaving the error path aside, I 
wasn't sure how to interpret the "main path" assembly from the D 
and Rust code, and whether they were practically equivalent 
(clearly the assembly posted looked different). Can anyone offer 
any interpretation there?
Oct 06 2019
parent Paul Backus <snarwin gmail.com> writes:
On Sunday, 6 October 2019 at 21:49:59 UTC, Joseph Rushton 
Wakeling wrote:

 I'm not fluent in assembly so, leaving the error path aside, I 
 wasn't sure how to interpret the "main path" assembly from the 
 D and Rust code, and whether they were practically equivalent 
 (clearly the assembly posted looked different). Can anyone 
 offer any interpretation there?


Both the D-with-SumType and Rust versions optimize the main path 
of the function down to a single array lookup. The reason they 
look different is that the assembly for the D version uses AT&T 
syntax (which is the default for GNU tools like objdump), and the 
assembly for the Rust version uses Intel syntax.
Oct 06 2019
prev sibling parent reply Seb <seb wilzba.ch> writes:
On Sunday, 6 October 2019 at 07:16:03 UTC, Joseph Rushton 
Wakeling wrote:

 On Sunday, 6 October 2019 at 03:47:25 UTC, Seb wrote:

 My earlier post tried to point out that SumType is an 
 excellent candidate for v2.


 Sorry, Seb, but I don't get this. There's no reason to wait for 
 a v2 to introduce a new SumType symbol that outperforms the old 
 Variant (assuming it's not possible to just rewrite the latter 
 implementation under the hood without changing behaviour).


Well, my guess it will be similar to that one time when `Option` 
(https://github.com/dlang/phobos/pull/3915) was rejected because 
it was only slightly better than `only` or the other time when it 
was attempted to improve `Nullable` (e.g. 
https://github.com/dlang/phobos/pull/6253)?
Alternatively, there is also the route pioneered by 
std.typecons.wrap whose improvements are - after more than four 
years - still stuck in std.experimental.wrap 
(https://github.com/dlang/phobos/pull/2945, 
https://dlang.org/phobos/std_experimental_typecons.html#.wrap).



 On the contrary, the best way to do a v2 is likely to be that 
 all or most of the stuff we want in it is already there and has 
 been battle tested in the wild.


A standard library is _not_ supposed to be a place where actual 
battle-testing happens. It's where things move when they have 
been tested and no longer change. This "readiness" condition is 
one of the reasons why only a single function has been added to 
Phobos in 2019 so far.

Please don't misunderstand me: I would love to see SumType in 
Phobos. There are tons of other great modules that would make a 
great base as well, e.g. 
http://rumbu13.github.io/decimal/doc/decimal.html, 
https://code.dlang.org/packages/optional, or 
https://github.com/dlang-community/std_data_json etc.
I plainly doubt that we will ever see them in the current Phobos.
Oct 06 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 6 October 2019 at 08:27:35 UTC, Seb wrote:

 Well, my guess it will be similar [...]


If you're not the one making those decisions it may be better not 
to prejudge them. A significant performance improvement is a 
different beast to moderate API/usability improvements.


 A standard library is _not_ supposed to be a place where actual 
 battle-testing happens. It's where things move when they have 
 been tested and no longer change.


You misunderstand what I mean by "battle-testing". Clearly 
designs should go through a high level of testing and usage 
before they go anywhere near the standard library. But the very 
fact of being placed in the standard library exposes them to 
orders of magnitude more usage, and hence gives a much stronger 
guarantee of establishing their correctness (or identifying their 
flaws).

It's much better to get newer and apparently better designs 
exposed to this level of scrutiny _before_ making them the only 
option in a new major release. That way you are much less likely 
to get hit by a showstopper edge case that no one anticipated.
Oct 06 2019
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/19 3:09 AM, Joseph Rushton Wakeling wrote:

 You misunderstand what I mean by "battle-testing". Clearly designs 
 should go through a high level of testing and usage before they go 
 anywhere near the standard library. But the very fact of being placed in 
 the standard library exposes them to orders of magnitude more usage, and 
 hence gives a much stronger guarantee of establishing their correctness 
 (or identifying their flaws).


Well also in the case of SumType there's not even much battle-testing. 
It's more of an implementation than a design. Take the existing variant, 
keep the interface, remove the cruft, put in the new nice code. Make it 
pass the unittests, and release. Of course it's an involved process, but 
definitely not more difficult than writing the new code to start with. 
The author receives credit as the primary author of the facility. It is 
wonderful service to the community - not only offers better speed for 
new code, but it instantly enhances speed of existing variant uses.
Oct 06 2019
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 7 October 2019 at 01:09:34 UTC, Andrei Alexandrescu 
wrote:

 Well also in the case of SumType there's not even much 
 battle-testing. It's more of an implementation than a design. 
 Take the existing variant, keep the interface, remove the 
 cruft, put in the new nice code. Make it pass the unittests, 
 and release. Of course it's an involved process, but definitely 
 not more difficult than writing the new code to start with. The 
 author receives credit as the primary author of the facility. 
 It is wonderful service to the community - not only offers 
 better speed for new code, but it instantly enhances speed of 
 existing variant uses.


Just to clarify: SumType isn't, and was never intended to be, a 
drop-in replacement for Algebraic. Their interfaces are similar 
enough that porting code from Algebraic to SumType shouldn't be 
too difficult, but even within the common subset, there are 
incompatibilities. For example, here's a post from an old sumtype 
announcement thread where I discuss the differences between 
SumType's `match` and Algebraic's `visit`:

https://forum.dlang.org/post/xipgrrfljdnyhrhnmsij forum.dlang.org

If SumType is ever added to Phobos, it will have to be alongside 
Algebraic, as an alternative, not as a replacement.
Oct 06 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 7 October 2019 at 01:38:04 UTC, Paul Backus wrote:

 Just to clarify: SumType isn't, and was never intended to be, a 
 drop-in replacement for Algebraic. Their interfaces are similar 
 enough that porting code from Algebraic to SumType shouldn't be 
 too difficult, but even within the common subset, there are 
 incompatibilities. For example, here's a post from an old 
 sumtype announcement thread where I discuss the differences 
 between SumType's `match` and Algebraic's `visit`:

 https://forum.dlang.org/post/xipgrrfljdnyhrhnmsij forum.dlang.org


I don't follow. The visit and match template signatures are 
identical, and from what you describe the SumType match 
implementation should support the same handler choices as visit, 
while allowing extra/more flexible choices.

Where's the breaking change? Are there selections of handlers 
that work for visit and don't work with match?
Oct 07 2019
parent Paul Backus <snarwin gmail.com> writes:
On Monday, 7 October 2019 at 08:07:01 UTC, Joseph Rushton 
Wakeling wrote:

 Where's the breaking change? Are there selections of handlers 
 that work for visit and don't work with match?


Yes.

https://run.dlang.io/is/UpaY2E
Oct 07 2019
prev sibling parent Arredondo <arm.plus gmail.com> writes:
On Sunday, 6 October 2019 at 03:47:25 UTC, Seb wrote:

 Phobos is amazing and stable, but exactly because of these 
 attributes there isn't much active development happening.


I thought that the work on dip1000 and the current push for 
borrow semantics and other lifetime related issues that have 
certainly been receiving a lot of attention were partly motivated 
because Andrei could not write the containers library that he 
wanted to write, and that one of the first applications that 
those things would see is a proper containers library in Phobos.
Oct 07 2019
prev sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Sunday, 6 October 2019 at 02:33:15 UTC, Walter Bright wrote:

 On 10/5/2019 6:58 AM, Seb wrote:

 Phobos is essentially dead/frozen (feature-wise).


 I beg to disagree. A couple cases in point:

     https://github.com/dlang/phobos/pull/7211

 which is a re-imagining, rethinking of hexString.

 and:

     https://github.com/dlang/phobos/pull/7130
     https://github.com/dlang/phobos/pull/7144

 both of which work to remove autodecode from Phobos. 7130 in 
 particular can use some help with anyone who wants to help 
 drive this forward.


Well, so there's hope that _very little_ improvements will be 
merged, in a way or another? I mean, there's some sort of policy 
for things like that:

    https://github.com/dlang/phobos/pull/6730

Frankly speaking, the actual situation it's a little 
discouraging...
Oct 06 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2019 2:59 AM, Paolo Invernizzi wrote:

 Well, so there's hope that _very little_ improvements will be merged, in a way 
 or another? I mean, there's some sort of policy for things like that:
 
     https://github.com/dlang/phobos/pull/6730
 
 Frankly speaking, the actual situation it's a little discouraging...


We want a much higher bar for merging things than historically. A smaller, 
higher quality library is preferable to a kitchen sink library.
Oct 06 2019
next sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Sunday, 6 October 2019 at 19:58:04 UTC, Walter Bright wrote:

 On 10/6/2019 2:59 AM, Paolo Invernizzi wrote:

 Well, so there's hope that _very little_ improvements will be 
 merged, in a way or another? I mean, there's some sort of 
 policy for things like that:
 
     https://github.com/dlang/phobos/pull/6730
 
 Frankly speaking, the actual situation it's a little 
 discouraging...


 We want a much higher bar for merging things than historically. 
 A smaller, higher quality library is preferable to a kitchen 
 sink library.


The pull request I've shown is pretty simple:

- std.socket is ... well... not the best piece of code out there
- the `receive` method is usually in the _hot_ code path
- it's not marked  nogc, and actually it does not allocate

So:

- adding  nogc will break derived classes that redefines the 
method (I still regret that the language was not shifted towards 
"final by default", years ago, as clearly that would be a *great* 
mitigation over that kind of problems)

- adding another method to a class, marked  nogc, and (maybe) 
deprecating the previous method is seen as 'annoying', also if 
it's a _clear_ improvement over the actual situation (you can 
write _better_ code with that in place compared to the actual 
situation, I mean)

I'm on the same boat with you, regarding what you wrote, but ... 
I still don't understand the number printed on the bar level.

There's a number of recurring patterns of simple things to fix 
like the one above, with the same kind of problem to address. I 
humbly suggest the core team to just forge a general recipe for 
some of them, and stick with it, so that the number of the bar is 
less blurred. That scales, and encourage contribution.

So, what do you think about starting a first one bases on cases 
similar to the above?
Oct 07 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2019 12:37 AM, Paolo Invernizzi wrote:

 - adding another method to a class, marked  nogc, and (maybe) deprecating the 
 previous method is seen as 'annoying', also if it's a _clear_ improvement over 
 the actual situation (you can write _better_ code with that in place compared
to 
 the actual situation, I mean)


 nogc doesn't actually enable writing better code. It doesn't change the 
generated code at all.



 I'm on the same boat with you, regarding what you wrote, but ... I still don't 
 understand the number printed on the bar level.


Atila is in charge of this, and he is because he's shown excellent judgement 
about these matters over the years.
Oct 11 2019
next sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Saturday, 12 October 2019 at 03:10:29 UTC, Walter Bright wrote:

 On 10/7/2019 12:37 AM, Paolo Invernizzi wrote:

 - adding another method to a class, marked  nogc, and (maybe) 
 deprecating the previous method is seen as 'annoying', also if 
 it's a _clear_ improvement over the actual situation (you can 
 write _better_ code with that in place compared to the actual 
 situation, I mean)


  nogc doesn't actually enable writing better code. It doesn't 
 change the generated code at all.


I meant, writing better _source_ code, especially for reviewers.


 I'm on the same boat with you, regarding what you wrote, but 
 ... I still don't understand the number printed on the bar 
 level.


 Atila is in charge of this, and he is because he's shown 
 excellent judgement about these matters over the years.


I'm faithful for Atila judgement, and at the same time I've 
always liked also your pragmatism. Anyway, I'll sit waiting for a 
policy decision on cases similar to the one mentioned.
Oct 14 2019
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 12 October 2019 at 03:10:29 UTC, Walter Bright wrote:

 On 10/7/2019 12:37 AM, Paolo Invernizzi wrote:

 - adding another method to a class, marked  nogc, and (maybe) 
 deprecating the previous method is seen as 'annoying', also if 
 it's a _clear_ improvement over the actual situation (you can 
 write _better_ code with that in place compared to the actual 
 situation, I mean)


  nogc doesn't actually enable writing better code. It doesn't 
 change the generated code at all.



 I'm on the same boat with you, regarding what you wrote, but 
 ... I still don't understand the number printed on the bar 
 level.


 Atila is in charge of this, and he is because he's shown 
 excellent judgement about these matters over the years.


I think that I need to ruminate on Phobos v2.

In the meanwhile, a much easier and shorter route to improving 
the D library ecosystem is to put something up on code.dlang.org, 
which requires no gatekeeping.
Oct 16 2019
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 16 October 2019 at 10:56:40 UTC, Atila Neves wrote:

 On Saturday, 12 October 2019 at 03:10:29 UTC, Walter Bright 
 wrote:

 On 10/7/2019 12:37 AM, Paolo Invernizzi wrote:

 - adding another method to a class, marked  nogc, and (maybe) 
 deprecating the previous method is seen as 'annoying', also 
 if it's a _clear_ improvement over the actual situation (you 
 can write _better_ code with that in place compared to the 
 actual situation, I mean)


  nogc doesn't actually enable writing better code. It doesn't 
 change the generated code at all.



 I'm on the same boat with you, regarding what you wrote, but 
 ... I still don't understand the number printed on the bar 
 level.


 Atila is in charge of this, and he is because he's shown 
 excellent judgement about these matters over the years.


 I think that I need to ruminate on Phobos v2.

 In the meanwhile, a much easier and shorter route to improving 
 the D library ecosystem is to put something up on 
 code.dlang.org, which requires no gatekeeping.


While I agree on the ecosystem, the problem of keeping the actual 
Phobos modules in a good shape still apply.

Please take a look at the cited pull request: it's a *trivial* 
Phobos patch, that can be added aside to the current 
implementation, blocked for months waiting for a _political_ 
decision.

I understand that "there's always something else better for the 
language to do", but Phobos is the current "home sweet home" for 
everyone approaching D, and it's the first library inspected in 
details.

It's simply embarrassing to explain to an external reviewer that 
a standard library method signature is inaccurate after 88 
releases of version 2 of the language. And that yes, 'assumeNoGC' 
is needed, 'trust' that, and yes, an issue was filed along with a 
potential fix.

I've full faith in your and Walter judgement, of course.
Oct 16 2019
parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 16 October 2019 at 12:32:28 UTC, Paolo Invernizzi 
wrote:

 On Wednesday, 16 October 2019 at 10:56:40 UTC, Atila Neves 
 wrote:

 On Saturday, 12 October 2019 at 03:10:29 UTC, Walter Bright 
 wrote:

 On 10/7/2019 12:37 AM, Paolo Invernizzi wrote:

 - adding another method to a class, marked  nogc, and 
 (maybe) deprecating the previous method is seen as 
 'annoying', also if it's a _clear_ improvement over the 
 actual situation (you can write _better_ code with that in 
 place compared to the actual situation, I mean)


  nogc doesn't actually enable writing better code. It doesn't 
 change the generated code at all.



 I'm on the same boat with you, regarding what you wrote, but 
 ... I still don't understand the number printed on the bar 
 level.


 Atila is in charge of this, and he is because he's shown 
 excellent judgement about these matters over the years.


 I think that I need to ruminate on Phobos v2.

 In the meanwhile, a much easier and shorter route to improving 
 the D library ecosystem is to put something up on 
 code.dlang.org, which requires no gatekeeping.


 While I agree on the ecosystem, the problem of keeping the 
 actual Phobos modules in a good shape still apply.

 Please take a look at the cited pull request: it's a *trivial* 
 Phobos patch, that can be added aside to the current 
 implementation, blocked for months waiting for a _political_ 
 decision.


I don't think it's political: the change implies breakage for 
downstream users who inherit from the class who might not even 
care about  nogc. This a technical point. The easiest way out in 
my opinion is to to inherit from it yourself and mark `receive` 
 nogc, as was suggested in the PR.


 I understand that "there's always something else better for the 
 language to do", but Phobos is the current "home sweet home" 
 for everyone approaching D, and it's the first library 
 inspected in details.


I understand that and sympathise.


 It's simply embarrassing to explain to an external reviewer 
 that a standard library method signature is inaccurate after 88 
 releases of version 2 of the language. And that yes, 
 'assumeNoGC' is needed, 'trust' that, and yes, an issue was 
 filed along with a potential fix.


Indeed. We're hardly alone in this: std::auto_ptr was/is an 
embarassment in C++. Then there's std::vector<bool>...
Oct 16 2019
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 16 October 2019 at 15:01:17 UTC, Atila Neves wrote:


 Please take a look at the cited pull request: it's a *trivial* 
 Phobos patch, that can be added aside to the current 
 implementation, blocked for months waiting for a _political_ 
 decision.


 I don't think it's political: the change implies breakage for 
 downstream users who inherit from the class who might not even 
 care about  nogc.


The proposed solution is to "add" a new  nogc method, with the 
correct signature, so that if someone want to write application 
and care about  nogc and  safe can rely on the D standard library 
being complaint to that.

What's the problem with that, if not a _political_ one? We have a 
"wrong" signature, we don't break anything, but we add "correct" 
signature. That's what already was done in Mutex with 
lock_nothrow, but it's seen as "annoying to have to define/use 
alternate names for all the methods, though"


 This a technical point. The easiest way out in my opinion is to 
 to inherit from it yourself and mark `receive`  nogc, as was 
 suggested in the PR.


That's can't be done without a cast, so we need to rely on 
trusted, and we go again to the starting point, as stated the 
pull request.


 It's simply embarrassing to explain to an external reviewer 
 that a standard library method signature is inaccurate after 
 88 releases of version 2 of the language. And that yes, 
 'assumeNoGC' is needed, 'trust' that, and yes, an issue was 
 filed along with a potential fix.


 Indeed. We're hardly alone in this: std::auto_ptr was/is an 
 embarassment in C++. Then there's std::vector<bool>...


And that's why I'm throwing a stone in the water: what's the 
'concrete' procedures that the gatekeepers have in mind to 
improve Phobos quality for cases like that?

Atila, that's really a _little_ change, if that can't be handled 
easily, what about handling _big_ changes?
Oct 16 2019
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 16 October 2019 at 15:23:01 UTC, Paolo Invernizzi 
wrote:

 On Wednesday, 16 October 2019 at 15:01:17 UTC, Atila Neves 
 wrote:


 [...]


 I don't think it's political: the change implies breakage for 
 downstream users who inherit from the class who might not even 
 care about  nogc.


 The proposed solution is to "add" a new  nogc method, with the 
 correct signature, so that if someone want to write application 
 and care about  nogc and  safe can rely on the D standard 
 library being complaint to that.

 What's the problem with that, if not a _political_ one? We have 
 a "wrong" signature, we don't break anything, but we add 
 "correct" signature. That's what already was done in Mutex with 
 lock_nothrow, but it's seen as "annoying to have to define/use 
 alternate names for all the methods, though"


Oh. I missed that.
Oct 16 2019
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Sunday, 6 October 2019 at 19:58:04 UTC, Walter Bright wrote:

 On 10/6/2019 2:59 AM, Paolo Invernizzi wrote:

 Well, so there's hope that _very little_ improvements will be 
 merged, in a way or another? I mean, there's some sort of 
 policy for things like that:
 
     https://github.com/dlang/phobos/pull/6730
 
 Frankly speaking, the actual situation it's a little 
 discouraging...


 We want a much higher bar for merging things than historically. 
 A smaller, higher quality library is preferable to a kitchen 
 sink library.


There is someone actively adding auto-merge labels to pull 
requests even when the pull request author specifically says the 
PR is not ready. So the bar has actually been lowered in recent 
times..

I'm not going to name names because it would be inappropriate, 
but people are beginning to notice.
Oct 08 2019
prev sibling next sibling parent Johan Engelen <j j.nl> writes:
On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options 
 we have in D, like Algebraic and SumType, so I did a simple 
 comparison of the generated assembly for each of them. You can 
 read about it at the link below. And as you can probably guess 
 from the title, D comes out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

 This is my first attempt at sharing something like this, so any 
 comment or feedback is very much appreciated!


It'd be interesting if you could investigate the case of 
non-trivial lambdas, where inlining the lambdas is not 
profitable. Perhaps easiest if you call opaque (declaration-only) 
functions in the visit list.

Conclusion, performance-wise:
- SumType is great
- std::visit is OK'ish
- D Algebraic is terrible
Do you agree?

cheers,
   Johan
Oct 05 2019
prev sibling next sibling parent user1234 <user1234 12.de> writes:
On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options 
 we have in D, like Algebraic and SumType, so I did a simple 
 comparison of the generated assembly for each of them. You can 
 read about it at the link below. And as you can probably guess 
 from the title, D comes out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

 This is my first attempt at sharing something like this, so any 
 comment or feedback is very much appreciated!


getting typeinfo in D leads to a double indirection. Making a 
tagged union is clearly a better alternative and the results of 
SumType vs Algebraic are not a surprise. I pointed this to the 
main maintainer of D-YAML a while back but at this time there was 
more serious bottlenecks. If you read this "El Pinguino"...
Oct 05 2019
prev sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options 
 we have in D, like Algebraic and SumType, so I did a simple 
 comparison of the generated assembly for each of them. You can 
 read about it at the link below. And as you can probably guess 
 from the title, D comes out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

 This is my first attempt at sharing something like this, so any 
 comment or feedback is very much appreciated!


Reddit:
https://www.reddit.com/r/programming/comments/ddmlfy/beating_stdvisit_without_really_trying/
Oct 05 2019
parent reply Mike Parker <aldacron gmail.com> writes:
On Saturday, 5 October 2019 at 12:27:06 UTC, Mike Parker wrote:

 On Saturday, 5 October 2019 at 02:59:58 UTC, Paul Backus wrote:

 I was curious how C++17's std::variant compared to the options 
 we have in D, like Algebraic and SumType, so I did a simple 
 comparison of the generated assembly for each of them. You can 
 read about it at the link below. And as you can probably guess 
 from the title, D comes out ahead, in the end.

 https://pbackus.github.io/blog/beating-stdvisit-without-really-trying.html

 This is my first attempt at sharing something like this, so 
 any comment or feedback is very much appreciated!


 Reddit:
 https://www.reddit.com/r/programming/comments/ddmlfy/beating_stdvisit_without_really_trying/


Correction:

https://www.reddit.com/r/programming/comments/ddi5wb/beating_stdvisit_without_really_trying/

I missed that someone had already shared it.
Oct 05 2019
parent reply SrMordred <patric.dexheimer gmail.com> writes:
I wonder about the generated Assembly:

only two lines.


In D, Look like the first assembly lines are checking for 
errors(or out of bounds index?).
But why if this is a final switch? (also assert(0); at the end 
don´t help )
Oct 05 2019
parent reply Dennis <dkorpel gmail.com> writes:
On Sunday, 6 October 2019 at 00:58:17 UTC, SrMordred wrote:

 But why if this is a final switch? (also assert(0); at the end 
 don´t help )


Because the optimization hasn't been implemented yet.
https://issues.dlang.org/show_bug.cgi?id=13169
Oct 06 2019
parent SrMordred <patric.dexheimer gmail.com> writes:
On Sunday, 6 October 2019 at 08:31:20 UTC, Dennis wrote:

 On Sunday, 6 October 2019 at 00:58:17 UTC, SrMordred wrote:

 But why if this is a final switch? (also assert(0); at the end 
 don´t help )


 Because the optimization hasn't been implemented yet.
 https://issues.dlang.org/show_bug.cgi?id=13169


1) Nice to know that is possible.
2) 2014 ... ouch.
Oct 06 2019