www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - I wish all qualifiers were revisited with an eye for simplification

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
(Background: qualifiers were introduced following my horror when I 
started writing D1 code and saw that strings are represented as char[]. 
So structs with string members would look like:

struct Widget
{
     int x, y;
     char[] name, parentName;
     ...
}

I found it just shocking that following a Widget's construction whoever 
aliased the same strings the outside could change members of the Widget 
without Widget knowing. Of course, other D1 coders disliked that as 
well, so they'd defensively duplicate in the constructor:

struct Widget
{
     int x, y;
     char[] name, parentName;
     this(char[] n, char[] p)
     {
         name = n.dup;
         parentName = p.dup;
     }
     ...
}

thus ensuring proliferation of the garbage whether duplication was 
needed or not.

I found this absolutely maddening, to the extent I didn't think D could 
be ever used at any considerable scale while dragging this anchor behind it.

The second problem was Walter was adamant about using arrays of 
characters at strings. He found the notion of a library-defined string 
type (a la C++) an absolute abomination. Stubborn about it like I've 
never seen him before or after. So my unstoppable requests for a string 
type were met with the proverbial immovable refusal. Ironically, much of 
his argument came from an efficiency angle, yet the unnecessary 
duplication was way less efficient than some reference counting/small 
string optimization/etc scheme that a dedicated string type would use.

Then we figured things would work out if we arranged things such that 
people could NOT change individual characters of a string. That would 
allow sharing without the danger of long-distance influence. After many 
discussions with Walter, Bartosz, Eric, Brad, and myself, immutable and 
const were born.

Then followed the other qualifiers, in order: shared and inout.)

* * *

The result is... there: https://dlang.org/spec/const3.html. It has the 
images https://dlang.org/images/qualifier-combinations.svg and 
https://dlang.org/images/qualifier-conversions.svg and a large table and 
a lot of rules. Whenever I code anything generic, I find myself going 
back to those damn images and tables way more than anyone ought to. (And 
it's ironic... I made those. Woe to the relative newcomer.)

It's all too complicated, making generic D programming into 3D chess 
instead of the difficult endeavor it already is. And what does it buy 
us? Well, we don't need to define a string library type. Yowzers. 
(Actually we should if we want to get rid of the GC. But then Walter 
would oppose that. So - stalemate once again.)

Far as I can tell, the power/weight ratio of qualifier is very poor. I 
wish a DIP would revisit qualifiers with a stated intent to simplify 
them as much as possible. Whenever I code generically I invariably run 
into these issues:

* Whatever I do, however I twitch, immutable finds the opportunity to 
lodge itself in a soft part of my body and cause constant pain. This 
doesn't work, that doesn't work. No solution for "tail immutable" - 
mutable references to immutable class instances can't be done without 
contortions. Can't assign to out immutable class references, though 
there's no reason for that (see Adam's recent post).

* No matter how I dice any significant piece of code, there will be five 
casts from immutable and/or back that I can't rid of and lose sleep at 
night trying to convince myself are justified.

* Every time "inout" comes within a radius of a mile of what I'm doing, 
it starts to stink like a skunk. I wish I could get a restraining order. 
I can't instantiate "inout" variables, so writing any tests or 
constraints becomes an advanced matter of defining functions and crap. I 
get frustrated, I protest to this forum, and immediately a cabal is 
raised under Timon's leadership. The cabal convinces me that inout is 
actually great and that I'm an idiot. I do get convinced, which is more 
of a proof that Timon is very good, than a testament to the conviviality 
of inout. Then I leave and get back to my code, and it stinks of inout 
again. And I hate it and myself for having to deal with it.

* Nobody - probably not even Timon - knows what "shared" does or is 
supposed to do and not do. The most I got from Walter ever is "shared is 
intentionally restricted so you can't do much without a cast". Yet the 
definition of "much" and the conditions under which casting is legit are 
not anywhere to be found.

* And of course, "shared" gladly partakes in qualifier combinations, 
thus spreading its stink around in a combinatorial manner. Believe it or 
not, the type "const inout shared T" exists. Of course, nobody knows 
what it really means or how it could be used.

A "Define All Qualifiers" DIP would be a radical improvement of the 
state of affairs.
Aug 02 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
wrote:
 The cabal convinces me that inout is actually great and that 
 I'm an idiot.
inout is great, but you are also not wrong about it. It has frustrating arbitrary limitations that should really just get lifted. Steven designed it conservatively thinking it was a good idea at the time but agrees now it was a mistake to limit it like this. Simplest step we could do is define inout == const if used for a local variable in a non-inout function. I think that'd eliminate the majority of the headache. The `is` expression should be more strict, but a local declaration should just fallback to const for practicality.
 Believe it or not, the type "const inout shared T" exists. Of 
 course, nobody knows what it really means or how it could be 
 used.
I don't even have an idea lol
Aug 02 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/2/20 5:30 PM, Adam D. Ruppe wrote:
 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:
 The cabal convinces me that inout is actually great and that I'm an 
 idiot.
inout is great, but you are also not wrong about it. It has frustrating arbitrary limitations that should really just get lifted.
Yes that would help a lot.
Aug 02 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/2/20 6:22 PM, Andrei Alexandrescu wrote:
 On 8/2/20 5:30 PM, Adam D. Ruppe wrote:
 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:
 The cabal convinces me that inout is actually great and that I'm an 
 idiot.
inout is great, but you are also not wrong about it. It has frustrating arbitrary limitations that should really just get lifted.
Yes that would help a lot.
If I could hack dmd, I would make this my first priority... -Steve
Aug 03 2020
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
wrote:
 (Background: qualifiers were introduced following my horror 
 when I started writing D1 code and saw that strings are 
 represented as char[]. So structs with string members would 
 look like:
[lots of stuff] const inout shared T is simply the same as const shared T. shared means simply: this is not thread local, therefore this thing might be used by other threads therefore you read or write it in an expression directly. You can however have a function take a shared argument and leave the scheduling or locking or whatever synchronization you use, to enact an operation in a safe way.
Aug 02 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/2/20 6:55 PM, Stefan Koch wrote:
 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:
 (Background: qualifiers were introduced following my horror when I 
 started writing D1 code and saw that strings are represented as 
 char[]. So structs with string members would look like:
 [lots of stuff] const inout shared T is simply the same as const shared T.
Sadly no: https://run.dlang.io/is/etdPtP
 shared means simply: this is not thread local,
 therefore this thing might be used by other threads
 therefore you read or write it in an expression directly.
 You can however have a function take a shared argument and leave
 the scheduling or locking or whatever synchronization you use,
 to enact an operation in a safe way.
Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
Aug 02 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 3 August 2020 at 01:23:26 UTC, Andrei Alexandrescu 
wrote:
 On 8/2/20 6:55 PM, Stefan Koch wrote:
 [...]
Sadly no: https://run.dlang.io/is/etdPtP
 [...]
Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
No it should not. With the right -preview switch it won't compile. The programmer is to ensure that atomics or other needed syncronisation primitives are used.
Aug 02 2020
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, August 2, 2020 7:23:26 PM MDT Andrei Alexandrescu via Digitalmars-d 
wrote:
 On 8/2/20 6:55 PM, Stefan Koch wrote:
 shared means simply: this is not thread local,
 therefore this thing might be used by other threads
 therefore you read or write it in an expression directly.
 You can however have a function take a shared argument and leave
 the scheduling or locking or whatever synchronization you use,
 to enact an operation in a safe way.
Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
In princip le, shared should be preventing any and all read-write operations which it cannot guarantee are atomic (which probably means banning all read-write operations, since that's going to vary across architectures). All code with shared then either needs to either use atomics or cast away shared and use mutexes, semaphores, or whatever other threading primitive is appropriate to ensure that the data is only accessed by that single thread while it's typed as thread-local, leaving it up to the programmer to then ensure that they're dealing with the threading correctly (just like in C++). However, unlike C++, all of the code operating on data shared across threads would then be segregated (it would also likely be system in most cases). More complex objects can then have shared member functions which deal with all of the threading stuff internally rather than requiring that everyone using them deal with it directly, but at the core, it's ultimately going to be a question of atomics or casts if we want the guarantee that shared objects aren't being accessed across threads when they're not properly protected. However, the core problem with all of this is that the restrictions that need to be put on shared have never really been put in place. For years, about all that it really did was disallow implicit conversions between shared and thread-local (and even then, it didn't always do it - e.g. with integers). Over time, additional restrictions have been added (I think mostly to numeric types), but it's all piecemeal. IIRC, there was a DIP that was supposed to outright make reading and writing to shared variables illegal, but I'm not quite sure where that stands. We also potentially need some language improvements to make it cleaner, because otherwise in order to read or write value types, we'll have to do stuff like take their address and cast the resulting pointer so that we can operate on the object as thread-local while the mutex is locked (or whatever or threading protections are in place). So, while I think that the basic idea of shared is solid, it's never been properly implemented. - Jonathan M Davis
Aug 12 2020
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, August 13, 2020 12:44:54 AM MDT Jonathan M Davis via Digitalmars-
d wrote:
 On Sunday, August 2, 2020 7:23:26 PM MDT Andrei Alexandrescu via
 Digitalmars-d
 wrote:
 On 8/2/20 6:55 PM, Stefan Koch wrote:
 shared means simply: this is not thread local,
 therefore this thing might be used by other threads
 therefore you read or write it in an expression directly.
 You can however have a function take a shared argument and leave
 the scheduling or locking or whatever synchronization you use,
 to enact an operation in a safe way.
Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
In princip le, shared should be preventing any and all read-write operations which it cannot guarantee are atomic (which probably means banning all read-write operations, since that's going to vary across architectures). All code with shared then either needs to either use atomics or cast away shared and use mutexes, semaphores, or whatever other threading primitive is appropriate to ensure that the data is only accessed by that single thread while it's typed as thread-local, leaving it up to the programmer to then ensure that they're dealing with the threading correctly (just like in C++). However, unlike C++, all of the code operating on data shared across threads would then be segregated (it would also likely be system in most cases). More complex objects can then have shared member functions which deal with all of the threading stuff internally rather than requiring that everyone using them deal with it directly, but at the core, it's ultimately going to be a question of atomics or casts if we want the guarantee that shared objects aren't being accessed across threads when they're not properly protected. However, the core problem with all of this is that the restrictions that need to be put on shared have never really been put in place. For years, about all that it really did was disallow implicit conversions between shared and thread-local (and even then, it didn't always do it - e.g. with integers). Over time, additional restrictions have been added (I think mostly to numeric types), but it's all piecemeal. IIRC, there was a DIP that was supposed to outright make reading and writing to shared variables illegal, but I'm not quite sure where that stands. We also potentially need some language improvements to make it cleaner, because otherwise in order to read or write value types, we'll have to do stuff like take their address and cast the resulting pointer so that we can operate on the object as thread-local while the mutex is locked (or whatever or threading protections are in place). So, while I think that the basic idea of shared is solid, it's never been properly implemented.
LOL. And reading through the rest of the thread, I see that Manu already brought this up and that the DIP has an implementation with the -preview flag. - Jonathan M Davis
Aug 13 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Thu, Aug 13, 2020 at 5:24 PM Jonathan M Davis via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Thursday, August 13, 2020 12:44:54 AM MDT Jonathan M Davis via
 Digitalmars-
 d wrote:
 On Sunday, August 2, 2020 7:23:26 PM MDT Andrei Alexandrescu via
 Digitalmars-d
 wrote:
 On 8/2/20 6:55 PM, Stefan Koch wrote:
 shared means simply: this is not thread local,
 therefore this thing might be used by other threads
 therefore you read or write it in an expression directly.
 You can however have a function take a shared argument and leave
 the scheduling or locking or whatever synchronization you use,
 to enact an operation in a safe way.
Yes, the intent was generous. The realization, not quite. However, I
did
 learn from Walter that a number of invalid operations on shared
numerics
 have now been disabled. However, copying a shared int into an int is
 allowed and as far as I know with no special treatment:

 https://run.dlang.io/is/ZP124F

 At least on ARM that should generate a read barrier.
In princip le, shared should be preventing any and all read-write
operations
 which it cannot guarantee are atomic (which probably means banning all
 read-write operations, since that's going to vary across architectures)=
.
 All code with shared then either needs to either use atomics or cast aw=
ay
 shared and use mutexes, semaphores, or whatever other threading primiti=
ve
 is appropriate to ensure that the data is only accessed by that single
 thread while it's typed as thread-local, leaving it up to the programme=
r
 to
 then ensure that they're dealing with the threading correctly (just lik=
e
 in
 C++). However, unlike C++, all of the code operating on data shared
across
 threads would then be segregated (it would also likely be  system in mo=
st
 cases). More complex objects can then have shared member functions whic=
h
 deal with all of the threading stuff internally rather than requiring
that
 everyone using them deal with it directly, but at the core, it's
ultimately
 going to be a question of atomics or casts if we want the guarantee tha=
t
 shared objects aren't being accessed across threads when they're not
 properly protected.

 However, the core problem with all of this is that the restrictions tha=
t
 need to be put on shared have never really been put in place. For years=
,
 about all that it really did was disallow implicit conversions between
 shared and thread-local (and even then, it didn't always do it - e.g.
with
 integers). Over time, additional restrictions have been added (I think
 mostly to numeric types), but it's all piecemeal. IIRC, there was a DIP
that
 was supposed to outright make reading and writing to shared variables
 illegal, but I'm not quite sure where that stands. We also potentially
need
 some language improvements to make it cleaner, because otherwise in ord=
er
 to read or write value types, we'll have to do stuff like take their
 address and cast the resulting pointer so that we can operate on the
object
 as thread-local while the mutex is locked (or whatever or threading
 protections are in place).

 So, while I think that the basic idea of shared is solid, it's never be=
en
 properly implemented.
LOL. And reading through the rest of the thread, I see that Manu already brought this up and that the DIP has an implementation with the -preview flag. - Jonathan M Davis
=F0=9F=91=8D You're right though! :P
Aug 13 2020
prev sibling next sibling parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
wrote:

 * Whatever I do, however I twitch, immutable finds the 
 opportunity to lodge itself in a soft part of my body and cause 
 constant pain. This doesn't work, that doesn't work. No 
 solution for "tail immutable" - mutable references to immutable 
 class instances can't be done without contortions. Can't assign 
 to out immutable class references, though there's no reason for 
 that (see Adam's recent post).
Maybe a library implementation of `TailMutable(T) if (is(T == immutable) || is(T == const))` for reference types can fix this without causing breakage and leaving immutable/const transitivity intact.
Aug 02 2020
parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Monday, 3 August 2020 at 03:14:15 UTC, RazvanN wrote:
 Maybe a library implementation of `TailMutable(T) if (is(T == 
 immutable) || is(T == const))` for reference types can fix this 
 without causing breakage and leaving immutable/const 
 transitivity intact.
There is already Rebindable structure for that.
Aug 03 2020
prev sibling next sibling parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
wrote:
 * Nobody - probably not even Timon - knows what "shared" does 
 or is supposed to do and not do. The most I got from Walter 
 ever is "shared is intentionally restricted so you can't do 
 much without a cast". Yet the definition of "much" and the 
 conditions under which casting is legit are not anywhere to be 
 found.
Shared today is a half-baked implementation of a half-thought idea. Manu wrote[0] almost two years ago what it should be: you can't read or write from a shared object, and only shared methods may be called on it. This was made into a DIP[1], and is half-way(?) implemented in the compiler as -preview=nosharedaccess. The semantics are effective and easy to understand. -- Simen [0]: https://forum.dlang.org/post/mailman.4299.1539629222.29801.digitalmars-d puremagic.com [1]: https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1024.md
Aug 03 2020
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/3/20 7:45 AM, Simen Kjærås wrote:
 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:
 * Nobody - probably not even Timon - knows what "shared" does or is 
 supposed to do and not do. The most I got from Walter ever is "shared 
 is intentionally restricted so you can't do much without a cast". Yet 
 the definition of "much" and the conditions under which casting is 
 legit are not anywhere to be found.
Shared today is a half-baked implementation of a half-thought idea. Manu wrote[0] almost two years ago what it should be: you can't read or write from a shared object, and only shared methods may be called on it. This was made into a DIP[1], and is half-way(?) implemented in the compiler as -preview=nosharedaccess. The semantics are effective and easy to understand. --   Simen [0]: https://forum.dlang.org/post/mailman.4299.1539629222.29801.digital ars-d puremagic.com [1]: https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1024.md
Found out about that after I wrote my diatribe. Thanks!
Aug 03 2020
prev sibling parent Bruce Carneal <bcarneal gmail.com> writes:
On Monday, 3 August 2020 at 11:45:22 UTC, Simen Kjærås wrote:
 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
 wrote:
 [...]
Shared today is a half-baked implementation of a half-thought idea. Manu wrote[0] almost two years ago what it should be: you can't read or write from a shared object, and only shared methods may be called on it. This was made into a DIP[1], and is half-way(?) implemented in the compiler as -preview=nosharedaccess. The semantics are effective and easy to understand. -- Simen [0]: https://forum.dlang.org/post/mailman.4299.1539629222.29801.digitalmars-d puremagic.com [1]: https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1024.md
Very much looking forward to activating shared with -preview=nosharedaccess. Will do so widely once "hello world" compiles again.
Aug 03 2020
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Mon, Aug 3, 2020 at 6:55 AM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 (Background: qualifiers were introduced following my horror when I
 started writing D1 code and saw that strings are represented as char[].
 So structs with string members would look like:

 struct Widget
 {
      int x, y;
      char[] name, parentName;
      ...
 }

 I found it just shocking that following a Widget's construction whoever
 aliased the same strings the outside could change members of the Widget
 without Widget knowing. Of course, other D1 coders disliked that as
 well, so they'd defensively duplicate in the constructor:

 struct Widget
 {
      int x, y;
      char[] name, parentName;
      this(char[] n, char[] p)
      {
          name = n.dup;
          parentName = p.dup;
      }
      ...
 }

 thus ensuring proliferation of the garbage whether duplication was
 needed or not.

 I found this absolutely maddening, to the extent I didn't think D could
 be ever used at any considerable scale while dragging this anchor behind
 it.

 The second problem was Walter was adamant about using arrays of
 characters at strings. He found the notion of a library-defined string
 type (a la C++) an absolute abomination. Stubborn about it like I've
 never seen him before or after. So my unstoppable requests for a string
 type were met with the proverbial immovable refusal. Ironically, much of
 his argument came from an efficiency angle, yet the unnecessary
 duplication was way less efficient than some reference counting/small
 string optimization/etc scheme that a dedicated string type would use.

 Then we figured things would work out if we arranged things such that
 people could NOT change individual characters of a string. That would
 allow sharing without the danger of long-distance influence. After many
 discussions with Walter, Bartosz, Eric, Brad, and myself, immutable and
 const were born.

 Then followed the other qualifiers, in order: shared and inout.)

 * * *

 The result is... there: https://dlang.org/spec/const3.html. It has the
 images https://dlang.org/images/qualifier-combinations.svg and
 https://dlang.org/images/qualifier-conversions.svg and a large table and
 a lot of rules. Whenever I code anything generic, I find myself going
 back to those damn images and tables way more than anyone ought to. (And
 it's ironic... I made those. Woe to the relative newcomer.)

 It's all too complicated, making generic D programming into 3D chess
 instead of the difficult endeavor it already is. And what does it buy
 us? Well, we don't need to define a string library type. Yowzers.
 (Actually we should if we want to get rid of the GC. But then Walter
 would oppose that. So - stalemate once again.)

 Far as I can tell, the power/weight ratio of qualifier is very poor. I
 wish a DIP would revisit qualifiers with a stated intent to simplify
 them as much as possible. Whenever I code generically I invariably run
 into these issues:

 * Whatever I do, however I twitch, immutable finds the opportunity to
 lodge itself in a soft part of my body and cause constant pain. This
 doesn't work, that doesn't work. No solution for "tail immutable" -
 mutable references to immutable class instances can't be done without
 contortions. Can't assign to out immutable class references, though
 there's no reason for that (see Adam's recent post).

 * No matter how I dice any significant piece of code, there will be five
 casts from immutable and/or back that I can't rid of and lose sleep at
 night trying to convince myself are justified.

 * Every time "inout" comes within a radius of a mile of what I'm doing,
 it starts to stink like a skunk. I wish I could get a restraining order.
 I can't instantiate "inout" variables, so writing any tests or
 constraints becomes an advanced matter of defining functions and crap. I
 get frustrated, I protest to this forum, and immediately a cabal is
 raised under Timon's leadership. The cabal convinces me that inout is
 actually great and that I'm an idiot. I do get convinced, which is more
 of a proof that Timon is very good, than a testament to the conviviality
 of inout. Then I leave and get back to my code, and it stinks of inout
 again. And I hate it and myself for having to deal with it.

 * Nobody - probably not even Timon - knows what "shared" does or is
 supposed to do and not do. The most I got from Walter ever is "shared is
 intentionally restricted so you can't do much without a cast". Yet the
 definition of "much" and the conditions under which casting is legit are
 not anywhere to be found.

 * And of course, "shared" gladly partakes in qualifier combinations,
 thus spreading its stink around in a combinatorial manner. Believe it or
 not, the type "const inout shared T" exists. Of course, nobody knows
 what it really means or how it could be used.

 A "Define All Qualifiers" DIP would be a radical improvement of the
 state of affairs.
Shared recently received a `-preview` which makes it really mean something; this is what shared means: 1. The data is shared; therefore, it is invalid to read or write that memory. 2. The reason this is useful as an attribute, is because you are able to attribute methods. Ability to author a set of threadsafe methods and clearly distinguish them from non-threadsafe methods is a powerful advantage over C++. isn't strengthened, then it is my opinion that `shared` is completely useless, and `struct Shared(T) { private: T* sharedThing; }` is just as good. I think shared has potential to offer critical benefit to D, and we've talked about this personally to some length. Don't kill it yet, let's try and fix it. Although it's worth recognising that if we don't fix it, then it might as well be killed as it stands today.
Aug 03 2020
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/3/20 5:56 PM, Manu wrote:
 I think shared has potential to offer critical benefit to D, and we've 
 talked about this personally to some length. Don't kill it yet, let's 
 try and fix it. Although it's worth recognising that if we don't fix it, 
 then it might as well be killed as it stands today.
Of course fixing it would be great! I'm glad you pushed the restriction through. At least now generic code could see `shared` as "shrouded in opacity, not subject to the usual operations".
Aug 03 2020
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04.08.20 04:13, Andrei Alexandrescu wrote:
 On 8/3/20 5:56 PM, Manu wrote:
 I think shared has potential to offer critical benefit to D, and we've 
 talked about this personally to some length. Don't kill it yet, let's 
 try and fix it. Although it's worth recognising that if we don't fix 
 it, then it might as well be killed as it stands today.
Of course fixing it would be great!
It's being fixed, but Manu's vision for "fixing" it is really to do more than fixing it. Manu thinks "unshared" is useless as a type qualifier. He wants to change the meaning of `shared` so it no longer means "shared", but instead means "thread safe". For plain variables, this "thread safe" annotation would also imply "shared" (in an unprincipled manner, breaking the type system). "thread safe" is to "shared" approximately as "const" is to "mutable". That's also why all variables can implicitly convert to "thread safe".
 I'm glad you pushed the restriction 
 through. At least now generic code could see `shared` as "shrouded in 
 opacity, not subject to the usual operations".
Luckily the restriction makes sense for `shared` even if its meaning is not changed to be different from its name.
Aug 04 2020
parent Manu <turkeyman gmail.com> writes:
On Wed, Aug 5, 2020 at 3:20 AM Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 04.08.20 04:13, Andrei Alexandrescu wrote:
 On 8/3/20 5:56 PM, Manu wrote:
 I think shared has potential to offer critical benefit to D, and we've
 talked about this personally to some length. Don't kill it yet, let's
 try and fix it. Although it's worth recognising that if we don't fix
 it, then it might as well be killed as it stands today.
Of course fixing it would be great!
It's being fixed, but Manu's vision for "fixing" it is really to do more than fixing it. Manu thinks "unshared" is useless as a type qualifier. He wants to change the meaning of `shared` so it no longer means "shared", but instead means "thread safe".
This's a highly speculative end-goal, but a lot of my work applies to 'shared' equally with respect to today's definition, and also that potential definition. Leaving the 'thread-safe' definition aside, the key goal I want to achieve is that you may perform an implicit conversion to `scope shared`; since it can't escape the callee and therefore no references should co-exist with the thread-local instance in the calling scope. This would be immensely useful to enable various parallel workloads, parallel-for is a key target. The trouble is, and I think you pointed it out before, when multiple arguments alias eachother: void fun(ref X a, ref scope shared(X) b) { /* a and b are a shared + unshared alias... */ } X x; fun(x, x); Allowing implicit conversion to `scope shared` requires that the caller and callee's scopes are a clear division between the unshared and the promoted-to-shared instances. scope enforces that we can't escape the callee's promoted-to-shared ref to alias the unshared ref in the callers scope, but that division can be violated by aliasing the caller's reference, and sneaking an unshared reference into the callee beside the promoted reference. That problem makes the implicit conversion in the presence of scope challenging, and that is what leads to the design where `shared` comes to mean 'thread-safe' instead of simply shared; which insists a stronger set of requirements, and under that changed definition, it allows the alias to exist. If there was another way to prevent the caller's unshared ref from aliasing its way into the callee, that would be preferable to the scheme I described before. For plain variables, this
 "thread safe" annotation would also imply "shared" (in an unprincipled
 manner, breaking the type system).

 "thread safe" is to "shared" approximately as "const" is to "mutable".
 That's also why all variables can implicitly convert to "thread safe".

 I'm glad you pushed the restriction
 through. At least now generic code could see `shared` as "shrouded in
 opacity, not subject to the usual operations".
Luckily the restriction makes sense for `shared` even if its meaning is not changed to be different from its name.
It's not really 'lucky', it's fundamental to any conceivable definition of shared. It's a starting point that's actually possible to work out from.
Aug 04 2020
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:
 Shared recently received a `-preview` which makes it really 
 mean something;
 this is what shared means:
 1. The data is shared; therefore, it is invalid to read or 
 write that
 memory.
 2. The reason this is useful as an attribute, is because you 
 are able to
 attribute methods. Ability to author a set of threadsafe 
 methods and
 clearly distinguish them from non-threadsafe methods is a 
 powerful
 advantage over C++.
For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.

 years.
Thanks!
 I think shared has potential to offer critical benefit to D, 
 and we've
 talked about this personally to some length. Don't kill it yet, 
 let's try
 and fix it. Although it's worth recognising that if we don't 
 fix it, then
 it might as well be killed as it stands today.
There might be room for improvement, but it is useful for me already. For instance, I often deal with delegates that can be called from any execution context. I make them shared to convey those semantics and the compiler even helps! Very nice.
Aug 04 2020
next sibling parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Tuesday, 4 August 2020 at 07:38:26 UTC, Sebastiaan Koppe wrote:
 On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:
 Shared recently received a `-preview` which makes it really 
 mean something;
 this is what shared means:
 1. The data is shared; therefore, it is invalid to read or 
 write that
 memory.
 2. The reason this is useful as an attribute, is because you 
 are able to
 attribute methods. Ability to author a set of threadsafe 
 methods and
 clearly distinguish them from non-threadsafe methods is a 
 powerful
 advantage over C++.
For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.
With DIP1024, any type should be implicitly castable to shared. This means you should always be able to call shared methods on any object. Far as I can see, this is not implemented. For a type with no thread-safe interface that means anyone with a shared reference to it can call absolutely no methods on it, and not read or write any of its fields. Kinda useless, but that's the point. If you want to manipulate it, you will need to cast away shared, which is non- safe and a red flag. For a type with a thread-safe interface, there may be methods that can't be called, but some subset will be callable. These must be written such that they are thread-safe, and any non-shared methods must be written such that they do not break the thread-safety of shared methods (but non-shared methods may be non-thread-safe if called on multiple threads, since that should not happen). In other words, DIP1024 assumes you will have at most one non-shared reference to an object, and that no methods on an object are written in such a way that they break thread-safety under this assumption. -- Simen
Aug 04 2020
parent reply Manu <turkeyman gmail.com> writes:
On Tue, Aug 4, 2020 at 6:26 PM Simen Kj=C3=A6r=C3=A5s via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 4 August 2020 at 07:38:26 UTC, Sebastiaan Koppe wrote:
 On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:
 Shared recently received a `-preview` which makes it really
 mean something;
 this is what shared means:
 1. The data is shared; therefore, it is invalid to read or
 write that
 memory.
 2. The reason this is useful as an attribute, is because you
 are able to
 attribute methods. Ability to author a set of threadsafe
 methods and
 clearly distinguish them from non-threadsafe methods is a
 powerful
 advantage over C++.
For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.
With DIP1024, any type should be implicitly castable to shared. This means you should always be able to call shared methods on any object. Far as I can see, this is not implemented. For a type with no thread-safe interface that means anyone with a shared reference to it can call absolutely no methods on it, and not read or write any of its fields. Kinda useless, but that's the point. If you want to manipulate it, you will need to cast away shared, which is non- safe and a red flag. For a type with a thread-safe interface, there may be methods that can't be called, but some subset will be callable. These must be written such that they are thread-safe, and any non-shared methods must be written such that they do not break the thread-safety of shared methods (but non-shared methods may be non-thread-safe if called on multiple threads, since that should not happen). In other words, DIP1024 assumes you will have at most one non-shared reference to an object, and that no methods on an object are written in such a way that they break thread-safety under this assumption. -- Simen
I don't think what you describe has anything to do with DIP1024. What you're describing is the scheme I was arguing for 1-2 years ago, but Walter rejected it and presented DIP1024 instead. DIP1024 was a step in the right direction, so I supported it at that time, but it doesn't change any definitions about thread-safety of methods, and it's not a complete solution. It just adds the restrictions that should have been there from the start; that is, `shared` has no read or write access to data.
Aug 04 2020
parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Tuesday, 4 August 2020 at 13:06:35 UTC, Manu wrote:
 On Tue, Aug 4, 2020 at 6:26 PM Simen Kjærås via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:
[stuff]
 I don't think what you describe has anything to do with DIP1024.
 What you're describing is the scheme I was arguing for 1-2 
 years ago, but
 Walter rejected it and presented DIP1024 instead.
 DIP1024 was a step in the right direction, so I supported it at 
 that time,
 but it doesn't change any definitions about thread-safety of 
 methods, and
 it's not a complete solution. It just adds the restrictions 
 that should
 have been there from the start; that is, `shared` has no read 
 or write
 access to data.
You're right. I was under the impression that DIP1024 was going to do all the things in your proposal (and I still hope they will be added to D eventually), but it seems I was wrong. -- Simen
Aug 04 2020
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:
 Shared recently received a `-preview` which makes it really
 mean something;
 this is what shared means:
 1. The data is shared; therefore, it is invalid to read or
 write that
 memory.
 2. The reason this is useful as an attribute, is because you
 are able to
 attribute methods. Ability to author a set of threadsafe
 methods and
 clearly distinguish them from non-threadsafe methods is a
 powerful
 advantage over C++.
For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.
Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change. In the mean-time, until the shared usage patterns are more well-proven, I recommend you try to use the 'alias this' pattern I show above, and report on any issues you encounter using this scheme. If no issues are identified with extensive real-world usage data, I will push again for that implicit conversion rule.
Aug 04 2020
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:
 On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:
 There is just one thing about shared I don't understand. If I 
 design my object such that the non-shared methods are to be 
 used thread-local and the shared methods from any thread, it 
 follows that I should be able to call said shared methods from 
 both a shared and non-shared instance of that object.
Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.
What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method. Hmm, well I guess if members themselves were also implicitly promoted to shared that could cause havoc.
 In the mean-time, until the shared usage patterns are more 
 well-proven, I
 recommend you try to use the 'alias this' pattern I show above, 
 and report
 on any issues you encounter using this scheme. If no issues are 
 identified
 with extensive real-world usage data, I will push again for 
 that implicit
 conversion rule.
Great, I'll try it. Thanks.
Aug 04 2020
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04.08.20 15:52, Sebastiaan Koppe wrote:
 On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:
 On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:
 There is just one thing about shared I don't understand. If I design 
 my object such that the non-shared methods are to be used 
 thread-local and the shared methods from any thread, it follows that 
 I should be able to call said shared methods from both a shared and 
 non-shared instance of that object.
Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing {   ref shared(Thing) implSharedCast() { return *cast(shared)&this; }   alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.
What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method. ...
Nope. You can access all members, but they will be treated as `shared`.
 Hmm, well I guess if members themselves were also implicitly promoted to 
 shared that could cause havoc.
Which is what the code above does, and also what Manu's "thread safe" qualifier would do. `shared` would no longer mean "shared".
Aug 04 2020
parent Manu <turkeyman gmail.com> writes:
On Wed, Aug 5, 2020 at 3:40 AM Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 04.08.20 15:52, Sebastiaan Koppe wrote:
 On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:
 On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:
 There is just one thing about shared I don't understand. If I design
 my object such that the non-shared methods are to be used
 thread-local and the shared methods from any thread, it follows that
 I should be able to call said shared methods from both a shared and
 non-shared instance of that object.
Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's
a
 lot of resistance to this change.
What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method. ...
Nope. You can access all members, but they will be treated as `shared`.
"treated as `shared`"; which is, under the `-preview` merged last year some time, that they can not be read or written at all. So you can take the address of all members, but not read or write to them.
 Hmm, well I guess if members themselves were also implicitly promoted to
 shared that could cause havoc.
Which is what the code above does, and also what Manu's "thread safe" qualifier would do. `shared` would no longer mean "shared".
I described an alternative approach in my reply to you. If there is any way to inhibit an un-shared alias from the caller being propagated to the callee, maintaining a separation at the call boundary between the unshared and shared instances, then it's safe without any of my other speculative rules.
Aug 04 2020
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Aug 4, 2020 at 11:55 PM Sebastiaan Koppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:
 On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:
 There is just one thing about shared I don't understand. If I
 design my object such that the non-shared methods are to be
 used thread-local and the shared methods from any thread, it
 follows that I should be able to call said shared methods from
 both a shared and non-shared instance of that object.
Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.
What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method.
Marking the function shared (and therefore the data accessible) doesn't magically make the function body threadsafe; what it does it makes the function potentially racy, and subject to VERY careful implementation. Sadly, I'm not aware of any CS research that can help you write atomic/threadsafe functions with any degree of proof in an environment like this (ie, without isolation and/or things like copy-on-write, etc). Calling shared methods from shared methods isn't safe either. Each call may be threadsafe atomically, but you can't author a leaf function without just as much care (or more) than the lower-level ones. If the function is more than 1-line, and carries some state across a few statements that call lower-level shared methods, then you need to be confident about the atomic state guarantees (or not! which is more likely) of the dependency API's, such that you don't create a state race between calls. Needless to say, it's still tricky, and there's really nothing the language can help you do... other than provide a strong mechanism to lock it off from normal code. As long as shared methods are few, and very well defined, then it is possible to implement very interesting and useful machines with this scheme, and also MASSIVELY help with validation of your ecosystem, and I have done so... but it's not magic, it's just a lot better than C++. The thing `shared` protects most against, is failure to correct multithreaded code when refactoring. I maintain a majorly-superscalar engine, and 90% of the race bugs we have had to spend heaps of time chasing down are the result of refactors or changes that had very peripheral contact with the multithreaded core; contact was narrow enough that we didn't notice then making changes, and there was nothing in the type system to alert us. For this reason alone, `shared` is worth its weight in gold. Hmm, well I guess if members themselves were also implicitly
 promoted to shared that could cause havoc.
The catch though, is that if you allow implicit conversion from unshared->shared, then the rule changes from "shared methods must be threadsafe", to " shared methods must be threadsafe, AND unshared methods must also be threadsafe in the event they are called in conjunction with any of the shared methods (just not with eachother)". While I have plenty of small tools where the second rule is easy to implement, I can also think of many situations where even the first rule is hard to implement, and the second rule is virtually impossible. Unless we learn better ways to handle this, I don't think implicit conversion that way can scale. For the time being, you can implement the implicit conversion for the tools that may support that using `alias this` like I showed above, and I think that's appropriate for the time being. In general, I'd like to find ways we can enhance shared to be more than a marker; consider TSAN (ThreadSanitizer) in Clang; it instruments atomics with runtime tracking to detect races at runtime. I think it would be possible to instrument shared data/methods in a similar way in debug builds, and then we'd have meaningful compiler assistance.
Aug 04 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Aug 4, 2020 at 11:55 PM Sebastiaan Koppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:
 On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:
 There is just one thing about shared I don't understand. If I
 design my object such that the non-shared methods are to be
 used thread-local and the shared methods from any thread, it
 follows that I should be able to call said shared methods from
 both a shared and non-shared instance of that object.
Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.
What exactly does the compiler need to prove?
It would be ideal if the compiler could validate that the un-shared methods are safe with respect to any shared methods. If a shared (ie, 'thread-safe') method were called in parallel to an un-shared method as is possible if the implicit conversion was allowed, then it is necessary that un-shared methods need to be tolerant of their 'thread-safe' API being called at any time. That's the change in meaning which is highly dangerous without some serious tooling. Essentially, adding a shared method to an object applies a contract to all other NOT-shared methods, and that doesn't feel good; "touching this here affects those other things"... it will be super hard to audit. Technology that looks like ThreadSanitizer might help, but it gets more tricky again when UFCS is used instead of member functions; those associations are harder to track. I think that idea is doomed, and more recently I'm thinking about ways to prevent sneaking un-shared aliases across the conversion boundary... but I don't know how to achieve that without some sort of aliasing rules which seem unlikely. The restrictions
 are all in place, you can only call shared methods on a shared
 object, and you can only access shared members in a shared method.
Ummmm no, under any of my proposals, you can't access shared members in a shared method... a shared method's `this` pointer is shared, which means all members are shared, and shared data may not be read or written at all... shared methods have no safe access to members. They can only call other shared methods safe-ly. Hmm, well I guess if members themselves were also implicitly
 promoted to shared that could cause havoc.
Members inherit the qualifier of methods... in a shared method, members are shared, which means you can't read or write them.
Aug 04 2020
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04.08.20 14:52, Manu wrote:
 slight change of meaning of shared
It's not slight. What happens to module level variables?
Aug 04 2020
parent Manu <turkeyman gmail.com> writes:
On Wed, Aug 5, 2020 at 3:35 AM Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 04.08.20 14:52, Manu wrote:
 slight change of meaning of shared
It's not slight. What happens to module level variables?
I'm not sure what you're asking?
Aug 04 2020
prev sibling next sibling parent Ronny <weibers jjmail.com> writes:
So, are you guys still surprised nobody wants to use D in 
production? The current language state is simply not suitable for 
development. D is just a bunch of features and most of them 
aren't even designed well. This stagnation has been going on for 
years without any improvement. It's time for D3.
Aug 04 2020
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
General disclaimer: My post may contain rebuttals to low-level technical 
points which were made in support of some high-level claims. Apparently 
there has been some confusion: When I disagree with a technical point, I 
am not automatically taking a stance against the corresponding 
high-level claim.

On 02.08.20 22:50, Andrei Alexandrescu wrote:
 (Background: qualifiers were introduced following my horror when I 
 started writing D1 code and saw that strings are represented as char[]. 
 So structs with string members would look like:
 
 struct Widget
 {
      int x, y;
      char[] name, parentName;
      ...
 }
 
 I found it just shocking that following a Widget's construction whoever 
 aliased the same strings the outside could change members of the Widget 
 without Widget knowing. Of course, other D1 coders disliked that as 
 well, so they'd defensively duplicate in the constructor:
 
 struct Widget
 {
      int x, y;
      char[] name, parentName;
      this(char[] n, char[] p)
      {
          name = n.dup;
          parentName = p.dup;
      }
      ...
 }
 
 thus ensuring proliferation of the garbage whether duplication was 
 needed or not.
 
 I found this absolutely maddening, to the extent I didn't think D could 
 be ever used at any considerable scale while dragging this anchor behind 
 it.
 
 The second problem was Walter was adamant about using arrays of 
 characters at strings. He found the notion of a library-defined string 
 type (a la C++) an absolute abomination. Stubborn about it like I've 
 never seen him before or after. So my unstoppable requests for a string 
 type were met with the proverbial immovable refusal. Ironically, much of 
 his argument came from an efficiency angle, yet the unnecessary 
 duplication was way less efficient than some reference counting/small 
 string optimization/etc scheme that a dedicated string type would use.
 ...
At the cost of efficient slicing?
 Then we figured things would work out if we arranged things such that 
 people could NOT change individual characters of a string. That would 
 allow sharing without the danger of long-distance influence. After many 
 discussions with Walter, Bartosz, Eric, Brad, and myself, immutable and 
 const were born.
 ...
And it has to be said that they do a great job at preventing individual chars in a `char[]` from being mutated. :-)
 Then followed the other qualifiers, in order: shared and inout.)
 
 * * *
 
 The result is... there: https://dlang.org/spec/const3.html. It has the 
 images https://dlang.org/images/qualifier-combinations.svg and 
 https://dlang.org/images/qualifier-conversions.svg and a large table and 
 a lot of rules. Whenever I code anything generic, I find myself going 
 back to those damn images and tables way more than anyone ought to. (And 
 it's ironic... I made those. Woe to the relative newcomer.)
 ...
Well, those tables and figures are easy to make, because they are based on simple rules. The combinatorial explosion is artificial.
 It's all too complicated, making generic D programming into 3D chess 
 instead of the difficult endeavor it already is. And what does it buy 
 us? Well, we don't need to define a string library type. Yowzers. 
 (Actually we should if we want to get rid of the GC. But then Walter 
 would oppose that. So - stalemate once again.)
 ...
In general, Walter opposes memory ownership to be mediated by library types.
 Far as I can tell, the power/weight ratio of qualifier is very poor. I 
 wish a DIP would revisit qualifiers with a stated intent to simplify 
 them as much as possible. Whenever I code generically I invariably run 
 into these issues:
 
 * Whatever I do, however I twitch, immutable finds the opportunity to 
 lodge itself in a soft part of my body and cause constant pain. This 
 doesn't work, that doesn't work. No solution for "tail immutable" - 
 mutable references to immutable class instances can't be done without 
 contortions. Can't assign to out immutable class references, though 
 there's no reason for that (see Adam's recent post).
 ...
Personally I don't use it where it causes those problems, but maybe the standard library is not afforded those conveniences.
 * No matter how I dice any significant piece of code, there will be five 
 casts from immutable and/or back that I can't rid of and lose sleep at 
 night trying to convince myself are justified.
 ...
Maybe `immutable` is overused in that code base.
 * Every time "inout" comes within a radius of a mile of what I'm doing, 
 it starts to stink like a skunk. I wish I could get a restraining order. 
 I can't instantiate "inout" variables, so writing any tests or 
 constraints becomes an advanced matter of defining functions and crap.
I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.
 I get frustrated, I protest to this forum, and immediately a cabal is 
 raised under Timon's leadership. The cabal convinces me that inout is 
 actually great and that I'm an idiot. I do get convinced, which is more 
 of a proof that Timon is very good, than a testament to the conviviality 
 of inout. Then I leave and get back to my code, and it stinks of inout 
 again. And I hate it and myself for having to deal with it.
 ...
I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day. I'm attaching my draft write-up on `inout` from 2018.
 * Nobody - probably not even Timon - knows what "shared" does or is 
 supposed to do and not do. The most I got from Walter ever is "shared is 
 intentionally restricted so you can't do much without a cast". Yet the 
 definition of "much" and the conditions under which casting is legit are 
 not anywhere to be found.
 ...
Well, I think I know what it is supposed to do, but I think Walter does not fully agree, as evidenced by the `scope shared` discussion. However, think of it this way: it is clear what _unshared_ does; it's a type system assertion that there will not be concurrent accesses to the given memory location. Casts to and from shared have to uphold that. That's essentially it. Then, semantically, shared variables are just standard variables from C or C++. Of course, that means `shared` variables should not be accessed using unsynchronized reads/writes in ` safe` functions, but I think Manu and Walter have been working on adding necessary type checks.
 * And of course, "shared" gladly partakes in qualifier combinations, 
 thus spreading its stink around in a combinatorial manner.
That's true of other language features. "Any field of any type can be of type `int`, so `int` is spreading its stink around in a combinatorial manner." If you consider type constructors to be type constructors instead of flags on top of a type, I think they are easier to understand and you don't have to handle nearly the same amount of combinatorial explosion.
 Believe it or 
 not, the type "const inout shared T" exists. Of course, nobody knows 
 what it really means or how it could be used.
 ...
I beg to differ. It's either const(shared(T)) or immutable(T), but you don't know which. An use case is you have a function that may return references to its `shared` input, which it does not modify, and you want to preserve `immutable` qualifiers across function calls while also allowing the function to be called with mutable shared data. Unfortunately this is only obvious if you understand `inout`, but unfortunately, `inout` is harder to understand than type theory concepts that are more general than `inout`...
 A "Define All Qualifiers" DIP would be a radical improvement of the 
 state of affairs.
My attached draft has some overlap with that.
Aug 04 2020
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 8/4/20 12:57 PM, Timon Gehr wrote:
 My attached draft has some overlap with that.
Thanks, Timon. Mike, looks like we have a great post for the blog in the making!
Aug 04 2020
prev sibling next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
Alternative syntax (even though it is in the template parameters, it is 
in fact erased and should not end in duplication):






T(U) first(void T=inout, U)(T(U)[] data) {
	assert(data.length > 0);
	return data[0];
}

--------

struct Foo(void Qual=inout) {
	private int[] _payload;
	
	this(Qual int[] payload) void(Qual) {
		this._payload = payload;
	}
	
	 property Qual(int)[] payload() void(Qual) {
		return this._payload;
	}
}

--------

void assign(void Writable=inout, void Readable=inout)(ref Writable(int)* 
a,Readable(int)* b,ref Writable(int)* c,Readable(int)* d) {
     a=b;
     c=d;
}

--------

Qual(int)* id(void Qual=inout)(Qual(int)* p) {
     return p;
}

--------

static assert(is(typeof(&id!mutable)==int* function(int*)))
static assert(is(typeof(&id!immutable)==immutable(int)* 
function(immutable(int)*)))
static assert(is(typeof(&id!const)==const(int)* function(const(int)*)))

That random mutable identifier tho...
Aug 04 2020
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/4/20 12:57 PM, Timon Gehr wrote:
 * Every time "inout" comes within a radius of a mile of what I'm 
 doing, it starts to stink like a skunk. I wish I could get a 
 restraining order. I can't instantiate "inout" variables, so writing 
 any tests or constraints becomes an advanced matter of defining 
 functions and crap.
I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.
 I get frustrated, I protest to this forum, and immediately a cabal is 
 raised under Timon's leadership. The cabal convinces me that inout is 
 actually great and that I'm an idiot. I do get convinced, which is 
 more of a proof that Timon is very good, than a testament to the 
 conviviality of inout. Then I leave and get back to my code, and it 
 stinks of inout again. And I hate it and myself for having to deal 
 with it.
 ...
I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day.
Well for what it's worth I have a simple question: how can I assess in druntime if a type T is copyable? I add the informal requirement that it's a simple query so it should be served with a proportionally simple answer. My initial take: static if (is(typeof((T x) { T y = x; }))) { ... } i.e. a lambda can be created that takes a T and creates a copy of it. Beautiful. This test, however, passes for inout types. And inout types cannot be considered really copyable, because they cannot be used in many places where one would expect to use a copyable type. To wit, a variety of unittests will fail (such as structs with copyable members), all protesting to the attempt of classifying inout types as copyable. Second attempt: static if (is(typeof((T x) { T y = x; })) && !is(T == inout U, U) { ... } So a type is copyable as before, just let's special case inout for exclusion. This already gets my diaper in a bunch because I need to special case a type of which utility I already am suspicious. And it's not only here - it's many, many similar places. Also, this also does NOT work because inout(const(int)) passes the test. This could probably be classified as a bug in the language or its compiler. So now I'm looking at things like importing "core.lifetime : emplace" and see if that compiles. Because the very complex implementation of emplace uses a complex mechanism to handle inout. I could be convinced that this awful complexity is justified given the choices made in the definition of this or that, but it would be more difficult to convince ourselves this is good programming language design. Simple questions should have simple answers.
Aug 14 2020
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/14/20 9:45 AM, Andrei Alexandrescu wrote:

 Well for what it's worth I have a simple question: how can I assess in 
 druntime if a type T is copyable? I add the informal requirement that 
 it's a simple query so it should be served with a proportionally simple 
 answer.
 
 My initial take:
 
 static if (is(typeof((T x) { T y = x; }))) { ... }
 
 i.e. a lambda can be created that takes a T and creates a copy of it. 
 Beautiful.
 
 This test, however, passes for inout types. And inout types cannot be 
 considered really copyable, because they cannot be used in many places 
 where one would expect to use a copyable type. To wit, a variety of 
 unittests will fail (such as structs with copyable members), all 
 protesting to the attempt of classifying inout types as copyable.
inout types aren't any less copyable than const or immutable types.
 
 Second attempt:
 
 static if (is(typeof((T x) { T y = x; })) && !is(T == inout U, U) { ... }
 
 So a type is copyable as before, just let's special case inout for 
 exclusion.
This seems silly, if it's copyable it's copyable. The unrestricted test should be valid.
 
 This already gets my diaper in a bunch because I need to special case a 
 type of which utility I already am suspicious. And it's not only here - 
 it's many, many similar places.
 
 Also, this also does NOT work because inout(const(int)) passes the test. 
 This could probably be classified as a bug in the language or its compiler.
 
 So now I'm looking at things like importing "core.lifetime : emplace" 
 and see if that compiles. Because the very complex implementation of 
 emplace uses a complex mechanism to handle inout.
 
 I could be convinced that this awful complexity is justified given the 
 choices made in the definition of this or that, but it would be more 
 difficult to convince ourselves this is good programming language 
 design. Simple questions should have simple answers.
Is there a specific example? I suspect we are going to get into one of the foolish restrictions of inout -- can't be struct members or can't be returned if you don't have inout parameters. The answer is still -- fix inout so it's not so foolish. -Steve
Aug 14 2020
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, Aug 14, 2020 at 11:50 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/4/20 12:57 PM, Timon Gehr wrote:
 * Every time "inout" comes within a radius of a mile of what I'm
 doing, it starts to stink like a skunk. I wish I could get a
 restraining order. I can't instantiate "inout" variables, so writing
 any tests or constraints becomes an advanced matter of defining
 functions and crap.
I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.
 I get frustrated, I protest to this forum, and immediately a cabal is
 raised under Timon's leadership. The cabal convinces me that inout is
 actually great and that I'm an idiot. I do get convinced, which is
 more of a proof that Timon is very good, than a testament to the
 conviviality of inout. Then I leave and get back to my code, and it
 stinks of inout again. And I hate it and myself for having to deal
 with it.
 ...
I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day.
Well for what it's worth I have a simple question: how can I assess in druntime if a type T is copyable? I add the informal requirement that it's a simple query so it should be served with a proportionally simple answer. My initial take: static if (is(typeof((T x) { T y = x; }))) { ... } i.e. a lambda can be created that takes a T and creates a copy of it. Beautiful. This test, however, passes for inout types. And inout types cannot be considered really copyable, because they cannot be used in many places where one would expect to use a copyable type. To wit, a variety of unittests will fail (such as structs with copyable members), all protesting to the attempt of classifying inout types as copyable. Second attempt: static if (is(typeof((T x) { T y = x; })) && !is(T == inout U, U) { ... } So a type is copyable as before, just let's special case inout for exclusion. This already gets my diaper in a bunch because I need to special case a type of which utility I already am suspicious. And it's not only here - it's many, many similar places. Also, this also does NOT work because inout(const(int)) passes the test. This could probably be classified as a bug in the language or its compiler. So now I'm looking at things like importing "core.lifetime : emplace" and see if that compiles. Because the very complex implementation of emplace uses a complex mechanism to handle inout. I could be convinced that this awful complexity is justified given the choices made in the definition of this or that, but it would be more difficult to convince ourselves this is good programming language design. Simple questions should have simple answers.
I'm delighted by this post. Keep talking this way, and we have a chance to make the kind of progress we've been waiting for! If you're looking into emplace (and friends) you should be familiar with Suleyman's move constructor work-in-progress. I am convinced that's the only reasonable path forwards on that front.
Aug 14 2020
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 14.08.20 15:45, Andrei Alexandrescu wrote:
 On 8/4/20 12:57 PM, Timon Gehr wrote:
 * Every time "inout" comes within a radius of a mile of what I'm 
 doing, it starts to stink like a skunk. I wish I could get a 
 restraining order. I can't instantiate "inout" variables, so writing 
 any tests or constraints becomes an advanced matter of defining 
 functions and crap.
I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.
 I get frustrated, I protest to this forum, and immediately a cabal is 
 raised under Timon's leadership. The cabal convinces me that inout is 
 actually great and that I'm an idiot. I do get convinced, which is 
 more of a proof that Timon is very good, than a testament to the 
 conviviality of inout. Then I leave and get back to my code, and it 
 stinks of inout again. And I hate it and myself for having to deal 
 with it.
 ...
I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day.
Well for what it's worth I have a simple question: how can I assess in druntime if a type T is copyable? I add the informal requirement that it's a simple query so it should be served with a proportionally simple answer. My initial take: static if (is(typeof((T x) { T y = x; }))) { ... } i.e. a lambda can be created that takes a T and creates a copy of it. Beautiful. This test, however, passes for inout types. And inout types cannot be considered really copyable, because they cannot be used in many places where one would expect to use a copyable type. To wit, a variety of unittests will fail (such as structs with copyable members), all protesting to the attempt of classifying inout types as copyable. ...
Those are protesting to the attempt of using `inout` types as member variables, not to the attempt of classifying them as copyable. The question is whether we really want types that can't be used as fields, and I think no, we don't want that. (But all of those weird limitations would not exist if `inout` was designed right!)
 ...
 Also, this also does NOT work because inout(const(int)) passes the test. 
 This could probably be classified as a bug in the language or its compiler.
 ...
It most definitely is.
 So now I'm looking at things like importing "core.lifetime : emplace" 
 and see if that compiles. Because the very complex implementation of 
 emplace uses a complex mechanism to handle inout.
 
 I could be convinced that this awful complexity is justified given the 
 choices made in the definition of this or that, but it would be more 
 difficult to convince ourselves this is good programming language 
 design.
The complexity is not justified, it is not good programming language design and I never claimed otherwise. Good programming language features are orthogonal to each other.
 Simple questions should have simple answers.
If the above implementation of isCopyable is not the one you want, something else is off.
Aug 14 2020
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 15.08.20 03:29, Timon Gehr wrote:
 
 Simple questions should have simple answers.
If the above implementation of isCopyable is not the one you want, something else is off.
Oh, actually I misspoke, didn't notice that you changed my suggestion from the github issue from __traits(compiles, ...) to is(typeof(...)). Use __traits(compiles, ...), not is(typeof(...)). typeof assigns types to expressions that would not otherwise compile, even lambdas: void main(){ int x; static void foo(){ pragma(msg, typeof(()=>x)); // int delegate() pure nothrow nogc safe pragma(msg, __traits(compiles, ()=>x)); // false } } There is this persistent myth that __traits(compiles, ...) is the same as is(typeof(...)) this is not the case and IIRC I have used it to demonstrate that most template constraints in Phobos don't work correctly.
Aug 14 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/14/20 9:35 PM, Timon Gehr wrote:
 On 15.08.20 03:29, Timon Gehr wrote:
 Simple questions should have simple answers.
If the above implementation of isCopyable is not the one you want, something else is off.
Oh, actually I misspoke, didn't notice that you changed my suggestion from the github issue from __traits(compiles, ...) to is(typeof(...)). Use __traits(compiles, ...), not is(typeof(...)). typeof assigns types to expressions that would not otherwise compile, even lambdas: void main(){     int x;     static void foo(){         pragma(msg, typeof(()=>x));             // int delegate() pure nothrow nogc safe         pragma(msg, __traits(compiles, ()=>x)); // false     } } There is this persistent myth that __traits(compiles, ...) is the same as is(typeof(...)) this is not the case and IIRC I have used it to demonstrate that most template constraints in Phobos don't work correctly.
This is a problem as __traits should be a lower-level, seldom-used resort. Are the distinctions between the two documented somewhere?
Aug 17 2020
parent Nick Treleaven <nick geany.org> writes:
On Monday, 17 August 2020 at 11:23:04 UTC, Andrei Alexandrescu 
wrote:
 On 8/14/20 9:35 PM, Timon Gehr wrote:
 There is this persistent myth that __traits(compiles, ...) is 
 the same as is(typeof(...)) this is not the case and IIRC I 
 have used it to demonstrate that most template constraints in 
 Phobos don't work correctly.
This is a problem as __traits should be a lower-level, seldom-used resort.
__traits(compiles) is not really a trait. Whether an expression is valid or not isn't really metadata of the expression - because in e.g. D, an expression must be valid in order to have metadata. Traits can be wrapped by templates, this can't. Besides these points, it would be nicer syntax to have something like __compiles(expr) instead.
Aug 25 2020
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
wrote:
 * Nobody - probably not even Timon - knows what "shared" does 
 or is supposed to do and not do. The most I got from Walter 
 ever is "shared is intentionally restricted so you can't do 
 much without a cast". Yet the definition of "much" and the 
 conditions under which casting is legit are not anywhere to be 
 found.
Shouldn't shared just be a relaxation in compiler checks so that any data that has the "shared" qualifier is allowed to be shared between threads. This is pretty much what __gshared is about. For the compiler to check if you really can share the data is currently impossible with current compiler technology. It can be atomic integers, Intel TSX, some very elaborate lockless algorithm, lockless COW, even mutexes/spinlocks if necessary. Programmer must "manually" ensure that the data can be shared, there is no other way. So yes, a revisit of the qualifiers should be done so that they can be precisely defined.
Aug 13 2020
parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Thursday, 13 August 2020 at 19:22:29 UTC, IGotD- wrote:
 Shouldn't shared just be a relaxation in compiler checks so 
 that any data that has the "shared" qualifier is allowed to be 
 shared between threads. This is pretty much what __gshared is 
 about.
It should help the programmer write bug-free code, which __gshared does not. A panacea would be great, but more realistically we should disallow reading, writing and calling non-shared methods on shared data in safe code. This limits the number of places where races can exist, but still (sadly) requires the programmer to know what he's doing when he writes shared code. -- Simen
Aug 13 2020
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
wrote:
 A "Define All Qualifiers" DIP would be a radical improvement of 
 the state of affairs.
Make sure there is a separate DIP for each qualifier. I have a feeling some qualifiers will be quite a battle with different opinions.
Aug 14 2020
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/14/20 4:30 PM, IGotD- wrote:
 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:
 A "Define All Qualifiers" DIP would be a radical improvement of the 
 state of affairs.
Make sure there is a separate DIP for each qualifier. I have a feeling some qualifiers will be quite a battle with different opinions.
Conversions and interactions make this a package deal.
Aug 17 2020
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu 
wrote:
 (Background: qualifiers were introduced following my horror 
 when I started writing D1 code and saw that strings are 
 represented as char[]. So structs with string members would 
 look like:

 [...]
Nearly every time I use inout I run into problems. Something doesn't compile, I get annoyed, and switch to templated this. Can anyone tell me what inout does that templated this doesn't?
Aug 24 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/24/20 12:13 PM, Atila Neves wrote:
 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:
 (Background: qualifiers were introduced following my horror when I 
 started writing D1 code and saw that strings are represented as 
 char[]. So structs with string members would look like:

 [...]
Nearly every time I use inout I run into problems. Something doesn't compile, I get annoyed, and switch to templated this.
I have the opposite experience -- inout just works most of the time.
 Can anyone tell me 
 what inout does that templated this doesn't?
1. It does not create a separate identical implementation for all flavors of mutability 2. It is not mutable even when the this reference is. The problems with inout have nothing to do with its utility. They are artificial limitations that should not have been included. -Steve
Aug 24 2020
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.08.20 18:27, Steven Schveighoffer wrote:
 
 Can anyone tell me what inout does that templated this doesn't?
1. It does not create a separate identical implementation for all flavors of mutability
- It works with virtual functions/function pointers/delegates. - The implementation is type checked before instantiation.
 2. It is not mutable even when the this reference is.
(That's a hard property to keep in a sound version without artificial limitations.)
Aug 24 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/24/20 1:09 PM, Timon Gehr wrote:
 On 24.08.20 18:27, Steven Schveighoffer wrote:
 2. It is not mutable even when the this reference is.
(That's a hard property to keep in a sound version without artificial limitations.)
I'm not sure what you mean -- inout is like const in that you cannot mutate data via that reference. with template this parameters, you can statically check if it's mutable, and mutate. It is only important if your goal is to identify that a function will not modify data passed to it. In essence, if you care about const functions. -Steve
Aug 24 2020
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 24.08.20 19:19, Steven Schveighoffer wrote:

 
 I'm not sure what you mean -- inout is like const in that you cannot 
 mutate data via that reference.
With a better type system the caller might pass in delegates that know about the mutability. void foo[Qualifier q](ref q int x, void delegate(ref q int x) process){ process(x); } void main(){ int x=0; foo(x,(ref int a){ a=2; }); // (q is mutable) immutable int y=2; foo(y,(ref immutable int b){ x=b; }); // (q is `immutable`) }
Aug 24 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Aug 25, 2020 at 2:15 AM Atila Neves via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu
 wrote:
 (Background: qualifiers were introduced following my horror
 when I started writing D1 code and saw that strings are
 represented as char[]. So structs with string members would
 look like:

 [...]
Nearly every time I use inout I run into problems. Something doesn't compile, I get annoyed, and switch to templated this. Can anyone tell me what inout does that templated this doesn't?
Generates one instance, rather than many. Is a 'hard' function; ie, can by virtual...
Aug 24 2020