digitalmars.D - I wish all qualifiers were revisited with an eye for simplification
- Andrei Alexandrescu (88/88) Aug 02 2020 (Background: qualifiers were introduced following my horror when I
- Adam D. Ruppe (13/18) Aug 02 2020 inout is great, but you are also not wrong about it. It has
- Andrei Alexandrescu (2/8) Aug 02 2020 Yes that would help a lot.
- Steven Schveighoffer (3/12) Aug 03 2020 If I could hack dmd, I would make this my first priority...
- Stefan Koch (10/14) Aug 02 2020 [lots of stuff]
- Andrei Alexandrescu (8/21) Aug 02 2020 Yes, the intent was generous. The realization, not quite. However, I did...
- Stefan Koch (6/17) Aug 02 2020 No it should not.
- Jonathan M Davis (34/47) Aug 12 2020 In princip@le, shared should be preventing any and all read-write operat...
- Jonathan M Davis (6/56) Aug 13 2020 LOL. And reading through the rest of the thread, I see that Manu already
- Manu (16/88) Aug 13 2020 .
- RazvanN (6/13) Aug 02 2020 Maybe a library implementation of `TailMutable(T) if (is(T ==
- Alexandru Ermicioi (2/6) Aug 03 2020 There is already Rebindable structure for that.
- Simen =?UTF-8?B?S2rDpnLDpXM=?= (15/21) Aug 03 2020 Shared today is a half-baked implementation of a half-thought
- Andrei Alexandrescu (2/23) Aug 03 2020 Found out about that after I wrote my diatribe. Thanks!
- Bruce Carneal (4/20) Aug 03 2020 Very much looking forward to activating shared with
- Manu (18/107) Aug 03 2020 Shared recently received a `-preview` which makes it really mean somethi...
- Andrei Alexandrescu (4/8) Aug 03 2020 Of course fixing it would be great! I'm glad you pushed the restriction
- Timon Gehr (11/21) Aug 04 2020 It's being fixed, but Manu's vision for "fixing" it is really to do more...
- Manu (35/56) Aug 04 2020 This's a highly speculative end-goal, but a lot of my work applies to
- Sebastiaan Koppe (16/38) Aug 04 2020 For some reason I often end up with multiple threads and the
- Simen =?UTF-8?B?S2rDpnLDpXM=?= (22/46) Aug 04 2020 With DIP1024, any type should be implicitly castable to shared.
- Manu (10/59) Aug 04 2020 I don't think what you describe has anything to do with DIP1024.
- Simen =?UTF-8?B?S2rDpnLDpXM=?= (7/22) Aug 04 2020 You're right. I was under the impression that DIP1024 was going
- Manu (20/44) Aug 04 2020 Yes, this is a thing I talked at great length 1-2 years ago.
- Sebastiaan Koppe (7/41) Aug 04 2020 What exactly does the compiler need to prove? The restrictions
- Timon Gehr (4/36) Aug 04 2020 Which is what the code above does, and also what Manu's "thread safe"
- Manu (10/47) Aug 04 2020 "treated as `shared`"; which is, under the `-preview` merged last year s...
- Manu (47/80) Aug 04 2020 Marking the function shared (and therefore the data accessible) doesn't
- Manu (28/61) Aug 04 2020 It would be ideal if the compiler could validate that the un-shared meth...
- Timon Gehr (2/3) Aug 04 2020 It's not slight. What happens to module level variables?
- Manu (3/6) Aug 04 2020 I'm not sure what you're asking?
- Ronny (5/5) Aug 04 2020 So, are you guys still surprised nobody wants to use D in
- Timon Gehr (49/157) Aug 04 2020 General disclaimer: My post may contain rebuttals to low-level technical...
- Andrei Alexandrescu (3/4) Aug 04 2020 Thanks, Timon. Mike, looks like we have a great post for the blog in the...
- rikki cattermole (34/34) Aug 04 2020 Alternative syntax (even though it is in the template parameters, it is
- Andrei Alexandrescu (30/54) Aug 14 2020 Well for what it's worth I have a simple question: how can I assess in
- Steven Schveighoffer (9/49) Aug 14 2020 This seems silly, if it's copyable it's copyable. The unrestricted test
- Manu (7/61) Aug 14 2020 I'm delighted by this post. Keep talking this way, and we have a chance ...
- Timon Gehr (12/70) Aug 14 2020 Those are protesting to the attempt of using `inout` types as member
- Timon Gehr (16/21) Aug 14 2020 Oh, actually I misspoke, didn't notice that you changed my suggestion
- Andrei Alexandrescu (3/28) Aug 17 2020 This is a problem as __traits should be a lower-level, seldom-used
- Nick Treleaven (8/15) Aug 25 2020 __traits(compiles) is not really a trait. Whether an expression
- IGotD- (13/19) Aug 13 2020 Shouldn't shared just be a relaxation in compiler checks so that
- Simen =?UTF-8?B?S2rDpnLDpXM=?= (10/14) Aug 13 2020 It should help the programmer write bug-free code, which
- IGotD- (5/7) Aug 14 2020 Make sure there is a separate DIP for each qualifier. I have a
- Andrei Alexandrescu (2/9) Aug 17 2020 Conversions and interactions make this a package deal.
- Atila Neves (5/10) Aug 24 2020 Nearly every time I use inout I run into problems. Something
- Steven Schveighoffer (8/19) Aug 24 2020 1. It does not create a separate identical implementation for all
- Timon Gehr (5/11) Aug 24 2020 - It works with virtual functions/function pointers/delegates.
- Steven Schveighoffer (8/13) Aug 24 2020 I'm not sure what you mean -- inout is like const in that you cannot
- Timon Gehr (12/16) Aug 24 2020 With a better type system the caller might pass in delegates that know
- Manu (4/15) Aug 24 2020 Generates one instance, rather than many. Is a 'hard' function; ie, can ...
(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like: struct Widget { int x, y; char[] name, parentName; ... } I found it just shocking that following a Widget's construction whoever aliased the same strings the outside could change members of the Widget without Widget knowing. Of course, other D1 coders disliked that as well, so they'd defensively duplicate in the constructor: struct Widget { int x, y; char[] name, parentName; this(char[] n, char[] p) { name = n.dup; parentName = p.dup; } ... } thus ensuring proliferation of the garbage whether duplication was needed or not. I found this absolutely maddening, to the extent I didn't think D could be ever used at any considerable scale while dragging this anchor behind it. The second problem was Walter was adamant about using arrays of characters at strings. He found the notion of a library-defined string type (a la C++) an absolute abomination. Stubborn about it like I've never seen him before or after. So my unstoppable requests for a string type were met with the proverbial immovable refusal. Ironically, much of his argument came from an efficiency angle, yet the unnecessary duplication was way less efficient than some reference counting/small string optimization/etc scheme that a dedicated string type would use. Then we figured things would work out if we arranged things such that people could NOT change individual characters of a string. That would allow sharing without the danger of long-distance influence. After many discussions with Walter, Bartosz, Eric, Brad, and myself, immutable and const were born. Then followed the other qualifiers, in order: shared and inout.) * * * The result is... there: https://dlang.org/spec/const3.html. It has the images https://dlang.org/images/qualifier-combinations.svg and https://dlang.org/images/qualifier-conversions.svg and a large table and a lot of rules. Whenever I code anything generic, I find myself going back to those damn images and tables way more than anyone ought to. (And it's ironic... I made those. Woe to the relative newcomer.) It's all too complicated, making generic D programming into 3D chess instead of the difficult endeavor it already is. And what does it buy us? Well, we don't need to define a string library type. Yowzers. (Actually we should if we want to get rid of the GC. But then Walter would oppose that. So - stalemate once again.) Far as I can tell, the power/weight ratio of qualifier is very poor. I wish a DIP would revisit qualifiers with a stated intent to simplify them as much as possible. Whenever I code generically I invariably run into these issues: * Whatever I do, however I twitch, immutable finds the opportunity to lodge itself in a soft part of my body and cause constant pain. This doesn't work, that doesn't work. No solution for "tail immutable" - mutable references to immutable class instances can't be done without contortions. Can't assign to out immutable class references, though there's no reason for that (see Adam's recent post). * No matter how I dice any significant piece of code, there will be five casts from immutable and/or back that I can't rid of and lose sleep at night trying to convince myself are justified. * Every time "inout" comes within a radius of a mile of what I'm doing, it starts to stink like a skunk. I wish I could get a restraining order. I can't instantiate "inout" variables, so writing any tests or constraints becomes an advanced matter of defining functions and crap. I get frustrated, I protest to this forum, and immediately a cabal is raised under Timon's leadership. The cabal convinces me that inout is actually great and that I'm an idiot. I do get convinced, which is more of a proof that Timon is very good, than a testament to the conviviality of inout. Then I leave and get back to my code, and it stinks of inout again. And I hate it and myself for having to deal with it. * Nobody - probably not even Timon - knows what "shared" does or is supposed to do and not do. The most I got from Walter ever is "shared is intentionally restricted so you can't do much without a cast". Yet the definition of "much" and the conditions under which casting is legit are not anywhere to be found. * And of course, "shared" gladly partakes in qualifier combinations, thus spreading its stink around in a combinatorial manner. Believe it or not, the type "const inout shared T" exists. Of course, nobody knows what it really means or how it could be used. A "Define All Qualifiers" DIP would be a radical improvement of the state of affairs.
Aug 02 2020
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:The cabal convinces me that inout is actually great and that I'm an idiot.inout is great, but you are also not wrong about it. It has frustrating arbitrary limitations that should really just get lifted. Steven designed it conservatively thinking it was a good idea at the time but agrees now it was a mistake to limit it like this. Simplest step we could do is define inout == const if used for a local variable in a non-inout function. I think that'd eliminate the majority of the headache. The `is` expression should be more strict, but a local declaration should just fallback to const for practicality.Believe it or not, the type "const inout shared T" exists. Of course, nobody knows what it really means or how it could be used.I don't even have an idea lol
Aug 02 2020
On 8/2/20 5:30 PM, Adam D. Ruppe wrote:On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:Yes that would help a lot.The cabal convinces me that inout is actually great and that I'm an idiot.inout is great, but you are also not wrong about it. It has frustrating arbitrary limitations that should really just get lifted.
Aug 02 2020
On 8/2/20 6:22 PM, Andrei Alexandrescu wrote:On 8/2/20 5:30 PM, Adam D. Ruppe wrote:If I could hack dmd, I would make this my first priority... -SteveOn Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:Yes that would help a lot.The cabal convinces me that inout is actually great and that I'm an idiot.inout is great, but you are also not wrong about it. It has frustrating arbitrary limitations that should really just get lifted.
Aug 03 2020
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like:[lots of stuff] const inout shared T is simply the same as const shared T. shared means simply: this is not thread local, therefore this thing might be used by other threads therefore you read or write it in an expression directly. You can however have a function take a shared argument and leave the scheduling or locking or whatever synchronization you use, to enact an operation in a safe way.
Aug 02 2020
On 8/2/20 6:55 PM, Stefan Koch wrote:On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:Sadly no: https://run.dlang.io/is/etdPtP(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like:[lots of stuff] const inout shared T is simply the same as const shared T.shared means simply: this is not thread local, therefore this thing might be used by other threads therefore you read or write it in an expression directly. You can however have a function take a shared argument and leave the scheduling or locking or whatever synchronization you use, to enact an operation in a safe way.Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
Aug 02 2020
On Monday, 3 August 2020 at 01:23:26 UTC, Andrei Alexandrescu wrote:On 8/2/20 6:55 PM, Stefan Koch wrote:No it should not. With the right -preview switch it won't compile. The programmer is to ensure that atomics or other needed syncronisation primitives are used.[...]Sadly no: https://run.dlang.io/is/etdPtP[...]Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
Aug 02 2020
On Sunday, August 2, 2020 7:23:26 PM MDT Andrei Alexandrescu via Digitalmars-d wrote:On 8/2/20 6:55 PM, Stefan Koch wrote:In princip le, shared should be preventing any and all read-write operations which it cannot guarantee are atomic (which probably means banning all read-write operations, since that's going to vary across architectures). All code with shared then either needs to either use atomics or cast away shared and use mutexes, semaphores, or whatever other threading primitive is appropriate to ensure that the data is only accessed by that single thread while it's typed as thread-local, leaving it up to the programmer to then ensure that they're dealing with the threading correctly (just like in C++). However, unlike C++, all of the code operating on data shared across threads would then be segregated (it would also likely be system in most cases). More complex objects can then have shared member functions which deal with all of the threading stuff internally rather than requiring that everyone using them deal with it directly, but at the core, it's ultimately going to be a question of atomics or casts if we want the guarantee that shared objects aren't being accessed across threads when they're not properly protected. However, the core problem with all of this is that the restrictions that need to be put on shared have never really been put in place. For years, about all that it really did was disallow implicit conversions between shared and thread-local (and even then, it didn't always do it - e.g. with integers). Over time, additional restrictions have been added (I think mostly to numeric types), but it's all piecemeal. IIRC, there was a DIP that was supposed to outright make reading and writing to shared variables illegal, but I'm not quite sure where that stands. We also potentially need some language improvements to make it cleaner, because otherwise in order to read or write value types, we'll have to do stuff like take their address and cast the resulting pointer so that we can operate on the object as thread-local while the mutex is locked (or whatever or threading protections are in place). So, while I think that the basic idea of shared is solid, it's never been properly implemented. - Jonathan M Davisshared means simply: this is not thread local, therefore this thing might be used by other threads therefore you read or write it in an expression directly. You can however have a function take a shared argument and leave the scheduling or locking or whatever synchronization you use, to enact an operation in a safe way.Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
Aug 12 2020
On Thursday, August 13, 2020 12:44:54 AM MDT Jonathan M Davis via Digitalmars- d wrote:On Sunday, August 2, 2020 7:23:26 PM MDT Andrei Alexandrescu via Digitalmars-d wrote:LOL. And reading through the rest of the thread, I see that Manu already brought this up and that the DIP has an implementation with the -preview flag. - Jonathan M DavisOn 8/2/20 6:55 PM, Stefan Koch wrote:In princip le, shared should be preventing any and all read-write operations which it cannot guarantee are atomic (which probably means banning all read-write operations, since that's going to vary across architectures). All code with shared then either needs to either use atomics or cast away shared and use mutexes, semaphores, or whatever other threading primitive is appropriate to ensure that the data is only accessed by that single thread while it's typed as thread-local, leaving it up to the programmer to then ensure that they're dealing with the threading correctly (just like in C++). However, unlike C++, all of the code operating on data shared across threads would then be segregated (it would also likely be system in most cases). More complex objects can then have shared member functions which deal with all of the threading stuff internally rather than requiring that everyone using them deal with it directly, but at the core, it's ultimately going to be a question of atomics or casts if we want the guarantee that shared objects aren't being accessed across threads when they're not properly protected. However, the core problem with all of this is that the restrictions that need to be put on shared have never really been put in place. For years, about all that it really did was disallow implicit conversions between shared and thread-local (and even then, it didn't always do it - e.g. with integers). Over time, additional restrictions have been added (I think mostly to numeric types), but it's all piecemeal. IIRC, there was a DIP that was supposed to outright make reading and writing to shared variables illegal, but I'm not quite sure where that stands. We also potentially need some language improvements to make it cleaner, because otherwise in order to read or write value types, we'll have to do stuff like take their address and cast the resulting pointer so that we can operate on the object as thread-local while the mutex is locked (or whatever or threading protections are in place). So, while I think that the basic idea of shared is solid, it's never been properly implemented.shared means simply: this is not thread local, therefore this thing might be used by other threads therefore you read or write it in an expression directly. You can however have a function take a shared argument and leave the scheduling or locking or whatever synchronization you use, to enact an operation in a safe way.Yes, the intent was generous. The realization, not quite. However, I did learn from Walter that a number of invalid operations on shared numerics have now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.
Aug 13 2020
On Thu, Aug 13, 2020 at 5:24 PM Jonathan M Davis via Digitalmars-d < digitalmars-d puremagic.com> wrote:On Thursday, August 13, 2020 12:44:54 AM MDT Jonathan M Davis via Digitalmars- d wrote:.On Sunday, August 2, 2020 7:23:26 PM MDT Andrei Alexandrescu via Digitalmars-d wrote:didOn 8/2/20 6:55 PM, Stefan Koch wrote:shared means simply: this is not thread local, therefore this thing might be used by other threads therefore you read or write it in an expression directly. You can however have a function take a shared argument and leave the scheduling or locking or whatever synchronization you use, to enact an operation in a safe way.Yes, the intent was generous. The realization, not quite. However, Inumericslearn from Walter that a number of invalid operations on sharedoperationshave now been disabled. However, copying a shared int into an int is allowed and as far as I know with no special treatment: https://run.dlang.io/is/ZP124F At least on ARM that should generate a read barrier.In princip le, shared should be preventing any and all read-writewhich it cannot guarantee are atomic (which probably means banning all read-write operations, since that's going to vary across architectures)=ayAll code with shared then either needs to either use atomics or cast aw=veshared and use mutexes, semaphores, or whatever other threading primiti=ris appropriate to ensure that the data is only accessed by that single thread while it's typed as thread-local, leaving it up to the programme=toethen ensure that they're dealing with the threading correctly (just lik=instC++). However, unlike C++, all of the code operating on data sharedacrossthreads would then be segregated (it would also likely be system in mo=hcases). More complex objects can then have shared member functions whic=tdeal with all of the threading stuff internally rather than requiringthateveryone using them deal with it directly, but at the core, it'sultimatelygoing to be a question of atomics or casts if we want the guarantee tha=tshared objects aren't being accessed across threads when they're not properly protected. However, the core problem with all of this is that the restrictions tha=,need to be put on shared have never really been put in place. For years=erabout all that it really did was disallow implicit conversions between shared and thread-local (and even then, it didn't always do it - e.g.withintegers). Over time, additional restrictions have been added (I think mostly to numeric types), but it's all piecemeal. IIRC, there was a DIPthatwas supposed to outright make reading and writing to shared variables illegal, but I'm not quite sure where that stands. We also potentiallyneedsome language improvements to make it cleaner, because otherwise in ord=ento read or write value types, we'll have to do stuff like take their address and cast the resulting pointer so that we can operate on theobjectas thread-local while the mutex is locked (or whatever or threading protections are in place). So, while I think that the basic idea of shared is solid, it's never be==F0=9F=91=8D You're right though! :Pproperly implemented.LOL. And reading through the rest of the thread, I see that Manu already brought this up and that the DIP has an implementation with the -preview flag. - Jonathan M Davis
Aug 13 2020
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:* Whatever I do, however I twitch, immutable finds the opportunity to lodge itself in a soft part of my body and cause constant pain. This doesn't work, that doesn't work. No solution for "tail immutable" - mutable references to immutable class instances can't be done without contortions. Can't assign to out immutable class references, though there's no reason for that (see Adam's recent post).Maybe a library implementation of `TailMutable(T) if (is(T == immutable) || is(T == const))` for reference types can fix this without causing breakage and leaving immutable/const transitivity intact.
Aug 02 2020
On Monday, 3 August 2020 at 03:14:15 UTC, RazvanN wrote:Maybe a library implementation of `TailMutable(T) if (is(T == immutable) || is(T == const))` for reference types can fix this without causing breakage and leaving immutable/const transitivity intact.There is already Rebindable structure for that.
Aug 03 2020
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:* Nobody - probably not even Timon - knows what "shared" does or is supposed to do and not do. The most I got from Walter ever is "shared is intentionally restricted so you can't do much without a cast". Yet the definition of "much" and the conditions under which casting is legit are not anywhere to be found.Shared today is a half-baked implementation of a half-thought idea. Manu wrote[0] almost two years ago what it should be: you can't read or write from a shared object, and only shared methods may be called on it. This was made into a DIP[1], and is half-way(?) implemented in the compiler as -preview=nosharedaccess. The semantics are effective and easy to understand. -- Simen [0]: https://forum.dlang.org/post/mailman.4299.1539629222.29801.digitalmars-d puremagic.com [1]: https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1024.md
Aug 03 2020
On 8/3/20 7:45 AM, Simen Kjærås wrote:On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:Found out about that after I wrote my diatribe. Thanks!* Nobody - probably not even Timon - knows what "shared" does or is supposed to do and not do. The most I got from Walter ever is "shared is intentionally restricted so you can't do much without a cast". Yet the definition of "much" and the conditions under which casting is legit are not anywhere to be found.Shared today is a half-baked implementation of a half-thought idea. Manu wrote[0] almost two years ago what it should be: you can't read or write from a shared object, and only shared methods may be called on it. This was made into a DIP[1], and is half-way(?) implemented in the compiler as -preview=nosharedaccess. The semantics are effective and easy to understand. -- Simen [0]: https://forum.dlang.org/post/mailman.4299.1539629222.29801.digital ars-d puremagic.com [1]: https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1024.md
Aug 03 2020
On Monday, 3 August 2020 at 11:45:22 UTC, Simen Kjærås wrote:On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:Very much looking forward to activating shared with -preview=nosharedaccess. Will do so widely once "hello world" compiles again.[...]Shared today is a half-baked implementation of a half-thought idea. Manu wrote[0] almost two years ago what it should be: you can't read or write from a shared object, and only shared methods may be called on it. This was made into a DIP[1], and is half-way(?) implemented in the compiler as -preview=nosharedaccess. The semantics are effective and easy to understand. -- Simen [0]: https://forum.dlang.org/post/mailman.4299.1539629222.29801.digitalmars-d puremagic.com [1]: https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1024.md
Aug 03 2020
On Mon, Aug 3, 2020 at 6:55 AM Andrei Alexandrescu via Digitalmars-d < digitalmars-d puremagic.com> wrote:(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like: struct Widget { int x, y; char[] name, parentName; ... } I found it just shocking that following a Widget's construction whoever aliased the same strings the outside could change members of the Widget without Widget knowing. Of course, other D1 coders disliked that as well, so they'd defensively duplicate in the constructor: struct Widget { int x, y; char[] name, parentName; this(char[] n, char[] p) { name = n.dup; parentName = p.dup; } ... } thus ensuring proliferation of the garbage whether duplication was needed or not. I found this absolutely maddening, to the extent I didn't think D could be ever used at any considerable scale while dragging this anchor behind it. The second problem was Walter was adamant about using arrays of characters at strings. He found the notion of a library-defined string type (a la C++) an absolute abomination. Stubborn about it like I've never seen him before or after. So my unstoppable requests for a string type were met with the proverbial immovable refusal. Ironically, much of his argument came from an efficiency angle, yet the unnecessary duplication was way less efficient than some reference counting/small string optimization/etc scheme that a dedicated string type would use. Then we figured things would work out if we arranged things such that people could NOT change individual characters of a string. That would allow sharing without the danger of long-distance influence. After many discussions with Walter, Bartosz, Eric, Brad, and myself, immutable and const were born. Then followed the other qualifiers, in order: shared and inout.) * * * The result is... there: https://dlang.org/spec/const3.html. It has the images https://dlang.org/images/qualifier-combinations.svg and https://dlang.org/images/qualifier-conversions.svg and a large table and a lot of rules. Whenever I code anything generic, I find myself going back to those damn images and tables way more than anyone ought to. (And it's ironic... I made those. Woe to the relative newcomer.) It's all too complicated, making generic D programming into 3D chess instead of the difficult endeavor it already is. And what does it buy us? Well, we don't need to define a string library type. Yowzers. (Actually we should if we want to get rid of the GC. But then Walter would oppose that. So - stalemate once again.) Far as I can tell, the power/weight ratio of qualifier is very poor. I wish a DIP would revisit qualifiers with a stated intent to simplify them as much as possible. Whenever I code generically I invariably run into these issues: * Whatever I do, however I twitch, immutable finds the opportunity to lodge itself in a soft part of my body and cause constant pain. This doesn't work, that doesn't work. No solution for "tail immutable" - mutable references to immutable class instances can't be done without contortions. Can't assign to out immutable class references, though there's no reason for that (see Adam's recent post). * No matter how I dice any significant piece of code, there will be five casts from immutable and/or back that I can't rid of and lose sleep at night trying to convince myself are justified. * Every time "inout" comes within a radius of a mile of what I'm doing, it starts to stink like a skunk. I wish I could get a restraining order. I can't instantiate "inout" variables, so writing any tests or constraints becomes an advanced matter of defining functions and crap. I get frustrated, I protest to this forum, and immediately a cabal is raised under Timon's leadership. The cabal convinces me that inout is actually great and that I'm an idiot. I do get convinced, which is more of a proof that Timon is very good, than a testament to the conviviality of inout. Then I leave and get back to my code, and it stinks of inout again. And I hate it and myself for having to deal with it. * Nobody - probably not even Timon - knows what "shared" does or is supposed to do and not do. The most I got from Walter ever is "shared is intentionally restricted so you can't do much without a cast". Yet the definition of "much" and the conditions under which casting is legit are not anywhere to be found. * And of course, "shared" gladly partakes in qualifier combinations, thus spreading its stink around in a combinatorial manner. Believe it or not, the type "const inout shared T" exists. Of course, nobody knows what it really means or how it could be used. A "Define All Qualifiers" DIP would be a radical improvement of the state of affairs.Shared recently received a `-preview` which makes it really mean something; this is what shared means: 1. The data is shared; therefore, it is invalid to read or write that memory. 2. The reason this is useful as an attribute, is because you are able to attribute methods. Ability to author a set of threadsafe methods and clearly distinguish them from non-threadsafe methods is a powerful advantage over C++. isn't strengthened, then it is my opinion that `shared` is completely useless, and `struct Shared(T) { private: T* sharedThing; }` is just as good. I think shared has potential to offer critical benefit to D, and we've talked about this personally to some length. Don't kill it yet, let's try and fix it. Although it's worth recognising that if we don't fix it, then it might as well be killed as it stands today.
Aug 03 2020
On 8/3/20 5:56 PM, Manu wrote:I think shared has potential to offer critical benefit to D, and we've talked about this personally to some length. Don't kill it yet, let's try and fix it. Although it's worth recognising that if we don't fix it, then it might as well be killed as it stands today.Of course fixing it would be great! I'm glad you pushed the restriction through. At least now generic code could see `shared` as "shrouded in opacity, not subject to the usual operations".
Aug 03 2020
On 04.08.20 04:13, Andrei Alexandrescu wrote:On 8/3/20 5:56 PM, Manu wrote:It's being fixed, but Manu's vision for "fixing" it is really to do more than fixing it. Manu thinks "unshared" is useless as a type qualifier. He wants to change the meaning of `shared` so it no longer means "shared", but instead means "thread safe". For plain variables, this "thread safe" annotation would also imply "shared" (in an unprincipled manner, breaking the type system). "thread safe" is to "shared" approximately as "const" is to "mutable". That's also why all variables can implicitly convert to "thread safe".I think shared has potential to offer critical benefit to D, and we've talked about this personally to some length. Don't kill it yet, let's try and fix it. Although it's worth recognising that if we don't fix it, then it might as well be killed as it stands today.Of course fixing it would be great!I'm glad you pushed the restriction through. At least now generic code could see `shared` as "shrouded in opacity, not subject to the usual operations".Luckily the restriction makes sense for `shared` even if its meaning is not changed to be different from its name.
Aug 04 2020
On Wed, Aug 5, 2020 at 3:20 AM Timon Gehr via Digitalmars-d < digitalmars-d puremagic.com> wrote:On 04.08.20 04:13, Andrei Alexandrescu wrote:This's a highly speculative end-goal, but a lot of my work applies to 'shared' equally with respect to today's definition, and also that potential definition. Leaving the 'thread-safe' definition aside, the key goal I want to achieve is that you may perform an implicit conversion to `scope shared`; since it can't escape the callee and therefore no references should co-exist with the thread-local instance in the calling scope. This would be immensely useful to enable various parallel workloads, parallel-for is a key target. The trouble is, and I think you pointed it out before, when multiple arguments alias eachother: void fun(ref X a, ref scope shared(X) b) { /* a and b are a shared + unshared alias... */ } X x; fun(x, x); Allowing implicit conversion to `scope shared` requires that the caller and callee's scopes are a clear division between the unshared and the promoted-to-shared instances. scope enforces that we can't escape the callee's promoted-to-shared ref to alias the unshared ref in the callers scope, but that division can be violated by aliasing the caller's reference, and sneaking an unshared reference into the callee beside the promoted reference. That problem makes the implicit conversion in the presence of scope challenging, and that is what leads to the design where `shared` comes to mean 'thread-safe' instead of simply shared; which insists a stronger set of requirements, and under that changed definition, it allows the alias to exist. If there was another way to prevent the caller's unshared ref from aliasing its way into the callee, that would be preferable to the scheme I described before. For plain variables, thisOn 8/3/20 5:56 PM, Manu wrote:It's being fixed, but Manu's vision for "fixing" it is really to do more than fixing it. Manu thinks "unshared" is useless as a type qualifier. He wants to change the meaning of `shared` so it no longer means "shared", but instead means "thread safe".I think shared has potential to offer critical benefit to D, and we've talked about this personally to some length. Don't kill it yet, let's try and fix it. Although it's worth recognising that if we don't fix it, then it might as well be killed as it stands today.Of course fixing it would be great!"thread safe" annotation would also imply "shared" (in an unprincipled manner, breaking the type system). "thread safe" is to "shared" approximately as "const" is to "mutable". That's also why all variables can implicitly convert to "thread safe".It's not really 'lucky', it's fundamental to any conceivable definition of shared. It's a starting point that's actually possible to work out from.I'm glad you pushed the restriction through. At least now generic code could see `shared` as "shrouded in opacity, not subject to the usual operations".Luckily the restriction makes sense for `shared` even if its meaning is not changed to be different from its name.
Aug 04 2020
On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:Shared recently received a `-preview` which makes it really mean something; this is what shared means: 1. The data is shared; therefore, it is invalid to read or write that memory. 2. The reason this is useful as an attribute, is because you are able to attribute methods. Ability to author a set of threadsafe methods and clearly distinguish them from non-threadsafe methods is a powerful advantage over C++.For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.years.Thanks!I think shared has potential to offer critical benefit to D, and we've talked about this personally to some length. Don't kill it yet, let's try and fix it. Although it's worth recognising that if we don't fix it, then it might as well be killed as it stands today.There might be room for improvement, but it is useful for me already. For instance, I often deal with delegates that can be called from any execution context. I make them shared to convey those semantics and the compiler even helps! Very nice.
Aug 04 2020
On Tuesday, 4 August 2020 at 07:38:26 UTC, Sebastiaan Koppe wrote:On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:With DIP1024, any type should be implicitly castable to shared. This means you should always be able to call shared methods on any object. Far as I can see, this is not implemented. For a type with no thread-safe interface that means anyone with a shared reference to it can call absolutely no methods on it, and not read or write any of its fields. Kinda useless, but that's the point. If you want to manipulate it, you will need to cast away shared, which is non- safe and a red flag. For a type with a thread-safe interface, there may be methods that can't be called, but some subset will be callable. These must be written such that they are thread-safe, and any non-shared methods must be written such that they do not break the thread-safety of shared methods (but non-shared methods may be non-thread-safe if called on multiple threads, since that should not happen). In other words, DIP1024 assumes you will have at most one non-shared reference to an object, and that no methods on an object are written in such a way that they break thread-safety under this assumption. -- SimenShared recently received a `-preview` which makes it really mean something; this is what shared means: 1. The data is shared; therefore, it is invalid to read or write that memory. 2. The reason this is useful as an attribute, is because you are able to attribute methods. Ability to author a set of threadsafe methods and clearly distinguish them from non-threadsafe methods is a powerful advantage over C++.For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.
Aug 04 2020
On Tue, Aug 4, 2020 at 6:26 PM Simen Kj=C3=A6r=C3=A5s via Digitalmars-d < digitalmars-d puremagic.com> wrote:On Tuesday, 4 August 2020 at 07:38:26 UTC, Sebastiaan Koppe wrote:I don't think what you describe has anything to do with DIP1024. What you're describing is the scheme I was arguing for 1-2 years ago, but Walter rejected it and presented DIP1024 instead. DIP1024 was a step in the right direction, so I supported it at that time, but it doesn't change any definitions about thread-safety of methods, and it's not a complete solution. It just adds the restrictions that should have been there from the start; that is, `shared` has no read or write access to data.On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:With DIP1024, any type should be implicitly castable to shared. This means you should always be able to call shared methods on any object. Far as I can see, this is not implemented. For a type with no thread-safe interface that means anyone with a shared reference to it can call absolutely no methods on it, and not read or write any of its fields. Kinda useless, but that's the point. If you want to manipulate it, you will need to cast away shared, which is non- safe and a red flag. For a type with a thread-safe interface, there may be methods that can't be called, but some subset will be callable. These must be written such that they are thread-safe, and any non-shared methods must be written such that they do not break the thread-safety of shared methods (but non-shared methods may be non-thread-safe if called on multiple threads, since that should not happen). In other words, DIP1024 assumes you will have at most one non-shared reference to an object, and that no methods on an object are written in such a way that they break thread-safety under this assumption. -- SimenShared recently received a `-preview` which makes it really mean something; this is what shared means: 1. The data is shared; therefore, it is invalid to read or write that memory. 2. The reason this is useful as an attribute, is because you are able to attribute methods. Ability to author a set of threadsafe methods and clearly distinguish them from non-threadsafe methods is a powerful advantage over C++.For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.
Aug 04 2020
On Tuesday, 4 August 2020 at 13:06:35 UTC, Manu wrote:On Tue, Aug 4, 2020 at 6:26 PM Simen Kjærås via Digitalmars-d < digitalmars-d puremagic.com> wrote:[stuff]I don't think what you describe has anything to do with DIP1024. What you're describing is the scheme I was arguing for 1-2 years ago, but Walter rejected it and presented DIP1024 instead. DIP1024 was a step in the right direction, so I supported it at that time, but it doesn't change any definitions about thread-safety of methods, and it's not a complete solution. It just adds the restrictions that should have been there from the start; that is, `shared` has no read or write access to data.You're right. I was under the impression that DIP1024 was going to do all the things in your proposal (and I still hope they will be added to D eventually), but it seems I was wrong. -- Simen
Aug 04 2020
On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:On Monday, 3 August 2020 at 21:56:34 UTC, Manu wrote:Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change. In the mean-time, until the shared usage patterns are more well-proven, I recommend you try to use the 'alias this' pattern I show above, and report on any issues you encounter using this scheme. If no issues are identified with extensive real-world usage data, I will push again for that implicit conversion rule.Shared recently received a `-preview` which makes it really mean something; this is what shared means: 1. The data is shared; therefore, it is invalid to read or write that memory. 2. The reason this is useful as an attribute, is because you are able to attribute methods. Ability to author a set of threadsafe methods and clearly distinguish them from non-threadsafe methods is a powerful advantage over C++.For some reason I often end up with multiple threads and the coordination that comes with it. Shared has been very helpful for me and I am using no. 2 with good success. There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object. Often I workaround it be introducing a non shared method that forwards to the shared method by means of casting.
Aug 04 2020
On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method. Hmm, well I guess if members themselves were also implicitly promoted to shared that could cause havoc.There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object.Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.In the mean-time, until the shared usage patterns are more well-proven, I recommend you try to use the 'alias this' pattern I show above, and report on any issues you encounter using this scheme. If no issues are identified with extensive real-world usage data, I will push again for that implicit conversion rule.Great, I'll try it. Thanks.
Aug 04 2020
On 04.08.20 15:52, Sebastiaan Koppe wrote:On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:Nope. You can access all members, but they will be treated as `shared`.On Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method. ...There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object.Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.Hmm, well I guess if members themselves were also implicitly promoted to shared that could cause havoc.Which is what the code above does, and also what Manu's "thread safe" qualifier would do. `shared` would no longer mean "shared".
Aug 04 2020
On Wed, Aug 5, 2020 at 3:40 AM Timon Gehr via Digitalmars-d < digitalmars-d puremagic.com> wrote:On 04.08.20 15:52, Sebastiaan Koppe wrote:"treated as `shared`"; which is, under the `-preview` merged last year some time, that they can not be read or written at all. So you can take the address of all members, but not read or write to them.On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:aOn Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object.Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there'sNope. You can access all members, but they will be treated as `shared`.lot of resistance to this change.What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method. ...Hmm, well I guess if members themselves were also implicitly promoted toI described an alternative approach in my reply to you. If there is any way to inhibit an un-shared alias from the caller being propagated to the callee, maintaining a separation at the call boundary between the unshared and shared instances, then it's safe without any of my other speculative rules.shared that could cause havoc.Which is what the code above does, and also what Manu's "thread safe" qualifier would do. `shared` would no longer mean "shared".
Aug 04 2020
On Tue, Aug 4, 2020 at 11:55 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:Marking the function shared (and therefore the data accessible) doesn't magically make the function body threadsafe; what it does it makes the function potentially racy, and subject to VERY careful implementation. Sadly, I'm not aware of any CS research that can help you write atomic/threadsafe functions with any degree of proof in an environment like this (ie, without isolation and/or things like copy-on-write, etc). Calling shared methods from shared methods isn't safe either. Each call may be threadsafe atomically, but you can't author a leaf function without just as much care (or more) than the lower-level ones. If the function is more than 1-line, and carries some state across a few statements that call lower-level shared methods, then you need to be confident about the atomic state guarantees (or not! which is more likely) of the dependency API's, such that you don't create a state race between calls. Needless to say, it's still tricky, and there's really nothing the language can help you do... other than provide a strong mechanism to lock it off from normal code. As long as shared methods are few, and very well defined, then it is possible to implement very interesting and useful machines with this scheme, and also MASSIVELY help with validation of your ecosystem, and I have done so... but it's not magic, it's just a lot better than C++. The thing `shared` protects most against, is failure to correct multithreaded code when refactoring. I maintain a majorly-superscalar engine, and 90% of the race bugs we have had to spend heaps of time chasing down are the result of refactors or changes that had very peripheral contact with the multithreaded core; contact was narrow enough that we didn't notice then making changes, and there was nothing in the type system to alert us. For this reason alone, `shared` is worth its weight in gold. Hmm, well I guess if members themselves were also implicitlyOn Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:What exactly does the compiler need to prove? The restrictions are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method.There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object.Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.promoted to shared that could cause havoc.The catch though, is that if you allow implicit conversion from unshared->shared, then the rule changes from "shared methods must be threadsafe", to " shared methods must be threadsafe, AND unshared methods must also be threadsafe in the event they are called in conjunction with any of the shared methods (just not with eachother)". While I have plenty of small tools where the second rule is easy to implement, I can also think of many situations where even the first rule is hard to implement, and the second rule is virtually impossible. Unless we learn better ways to handle this, I don't think implicit conversion that way can scale. For the time being, you can implement the implicit conversion for the tools that may support that using `alias this` like I showed above, and I think that's appropriate for the time being. In general, I'd like to find ways we can enhance shared to be more than a marker; consider TSAN (ThreadSanitizer) in Clang; it instruments atomics with runtime tracking to detect races at runtime. I think it would be possible to instrument shared data/methods in a similar way in debug builds, and then we'd have meaningful compiler assistance.
Aug 04 2020
On Tue, Aug 4, 2020 at 11:55 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:On Tuesday, 4 August 2020 at 12:52:01 UTC, Manu wrote:It would be ideal if the compiler could validate that the un-shared methods are safe with respect to any shared methods. If a shared (ie, 'thread-safe') method were called in parallel to an un-shared method as is possible if the implicit conversion was allowed, then it is necessary that un-shared methods need to be tolerant of their 'thread-safe' API being called at any time. That's the change in meaning which is highly dangerous without some serious tooling. Essentially, adding a shared method to an object applies a contract to all other NOT-shared methods, and that doesn't feel good; "touching this here affects those other things"... it will be super hard to audit. Technology that looks like ThreadSanitizer might help, but it gets more tricky again when UFCS is used instead of member functions; those associations are harder to track. I think that idea is doomed, and more recently I'm thinking about ways to prevent sneaking un-shared aliases across the conversion boundary... but I don't know how to achieve that without some sort of aliasing rules which seem unlikely. The restrictionsOn Tue, Aug 4, 2020 at 5:40 PM Sebastiaan Koppe via Digitalmars-d < digitalmars-d puremagic.com> wrote:What exactly does the compiler need to prove?There is just one thing about shared I don't understand. If I design my object such that the non-shared methods are to be used thread-local and the shared methods from any thread, it follows that I should be able to call said shared methods from both a shared and non-shared instance of that object.Yes, this is a thing I talked at great length 1-2 years ago. If you take shared to mean "is thread-safe", then my idea was that not-shared -> shared implicit conversion should be possible. What I often do is this: struct Thing { ref shared(Thing) implSharedCast() { return *cast(shared)&this; } alias implSharedCast this; } If that were an implicit conversion, that implies a slight change of meaning of shared (to one that I consider immensely more useful), but it's more challenging for the compiler to prove with confidence, and there's a lot of resistance to this change.are all in place, you can only call shared methods on a shared object, and you can only access shared members in a shared method.Ummmm no, under any of my proposals, you can't access shared members in a shared method... a shared method's `this` pointer is shared, which means all members are shared, and shared data may not be read or written at all... shared methods have no safe access to members. They can only call other shared methods safe-ly. Hmm, well I guess if members themselves were also implicitlypromoted to shared that could cause havoc.Members inherit the qualifier of methods... in a shared method, members are shared, which means you can't read or write them.
Aug 04 2020
On 04.08.20 14:52, Manu wrote:slight change of meaning of sharedIt's not slight. What happens to module level variables?
Aug 04 2020
On Wed, Aug 5, 2020 at 3:35 AM Timon Gehr via Digitalmars-d < digitalmars-d puremagic.com> wrote:On 04.08.20 14:52, Manu wrote:I'm not sure what you're asking?slight change of meaning of sharedIt's not slight. What happens to module level variables?
Aug 04 2020
So, are you guys still surprised nobody wants to use D in production? The current language state is simply not suitable for development. D is just a bunch of features and most of them aren't even designed well. This stagnation has been going on for years without any improvement. It's time for D3.
Aug 04 2020
General disclaimer: My post may contain rebuttals to low-level technical points which were made in support of some high-level claims. Apparently there has been some confusion: When I disagree with a technical point, I am not automatically taking a stance against the corresponding high-level claim. On 02.08.20 22:50, Andrei Alexandrescu wrote:(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like: struct Widget { int x, y; char[] name, parentName; ... } I found it just shocking that following a Widget's construction whoever aliased the same strings the outside could change members of the Widget without Widget knowing. Of course, other D1 coders disliked that as well, so they'd defensively duplicate in the constructor: struct Widget { int x, y; char[] name, parentName; this(char[] n, char[] p) { name = n.dup; parentName = p.dup; } ... } thus ensuring proliferation of the garbage whether duplication was needed or not. I found this absolutely maddening, to the extent I didn't think D could be ever used at any considerable scale while dragging this anchor behind it. The second problem was Walter was adamant about using arrays of characters at strings. He found the notion of a library-defined string type (a la C++) an absolute abomination. Stubborn about it like I've never seen him before or after. So my unstoppable requests for a string type were met with the proverbial immovable refusal. Ironically, much of his argument came from an efficiency angle, yet the unnecessary duplication was way less efficient than some reference counting/small string optimization/etc scheme that a dedicated string type would use. ...At the cost of efficient slicing?Then we figured things would work out if we arranged things such that people could NOT change individual characters of a string. That would allow sharing without the danger of long-distance influence. After many discussions with Walter, Bartosz, Eric, Brad, and myself, immutable and const were born. ...And it has to be said that they do a great job at preventing individual chars in a `char[]` from being mutated. :-)Then followed the other qualifiers, in order: shared and inout.) * * * The result is... there: https://dlang.org/spec/const3.html. It has the images https://dlang.org/images/qualifier-combinations.svg and https://dlang.org/images/qualifier-conversions.svg and a large table and a lot of rules. Whenever I code anything generic, I find myself going back to those damn images and tables way more than anyone ought to. (And it's ironic... I made those. Woe to the relative newcomer.) ...Well, those tables and figures are easy to make, because they are based on simple rules. The combinatorial explosion is artificial.It's all too complicated, making generic D programming into 3D chess instead of the difficult endeavor it already is. And what does it buy us? Well, we don't need to define a string library type. Yowzers. (Actually we should if we want to get rid of the GC. But then Walter would oppose that. So - stalemate once again.) ...In general, Walter opposes memory ownership to be mediated by library types.Far as I can tell, the power/weight ratio of qualifier is very poor. I wish a DIP would revisit qualifiers with a stated intent to simplify them as much as possible. Whenever I code generically I invariably run into these issues: * Whatever I do, however I twitch, immutable finds the opportunity to lodge itself in a soft part of my body and cause constant pain. This doesn't work, that doesn't work. No solution for "tail immutable" - mutable references to immutable class instances can't be done without contortions. Can't assign to out immutable class references, though there's no reason for that (see Adam's recent post). ...Personally I don't use it where it causes those problems, but maybe the standard library is not afforded those conveniences.* No matter how I dice any significant piece of code, there will be five casts from immutable and/or back that I can't rid of and lose sleep at night trying to convince myself are justified. ...Maybe `immutable` is overused in that code base.* Every time "inout" comes within a radius of a mile of what I'm doing, it starts to stink like a skunk. I wish I could get a restraining order. I can't instantiate "inout" variables, so writing any tests or constraints becomes an advanced matter of defining functions and crap.I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.I get frustrated, I protest to this forum, and immediately a cabal is raised under Timon's leadership. The cabal convinces me that inout is actually great and that I'm an idiot. I do get convinced, which is more of a proof that Timon is very good, than a testament to the conviviality of inout. Then I leave and get back to my code, and it stinks of inout again. And I hate it and myself for having to deal with it. ...I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day. I'm attaching my draft write-up on `inout` from 2018.* Nobody - probably not even Timon - knows what "shared" does or is supposed to do and not do. The most I got from Walter ever is "shared is intentionally restricted so you can't do much without a cast". Yet the definition of "much" and the conditions under which casting is legit are not anywhere to be found. ...Well, I think I know what it is supposed to do, but I think Walter does not fully agree, as evidenced by the `scope shared` discussion. However, think of it this way: it is clear what _unshared_ does; it's a type system assertion that there will not be concurrent accesses to the given memory location. Casts to and from shared have to uphold that. That's essentially it. Then, semantically, shared variables are just standard variables from C or C++. Of course, that means `shared` variables should not be accessed using unsynchronized reads/writes in ` safe` functions, but I think Manu and Walter have been working on adding necessary type checks.* And of course, "shared" gladly partakes in qualifier combinations, thus spreading its stink around in a combinatorial manner.That's true of other language features. "Any field of any type can be of type `int`, so `int` is spreading its stink around in a combinatorial manner." If you consider type constructors to be type constructors instead of flags on top of a type, I think they are easier to understand and you don't have to handle nearly the same amount of combinatorial explosion.Believe it or not, the type "const inout shared T" exists. Of course, nobody knows what it really means or how it could be used. ...I beg to differ. It's either const(shared(T)) or immutable(T), but you don't know which. An use case is you have a function that may return references to its `shared` input, which it does not modify, and you want to preserve `immutable` qualifiers across function calls while also allowing the function to be called with mutable shared data. Unfortunately this is only obvious if you understand `inout`, but unfortunately, `inout` is harder to understand than type theory concepts that are more general than `inout`...A "Define All Qualifiers" DIP would be a radical improvement of the state of affairs.My attached draft has some overlap with that.
Aug 04 2020
On 8/4/20 12:57 PM, Timon Gehr wrote:My attached draft has some overlap with that.Thanks, Timon. Mike, looks like we have a great post for the blog in the making!
Aug 04 2020
Alternative syntax (even though it is in the template parameters, it is in fact erased and should not end in duplication): T(U) first(void T=inout, U)(T(U)[] data) { assert(data.length > 0); return data[0]; } -------- struct Foo(void Qual=inout) { private int[] _payload; this(Qual int[] payload) void(Qual) { this._payload = payload; } property Qual(int)[] payload() void(Qual) { return this._payload; } } -------- void assign(void Writable=inout, void Readable=inout)(ref Writable(int)* a,Readable(int)* b,ref Writable(int)* c,Readable(int)* d) { a=b; c=d; } -------- Qual(int)* id(void Qual=inout)(Qual(int)* p) { return p; } -------- static assert(is(typeof(&id!mutable)==int* function(int*))) static assert(is(typeof(&id!immutable)==immutable(int)* function(immutable(int)*))) static assert(is(typeof(&id!const)==const(int)* function(const(int)*))) That random mutable identifier tho...
Aug 04 2020
On 8/4/20 12:57 PM, Timon Gehr wrote:Well for what it's worth I have a simple question: how can I assess in druntime if a type T is copyable? I add the informal requirement that it's a simple query so it should be served with a proportionally simple answer. My initial take: static if (is(typeof((T x) { T y = x; }))) { ... } i.e. a lambda can be created that takes a T and creates a copy of it. Beautiful. This test, however, passes for inout types. And inout types cannot be considered really copyable, because they cannot be used in many places where one would expect to use a copyable type. To wit, a variety of unittests will fail (such as structs with copyable members), all protesting to the attempt of classifying inout types as copyable. Second attempt: static if (is(typeof((T x) { T y = x; })) && !is(T == inout U, U) { ... } So a type is copyable as before, just let's special case inout for exclusion. This already gets my diaper in a bunch because I need to special case a type of which utility I already am suspicious. And it's not only here - it's many, many similar places. Also, this also does NOT work because inout(const(int)) passes the test. This could probably be classified as a bug in the language or its compiler. So now I'm looking at things like importing "core.lifetime : emplace" and see if that compiles. Because the very complex implementation of emplace uses a complex mechanism to handle inout. I could be convinced that this awful complexity is justified given the choices made in the definition of this or that, but it would be more difficult to convince ourselves this is good programming language design. Simple questions should have simple answers.* Every time "inout" comes within a radius of a mile of what I'm doing, it starts to stink like a skunk. I wish I could get a restraining order. I can't instantiate "inout" variables, so writing any tests or constraints becomes an advanced matter of defining functions and crap.I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.I get frustrated, I protest to this forum, and immediately a cabal is raised under Timon's leadership. The cabal convinces me that inout is actually great and that I'm an idiot. I do get convinced, which is more of a proof that Timon is very good, than a testament to the conviviality of inout. Then I leave and get back to my code, and it stinks of inout again. And I hate it and myself for having to deal with it. ...I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day.
Aug 14 2020
On 8/14/20 9:45 AM, Andrei Alexandrescu wrote:Well for what it's worth I have a simple question: how can I assess in druntime if a type T is copyable? I add the informal requirement that it's a simple query so it should be served with a proportionally simple answer. My initial take: static if (is(typeof((T x) { T y = x; }))) { ... } i.e. a lambda can be created that takes a T and creates a copy of it. Beautiful. This test, however, passes for inout types. And inout types cannot be considered really copyable, because they cannot be used in many places where one would expect to use a copyable type. To wit, a variety of unittests will fail (such as structs with copyable members), all protesting to the attempt of classifying inout types as copyable.inout types aren't any less copyable than const or immutable types.Second attempt: static if (is(typeof((T x) { T y = x; })) && !is(T == inout U, U) { ... } So a type is copyable as before, just let's special case inout for exclusion.This seems silly, if it's copyable it's copyable. The unrestricted test should be valid.This already gets my diaper in a bunch because I need to special case a type of which utility I already am suspicious. And it's not only here - it's many, many similar places. Also, this also does NOT work because inout(const(int)) passes the test. This could probably be classified as a bug in the language or its compiler. So now I'm looking at things like importing "core.lifetime : emplace" and see if that compiles. Because the very complex implementation of emplace uses a complex mechanism to handle inout. I could be convinced that this awful complexity is justified given the choices made in the definition of this or that, but it would be more difficult to convince ourselves this is good programming language design. Simple questions should have simple answers.Is there a specific example? I suspect we are going to get into one of the foolish restrictions of inout -- can't be struct members or can't be returned if you don't have inout parameters. The answer is still -- fix inout so it's not so foolish. -Steve
Aug 14 2020
On Fri, Aug 14, 2020 at 11:50 PM Andrei Alexandrescu via Digitalmars-d < digitalmars-d puremagic.com> wrote:On 8/4/20 12:57 PM, Timon Gehr wrote:I'm delighted by this post. Keep talking this way, and we have a chance to make the kind of progress we've been waiting for! If you're looking into emplace (and friends) you should be familiar with Suleyman's move constructor work-in-progress. I am convinced that's the only reasonable path forwards on that front.Well for what it's worth I have a simple question: how can I assess in druntime if a type T is copyable? I add the informal requirement that it's a simple query so it should be served with a proportionally simple answer. My initial take: static if (is(typeof((T x) { T y = x; }))) { ... } i.e. a lambda can be created that takes a T and creates a copy of it. Beautiful. This test, however, passes for inout types. And inout types cannot be considered really copyable, because they cannot be used in many places where one would expect to use a copyable type. To wit, a variety of unittests will fail (such as structs with copyable members), all protesting to the attempt of classifying inout types as copyable. Second attempt: static if (is(typeof((T x) { T y = x; })) && !is(T == inout U, U) { ... } So a type is copyable as before, just let's special case inout for exclusion. This already gets my diaper in a bunch because I need to special case a type of which utility I already am suspicious. And it's not only here - it's many, many similar places. Also, this also does NOT work because inout(const(int)) passes the test. This could probably be classified as a bug in the language or its compiler. So now I'm looking at things like importing "core.lifetime : emplace" and see if that compiles. Because the very complex implementation of emplace uses a complex mechanism to handle inout. I could be convinced that this awful complexity is justified given the choices made in the definition of this or that, but it would be more difficult to convince ourselves this is good programming language design. Simple questions should have simple answers.* Every time "inout" comes within a radius of a mile of what I'm doing, it starts to stink like a skunk. I wish I could get a restraining order. I can't instantiate "inout" variables, so writing any tests or constraints becomes an advanced matter of defining functions and crap.I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.I get frustrated, I protest to this forum, and immediately a cabal is raised under Timon's leadership. The cabal convinces me that inout is actually great and that I'm an idiot. I do get convinced, which is more of a proof that Timon is very good, than a testament to the conviviality of inout. Then I leave and get back to my code, and it stinks of inout again. And I hate it and myself for having to deal with it. ...I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day.
Aug 14 2020
On 14.08.20 15:45, Andrei Alexandrescu wrote:On 8/4/20 12:57 PM, Timon Gehr wrote:Those are protesting to the attempt of using `inout` types as member variables, not to the attempt of classifying them as copyable. The question is whether we really want types that can't be used as fields, and I think no, we don't want that. (But all of those weird limitations would not exist if `inout` was designed right!)Well for what it's worth I have a simple question: how can I assess in druntime if a type T is copyable? I add the informal requirement that it's a simple query so it should be served with a proportionally simple answer. My initial take: static if (is(typeof((T x) { T y = x; }))) { ... } i.e. a lambda can be created that takes a T and creates a copy of it. Beautiful. This test, however, passes for inout types. And inout types cannot be considered really copyable, because they cannot be used in many places where one would expect to use a copyable type. To wit, a variety of unittests will fail (such as structs with copyable members), all protesting to the attempt of classifying inout types as copyable. ...* Every time "inout" comes within a radius of a mile of what I'm doing, it starts to stink like a skunk. I wish I could get a restraining order. I can't instantiate "inout" variables, so writing any tests or constraints becomes an advanced matter of defining functions and crap.I have never understood the `(inout int){ T.init; }` idiom. Just use `(T value){ value; }`.I get frustrated, I protest to this forum, and immediately a cabal is raised under Timon's leadership. The cabal convinces me that inout is actually great and that I'm an idiot. I do get convinced, which is more of a proof that Timon is very good, than a testament to the conviviality of inout. Then I leave and get back to my code, and it stinks of inout again. And I hate it and myself for having to deal with it. ...I am sure you are sincere, but I still think this is a misrepresentation. I don't think I ever claimed that `inout` is great. I merely understand what `inout` is supposed to be, but it comes way short. See all of the issues I have opened that show that type checking for `inout` is broken. When I tried to document inout properly in 2018 I found multiple new type system holes, I think they are open to this day.... Also, this also does NOT work because inout(const(int)) passes the test. This could probably be classified as a bug in the language or its compiler. ...It most definitely is.So now I'm looking at things like importing "core.lifetime : emplace" and see if that compiles. Because the very complex implementation of emplace uses a complex mechanism to handle inout. I could be convinced that this awful complexity is justified given the choices made in the definition of this or that, but it would be more difficult to convince ourselves this is good programming language design.The complexity is not justified, it is not good programming language design and I never claimed otherwise. Good programming language features are orthogonal to each other.Simple questions should have simple answers.If the above implementation of isCopyable is not the one you want, something else is off.
Aug 14 2020
On 15.08.20 03:29, Timon Gehr wrote:Oh, actually I misspoke, didn't notice that you changed my suggestion from the github issue from __traits(compiles, ...) to is(typeof(...)). Use __traits(compiles, ...), not is(typeof(...)). typeof assigns types to expressions that would not otherwise compile, even lambdas: void main(){ int x; static void foo(){ pragma(msg, typeof(()=>x)); // int delegate() pure nothrow nogc safe pragma(msg, __traits(compiles, ()=>x)); // false } } There is this persistent myth that __traits(compiles, ...) is the same as is(typeof(...)) this is not the case and IIRC I have used it to demonstrate that most template constraints in Phobos don't work correctly.Simple questions should have simple answers.If the above implementation of isCopyable is not the one you want, something else is off.
Aug 14 2020
On 8/14/20 9:35 PM, Timon Gehr wrote:On 15.08.20 03:29, Timon Gehr wrote:This is a problem as __traits should be a lower-level, seldom-used resort. Are the distinctions between the two documented somewhere?Oh, actually I misspoke, didn't notice that you changed my suggestion from the github issue from __traits(compiles, ...) to is(typeof(...)). Use __traits(compiles, ...), not is(typeof(...)). typeof assigns types to expressions that would not otherwise compile, even lambdas: void main(){ int x; static void foo(){ pragma(msg, typeof(()=>x)); // int delegate() pure nothrow nogc safe pragma(msg, __traits(compiles, ()=>x)); // false } } There is this persistent myth that __traits(compiles, ...) is the same as is(typeof(...)) this is not the case and IIRC I have used it to demonstrate that most template constraints in Phobos don't work correctly.Simple questions should have simple answers.If the above implementation of isCopyable is not the one you want, something else is off.
Aug 17 2020
On Monday, 17 August 2020 at 11:23:04 UTC, Andrei Alexandrescu wrote:On 8/14/20 9:35 PM, Timon Gehr wrote:__traits(compiles) is not really a trait. Whether an expression is valid or not isn't really metadata of the expression - because in e.g. D, an expression must be valid in order to have metadata. Traits can be wrapped by templates, this can't. Besides these points, it would be nicer syntax to have something like __compiles(expr) instead.There is this persistent myth that __traits(compiles, ...) is the same as is(typeof(...)) this is not the case and IIRC I have used it to demonstrate that most template constraints in Phobos don't work correctly.This is a problem as __traits should be a lower-level, seldom-used resort.
Aug 25 2020
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:* Nobody - probably not even Timon - knows what "shared" does or is supposed to do and not do. The most I got from Walter ever is "shared is intentionally restricted so you can't do much without a cast". Yet the definition of "much" and the conditions under which casting is legit are not anywhere to be found.Shouldn't shared just be a relaxation in compiler checks so that any data that has the "shared" qualifier is allowed to be shared between threads. This is pretty much what __gshared is about. For the compiler to check if you really can share the data is currently impossible with current compiler technology. It can be atomic integers, Intel TSX, some very elaborate lockless algorithm, lockless COW, even mutexes/spinlocks if necessary. Programmer must "manually" ensure that the data can be shared, there is no other way. So yes, a revisit of the qualifiers should be done so that they can be precisely defined.
Aug 13 2020
On Thursday, 13 August 2020 at 19:22:29 UTC, IGotD- wrote:Shouldn't shared just be a relaxation in compiler checks so that any data that has the "shared" qualifier is allowed to be shared between threads. This is pretty much what __gshared is about.It should help the programmer write bug-free code, which __gshared does not. A panacea would be great, but more realistically we should disallow reading, writing and calling non-shared methods on shared data in safe code. This limits the number of places where races can exist, but still (sadly) requires the programmer to know what he's doing when he writes shared code. -- Simen
Aug 13 2020
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:A "Define All Qualifiers" DIP would be a radical improvement of the state of affairs.Make sure there is a separate DIP for each qualifier. I have a feeling some qualifiers will be quite a battle with different opinions.
Aug 14 2020
On 8/14/20 4:30 PM, IGotD- wrote:On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:Conversions and interactions make this a package deal.A "Define All Qualifiers" DIP would be a radical improvement of the state of affairs.Make sure there is a separate DIP for each qualifier. I have a feeling some qualifiers will be quite a battle with different opinions.
Aug 17 2020
On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like: [...]Nearly every time I use inout I run into problems. Something doesn't compile, I get annoyed, and switch to templated this. Can anyone tell me what inout does that templated this doesn't?
Aug 24 2020
On 8/24/20 12:13 PM, Atila Neves wrote:On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:I have the opposite experience -- inout just works most of the time.(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like: [...]Nearly every time I use inout I run into problems. Something doesn't compile, I get annoyed, and switch to templated this.Can anyone tell me what inout does that templated this doesn't?1. It does not create a separate identical implementation for all flavors of mutability 2. It is not mutable even when the this reference is. The problems with inout have nothing to do with its utility. They are artificial limitations that should not have been included. -Steve
Aug 24 2020
On 24.08.20 18:27, Steven Schveighoffer wrote:- It works with virtual functions/function pointers/delegates. - The implementation is type checked before instantiation.Can anyone tell me what inout does that templated this doesn't?1. It does not create a separate identical implementation for all flavors of mutability2. It is not mutable even when the this reference is.(That's a hard property to keep in a sound version without artificial limitations.)
Aug 24 2020
On 8/24/20 1:09 PM, Timon Gehr wrote:On 24.08.20 18:27, Steven Schveighoffer wrote:I'm not sure what you mean -- inout is like const in that you cannot mutate data via that reference. with template this parameters, you can statically check if it's mutable, and mutate. It is only important if your goal is to identify that a function will not modify data passed to it. In essence, if you care about const functions. -Steve2. It is not mutable even when the this reference is.(That's a hard property to keep in a sound version without artificial limitations.)
Aug 24 2020
On 24.08.20 19:19, Steven Schveighoffer wrote:With a better type system the caller might pass in delegates that know about the mutability. void foo[Qualifier q](ref q int x, void delegate(ref q int x) process){ process(x); } void main(){ int x=0; foo(x,(ref int a){ a=2; }); // (q is mutable) immutable int y=2; foo(y,(ref immutable int b){ x=b; }); // (q is `immutable`) }I'm not sure what you mean -- inout is like const in that you cannot mutate data via that reference.
Aug 24 2020
On Tue, Aug 25, 2020 at 2:15 AM Atila Neves via Digitalmars-d < digitalmars-d puremagic.com> wrote:On Sunday, 2 August 2020 at 20:50:14 UTC, Andrei Alexandrescu wrote:Generates one instance, rather than many. Is a 'hard' function; ie, can by virtual...(Background: qualifiers were introduced following my horror when I started writing D1 code and saw that strings are represented as char[]. So structs with string members would look like: [...]Nearly every time I use inout I run into problems. Something doesn't compile, I get annoyed, and switch to templated this. Can anyone tell me what inout does that templated this doesn't?
Aug 24 2020