www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Deprecating this(this)

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
We need to have a simple recipe on how to define a canonical object. 
That would include requirements such as:

* should work with mutable, const, immutable, and shared
* the right way to define constructor(s)
* the right way to define copying
* the right way to define destructor(s)

I've asked my student Razvan to document the behavior of this(this). His 
work:

https://github.com/dlang/dmd/pull/8055
https://github.com/dlang/dlang.org/pull/2281
https://github.com/dlang/dlang.org/pull/2299

... reveals a puzzling array of behaviors. Sometimes the typechecking is 
wrong, too.

I think it's very important for us to have a simple, correct, and 
canonical way of defining structs in the D language that work with the 
language features: qualifiers, pure, safe, and nogc.

Once we have that, we can encapsulate desirable abstractions (such as 
 nogc safe collections that work in pure code), regardless of how 
difficult their implementations might be. It seems that currently 
this(this) does not allow us to do that.

Eduard, another student I work with, has made steps toward a collections 
library that rant into difficulties:

*  safe is achievable with relative ease

* immutable and const are very difficult, but we have an attack 
(assuming copy construction gets taken care of)

*  nogc is doable with a couple of conventions for allocators

* pure is difficult

* making them work together is very difficult

We need to offer features, tools, and guidance toward creating simple 
encapsulated types that work well with D's own abstractions. Once we 
have that we can build libraries to work satisfactorily for any domain.

I think the way to move forward is to deprecate this(this) entirely and 
create a DIP that allows people to define truly encapsulated structs. 
This is important, urgent, and of huge impact.

I am looking for folks to assist me in creating a DIP for that. There 
will be a _lot_ of work involved, so don't take it lightly.


Thanks,

Andrei
Mar 31 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, March 31, 2018 19:38:06 Andrei Alexandrescu via Digitalmars-d 
wrote:
 We need to have a simple recipe on how to define a canonical object.
 That would include requirements such as:

 * should work with mutable, const, immutable, and shared
 * the right way to define constructor(s)
 * the right way to define copying
 * the right way to define destructor(s)

 I've asked my student Razvan to document the behavior of this(this). His
 work:

 https://github.com/dlang/dmd/pull/8055
 https://github.com/dlang/dlang.org/pull/2281
 https://github.com/dlang/dlang.org/pull/2299

 ... reveals a puzzling array of behaviors. Sometimes the typechecking is
 wrong, too.

 I think it's very important for us to have a simple, correct, and
 canonical way of defining structs in the D language that work with the
 language features: qualifiers, pure, safe, and nogc.

 Once we have that, we can encapsulate desirable abstractions (such as
  nogc safe collections that work in pure code), regardless of how
 difficult their implementations might be. It seems that currently
 this(this) does not allow us to do that.

 Eduard, another student I work with, has made steps toward a collections
 library that rant into difficulties:

 *  safe is achievable with relative ease

 * immutable and const are very difficult, but we have an attack
 (assuming copy construction gets taken care of)

 *  nogc is doable with a couple of conventions for allocators

 * pure is difficult

 * making them work together is very difficult

 We need to offer features, tools, and guidance toward creating simple
 encapsulated types that work well with D's own abstractions. Once we
 have that we can build libraries to work satisfactorily for any domain.

 I think the way to move forward is to deprecate this(this) entirely and
 create a DIP that allows people to define truly encapsulated structs.
 This is important, urgent, and of huge impact.

 I am looking for folks to assist me in creating a DIP for that. There
 will be a _lot_ of work involved, so don't take it lightly.
So, is the idea then that we'd switch to copy constructors? - Jonathan M Davis
Mar 31 2018
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/31/18 8:25 PM, Jonathan M Davis wrote:
 So, is the idea then that we'd switch to copy constructors?
Something like that. We'll need to define them carefully to obey safety and purity.
Mar 31 2018
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Mar 31, 2018 at 07:38:06PM -0400, Andrei Alexandrescu via Digitalmars-d
wrote:
[...]
 Once we have that, we can encapsulate desirable abstractions (such as
  nogc safe collections that work in pure code), regardless of how
 difficult their implementations might be. It seems that currently
 this(this) does not allow us to do that.
What exactly is it about this(this) that blocks us from doing that? Removing this(this) is going to be a huge breaking change far bigger than, say, removing autodecoding ever will be.
 Eduard, another student I work with, has made steps toward a
 collections library that rant into difficulties:
 
 *  safe is achievable with relative ease
 
 * immutable and const are very difficult, but we have an attack
 (assuming copy construction gets taken care of)
A lot of us here have essentially given up on const except for a few very narrow cases. The transitive nature of const makes it extremely difficult to work with in the general case, even though the simplest use cases are workable. One of the biggest stumbling blocks is that whenever ranges are involved, const is practically out of the question, because even though it can be made to work for most cases, there will almost always be that one pathological case where it's impossible / too hard to work around, and that ruins it for everything else, so that it's much simpler to just avoid it altogether. Also, as far as containers are concerned, the lack of a standard way to construct head-mutable / tail-const types that works analogously with built-in arrays makes it very difficult to write generic containers that work well with const/immutable. It's not too hard to make it work for specific types, but very difficult to write a truly *generic* container that can be deployed in all situations where const/immutable are involved. [...]
 * pure is difficult
[...] The one nagging question I've been having about pure is: how much are we actually taking advantage of the guarantees provided by pure? We have developed very clever ways of extending the traditional definition of pure and invented creative ways of making more things pure, which is all great. But AFAIK the only place where it's actually taken advantage of is to elide some redundant function calls inside a single expression. And perhaps infer uniqueness in some cases for implicit casting to immutable. While these are indisputably useful, they seem so far to be only relatively minor benefits. If pure is indeed so difficult to support generically, it begs the question, is it worth the effort just to gain these niggling benefits? Whatever happened to larger-scale benefits conferred by purity? T -- Computers aren't intelligent; they only think they are.
Mar 31 2018
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/31/18 8:32 PM, H. S. Teoh wrote:
 On Sat, Mar 31, 2018 at 07:38:06PM -0400, Andrei Alexandrescu via
Digitalmars-d wrote:
 [...]
 Once we have that, we can encapsulate desirable abstractions (such as
  nogc safe collections that work in pure code), regardless of how
 difficult their implementations might be. It seems that currently
 this(this) does not allow us to do that.
What exactly is it about this(this) that blocks us from doing that?
See the updated docs. Too many bugs in design and implementation.
 Removing this(this) is going to be a huge breaking change far bigger
 than, say, removing autodecoding ever will be.
We're not removing it as much as evolving it: we define an alternate copying mechanism, and once that is in tip-top shape, we deprecate this(this).
 A lot of us here have essentially given up on const except for a few
 very narrow cases.  The transitive nature of const makes it extremely
 difficult to work with in the general case, even though the simplest use
 cases are workable.
Immutable is where it's at, in terms of usefulness. Const is a mere servant of it (and of mutable).
 One of the biggest stumbling blocks is that whenever ranges are
 involved, const is practically out of the question, because even though
 it can be made to work for most cases, there will almost always be that
 one pathological case where it's impossible / too hard to work around,
 and that ruins it for everything else, so that it's much simpler to just
 avoid it altogether.
Yah, the DIP might address that, too. Consider: void fun(R)(R arr) { pragma(msg, typeof(arr)); } void main() { immutable(int[]) arr = [ 1, 2, 3 ]; pragma(msg, typeof(arr)); fun(arr); } The program prints during compilation: immutable(int[]) immutable(int)[] Interesting! So the type of the array changes during template matching, which is an exception to the rule that templates always glom to the exact type passed. This is a hack introduced in the compiler in response to the issues you mention. But we don't need hacks and special casing - we need a means for types to say "here's what needs to happen when a template parameter is matched against this type". So the DIP would address manipulating qualified ranges as a perk.
 The one nagging question I've been having about pure is: how much are we
 actually taking advantage of the guarantees provided by pure?
Very little, but that doesn't matter. The problem is it's underspecified. So now it's like a vague threat - whenever we mess with fear somebody comes asking, but what about an aggressive compiler doing some unexpected optimizations based on such and such interpretation? We need to lock pure down. Andrei
Mar 31 2018
parent reply ag0aep6g <anonymous example.com> writes:
On 04/01/2018 03:08 AM, Andrei Alexandrescu wrote:
 On 3/31/18 8:32 PM, H. S. Teoh wrote:
[...]
 What exactly is it about this(this) that blocks us from doing that?
See the updated docs. Too many bugs in design and implementation.
 Removing this(this) is going to be a huge breaking change far bigger
 than, say, removing autodecoding ever will be.
We're not removing it as much as evolving it: we define an alternate copying mechanism, and once that is in tip-top shape, we deprecate this(this).
Is there a fundamental flaw in the postblit idea, or are you just going to give postblit a new syntax, and try to avoid all the issues that `this(this)` currently has? If there's a fundamental flaw, I'd be interested in what it is. I can't make it out in your additions to the spec, if it's in there. I can see that `this(this)` is a mess, but it also looks like a lot could be fixed. For example, how it interacts with const/immutable is ridiculous, but that could probably be fixed. If you're just going for a clean slate, I can see the appeal. You avoid dealing with the hard breakage that fixing `this(this)` would most probably bring.
Apr 01 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, April 01, 2018 14:55:07 ag0aep6g via Digitalmars-d wrote:
 On 04/01/2018 03:08 AM, Andrei Alexandrescu wrote:
 On 3/31/18 8:32 PM, H. S. Teoh wrote:
[...]
 What exactly is it about this(this) that blocks us from doing that?
See the updated docs. Too many bugs in design and implementation.
 Removing this(this) is going to be a huge breaking change far bigger
 than, say, removing autodecoding ever will be.
We're not removing it as much as evolving it: we define an alternate copying mechanism, and once that is in tip-top shape, we deprecate this(this).
Is there a fundamental flaw in the postblit idea, or are you just going to give postblit a new syntax, and try to avoid all the issues that `this(this)` currently has? If there's a fundamental flaw, I'd be interested in what it is. I can't make it out in your additions to the spec, if it's in there. I can see that `this(this)` is a mess, but it also looks like a lot could be fixed. For example, how it interacts with const/immutable is ridiculous, but that could probably be fixed.
One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. To cleanly deal with const, you need something more like a copy constructor where you initialize it with the adjusted values directly rather than mutating the copy.
 If you're just going for a clean slate, I can see the appeal. You avoid
 dealing with the hard breakage that fixing `this(this)` would most
 probably bring.
Avoiding any breakage would be ideal, but it's unlikely that that can be done if we want to make copying const objects work properly - not unless we did something like use postblit constructors for mutable objects and copy constructors for const objects, and that would just cause other problems (including having to duplicate the code that deals with copying an object if it's going to work with const). On the bright side, if we're replacing postblit constructors with some other type of constructor for copying, it should be a pretty straightforward process. But there isn't much point in worrying about how much breakage there's going to be before we really know where we want to go with this. At this point, it's just clear that as things stand, postblit constructors have some definite problems (some which are implementation issues and some which are language design issues), and based on past discussions on this and previous attempts to fix some of the problems with postblit constructors, it seems pretty unlikely that we can fully fix postblit constructors. I'm sure that we could fix some of the issues, but others (most notably, the issues with const) seem pretty intractable. - Jonathan M Davis
Apr 01 2018
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't 
 work with const. The problem is that a postblit constructor 
 works by copying the object and _then_ mutating it, and you 
 can't mutate a const object. To cleanly deal with const, you 
 need something more like a copy constructor where you 
 initialize it with the adjusted values directly rather than 
 mutating the copy.
I've always wondered about that, is the difference between that anything more than philosophical? Put another way if a this(this) is weakly pure, is there any safety issues with the compiler permitting the mutation on a (non-shared? not sure if this would be a requirement) const object? I'm not sure what the spec says, but if you take the view that the const object is no fully initialised until the postblit is done, then I don't see the problem.
Apr 01 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Apr 01, 2018 at 02:31:06PM +0000, Nicholas Wilson via Digitalmars-d
wrote:
 On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't work
 with const. The problem is that a postblit constructor works by
 copying the object and _then_ mutating it, and you can't mutate a
 const object. To cleanly deal with const, you need something more
 like a copy constructor where you initialize it with the adjusted
 values directly rather than mutating the copy.
I've always wondered about that, is the difference between that anything more than philosophical? Put another way if a this(this) is weakly pure, is there any safety issues with the compiler permitting the mutation on a (non-shared? not sure if this would be a requirement) const object? I'm not sure what the spec says, but if you take the view that the const object is no fully initialised until the postblit is done, then I don't see the problem.
Yeah I've been wondering about this too. I mean, currently, ctors are allowed to assign to immutable fields, because, well, it's initialization, not after-the-fact mutation. Why can't we extend this to postblits? Now, there's certainly the issue of ctors "leaking" mutable references to immutable fields if not implemented properly, but since postblits are run after a built-into-the-language copying of the original, supposedly an opaque process, it seems reasonable enough to allow the postblit to be regarded as initialization and able to assign to const/immutable fields once. At the very least, allow rebinding of const/immutable references in the postblit. (Allowing straight-out reassignment may have adverse effects if the copy shares a reference to an immutable object.) T -- What is Matter, what is Mind? Never Mind, it doesn't Matter.
Apr 02 2018
prev sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, April 02, 2018 08:56:41 H. S. Teoh via Digitalmars-d wrote:
 On Sun, Apr 01, 2018 at 02:31:06PM +0000, Nicholas Wilson via Digitalmars-
d wrote:
 On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't work
 with const. The problem is that a postblit constructor works by
 copying the object and _then_ mutating it, and you can't mutate a
 const object. To cleanly deal with const, you need something more
 like a copy constructor where you initialize it with the adjusted
 values directly rather than mutating the copy.
I've always wondered about that, is the difference between that anything more than philosophical? Put another way if a this(this) is weakly pure, is there any safety issues with the compiler permitting the mutation on a (non-shared? not sure if this would be a requirement) const object? I'm not sure what the spec says, but if you take the view that the const object is no fully initialised until the postblit is done, then I don't see the problem.
Yeah I've been wondering about this too. I mean, currently, ctors are allowed to assign to immutable fields, because, well, it's initialization, not after-the-fact mutation. Why can't we extend this to postblits? Now, there's certainly the issue of ctors "leaking" mutable references to immutable fields if not implemented properly, but since postblits are run after a built-into-the-language copying of the original, supposedly an opaque process, it seems reasonable enough to allow the postblit to be regarded as initialization and able to assign to const/immutable fields once. At the very least, allow rebinding of const/immutable references in the postblit. (Allowing straight-out reassignment may have adverse effects if the copy shares a reference to an immutable object.)
The core problem is that in a postblit, you're reading a member variable and then assigning to it, which violates const. For some simple cases, we could essentially make the member variables tail-const within the postblit, because we could safely say that the member variable was a distinct copy and that overwriting it wouldn't really cause problems - e.g. assigning to an int wouldn't really be a problem, and assigning to const(T)* wouldn't really be a problem. For classes, it's a bit hinky, because there's no such thing as tail-const for classes, but they're conceptually the same as const(T)*, so we could cheat in a postblit and still let the reference be assigned once. However, the real problem is once you start dealing with structs. Not only is there no real concept of tail-const with structs, and they can contain any combination of value types, reference types, pseudo-reference types, etc. but they can overload opAssign and postblit. And of course, the postblit constructor for member variables is run before the postblit constructor for the object containing them, and there's no guarantee that the postblit actually did _anything_ involving making a deep copy (e.g. it could simply have been printing out that the object was copied). So, figuring out how independent a copy the struct is from the original isn't necessarily very straightforward, and even if it were, how would we do the equivalent of const(T)* to allow the struct to be reassigned without mucking with any data that's not independent? It might contain members that have their data directly embedded in the struct and members which are reference types, but it's still one unit, and stuff like opAssign is not designed with the idea that you just overwrite some of the struct. And of course, once opAssign is overloaded, who knows what the semantics of assigning to the struct are. As such, how do you reason about the safety of relaxing the type system to allow assignment to a struct that has overloaded opAssign? At one point, Kenji was working on a solution to the problem (I _think_ that he had a DIP on it, but it's been a while, so I don't remember), and as I recall, Walter and Andrei vetoed it, because it was too complicated. And I don't even know if he actually, fully solved the problem. Ultimately, this is all much, much cleaner if copying an object involves initializing it directly rather than doing a shallow copy and then doing a deep copy and reassigning pieces of the object. That sucks for the simple cases, because if you don't care about stuff like const, and your object has a bunch of members in it, with a postblit constructor, you might only have to manually do something with a few of them (whereas in C++, you'd have to list them all individually), but as far as I can tell, it's pretty much required for the complex cases. Off the top of my head, what I would probably do if I were redesigning this would be to go with copy constructors but improve the syntax so that you don't have to list any of the member variables unless you're actually going to be giving them different values. If that could be done cleanly, then it might end up being a bit more verbose depending on what we had to do, or it might look pretty much the same as now, except that you'd then be assigning from a parameter (be it implicit or explicit) representing the object being copied rather than the object being constructed. Either way, I wouldn't go with an actual postblit constructor. It's a great idea if const isn't a thing, but since const is a thing, it's not so great an idea. It also isn't possible with postblit constructors to look at the original object to get stuff like its address, which could matter in the uses cases that the DIP for opMove is trying to solve. So, I'm inclined to think that copy constructors would ultimately be a better solution - especially if we can keep the syntax such that you don't have to explicitly initialize any members that you just want copied. At that point, syntactically, they'd be pretty much the same as postblit constructors. - Jonathan M Davis
Apr 02 2018
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/1/18 9:37 AM, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't work with const.
Actually they do...
Apr 01 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, April 01, 2018 10:31:46 Andrei Alexandrescu via Digitalmars-d 
wrote:
 On 4/1/18 9:37 AM, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't work with
 const.
Actually they do...
How so? In the postblit, you're dealing with a copy of an object where everything is already initialized. Mutating the object would violate const. It could be made to work for primitive types that the compiler understands and knows that the member variable is truly independent - e.g. it could be allowed to mutate an int, or it could be allowed to mutate a pointer while treating what it points to as const, but as soon as you're dealing with user-defined types, you're screwed - especially if you're dealing with something like a struct with a user-defined opAssign. You're reading an existing value and then mutating it, and it has to be at least tail-const, because the original was const - and tail-const is pretty meaningless for structs and can't really be represented in the type system for classes. So, I don't see how postblit could be made to work with a const object of any real complexity. It can be made to work in some corner cases but not in general. Kenji worked on a solution to the problem with const and postblit several years ago (and I'm not sure how close he got to really solving it), but as I recall, you and Walter shot it down because it was overly complicated. How are you proposing that const work with postblit? - Jonathan M Davis
Apr 01 2018
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't 
 work with const. The problem is that a postblit constructor 
 works by copying the object and _then_ mutating it, and you 
 can't mutate a const object.
I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway.
Apr 01 2018
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 1 April 2018 at 14:34:01 UTC, ag0aep6g wrote:
 On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't 
 work with const. The problem is that a postblit constructor 
 works by copying the object and _then_ mutating it, and you 
 can't mutate a const object.
I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway.
Ah, you said it much better than I did.
Apr 01 2018
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/1/18 10:34 AM, ag0aep6g wrote:
 On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't work with 
 const. The problem is that a postblit constructor works by copying the 
 object and _then_ mutating it, and you can't mutate a const object.
I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway.
Yes, precisely what I had been arguing here: https://issues.dlang.org/show_bug.cgi?id=18417#c5, however const/immutable postblit was recently deprecated by: https://github.com/dlang/dmd/pull/8032 So I don't think D is going to allow const postblit any more. Maybe not even postblit any more. -Steve
Apr 02 2018
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 2 April 2018 at 14:07:21 UTC, Steven Schveighoffer 
wrote:
 On 4/1/18 10:34 AM, ag0aep6g wrote:
 On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis 
 wrote:
 One issue is that postblit constructors fundamentally don't 
 work with const. The problem is that a postblit constructor 
 works by copying the object and _then_ mutating it, and you 
 can't mutate a const object.
I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway.
Yes, precisely what I had been arguing here: https://issues.dlang.org/show_bug.cgi?id=18417#c5, however const/immutable postblit was recently deprecated by: https://github.com/dlang/dmd/pull/8032 So I don't think D is going to allow const postblit any more. Maybe not even postblit any more. -Steve
Andrei did post an example where treating the designation as mutable, stuffing references to mutable data into is and then treating it as immutable leaves you with an immutable reference to immutable data. However i think that loophole is fixed if you only allow assignment to const/immutable from a pure postblit.
Apr 02 2018
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/2/18 10:24 AM, Nicholas Wilson wrote:
 On Monday, 2 April 2018 at 14:07:21 UTC, Steven Schveighoffer wrote:
 On 4/1/18 10:34 AM, ag0aep6g wrote:
 On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
 One issue is that postblit constructors fundamentally don't work 
 with const. The problem is that a postblit constructor works by 
 copying the object and _then_ mutating it, and you can't mutate a 
 const object.
I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway.
Yes, precisely what I had been arguing here: https://issues.dlang.org/show_bug.cgi?id=18417#c5, however const/immutable postblit was recently deprecated by: https://github.com/dlang/dmd/pull/8032 So I don't think D is going to allow const postblit any more. Maybe not even postblit any more.
Andrei did post an example where treating the designation as mutable, stuffing references to mutable data into is and then treating it as immutable leaves you with an immutable reference to immutable data.
I think you meant immutable reference to mutable data.
 
 However i think that loophole is fixed if you only allow assignment to 
 const/immutable from a pure postblit.
 
It's pretty straightforward I would think: If a postblit is being called on an immutable, it *must* be new data, because if it was data that just got overwritten, that data couldn't possibly be immutable. Therefore, there's leeway to mutate the head at that point, because nothing else can see it yet. Doesn't need to be pure. Same leeway as an immutable constructor I think. But all this discussion is moot if the postblit is going away. -Steve
Apr 02 2018
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On Monday, 2 April 2018 at 14:24:20 UTC, Nicholas Wilson wrote:
 On Monday, 2 April 2018 at 14:07:21 UTC, Steven Schveighoffer 
 wrote:
 On 4/1/18 10:34 AM, ag0aep6g wrote:
[...]
 I'm not so sure if that's fundamental. Can't we just say that 
 the copy is head-mutable at the time when the postblit 
 function is called, and it only becomes fully const after 
 that?
[...]
 Yes, precisely what I had been arguing here: 
 https://issues.dlang.org/show_bug.cgi?id=18417#c5
[...]
 Andrei did post an example where treating the designation as 
 mutable, stuffing references to mutable data into is and then 
 treating it as immutable leaves you with an immutable reference 
 to immutable data.
This one? ---- int[] sneaky; struct A { private int[] innocent; this(this) { sneaky = innocent; } } void main() { immutable a = A([1, 2, 3]); auto b = a; sneaky[1] = 42; // oops import std.stdio; writeln(a.innocent); // ooooops } ---- That wouldn't be possible if `innocent` were only head-mutable in the postblit function, instead of fully mutable as it is currently (bug). `innocent` would be typed as `immutable(int)[]`, and you could not assign it to the fully mutable `sneaky`.
 However i think that loophole is fixed if you only allow 
 assignment to const/immutable from a pure postblit.
Still must be head-mutable only. With fully mutable, you can mutate data that is seen as immutable elsewehre, even if the postblit function is pure: ---- struct S { int* x; this(this) pure { *x = 13; } } void main() { auto s = immutable S(new int(42)); auto s2 = s; assert(*s.x == 42); /* fails; immutability has been broken */ } ---- Issue 18357 covers both Andrei's example, and this one. https://issues.dlang.org/show_bug.cgi?id=18357
Apr 02 2018
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/02/2018 10:59 AM, ag0aep6g wrote:
 
 That wouldn't be possible if `innocent` were only head-mutable in the 
 postblit function, instead of fully mutable as it is currently (bug).
 
 `innocent` would be typed as `immutable(int)[]`, and you could not 
 assign it to the fully mutable `sneaky`.
Problem is we don't have head-mutable in the language. Yes, for built-in slices the mechanism is simple - just change qualifier(T[]) to qualifier(T)[]. For a struct S, there is no way to convert from qualifier(S) to tailqualifier(S). I plan to attack this directly in the DIP - provide a way for structs to express "here's what implicit conversion should be applied when doing template matching". Andrei
Apr 02 2018
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/2/18 11:57 AM, Andrei Alexandrescu wrote:
 On 04/02/2018 10:59 AM, ag0aep6g wrote:
 That wouldn't be possible if `innocent` were only head-mutable in the 
 postblit function, instead of fully mutable as it is currently (bug).

 `innocent` would be typed as `immutable(int)[]`, and you could not 
 assign it to the fully mutable `sneaky`.
Problem is we don't have head-mutable in the language.
This is the wrong way to look at it. Head mutable is a specialized concept already implemented in constructors: struct S { int x; int *ptr; int *ptr2; this(int xval, immutable int *ptrval, int *cantuse) immutable { x = xval; // ok, head mutable //x = x + 1; // error, immutable field initialized multiple times ptr = ptrval; // ok // ptr2 = cantuse; // error, can't assign mutable to immutable } } We could have the same mechanism for postblit. Essentially, you should be able to assign immutable fields once, but they shouldn't lose their const protections (note the cantuse example). As was mentioned, because postblit on an immutable (or const) is ONLY allowed for new data, there shouldn't be an issue. -Steve
Apr 02 2018
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/02/2018 12:53 PM, Steven Schveighoffer wrote:
 On 4/2/18 11:57 AM, Andrei Alexandrescu wrote:
 On 04/02/2018 10:59 AM, ag0aep6g wrote:
 That wouldn't be possible if `innocent` were only head-mutable in the 
 postblit function, instead of fully mutable as it is currently (bug).

 `innocent` would be typed as `immutable(int)[]`, and you could not 
 assign it to the fully mutable `sneaky`.
Problem is we don't have head-mutable in the language.
This is the wrong way to look at it. Head mutable is a specialized concept already implemented in constructors: struct S {    int x;    int *ptr;    int *ptr2;    this(int xval, immutable int *ptrval, int *cantuse) immutable    {       x = xval; // ok, head mutable       //x = x + 1; // error, immutable field initialized multiple times       ptr = ptrval; // ok       // ptr2 = cantuse; // error, can't assign mutable to immutable    } }
I've asked Razvan to document how immutable constructors are exactly typechecked. My understanding is that they don't rely on some form of internal "head const" but instead on simple data flow - the first assignment counts as a constructor call. Consider the following example, which contains opaque methods instead of built-ins: struct S { int[] payload; S* another; this(int) immutable; } struct HasS { S member; this(int x) immutable { member = immutable S(1); } } This fails to link; the assignment is just a call to the (declared but undefined) constructor. Such behavior is nice and easy to generalize to copy construction.
 We could have the same mechanism for postblit. Essentially, you should 
 be able to assign immutable fields once, but they shouldn't lose their 
 const protections (note the cantuse example).
Yes, with the distinction that is not "assign" but really "construction with assignment syntax".
 As was mentioned, because postblit on an immutable (or const) is ONLY 
 allowed for new data, there shouldn't be an issue.
The problem with postblit is there's "double construction", one done by the compiler, after which the user may want to assign something else. That's more difficult to typecheck than direct initialization. Andrei
Apr 02 2018
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/2/18 1:08 PM, Andrei Alexandrescu wrote:
 On 04/02/2018 12:53 PM, Steven Schveighoffer wrote:
 As was mentioned, because postblit on an immutable (or const) is ONLY 
 allowed for new data, there shouldn't be an issue.
The problem with postblit is there's "double construction", one done by the compiler, after which the user may want to assign something else. That's more difficult to typecheck than direct initialization.
I was going to argue that it's not full construction -- it's just a copy of bits. But I am incorrect. My understanding was that postblit is called after the bits are copied, but that isn't the case. Currently, the postblit of members is called BEFORE the postblit of the container (or maybe as part of the postblit of the container, but always at the start). This means that a member is not a moved copy at that point, but a fully postblitted item. This also means that it's not safe to assume head-const-ness. This makes some sense -- you don't want to have to deal with member postblits if you don't have to. But it also makes it impossible to intercept copying to do something different (as you would need to do for this case). Indeed, something other than the current postblit mechanism looks more attractive and powerful, even if it isn't as straightforward. -Steve
Apr 02 2018
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 2 Apr 2018 11:57:55 -0400
schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:

 Problem is we don't have head-mutable in the language. Yes, for built-in 
 slices the mechanism is simple - just change qualifier(T[]) to 
 qualifier(T)[]. For a struct S, there is no way to convert from 
 qualifier(S) to tailqualifier(S).
 
 I plan to attack this directly in the DIP - provide a way for structs to 
 express "here's what implicit conversion should be applied when doing 
 template matching".
 
 Andrei
You are hitting a prominent type system flaw here. What may look like a hurdle on the path to fix this(this) is also at the core of getting "shared" into a good shape and probably affects how we will discuss "immutable destructors" and their kin in the future. The question is "How transitive is a qualifier when we strip it top-level on an aggregate?" In https://issues.dlang.org/show_bug.cgi?id=8295 I've been arguing for removing all qualifiers on shallow copies and the case you mentioned where top level qualifiers are stripped for template matching reconfirms me that there is generally some merit to that semantic, that should be explored. Shared structs need elaborate code to be copied, that's for sure. There may be a mutex to be used or values may be copied using atomic loads. The result would be what you dubbed "tailqualifier(S)". I.e. in case of shared, a thread-local copy of the fields that make up the struct. But then it starts to become messy: * Are there cases where we want references contained in the struct to become unshared, too? * If yes, what if these references were marked shared themselves in the struct's definition? * If all fields become unshared, shouldn't the now superfluous mutex be removed from the struct? If so, what started out as a bit blit, now produces a different type entirely. I'm interested to hear more on your thoughs on "tailqualifier(S)". -- Marco
Apr 02 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, April 02, 2018 20:47:31 Marco Leise via Digitalmars-d wrote:
 Am Mon, 2 Apr 2018 11:57:55 -0400

 schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:
 Problem is we don't have head-mutable in the language. Yes, for built-in
 slices the mechanism is simple - just change qualifier(T[]) to
 qualifier(T)[]. For a struct S, there is no way to convert from
 qualifier(S) to tailqualifier(S).

 I plan to attack this directly in the DIP - provide a way for structs to
 express "here's what implicit conversion should be applied when doing
 template matching".

 Andrei
You are hitting a prominent type system flaw here. What may look like a hurdle on the path to fix this(this) is also at the core of getting "shared" into a good shape and probably affects how we will discuss "immutable destructors" and their kin in the future. The question is "How transitive is a qualifier when we strip it top-level on an aggregate?" In https://issues.dlang.org/show_bug.cgi?id=8295 I've been arguing for removing all qualifiers on shallow copies and the case you mentioned where top level qualifiers are stripped for template matching reconfirms me that there is generally some merit to that semantic, that should be explored. Shared structs need elaborate code to be copied, that's for sure. There may be a mutex to be used or values may be copied using atomic loads. The result would be what you dubbed "tailqualifier(S)". I.e. in case of shared, a thread-local copy of the fields that make up the struct. But then it starts to become messy: * Are there cases where we want references contained in the struct to become unshared, too? * If yes, what if these references were marked shared themselves in the struct's definition? * If all fields become unshared, shouldn't the now superfluous mutex be removed from the struct? If so, what started out as a bit blit, now produces a different type entirely. I'm interested to hear more on your thoughs on "tailqualifier(S)".
Copying shared structs is a bit sketchy in general. In theory, we've been trying to make it so non-atomic operations on shared objects aren't legal, because they're not safe unless the object is protected by a mutex. So, the normal way to deal with shared if you want to do anything with it is to protect a section of code with a mutex and temporarily cast away shared within that section of code (if we had synchronized classes, then in some cases, that cast would be automatic, but as it stands, it's always manual). And if doing a non-atomic operation on a shared object isn't legal, then copying a shared object doesn't really make sense. In that case, I would expect that copying a shared object wouldn't be legal (and as such the way to copy a shared object would be to protect that with a mutex and cast away shared when it's copied, and then cast the new object to shared if appropriate). On the other hand, that could get _really_ annoying, and if we were dealing with a copy constructor rather than a postblit constructor, then it would probably be possible to lock a mutex and inside the copy constructor so that the copy and whatever casting was necessary could be done safely inside the struct rather than having to externally protect it with a mutex and cast away shared there. So, I don't know what the answer is, but if we're trying to make operations which aren't guaranteed to be thread-safe illegal on shared objects, then we have a bit of a problem with what to do with copying shared objects (or assigning shared objects). But I suppose that part of the problem is that while the _idea_ of shared is very well defined, the actual details aren't (e.g. a number of non-atomic operations aren't legal on shared objects, but some are). And copying is definitely one area where that has not been properly sorted out yet. - Jonathan M Davis
Apr 02 2018
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/02/2018 02:47 PM, Marco Leise wrote:
 Am Mon, 2 Apr 2018 11:57:55 -0400
 schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:
 
 Problem is we don't have head-mutable in the language. Yes, for built-in
 slices the mechanism is simple - just change qualifier(T[]) to
 qualifier(T)[]. For a struct S, there is no way to convert from
 qualifier(S) to tailqualifier(S).

 I plan to attack this directly in the DIP - provide a way for structs to
 express "here's what implicit conversion should be applied when doing
 template matching".

 Andrei
You are hitting a prominent type system flaw here. What may look like a hurdle on the path to fix this(this) is also at the core of getting "shared" into a good shape and probably affects how we will discuss "immutable destructors" and their kin in the future. The question is "How transitive is a qualifier when we strip it top-level on an aggregate?"
Roger. My hope is to solve that for primitive types, then use that to typecheck constructors and destructors, then use the signatures of (typechecked) constructors and destructors to address composition. Ideally we'd get away without defining another kind of qualifier - tail(const) or whatever. That would complicate the language a great deal. Andrei
Apr 03 2018
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/1/18 8:55 AM, ag0aep6g wrote:
 On 04/01/2018 03:08 AM, Andrei Alexandrescu wrote:
 On 3/31/18 8:32 PM, H. S. Teoh wrote:
[...]
 What exactly is it about this(this) that blocks us from doing that?
See the updated docs. Too many bugs in design and implementation.
 Removing this(this) is going to be a huge breaking change far bigger
 than, say, removing autodecoding ever will be.
We're not removing it as much as evolving it: we define an alternate copying mechanism, and once that is in tip-top shape, we deprecate this(this).
Is there a fundamental flaw in the postblit idea, or are you just going to give postblit a new syntax, and try to avoid all the issues that `this(this)` currently has? If there's a fundamental flaw, I'd be interested in what it is. I can't make it out in your additions to the spec, if it's in there. I can see that `this(this)` is  a mess, but it also looks like a lot could be fixed. For example, how it interacts with const/immutable is ridiculous, but that could probably be fixed. If you're just going for a clean slate, I can see the appeal. You avoid dealing with the hard breakage that fixing `this(this)` would most probably bring.
There's a mix of fundamental flaws and bugs. I'll get to the flaws in a second. About the bugs: people have altered their code in various ways to work with the bizarre semantics of this(this). Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative. I found two fundamental flaws with this(this): 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust. 2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically. There'd be an additional issue - this(this) is non-templated, which requires combinatorial additions when qualifiers are present on the source or destination side. Please note that fixing one or two of these issues doesn't make this(this) viable - I'm mentioning various issues, each of which is a showstopper. Nevertheless knowing them is necessary so we don't make the same mistake again! Andrei
Apr 01 2018
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu 
wrote:
 There's a mix of fundamental flaws and bugs. I'll get to the 
 flaws in a second. About the bugs: people have altered their 
 code in various ways to work with the bizarre semantics of 
 this(this). Now, if we fix various bugs in this(this) by 
 virtually redefining it, then we'll break a lot of code in a 
 lot of ways. To wit, we fixed a small issue and it already 
 created problems: https://github.com/dlang/dmd/pull/8032. That 
 didn't contribute to the decision but is quite illustrative.

 I found two fundamental flaws with this(this):

 1. For immutable objects, typechecking in the presence of 
 successive modifications of data (first assignment by the 
 compiler, then modification by the user) is very difficult if 
 not impossible. I don't know how to do it. The single 
 initialization model (raw/cooked) used currently in regular 
 immutable constructors works reasonably well and is robust.

 2. For shared objects, the part done by the compiler and the 
 part done by this(this) should be synchronized together. This 
 makes it impossible for the user to e.g. define a struct that 
 gets copied atomically.
See my other reply: but why is it necessary to consider the blit logically distinct from the postblit w.r.t to program flow observability? for 1. consider immutable foo = ...; immutable bar = foo; to be immutable foo = ...; immutable bar = () {mutable _ = bitcopy(foo); _.__postblit(); return _;}(); for 2. you would have to synchronize anyway for shared, it makes no difference.
 There'd be an additional issue - this(this) is non-templated, 
 which requires combinatorial additions when qualifiers are 
 present on the source or destination side.
(Perhaps this is what you're referring to, but all you have said so far is "this doesn't work and we need to fix it") the post blit is surely like a destructor: there's only one way to do it, irrespective of the attributes, especially of the intermediate is considered mutable until the end of post blit, like static module constructors initialising global immutables.
 Please note that fixing one or two of these issues doesn't make 
 this(this) viable - I'm mentioning various issues, each of 
 which is a showstopper. Nevertheless knowing them is necessary 
 so we don't make the same mistake again!


 Andrei
I agree that we should fix any type checking bugs that may be present, and that we should strive to not make the same mistake twice, however I wouldn't call either of the above cases a showstopper. You will need to show more of what is broken and why it is broken given the expected breakage.
Apr 01 2018
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/1/18 10:59 AM, Nicholas Wilson wrote:
 On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote:
 There's a mix of fundamental flaws and bugs. I'll get to the flaws in 
 a second. About the bugs: people have altered their code in various 
 ways to work with the bizarre semantics of this(this). Now, if we fix 
 various bugs in this(this) by virtually redefining it, then we'll 
 break a lot of code in a lot of ways. To wit, we fixed a small issue 
 and it already created problems: 
 https://github.com/dlang/dmd/pull/8032. That didn't contribute to the 
 decision but is quite illustrative.

 I found two fundamental flaws with this(this):

 1. For immutable objects, typechecking in the presence of successive 
 modifications of data (first assignment by the compiler, then 
 modification by the user) is very difficult if not impossible. I don't 
 know how to do it. The single initialization model (raw/cooked) used 
 currently in regular immutable constructors works reasonably well and 
 is robust.

 2. For shared objects, the part done by the compiler and the part done 
 by this(this) should be synchronized together. This makes it 
 impossible for the user to e.g. define a struct that gets copied 
 atomically.
See my other reply: but why is it necessary to consider the blit logically distinct from the postblit w.r.t to program flow observability? for 1. consider immutable foo = ...; immutable bar = foo; to be immutable foo = ...; immutable bar = () {mutable _ = bitcopy(foo); _.__postblit(); return _;}();
Negative. The problem is typechecking postblit itself, not its invocation.
 for 2. you would have to synchronize anyway for shared, it makes no 
 difference.
Negative. Consider: shared struct Point { private long x, y, z; private Mutex mutex; ... } Task: define the copy primitive of Point so atomically copy x, y, and z using mutex. The problem is, the compiler will memcpy the three longs non-atomically before the user even gets a crack at intercepting the operation.
 There'd be an additional issue - this(this) is non-templated, which 
 requires combinatorial additions when qualifiers are present on the 
 source or destination side.
(Perhaps this is what you're referring to, but all you have said so far is "this doesn't work and we need to fix it") the post blit is surely like a destructor: there's only one way to do it, irrespective of the attributes, especially of the intermediate is considered mutable until the end of post blit, like static module constructors initialising global immutables.
Negative. Ignoring qualifiers during copying opens holes in the type system the size of China. Or at least Australia as it were :o). Consider: int[] sneaky; struct A { private int[] innocent; this(this) { sneaky = innocent; } } void main() { immutable a = A([1, 2, 3]); auto b = a; sneaky[1] = 42; // oops import std.stdio; writeln(a.innocent); // ooooops } Sadly this (and many similar ones) compiles and runs warning-free on today's compiler. We really need to close this loop, like, five years ago.
 I agree that we should fix any type checking bugs that may be present, 
 and that we should strive to not make the same mistake twice, however I 
 wouldn't call either of the above cases a showstopper. You will need to 
 show more of what is broken and why it is broken given the expected 
 breakage.
Such discussions will be indeed present in the DIP. Andrei
Apr 01 2018
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 1 April 2018 at 17:08:37 UTC, Andrei Alexandrescu 
wrote:
 On 4/1/18 10:59 AM, Nicholas Wilson wrote:
 for 1. consider
 immutable foo = ...;
 immutable bar = foo;
 to be
 immutable foo = ...;
 immutable bar = () {mutable _ = bitcopy(foo); _.__postblit(); 
 return _;}();
Negative. The problem is typechecking postblit itself, not its invocation.
Nit sure that I follow but, see comment below on your immutable example.
 for 2. you would have to synchronize anyway for shared, it 
 makes no difference.
Negative. Consider: shared struct Point { private long x, y, z; private Mutex mutex; ... } Task: define the copy primitive of Point so atomically copy x, y, and z using mutex. The problem is, the compiler will memcpy the three longs non-atomically before the user even gets a crack at intercepting the operation.
What I meant was: Point p = ...; auto pp = p; // bit copy + postblit = WRONG Point pp; synchronized (p.mutex) { pp = p; } // synchronised: bitcopy is inseparable from the postblit Perhaps there is a way to make that automatic / nicer to use?
 [...]
 int[] sneaky;
 struct A
 {
     private int[] innocent;
     this(this)
     {
         sneaky = innocent;
     }
 }
 void main()
 {
     immutable a = A([1, 2, 3]);
     auto b = a;
     sneaky[1] = 42; // oops
     import std.stdio;
     writeln(a.innocent); // ooooops
 }

 Sadly this (and many similar ones) compiles and runs 
 warning-free on today's compiler. We really need to close this 
 loop, like, five years ago.
How much of this class of bug would be eliminated by requiring that `this(this)` be pure for assignment to const and immutable objects? Arguably this(this) should always be pure in any sane program. The only reason I can think of is if you're trying to perf the number of copies you're making, but there is compiler help for that.
Apr 01 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, April 02, 2018 00:25:52 Nicholas Wilson via Digitalmars-d wrote:
 On Sunday, 1 April 2018 at 17:08:37 UTC, Andrei Alexandrescu
 wrote:
 On 4/1/18 10:59 AM, Nicholas Wilson wrote:
 [...]
 int[] sneaky;
 struct A
 {

     private int[] innocent;
     this(this)
     {

         sneaky = innocent;

     }

 }
 void main()
 {

     immutable a = A([1, 2, 3]);
     auto b = a;
     sneaky[1] = 42; // oops
     import std.stdio;
     writeln(a.innocent); // ooooops

 }

 Sadly this (and many similar ones) compiles and runs
 warning-free on today's compiler. We really need to close this
 loop, like, five years ago.
How much of this class of bug would be eliminated by requiring that `this(this)` be pure for assignment to const and immutable objects? Arguably this(this) should always be pure in any sane program. The only reason I can think of is if you're trying to perf the number of copies you're making, but there is compiler help for that.
All kinds of things could be done with a postlbit costructor that don't actually involve copying. These include logging or printing something, and they could include stuff like reference counting, which may or may not need access to something external. debug statements would solve some uses cases but not all. Requiring that this(this) be pure has some of the same issues that requiring opEquals to be pure or const or whatever has. It makes sense in _most_ cases, but occasionally, there are good reasons for it not to be - especially in a systems language. We have to be _very_ careful about requiring any particular attribute much of anywhere. Doing so typically causes problems - e.g. those we have with Object's opEquals, opCmp, toHash, and toString. That decision really needs to be left up to a particular code base. It's also a big part of why templates infer attributes. Someone can write their code in such a way that their code base requires a particular attribute, but in general, language features shouldn't be requiring any specific attributes, or we'll just be backing ourselves into another corner. - Jonathan M Davis
Apr 01 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 2 April 2018 at 00:44:14 UTC, Jonathan M Davis wrote:
 On Monday, April 02, 2018 00:25:52 Nicholas Wilson via 
 Digitalmars-d wrote:
 On Sunday, 1 April 2018 at 17:08:37 UTC, Andrei Alexandrescu 
 wrote:
 On 4/1/18 10:59 AM, Nicholas Wilson wrote:
 [...]
 int[] sneaky;
 struct A
 {

     private int[] innocent;
     this(this)
     {

         sneaky = innocent;

     }

 }
 void main()
 {

     immutable a = A([1, 2, 3]);
     auto b = a;
     sneaky[1] = 42; // oops
     import std.stdio;
     writeln(a.innocent); // ooooops

 }

 Sadly this (and many similar ones) compiles and runs 
 warning-free on today's compiler. We really need to close 
 this loop, like, five years ago.
How much of this class of bug would be eliminated by requiring that `this(this)` be pure for assignment to const and immutable objects? Arguably this(this) should always be pure in any sane program. The only reason I can think of is if you're trying to perf the number of copies you're making, but there is compiler help for that.
All kinds of things could be done with a postlbit costructor that don't actually involve copying. These include logging or printing something, and they could include stuff like reference counting, which may or may not need access to something external. debug statements would solve some uses cases but not all. Requiring that this(this) be pure has some of the same issues that requiring opEquals to be pure or const or whatever has. It makes sense in _most_ cases, but occasionally, there are good reasons for it not to be - especially in a systems language.
I wasn't suggesting this a global requirement of all this(this)s, only to the _postblit assignment to const and immutable objects_ ( where being pure would to disallow the above bug). Note that this should still be able to be worked around by cast()s (which are un safe) and therefore require trusted to work in safe code, in which case the programmer has presumably thought about the situation and knows what he's doing.
Apr 02 2018
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu 
wrote:
 Now, if we fix various bugs in this(this) by virtually 
 redefining it, then we'll break a lot of code in a lot of ways. 
 To wit, we fixed a small issue and it already created problems: 
 https://github.com/dlang/dmd/pull/8032. That didn't contribute 
 to the decision but is quite illustrative.
Yeah, I absolutely see value in starting fresh with different syntax, even if you were just implementing the same postblit idea again.
 I found two fundamental flaws with this(this):

 1. For immutable objects, typechecking in the presence of 
 successive modifications of data (first assignment by the 
 compiler, then modification by the user) is very difficult if 
 not impossible. I don't know how to do it. The single 
 initialization model (raw/cooked) used currently in regular 
 immutable constructors works reasonably well and is robust.
I'd think that just letting the const/immutable postblit function see head-mutable fields would work. But maybe that's way harder to implement than writing it down the forum. If that's so, then fair enough. I know that I won't be able to implement it.
 2. For shared objects, the part done by the compiler and the 
 part done by this(this) should be synchronized together. This 
 makes it impossible for the user to e.g. define a struct that 
 gets copied atomically.
Interesting. I've got no armchair expertise on this one.
 There'd be an additional issue - this(this) is non-templated, 
 which requires combinatorial additions when qualifiers are 
 present on the source or destination side.
I think I don't understand this one. Could you give an example in code? Are you saying that we'd need to define all these: this(this) this(this) const this(this) immutable even if they do the same thing? Wouldn't `this(this) inout` take care of this?
Apr 01 2018
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu 
wrote:
 1. For immutable objects, typechecking in the presence of 
 successive modifications of data (first assignment by the 
 compiler, then modification by the user) is very difficult if 
 not impossible. I don't know how to do it. The single 
 initialization model (raw/cooked) used currently in regular 
 immutable constructors works reasonably well and is robust.
Do the same as in const constructor.
 2. For shared objects, the part done by the compiler and the 
 part done by this(this) should be synchronized together. This 
 makes it impossible for the user to e.g. define a struct that 
 gets copied atomically.
If you mean reference counting, the easy solution with copy constructor is bad performance wise. Good solution can be done architecturally now and doesn't rely on copy constructor.
Apr 02 2018
parent reply ag0aep6g <anonymous example.com> writes:
On Monday, 2 April 2018 at 14:01:22 UTC, Kagamin wrote:
 On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu 
 wrote:
 1. For immutable objects, typechecking in the presence of 
 successive modifications of data (first assignment by the 
 compiler, then modification by the user) is very difficult if 
 not impossible. I don't know how to do it. The single 
 initialization model (raw/cooked) used currently in regular 
 immutable constructors works reasonably well and is robust.
Do the same as in const constructor.
The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment. In a postblit function, we can't say it's initialization, because the field already has a value that can't be ignored.
Apr 02 2018
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/02/2018 10:42 AM, ag0aep6g wrote:
 On Monday, 2 April 2018 at 14:01:22 UTC, Kagamin wrote:
 On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote:
 1. For immutable objects, typechecking in the presence of successive 
 modifications of data (first assignment by the compiler, then 
 modification by the user) is very difficult if not impossible. I 
 don't know how to do it. The single initialization model (raw/cooked) 
 used currently in regular immutable constructors works reasonably 
 well and is robust.
Do the same as in const constructor.
The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment.
Affirmative. First assignment is a call to the member's constructor. We typecheck that reasonably well already in qualified constructors, and it's the most promising approach for the DIP.
 In a postblit function, we can't say it's initialization, because the 
 field already has a value that can't be ignored.
Affirmative. That's what makes it so darn difficult to typecheck. If we don't let the compiler do the initial blitting (and instead start with T.init), the copy ctor is typechecked exactly like the regular ctor. Andrei
Apr 02 2018
prev sibling parent reply Kagamin <spam here.lot> writes:
On Monday, 2 April 2018 at 14:42:17 UTC, ag0aep6g wrote:
 The way it works in a const constructor is that `this.foo = 
 bar;` is considered initialization, not assignment.
Do you mean the spec? Andrei complained about implementation. Const constructors are already implemented as needed for postblit.
 In a postblit function, we can't say it's initialization, 
 because the field already has a value that can't be ignored.
Fields are initialized before const constructor too.
Apr 03 2018
parent reply ag0aep6g <anonymous example.com> writes:
On 04/03/2018 09:39 AM, Kagamin wrote:
 On Monday, 2 April 2018 at 14:42:17 UTC, ag0aep6g wrote:
 The way it works in a const constructor is that `this.foo = bar;` is 
 considered initialization, not assignment.
Do you mean the spec? Andrei complained about implementation.
Andrei complained about both, no? His words: "Too many bugs in design and implementation." Design = spec. I'm also talking about the implementation. `this.foo = bar;` in a constructor is at least not normal assignment. If foo has opAssign, that won't be called.
 Const constructors are already implemented as needed for postblit.
Maybe. But we can't explain the special assignment semantics with it being initialization. Or can we? For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost. We can do the same with the postblit function: First assignment is actually initialization. When the compiler sees that the postblit function initializes a field, it can skip that field when blitting. But it can also just blit the whole struct, because it doesn't matter if the value just gets overwritten. In other words, a postblit function can either: 1) use the blitted value as a starting point, like a constructor can use the .init value, or it can 2) initialize the field itself. Would make perfect sense to me.
Apr 03 2018
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/03/2018 07:36 AM, ag0aep6g wrote:
 For constructors, we say that the first assignment is actually 
 initialization. The compiler might or might not put the .init value down 
 before calling the constructor. Doesn't matter, because the constructor 
 will overwrite it anyway, and nothing of value is lost.
What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.
 We can do the same with the postblit function: First assignment is 
 actually initialization. When the compiler sees that the postblit 
 function initializes a field, it can skip that field when blitting.
What if the user code reads the value? * Often people use this(this) to bump a reference count a la "if (pcnt) ++*pcnt;" * People may pass the field by reference to an opaque function. What type does the field have?
 But 
 it can also just blit the whole struct, because it doesn't matter if the 
 value just gets overwritten.
 
 In other words, a postblit function can either:
 
 1) use the blitted value as a starting point, like a constructor can use 
 the .init value, or it can
 2) initialize the field itself.
 
 Would make perfect sense to me.
In case (1) things can get quite confusing. Inside a postblit, field = TypeOfField(100); is a call to the constructor, whereas field = TypeOfField(field.x + 100); is a call to the assignment operator. Andrei
Apr 03 2018
parent reply ag0aep6g <anonymous example.com> writes:
On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu 
wrote:
 On 04/03/2018 07:36 AM, ag0aep6g wrote:
 For constructors, we say that the first assignment is actually 
 initialization. The compiler might or might not put the .init 
 value down before calling the constructor. Doesn't matter, 
 because the constructor will overwrite it anyway, and nothing 
 of value is lost.
What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.
So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want. [...]
 What if the user code reads the value?

 * Often people use this(this) to bump a reference count a la 
 "if (pcnt) ++*pcnt;"
Because of the indirection, that can only be done in a mutable `this(this)`. Otherwise you violate the const/immutable guarantee of the original object.
 * People may pass the field by reference to an opaque function. 
 What type does the field have?
Fully const. Same as in a constructor. [...]
 In case (1) things can get quite confusing. Inside a postblit,

 field = TypeOfField(100);

 is a call to the constructor, whereas

 field = TypeOfField(field.x + 100);

 is a call to the assignment operator.
A const constructor currently accepts both of those. So the second one can apparently be considered "initialization" as well? Or should a const constructor not be allowed to do that?
Apr 03 2018
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/3/18 10:21 AM, ag0aep6g wrote:
 On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote:
 On 04/03/2018 07:36 AM, ag0aep6g wrote:
 For constructors, we say that the first assignment is actually 
 initialization. The compiler might or might not put the .init value 
 down before calling the constructor. Doesn't matter, because the 
 constructor will overwrite it anyway, and nothing of value is lost.
What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.
So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want.
Unfortunately, I found out that it's not just "pre-filled with some values". Member postblits are run before the containing postblit. https://run.dlang.io/is/mt6eGa So this means, the data that is available to the postblit has already been processed. It would only make sense to allow const postblits to have the same constructor mechanism if the members all had no postblits. -Steve
Apr 03 2018
parent reply ag0aep6g <anonymous example.com> writes:
On 04/03/2018 05:13 PM, Steven Schveighoffer wrote:
 Unfortunately, I found out that it's not just "pre-filled with some 
 values". Member postblits are run before the containing postblit.
 
 https://run.dlang.io/is/mt6eGa
 
 So this means, the data that is available to the postblit has already 
 been processed.
There's a similar situation with constructors: A constructor can call another constructor, which can lead to double initialization of fields. Example: ---- class C { int x; this() immutable { this(42); /* Initializes x. */ x = 13; /* Breaking immutable, or ok? */ } this(int x) immutable { this.x = x; } } ---- If there's a problem with running two postblits on the same field, then I think constructors probably have similar issue. I'm having a hard time finding a good example, though. One where we could break immutable in an obvious way or some such.
 It would only make sense to allow const postblits to have the same 
 constructor mechanism if the members all had no postblits.
Less drastically, we could also disallow writing to those fields that have their own postblit. The analog with constructors would be disallowing writing to fields that have already been initialized by an implicit `super` call. Actually, that seems to be how it works, kinda-sorta: ---- class C { int x; } class D : C { int y; this() immutable { y = 1; /* accepted */ x = 2; /* Error: cannot modify this.x in immutable function */ } } ---- Obviously, DMD doesn't check what C's constructor actually does (because it generally can't check that). It just considers initializing x to be C's responsibility.
Apr 03 2018
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/3/18 4:26 PM, ag0aep6g wrote:
 On 04/03/2018 05:13 PM, Steven Schveighoffer wrote:
 Unfortunately, I found out that it's not just "pre-filled with some 
 values". Member postblits are run before the containing postblit.

 https://run.dlang.io/is/mt6eGa

 So this means, the data that is available to the postblit has already 
 been processed.
There's a similar situation with constructors: A constructor can call another constructor, which can lead to double initialization of fields. Example: ---- class C {     int x;     this() immutable     {         this(42); /* Initializes x. */         x = 13; /* Breaking immutable, or ok? */
IMO, breaking immutable.
      }
      this(int x) immutable
      {
          this.x = x;
      }
 }
 ----
 
 If there's a problem with running two postblits on the same field, then 
 I think constructors probably have similar issue. I'm having a hard time 
 finding a good example, though. One where we could break immutable in an 
 obvious way or some such.
You may NOT want to run a postblit on the member. If all you are going to do, for example, is reassign a variable, then running the postblit, and then the destructor, just so you can overwrite it is pointless. But more importantly, if the postblit of the member does something crazy like stores a reference to itself as an immutable elsewhere, and then the compiler allows overwriting, we now have problems. I think the better mechanism for immutable copying would be to have a copy constructor that starts off with T.init, and is passed the object to copy from. That seems to be a direction Andrei is considering. -Steve
Apr 03 2018
parent reply ag0aep6g <anonymous example.com> writes:
On 04/03/2018 10:51 PM, Steven Schveighoffer wrote:
 On 4/3/18 4:26 PM, ag0aep6g wrote:
[...]
 If there's a problem with running two postblits on the same field, 
 then I think constructors probably have similar issue. I'm having a 
 hard time finding a good example, though. One where we could break 
 immutable in an obvious way or some such.
You may NOT want to run a postblit on the member. If all you are going to do, for example, is reassign a variable, then running the postblit, and then the destructor, just so you can overwrite it is pointless.
Same with class constructors: You may not want to run `super` when you're just going to overwrite what it did. But the language doesn't give you a choice. It'll be called one way or another. I'm not saying that imitating how constructors work will make the best possible copying mechanism. Something else might be superior in every way. It's just that so far the arguments against a constructor-like postblit also seem to apply to constructors as they are implemented. So I'm thinking that a postblit modeled after constructors could work as well as they do. But maybe the real takeaway is that constructors don't work very well, and shouldn't be imitated.
 But more importantly, if the postblit of the member does something crazy 
 like stores a reference to itself as an immutable elsewhere, and then 
 the compiler allows overwriting, we now have problems.
I'd love to see an example of this in code. The best I can come up with would be something like this (doesn't compile): ---- import std.stdio; immutable(int)* p; struct S { int x; this(this) immutable { x = 42; /* First write. */ .p = &this.x; writeln(p, " ", *p); /* Prints some address and 42. */ } } struct T { S s; this(this) immutable { s = S(13); /* Second write. Breaking immutable? */ writeln(p, " ", *p); /* Same address, but 13. */ } } void main() { immutable T foo; immutable bar = foo; } ---- But that's essentially the same as the class example I posted. `*p` would only change values during the postblit run. Just like a constructor chain can write to the same field multiple times. That's kinda iffy, but I can't find a way to demonstrate some real, obvious damage.
 I think the better mechanism for immutable copying would be to have a 
 copy constructor that starts off with T.init, and is passed the object 
 to copy from. That seems to be a direction Andrei is considering.
No objection from me. If Andrei et al. can find a better solution, great.
Apr 03 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/3/18 5:44 PM, ag0aep6g wrote:
 On 04/03/2018 10:51 PM, Steven Schveighoffer wrote:
 On 4/3/18 4:26 PM, ag0aep6g wrote:
[...]
 If there's a problem with running two postblits on the same field, 
 then I think constructors probably have similar issue. I'm having a 
 hard time finding a good example, though. One where we could break 
 immutable in an obvious way or some such.
You may NOT want to run a postblit on the member. If all you are going to do, for example, is reassign a variable, then running the postblit, and then the destructor, just so you can overwrite it is pointless.
Same with class constructors: You may not want to run `super` when you're just going to overwrite what it did. But the language doesn't give you a choice. It'll be called one way or another.
At least you can invoke the one you want, with postblit there is only one "choice". But this is a red herring -- we already have struct constructors, and that requirement of invoking constructors on members is not present. With structs, we have the possibility of initialization via different mechanisms: constructor, postblit, .init. All of these are supported by the struct member, but currently you can only invoke postblit if you are in a postblit. And only at the beginning. I would like to see more flexibility for copying.
 I'm not saying that imitating how constructors work will make the best 
 possible copying mechanism. Something else might be superior in every 
 way. It's just that so far the arguments against a constructor-like 
 postblit also seem to apply to constructors as they are implemented.
For structs, using .init is a valid initialization, so it's completely different from classes, where a constructor MUST be invoked. Indeed, there is no mechanism to require calling struct member constructors in the owner's ctor. Stop thinking class constructors, and think struct constructors instead.
 But more importantly, if the postblit of the member does something 
 crazy like stores a reference to itself as an immutable elsewhere, and 
 then the compiler allows overwriting, we now have problems.
I'd love to see an example of this in code. The best I can come up with would be something like this (doesn't compile): ---- import std.stdio; immutable(int)* p; struct S {     int x;     this(this) immutable     {         x = 42; /* First write. */         .p = &this.x;         writeln(p, " ", *p); /* Prints some address and 42. */     } } struct T {     S s;     this(this) immutable     {         s = S(13); /* Second write. Breaking immutable? */
Of course this doesn't compile, because s is considered immutable by now. What I was saying is that we can't allow postblit to modify data that has already been postblitted, because of the reason this example is showing.
          writeln(p, " ", *p); /* Same address, but 13. */
      }
 }
 
 void main()
 {
      immutable T foo;
      immutable bar = foo;
 }
 ----
 
 But that's essentially the same as the class example I posted. `*p` 
 would only change values during the postblit run. Just like a 
 constructor chain can write to the same field multiple times.
I don't think you should be able to write to the same field multiple times in an immutable/const constructor. If so, that's a bug.
 That's kinda iffy, but I can't find a way to demonstrate some real, 
 obvious damage.
Any place where an immutable can be observed to change between two reads is breaking immutable. -Steve
Apr 04 2018
next sibling parent ag0aep6g <anonymous example.com> writes:
On 04/04/2018 12:37 PM, Steven Schveighoffer wrote:
 With structs, we have the possibility of initialization via different 
 mechanisms: constructor, postblit, .init. All of these are supported by 
 the struct member, but currently you can only invoke postblit if you are 
 in a postblit. And only at the beginning. I would like to see more 
 flexibility for copying.
[...]
 For structs, using .init is a valid initialization, so it's completely
 different from classes, where a constructor MUST be invoked. Indeed,
 there is no mechanism to require calling struct member constructors in 
 the owner's ctor.
[...] To paraphrase your point: We should be able to initialize a member with .init or by calling its constructor (over .init). We should not be forced to initialize it with its postblit. Makes sense. But I see one tricky case: Can I also choose to use the blitted value of the member without calling its postblit? I'd say that can't be allowed. So the compiler would have to identify that case and reject it. And now we're in uncharted territory. As far as I see, this is not something that constructors do (struct or class). They happily accept any .init value. But postblits can't accept any blitted value. So we can't say: "Just do what constructors do." We have to come up with new rules. But we don't want to come up with new rules, we want to say: "Just do what constructors do." So we require the member postblit, and we say that the outer postblit operates on the resulting value like a constructor operates on .init. And then we see that it breaks the type system, and that any analog behavior of constructors just means that constructors/.init are broken, too. [...]
 struct S
 {
      int x;
      this(this) immutable
      {
          x = 42; /* First write. */
[...]
      }
 }

 struct T
 {
      S s;
      this(this) immutable
      {
          s = S(13); /* Second write. Breaking immutable? */
Of course this doesn't compile, because s is considered immutable by now.
It doesn't compile, because this(this) is a broken mess currently. The first write doesn't compile either. Obviously, with a properly implemented immutable this(this), you would at least be allowed to write once. With mutable(!) `this(this)`s, both writes are accepted.
 What I was saying is that we can't allow postblit to modify data 
 that has already been postblitted, because of the reason this example is 
 showing.
Still, I'd like to have a more explosive example. The breakage here is very localized, and could possibly be defined away somehow. Like: "Unique immutable data is considered 'raw' in constructors and postblit functions. That means, the data can be mutated. Only when the constructor/postblit returns does it become 'cooked' and truly immutable." [...]
 ----
[...]
 I don't think you should be able to write to the same field multiple 
 times in an immutable/const constructor. If so, that's a bug.
Are you counting .init as a write, or is a constructor allowed to build on that? ---- import std.stdio; struct S { int x = 1; int y = 3; this(int dummy) immutable { writeln(x); /* 1 */ ++x; /* Breaking immutable? */ writeln(x); /* 2 */ ++y; /* Breaking immutable even though y hasn't been printed yet? */ writeln(y); /* 4 */ } } void main() { auto s = immutable S(0); } ---- I think those increments could be considered breaking immutable. Maybe they must be. If this were to be outlawed, I'd agree a constructors and postblits are fundamentally different.
Apr 04 2018
prev sibling next sibling parent Kagamin <spam here.lot> writes:
On Wednesday, 4 April 2018 at 10:37:57 UTC, Steven Schveighoffer 
wrote:
 I would like to see more flexibility for copying.
It's a tradeoff between control and ergonomics.
 import std.stdio;
 
 immutable(int)* p;
 
 struct S
 {
      int x;
      this(this) immutable
      {
          x = 42; /* First write. */
          .p = &this.x;
          writeln(p, " ", *p); /* Prints some address and 42. */
      }
 }
 
 struct T
 {
      S s;
      this(this) immutable
      {
          s = S(13); /* Second write. Breaking immutable? */
Of course this doesn't compile, because s is considered immutable by now. What I was saying is that we can't allow postblit to modify data that has already been postblitted, because of the reason this example is showing.
Assignment ends lifetime of the previous value, it will be just a dangling pointer. AFAIK this is how reassignment works wrt destructors: { immutable tmp=S(13); s.__dtor(); //if any s=tmp; //move into immutable storage }
 I don't think you should be able to write to the same field 
 multiple times in an immutable/const constructor. If so, that's 
 a bug.
Stores to const fields are special, so it should be possible to track them in postblits too.
Apr 12 2018
prev sibling parent Kagamin <spam here.lot> writes:
On Wednesday, 4 April 2018 at 10:37:57 UTC, Steven Schveighoffer 
wrote:
 Any place where an immutable can be observed to change between 
 two reads is breaking immutable.
That happens in constructor too.
Apr 12 2018
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/03/2018 04:26 PM, ag0aep6g wrote:
 On 04/03/2018 05:13 PM, Steven Schveighoffer wrote:
 Unfortunately, I found out that it's not just "pre-filled with some 
 values". Member postblits are run before the containing postblit.

 https://run.dlang.io/is/mt6eGa

 So this means, the data that is available to the postblit has already 
 been processed.
There's a similar situation with constructors: A constructor can call another constructor, which can lead to double initialization of fields. Example: ---- class C {     int x;     this() immutable     {         this(42); /* Initializes x. */         x = 13; /* Breaking immutable, or ok? */     }     this(int x) immutable     {         this.x = x;     } } ----
Let's replace "int" with an UDT: struct S { int x = -1; this(int y) immutable { x = y; } void opAssign(int) immutable; } class C { S x; this() immutable { this(42); /* Initializes x. */ x = 13; /* Breaking immutable, or ok? */ } this(int x) immutable { this.x = x; } } This code compiles, and calls the constructor twice for the same object. Clearly that shouldn't be allowed to pass. I've submitted https://issues.dlang.org/show_bug.cgi?id=18719 - thanks! (The problem seems to occur even without immutable, it's endemic to forwarding constructors.) Andre
Apr 03 2018
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/03/2018 10:21 AM, ag0aep6g wrote:
 On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote:
 On 04/03/2018 07:36 AM, ag0aep6g wrote:
 For constructors, we say that the first assignment is actually 
 initialization. The compiler might or might not put the .init value 
 down before calling the constructor. Doesn't matter, because the 
 constructor will overwrite it anyway, and nothing of value is lost.
What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.
So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want.
Well... not really. This is because .init is really an inert state - null indirections, no state allocated etc. Makes typechecking easy, and calling constructor on top of .init is what happens already. In contrast, the postblit situash is very different - the fields already contain "interesting" data, allocated resources etc. Calling a constructor on top of that is not defined.
Apr 03 2018
parent reply ag0aep6g <anonymous example.com> writes:
On 04/03/2018 08:57 PM, Andrei Alexandrescu wrote:
 Well... not really. This is because .init is really an inert state - 
 null indirections, no state allocated etc.
.init can have non-null indirections: struct S { int[] a = [1, 2, 3]; } static assert(S.init.a.ptr !is null); /* passes */ But maybe it shouldn't. It leads to problems with immutable, because the same .init with the same `a` is used for both mutable and immutable objects. https://issues.dlang.org/show_bug.cgi?id=10376 As far as I see, a const/immutable postblit working like a constructor wouldn't have that particular problem, because the copy is always made from another const/immutable object, so there wouldn't be aliasing with mutable data. But there might of course be other problems with indirections, which I don't see right now. I'd love to see any examples, though. They probably reveal weaknesses in .init/constructors, too.
 Makes typechecking easy, and 
 calling constructor on top of .init is what happens already. In 
 contrast, the postblit situash is very different - the fields already 
 contain "interesting" data, allocated resources etc. Calling a 
 constructor on top of that is not defined.
Well, the idea would be to define it. But if postblit goes away for other reasons anyway (like the atomic copy thing, or another mechanism being simply superior), then there's no point in pursuing this, of course.
Apr 03 2018
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/03/2018 04:29 PM, ag0aep6g wrote:
 But if postblit goes away for other reasons anyway (like the atomic copy 
 thing, or another mechanism being simply superior), then there's no 
 point in pursuing this, of course.
The DIP will definitely need to make a solid case supporting whatever it proposes.
Apr 03 2018
prev sibling parent Kagamin <spam here.lot> writes:
On Tuesday, 3 April 2018 at 11:36:27 UTC, ag0aep6g wrote:
 Maybe. But we can't explain the special assignment semantics 
 with it being initialization.

 Or can we?
Postblit is a part of initialization of a copy. Until postblit completes the object is in invalid state, same as in constructor.
Apr 05 2018
prev sibling parent Kagamin <spam here.lot> writes:
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu 
wrote:
 There's a mix of fundamental flaws and bugs. I'll get to the 
 flaws in a second. About the bugs: people have altered their 
 code in various ways to work with the bizarre semantics of 
 this(this). Now, if we fix various bugs in this(this) by 
 virtually redefining it, then we'll break a lot of code in a 
 lot of ways. To wit, we fixed a small issue and it already 
 created problems: https://github.com/dlang/dmd/pull/8032. That 
 didn't contribute to the decision but is quite illustrative.
Since poorly thought decisions keep being made for the language it's doubtful that "no breakage" policy has real backing.
Apr 02 2018
prev sibling parent reply Shachar Shemesh <shachar weka.io> writes:
On 01/04/18 03:32, H. S. Teoh wrote:
 The one nagging question I've been having about pure is: how much are we
 actually taking advantage of the guarantees provided by pure?
My problem is that pure do not provide many guarantees.
  We have
 developed very clever ways of extending the traditional definition of
 pure and invented creative ways of making more things pure, which is all
 great.
Can anyone explain to me what good are the guarantees provided by a function that is pure but not strictly pure? I couldn't find them.
  But AFAIK the only place where it's actually taken advantage of
 is to elide some redundant function calls inside a single expression.
You cannot even do that unless the function is strictly pure. For all D's extension of the pure concept, it weakened, rather than enhanced, what it means.
 And perhaps infer uniqueness in some cases for implicit casting to
 immutable.
Can you expand on that one? Shachar
Apr 01 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, April 02, 2018 09:56:19 Shachar Shemesh via Digitalmars-d wrote:
 On 01/04/18 03:32, H. S. Teoh wrote:
 The one nagging question I've been having about pure is: how much are we
 actually taking advantage of the guarantees provided by pure?
My problem is that pure do not provide many guarantees.
  We have

 developed very clever ways of extending the traditional definition of
 pure and invented creative ways of making more things pure, which is all
 great.
Can anyone explain to me what good are the guarantees provided by a function that is pure but not strictly pure? I couldn't find them.
Honestly, I think at this point pure is easier to understand if you think of it as noglobal and don't think about functional purity at all. What pure does is make it so that the function cannot access global, mutable state except through its arguments. So, all that it's working with (except for constants such as enums or immutable, module-level variables) is what it's given via its arguments. As such, the primary benefit of pure is that you know that no global state is being mucked with unless it was passed to the function as an argument. _That_ is the guarantee that pure provides. Everything else about pure is just built on top of that guarantee and what the compiler can infer from it. If the function is "strongly" pure (i.e. its parameters are immutable or implicitly convertible to immutable) then the compiler knows that if the function is called multiple times with the exact same arguments, then it knows that the result will be the same each time (though it does have to take into account the fact that each call could return a newly allocated object - they'd just be equivalent objects every time). So, when you're dealing with a strongly pure function, you're then dealing with actual, functional purity, and the compiler can choose to optimize calls such as foo(5) * foo(5) so that foo(5) is only called once. Weakly pure functions (i.e. pure functions that aren't strongly pure) don't have those same optimization benefits, because calling them could result in the arguments being mutated, but because weakly pure functions don't access global, mutable state, the compiler can safely call them from a strongly pure function without violating the guarantee that multiple calls with the same arguments to a strongly pure function are supposed to give the same result. Most pure functions are weakly pure (e.g. unless a pure member function is immutable, then it's weakly pure), so the optimization benefits are pretty minimal in most cases. And even if a function is strongly pure, the optimization is only done within an expression (or maybe statement - I can never remember which), because going beyond that would require data flow analysis, which the compiler rarely does. As such, pretty much the only time you end up with function calls being elided thanks to pure is when you have a strongly pure function where you do something like foo(5) * foo(5). So, the optimization benefits of pure are pretty minimal. It's that guarantee about not touching global, mutable state which is the main benefit. As such, if we were adding pure now, I would strongly argue for calling it something like noglobal. I think that it would reduce the confusion considerably. As it stands, while it helps make functional purity possible in some cases, it ultimately doesn't have much to do with functional purity in spite of its name.
 And perhaps infer uniqueness in some cases for implicit casting to
 immutable.
Can you expand on that one?
int[] foo() { return [1, 2, 3, 4, 5]; } void main() { immutable arr = foo(); } does not compile, because the return value is mutable, and you can't implicitly convert int[] to immutable int[]. However, int[] foo() pure { return [1, 2, 3, 4, 5]; } void main() { immutable arr = foo(); } compiles just fine. Because foo is pure, and the compiler knows that the return value could not possibly have come from the function's arguments, it knows that the return value is unique and that it won't violate the type system to implicitly cast it to immutable. As such, you can write a function as complicated as you want to create the return value, and so long as the function is pure, and the compiler can determine that the return value did not come via an argument, you can implicitly convert the return value to immutable rather than having to use something like std.exception.assumeUnique or an explicit cast, which relies on the programmer verifying that the object being cast is indeed unique rather than having the compiler guarantee it. Whether the implicit cast is allowed ultimately depends on the types of the pure function's parameters and the actual arguments, so it's not always obvious whether it will work or not, but in general, it works very well for initializing complex, immutable objects without having to rely on getting casts right. e.g. this still compiles int[] foo(int[] a) pure { return [1, 2, 3, 4, 5]; } void main() { immutable arr = foo([1, 2]); } because the compiler can see that the argument being passed is unique and that therefore the return value is unique whether it came from the argument or not, whereas this doesn't compile int[] foo(int[] a) pure { return [1, 2, 3, 4, 5]; } void main() { int[] a = [1, 2]; immutable arr = foo(a); } because the compiler can't guarantee that the return value isn't a slice of the argument (at least not without looking at the implementation, but all the compiler looks at is the signature). - Jonathan M Davis
Apr 02 2018
parent reply Shachar Shemesh <shachar weka.io> writes:
On 02/04/18 10:45, Jonathan M Davis wrote:
 Honestly, I think at this point pure is easier to understand if you think of
 it as  noglobal and don't think about functional purity at all.
That's fine. My point was that the only optimizations possible are possible on strictly pure functions (the immutable cast one included). Weakly pure functions add nothing. But merely having them around means that when I annotate a function with "pure", I do not promise any guarantees that the compiler can actually use to perform optimizations. Shachar
Apr 02 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, April 02, 2018 11:04:08 Shachar Shemesh via Digitalmars-d wrote:
 On 02/04/18 10:45, Jonathan M Davis wrote:
 Honestly, I think at this point pure is easier to understand if you
 think of it as  noglobal and don't think about functional purity at
 all.
That's fine. My point was that the only optimizations possible are possible on strictly pure functions (the immutable cast one included). Weakly pure functions add nothing. But merely having them around means that when I annotate a function with "pure", I do not promise any guarantees that the compiler can actually use to perform optimizations.
It means that a strongly pure function could call the function, which is the entire reason that the definition of pure was widened to simply mean that it guarantees that the function doesn't access global, mutable state instead of also including the requirements about parameters which are placed on strongly pure functions. So, a weakly pure function _can_ help with optimizations in that it helps to implement strongly pure functions, and without weakly pure functions, what you can do with strongly pure functions can be very limited, making weakly pure functions very important even if all you care about is optimizations, but no, a call to a weakly pure function cannot be elided based on the fact that it's weakly pure. Now, pure combined with const could provide some optimizations in rare cases, since the compiler can guarantee that a pure function doesn't mutate a const argument via another reference if no such mutable reference could be accessed through one of the arguments, but I doubt that such optimizations are done at this point, and it wouldn't involve eliding function calls. But in theory, the fact that the compiler knows that a function can't access anything except through its arguments could allow the compiler to optimize some code, even if it doesn't involve eliding function calls. Ultimately, I think that it's a mistake to think about pure having much to do with optimizations. Much as such optimizations do exist, they're just too limited. The primary advantage is that when you see that a function is pure, you know that the function is just using what it's given via its arguments, just like when you see a variable is immutable, you know that it can't be mutated. Any optimizations that can be gotten via pure are therefore mostly just gravy. So, if your only motivation in dealing with pure is optimizations, then there's a good chance that it really isn't worth your time, but personally, I think that it's quite valuably simply for guaranteeing the that function doesn't access global, mutable state. - Jonathan M Davis
Apr 02 2018
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/2/18 4:04 AM, Shachar Shemesh wrote:
 On 02/04/18 10:45, Jonathan M Davis wrote:
 Honestly, I think at this point pure is easier to understand if you 
 think of
 it as  noglobal and don't think about functional purity at all.
That's fine. My point was that the only optimizations possible are possible on strictly pure functions (the immutable cast one included). Weakly pure functions add nothing. But merely having them around means that when I annotate a function with "pure", I do not promise any guarantees that the compiler can actually use to perform optimizations. Shachar
This is a good article motivating the relaxed purity model we have: http://klickverbot.at/blog/2012/05/purity-in-d/
Apr 02 2018
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02.04.2018 10:04, Shachar Shemesh wrote:
 On 02/04/18 10:45, Jonathan M Davis wrote:
 Honestly, I think at this point pure is easier to understand if you 
 think of
 it as  noglobal and don't think about functional purity at all.
That's fine. My point was that the only optimizations possible are possible on strictly pure functions (the immutable cast one included). Weakly pure functions add nothing. ...
That point is wrong. It would be wrong even if your next point was true.
 But merely having them around means that when I annotate a function with 
 "pure", I do not promise any guarantees that the compiler can actually 
 use to perform optimizations.
 
 Shachar
Yes, actually you do. For example, if you have: void foo(int[] x)pure; void main(){ auto x = [1]; auto y = x.dup; foo(x); foo(y); } This can be transformed into: void main(){ auto x = [1]; foo(x); auto y = x.dup; } Pure functions have deterministic side-effects.
Apr 02 2018
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02.04.2018 08:56, Shachar Shemesh wrote:
 On 01/04/18 03:32, H. S. Teoh wrote:
 The one nagging question I've been having about pure is: how much are we
 actually taking advantage of the guarantees provided by pure?
My problem is that pure do not provide many guarantees. ...
It guarantees that no global state is accessed.
  We have
 developed very clever ways of extending the traditional definition of
 pure and invented creative ways of making more things pure, which is all
 great.
Can anyone explain to me what good are the guarantees provided by a function that is pure but not strictly pure? I couldn't find them. ...
You can use weakly pure functions to compose strongly pure functions.
  But AFAIK the only place where it's actually taken advantage of
 is to elide some redundant function calls inside a single expression.
You cannot even do that unless the function is strictly pure. For all D's extension of the pure concept, it weakened, rather than enhanced, what it means. ...
There is no such weakening.
 And perhaps infer uniqueness in some cases for implicit casting to
 immutable.
Can you expand on that one? Shachar
Apr 02 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, March 31, 2018 17:32:10 H. S. Teoh via Digitalmars-d wrote:
 On Sat, Mar 31, 2018 at 07:38:06PM -0400, Andrei Alexandrescu via
 Digitalmars-d wrote: [...]

 Once we have that, we can encapsulate desirable abstractions (such as
  nogc safe collections that work in pure code), regardless of how
 difficult their implementations might be. It seems that currently
 this(this) does not allow us to do that.
What exactly is it about this(this) that blocks us from doing that? Removing this(this) is going to be a huge breaking change far bigger than, say, removing autodecoding ever will be.
Well, as far const goes, you have the inherent problem that the object you have in this(this) has already been initialized, and you can't mutate const. I think that other attribute issues are mostly implementation issues. And as far as auto-decoding goes, removing postblit constructors would actually be far easier, because it's simply a matter of deprecating a particular function. Some traits that check for postblits would have to be adjusted, but all those really care about is that there's extra code beyond the default copying code, and they could easily be made to work with both postblits and whatever the new solution is at the same time until postblits are actually gone. Auto-decoding on the other hand, affects far more than just the functions in std.range.primitives, and there is no clear deprecation path. So, while it _might_ be true that deprecating postblits would break more code (and I'm honestly not convinced that it would), postblit constructors would actually have a clean deprecation path. So, while it breaks code, it does so in a way that can easily be managed, whereas the removal of auto-decoding is not straightforward at all. It could be done by just flipping the switch so-to-speak, but AFAIK, no one has yet presented a workable deprecation path. And ultimately, I think that _that_ is what's preventing us from fixing auto-decoding. If a clean deprecation path were found, then we could discuss whether the resulting breakage would be worth it, but without a clean deprecation path, I don't see how we could ever do it. Also, unless we can fix postblit constructors (and the evidence thus far is that if we can, it's way too complicated - e.g. Kenji's attempt several years ago was rejected because it was way too complicated), if we don't switch to a differnt solution, we're talking about permanently not supporting copying const types that require user-defined copying.
 A lot of us here have essentially given up on const except for a few
 very narrow cases.  The transitive nature of const makes it extremely
 difficult to work with in the general case, even though the simplest use
 cases are workable.

 One of the biggest stumbling blocks is that whenever ranges are
 involved, const is practically out of the question, because even though
 it can be made to work for most cases, there will almost always be that
 one pathological case where it's impossible / too hard to work around,
 and that ruins it for everything else, so that it's much simpler to just
 avoid it altogether.
I don't think that it's even the case that it can be made to work in most cases - or if it can, it involves a lot of static ifs. The range API does not require const for _anything_ (and really can't due to how restricted const is), so _no_ generic range-based code can assume that even something like length or empty can be called if the range is const or inout. As such, the only ranges that can mark anything with const are ones that either aren't generic or which have a bunch of static ifs presenting const and non-const versions depending on the template arguments, and IMHO, that's just not workable. I've done it before, and it's a mess. As such, ranges and const really don't work together at all. const really only works when you're dealing with a very constrainted set of types where you can actually guarantee that they work with const.
 * pure is difficult
[...] The one nagging question I've been having about pure is: how much are we actually taking advantage of the guarantees provided by pure? We have developed very clever ways of extending the traditional definition of pure and invented creative ways of making more things pure, which is all great. But AFAIK the only place where it's actually taken advantage of is to elide some redundant function calls inside a single expression. And perhaps infer uniqueness in some cases for implicit casting to immutable. While these are indisputably useful, they seem so far to be only relatively minor benefits. If pure is indeed so difficult to support generically, it begs the question, is it worth the effort just to gain these niggling benefits? Whatever happened to larger-scale benefits conferred by purity?
Honestly, I think that the main benefit of pure is that if you know that a function is pure, you know that it doesn't access any global, mutable state except through its arguments. The secondary benefit is that it can allow for functions which construct immutable objects using mutable state and without casts (though such functions are usually small in scale and don't require that large portions of the code base be pure). I think that the idea that pure is going to result in compiler optimizations is mostly a joke. Not only can it only work with strongly pure functions (which most pure functions aren't and can't be), but the way it's currently implemented, it can only elide calls within a single statement (and it might actually be a single expression - I'm not sure which). Going farther than that requires code-flow analysis, which Walter is almost always against, so it's almost certainly not happening. And even if it did, I don't think that it would help much. How often do you call the same function with the same arguments within a single function body? I expect that that's pretty rare. The place that it would likely be of the most benefit would be math code, and even there, I'm not sure that it happens much. So, I think that the only large-scale benefit thet exists for pure and really can exist for pure is the fact that you know that the function doesn't access global, mutable state. Everything else it does is just gravy and too limited to be a "large-scale" benefit. Certainly, optimizations are clearly _not_ the main benefit of pure, since they almost don't exist. But over time, we have managed to add more gravy here and there as we've figured out assumptions that can be made based on pure (like the case where we can convert the result of a pure function to immutable). - Jonathan M Davis
Mar 31 2018
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/31/18 9:01 PM, Jonathan M Davis wrote:
 And as far as auto-decoding goes
Let's keep this about construction and not about auto-decoding. Thanks.
Mar 31 2018
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 1 April 2018 at 01:01:24 UTC, Jonathan M Davis wrote:
 So, I think that the only large-scale benefit thet exists for 
 pure and really can exist for pure is the fact that you know 
 that the function doesn't access global, mutable state. 
 Everything else it does is just gravy and too limited to be a 
 "large-scale" benefit. Certainly, optimizations are clearly 
 _not_ the main benefit of pure, since they almost don't exist. 
 But over time, we have managed to add more gravy here and there 
 as we've figured out assumptions that can be made based on pure 
 (like the case where we can convert the result of a pure 
 function to immutable).

 - Jonathan M Davis
Great write-up. Why keep pure when the benefits obviously don't outweight the costs? Here is what I interpret reading this: "lost productivity".
Apr 01 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, April 01, 2018 11:37:21 Guillaume Piolat via Digitalmars-d wrote:
 On Sunday, 1 April 2018 at 01:01:24 UTC, Jonathan M Davis wrote:
 So, I think that the only large-scale benefit thet exists for
 pure and really can exist for pure is the fact that you know
 that the function doesn't access global, mutable state.
 Everything else it does is just gravy and too limited to be a
 "large-scale" benefit. Certainly, optimizations are clearly
 _not_ the main benefit of pure, since they almost don't exist.
 But over time, we have managed to add more gravy here and there
 as we've figured out assumptions that can be made based on pure
 (like the case where we can convert the result of a pure
 function to immutable).

 - Jonathan M Davis
Great write-up. Why keep pure when the benefits obviously don't outweight the costs? Here is what I interpret reading this: "lost productivity".
Personally, I don't think that the costs outweigh the benefits. Yes, if your primary goal is optimizations, then pure falls flat on its face, but personally, I think that the fact that pure allows you to prove that a function doesn't access any global, mutable state except through its arguments is quite valuable. There are still some things in druntime and Phobos which don't work with pure like they should, but in general, I haven't found that it's really a big problem. Some code can't be pure, and that's life, but a _lot_ can be. If anything, I find that const is the attribute that causes problems, not pure. But if you don't want to use pure, then don't use pure. Nothing is forcing you to use it, and if someone else insists on making their code work with pure, it's only a problem for you if they're trying to call your code, and they complain about the fact that your code doesn't work with pure. Certainly, code that isn't pure has no problems calling code that is, so if other folks go to the effort of making their code work with pure, it won't cause you problems. - Jonathan M Davis
Apr 01 2018
prev sibling next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 01/04/2018 11:38 AM, Andrei Alexandrescu wrote:
 We need to have a simple recipe on how to define a canonical object. 
 That would include requirements such as:
 
 * should work with mutable, const, immutable, and shared
 * the right way to define constructor(s)
 * the right way to define copying
 * the right way to define destructor(s)
We will also need to make this consistent for classes. *** So lets go back to the basics. From the primordial goo that is My Own Little Language (MOLL heavily WIP): __userType [Identifier:name] { (Declaration|Definition)* } Is a value type, acts pretty much like struct. Why do I not call it a struct? Because it shouldn't be used in user code. This exists only in mental models for D users currently. So something like: __userType Foo { int x; void __constructor(int x) { this.x = x; } void __postblit() { this.x = x+1; } void __deconstructor() { // free? } } Is comparable to D code of: struct Foo { int x; this(this) { this.x = x + 1; } ~this() { // free? } } So what does a class look like? __userType Foo { enum __IS_CLASS = true; static TypeInfo_Class TypeInfo = TypeInfo_Class(...); static void*[][] __vtables = [ [&TypeInfo, ...], [&TypeInfo, ...] ]; void*[] __vtable; void* __ptr; auto __cast(T)() if(T.__IS_CLASS) { // ... } } Why is this important? Because it shows that our current lifetime management strategies for ref counting with structs can be applied directly to classes. *** At this point in time, lets look at how each qualifier is perceived: mutable: default, good, everything else is a pain const: library code and the author doesn't want you to go touchy immutable: in binary only, quite yucky shared: a message from the library author and not much else I do not believe we can continue this conversation with the above perceptions. First and foremost qualifiers are there to tell other developers (and yourself in the future) what you intend on doing with a given bit of memory. Now we do need a way to express head-const. Because it allows us to say that we personally can't modify a bit of memory but we can call a method that does (from it). Good engineering requires us to consider psychology on these issues. To create a single unified view of this, will be tricky I think, but well worth it.
Mar 31 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, March 31, 2018 19:38:06 Andrei Alexandrescu via Digitalmars-d 
wrote:
 We need to have a simple recipe on how to define a canonical object.
 That would include requirements such as:

 * should work with mutable, const, immutable, and shared
 * the right way to define constructor(s)
 * the right way to define copying
 * the right way to define destructor(s)

 I've asked my student Razvan to document the behavior of this(this). His
 work:

 https://github.com/dlang/dmd/pull/8055
 https://github.com/dlang/dlang.org/pull/2281
 https://github.com/dlang/dlang.org/pull/2299

 ... reveals a puzzling array of behaviors. Sometimes the typechecking is
 wrong, too.

 I think it's very important for us to have a simple, correct, and
 canonical way of defining structs in the D language that work with the
 language features: qualifiers, pure, safe, and nogc.

 Once we have that, we can encapsulate desirable abstractions (such as
  nogc safe collections that work in pure code), regardless of how
 difficult their implementations might be. It seems that currently
 this(this) does not allow us to do that.

 Eduard, another student I work with, has made steps toward a collections
 library that rant into difficulties:

 *  safe is achievable with relative ease

 * immutable and const are very difficult, but we have an attack
 (assuming copy construction gets taken care of)

 *  nogc is doable with a couple of conventions for allocators

 * pure is difficult

 * making them work together is very difficult

 We need to offer features, tools, and guidance toward creating simple
 encapsulated types that work well with D's own abstractions. Once we
 have that we can build libraries to work satisfactorily for any domain.

 I think the way to move forward is to deprecate this(this) entirely and
 create a DIP that allows people to define truly encapsulated structs.
 This is important, urgent, and of huge impact.

 I am looking for folks to assist me in creating a DIP for that. There
 will be a _lot_ of work involved, so don't take it lightly.
Another potential issue is whether any of this does or should relate to https://github.com/dlang/DIPs/pull/109 and it's solution for hooking into to moves. I'm not at all sure that what happens with that needs to be related to this at all, but it might. - Jonathan M Davis
Mar 31 2018
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Sunday, 1 April 2018 at 01:56:40 UTC, Jonathan M Davis wrote:
 Another potential issue is whether any of this does or should 
 relate to

 https://github.com/dlang/DIPs/pull/109

 and it's solution for hooking into to moves. I'm not at all 
 sure that what happens with that needs to be related to this at 
 all, but it might.

 - Jonathan M Davis
And before we think about `opMove` we should, IMO, make the compiler pass by move in more cases, for instance, in range constructors such as this(Source source) { this.source = source; // last occurrence of `source` can be moved } I'd be happy to help out with adding this in dmd. Andrei has already showed interest in this idea.
Apr 01 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, April 01, 2018 12:23:29 Per Nordlw via Digitalmars-d wrote:
 On Sunday, 1 April 2018 at 01:56:40 UTC, Jonathan M Davis wrote:
 Another potential issue is whether any of this does or should
 relate to

 https://github.com/dlang/DIPs/pull/109

 and it's solution for hooking into to moves. I'm not at all
 sure that what happens with that needs to be related to this at
 all, but it might.

 - Jonathan M Davis
And before we think about `opMove` we should, IMO, make the compiler pass by move in more cases, for instance, in range constructors such as this(Source source) { this.source = source; // last occurrence of `source` can be moved } I'd be happy to help out with adding this in dmd. Andrei has already showed interest in this idea.
I don't see what that has to do with opMove other than the fact that in such cases, you'd then end up with opMove being called instead of the postblit constructor. It seems to me that whether opMove is part of the language or not when such improvements are made is irrelevant. They should be able to be done independently of one other. As I understand it, the motivations behind opMove really have nothing to do with how often moves are made vs copying. It has to do with types that have serious issues if they're moved. The Weka guys (and probably others) have use cases where the fact that an object is moved without any way for them to detect it and do stuff like update pointers to the object is a serious problem. If anything, their use case might be better off right now if _no_ moves were ever done (at least until something like opMove is in the language), since the moves result in bugs. - Jonathan M Davis
Apr 01 2018
prev sibling parent Shachar Shemesh <shachar weka.io> writes:
On 01/04/18 04:56, Jonathan M Davis wrote:
 Another potential issue is whether any of this does or should relate to
 
 https://github.com/dlang/DIPs/pull/109
 
 and it's solution for hooking into to moves. I'm not at all sure that what
 happens with that needs to be related to this at all, but it might.
 
 - Jonathan M Davis
I was actually going to start a new thread about it. On the one hand, nothing in the opMove DIP is directly affected by this. On the other, if we're moving away from "copy and then fix" mentality, then opMove should also reflect this. The problem is that the alternative solution has a much bigger impact on backward compatibility. I'm really tempted to try and push this DIP as is, only renaming "opMove" to "opPostMove". This way, we can push the simpler version as is, and when (and it will take a while) "this(this)" deprecation finally comes, we can implement "opMove" as an in-process user hook. Shachar
Apr 04 2018
prev sibling next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
Am 01.04.2018 um 01:38 schrieb Andrei Alexandrescu:
 We need to have a simple recipe on how to define a canonical object.
 That would include requirements such as:
 
 * should work with mutable, const, immutable, and shared
 * the right way to define constructor(s)
 * the right way to define copying
 * the right way to define destructor(s)
 
 I've asked my student Razvan to document the behavior of this(this). His
 work:
 
 https://github.com/dlang/dmd/pull/8055
 https://github.com/dlang/dlang.org/pull/2281
 https://github.com/dlang/dlang.org/pull/2299
 
 ... reveals a puzzling array of behaviors. Sometimes the typechecking is
 wrong, too.
 
 I think it's very important for us to have a simple, correct, and
 canonical way of defining structs in the D language that work with the
 language features: qualifiers, pure, safe, and nogc.
 
 Once we have that, we can encapsulate desirable abstractions (such as
  nogc safe collections that work in pure code), regardless of how
 difficult their implementations might be. It seems that currently
 this(this) does not allow us to do that.
 
 Eduard, another student I work with, has made steps toward a collections
 library that rant into difficulties:
 
 *  safe is achievable with relative ease
 
 * immutable and const are very difficult, but we have an attack
 (assuming copy construction gets taken care of)
 
 *  nogc is doable with a couple of conventions for allocators
 
 * pure is difficult
 
 * making them work together is very difficult
 
 We need to offer features, tools, and guidance toward creating simple
 encapsulated types that work well with D's own abstractions. Once we
 have that we can build libraries to work satisfactorily for any domain.
 
 I think the way to move forward is to deprecate this(this) entirely and
 create a DIP that allows people to define truly encapsulated structs.
 This is important, urgent, and of huge impact.
 
 I am looking for folks to assist me in creating a DIP for that. There
 will be a _lot_ of work involved, so don't take it lightly.
 
 
 Thanks,
 
 Andrei
This seems really sudden, april fool's joke? Not really sure, as there are real problems with this(this)...
Apr 01 2018
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/1/18 6:04 AM, Johannes Loher wrote:
 This seems really sudden, april fool's joke? Not really sure, as there
 are real problems with this(this)...
I'm glad I've sent it yesterday then at least in my time zone :o). This looks sudden but isn't. Eduard and I have been blocked by this problem seriously whilst working on the collections library. Andrei
Apr 01 2018
prev sibling parent reply bachmeier <no spam.net> writes:
On Sunday, 1 April 2018 at 10:04:04 UTC, Johannes Loher wrote:
 This seems really sudden, april fool's joke? Not really sure, 
 as there are real problems with this(this)...
What I was wondering too. I mean, breaking changes just don't happen to this language. Now there will be, without even an indication of how existing code would have to be rewritten, or how this large-scale breakage is different than the breakages that just can't happen because reasons. I guess that's why there's always the disclaimer, "We'll only break code if there's a really good reason." That reason is "in case we want to".
Apr 01 2018
next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Sunday, 1 April 2018 at 10:49:22 UTC, bachmeier wrote:
 On Sunday, 1 April 2018 at 10:04:04 UTC, Johannes Loher wrote:
 This seems really sudden, april fool's joke? Not really sure, 
 as there are real problems with this(this)...
What I was wondering too. I mean, breaking changes just don't happen to this language. Now there will be, without even an indication of how existing code would have to be rewritten, or how this large-scale breakage is different than the breakages that just can't happen because reasons. I guess that's why there's always the disclaimer, "We'll only break code if there's a really good reason." That reason is "in case we want to".
Nothing has been lay out yet and people are already freaking out. No wonder nothing gets done anymore.
Apr 01 2018
parent reply bachmeier <no spam.net> writes:
On Sunday, 1 April 2018 at 11:17:38 UTC, Patrick Schluter wrote:

 What I was wondering too. I mean, breaking changes just don't 
 happen to this language. Now there will be, without even an 
 indication of how existing code would have to be rewritten, or 
 how this large-scale breakage is different than the breakages 
 that just can't happen because reasons. I guess that's why 
 there's always the disclaimer, "We'll only break code if 
 there's a really good reason." That reason is "in case we want 
 to".
Nothing has been lay out yet and people are already freaking out. No wonder nothing gets done anymore.
No, the reason nothing gets done is because "that would break code" is used to kill every proposal that comes along. Someone that only responds to proposals with "write a DIP" proceeds to announce a major piece of the language will be deprecated without writing a DIP himself. Corporate leadership doesn't work with an open source project. I could have gotten more involved long ago, but I'd rather not jump on a ship that's sailing in circles. From the many comments I've seen, I'm not the only one.
Apr 02 2018
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 2 April 2018 at 13:55:25 UTC, bachmeier wrote:

 No, the reason nothing gets done is because "that would break 
 code" is used to kill every proposal that comes along. Someone 
 that only responds to proposals with "write a DIP" proceeds to 
 announce a major piece of the language will be deprecated 
 without writing a DIP himself. Corporate leadership doesn't 
 work with an open source project. I could have gotten more 
 involved long ago, but I'd rather not jump on a ship that's 
 sailing in circles. From the many comments I've seen, I'm not 
 the only one.
Andrei wrote in the message
I am looking for folks to assist me in creating a DIP for that.
There will be a _lot_ of work involved, so don't take it lightly.
So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision. I'm +1 on major breaking changes if they drive D towards a better shape. /Paolo
Apr 02 2018
parent reply bachmeier <no spam.net> writes:
On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:

 Andrei wrote in the message

I am looking for folks to assist me in creating a DIP for that.
There will be a _lot_ of work involved, so don't take it 
lightly.
So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision. I'm +1 on major breaking changes if they drive D towards a better shape.
Andrei is asking others to write a DIP to formalize a decision he has already made. Yet when Manu posts here, he responds:
 The good news is there is a way to ensure your proposal gets a 
 fair shake of the stick: write a DIP.
 Filing a DIP is like filing a police report: once it's in the 
 system, we're obligated to work on it. There's a guarantee of a 
 response. In the case of acceptance, we commit to implementing 
 the proposal. In the case of rejection, we give a clear 
 motivation of the reasons we had. In the case we ask for 
 further review, we provide clear feedback of what would take 
 the DIP through another iteration.
 Forum discussions are the equivalent of complaining loudly in a 
 bar to people you know and also to strangers within earshot 
 that your house was broken into. Until you file a report, the 
 police will not look into it.
There's not even an attempt made to pretend there's symmetry. The only way for Manu (and basically anyone else) to propose a change is to write a DIP. Andrei won't even participate in discussions without a DIP. That's probably a good idea. What's not a good idea is to make unilateral decisions about major breaking changes, posting in the forum, and then asking others to write the DIP. That's corporate software development, and it's very discouraging to potential contributors.
Apr 02 2018
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/02/2018 12:00 PM, bachmeier wrote:
 On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:
 
 Andrei wrote in the message

 I am looking for folks to assist me in creating a DIP for that.
 There will be a _lot_ of work involved, so don't take it lightly.
So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision. I'm +1 on major breaking changes if they drive D towards a better shape.
Andrei is asking others to write a DIP to formalize a decision he has already made. Yet when Manu posts here, he responds:
Apologies for the misunderstanding. I'll be the first author of the DIP and plan to dedicate a lot of time to it. I was just asking for others to join me in this important and urgent effort. Andrei
Apr 02 2018
prev sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 2 April 2018 at 16:00:11 UTC, bachmeier wrote:
 On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:
 Andrei wrote in the message
I am looking for folks to assist me in creating a DIP for that.
There will be a _lot_ of work involved, so don't take it 
lightly.
 Andrei is asking others to write a DIP to formalize a decision
<snip>
 There's not even an attempt made to pretend there's symmetry. 
 The only way for Manu (and basically anyone else) to propose a 
 change is to write a DIP. Andrei won't even participate in 
 discussions without a DIP. That's probably a good idea. What's 
 not a good idea is to make unilateral decisions about major 
 breaking changes, posting in the forum, and then asking others 
 to write the DIP. That's corporate software development, and 
 it's very discouraging to potential contributors.
I think you are plain wrong on this: the 'P' in a DIP stands for Proposal, so any decision is not taken yet. And I'll bet: - Andrei will be the main author or he will partecipate in the writing. - the DIP will follow the usual proceeding, exactly like the others. /Paolo
Apr 02 2018
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/02/2018 12:14 PM, Paolo Invernizzi wrote:
 On Monday, 2 April 2018 at 16:00:11 UTC, bachmeier wrote:
 On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:
 Andrei wrote in the message
 I am looking for folks to assist me in creating a DIP for that.
 There will be a _lot_ of work involved, so don't take it lightly.
 Andrei is asking others to write a DIP to formalize a decision
<snip>
 There's not even an attempt made to pretend there's symmetry. The only 
 way for Manu (and basically anyone else) to propose a change is to 
 write a DIP. Andrei won't even participate in discussions without a 
 DIP. That's probably a good idea. What's not a good idea is to make 
 unilateral decisions about major breaking changes, posting in the 
 forum, and then asking others to write the DIP. That's corporate 
 software development, and it's very discouraging to potential 
 contributors.
I think you are plain wrong on this: the 'P' in a DIP stands for Proposal, so any decision is not taken yet. And I'll bet: - Andrei will be the main author or he will partecipate in the writing. - the DIP will follow the usual proceeding, exactly like the others.
Affirmative. No need to discuss this further, it's a simple misunderstanding. I'd agree with bachmeier if his perception was correct.
Apr 02 2018
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/1/2018 3:49 AM, bachmeier wrote:
 What I was wondering too. I mean, breaking changes just don't happen to this 
 language. Now there will be, without even an indication of how existing code 
 would have to be rewritten, or how this large-scale breakage is different than 
 the breakages that just can't happen because reasons. I guess that's why
there's 
 always the disclaimer, "We'll only break code if there's a really good
reason." 
 That reason is "in case we want to".
The idea is not to introduce a breaking change. Postblits will remain. The copy constructor will be an additional construct, and if one is defined, it will be preferred. Eventually (years later) postblits will be slowly deprecated and eventually removed.
Apr 04 2018
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 4 April 2018 at 22:30:39 UTC, Walter Bright wrote:
 On 4/1/2018 3:49 AM, bachmeier wrote:
 What I was wondering too. I mean, breaking changes just don't 
 happen to this language. Now there will be, without even an 
 indication of how existing code would have to be rewritten, or 
 how this large-scale breakage is different than the breakages 
 that just can't happen because reasons. I guess that's why 
 there's always the disclaimer, "We'll only break code if 
 there's a really good reason." That reason is "in case we want 
 to".
The idea is not to introduce a breaking change. Postblits will remain. The copy constructor will be an additional construct, and if one is defined, it will be preferred. Eventually (years later) postblits will be slowly deprecated and eventually removed.
Could you describe how a copy constructor differs from postblit ?
Apr 04 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, April 04, 2018 22:41:39 Stefan Koch via Digitalmars-d wrote:
 On Wednesday, 4 April 2018 at 22:30:39 UTC, Walter Bright wrote:
 On 4/1/2018 3:49 AM, bachmeier wrote:
 What I was wondering too. I mean, breaking changes just don't
 happen to this language. Now there will be, without even an
 indication of how existing code would have to be rewritten, or
 how this large-scale breakage is different than the breakages
 that just can't happen because reasons. I guess that's why
 there's always the disclaimer, "We'll only break code if
 there's a really good reason." That reason is "in case we want
 to".
The idea is not to introduce a breaking change. Postblits will remain. The copy constructor will be an additional construct, and if one is defined, it will be preferred. Eventually (years later) postblits will be slowly deprecated and eventually removed.
Could you describe how a copy constructor differs from postblit ?
With a postblit, all of the members are copied _before_ the postblit constructor is called, meaning that the struct was blitted, and then the postblit constructors were called for each member variable that has a postblit constructor, before the struct's postblit constructor is called. So, inside the postblit constructor, you're mutating an already copied object. It's doing work post the copy - hence postblit. With a copy constructor, the copy is being directly constructed. Each member will be initialized exactly once rather than being initialized and then mutated. Ideally, each member that was not explicitly initialized by the copy constructor would be automatically copied from the original so that you wouldn't have to explicitly copy each member, so syntactically, it may look very much like a postblit constructor, but the key difference is that with a copy constructor, the copy is directly constructed with whatever changes are done by the copy constructor as opposed to having it copied and then mutated. - Jonathan M Davis
Apr 04 2018
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 04, 2018 at 03:30:39PM -0700, Walter Bright via Digitalmars-d wrote:
 On 4/1/2018 3:49 AM, bachmeier wrote:
 What I was wondering too. I mean, breaking changes just don't happen
 to this language. Now there will be, without even an indication of
 how existing code would have to be rewritten, or how this
 large-scale breakage is different than the breakages that just can't
 happen because reasons. I guess that's why there's always the
 disclaimer, "We'll only break code if there's a really good reason."
 That reason is "in case we want to".
The idea is not to introduce a breaking change. Postblits will remain. The copy constructor will be an additional construct, and if one is defined, it will be preferred.
This may lead to trouble, unless we explicitly make it illegal for something to have both a postblit and a copy ctor. Having the two interact with each other seems likely to lead to a lot of pathological, hard-to-understand results.
 Eventually (years later) postblits will be slowly deprecated and
 eventually removed.
Actually, if it's possible to have them coexist and there are no disadvantages to doing so, why not just leave both of them in? Then cases where postblits give the best performance could just be left as-is, and cases that require a copy ctor can use a copy ctor. T -- Klein bottle for rent ... inquire within. -- Stephen Mulraney
Apr 04 2018
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, April 04, 2018 15:51:25 H. S. Teoh via Digitalmars-d wrote:
 On Wed, Apr 04, 2018 at 03:30:39PM -0700, Walter Bright via Digitalmars-d 
wrote:
 On 4/1/2018 3:49 AM, bachmeier wrote:
 What I was wondering too. I mean, breaking changes just don't happen
 to this language. Now there will be, without even an indication of
 how existing code would have to be rewritten, or how this
 large-scale breakage is different than the breakages that just can't
 happen because reasons. I guess that's why there's always the
 disclaimer, "We'll only break code if there's a really good reason."
 That reason is "in case we want to".
The idea is not to introduce a breaking change. Postblits will remain. The copy constructor will be an additional construct, and if one is defined, it will be preferred.
This may lead to trouble, unless we explicitly make it illegal for something to have both a postblit and a copy ctor. Having the two interact with each other seems likely to lead to a lot of pathological, hard-to-understand results.
+1
 Eventually (years later) postblits will be slowly deprecated and
 eventually removed.
Actually, if it's possible to have them coexist and there are no disadvantages to doing so, why not just leave both of them in? Then cases where postblits give the best performance could just be left as-is, and cases that require a copy ctor can use a copy ctor.
The big disadvantage is that you would have to deal with two distinct ways to copy objects and would have to explain them both to everyone learnig D. AFAIK, the only real advantage to a postblit constructor is that you don't have to explicitly copy everything like you do when declaring a copy constructor in C++ - you just change the parts that need to be changed after the copy is made in order to make it a deep copy or do whatever you need to do to finish making the copy what it should be. As such, if we define copy constructors in a way that only requires listing the members that are actually going to be different from the default copy, then I really don't see an advantage to postblit constructors. Hmmm. And actually, thinking about that, I almost wonder if we could just change this(this) to be a copy constructor. Assume for a moment that inside this(this), we provide a new symbol that is the this pointer/reference for the original. We then have make it so that any member variable in the newly constructed object which is read before it is assigned is default-copied. As such, this(this) { } would just default-copy everything basically as it does now. And if you currently had this(this) { _foo = _foo.dup; } then after the change, it would still have the same semantics, because the _foo is used before it is assigned, so it must be default-copied. However, if we provide the object being copied via an invisible ref (like this) called orig, doing this(this) { _foo = orig._foo; } then that would directly initialize _foo without it being default-copied. As such, existing postblit constructors should continue to work, but it would be straightforward to turn them into copy constructors, and most of the changes would be underneath the hood. I don't know if orig is the best name (and making it a keyword would be really annoying, since it's a name that I use all the time), but on the surface at least, the idea seems sound to me. And assuming that there isn't a big implementation issue that makes it a serious problem, it should actually make transitioning from postblit constructors to copy constructors very clean and negate the need for any kind of deprecation process. - Jonathan M Davis
Apr 04 2018
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 04, 2018 at 05:08:26PM -0600, Jonathan M Davis via Digitalmars-d
wrote:
[...]
 AFAIK, the only real advantage to a postblit constructor is that you
 don't have to explicitly copy everything like you do when declaring a
 copy constructor in C++ - you just change the parts that need to be
 changed after the copy is made in order to make it a deep copy or do
 whatever you need to do to finish making the copy what it should be.
 As such, if we define copy constructors in a way that only requires
 listing the members that are actually going to be different from the
 default copy, then I really don't see an advantage to postblit
 constructors.
That's a very interesting concept. It could be a nice way of bridging current semantics over to the new semantics. Some of the existing postblit code might even be usable as-is, when reinterpreted in this way!
 Hmmm. And actually, thinking about that, I almost wonder if we could
 just change this(this) to be a copy constructor. Assume for a moment
 that inside this(this), we provide a new symbol that is the this
 pointer/reference for the original. We then have make it so that any
 member variable in the newly constructed object which is read before
 it is assigned is default-copied. As such,
 
 this(this)
 {
 }
 
 would just default-copy everything basically as it does now.
Why even bother with declaring an empty this(this)? Just leave it out, and the compiler assumes default-copy.
 And if you currently had
 
 this(this)
 {
     _foo = _foo.dup;
 }
 
 then after the change, it would still have the same semantics, because
 the _foo is used before it is assigned, so it must be default-copied.
That's a pretty neat way of transitioning from current semantics to new semantics. I like it.
 However, if we provide the object being copied via an invisible ref
 (like this) called orig, doing
 
 this(this)
 {
     _foo = orig._foo;
 }
 
 then that would directly initialize _foo without it being
 default-copied. As such, existing postblit constructors should
 continue to work, but it would be straightforward to turn them into
 copy constructors, and most of the changes would be underneath the
 hood.
[...] I like this idea. Except the syntax could be improved: this(this orig) // <-- N.B. { _foo = orig._foo; } Let the user specify the symbol, instead of introducing yet another implicit magic identifier. :-) T -- "I suspect the best way to deal with procrastination is to put off the procrastination itself until later. I've been meaning to try this, but haven't gotten around to it yet. " -- swr
Apr 05 2018
parent Nick Treleaven <nick geany.org> writes:
On Thursday, 5 April 2018 at 18:46:25 UTC, H. S. Teoh wrote:
 I like this idea.  Except the syntax could be improved:

 	this(this orig)		// <-- N.B.
 	{
It's already valid syntax, equivalent to `this(typeof(this) orig)`. It can be called explicitly, but is ignored for implicit construction. Perhaps we will just enable `this(typeof(this) orig)` for implicit construction too, if there's no postblit defined.
Apr 11 2018
prev sibling next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Saturday, 31 March 2018 at 23:38:06 UTC, Andrei Alexandrescu 
wrote:
 * immutable and const are very difficult, but we have an attack 
 (assuming copy construction gets taken care of)

 * pure is difficult
How about removing pure, const and immutable?
Apr 01 2018
prev sibling next sibling parent Shachar Shemesh <shachar weka.io> writes:
By sheer coincidence, I've just stumbled upon another limitation of 
this(this). I am not sure whether it is already documented.

Let's define struct S1 with no copying allowed, and put it as a member 
of struct S2. Under C++ as well as under D, this automatically makes S2 
also non-copyable.
Under C++, however, I can do this:

struct S1 {
     S1() {}

     // Make S1 non-copyable
     S1(const S1 &that) = delete;
     S1 &operator=(const S1 &that) = delete;
};

struct S2 {
     int a;
     S1 s;

     S2(int _a) : a(_a) {
     }

     S2(const S2 &that) : a(that.a) {
     }
};

int main() {
     S2 s(17);
     S2 v(s); // This compiles, invoking S2's copy ctor
}


In other words, I can tell the compiler that I know how to copy S2 
without having to copy the member S1.

Under D, this simply doesn't work. If S1 has  disable this(this), any 
struct that has S1 as a member will be uncopyable, and this is not 
overridable.

Shachar
Apr 02 2018
prev sibling next sibling parent Kagamin <spam here.lot> writes:
On Saturday, 31 March 2018 at 23:38:06 UTC, Andrei Alexandrescu 
wrote:
 * should work with mutable, const, immutable, and shared
The problem is that postblit is not overloadable, so make it overloadable, and problems with overloading will be solved.
 * immutable and const are very difficult, but we have an attack 
 (assuming copy construction gets taken care of)
Collections must be filled somehow, so they are inherently mutable, immutable collections need a whole different design approach, it doesn't look specific to postblit.
 * pure is difficult
Purity depends on written code. Running impure code in copy constructor won't make it pure.
Apr 02 2018
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Saturday, 31 March 2018 at 23:38:06 UTC, Andrei Alexandrescu 
wrote:
 [snip]

 * immutable and const are very difficult, but we have an attack 
 (assuming copy construction gets taken care of)
Would it be easier if the const/immutable containers were considered separate types? For instance, in the code below, there is InoutFoo and then Foo takes InoutFoo as an alias this (you could do the same thing with immutable, but then you’d have to include two get functions). This would be like inheriting the InoutFoo. With better syntax, InoutFoo would be something like inout(Foo) and the compiler could recognize that the mutable constructor is also defined and to call that when appropriate. struct InoutFoo { int a; this(int b) inout { this.a = b; } int get() inout { return a; } } struct Foo { InoutFoo inoutfoo; alias inoutfoo this; this(int b) { a = b; } void set(int b) { a = b; } } void main() { auto x = immutable InoutFoo(1); auto y = Foo(1); assert(is(typeof(y) : typeof(x))); //x.a++; //not allowed y.a++; assert(x.a == 1); assert(y.a == 2); assert(x.get == 1); y.set(3); assert(y.get == 3); }
Apr 03 2018
prev sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, April 05, 2018 11:46:25 H. S. Teoh via Digitalmars-d wrote:
 On Wed, Apr 04, 2018 at 05:08:26PM -0600, Jonathan M Davis via
 Digitalmars-d wrote: [...]
 Hmmm. And actually, thinking about that, I almost wonder if we could
 just change this(this) to be a copy constructor. Assume for a moment
 that inside this(this), we provide a new symbol that is the this
 pointer/reference for the original. We then have make it so that any
 member variable in the newly constructed object which is read before
 it is assigned is default-copied. As such,

 this(this)
 {
 }

 would just default-copy everything basically as it does now.
Why even bother with declaring an empty this(this)? Just leave it out, and the compiler assumes default-copy.
You wouldn't declare it. I was just talking about it to discuss the semantics.
 However, if we provide the object being copied via an invisible ref
 (like this) called orig, doing

 this(this)
 {

     _foo = orig._foo;

 }

 then that would directly initialize _foo without it being
 default-copied. As such, existing postblit constructors should
 continue to work, but it would be straightforward to turn them into
 copy constructors, and most of the changes would be underneath the
 hood.
[...] I like this idea. Except the syntax could be improved: this(this orig) // <-- N.B. { _foo = orig._foo; } Let the user specify the symbol, instead of introducing yet another implicit magic identifier. :-)
Well, that might be better in that it allows the user to choose the name, but it then opens up the question of whether this(this) should be deprecated, and part of the point was to try and just do it all in-place and simply turn postblit constructors into copy constructors - though it could be argued that this(this) was just a copy constructor where you didn't bother to use the original, and anyone actually wanting to take advantage of copy constructors is going to need to make changes anyway. But if this(this) simply becomes a copy constructor (whether a default name is used or something like this(this orig) is used to define the name), then we hopefully would be able to avoid actually deprecating anything and just make more code work and making it so that only minor tweaks are necessary to actually take full advantage of it being a copy constructor instead of mutating the copy in the copy constructor. The other issue that occurred to me after posting about this was the question of what to do about const. Ideally, this(this) or this(this orig) would work just fine with const (assuming that it didn't do anything that would mutate the object), but someone might want to actually require const or overload the copy constructor for the const case, in which case, I was thinking making this(const this), but with your suggestion, I guess that it would become either this(const this orig) or this(this const orig). I don't know. As usual, adding const into the mix makes things messier, but part of the point of all of this is to make it so that we can copy arbitrarily complex const objects. In any case, I think that the idea of trying to simply turn postblit constructors into copy constructors has merit. I certainly don't think that we need to be doing anything like struct S { this(ref S orig) { ... } } where everyone will have to convert their postblits over time in order to avoid the deprecation and where it actually risks conflicting with existing constructors. - Jonathan M Davis
Apr 05 2018