www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP 1014

reply Manu <turkeyman gmail.com> writes:
Who knows about DIP 1014? (struct move hook)
Is it well received? Is it likely to be accepted soon?

I'm working on the std::string binding, it's almost finished... but
then I hit a brick wall.
GNU's std::string implementation stores an interior pointer! >_<

No other implementation does this. It's a really bad implementation
actually, quite inefficient. It could make better use of its space for
small-strings if it wasn't wasting 8-bytes for an interior pointer to
a small string buffer...

Anyway, I'm blocked until this DIP is accepted; so, is it looking promising?
Sep 29 2018
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Sunday, 30 September 2018 at 04:34:20 UTC, Manu wrote:
 Who knows about DIP 1014? (struct move hook)
 Is it well received? Is it likely to be accepted soon?

 I'm working on the std::string binding, it's almost finished... 
 but
 then I hit a brick wall.
 GNU's std::string implementation stores an interior pointer! >_<

 No other implementation does this. It's a really bad 
 implementation actually, quite inefficient. It could make 
 better use of its space for small-strings if it wasn't wasting 
 8-bytes for an interior pointer to a small string buffer...

 Anyway, I'm blocked until this DIP is accepted; so, is it 
 looking promising?
It's pending a decision from Walter & Andrei, which I hope to hear soon.
Sep 29 2018
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2018 9:34 PM, Manu wrote:
 GNU's std::string implementation stores an interior pointer! >_<
 
 No other implementation does this. It's a really bad implementation
 actually, quite inefficient. It could make better use of its space for
 small-strings if it wasn't wasting 8-bytes for an interior pointer to
 a small string buffer...
Could you post a synopsis of the layout of std::string?
Sep 29 2018
parent reply Manu <turkeyman gmail.com> writes:
On Sat, Sep 29, 2018 at 11:50 PM Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 9/29/2018 9:34 PM, Manu wrote:
 GNU's std::string implementation stores an interior pointer! >_<

 No other implementation does this. It's a really bad implementation
 actually, quite inefficient. It could make better use of its space for
 small-strings if it wasn't wasting 8-bytes for an interior pointer to
 a small string buffer...
Could you post a synopsis of the layout of std::string?
The code's all in the PR if you wanna dig into it. The synopsis is: struct string { char* ptr; size_t len; union { char[16] localBuffer; size_type allocatedCapacity; } bool isAllocated() { return ptr != &localBuffer[0]; } bool capacity() { return isAllocated() ? allocatedCapacity : localBuffer.length; } this(DefaultCtor) { ptr = &localBuffer[0]; } // <- and here it is. interior pointer that breaks move semantics } Other implementations make much better use of that built-in space by not wasting 8 bytes on an interior pointer for small-strings.
Sep 30 2018
parent reply Shachar Shemesh <shachar weka.io> writes:
On 30/09/18 10:26, Manu wrote:
 
 Other implementations make much better use of that built-in space by
 not wasting 8 bytes on an interior pointer for small-strings.
 
I will point out that a pointer that *sometimes* points to an internal member was one of the use cases I documented when I submitted the DIP. Starting a long discussion about the merits of the design is a bit off-topic. I will point out that branch prediction considerations *might* make this a wise choice, despite the loss of 8 bytes of potential storage. Either way, this is a design that is highly sensitive to precise use pattern, which, admitably, GNU's std::string probably can't know. Shachar
Sep 30 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, September 30, 2018 1:35:28 AM MDT Shachar Shemesh via 
Digitalmars-d wrote:
 On 30/09/18 10:26, Manu wrote:
 Other implementations make much better use of that built-in space by
 not wasting 8 bytes on an interior pointer for small-strings.
I will point out that a pointer that *sometimes* points to an internal member was one of the use cases I documented when I submitted the DIP. Starting a long discussion about the merits of the design is a bit off-topic. I will point out that branch prediction considerations *might* make this a wise choice, despite the loss of 8 bytes of potential storage. Either way, this is a design that is highly sensitive to precise use pattern, which, admitably, GNU's std::string probably can't know.
I think that the key thing here is that if GNU's std::string is using a design like this, it's that much more critical that we have a way to hook into moves to do stuff like adjust pointers. It's one more example of a real world use case where the DIP (or something like it) is needed, or there are things that we simply won't be able to do in D - and given the push to interface with C++, it's that much more important. And while a discussion could certainly be had as to whether GNU's design decision was a good one or not, it's secondary to what really matters here, which is what the state of the DIP is how we're going to deal with interfacing with this C++ code. We need to worry about how to interface with it whether it's the best design ever or whether it's the worst design ever - and we have to take into account the fact the its design could actually change in future versions if they decide that a different way is better (e.g. GNU could change to match other implementations, or other implementations could change to match GNU, depending on what actually turned out to be best in practice when all of the various factors were taken into account - including developers making future decisions that aren't necessarily good ones; we have to interface with the code whether it's good or bad). All in all though, if anything, I have to think that this issue increases the chances of the DIP being accepted given the importance that Walter and Andrei have been placing on interfacing with C++. And having it come up while they're in the middle of discussing it probably doesn't hurt - though maybe they were already going to accept it. I don't know. Personally, while I tend to think that it's generally better to avoid designs where opPostMove is necessary if possible, I think that the case was well made that we need a solution like it in certain cases, and if we want to interface with C++, which can do more or less arbitrary stuff in its move constructors, I don't see how we can avoid having an analogue unless we want to give up on interfacing with that code without an extra compatibility layer. - Jonathan M Davis
Sep 30 2018
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2018 9:34 PM, Manu wrote:
 Who knows about DIP 1014? (struct move hook)
When discussing DIP 1014, a link is helpful: https://github.com/dlang/DIPs/blob/38cec74a7471735559e3b8a7553f55102d289d28/DIPs/DIP1014.md
Oct 02 2018
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2018 9:34 PM, Manu wrote:
 Who knows about DIP 1014? (struct move hook)
 Is it well received? Is it likely to be accepted soon?
 
 I'm working on the std::string binding, it's almost finished... but
 then I hit a brick wall.
 GNU's std::string implementation stores an interior pointer! >_<
The rationale behind D allowing structs to be moveable is to enable a copying garbage collector. Some solutions to this problem: 1. Don't allow moving of C++ structs 2. Add a struct attribute that means "not moveable" 3. DIP 1014, which is add a __move_post_blit() function (most complex solution) 4. Use copy/destruct for C++ structs that have copy constructors (this is the old C++ solution, and is less efficient than the move constructor) A discussion of the rationale for the C++ move constructor is: https://akrzemi1.wordpress.com/2011/08/11/move-constructor/
 Anyway, I'm blocked until this DIP is accepted; so, is it looking promising?
Pragmatically, I suggest for the moment just ignore the problem, file a bug report for std::string, and move on.
Oct 02 2018
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/2/2018 2:17 AM, Walter Bright wrote:
 1. Don't allow moving of C++ structs
 2. Add a struct attribute that means "not moveable"
 3. DIP 1014, which is add a __move_post_blit() function (most complex solution)
 4. Use copy/destruct for C++ structs that have copy constructors (this is the 
 old C++ solution, and is less efficient than the move constructor)
The postblit solution can also work today, https://issues.dlang.org/show_bug.cgi?id=17448#c37 as the DMD compiler doesn't actually move structs. So you're OK for the time being until DMD does, or a copying garbage collector is implemented.
Oct 02 2018
next sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Oct 2, 2018 at 2:40 AM Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 10/2/2018 2:17 AM, Walter Bright wrote:
 1. Don't allow moving of C++ structs
 2. Add a struct attribute that means "not moveable"
 3. DIP 1014, which is add a __move_post_blit() function (most complex solution)
 4. Use copy/destruct for C++ structs that have copy constructors (this is the
 old C++ solution, and is less efficient than the move constructor)
The postblit solution can also work today, https://issues.dlang.org/show_bug.cgi?id=17448#c37 as the DMD compiler doesn't actually move structs.
Sorry... what?! DMD doesn't move? O_O We've been talking endlessly about move semantics for like, 10 years... Do the other compilers move? I don't understand... I've missed something.
  So you're OK for the time
 being until DMD does, or a copying garbage collector is implemented.
Ummm...
Oct 02 2018
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, October 2, 2018 11:54:57 AM MDT Manu via Digitalmars-d wrote:
 On Tue, Oct 2, 2018 at 2:40 AM Walter Bright via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:
 On 10/2/2018 2:17 AM, Walter Bright wrote:
 1. Don't allow moving of C++ structs
 2. Add a struct attribute that means "not moveable"
 3. DIP 1014, which is add a __move_post_blit() function (most complex
 solution) 4. Use copy/destruct for C++ structs that have copy
 constructors (this is the old C++ solution, and is less efficient
 than the move constructor)>
The postblit solution can also work today, https://issues.dlang.org/show_bug.cgi?id=17448#c37 as the DMD compiler doesn't actually move structs.
Sorry... what?! DMD doesn't move? O_O We've been talking endlessly about move semantics for like, 10 years... Do the other compilers move? I don't understand... I've missed something.
Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved structs in a number of situations - e.g. when the return value was an rvalue. Something like A foo(); void bar(A); bar(foo()); is supposed to be guaranteed to move and not copy the return value. Maybe it manages to place the return value in a way that doesn't require actually moving it or copying it, but structs being moved was supposed to be a key thing that D could do. Are we currently getting a bunch of copies that we shouldn't be getting because the compiler isn't yet doing moves when it should be? A number of us have been answering questions for years based on what the spec and TDPL say indicating that the compiler moves structs. And while I'm all for having objects be moveable by default, I confess that I don't understand what the problem is with the opPostMove idea that the DIP is presenting. I don't think that it should be the norm by any means, but it sure seems like it's cleanly solving a major problem with interacting with C++, and it makes it possible to do stuff like have pointers and dynamic arrays refer to a structs internals in a way that we can't safely do right now (even though occasionally, it really would be useful to be able to do it). And it sure seems like opPostMove fits in cleanly with the current design, though maybe I'm missing something. Either way, without something like it, we're clearly missing some key functionality for a systems language - particularly one that wants to interoperate with C++. - Jonathan M Davis
Oct 02 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis 
wrote:
 Yeah. IIRC, it was supposed to be _guaranteed_ that the 
 compiler moved structs in a number of situations - e.g. when 
 the return value was an rvalue. Something like
Eh, I don't think that moves it, but rather just constructs it in-place for the next call.
Oct 02 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
 On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
 Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved structs 
 in a number of situations - e.g. when the return value was an rvalue. 
 Something like
Eh, I don't think that moves it, but rather just constructs it in-place for the next call.
The technical term for that is "copy elision".
Oct 02 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
 On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
 Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved structs
 in a number of situations - e.g. when the return value was an rvalue.
 Something like
Eh, I don't think that moves it, but rather just constructs it in-place for the next call.
The technical term for that is "copy elision".
Okay, so copy elision is working... but moves otherwise are not? That's still not what we've been peddling all these years. A whole lot of design surface area is dedicated to implicit move semantics... and they don't work? What does it do? postblit unnecessarily?
Oct 03 2018
next sibling parent reply Corel <arguile gmail.com> writes:
On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote:
 On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
 On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis 
 wrote:
 Yeah. IIRC, it was supposed to be _guaranteed_ that the 
 compiler moved structs in a number of situations - e.g. 
 when the return value was an rvalue. Something like
Eh, I don't think that moves it, but rather just constructs it in-place for the next call.
The technical term for that is "copy elision".
Okay, so copy elision is working... but moves otherwise are not? That's still not what we've been peddling all these years. A whole lot of design surface area is dedicated to implicit move semantics... and they don't work? What does it do? postblit unnecessarily?
The impression is that you are complaining about the continuous lack of "things" based on an incomplete knowledge of how D works in detail ... tragically you invoke low-level features, and you do not know the question. The fact that in D the structures to date are not moved, is known for years ... take advantage of this fact, and move on. Work on an implementation that works, AFTER profile it, and possibly complain about performance.
Oct 03 2018
next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Oct 3, 2018 at 2:50 AM Corel via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote:
 On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
 On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis
 wrote:
 Yeah. IIRC, it was supposed to be _guaranteed_ that the
 compiler moved structs in a number of situations - e.g.
 when the return value was an rvalue. Something like
Eh, I don't think that moves it, but rather just constructs it in-place for the next call.
The technical term for that is "copy elision".
Okay, so copy elision is working... but moves otherwise are not? That's still not what we've been peddling all these years. A whole lot of design surface area is dedicated to implicit move semantics... and they don't work? What does it do? postblit unnecessarily?
The impression is that you are complaining about the continuous lack of "things" based on an incomplete knowledge of how D works in detail ... tragically you invoke low-level features, and you do not know the question. The fact that in D the structures to date are not moved, is known for years ... take advantage of this fact, and move on. Work on an implementation that works, AFTER profile it, and possibly complain about performance.
O_o .. this is one of the stranger replies I've ever gotten here.
Oct 03 2018
prev sibling parent Shachar Shemesh <shachar weka.io> writes:
On 03/10/18 12:48, Corel wrote:
 The fact that in D the structures to date are not moved, is known for 
 years ... take advantage of this fact, and move on.
I have no idea where you got this fact: import std.stdio; struct MoveTest { static uint counter=1; uint id; disable this(this); disable this(MoveTest); this(uint dummy) { id = counter++; writefln("Constructed %s id %s", &this, id); } ~this() { writefln("Id %s destroyed at %s", id, &this); } } MoveTest func1() { return MoveTest(3); } void func2(MoveTest m) { } int main() { func2(func1()); return 0; } $ rdmd movetest.d Constructed 7FFDC7A663E0 id 1 Id 1 destroyed at 7FFDC7A66400 Our instance was constructed at one address, but destroyed at another. In other words, it was moved. Can we, please, put that myth to rest? Shachar
Oct 03 2018
prev sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote:

 Okay, so copy elision is working... but moves otherwise are 
 not? That's still not what we've been peddling all these years. 
 A whole lot of design surface area is dedicated to implicit 
 move semantics... and they don't work? What does it do? 
 postblit unnecessarily?
No. The problem is that the language is under-specified. It is built on the *assumption* that no one ever should create self-referencing data. But it does not enforce that. Which eventually leads to someone trying to make such data and then run into a wall, or worse, a heisenbug. Thing is, there isn't anything wrong with self-referencing data per se. It's that the language plumbing should either disallow it wholesale (i.e. Rust) or allow a graceful way of handling it. Neither is present in D. The latter could be added though, that's what the DIP is about.
Oct 03 2018
prev sibling parent reply Shachar Shemesh <shachar weka.io> writes:
On 03/10/18 04:10, Walter Bright wrote:
 On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
 On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
 Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler 
 moved structs in a number of situations - e.g. when the return value 
 was an rvalue. Something like
Eh, I don't think that moves it, but rather just constructs it in-place for the next call.
The technical term for that is "copy elision".
I'm not sure I follow. First of all, you cannot elide the copy if there is more than one potential local variable you are returning, ala: A someFunc() { A a, b; manipulate(a); manipulate(b); if( someRandomCondition ) return a; return b; } What happens then? What happens if A has disable this(this)? What happens if we explicitly call std.algorithm.move? Shachar
Oct 03 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
Shachar, as I don't see a better place of discussing that DIP at 
the moment, I'll pour some observations and thoughts in here if 
you don't mind, will add some comments on GitHub later.
As I see it right now, it's a case of over-engineering of a quite 
simple concept.

1. A new function, called __move_post_blt, will be added to 
DRuntime.

That's unnecessary, if not downright harmful for the language. We 
should strive to remove things from DRuntime, not add to it. The 
core language should deal with type memory, not a .so or dll. And 
it's extraneous, because...

2 and 3. onPostMove and __move_post_blt:

They're unnecessary as well. All that's required is to allow a 
by-value constructor, e.g:

struct S {
     this(S rhs);
}

Any function in D that has a signature of the form

ReturnType foo(Type x);

in C++ would have an equivalent signature of

ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);

because passing by value in D always implies a possible move. The 
'x' in such functions can be safely cannibalized without any 
repercussions, as it is either a temporary on the call site, or, 
which is especially pertaining to the original bugzilla 
discussion, constructed in place via copy elision.

Thus in effect this(S) would be an equivalent of C++'s move 
constructor. We already have a de-facto move-assignment in the 
form of opAssign(S), this(S) would be a natural extension to that.

Note, per above, that it is NOT a copy constructor, although user 
code may want to create a copy *before* calling it, to create the 
temporary.

Such approach reduces added complexity. The only potential 
problem with it would be a need to "special-case" initialization 
from .init, although at the moment, I think even that may be 
unnecessary: this is a hook after all.

Your example from the DIP would become:

struct Tracker {
     static uint globalCounter;
     uint localCounter;
     uint* counter;

      disable this(this);

     this(bool local) {
         localCounter = 0;
         if( local )
             counter = &localCounter;
         else
             counter = &globalCounter;
     }

     this(Tracker oldLocation) {
         if( counter is &oldLocation.localCounter )
             counter = &localCounter;
     }

     void increment() {
         (*counter)++;
     }

}

Usage:

auto old = Tracker(true);
// ...
auto new = move(old); // calls Tracker.this(Tracker);

...this avoids any need to inject special postblits into user 
code.

As I see it, in addition to the above, what would be really 
desirable is for move() and emplace() family of calls to become 
compiler intrinsics instead of library constructs. Those at the 
moment are complete poison: being templates they infect user code 
with dependencies on libc (moveEmplace calls memset and memcpy of 
all things) and unnecessary calls to DRuntime (typeid), and they 
of course blow up the amount of generated code in the form of 
template instantiations. That's despite the fact that the 
compiler possesses ALL the necessary knowledge at the time of 
those calls.
Oct 03 2018
next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 13:56:29 UTC, Stanislav Blinov 
wrote:

Aendment, this should of course be:

     this(Tracker oldLocation) {
         localCounter = oldLocation.locaclCounter;
         counter = oldLocation.counter;
         if( counter is &oldLocation.localCounter )
             counter = &localCounter;
     }
Oct 03 2018
prev sibling next sibling parent reply Shachar Shemesh <shachar weka.io> writes:
On 03/10/18 16:56, Stanislav Blinov wrote:
 struct S {
      this(S rhs);
OMG, that's so simple!!! Why didn't I think of it? Oh wait, I did. And this simply and utterly doesn't work. If you read the DIP, you will notice that the *address* in which the old instance resides is quite important for performing the actual move. This is not available with the interface you're suggesting, mainly because by the time you have rhs, it has already moved. In other words, for the interface above to work, the type must already be movable, which kinda contradict what we're trying to achieve here.
 in C++ would have an equivalent signature of
 
 ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
No, it is not. You see, in C++, x is an "rvalue *reference*". x has not moved by this point in the run, it has simply had its address passed to foo. Please see https://stackoverflow.com/questions/28483250/rvalue-reference-is-treated-as-an-lvalue Shachar
Oct 03 2018
next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 14:07:58 UTC, Shachar Shemesh 
wrote:
 On 03/10/18 16:56, Stanislav Blinov wrote:
 struct S {
      this(S rhs);
OMG, that's so simple!!! Why didn't I think of it? Oh wait, I did.
Now I see why sometimes your posts are greeted with hostility.
 And this simply and utterly doesn't work.

 If you read the DIP, you will notice that the *address* in 
 which the old instance resides is quite important for 
 performing the actual move. This is not available with the 
 interface you're suggesting, mainly because by the time you 
 have rhs, it has already moved.

 In other words, for the interface above to work, the type must 
 already be movable, which kinda contradict what we're trying to 
 achieve here.
In the presence of such a constructor, the compiler will have to call it every time it moves the value, same as what you're proposing for __move_post_blt. This obviates the need of an address: address of the argument will always already be sufficient, even though it's not ref, as the chain of calls for this(S) will inevitably start with the address of something constructed in place.
 in C++ would have an equivalent signature of
 
 ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
No, it is not. You see, in C++, x is an "rvalue *reference*". x has not moved by this point in the run, it has simply had its address passed to foo.
You've misunderstood me. Yes, in C++ there's an obvious difference between pass-by-value and pass-by-rvalue-reference, and it is always user's responsibility to write a move ctor. Not so in D. In D, you can always assume that anything passed by value *is* an rvalue reference, precisely because of D's take on move semantics. I.e. any argument passed by value can assumed to be moved or constructed in place (that's the main difference from C++, where it must be explicitly specified).
Oct 03 2018
parent reply Shachar Shemesh <shachar weka.io> writes:
On 03/10/18 17:29, Stanislav Blinov wrote:
 OMG, that's so simple!!! Why didn't I think of it?

 Oh wait, I did.
Now I see why sometimes your posts are greeted with hostility.
Yes. I am actually sorry about that. I was responding to your assumption that I'm wrong. Had your post been phrased as "why didn't you", instead of "you're wrong wrong wrong" I wouldn't have responded that way. Like I said, I am sorry.
 Allow me to further illustrate with something that can be written in 
D > today: I am not sure what you were trying to demonstrate, so instead I wanted to see if you succeeded. I added the following to your Tracker struct: ~this() { writefln("%s destructed", &this); assert(counter is null || counter is &localCounter); } I.e. - I am asserting if a move was not caught. The program fails to run on either ldc or dmd. To me, this makes perfect sense as for the way D is built. In essence, opAssign isn't guaranteed to run. Feel free to build a struct where that assert passes to convince me. Here is the flaw in your logic: void opAssign(Tracker rhs) rhs is passed by value. This means that already at the point opAssign is called, rhs *already* has a different address than the one it was passed in with. I did not follow your logic on why this isn't so, but I don't see how you can make it not so without changing the ABI quite drastically. Shachar
Oct 03 2018
next sibling parent Shachar Shemesh <shachar weka.io> writes:
On 03/10/18 18:33, Shachar Shemesh wrote:
       ~this() {
          writefln("%s destructed", &this);
          assert(counter is null || counter is &localCounter);
      }
You might also want to add disable this(this); and remove the dead code (i.e. - the case where the pointer is global) to reduce noise. I verified that neither one changes anything in the outcome. Shachar
Oct 03 2018
prev sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh 
wrote:
 On 03/10/18 17:29, Stanislav Blinov wrote:
 OMG, that's so simple!!! Why didn't I think of it?

 Oh wait, I did.
Now I see why sometimes your posts are greeted with hostility.
Yes. I am actually sorry about that. I was responding to your assumption that I'm wrong. Had your post been phrased as "why didn't you", instead of "you're wrong wrong wrong" I wouldn't have responded that way. Like I said, I am sorry.
I am sorry as well since I wasn't clear in my initial post.
 Allow me to further illustrate with something that can be
written in D > today: I am not sure what you were trying to demonstrate, so instead I wanted to see if you succeeded. I added the following to your Tracker struct: ~this() { writefln("%s destructed", &this); assert(counter is null || counter is &localCounter); } I.e. - I am asserting if a move was not caught. The program fails to run on either ldc or dmd. To me, this makes perfect sense as for the way D is built. In essence, opAssign isn't guaranteed to run. Feel free to build a struct where that assert passes to convince me.
That's a slightly different issue here. Look at the output. The operator is being run, it can't *not* run, unlike postblit (ironically, right now it doesn't run on fixed-size arrays though). In fact, as soon as you define a destructor, the compiler will generate a by-value opAssign if you haven't defined one. That's a separate problem. Currently, presence of a destructor makes the compilers generate different code, because it cannot elide destruction of arguments, because explicit move semantics do not exist in the language. That's why I haven't included a destructor in the example to begin with.
 Here is the flaw in your logic:

     void opAssign(Tracker rhs)

 rhs is passed by value. This means that already at the point 
 opAssign is called, rhs *already* has a different address than 
 the one it was passed in with.
Currently that is only true if you define a destructor. That would not be true, however, if a move hook in any form existed in the language. That was my point. I only used opAssign as something resembling the supposed new behavior, not as a "look, it already works". In the presence of a move hook, 'rhs' would first have to pass through that hook, which will not take destructors into account at all. Consider your own DIP: what you're suggesting is the ability to take the address of the original when a move is taking place. My example shows that in the simplest case even today, address of the original is already the address of the argument. Except it cannot be enforced in any way right now. A move hook will have to enforce that, as it will have to be called for every move.
 I did not follow your logic on why this isn't so, but I don't 
 see how you can make it not so without changing the ABI quite 
 drastically.
The changes are literally the same as the ones you're proposing: "When moving a struct's instance, the compiler MUST call __move_post_blt giving it both new and old instances' addresses." That is the same that would have to happen with this(typeof(this) rhs), where &this is the address of new instance, and &rhs is the address of old instance, but there's no need for opPostMove then. I guess what I should've said from the start is that the semantics you're proposing fit nicely within one special function, instead of two. this(typeof(this)), of course, would need to be special in the ABI, but again, that's one special function instead of two. Let's take a step back for a moment and look at what should actually be happening for this hook to work (which you briefly mention in the DIP): 1. The compiler constructs the value. In your case, it constructs two: the original and the new one. In my case, it constructs the original and then passes it over to the move ctor (one blit potentially avoided). 2. It calls the hook (move ctor). 3. In your case, it calls the opPostMove. 4. In any case, it *doesn't* destruct the original. Ever. The alternative would be to force the programmer to put the original back into valid state, and suddenly we're back to C++ with all it's pleasantries. That last part is quite different from the current model, in which the compiler always destructs function arguments. That's why my example fails when a destructor is present. The other thing to note (again something that you mention but don't expand on), and that's nodding back to my comment about making move() and emplace() intrinsics, is that creating such a hook *will* invalidate current behavior of move(). Which is perhaps more easily fixed with your implementation, actually, *except* for the part about eliding destruction. Unions are unreliable for that unless we also change the spec that talks about them. But IMHO, it's something that should be fixed by not making these facilities built into the language.
Oct 03 2018
next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 17:43:08 UTC, Stanislav Blinov 
wrote:

 But IMHO, it's something that should be fixed by not making 
 these facilities built into the language.
s/not//
Oct 03 2018
prev sibling parent reply Shachar Shemesh <shachar weka.io> writes:
On 03/10/18 20:43, Stanislav Blinov wrote:
 On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh wrote:
 I.e. - I am asserting if a move was not caught. The program fails to 
 run on either ldc or dmd. To me, this makes perfect sense as for the 
 way D is built. In essence, opAssign isn't guaranteed to run. Feel 
 free to build a struct where that assert passes to convince me.
That's a slightly different issue here.
Well, I view this issue as a deal breaker. If you need to move the object *in order* to pass it to your move hook, then anything that requires knowing the address of the old instance will, by definition, not work.
 Look at the output. The operator is being run, it can't *not* run, 
Sure it can. Just look at the example I posted on the other thread (https://forum.dlang.org/post/pp2v16$1014$1 digitalmars.com). The hook you mention is downright disabled there. In fact, had that not been the case, this DIP would never have happened.
 
 Here is the flaw in your logic:

     void opAssign(Tracker rhs)

 rhs is passed by value. This means that already at the point opAssign 
 is called, rhs *already* has a different address than the one it was 
 passed in with.
Currently that is only true if you define a destructor.
No, that's not true. Try printing the instance's address in the constructor and again in your operator.
 In the presence of a move hook, 'rhs' 
 would first have to pass through that hook, which will not take 
 destructors into account at all.
I'm sorry, I'm not following. What is the difference between what you're proposing and opPostMove as defined?
 Consider your own DIP: what you're suggesting is the ability to take the 
 address of the original when a move is taking place. My example shows 
 that in the simplest case even today, address of the original is already 
 the address of the argument.
This is the run I got: $ ./movetest2 Address of temporary is 'b382e390', counter points to 'b382e390' ... which is '0' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to '8eb82b60' ... which is '-966984906800' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to 'b382e390' ... which is '0' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to '8eb82b60' ... which is '-966984906800' bytes from the address of temporary. I'm not sure what I should have seen, or what I should have concluded from it. This is your original program, unmodified.
 The changes are literally the same as the ones you're proposing:
 
 "When moving a struct's instance, the compiler MUST call __move_post_blt 
 giving it both new and old instances' addresses."
 
 That is the same that would have to happen with this(typeof(this) rhs), 
 where &this is the address of new instance, and &rhs is the address of 
 old instance, but there's no need for opPostMove then. I guess what I 
 should've said from the start is that the semantics you're proposing fit 
 nicely within one special function, instead of two.
Except, like I said, it's not working for me, and I find it hard to understand how it *can* work (inlining notwithstanding), which is why I did not propose it.
 
 this(typeof(this)), of course, would need to be special in the ABI, but 
 again, that's one special function instead of two.
No. My proposal requires one amendment to argument passing in the ABI, but no special cases at all. Changes to the ABI are not the same as changes to the run time library.
 
 Let's take a step back for a moment and look at what should actually be 
 happening for this hook to work (which you briefly mention in the DIP):
 
 1. The compiler constructs the value. In your case, it constructs two:
It does not. It copies the bits from one to the other. This also means that a struct where some of its members have a hook is copied wholesale and patched, which is typically faster than copying in parts.
 the original and the new one. In my case, it constructs the original and 
 then passes it over to the move ctor (one blit potentially avoided).
 2. It calls the hook (move ctor).
I'm not sure I follow you on that one. What did you mean?
 3. In your case, it calls the opPostMove.
In all cases, you need to call the hook, whatever it is, for those structs that have it, and do some default handling for those that don't.
 4. In any case, it *doesn't* destruct the original. Ever. The 
 alternative would be to force the programmer to put the original back 
 into valid state, and suddenly we're back to C++ with all it's 
 pleasantries.
 
 That last part is quite different from the current model, in which the 
 compiler always destructs function arguments. That's why my example 
 fails when a destructor is present.
Like I said above, I don't think that's correct.
 
 The other thing to note (again something that you mention but don't 
 expand on), and that's nodding back to my comment about making move() 
 and emplace() intrinsics, is that creating such a hook *will* invalidate 
 current behavior of move(). Which is perhaps more easily fixed with your 
 implementation, actually, *except* for the part about eliding 
 destruction.
Convince me that the pointers indeed don't change when passed to the function, and then we can discuss whether this point is correct. Shachar
Oct 03 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 18:58:37 UTC, Shachar Shemesh 
wrote:
 On 03/10/18 20:43, Stanislav Blinov wrote:
 On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh 
 wrote:
 I.e. - I am asserting if a move was not caught. The program 
 fails to run on either ldc or dmd. To me, this makes perfect 
 sense as for the way D is built. In essence, opAssign isn't 
 guaranteed to run. Feel free to build a struct where that 
 assert passes to convince me.
That's a slightly different issue here.
Well, I view this issue as a deal breaker. If you need to move the object *in order* to pass it to your move hook, then anything that requires knowing the address of the old instance will, by definition, not work.
I feel like we're still not on the same page here. this(typeof(this)) doesn't work like a move hook in D right now. I'm *suggesting* making that be a move ctor, instead of opPostMove from your DIP (see below).
 Look at the output. The operator is being run, it can't *not* 
 run,
Sure it can. Just look at the example I posted on the other thread (https://forum.dlang.org/post/pp2v16$1014$1 digitalmars.com). The hook you mention is downright disabled there. In fact, had that not been the case, this DIP would never have happened.
Yup, we're definitely not on the same page :) That's not what I'm talking about at all.
 Here is the flaw in your logic:

     void opAssign(Tracker rhs)

 rhs is passed by value. This means that already at the point 
 opAssign is called, rhs *already* has a different address 
 than the one it was passed in with.
Currently that is only true if you define a destructor.
No, that's not true. Try printing the instance's address in the constructor and again in your operator.
It *is* true when the type doesn't have a destructor. Extending that to a move hook, it will also be true because destruction will be elided. I know what you're talking about, that happens for types that have destructors.
 In the presence of a move hook, 'rhs' would first have to pass 
 through that hook, which will not take destructors into 
 account at all.
I'm sorry, I'm not following. What is the difference between what you're proposing and opPostMove as defined?
1. with this(typeof(this)) the type of argument would never change. With opPostMove, it may. Remember that 'is(Tracker == const(Tracker))' is false. 2. you won't have to always move all the bits unconditionally. 3. symmetry with this(this)
 This is the run I got:
 $ ./movetest2
 Address of temporary is 'b382e390', counter points to 'b382e390'
 ... which is '0' bytes from the address of temporary.
 ...I'm not sure what I should have seen, or what I should have 
 concluded from it. This is your original program, unmodified.
This illustrates the intended behavior of a move hook if it existed in the language. The 'rhs' that was passed to the call was constructed at that same address (&rhs.counter == &rhs.localCounter). I.e. this is how I'm suggesting a this(typeof(this)) *could* work.
 this(typeof(this)), of course, would need to be special in the 
 ABI, but again, that's one special function instead of two.
No. My proposal requires one amendment to argument passing in the ABI, but no special cases at all. Changes to the ABI are not the same as changes to the run time library.
How so? Or, more to the point, what's argument passing OR runtime have to do with this?
 Let's take a step back for a moment and look at what should 
 actually be happening for this hook to work (which you briefly 
 mention in the DIP):
 
 1. The compiler constructs the value. In your case, it 
 constructs two:
It does not. It copies the bits from one to the other.
Poor choice of words on my part. It *creates* two. Whereas with this(typeof(this)) no implicit copying of bits is required, a-la a C++ move constructor. The programmer is free to choose the bits they need, or do a blit-and-patch if so desired. Except that unlike a C++ move constructor, no state bookkeeping would be necessary (same is true with your DIP as is).
 the original and the new one. In my case, it constructs the 
 original and then passes it over to the move ctor (one blit 
 potentially avoided).
 2. It calls the hook (move ctor).
I'm not sure I follow you on that one. What did you mean?
In your case, that would be when it calls __move_post_blt.
 3. In your case, it calls the opPostMove.
In all cases, you need to call the hook, whatever it is, for those structs that have it, and do some default handling for those that don't.
Correct, which would just be whatever the compilers already do.
 4. In any case, it *doesn't* destruct the original. Ever. The 
 alternative would be to force the programmer to put the 
 original back into valid state, and suddenly we're back to C++ 
 with all it's pleasantries.
 
 That last part is quite different from the current model, in 
 which the compiler always destructs function arguments. That's 
 why my example fails when a destructor is present.
Like I said above, I don't think that's correct.
I'm assuming you're talking about why the example fails with the destructor. That's because what the DIP and you and me are currently discussing do not exist in the language. You add a destructor, you force the compiler to construct an additional temporary. That wouldn't happen if we were looking at an actual move hook, not that haphazard attempt to illustrate how one could work.
 The other thing to note (again something that you mention but 
 don't expand on), and that's nodding back to my comment about 
 making move() and emplace() intrinsics, is that creating such 
 a hook *will* invalidate current behavior of move(). Which is 
 perhaps more easily fixed with your implementation, actually, 
 *except* for the part about eliding destruction.
Convince me that the pointers indeed don't change when passed to the function, and then we can discuss whether this point is correct.
They don't in my unmodified example. The reason they do at all are (1) destructors and (2) under-specification. The (1) won't be an issue for a move ctor, as the compiler won't need to destruct the original, and the (2) would be obviously avoided for types that do define the hook. I guess I've just created more confusion than explanation, so to reiterate: You're proposing: 1. Make the compiler emit extra code every time a struct is moved. 2. Allow users to provide a custom opPostBlit that takes an address of the original, called by (1). I'm proposing: 1. Allow users to provide a custom ctor with a signature this(typeof(this) rhs). If none is provided but any member of the struct has one, generate one implicitly. 2. Such ctor should be called whenever a value needs be moved, with &this being the target address and &rhs the source address.
Oct 03 2018
parent reply Shachar Shemesh <shachar weka.io> writes:
On 03/10/18 23:25, Stanislav Blinov wrote:
 It *is* true when the type doesn't have a destructor. Extending that to 
 a move hook, it will also be true because destruction will be elided.
 I know what you're talking about, that happens for types that have 
 destructors.
No, destructors have nothing to do with it, as well they shouldn't. The whole point of D moving structs around is that no destruction is needed. It took me a while to figure out why your program does appear to work. At first I thought it was because of inlining, but that was wrong. The reason your test case works (sometimes, if you don't breath on it too heavily) is because the object is actually moved twice. Once when returning from the function into the variable, and another when copied into opAssign's argument. This results in it returning to its original address. If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working. You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong. To verify my guess is right, I tried the following change: add to createCounter and createCounterNoNRV in your original program (no destructors) the following two lines: int a; write(a); You have added another local variable to the functions, but otherwise changed absolutely nothing. You will notice your program now has an offset. Shachar
Oct 03 2018
next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
 Manu,  Jonathan M Davis

 GNU's std::string implementation stores an interior pointer! >_<
it's not just GNU's std::string ; it can crop up in other places, see https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep) On Wed, Oct 3, 2018 at 8:10 PM Shachar Shemesh via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 On 03/10/18 23:25, Stanislav Blinov wrote:
 It *is* true when the type doesn't have a destructor. Extending that to
 a move hook, it will also be true because destruction will be elided.
 I know what you're talking about, that happens for types that have
 destructors.
No, destructors have nothing to do with it, as well they shouldn't. The whole point of D moving structs around is that no destruction is needed. It took me a while to figure out why your program does appear to work. At first I thought it was because of inlining, but that was wrong. The reason your test case works (sometimes, if you don't breath on it too heavily) is because the object is actually moved twice. Once when returning from the function into the variable, and another when copied into opAssign's argument. This results in it returning to its original address. If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working. You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong. To verify my guess is right, I tried the following change: add to createCounter and createCounterNoNRV in your original program (no destructors) the following two lines: int a; write(a); You have added another local variable to the functions, but otherwise changed absolutely nothing. You will notice your program now has an offset. Shachar
Oct 03 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Oct 3, 2018 at 11:00 PM Timothee Cour via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
  Manu,  Jonathan M Davis

 GNU's std::string implementation stores an interior pointer! >_<
it's not just GNU's std::string ; it can crop up in other places, see https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep)
Sure. Certainly, it shows up in C++ fairly often... but I'm working on the STL containers, and I didn't think there were any implementations that did that (because inefficient use of space), but turns out the GNU implementation bent me over, at least as far as I've encountered yet. It's kinda got me stuck.
Oct 03 2018
prev sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh 
wrote:

 If you do *anything* to that program, and that includes even 
 changing its compilation flags (try enabling inlining), it will 
 stop working.

 You should have known that when you found out it doesn't work 
 on ldc: ldc and dmd use the same front-end. If you think 
 something works fundamentally different between the two, you 
 are probably wrong.
For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much.
Oct 04 2018
parent reply Shachar Shemesh <shachar weka.io> writes:
On 04/10/18 11:05, Stanislav Blinov wrote:
 On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote:
 
 If you do *anything* to that program, and that includes even changing 
 its compilation flags (try enabling inlining), it will stop working.

 You should have known that when you found out it doesn't work on ldc: 
 ldc and dmd use the same front-end. If you think something works 
 fundamentally different between the two, you are probably wrong.
For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much.
The example isn't brittle. It is simply not an example. If you want to leave it out, however, then I think you should submit an orderly proposal. The changes you seem to be suggesting have consequences that go beyond what I think you understand, and there can be no serious discussion of it while it is not clear from your posts which part of what you say is the relevant one. Shachar
Oct 04 2018
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Thursday, 4 October 2018 at 08:10:31 UTC, Shachar Shemesh 
wrote:
 On 04/10/18 11:05, Stanislav Blinov wrote:
 On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh 
 wrote:
 
 [...]
For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much.
The example isn't brittle. It is simply not an example. If you want to leave it out, however, then I think you should submit an orderly proposal. The changes you seem to be suggesting have consequences that go beyond what I think you understand, and there can be no serious discussion of it while it is not clear from your posts which part of what you say is the relevant one. Shachar
While I want to thank you both, about the quality of this thread, what kind of "consequences that go beyond what I think you understand" are you thinking of? Can you give an example? Thanks, Paolo
Oct 04 2018
parent reply Shachar Shemesh <shachar weka.io> writes:
On 04/10/18 11:16, Paolo Invernizzi wrote:
 While I want to thank you both, about the quality of this thread, what 
 kind of "consequences that go beyond what I think you understand" are 
 you thinking of? Can you give an example?
Assuming I understand Stanislav's proposal correctly (an assumption I'm reluctant to make, hence my request for something more formal), it boils down to two points: * move the data as part of the call hook rather than before * Use a different name and signature on the hook function The first one we can argue for or against. My original proposal was phrased the way it was precisely because that's the way copying works in D (copy first, patch the data later). About a week after I submitted it, Andrei came forward with requesting to move to copy constructors. The second, to me, is a non-starter. The only way you'd get a function who's signature is: void someName(Type rhs); But which actually maintains rhs's address from before the call is if the compiler treats "someName" as a special case, and emits code which would normally be emitted for the function: void someName(ref Type rhs); That's why it was important for me to clear up whether there is *ever* a case in the current language where that happens (answer: only by accident, which is the same as saying "no"). So to get that to work, you'd need to insert a special case into the ABI of the language: if the function's name is someName, treat it differently. At this point, you might as well call a spade a spade, and just give the function that signature explicitly. s/someName/opPostMove/, and you get DIP 1014 (as far as point #2 is concerned). Shachar
Oct 04 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 4 October 2018 at 08:32:44 UTC, Shachar Shemesh 
wrote:
 On 04/10/18 11:16, Paolo Invernizzi wrote:
 While I want to thank you both, about the quality of this 
 thread, what kind of "consequences that go beyond what I think 
 you understand" are you thinking of? Can you give an example?
Assuming I understand Stanislav's proposal correctly (an
Yes it seems you do.
 assumption I'm reluctant to make, hence my request for 
 something more formal), it boils down to two points:

 * move the data as part of the call hook rather than before
 * Use a different name and signature on the hook function
Yes, exactly.
 The first one we can argue for or against. My original proposal 
 was phrased the way it was precisely because that's the way 
 copying works in D (copy first, patch the data later). About a 
 week after I submitted it, Andrei came forward with requesting 
 to move to copy constructors.
 The second, to me, is a non-starter. The only way you'd get a 
 function who's signature is:
 void someName(Type rhs);
 But which actually maintains rhs's address from before the call 
 is if the compiler treats "someName" as a special case, and 
 emits code which would normally be emitted for the function:
 void someName(ref Type rhs);
It would have to be special if you don't want to leave room for the compiler implementors. The calling convention for particular types (i.e. those that do have a move hook defined) would have to be enforced in some way. See the neighbor thread wrt move semantics by kinke.
 That's why it was important for me to clear up whether there is 
 *ever* a case in the current language where that happens 
 (answer: only by accident, which is the same as saying "no").
Which is, however, not a reason to formalize it and make it a requirement for an isolated specific case, such as this one, utilizing a syntax that is currently not used by the language. As opposed to trying to fit existing language semantics to something that the language didn't seem to want to allow in the first place.
 So to get that to work, you'd need to insert a special case 
 into the ABI of the language: if the function's name is 
 someName, treat it differently.
Yes, that's what I'm talking about.
 At this point, you might as well call a spade a spade, and just 
 give the function that signature explicitly. 
 s/someName/opPostMove/, and you get DIP 1014 (as far as point 
 #2 is concerned).
Oct 04 2018
parent reply Shachar Shemesh <shachar weka.io> writes:
On 04/10/18 13:43, Stanislav Blinov wrote:
 * move the data as part of the call hook rather than before
 * Use a different name and signature on the hook function
Yes, exactly.
 
 It would have to be special if you don't want to leave room for the 
 compiler implementors.
That's not how standards work. If you don't want compiler implementors to have a choice in the matter, you put MUST in the specs. Doing anything else is, by and large, considered harmful.
 The calling convention for particular types (i.e. 
 those that do have a move hook defined) would have to be enforced in 
 some way. See the neighbor thread wrt move semantics by kinke.
Two distinct things. Kinke was talking about how to pass a struct through the ABI. You are talking about special-casing a specific name. Not to mention, your special case is to transform it to something you can *already* specify in the language. Why?
 Which is, however, not a reason to formalize it and make it a 
 requirement for an isolated specific case, such as this one, utilizing a 
 syntax that is currently not used by the language.
There is positively nothing in DIP 1014 that is "syntax not used by the language". Quite the contrary.
 As opposed to trying 
 to fit existing language semantics to something that the language didn't 
 seem to want to allow in the first place.
Formalize it as a suggestion, and we can discuss the "as opposed to". Like I said, I think there's a lot you're glossing over here (such as backwards compatibility). Shachar
Oct 04 2018
parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 4 October 2018 at 12:08:38 UTC, Shachar Shemesh 
wrote:

 Two distinct things. Kinke was talking about how to pass a 
 struct through the ABI. You are talking about special-casing a 
 specific name.
Not just name, but argument passing as well.
 Not to mention, your special case is to transform it to 
 something you can *already* specify in the language. Why?
Because that syntax pertains specifically to construction, which is what a compiler move is; is not currently used by the language (the fact that the compiler doesn't error on it is an oversight); enforces calling convention.
 Which is, however, not a reason to formalize it and make it a 
 requirement for an isolated specific case, such as this one, 
 utilizing a syntax that is currently not used by the language.
There is positively nothing in DIP 1014 that is "syntax not used by the language". Quite the contrary.
Which is what I said in the very next sentence, so I'm not sure what your point is here. It's like we're having a discussion but we aren't at the same time.
 As opposed to trying to fit existing language semantics to 
 something that the language didn't seem to want to allow in 
 the first place.
 Formalize it as a suggestion, and we can discuss the "as 
 opposed to".
Alright, let's get back to it after the weekend then.
 Like I said, I think there's a lot you're glossing over here 
 (such as backwards compatibility).
Backwards compatibility? With what, exactly? Non-existing explicit moves?
Oct 04 2018
prev sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 14:07:58 UTC, Shachar Shemesh 
wrote:

 If you read the DIP, you will notice that the *address* in 
 which the old instance resides is quite important...
Allow me to further illustrate with something that can be written in D today: import std.stdio; struct Tracker { static int globalCounter; int localCounter; int* counter; this (bool local) { if (local) counter = &localCounter; else counter = &globalCounter; } // this should be this(Tracker rhs) void opAssign(Tracker rhs) { // note: taking address of local parameter // note: LDC will already 'optimize' the move which in the absence // of any move hooks will mess up the address; try with DMD printf("Address of temporary is '%x', counter points to '%x'\n", &rhs, rhs.counter); auto d = cast(void*) rhs.counter - cast(void*) &rhs; printf("... which is '%ld' bytes from the address of temporary.\n", d); localCounter = rhs.localCounter; counter = rhs.counter; if (counter is &rhs.localCounter) counter = &localCounter; } } auto createCounter(bool local = true) { Tracker result = Tracker(local); return result; } auto createCounterNoNRV(bool local = true) { return Tracker(local); } void main() { Tracker stale1, stale2; stale1 = createCounter(); stale2 = createCounter(false); Tracker stale3, stale4; stale3 = createCounterNoNRV(); stale4 = createCounterNoNRV(false); } If you run the above with DMD, you'll see what I mean about obviating the address. If we get this(typeof(this)) (that is *always* called on move) into the language, the behavior would be set in stone regardless of compiler.
Oct 03 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Shachar, as I don't see a better place of discussing that DIP at
 the moment, I'll pour some observations and thoughts in here if
 you don't mind, will add some comments on GitHub later.
 As I see it right now, it's a case of over-engineering of a quite
 simple concept.

 1. A new function, called __move_post_blt, will be added to
 DRuntime.

 That's unnecessary, if not downright harmful for the language. We
 should strive to remove things from DRuntime, not add to it. The
 core language should deal with type memory, not a .so or dll. And
 it's extraneous, because...

 2 and 3. onPostMove and __move_post_blt:

 They're unnecessary as well. All that's required is to allow a
 by-value constructor, e.g:

 struct S {
      this(S rhs);
 }

 Any function in D that has a signature of the form

 ReturnType foo(Type x);

 in C++ would have an equivalent signature of

 ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
What are you talking about? Equivalent C++ is: ReturnType foo(Type x); It's impossible to perform copy elision when passing an lvalue by-value *to* a function, but that's a case where you depend on move semantics. Also, within the function that receives an argument by value, you depend on move to construct something with that argument. Type&& passes a reference, which means you can perform the move direct from source to destination, without a stop at the middle man. This all has nothing to do with Walter's surprising claim that "as the DMD compiler doesn't actually move structs"... I'm still trying to understand this statement.
Oct 03 2018
parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 3 October 2018 at 18:38:50 UTC, Manu wrote:
 On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via 
 Digitalmars-d <digitalmars-d puremagic.com> wrote:
 Any function in D that has a signature of the form

 ReturnType foo(Type x);

 in C++ would have an equivalent signature of

 ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
What are you talking about? Equivalent C++ is: ReturnType foo(Type x);
C++ has rvalue references, move semantics are explicit. D doesn't have any of that. Perhaps I wasn't quite clear in that above statement though. Given some type Bar, compare these two calls in C++ in D, and tell me, which signature in C++ should correspond to D? I'm not talking about ABI, I'm talking about semantics. foo(std::move(bar)); // C++ foo(move(bar)); // D remembering that D's move() doesn't call postblit. void foo(Bar bar) wouldn't satisfy that last bit, would it? Yes, the semantics are different, as in D the move occurs before the call. But in D right now you *must* assume that the argument may have been moved, with all the consequences the language currently entails (and some of which the DIP attempts to resolve), whereas in C++ you can be explicit about it via overloading for rvalue references.
 It's impossible to perform copy elision when passing an lvalue
 by-value *to* a function, but that's a case where you depend on 
 move semantics.
Of course it's impossible. I'm not sure I understand your point here.
 Also, within the function that receives an argument by value, 
 you
 depend on move to construct something with that argument.

 Type&& passes a reference, which means you can perform the move 
 direct from source to destination, without a stop at the middle 
 man.
Yup, no argument there.
Oct 03 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Oct 2, 2018 at 2:20 AM Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 9/29/2018 9:34 PM, Manu wrote:
 Who knows about DIP 1014? (struct move hook)
 Is it well received? Is it likely to be accepted soon?

 I'm working on the std::string binding, it's almost finished... but
 then I hit a brick wall.
 GNU's std::string implementation stores an interior pointer! >_<
The rationale behind D allowing structs to be moveable is to enable a copying garbage collector.
Do we do that? What's the use of that? Does the opPostMove() concept not work here? If the GC wanted to move something, it can call it like any other code?
 Some solutions to this problem:

 1. Don't allow moving of C++ structs
std::string, std::vector, etc are quite impotent without move semantics. I'm pretty sure the first time I ever started using those containers was around 2015 when we became secure we could use C++11 in our code (supported by all compiler vendors). Before that, they were banned and we had alternative solutions.
 2. Add a struct attribute that means "not moveable"
They must be movable though.
 3. DIP 1014, which is add a __move_post_blit() function (most complex solution)
It's alleged you're working through the DIP... what's the story there? Are you unhappy with it?
 4. Use copy/destruct for C++ structs that have copy constructors (this is the
 old C++ solution, and is less efficient than the move constructor)
If that's where we land on this, I'll struggle to find value in my work. Containers become as impotent as back before 2011 :/
 Pragmatically, I suggest for the moment just ignore the problem, file a bug
 report for std::string, and move on.
But dangling pointer is an instant crash/memory corruption... it's a pretty bad 'bug'.
Oct 02 2018
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 10/2/18 1:51 PM, Manu wrote:
 But dangling pointer is an instant crash/memory corruption... it's a
 pretty bad 'bug'.
Yeah doesn't sound very brilliant. I think such a workaround wouldn't fare well. To keep momentum while we mull over a solution to this I suggest you look at porting other data structures.
Oct 02 2018
prev sibling parent reply Tremor <tesTremor gmail.com> writes:
On Sunday, 30 September 2018 at 04:34:20 UTC, Manu wrote:
 Who knows about DIP 1014? (struct move hook)
 Is it well received? Is it likely to be accepted soon?

 I'm working on the std::string binding, it's almost finished... 
 but
 then I hit a brick wall.
 GNU's std::string implementation stores an interior pointer! >_<

 No other implementation does this. It's a really bad 
 implementation actually, quite inefficient. It could make 
 better use of its space for small-strings if it wasn't wasting 
 8-bytes for an interior pointer to a small string buffer...

 Anyway, I'm blocked until this DIP is accepted; so, is it 
 looking promising?
Is any update for this ? DIP1014 is fatal for implement safe ref count, I don't trust a language call them self system language without safe ref count. This is also a major block for implement GNU CPP std::string, cpp ABI compatible is another import goal for D. I just with I has the skill to work on this, If nobody has no time to work on this. maybe some one can give a guide about how to implement this ? like the step to do: step 1: example to add some build in id for opPostMove step 2: which part code to modify to hook opPostMove with rvalue or lvalue move step 3: how to glue it with DMD and LDC backend to generate asm code. Maybe there should be a new thread to talk about this. The github user thewilsonator is like a hero to me, maybe when he get free time can take a look for this.
Jul 01
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 2 July 2019 at 05:32:07 UTC, Tremor wrote:
 Is any update for this ?
Les De Ridder has implemented the druntime function and made phobos use it for its move function, all thats left is to make the compiler insert a call to it when it moves structs around.
 DIP1014 is fatal for implement safe ref count,  I don't trust a 
 language call them self system language without safe ref count.

 This is also a major block for implement GNU CPP std::string, 
 cpp ABI compatible is  another import goal for D.

 I just with I has the skill to work on this,  If nobody has no 
 time to work on this. maybe some one can give a guide about how 
 to implement this ?

 like the step to do:

 step 1:  example to add some build in id for opPostMove
That would go here: https://github.com/dlang/dmd/blob/master/src/dmd/id.d#L63 you'd need one for https://github.com/dlang/druntime/blob/master/src/object.d#L151 as well
 step 2: which part code to modify to hook opPostMove with 
 rvalue or lvalue move
Would probably want to insert the call in, Razvan Nitu would probably know better/in more detail. https://github.com/dlang/dmd/blob/b359735d1034c77aa5b59dff9a8b815fb3eb14c9/src/dmd/expression.d#L468
 step 3: how to glue it with DMD and LDC backend to generate asm 
 code.
This should be no problem, all that is needed is to insert a call to, https://github.com/dlang/druntime/blob/master/src/object.d#L151 Its all front end stuff, i.e. LDC will be able to use whatever the front end does with no problem.
 Maybe there should be a new thread to talk about this.

 The github user thewilsonator is like a hero to me, maybe when 
 he get free time can take a look for this.
I'm humbled, but a bit busy. Don't let me stop you opening a PR though. I'm sure Razvan and I can try to help you there.
Jul 02
parent reply Tremor <tesTremor gmail.com> writes:
On Tuesday, 2 July 2019 at 07:19:10 UTC, Nicholas Wilson wrote:
 That would go here:

 https://github.com/dlang/dmd/blob/master/src/dmd/id.d#L63
Thanks for the tips, I manage to made a patch: https://paste.ofcode.org/xGnhS2KcXvtmENwyRPQXua The rest hard work is made the code to be call from https://github.com/dlang/dmd/blob/b359735d1034c77aa5b59dff9a8b815fb3eb14c9/src/d d/expression.d#L468 or src/dmd/dinterpret.d (I need help here). On Tuesday, 2 July 2019 at 07:48:06 UTC, RazvanN wrote:
 After implementing the copy constructor as a substitute for the 
 postblit, we realized that the fundamental flaw that the 
 postblit had (automatic copying) would also manifest in the 
 case of opPostMove; in Shachars' there is no mention what 
 happens when the source and destination are differently 
 qualified. I suspect that before implementing the DIP we need 
 to sort this out and it might be preferable to imlement a move 
 constructor a la C++ rather then rely on automatic moving.
Correct me if I am wrong. I think the opPostMove should always use for the same qualified type. for example if a function return a shared(struct), then the opPostMove also accept shared(struct) from a shared(struct) instance. Please feel free to use this patch https://paste.ofcode.org/xGnhS2KcXvtmENwyRPQXua to made PR and start the work. I can made the PR but I am afraid this is all I can do for this task(I will try, but I can not understand any of dinterpret.d/expression.d).
Jul 02
parent 12345swordy <alexanderheistermann gmail.com> writes:
On Tuesday, 2 July 2019 at 08:05:35 UTC, Tremor wrote:
 On Tuesday, 2 July 2019 at 07:19:10 UTC, Nicholas Wilson wrote:
 That would go here:

 https://github.com/dlang/dmd/blob/master/src/dmd/id.d#L63
Thanks for the tips, I manage to made a patch: https://paste.ofcode.org/xGnhS2KcXvtmENwyRPQXua The rest hard work is made the code to be call from https://github.com/dlang/dmd/blob/b359735d1034c77aa5b59dff9a8b815fb3eb14c9/src/d d/expression.d#L468 or src/dmd/dinterpret.d (I need help here). On Tuesday, 2 July 2019 at 07:48:06 UTC, RazvanN wrote:
 After implementing the copy constructor as a substitute for 
 the postblit, we realized that the fundamental flaw that the 
 postblit had (automatic copying) would also manifest in the 
 case of opPostMove; in Shachars' there is no mention what 
 happens when the source and destination are differently 
 qualified. I suspect that before implementing the DIP we need 
 to sort this out and it might be preferable to imlement a move 
 constructor a la C++ rather then rely on automatic moving.
Correct me if I am wrong. I think the opPostMove should always use for the same qualified type. for example if a function return a shared(struct), then the opPostMove also accept shared(struct) from a shared(struct) instance. Please feel free to use this patch https://paste.ofcode.org/xGnhS2KcXvtmENwyRPQXua to made PR and start the work. I can made the PR but I am afraid this is all I can do for this task(I will try, but I can not understand any of dinterpret.d/expression.d).
Make the PR, so that others can review it.
Jul 02
prev sibling parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Tuesday, 2 July 2019 at 05:32:07 UTC, Tremor wrote:
 On Sunday, 30 September 2018 at 04:34:20 UTC, Manu wrote:
 Who knows about DIP 1014? (struct move hook)
 Is it well received? Is it likely to be accepted soon?

 I'm working on the std::string binding, it's almost 
 finished... but
 then I hit a brick wall.
 GNU's std::string implementation stores an interior pointer!
_<
No other implementation does this. It's a really bad implementation actually, quite inefficient. It could make better use of its space for small-strings if it wasn't wasting 8-bytes for an interior pointer to a small string buffer... Anyway, I'm blocked until this DIP is accepted; so, is it looking promising?
Is any update for this ? DIP1014 is fatal for implement safe ref count, I don't trust a language call them self system language without safe ref count. This is also a major block for implement GNU CPP std::string, cpp ABI compatible is another import goal for D. I just with I has the skill to work on this, If nobody has no time to work on this. maybe some one can give a guide about how to implement this ? like the step to do: step 1: example to add some build in id for opPostMove step 2: which part code to modify to hook opPostMove with rvalue or lvalue move step 3: how to glue it with DMD and LDC backend to generate asm code. Maybe there should be a new thread to talk about this. The github user thewilsonator is like a hero to me, maybe when he get free time can take a look for this.
After implementing the copy constructor as a substitute for the postblit, we realized that the fundamental flaw that the postblit had (automatic copying) would also manifest in the case of opPostMove; in Shachars' there is no mention what happens when the source and destination are differently qualified. I suspect that before implementing the DIP we need to sort this out and it might be preferable to imlement a move constructor a la C++ rather then rely on automatic moving.
Jul 02
next sibling parent Tremor <tesTremor gmail.com> writes:
On Tuesday, 2 July 2019 at 07:48:06 UTC, RazvanN wrote:
 After implementing the copy constructor as a substitute for the 
 postblit, we realized that the fundamental flaw that the 
 postblit had (automatic copying) would also manifest in the 
 case of opPostMove; in Shachars' there is no mention what 
 happens when the source and destination are differently 
 qualified. I suspect that before implementing the DIP we need 
 to sort this out and it might be preferable to imlement a move 
 constructor a la C++ rather then rely on automatic moving.
Maybe this is work for us ? 1. opPostMove always apply to same qualified type 2. If the opPostMove apply to immutable instance, it should call copyConstructor instead. The diff from copyConstructor with opPostMove, the original instance is dropped without call dtor. 3. The original instance should be scope instance without any escape ref.
Jul 02
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jul 2, 2019 at 5:51 PM RazvanN via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 2 July 2019 at 05:32:07 UTC, Tremor wrote:
 On Sunday, 30 September 2018 at 04:34:20 UTC, Manu wrote:
 Who knows about DIP 1014? (struct move hook)
 Is it well received? Is it likely to be accepted soon?

 I'm working on the std::string binding, it's almost
 finished... but
 then I hit a brick wall.
 GNU's std::string implementation stores an interior pointer!
_<
No other implementation does this. It's a really bad implementation actually, quite inefficient. It could make better use of its space for small-strings if it wasn't wasting 8-bytes for an interior pointer to a small string buffer... Anyway, I'm blocked until this DIP is accepted; so, is it looking promising?
Is any update for this ? DIP1014 is fatal for implement safe ref count, I don't trust a language call them self system language without safe ref count. This is also a major block for implement GNU CPP std::string, cpp ABI compatible is another import goal for D. I just with I has the skill to work on this, If nobody has no time to work on this. maybe some one can give a guide about how to implement this ? like the step to do: step 1: example to add some build in id for opPostMove step 2: which part code to modify to hook opPostMove with rvalue or lvalue move step 3: how to glue it with DMD and LDC backend to generate asm code. Maybe there should be a new thread to talk about this. The github user thewilsonator is like a hero to me, maybe when he get free time can take a look for this.
After implementing the copy constructor as a substitute for the postblit, we realized that the fundamental flaw that the postblit had (automatic copying) would also manifest in the case of opPostMove; in Shachars' there is no mention what happens when the source and destination are differently qualified. I suspect that before implementing the DIP we need to sort this out and it might be preferable to imlement a move constructor a la C++ rather then rely on automatic moving.
Wouldn't it just call the opPostMove qualified identical to the type that was moved? Is it that you can't do qualifier conversions across a move, but you might like to? This kinda feels like a slightly separate issue.
Jul 02
parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Tuesday, 2 July 2019 at 23:05:14 UTC, Manu wrote:
 On Tue, Jul 2, 2019 at 5:51 PM RazvanN via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Tuesday, 2 July 2019 at 05:32:07 UTC, Tremor wrote:
 [...]
After implementing the copy constructor as a substitute for the postblit, we realized that the fundamental flaw that the postblit had (automatic copying) would also manifest in the case of opPostMove; in Shachars' there is no mention what happens when the source and destination are differently qualified. I suspect that before implementing the DIP we need to sort this out and it might be preferable to imlement a move constructor a la C++ rather then rely on automatic moving.
Wouldn't it just call the opPostMove qualified identical to the type that was moved? Is it that you can't do qualifier conversions across a move, but you might like to? This kinda feels like a slightly separate issue.
It is not a separate issue. Currently, the compiler may perform a move that co-occurs with an implicit conversion: struct T { ... } T fun(); const x = fun(); // a move may occur together with a qualifier change This is valid code. With DIP1014 it becomes invalid since the signature of __move_post_blt is : void __move_post_blt(S)(ref S newLocation, ref S oldLocation) nothrow if( is(S==struct) ); The above case is not even mentioned in the DIP and it should.
Jul 03
parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Wednesday, 3 July 2019 at 08:27:49 UTC, RazvanN wrote:
 On Tuesday, 2 July 2019 at 23:05:14 UTC, Manu wrote:
 On Tue, Jul 2, 2019 at 5:51 PM RazvanN via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
Also there is the problem of typechecking the move operator as a function.Let's take a look at a slightly modified version of the example that Shachar provided: struct Tracker { static uint globalCounter; uint localCounter; uint* counter; disable this(this); this(bool local) { localCounter = 0; if( local ) counter = &localCounter; else counter = &globalCounter; } void increment() { (*counter)++; } //void opPostMove(const ref Tracker oldLocation) { void opPostMove(immutable ref Tracker oldLocation) immutable { if( counter is &oldLocation.localCounter ) counter = &localCounter; } } This code will not compile even though it should. And here we open the door of the postblit problems.
Jul 03
parent reply aliak <something something.com> writes:
On Wednesday, 3 July 2019 at 09:31:38 UTC, RazvanN wrote:
 On Wednesday, 3 July 2019 at 08:27:49 UTC, RazvanN wrote:
 [...]
Also there is the problem of typechecking the move operator as a function.Let's take a look at a slightly modified version of the example that Shachar provided: [...]
Is there any situation where a move-related hook will do anything other than "fix" indirections in the destination of the move?
Jul 03
parent a11e99z <black80 bk.ru> writes:
On Wednesday, 3 July 2019 at 11:18:20 UTC, aliak wrote:
 On Wednesday, 3 July 2019 at 09:31:38 UTC, RazvanN wrote:
 On Wednesday, 3 July 2019 at 08:27:49 UTC, RazvanN wrote:

 Also there is the problem of typechecking the move operator as 
 a function.Let's take a look at a slightly modified version of 
 the example that Shachar provided:
Is there any situation where a move-related hook will do anything other than "fix" indirections in the destination of the move?
copy constructor or copy+postblit can be very expensive operations in some cases, and I am in doubt that compiler generate code only with copy without some move/memmoves. auto x = fun(); // a move may occur so at code generation levels in any case we have move/memmove operations, why not pull out it to user level where user can control it in some cases more predictably and easily? for some interop projects (like DPP https://dlang.org/blog/2019/04/08/project-highlight-dpp/ ) it can be helpful too: std::string uses Small String Optimization technique https://stackoverflow.com/a/28003328 with video about GCC string and fbstring https://youtu.be/kPR8h4-qZdk?t=654 (CppCon 2016: Nicholas Ormrod “The strange details of std::string at Facebook") - I just pointed to fbstring internals that most probably used as Clang strings (from SO answer), but M$ and GCC uses some kind of GCC ver>=5 string internals (find it in video too) probably it can be tied with DIP 1021 (Borrowing and Ownership) - when code doesn't contains another refs to object compiler can just move the one without expensive copy/constructor/destructor. nobody can assure (wrong word meaning? idk English) that "move" don't needed at all. so lets add it with user friendly details control.
Jul 18