www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - rvalues -> ref (yup... again!)

reply Manu <turkeyman gmail.com> writes:
Forked from the x^^y thread...

On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 3/23/2018 11:09 AM, Manu wrote:
 [...]
Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them.
Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); } How is that worse than the code you have to write: T temp = f(); T zero = 0; func(temp, zero); This 'workaround' is really upsetting. It's ugly, visually noisy, very annoying to implement over and over, and litters the local namespace with rubbish. There's no good name for most of this stuff, they just get named t0, t1, t2... Surely you can see how this objectively makes code worse...? This one more than anything has made my work trying to convince people that D is cool very hard. Most things I can explain around, or say it's a known issue and it'll be better soon. This one is kinda harder; "yeah... that's like, working exactly as intended! I know, isn't D cool!" ;)
 But at some level, D cannot replace C++ on a line-by-line basis. There's
 always going to be something different. If not in the core language, in the
 way the standard library works. If you're heavily using templates and stuff
 in C++, you're likely going to have to rethink how the code works to get it
 in D (or any other language).
I haven't experienced much friction on that front, or had it expressed by new users. They're generally willing to humour that D has differences in it's meta, probably because they understand C++ sucks and drastic changes must be made. They generally complain about choice of '!' for a few minutes and then move on. By contrast, people will NOT forgive the fact that they have to change: func(f(x), f(y), f(z)); to: T temp = f(x); T temp2 = f(y); T temp3 = f(z); func(temp, temp2, temp3); That's just hideous and in-defensible. A better story would be: func(f(x), f(y), f(z)); => func(x.f, y.f, z.f); Instead of being outraged, they'd be further seduced. Sadly, that's not our reality (yet). What a missed opportunity! ;)
 For example, in my efforts translating C to D, the clumsy part is the
 metaprogramming in the C preprocessor. There's nothing there D cannot do,
 but it has to be redesigned. The result is much better, but translating by
 rote is simply impossible.
I don't often work with C, and I think it's been considered pretty unsavoury to lean heavily on the preprocessor for a few decades now. Even if I did, I probably wouldn't 'port' C, so much as interact with it, and extern(C) works well. I'm more focused on C++, and as such, my experiences differ significantly ;) Reworking a field of preprocessor cruft is in quite a different space than "reworking all calls to func()". People are forgiving of a thing if they appreciate and understand it.
 Also, just try translating some of the code in Phobos to C++. It was tried
 to do ranges for C++, and the result was terrifying. (It worked, but that's
 about all that could be said for it.)
I've done comprehensive slices and ranges in C++. It works reasonably well. Certainly, the worst outcome was that I significantly diminished any arguments I had to switch to D... Not sure what your point is though? You'll find no argument from me that slice, ranges, algorithms are awesome, and one of D's main events... but new users need to get far enough into the woods to reach that point. We need to make sure we're the least amount likely to repel them prior to reaching that point as possible. I had a lunch discussion with a colleague today; he believes my claims about ranges/algorithms and stuff, but he has no feel for it, and can't really grok just how ground breaking that stuff is... he admits he needs some experience to comment, and he's actively checking it out. It's critical that he doesn't try to call a function that takes a ref before he reaches that moment, because if he does, chances are he'll get angry at me for wasting his time.
Mar 23 2018
next sibling parent reply Nick Sabalausky <a a.a> writes:
It never made any sense to me that there could be any problem 
with the compiler automatically creating a temporary hidden 
lvalue so a ref could be taken. If there IS any problem, I can 
only imagine it would be symptomatic of a different, larger 
problem.
Mar 23 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:
 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.
It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used). So, having ref in general accept rvalues would be a huge problem. However, that could be solved by having a different attribute indicate that the idea is that the compiler will accept rvalues and create temporaries for them if they're passed instead of an lvalue. Then, the situation is clear. As for rvalue references in general, I can never remember what exactly the problem is. IIRC, part of it relates to the fact that it makes it impossible for the compiler to know whether it's dealing with an actual lvalue or not, which has serious safety implications, and in the case of C++, complicates things considerably. Andrei has plenty of nasty things to say about how rvalue references in C++ were a huge mistake. It's just that I can never remember the arguments very well. The use of scope with DIP 1000 may reduce the problem such that scope ref could be made to work safely (I don't know), but that by itself isn't enough if you want it to be clear whether a function is supposed to be mutating the argument or if it's just trying to avoid copying it. C++ solves that problem by requiring const with rvalue references, but given how D's const works, that really wouldn't make sense. So, we'd have to do something else. I get the impression that this issue is one where it seems like a small one on the surface but that when you get into the guts of what it actually means to implement it, it gets nasty. I think that most of us have just take the approach of passing everything by value unless the type is clearly too expensive to copy for that to make sense, in which case, it's then passed by ref or auto ref, or we make it a reference type. I don't know how much the problem with the lack of rvalue references in D is a PR issue, because C++ programmers get annoyed about it and deem D to be subpar as a result, and how much it's an actual performance problem. I don't work in Manu's world, so I don't what really makes sense there. For me, passing by value is usually a non-issue, and it's easy enough to work around the problem in the rare cases where it is a problem. Clearly, it's a PR issue in Manu's world, but it may also be a performance problem. I don't know. Either way, it's clear that Manu and others like him think that the fact that D doesn't have rvalue references is a serious deficiency. - Jonathan M Davis
Mar 23 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/23/2018 07:46 PM, Jonathan M Davis wrote:
 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:
 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.
It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used).
???. That's equally true when an lvalue is passed in.
 However, that could be solved by having a different attribute indicate that
 the idea is that the compiler will accept rvalues and create temporaries for
 them if they're passed instead of an lvalue. Then, the situation is clear.
Why require the callee's author to add boilerplate? Just do it for all ref params that are given an rvalue.
 As for rvalue references in general, I can never remember what exactly the
 problem is. IIRC, part of it relates to the fact that it makes it impossible
 for the compiler to know whether it's dealing with an actual lvalue or not,
As long as the compiler takes the advocated "automatic hidden temporary" approach, then it *is* an actual lvalue. We're talking nothing more than sugar here. Sugar for what we're already forced to do manually. There *cannot* be a technical problem here that isn't *already* a problem with the author manually creating a temp var.
Mar 23 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d 
wrote:
 On 03/23/2018 07:46 PM, Jonathan M Davis wrote:
 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d 
wrote:
 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.
It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used).
???. That's equally true when an lvalue is passed in.
 However, that could be solved by having a different attribute indicate
 that the idea is that the compiler will accept rvalues and create
 temporaries for them if they're passed instead of an lvalue. Then, the
 situation is clear.
Why require the callee's author to add boilerplate? Just do it for all ref params that are given an rvalue.
Because if the point of the function accepting its argument by ref is because it's going to mutate the argument, then it makes no sense for it to accept rvalues, whereas if the point of the function accepting its argument by ref is to just avoid a copy, then it makes sense to accept both. It should be clear from the API which is intended. As it stands, because a function can't accept rvalues by ref, it's usually reasonable to assume that a function accepts its argument by ref because it's mutating that argument rather than simply because it's trying to avoid a copy. If ref suddenly starts accepting rvalues, then we lose that. With C++, you know the difference because, only const& accepts rvalues, and whether it's const or some other attribute, I'd very much like that any ref that accepts rvalues in D be marked with something to indicate that it does rather than making ref do double-duty and make it unclear what it's there for. - Jonathan M Davis
Mar 24 2018
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:

 As it stands, because a function can't accept rvalues by ref, it's
 usually reasonable to assume that a function accepts its argument by ref
 because it's mutating that argument rather than simply because it's
 trying to avoid a copy. If ref suddenly starts accepting rvalues, then
 we lose that.
Any reason you can't simply use `ref` to imply 'modifies value' and `const ref` as 'passed by ref for performance reasons'? -- Johannes
Mar 24 2018
parent Johannes Pfau <nospam example.com> writes:
Am Sat, 24 Mar 2018 17:10:53 +0000 schrieb Johannes Pfau:

 Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:
 
 As it stands, because a function can't accept rvalues by ref, it's
 usually reasonable to assume that a function accepts its argument by
 ref because it's mutating that argument rather than simply because it's
 trying to avoid a copy. If ref suddenly starts accepting rvalues, then
 we lose that.
Any reason you can't simply use `ref` to imply 'modifies value' and `const ref` as 'passed by ref for performance reasons'?
Sorry, I see Manu already asked the same question. -- Johannes
Mar 24 2018
prev sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/24/2018 03:03 AM, Jonathan M Davis wrote:
 On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d
 wrote:
 Why require the callee's author to add boilerplate? Just do it for all
 ref params that are given an rvalue.
Because if the point of the function accepting its argument by ref is because it's going to mutate the argument, then it makes no sense for it to accept rvalues,
1. That *isn't* always the core point of a function which takes a non-const ref argument. 2. It's the caller who decides whether or not the ref-result is needed, not the callee. 3. The frequent recurring complaints about no rvalue references are a testament that this is too common a use-case for the current "manually insert temporaries" workaround to be satisfactory. 4. If the whole point of disallowing rvalue references is to prevent accidents due to callers not realizing a param is being passed by ref (as it sounds like you're suggesting), then it's nothing but a broken half-solution, because it fails to provide that safety (and thus fails its own charter) when that same caller, once again not realizing a param is ref, passes an *lvalue* without expecting it to change. *If* the problem we want to solve here really is making sure a caller knows when a param is ref, we've failed, and the only way to actually accomplish it and raise an error when they don't.
Mar 24 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 00:04, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d
 wrote:
 On 03/23/2018 07:46 PM, Jonathan M Davis wrote:
 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d
wrote:
 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.
It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used).
???. That's equally true when an lvalue is passed in.
 However, that could be solved by having a different attribute indicate
 that the idea is that the compiler will accept rvalues and create
 temporaries for them if they're passed instead of an lvalue. Then, the
 situation is clear.
Why require the callee's author to add boilerplate? Just do it for all ref params that are given an rvalue.
Because if the point of the function accepting its argument by ref is because it's going to mutate the argument, then it makes no sense for it to accept rvalues, whereas if the point of the function accepting its argument by ref is to just avoid a copy, then it makes sense to accept both. It should be clear from the API which is intended.
Write const; it's as clear as day!
Mar 24 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 16:46, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:
 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.
It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used).
Okay, let's read 'const ref' every time I say 'ref'. I thought that would be fairly safe to assume. Sorry!
 However, that could be solved by having a different attribute indicate that
 the idea is that the compiler will accept rvalues and create temporaries for
 them if they're passed instead of an lvalue. Then, the situation is clear.
No, no attributes.. Just accept any argument to const-ref! The function's not gonna change it.
 As for rvalue references in general, I can never remember what exactly the
 problem is. IIRC, part of it relates to the fact that it makes it impossible
 for the compiler to know whether it's dealing with an actual lvalue or not,
 which has serious  safety implications, and in the case of C++, complicates
 things considerably. Andrei has plenty of nasty things to say about how
 rvalue references in C++ were a huge mistake. It's just that I can never
 remember the arguments very well.

 The use of scope with DIP 1000 may reduce the problem such that scope ref
 could be made to work  safely (I don't know)
True, but if you follow that line of reasoning, then this case must equally be banned: T temp = f(); func(temp); The safety fear is that the pointer to the stack arg may escape, and that's identical whether the argument is an explicit temp, or an implicit one.
 but that by itself isn't
 enough if you want it to be clear whether a function is supposed to be
 mutating the argument
Functions that receive const args make it pretty clear that they don't intend to mutate the arg.
 I get the impression that this issue is one where it seems like a small one
 on the surface but that when you get into the guts of what it actually means
 to implement it, it gets nasty.
It really doesn't... the compiler just make the same temp that I type with my hands. That's literally all that it needs to do. I'd bet money it's a one-paragraph change.
 Clearly, it's a PR issue in Manu's world, but it may also be a performance
problem. I
 don't know. Either way, it's clear that Manu and others like him think that
 the fact that D doesn't have rvalue references is a serious deficiency.
That's an understatement, but yes :P We want to call C++ code. The D code shouldn't be objectively worse than the C++ code we're trying to escape, otherwise we're undermining our gravity towards D. We can't have a situation that goes "I wrote this 3 line function in D, but now it's 6 lines because I had to break args to temps, it's objectively worse than the equivalent C++ function... why am I writing it in D again?".
Mar 23 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 01:35, Manu wrote:
 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!
Absolutely not. It makes absolutely no sense to restrict rvalue references to const objects. (Recall that const is transitive and actually prevents mutation. This is not C++.)
Mar 23 2018
parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 24.03.2018 01:35, Manu wrote:
 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!
Absolutely not. It makes absolutely no sense to restrict rvalue references to const objects. (Recall that const is transitive and actually prevents mutation. This is not C++.)
We're not talking about rvalue-references... were talking about not-rvalue-references to temporaries.
Mar 23 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 04:14, Manu wrote:
 On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 24.03.2018 01:35, Manu wrote:
 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!
Absolutely not. It makes absolutely no sense to restrict rvalue references to const objects. (Recall that const is transitive and actually prevents mutation. This is not C++.)
We're not talking about rvalue-references... were talking about not-rvalue-references to temporaries.
Let me rephrase: Absolutely not. It makes absolutely no sense to force const for not-rvalue-references to temporaries. (Recall that const is transitive and actually prevents mutation. This is not C++.)
Mar 23 2018
parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 20:19, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 24.03.2018 04:14, Manu wrote:
 On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 24.03.2018 01:35, Manu wrote:
 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!
Absolutely not. It makes absolutely no sense to restrict rvalue references to const objects. (Recall that const is transitive and actually prevents mutation. This is not C++.)
We're not talking about rvalue-references... were talking about not-rvalue-references to temporaries.
Let me rephrase: Absolutely not. It makes absolutely no sense to force const for not-rvalue-references to temporaries. (Recall that const is transitive and actually prevents mutation. This is not C++.)
Yes, I know... why would you mutate an argument who's life doesn't extend beyond the call? Any mutation is pointless.
Mar 23 2018
prev sibling next sibling parent reply MattCoder <nospam mail.com> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 ...
 By contrast, people will NOT forgive the fact that they have to 
 change:

     func(f(x), f(y), f(z));

 to:

     T temp = f(x);
     T temp2 = f(y);
     T temp3 = f(z);
     func(temp, temp2, temp3);
...
Or you could do this: func(tx=f(x), ty=f(y), tz=f(z)); I know it will not solve your problem, but except for the explicit variables, it's almost like your first example. Well, to be honest I still can't understand why would you want to pass a RValue as reference. Matt.
Mar 23 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.
Well, it's frequently the case that you don't want to copy an object if you don't have to - especially in the gaming world, where every ounce of performance matters. If a function accepts an argument by ref, then no copy is made, but then you have to pass an lvalue, making it really annoying to call the function when what you have is an rvalue. On the other hand, if the function doesn't accept its argument by ref, then you can pass an rvalue (and it will be moved, making that efficient), but then lvalues get copied when that's often not what you want. C++'s solution to this was rvalue references. That way, as long as the parameter is const, it can accept both rvalues and lvalues, and it won't copy unless it has to. The closest analogue that D has to this is auto ref, which requires templates, which may or may not be acceptable. In many cases, it works great, whereas in others, it doesn't work at all (e.g. virtual functions), and it can result in template bloat. The whole situation is complicated by the fact that sometimes it's actually faster to pass by value and copy the argument, even if it's an lvalue. Passing by const& can often result in unnecessary copies if anywhere in the chain passes the object by value, whereas if it's passed by value, the same object can often be moved multiple times, avoiding any copies. It's my understanding that prior to C++11, it was considered best practice to pass by const& as much as possible to avoid copies but that after C++11 (which added move constructors), it's often considered better to pass by value, because then in many cases, the compiler can move the object instead of copying it. So, C++ gives you control over which you do, and it's not necessarily straightforward as to which you should use (though plenty of older C++ programmers likely just use const& all over the place out of habit). D supports moving out of the box without move constructors, so it does a good job of handling the cases where a move is most appropriate, but it doesn't have rvalue references, so it doesn't have the same flexibility as C++ when you want to pass something by reference. So, anyone looking to pass stuff by reference all over the place (like Manu and his coworkers) is going to find the lack of rvalue references in D to be really annoying. - Jonathan M Davis
Mar 23 2018
parent reply MattCoder <nospam mail.com> writes:
On Friday, 23 March 2018 at 23:58:05 UTC, Jonathan M Davis wrote:
 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d 
 wrote:
 Well, to be honest I still can't understand why would you want 
 to pass a RValue as reference.
Well, it's frequently the case that you don't want to copy an object if you don't have to... - Jonathan M Davis
Well the concept it's OK. (Differences between passing by value vs reference, copy etc.). Except for the const thing in C++, because I don't know this language, and by the way thanks for explaining that. Question: In C++ the signature of the function which will receive the references like in this case, need to be "const ref" parameters, right? - If yes, then since it's const ref parameter, will not change the value passed, even if it's lvalue, right? Matt.
Mar 23 2018
parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 17:48, MattCoder via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Question:

 In C++ the signature of the function which will receive the references like
 in this case, need to be "const ref" parameters, right? - If yes, then since
 it's const ref parameter, will not change the value passed, even if it's
 lvalue, right?
Right. It's so obvious isn't it ;)
Mar 23 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.
Well, it's frequently the case that you don't want to copy an object if you don't have to - especially in the gaming world, where every ounce of performance matters. If a function accepts an argument by ref, then no copy is made, but then you have to pass an lvalue, making it really annoying to call the function when what you have is an rvalue. On the other hand, if the function doesn't accept its argument by ref, then you can pass an rvalue (and it will be moved, making that efficient), but then lvalues get copied when that's often not what you want. C++'s solution to this was rvalue references.
Ummm... rvalue-references are something completely different. rval-ref's are C++'s solution to move semantics. C++ just simply accepts rvalues passed to const& args. It makes a temp and passes the ref, as you expect.
 That way, as long as the parameter is const, it can accept both
 rvalues and lvalues, and it won't copy unless it has to. The closest
 analogue that D has to this is auto ref, which requires templates, which may
 or may not be acceptable.
auto-ref == template function, which by definition is NOT an extern(C++) function. auto-ref may also resolve to NOT a ref, which means a move... data structures that are large (ie, a vector or matrix) still have to copy a bunch of memory, even if the copy is algorithmically 'cheap'.
 In many cases, it works great, whereas in others,
 it doesn't work at all (e.g. virtual functions), and it can result in
 template bloat.
And templates, that's another case where it fails. auto-ref is something else unrelated to this topic, and it's useful in a different set of cases for a different set of uses/reasons. It's got nothing to do with this.
 The whole situation is complicated by the fact that sometimes it's actually
 faster to pass by value and copy the argument, even if it's an lvalue.
The api author wouldn't have made the arg a ref in that case.
 Passing by const& can often result in unnecessary copies if anywhere in the
 chain passes the object by value, whereas if it's passed by value, the same
 object can often be moved multiple times, avoiding any copies**.
**avoiding __calls to the copy constructor__. The memory is likely to still be moved around a bunch of times. The case you're thinking of is when values are *returned* by value, in that case, copy elision is possible, and the result can be constructed in place. That's a completely unrelated problem... you don't do return-by-ref ;)
 It's my
 understanding that prior to C++11, it was considered best practice to pass
 by const& as much as possible to avoid copies but that after C++11 (which
 added move constructors), it's often considered better to pass by value,
 because then in many cases, the compiler can move the object instead of
 copying it**.
** Assuming the object is tiny, but has an expensive copy constructor. In the case where an object is large (and has a primitive, or no copy constructor) it doesn't change the situation; you still wanna pass a big thing by ref in all cases; ie, vector/matrix.
 So, C++ gives you control over which you do, and it's not
 necessarily straightforward as to which you should use (though plenty of
 older C++ programmers likely just use const& all over the place out of
 habit).
D gives you the same set of options; except that passing args by ref is a PITA in D, and ruins your code.
Mar 23 2018
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote:
 but that by itself isn't
 enough if you want it to be clear whether a function is supposed to be
 mutating the argument
Functions that receive const args make it pretty clear that they don't intend to mutate the arg.
Yes, but with how restrictive const is in D, I have a very hard time believing that it's going to work well to start using const ref much even if it accepted rvalues. - Jonathan M Davis
Mar 23 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote:
 On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:
 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.
Well, it's frequently the case that you don't want to copy an object if you don't have to - especially in the gaming world, where every ounce of performance matters. If a function accepts an argument by ref, then no copy is made, but then you have to pass an lvalue, making it really annoying to call the function when what you have is an rvalue. On the other hand, if the function doesn't accept its argument by ref, then you can pass an rvalue (and it will be moved, making that efficient), but then lvalues get copied when that's often not what you want. C++'s solution to this was rvalue references.
Ummm... rvalue-references are something completely different. rval-ref's are C++'s solution to move semantics. C++ just simply accepts rvalues passed to const& args. It makes a temp and passes the ref, as you expect.
It was my understanding that that _was_ an rvalue reference, and I remember that Andrei was against const ref accepting rvalues in D due to issues with rvalue references. It's quite possible that I misremember though, and I certainly am not an expert on all of the issues with rvalues and references in C++, and my C++ knowledge is getting increasingly rusty, since I don't use C++ much anymore. In any case, I have a terrible time remembering Andrei's exact arguments, but he feels very strongly about them, so anyone looking to convince him is going to have a hard time of it. My biggest concern in all of this is that I don't want to see ref start accepting rvalues as has been occasionally discussed. It needs to be clear when a function is accept an argument by ref because it's going to mutate the object and when it's accepting by ref because it wants to avoid a copy. The addition of const solves that problem for C++, but given how restrictive const is in D, I doubt that much of anyone would ultimately be very happy with using const ref in their code very often.
 In many cases, it works great, whereas in others,
 it doesn't work at all (e.g. virtual functions), and it can result in
 template bloat.
And templates, that's another case where it fails. auto-ref is something else unrelated to this topic, and it's useful in a different set of cases for a different set of uses/reasons. It's got nothing to do with this.
auto ref has everything to do with a function accepting both rvalues and lvalues without making a copy. auto ref is the solution that was introduced into D to solve that very problem. It's just that it has limitations that make it inappropriate in a number of cases, so you don't consider it a solution to your problem.
 So, C++ gives you control over which you do, and it's not
 necessarily straightforward as to which you should use (though plenty of
 older C++ programmers likely just use const& all over the place out of
 habit).
D gives you the same set of options; except that passing args by ref is a PITA in D, and ruins your code.
Which is why I said that D doesn't give you the same set of options. - Jonathan M Davis
Mar 23 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
 
 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be clear
 when a function is accept an argument by ref because it's going to mutate
 the object and when it's accepting by ref because it wants to avoid a copy.
 
That ship sailed ages ago: It's already unclear. If we want to fix that, we can fix it, but blocking rvalue ref does nothing for that cause.
Mar 29 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d 
wrote:
 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants to
 avoid a copy.
That ship sailed ages ago: It's already unclear. If we want to fix that, we can fix it, but blocking rvalue ref does nothing for that cause.
Really? And how often does ref get used just to avoid copying? You can almost always look at a function, see that it accepts ref, and know that it's supposed to mutate the argument. Functions that want to avoid copying lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a number of folks would start using it all over the place to avoid copying. Right now, folks rarely use ref that way, because it becomes too annoying to call the function with an rvalue. So, while it might not be the case 100% of the time right now that ref is used with the purpose of mutating the argument, it almost always is. As such, you can pretty reliably look at a function signature and expect that if one of its parameters is ref, it's going to be mutating that argument. The function that accepts an argument by ref with no intention of mutating it is very much the exception, and I really don't want to see that change. - Jonathan M Davis
Mar 29 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d
 wrote:
 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants to
 avoid a copy.
That ship sailed ages ago: It's already unclear. If we want to fix that, we can fix it, but blocking rvalue ref does nothing for that cause.
Really? And how often does ref get used just to avoid copying? You can almost always look at a function, see that it accepts ref, and know that it's supposed to mutate the argument. Functions that want to avoid copying lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a number of folks would start using it all over the place to avoid copying. Right now, folks rarely use ref that way, because it becomes too annoying to call the function with an rvalue. So, while it might not be the case 100% of the time right now that ref is used with the purpose of mutating the argument, it almost always is. As such, you can pretty reliably look at a function signature and expect that if one of its parameters is ref, it's going to be mutating that argument. The function that accepts an argument by ref with no intention of mutating it is very much the exception, and I really don't want to see that change.
Interesting. Just understand that you're trading that feeling for a suite of edge cases though. Accepting asymmetric calling rules is a pretty big cost to pay for that 'nice thought'. That idea also dismisses the existence of the set of cases where ref is genuinely useful/correct. Your sentiment effectively puts those use cases into the position of 2nd-class citizens. I understand your sentiment, but I think the cost is not balanced, or even particularly fair with respect to users in those niche groups :/
Mar 30 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 17:58, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote:
 but that by itself isn't
 enough if you want it to be clear whether a function is supposed to be
 mutating the argument
Functions that receive const args make it pretty clear that they don't intend to mutate the arg.
Yes, but with how restrictive const is in D, I have a very hard time believing that it's going to work well to start using const ref much even if it accepted rvalues.
This is an interesting point, but I don't think it changes the balance in any way. Thinking of the majority of my anecdotal cases, I don't think it would be a problem. Something complex enough for const to be a problem likely doesn't conform to this pattern. Further, extern(C++) functions that receive const& args are 'const ref' regardless of D's const semantics. So for extern(C++), which I suspect is a high percentage of cases where this issue is significant (because D doesn't naturally follow C++'s pattern so much anyway), that point doesn't matter.
Mar 23 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 02:16, Manu wrote:
 This is an interesting point, but I don't think it changes the balance
 in any way. Thinking of the majority of my anecdotal cases, I don't
 think it would be a problem.
 Something complex enough for const to be a problem likely doesn't
 conform to this pattern.
Why aim for "it often works", when we want "it always works"? Forcing const upon people who want to pass rvalues by reference is just not good enough. It is bad language design. Also I think the point about documenting mutation intent is moot, as rvalues can be receivers for method calls, which will _already_ pass an rvalue by reference no matter whether it intends to mutate it or not. We can require some special annotation for this behavior, but I'd be perfectly fine without it.
Mar 23 2018
parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 24.03.2018 02:16, Manu wrote:
 This is an interesting point, but I don't think it changes the balance
 in any way. Thinking of the majority of my anecdotal cases, I don't
 think it would be a problem.
 Something complex enough for const to be a problem likely doesn't
 conform to this pattern.
Why aim for "it often works", when we want "it always works"? Forcing const upon people who want to pass rvalues by reference is just not good enough. It is bad language design.
I think you need to re-read the whole thread, and understand what we're even talking about. Nobody wants to pass rvalues by mutable-ref... that's completely pointless, since it's an rvalue that will timeout immediately anyway. Passing by const-ref is perfectly acceptable. I suspect Jonathan's talking about classic D situations with const like, I might pass by const-ref, but then I can't call a getter that caches the result. That's a classic problem with D's const, and that's not on debate here. I don't think that has any impact on this proposal; people understand what const means in D, and that's no different here than anywhere else.
 Also I think the point about documenting mutation intent is moot, as rvalues
 can be receivers for method calls, which will _already_ pass an rvalue by
 reference no matter whether it intends to mutate it or not. We can require
 some special annotation for this behavior, but I'd be perfectly fine without
 it.
I have no idea what this paragraph means... can you elaborate further what you're talking about?
Mar 23 2018
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 05:03, Manu wrote:
 On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 24.03.2018 02:16, Manu wrote:
 This is an interesting point, but I don't think it changes the balance
 in any way. Thinking of the majority of my anecdotal cases, I don't
 think it would be a problem.
 Something complex enough for const to be a problem likely doesn't
 conform to this pattern.
Why aim for "it often works", when we want "it always works"? Forcing const upon people who want to pass rvalues by reference is just not good enough. It is bad language design.
I think you need to re-read the whole thread, and understand what we're even talking about. ...
That will not be necessary. I wouldn't even have had to read it the first time. Those discussions always go the same way: M: I wish we could pass rvalue arguments to ref parameters. J: That would be terrible, as people would then pass rvalues as ref by accident and not see the mutation that the author of the function intended them to see. M: Only do it for const ref parameters then. T: No, this has nothing to do with const. (M can be replaced by a variety of other letters; this is a somewhat common feature request.)
 Nobody wants to pass rvalues by mutable-ref... that's completely
 pointless, since it's an rvalue that will timeout immediately anyway.
 Passing by const-ref is perfectly acceptable.
 ...
Your temporary might have mutable indirections. Maybe you don't want to be forced to annotate your methods as const, limiting your future options.
 I suspect Jonathan's talking about classic D situations with const
 like, I might pass by const-ref, but then I can't call a getter that
 caches the result.
 That's a classic problem with D's const, and that's not on debate
 here. I don't think that has any impact on this proposal; people
 understand what const means in D, and that's no different here than
 anywhere else.
 ...
Your proposal _changes_ the meaning of const. I.e., "const does all it did previously and now it also allows rvalues to be passed to ref functions". This is bad, as one has little to do with the other, yet now you couple them. Programmers who want to pass rvalues as ref do not necessarily want to use D const on their objects.
 
 Also I think the point about documenting mutation intent is moot, as rvalues
 can be receivers for method calls, which will _already_ pass an rvalue by
 reference no matter whether it intends to mutate it or not. We can require
 some special annotation for this behavior, but I'd be perfectly fine without
 it.
I have no idea what this paragraph means... can you elaborate further what you're talking about?
This works: struct S{ int x; void inc(){ this.x++; // note: 'this' is passed by ref } } void main(){ S().inc(); } but this does not: struct S{ int x; } void inc(ref S self){ self.x++; // note: 'self' is passed by ref } void main(){ S().inc(); } I.e. there is a special case where your rewrite is already applied. Note how "inc" cannot even be made const. What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice. There are only three sensible ways to fix the problem: 1. Just allow rvalue arguments to bind to ref parameters. (My preferred solution, though it will make the overloading rules slightly more complicated.) 2. Add some _new_ annotation for ref parameters that signifies that you want the same treatment for them that the implicit 'this' reference gets. (A close second.) 3. Continue to require code bloat (auto ref) or manual boilerplate (overloads). (I'm not very fond of this option, but it does not require a language change.)
Mar 24 2018
next sibling parent reply kinke <noone nowhere.com> writes:
On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
 What I'm saying is that I don't really buy Jonathan's argument. 
 Basically, you should just pass the correct arguments to 
 functions, as you always need to do. If you cannot use the 
 result of some mutation that you need to use, you will probably 
 notice.
I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too.
 There are only three sensible ways to fix the problem:

 1. Just allow rvalue arguments to bind to ref parameters. (My 
 preferred solution, though it will make the overloading rules 
 slightly more complicated.)
I always thought the main concern was potential escaping refs to the rvalue, which would be solvable by allowing rvalues to be bound to `scope ref` params only. That'd be my preferred choice as well.
 2. Add some _new_ annotation for ref parameters that signifies 
 that you want the same treatment for them that the implicit 
 'this' reference gets. (A close second.)
*Shudder*.
 3. Continue to require code bloat (auto ref) or manual 
 boilerplate (overloads). (I'm not very fond of this option, but 
 it does not require a language change.)
While `auto ref` seems to have worked out surprisingly well for code written in D, it doesn't solve the problem when interfacing with (many) external C++ functions taking structs (represented in D by structs as well) by (mostly const) ref. You're forced to declare lvalues for all of these args, uglifying the code substantially.
Mar 24 2018
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 15:56, kinke wrote:
 On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
 What I'm saying is that I don't really buy Jonathan's argument. 
 Basically, you should just pass the correct arguments to functions, as 
 you always need to do. If you cannot use the result of some mutation 
 that you need to use, you will probably notice.
I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too. ...
A warning is not viable. (There's no good way to fix it.)
 There are only three sensible ways to fix the problem:

 1. Just allow rvalue arguments to bind to ref parameters. (My 
 preferred solution, though it will make the overloading rules slightly 
 more complicated.)
I always thought the main concern was potential escaping refs to the rvalue, which would be solvable by allowing rvalues to be bound to `scope ref` params only. That'd be my preferred choice as well. ...
There is no difference between escaping refs to an rvalue and escaping refs to a short-lived lvalue, as the callee has no idea where the address is coming from anyway. According to Walter, ref parameters are not supposed to be escaped, and safe will enforce it. Also, AFAIU, "scope" in "scope ref T" already applies to "T", not "ref".
 2. Add some _new_ annotation for ref parameters that signifies that 
 you want the same treatment for them that the implicit 'this' 
 reference gets. (A close second.)
*Shudder*. ...
Well, it beats "const".
 3. Continue to require code bloat (auto ref) or manual boilerplate 
 (overloads). (I'm not very fond of this option, but it does not 
 require a language change.)
While `auto ref` seems to have worked out surprisingly well for code written in D, it doesn't solve the problem when interfacing with (many) external C++ functions taking structs (represented in D by structs as well) by (mostly const) ref. You're forced to declare lvalues for all of these args, uglifying the code substantially.
You can add additional overloads on the D side. (This can even be automated using a string mixin.)
Mar 24 2018
parent kinke <noone nowhere.com> writes:
On Saturday, 24 March 2018 at 15:36:14 UTC, Timon Gehr wrote:
 On 24.03.2018 15:56, kinke wrote:
 I agree, but restricting it to const ref would be enough for 
 almost all of my use cases. The MS C++ compiler just emits a 
 warning when binding an rvalue to a mutable ref ('nonstandard 
 extension used'), I'd find that absolutely viable for D too.
 ...
A warning is not viable. (There's no good way to fix it.)
As long as specific warnings cannot be suppressed via pragmas, one would need to predeclare the lvalue to get rid of it; fine IMHO for the, as I expect, very rare use cases.
 There is no difference between escaping refs to an rvalue and 
 escaping refs to a short-lived lvalue, as the callee has no 
 idea where the address is coming from anyway. According to 
 Walter, ref parameters are not supposed to be escaped, and 
  safe will enforce it.
Alright, the less keywords overhead, the better. :)
 You can add additional overloads on the D side. (This can even 
 be automated using a string mixin.)
Right I can, but I don't want to add 7 overloads for a C++ function taking 3 params by const ref. Even if autogenerated by some tool or fancy mixins, the code's legibility would suffer a lot. D's syntax is IMO one of its strongest selling points, and that shouldn't degrade when it comes to C(++) interop.
Mar 24 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 07:56, kinke via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
 What I'm saying is that I don't really buy Jonathan's argument. Basically,
 you should just pass the correct arguments to functions, as you always need
 to do. If you cannot use the result of some mutation that you need to use,
 you will probably notice.
I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too.
 There are only three sensible ways to fix the problem:

 1. Just allow rvalue arguments to bind to ref parameters. (My preferred
 solution, though it will make the overloading rules slightly more
 complicated.)
I always thought the main concern was potential escaping refs to the rvalue, which would be solvable by allowing rvalues to be bound to `scope ref` params only. That'd be my preferred choice as well.
I touched on this. It sounds reasonable at first glance. But the reasoning goes: 1. for safety reasons, we won't allow implicit stack temporaries to pass to functions that may escape them. 2. support passing temporaries only to 'scope ref' 3. realise that the implicit temp created by an rvalue is identical to the manually authored temp (or any other stack locals in the caller), you find that the only reasonable option to satisfy the alleged safety concern, is that all stack variables may never be passed to any function that's not 'scope ref'. Ie, you can't do this for safety reasons, because the exact same reasoning would exclude all stack variables ever from being passed the same way. The proposition is self-defeating. I was attracted to this idea for a short while, until I realised it was ridiculous. ;)
 3. Continue to require code bloat (auto ref) or manual boilerplate
 (overloads). (I'm not very fond of this option, but it does not require a
 language change.)
While `auto ref` seems to have worked out surprisingly well for code written in D, it doesn't solve the problem when interfacing with (many) external C++ functions taking structs (represented in D by structs as well) by (mostly const) ref. You're forced to declare lvalues for all of these args, uglifying the code substantially.
It's also not great for libraries. auto ref functions are template functions... what if I want to export that function? I can't. There are practical API and ABI reasons that you don't want every function to be a template function. Library authors carefully control which functions are 'real' functions and which are templates, for a wide variety of reasons. Binary libs are a thing. DLL's are a thing.
Mar 24 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 06:49, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 24.03.2018 05:03, Manu wrote:
 I have no idea what this paragraph means... can you elaborate further
 what you're talking about?
This works: struct S{ int x; void inc(){ this.x++; // note: 'this' is passed by ref } } void main(){ S().inc(); } but this does not: struct S{ int x; } void inc(ref S self){ self.x++; // note: 'self' is passed by ref } void main(){ S().inc(); } I.e. there is a special case where your rewrite is already applied. Note how "inc" cannot even be made const. What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice.
Your example demonstrates the exact reason why rvalue->ref is const& in C++, and illegal in D though. You mutate a temporary that times out at the end of the statement... your statement is never assigned to anything, and has no effect. If your statement is assigned to something, then you already have an lvalue to pass to such a function that receives mutable ref.
Mar 24 2018
parent Nick Treleaven <nick geany.org> writes:
On Saturday, 24 March 2018 at 17:34:09 UTC, Manu wrote:
 You mutate a temporary that times out at the end of the 
 statement...
 your statement is never assigned to anything, and has no effect.
That is solved by having the ref function return its argument (so it can be chained): struct S; ref S modify(return ref S s); S(data).modify.writeln; This rvalue pattern would be disallowed if argument `s` was const. Why not make `modify` just return a copy then, maybe the optimizer could remove the copy? So that you can also use it with lvalues: S s; modify(s);
Mar 29 2018
prev sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/24/2018 12:03 AM, Manu wrote:
 Why aim for "it often works", when we want "it always works"? Forcing const
 upon people who want to pass rvalues by reference is just not good enough.
 It is bad language design.
I think you need to re-read the whole thread, and understand what we're even talking about. Nobody wants to pass rvalues by mutable-ref...
I do. The ban serves no useful purpose (at least not any purpose that D doesn't *already* fail at - like knowing at the callsite whether a param is intended to be mutated). And it would permit the "avoid a copy" benefits in a completely orthogonal way - *without* relying on the param fitting D's transitive const requirements.
Mar 29 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 18:06, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote:
 On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:
 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.
Well, it's frequently the case that you don't want to copy an object if you don't have to - especially in the gaming world, where every ounce of performance matters. If a function accepts an argument by ref, then no copy is made, but then you have to pass an lvalue, making it really annoying to call the function when what you have is an rvalue. On the other hand, if the function doesn't accept its argument by ref, then you can pass an rvalue (and it will be moved, making that efficient), but then lvalues get copied when that's often not what you want. C++'s solution to this was rvalue references.
Ummm... rvalue-references are something completely different. rval-ref's are C++'s solution to move semantics. C++ just simply accepts rvalues passed to const& args. It makes a temp and passes the ref, as you expect.
It was my understanding that that _was_ an rvalue reference, and I remember that Andrei was against const ref accepting rvalues in D due to issues with rvalue references.
C++ const& to 'rvalues' are just lvalue refs to temporaries, exactly as I propose here. rval-ref's are something completely different (all about move semantics), and have nothing to do with this conversation. There is a potentially interesting parallel conversation which discusses how to interact with extern(C++) functions that receive rvalue ref's, but that actually is a complex conversation, and no simple answers exist.
 In any case, I have a terrible time remembering
 Andrei's exact arguments, but he feels very strongly about them, so anyone
 looking to convince him is going to have a hard time of it.
Fortunately, I'm not trying to make any sort of argument for rvalue-ref's in D.
 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be clear
 when a function is accept an argument by ref because it's going to mutate
 the object and when it's accepting by ref because it wants to avoid a copy.
It's not going to mutate the argument, because it's const.
 The addition of const solves that problem for C++, but given how restrictive
 const is in D, I doubt that much of anyone would ultimately be very happy
 with using const ref in their code very often.
I expect I'll be 100% satisfied. And if not, it'll be something very close to 100%. This pattern is quite unlikely to proliferate within D code natively (because auto ref, D move semantics, classes-as-ref-types, and such), but it's essential for interacting with C++.
 auto ref has everything to do with a function accepting both rvalues and
 lvalues without making a copy. auto ref is the solution that was introduced
 into D to solve that very problem.
Yeah, somehow that emerged from this conversation years ago. I aggressively expressed at the time that I never accepted it as a solution, because it's not. It's got nothing to do with this issue, and I said at the time that I'll be very annoyed if it starts getting raised in this context ;)
 It's just that it has limitations that
 make it inappropriate in a number of cases, so you don't consider it a
 solution to your problem.
It's orthogonal to this conversation. It's the ability for templates to automate the ref-ness of args for calling efficiency. It's something like scott myers 'universal references'; ie, `template<typename T> void func(T&& arg)`. It's really got nothing to do with this conversation.
 So, C++ gives you control over which you do, and it's not
 necessarily straightforward as to which you should use (though plenty of
 older C++ programmers likely just use const& all over the place out of
 habit).
D gives you the same set of options; except that passing args by ref is a PITA in D, and ruins your code.
Which is why I said that D doesn't give you the same set of options.
It does though; D just forces you to write the temps that should be implicit by hand. There's no reason for this. It just makes code ugly, people angry, and there is no advantage.
Mar 23 2018
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...

 On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On 3/23/2018 11:09 AM, Manu wrote:
 [...]
Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them.
Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); }
I understand what you want, but I'm struggling to understand why it's such a huge deal. The reason you want to pass by reference is for performance, to avoid copying the data at the call boundary. So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be passed. 1. The address of the lvalue is passed. 2. The rvalue is copied to a local, then the address of that local is passed. So in the rvalue case, you're not getting the performance benefit of passing by reference, because you have to copy to a local anyway. What I would do in D currently to get the same performance and API: void foo(float[32] v) { foo(v); } void foo(ref float[32] v) { ... } or void foo()(auto ref float[32] v) { ... } What is so totally unacceptable about those solutions? I personally like the second because it scales better to multiple parameters. I know you have said it's not relevant and annoying that people bring up auto ref, but I dont' get how or why. It's exactly D's solution to the problem. There's a little more work to be done when thinking about extern(C++) and/or virtual functions, but most code for most people isn't made of virtual extern(C++) functions that take large value types can't accept the cost of copying a few lvalues.
Mar 24 2018
next sibling parent MattCoder <nospam mail.com> writes:
On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote:
 I understand what you want, but I'm struggling to understand 
 why it's such a huge deal.
 ...
 What I would do in D currently to get the same performance and 
 API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }

 What is so totally unacceptable about those solutions?
I hope OP answers that, because that was what I tried to refer in my post. As your example, it works like in C and I'd prefer to use it in my code, instead of passing a rvalue as reference. Matt.
Mar 24 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 04:57, John Colvin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...

 On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 3/23/2018 11:09 AM, Manu wrote:
 [...]
Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them.
Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); }
I understand what you want, but I'm struggling to understand why it's such a huge deal.
Because it makes this kind of D code that interacts with C++ objectively worse than C++, and there's no reason for it. You can't say to someone who just frustrated-ly doubled their line count by manually introducing a bunch of temporaries in a tiny function that appears to do something so simple as 'call a function', that "oh yeah, isn't it cool that you can't just call your functions anymore! isn't D cool! we should switch to D right?" It's embarrassing. I've been put in the position where I have to try and 'explain' this feature quite some number of times... they usually just give me 'the look'™; ya know, quietly wondering if I'm still sane, and all I end up with is someone who's about 95% less convinced that D is cool than they were 5 seconds beforehand. What pisses me off is that's such a pointless thing to happen, because this issue is so trivial! In my experience, people are evaluating how D will materially impact the exact same code they're already writing in C++. This is one of those ways that they will be materially impacted, and it's almost enough all on its own to cause people to dismiss the entire thing on the spot. Pretty much the best case at this phase is that the D code is exactly the same as C++. If we can trim off a few parens here and there (ufcs?), maybe remove some '::' operators (modules that don't suck), that's a huge win.
 The reason you want to pass by reference is for performance, to avoid
 copying the data at the call boundary.

 So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be
 passed.

 1. The address of the lvalue is passed.

 2. The rvalue is copied to a local, then the address of that local is
 passed.

 So in the rvalue case, you're not getting the performance benefit of passing
 by reference, because you have to copy to a local anyway.

 What I would do in D currently to get the same performance and API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }
Can't be extern(C++), can't be virtual either (both is likely). I said before; you're talking about Scott Meyers 'universal references' as a language concept, and I'm just talking about calling a function.
 but I dont' get how or why. It's exactly D's solution to the problem.
It doesn't solve the problem... it doesn't even address the problem. You're talking about a totally different thing >_<
 There's a little more work to be done when thinking about extern(C++) and/or
 virtual functions, but most code for most people isn't made of virtual
 extern(C++) functions that take large value types can't accept the cost of
 copying a few lvalues.
Correct, as I said before; people are getting bent out of shape over a thing that will likely never affect them! This will likely have very little impact on normal D code because D const, auto ref, D move semantics, etc. It will have large impact on interaction with C++ code, and the impression D is able to make on that set of users. They're an important target market.
Mar 24 2018
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 24 March 2018 at 17:30:35 UTC, Manu wrote:
 On 24 March 2018 at 04:57, John Colvin via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...

 On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On 3/23/2018 11:09 AM, Manu wrote:
 [...]
Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them.
Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); }
I understand what you want, but I'm struggling to understand why it's such a huge deal.
Because it makes this kind of D code that interacts with C++ objectively worse than C++, and there's no reason for it. You can't say to someone who just frustrated-ly doubled their line count by manually introducing a bunch of temporaries in a tiny function that appears to do something so simple as 'call a function', that "oh yeah, isn't it cool that you can't just call your functions anymore! isn't D cool! we should switch to D right?" It's embarrassing. I've been put in the position where I have to try and 'explain' this feature quite some number of times... they usually just give me 'the look'™; ya know, quietly wondering if I'm still sane, and all I end up with is someone who's about 95% less convinced that D is cool than they were 5 seconds beforehand. What pisses me off is that's such a pointless thing to happen, because this issue is so trivial! In my experience, people are evaluating how D will materially impact the exact same code they're already writing in C++. This is one of those ways that they will be materially impacted, and it's almost enough all on its own to cause people to dismiss the entire thing on the spot. Pretty much the best case at this phase is that the D code is exactly the same as C++. If we can trim off a few parens here and there (ufcs?), maybe remove some '::' operators (modules that don't suck), that's a huge win.
 The reason you want to pass by reference is for performance, 
 to avoid copying the data at the call boundary.

 So there are 2 cases: an lvalue needs to be passed, or an 
 rvalue needs to be passed.

 1. The address of the lvalue is passed.

 2. The rvalue is copied to a local, then the address of that 
 local is passed.

 So in the rvalue case, you're not getting the performance 
 benefit of passing by reference, because you have to copy to a 
 local anyway.

 What I would do in D currently to get the same performance and 
 API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }
Can't be extern(C++), can't be virtual either (both is likely). I said before; you're talking about Scott Meyers 'universal references' as a language concept, and I'm just talking about calling a function.
 but I dont' get how or why. It's exactly D's solution to the 
 problem.
It doesn't solve the problem... it doesn't even address the problem. You're talking about a totally different thing >_<
Auto ref allows the unnecessary copy to be avoided for lvalues and creates a temporary (as part of passing the value) for rvalues. It has downsides (virtual functions and extern(C++), but it does directly address the problem you're talking about, unless I have totally misunderstood you. Here is a small proof of concept I made to demonstrate how easy it seems to be to use `auto ref` to call a C++ virtual const& function without incurring any more copies than would happen with the same calls from C++. I'm sure it could be improved a lot, but does the basic concept match what you would need? // D source file: /// mix this in to your extern(C++) class with a list of the functions where you /// want to be able to pass rvalues to ref parameters. auto rValueRefCalls(Funcs ...)() { string ret; foreach (foo; Funcs) ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ `(Args ...)(auto ref Args args) if (__traits(compiles, (&this.` ~ __traits(identifier, foo) ~ `)(args))) { (&this.foo)(args); }`; return ret; } extern(C++) { class A { void foo(const ref int v); mixin(rValueRefCalls!foo); } A makeA(); } void main() { int x = 3; auto a = makeA(); a.foo(x); a.foo(3); } // C++ source file: #include<cstdio> class A { public: virtual void foo(const int& v) { printf("%d\n", v); } }; A *makeA() { return new A; }
Mar 24 2018
next sibling parent reply Rubn <where is.this> writes:
On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
 Auto ref allows the unnecessary copy to be avoided for lvalues 
 and creates a temporary (as part of passing the value) for 
 rvalues. It has downsides (virtual functions and extern(C++), 
 but it does directly address the problem you're talking about, 
 unless I have totally misunderstood you.
You are trading syntax bloat for binary bloat. Having 4 parameters with auto ref can generate 16 different variants for the exact same function. If you are doing any sort of mathematical calculations you could cause a potential cache miss for calling the same exact function because one of your parameters didn't end up being the same type of value. You are causing all this bloat and potential slowdowns all to avoid having to do this: float value0 = a + b; float value1 = c + d; float value2 = e + f; someFunc(value0, value1, value2); That's an inherent flaw in design. You obviously agree that there is a problem, but how can you justify "auto ref" being a "good" solution to the problem? It's a horrible one, it causes excessive bloat and potential slowdowns just to get a cleaner readable syntax. That shouldn't be a solution that is deemed acceptable.
Mar 24 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote:
 On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
 Auto ref allows the unnecessary copy to be avoided for lvalues
 and creates a temporary (as part of passing the value) for
 rvalues. It has downsides (virtual functions and extern(C++),
 but it does directly address the problem you're talking about,
 unless I have totally misunderstood you.
You are trading syntax bloat for binary bloat. Having 4 parameters with auto ref can generate 16 different variants for the exact same function. If you are doing any sort of mathematical calculations you could cause a potential cache miss for calling the same exact function because one of your parameters didn't end up being the same type of value. You are causing all this bloat and potential slowdowns all to avoid having to do this: float value0 = a + b; float value1 = c + d; float value2 = e + f; someFunc(value0, value1, value2); That's an inherent flaw in design. You obviously agree that there is a problem, but how can you justify "auto ref" being a "good" solution to the problem? It's a horrible one, it causes excessive bloat and potential slowdowns just to get a cleaner readable syntax. That shouldn't be a solution that is deemed acceptable.
How good or bad it is depends on what you're doing and how many auto ref parameters there are in your code. If you're using it occasionally, it's not a problem at all, whereas if you're using it all over the place, you do get a lot of template bloat. Regardless, John's point was that auto ref solves the problem of being able to call a function with both lvalues and rvalues without copying lvalues, and he didn't understand why anyone was trying to argue that it doesn't do that. And I agree with him on that point. It does not help with virtual functions or extern(C++), and it creates a lot of template bloat if it's used heavily, so there are downsides to it, but it _does_ solve the basic problem of being able to call a function with both lvalues and rvalues without copying the lvalues. It just doesn't solve it in a way that everyone considers acceptable. auto ref also helps with forwarding refness, so it's useful for more than just avoiding copying lvalues, but the entire reason that auto ref was originally added to the language was to solve this exact problem. And maybe we need a different solution for some use cases (like virtual functions or cases where the template bloat is deemed unacceptable), but auto ref is in the language to solve this problem. So, much as at may make sense to argue that it's not a good solution to this problem, it really doesn't make sense to argue that it has nothing to do with it. - Jonathan M Davis
Mar 24 2018
parent Rubn <where is.this> writes:
On Sunday, 25 March 2018 at 01:43:43 UTC, Jonathan M Davis wrote:
 On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote:
 On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
 Auto ref allows the unnecessary copy to be avoided for 
 lvalues and creates a temporary (as part of passing the 
 value) for rvalues. It has downsides (virtual functions and 
 extern(C++), but it does directly address the problem you're 
 talking about, unless I have totally misunderstood you.
You are trading syntax bloat for binary bloat. Having 4 parameters with auto ref can generate 16 different variants for the exact same function. If you are doing any sort of mathematical calculations you could cause a potential cache miss for calling the same exact function because one of your parameters didn't end up being the same type of value. You are causing all this bloat and potential slowdowns all to avoid having to do this: float value0 = a + b; float value1 = c + d; float value2 = e + f; someFunc(value0, value1, value2); That's an inherent flaw in design. You obviously agree that there is a problem, but how can you justify "auto ref" being a "good" solution to the problem? It's a horrible one, it causes excessive bloat and potential slowdowns just to get a cleaner readable syntax. That shouldn't be a solution that is deemed acceptable.
How good or bad it is depends on what you're doing and how many auto ref parameters there are in your code. If you're using it occasionally, it's not a problem at all, whereas if you're using it all over the place, you do get a lot of template bloat. Regardless, John's point was that auto ref solves the problem of being able to call a function with both lvalues and rvalues without copying lvalues, and he didn't understand why anyone was trying to argue that it doesn't do that. And I agree with him on that point. It does not help with virtual functions or extern(C++), and it creates a lot of template bloat if it's used heavily, so there are downsides to it, but it _does_ solve the basic problem of being able to call a function with both lvalues and rvalues without copying the lvalues. It just doesn't solve it in a way that everyone considers acceptable.
That's a horrible way to look at it. You can still technically drive a car with square wheels, it may not be a very comfortable or efficient ride, but it still __does__ solve the basic problem of moving the car. Because it works we shouldn't look at any other solution.
 auto ref also helps with forwarding refness, so it's useful for 
 more than just avoiding copying lvalues, but the entire reason 
 that auto ref was originally added to the language was to solve 
 this exact problem. And maybe we need a different solution for 
 some use cases (like virtual functions or cases where the 
 template bloat is deemed unacceptable), but auto ref is in the 
 language to solve this problem. So, much as at may make sense 
 to argue that it's not a good solution to this problem, it 
 really doesn't make sense to argue that it has nothing to do 
 with it.
I never said it has nothing to do with it. It's just a really horrible solution. If you are going to add an exception just for two minor cases, at that point you might as well start looking at adding rvalue references, cause that's what it is going to lead to. It'd provided better integration with C++ and provide an __acceptable__ solution to the problem (not one that solves the problem in a roundabout square-wheel-like way) that doesn't create runtime bloat and slowdown just to have clean readable syntax.
Mar 24 2018
prev sibling parent Rubn <where is.this> writes:
On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
 Here is a small proof of concept I made to demonstrate how easy 
 it seems to be to use `auto ref` to call a C++ virtual const& 
 function without incurring any more copies than would happen 
 with the same calls from C++. I'm sure it could be improved a 
 lot, but does the basic concept match what you would need?

 // D source file:

 /// mix this in to your extern(C++) class with a list of the 
 functions where you
 /// want to be able to pass rvalues to ref parameters.
 auto rValueRefCalls(Funcs ...)()
 {
     string ret;
     foreach (foo; Funcs)
         ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ 
 `(Args ...)(auto ref Args args)
     if (__traits(compiles, (&this.` ~ __traits(identifier, foo) 
 ~ `)(args)))
     {
         (&this.foo)(args);
     }`;

     return ret;
 }

 extern(C++)
 {
     class A
     {
         void foo(const ref int v);
         mixin(rValueRefCalls!foo);
     }

     A makeA();
 }

 void main()
 {
     int x = 3;
     auto a = makeA();
     a.foo(x);
     a.foo(3);
 }


 // C++ source file:

 #include<cstdio>

 class A
 {
 public:
     virtual void foo(const int& v)
     {
         printf("%d\n", v);
     }
 };

 A *makeA()
 {
     return new A;
 }
That isn't going to scale to the number of potential functions that are going to need this. It's going to cause slow compile speeds and create a bloated binary.
Mar 24 2018
prev sibling parent Rubn <where is.this> writes:
On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote:
 I understand what you want, but I'm struggling to understand 
 why it's such a huge deal.

 The reason you want to pass by reference is for performance, to 
 avoid copying the data at the call boundary.
It's pretty simple: float foo() { ... } ref float bar() { ... } void someFunc(ref float); someFunc(bar()); // ok float temp = foo(); someFunc(temp); // Have to create a temporary anyways for the function to work someFunc(foo()); // Compile error, need to use the hideous code above So there really isn't any performance penalty cause if you want to call that function you are going to have to create a temporary variable anyways, but now you can't just call the function in one line. It requires multiple lines and a temporary variable name. This becomes especially horrible for math libraries: void someFunc(ref Vector3) { ... } someFunc(a + b); // Can't do this Vector3 temp = a + b; someFunc(temp); // Have to do this So there's no performance penalty for what he is requesting, but allows for a cleaner syntax to do the exact same thing.
 So there are 2 cases: an lvalue needs to be passed, or an 
 rvalue needs to be passed.

 1. The address of the lvalue is passed.

 2. The rvalue is copied to a local, then the address of that 
 local is passed.

 So in the rvalue case, you're not getting the performance 
 benefit of passing by reference, because you have to copy to a 
 local anyway.

 What I would do in D currently to get the same performance and 
 API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }

 What is so totally unacceptable about those solutions? I 
 personally like the second because it scales better to multiple 
 parameters. I know you have said it's not relevant and annoying 
 that people bring up auto ref, but I dont' get how or why. It's 
 exactly D's solution to the problem.
It doesn't scale better, that's part of the problem: void foo()(auto ref MyType1, auto ref MyType2, auto ref MyType3, auto ref MyType4) { ... } The above has the possibility of generating 16 different functions that basically all do the exact same thing. It creates excessive bloat, and now what if you want to take the address of the function? You can't cause you have to choose between one of the 16 variants. This is template bloat at it's finest, except it isn't even doing anything useful. I can only imagine telling someone they have to code gen 16 identical functions just to be able to call a function without having to create useless temporary variables in the scope a function is being called in.
Mar 24 2018
prev sibling next sibling parent Dgame <r.schuett.1987 gmail.com> writes:
Here is what I've used if I had to: 
https://p0nce.github.io/d-idioms/#Rvalue-references:-Understanding-auto-ref-and-then-not-using-it
Mar 24 2018
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...
 <snip>
There are too many replies on this thread, addressing all the comments would take forever and pollute the thread itself. So forgive me if I say something that was covered already by someone else. AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when calling C++ functions that take `const T&` (especially since that is common). I have not yet heard any other use for them. I'd be in favour of allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or have overloads for pass-by-value and pass-by-ref. I too, once a recent immigrant from the lands of C++, used to keep writing `ref const(T)`. I just pass by value now. C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T C++ T& -> D ref T If replacing const T& with T chafes, I understand. I used to feel that way too. It's _possible_ that would incur a penalty in copying/moving, but IME the cost is either 0, negligible, or negative (!). As mentioned above, if calling C++ code there's no choice about using T instead of const T&, so for pragmatic reasons that should be allowed. But only there, because...
 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
Rvalue references. They exist because of being able to bind temporaries to const T& in C++, which means there's no way of knowing if your const T& was originally a temporary or not. To disambiguate C++11 introduced the type system horror that are rvalue references (which also do double-duty in enabling perfect forwarding!). D doesn't have or need rvalue references _because_ of not allowing temporaries to bind to ref const(T). You get move semantics in D without the pain. That's a win in my book. Atila
Mar 26 2018
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...
 <snip>
There are too many replies on this thread, addressing all the comments would take forever and pollute the thread itself. So forgive me if I say something that was covered already by someone else. AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when calling C++ functions that take `const T&` (especially since that is common). I have not yet heard any other use for them. I'd be in favour of allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or have overloads for pass-by-value and pass-by-ref. I too, once a recent immigrant from the lands of C++, used to keep writing `ref const(T)`. I just pass by value now. C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T C++ T& -> D ref T If replacing const T& with T chafes, I understand. I used to feel that way too. It's _possible_ that would incur a penalty in copying/moving, but IME the cost is either 0, negligible, or negative (!).
I'm tearing my remaining stubs of hair out trying to understand why memory copies (not talking about copy constructors) are needed when passing an rvalue to a non-ref function: https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy
Mar 26 2018
parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 11:13, John Colvin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...
 <snip>
There are too many replies on this thread, addressing all the comments would take forever and pollute the thread itself. So forgive me if I say something that was covered already by someone else. AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when calling C++ functions that take `const T&` (especially since that is common). I have not yet heard any other use for them. I'd be in favour of allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or have overloads for pass-by-value and pass-by-ref. I too, once a recent immigrant from the lands of C++, used to keep writing `ref const(T)`. I just pass by value now. C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T C++ T& -> D ref T If replacing const T& with T chafes, I understand. I used to feel that way too. It's _possible_ that would incur a penalty in copying/moving, but IME the cost is either 0, negligible, or negative (!).
I'm tearing my remaining stubs of hair out trying to understand why memory copies (not talking about copy constructors) are needed when passing an rvalue to a non-ref function: https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy
Passing rvalues to non-ref functions may elide a memory copy. Moves can be very efficient. But we're talking about ref functions right? Not not-ref functions...?
Mar 26 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...
 <snip>
There are too many replies on this thread, addressing all the comments would take forever and pollute the thread itself. So forgive me if I say something that was covered already by someone else. AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when calling C++ functions that take `const T&` (especially since that is common). I have not yet heard any other use for them. I'd be in favour of allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or have overloads for pass-by-value and pass-by-ref. I too, once a recent immigrant from the lands of C++, used to keep writing `ref const(T)`. I just pass by value now. C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user.
 C++ T& -> D ref T
I agree the other 3 cases are correct.
Mar 26 2018
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/26/2018 12:24 PM, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user.
Copying does suck, I agree. Consider the following: void foo(T t) { foo(t); } <= add this overload void foo(ref T t) { ... } T aaa(); foo(aaa()); With inlining, I suspect we can get the compiler to not make any extra copies. It's not that different from NRVO. And as a marvy bonus, no weird semantic problems (as Atila mentioned).
Mar 26 2018
next sibling parent reply Rubn <where is.this> writes:
On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:
 On 3/26/2018 12:24 PM, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user.
Copying does suck, I agree. Consider the following: void foo(T t) { foo(t); } <= add this overload void foo(ref T t) { ... } T aaa(); foo(aaa()); With inlining, I suspect we can get the compiler to not make any extra copies. It's not that different from NRVO. And as a marvy bonus, no weird semantic problems (as Atila mentioned).
How do you add this overload for the following? void foo(ref T t) { ... } void function(ref int) func = &foo; int aaa(); func(aaa()); // err
Mar 26 2018
next sibling parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 16:21, Rubn via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:
 On 3/26/2018 12:24 PM, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user.
Copying does suck, I agree. Consider the following: void foo(T t) { foo(t); } <= add this overload void foo(ref T t) { ... } T aaa(); foo(aaa()); With inlining, I suspect we can get the compiler to not make any extra copies. It's not that different from NRVO. And as a marvy bonus, no weird semantic problems (as Atila mentioned).
How do you add this overload for the following? void foo(ref T t) { ... } void function(ref int) func = &foo; int aaa(); func(aaa()); // err
Exactly.
Mar 26 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 19:25, Manu <turkeyman gmail.com> wrote:
 On 26 March 2018 at 16:21, Rubn via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:
 On 3/26/2018 12:24 PM, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user.
Copying does suck, I agree. Consider the following: void foo(T t) { foo(t); } <= add this overload void foo(ref T t) { ... } T aaa(); foo(aaa()); With inlining, I suspect we can get the compiler to not make any extra copies. It's not that different from NRVO. And as a marvy bonus, no weird semantic problems (as Atila mentioned).
How do you add this overload for the following? void foo(ref T t) { ... } void function(ref int) func = &foo; int aaa(); func(aaa()); // err
Exactly.
We're just kicking the can. And the only reason to do so is ideological, as far as I can tell. I want to hear an argument against... or any issue that's introduced by allowing the implicit temporary?
Mar 26 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 15:48, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 3/26/2018 12:24 PM, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user.
Copying does suck, I agree. Consider the following: void foo(T t) { foo(t); } <= add this overload void foo(ref T t) { ... } T aaa(); foo(aaa()); With inlining, I suspect we can get the compiler to not make any extra copies. It's not that different from NRVO. And as a marvy bonus, no weird semantic problems (as Atila mentioned).
It's a terrible experience to add 2^n overloads, just to abuse the parameter list (as in your example) to create the temporary in an implicit manner. The compiler can easily create the exact same temporary without requiring a bunch of overloads to be declared. What's the advantage of requiring that effort from the user, rather than just doing it at the call site? There's side effects to that hack too. Now there are overloads; so taking function pointers becomes awkward (might be important in some cases). If the interface has a binary boundary (static lib/dll) then what does that look like with respect to inlining? Where do the overloads go if the function is virtual? I already know the answers to these questions, but the point is, there's a whole lot more baggage introduced into the scene that just doesn't need to be there. So, while I agree that's an existing workaround, it kinda misses the point. This thread isn't about inlining, it's about NOT inlining. You don't use ref args unless you have a reason to, and that reason is likely to have friction with that particular work-around.
Mar 26 2018
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...
 <snip>
C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do.
That's _if_ T is big and _if_ it even gets copied, the combination of which I think happens very rarely. When that happens I think that the temporary isn't a big deal. This code: struct Foo { int[1024] data; } int byValue(Foo f) { return f.data[42]; } Generates this assembly (ldc2 -O3, clang does the same for C++): 0000000000000000 <_D3foo7byValueFSQo3FooZi>: 0: 8b 84 24 b0 00 00 00 mov eax,DWORD PTR [rsp+0xb0] 7: c3 ret And I wrote a type for which a move and a copy are the same on purpose: in "real life" it's more likely that the memory will be dynamically allocated, probably held in a slice, and moved instead. Are there cases in which there will be an expensive copy? Yes, then pass by ref/pointer. But measure first, and prefer to pass by value by default. It's different in C++ - stick a std::vector or std::string in your struct and passing by value (usually) copies the dynamically allocated memory. In D it moves.
 As an API author, exactly as in C++, you will make a judgement 
 on a
 case-by-case basis on this matter. It may be by-value, it may 
 be by
 const-ref. It depends on a bunch of things, and they are points 
 for
 consideration by the API author, not the user.
You can still do that in D. There well may be a reason to pass by const ref. I'm just saying that there aren't that many, and in that in those rare cases a temporary is fine. Especially if the alternative are rvalue references. Atila
Mar 27 2018
prev sibling parent reply Rubn <where is.this> writes:
On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
 C++ T&& (Rvalue reference) -> D T
Not really, in C++ it is an actual reference and you get to choose which function actually does the move. In D it just does the copy when passed to the function. So you can't do this in D. void bar(T&& t) { // actually move contents of T } void foo(T&& t) { bar(std::forward<T>(t)); // Can't do this in D without making another actual copy cause it isn't a reference }
 If replacing const T& with T chafes, I understand. I used to 
 feel that way too. It's _possible_ that would incur a penalty 
 in copying/moving, but IME the cost is either 0, negligible, or 
 negative (!).

 As mentioned above, if calling C++ code there's no choice about 
 using T instead of const T&, so for pragmatic reasons that 
 should be allowed. But only there, because...

 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
Rvalue references. They exist because of being able to bind temporaries to const T& in C++, which means there's no way of knowing if your const T& was originally a temporary or not. To disambiguate C++11 introduced the type system horror that are rvalue references (which also do double-duty in enabling perfect forwarding!).
What's a concrete example that you would be required to know whether a const& is a temporary or not. I don't really see it otherwise. The solution everyone is saying to use as an alternative would be the literal case of how it would be implemented in the language.
 D doesn't have or need rvalue references _because_ of not 
 allowing temporaries to bind to ref const(T). You get move 
 semantics in D without the pain. That's a win in my book.

 Atila
I've come across a few pains of such. It make be easier to use but it comes at a performance hit. In part binaries become huge because of how "init" is implemented. struct StaticArray(T, size_t capacity) { size_t length; T[capacity] values; } Copying the above structure copies unnecessary data for any move/copy operation. Eg when length = 0, it'll still copy everything. This includes initialization.
Mar 26 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 17:30, Rubn via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
 C++ T&& (Rvalue reference) -> D T
Not really, in C++ it is an actual reference and you get to choose which function actually does the move. In D it just does the copy when passed to the function. So you can't do this in D. void bar(T&& t) { // actually move contents of T } void foo(T&& t) { bar(std::forward<T>(t)); // Can't do this in D without making another actual copy cause it isn't a reference }
You can still get there with D, if you assume the forwarding function is simple enough to inline (and the code is public). I was cautiously concerned about this for a long time, but I've never made a noise about it. I've digested and resolved that's okay though :) There are very few cases where you will fail to achieve equivalent efficiency, and I think D's amazingly simplified move semantics more than compensate for any conceivable loss.
 If replacing const T& with T chafes, I understand. I used to feel that way
 too. It's _possible_ that would incur a penalty in copying/moving, but IME
 the cost is either 0, negligible, or negative (!).

 As mentioned above, if calling C++ code there's no choice about using T
 instead of const T&, so for pragmatic reasons that should be allowed. But
 only there, because...

 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
Rvalue references. They exist because of being able to bind temporaries to const T& in C++, which means there's no way of knowing if your const T& was originally a temporary or not. To disambiguate C++11 introduced the type system horror that are rvalue references (which also do double-duty in enabling perfect forwarding!).
What's a concrete example that you would be required to know whether a const& is a temporary or not. I don't really see it otherwise. The solution everyone is saying to use as an alternative would be the literal case of how it would be implemented in the language.
He's trying to say that C++ introduced rvalue references because normal references weren't able to allow for move semantics to exist. It's a red-herring. D already has move semantics, they work well, and they're not on trial here. In C++'s case, it's not that references were deficient at being references that C++ needed rval-references, it's that references were deficient at being move-able. That is not a problem that exists in D. It's fine for references to just be references in D. We're not struggling to make references move-able in D, that's not a thing, we already have move semantics. Any extension of this conversation about references into C++ rvalue-references (T&&) and or move-semantics are red-herrings. There's no such problem in D that needs to be resolved, and the existing solution is excellent.
Mar 26 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 27 March 2018 at 02:41:12 UTC, Manu wrote:

 He's trying to say that C++ introduced rvalue references 
 because normal references weren't able to allow for move 
 semantics to exist. It's a red-herring. D already has move 
 semantics, they work well, and they're not on trial here.

 In C++'s case, it's not that references were deficient at being
 references that C++ needed rval-references, it's that 
 references were
 deficient at being move-able.
There were deficient at being moveable because temporaries can bind to const T&.
 That is not a problem that exists in D.
Because temporaries can't bind to ref const(T).
 It's fine for references to
 just be references in D. We're not struggling to make references
 move-able in D, that's not a thing, we already have move 
 semantics.
 Any extension of this conversation about references into C++
 rvalue-references (T&&) and or move-semantics are red-herrings.
 There's no such problem in D that needs to be resolved, and the
 existing solution is excellent.
If I'm reading you correctly (which I might not), you seem to be saying that there's a way forward in which: 1) D's move semantics aren't affected 2) No rvalue references are introduced 3) Temporaries can bind to ref const(T) I'd love to know what that would look like. Atila
Mar 27 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Forked from the x^^y thread...
 <snip>
C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T
Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do.
That's _if_ T is big and _if_ it even gets copied,
You've just described the exact anatomy of a ref function! You wouldn't write a function to receive T by ref UNLESS T was both big, and the function probably won't inline (therefore definitely copy), and that condition will be triggered by any of the list of reasons I've said a bunch of times (extern, dll, lib, virtual, etc). People don't just love writing ref (well, some people might), but they use it deliberately, typically in user-facing boundary API's for these exact reasons.
 the combination of which
 I think happens very rarely. When that happens I think that the temporary
 isn't a big deal. This code:

 struct Foo { int[1024] data; }
 int byValue(Foo f) { return f.data[42]; }

 Generates this assembly (ldc2 -O3, clang does the same for C++):

 0000000000000000 <_D3foo7byValueFSQo3FooZi>:
    0:   8b 84 24 b0 00 00 00    mov    eax,DWORD PTR [rsp+0xb0]
    7:   c3                      ret

 And I wrote a type for which a move and a copy are the same on purpose: in
 "real life" it's more likely that the memory will be dynamically allocated,
 probably held in a slice, and moved instead.

 Are there cases in which there will be an expensive copy? Yes, then pass by
 ref/pointer. But measure first, and prefer to pass by value by default.
Right. I'm talking about deliberate use of ref... Or *existing* (deliberate) use of ref, as is the case in almost all my my cases. The code already exists. As you assess, use of ref in D is fairly rare. I've been looking for cases where other people are inconvenienced by this... hard to find. I suspect there are reasons for this. One of them is that this inconvenience suppresses it; ie, you will choose not to use a ref even when you might prefer to. Others include the fact that extern(C++) isn't super popular, DLL's aren't popular, closed-source distributed code is non popular, OOP is not popular, etc.
 It's different in C++ - stick a std::vector or std::string in your struct
 and passing by value (usually) copies the dynamically allocated memory. In D
 it moves.
Only if you ARE moving, and not copying. D must deep copy too if you actually copy. Your example assumes C++ doesn't have a move constructor. D has implicit move semantics, so you can only make an equivalent comparison where C++ also defines the move constructor so the move case doesn't pollute the ref comparison. Also, irrespective of whether move semantics are performed (eliding potential deep copying, as in your example), the binary memcpy still has to be performed when handling values by-val, unless RVO (we're not talking about return values), or inlining is able to eliminate it. Also, we're not talking about move semantics!
 As an API author, exactly as in C++, you will make a judgement on a
 case-by-case basis on this matter. It may be by-value, it may be by
 const-ref. It depends on a bunch of things, and they are points for
 consideration by the API author, not the user.
You can still do that in D. There well may be a reason to pass by const ref. I'm just saying that there aren't that many, and in that in those rare cases a temporary is fine. Especially if the alternative are rvalue references.
We're not talking about rvalue references, we're talking about normal references >_<
 He's trying to say that C++ introduced rvalue references because normal
 references weren't able to allow for move semantics to exist. It's a
 red-herring. D already has move semantics, they work well, and they're not
 on trial here.

 In C++'s case, it's not that references were deficient at being
 references that C++ needed rval-references, it's that references were
 deficient at being move-able.
There were deficient at being moveable because temporaries can bind to const T&.
... what? That's just not true at all. If temporaries couldn't bind to C++ ref, then you *definitely* wouldn't be able to move it, because you can guarantee that someone else owns the reference. You can't move references, under any circumstances, in either language... and that's actually the whole point of references. rvalue references were introduced in C++ to capture the calls with rvalues into a separate function call, exactly the same way as the by-value overload will catch the rvalues in D (and perform an implicit move). It was impossible for C++ to implement D's implicit move semantics when receiving by-value for reasons that have nothing to do with references; C++ allows interior pointers which breaks implicit moving, C++ can overload default constructor which also breaks implicit moving (no implicit way to reset the state of the prior owner). That's the reason that C++ was forced to introduce rvalue references, rather than implement implicit move logic like in D. It was the choice of D to ban interior pointers, and dis-allow overloading the default struct constructor which supports D's implicit move semantics. If D didn't have those 2 things, it would need rvalue-ref's the same as C++ for the same reasons. References have no interaction with move semantics. But again, we're not talking about move semantics here, we're just talking about references ;)
 That is not a problem that exists in D.
Because temporaries can't bind to ref const(T).
No. This truly has nothing to do with it. rvalues would continue to choose the by-val function, performing a move whenever possible. All existing rules are correct as they are.
 It's fine for references to
 just be references in D. We're not struggling to make references
 move-able in D, that's not a thing, we already have move semantics.
 Any extension of this conversation about references into C++
 rvalue-references (T&&) and or move-semantics are red-herrings.
 There's no such problem in D that needs to be resolved, and the
 existing solution is excellent.
If I'm reading you correctly (which I might not), you seem to be saying that there's a way forward in which: 1) D's move semantics aren't affected 2) No rvalue references are introduced 3) Temporaries can bind to ref const(T) I'd love to know what that would look like.
That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated) As far as I can tell, it's completely benign, it just eliminates the annoying edge cases when interacting with functions that take arguments by ref. There's no spill-over affect anywhere that I'm aware of, and if you can find a single wart, I definitely want to know about it. I've asked so many times for a technical destruction, nobody will present any opposition that is anything other than a rejection *in principle*. This is a holy war, not a technical one. And as far as I can tell, it basically only affects me, because I do so much work against established C++ code! >_<
Mar 27 2018
next sibling parent Peter Campbell <peter spcampbell.co.uk> writes:
On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:
 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
  (contribution appreciated)
I've followed this thread since it was made as this has been one of the very few disappointments of the language for me. I only tend to write game code and use D for hobbyist projects whilst using C++ full-time as a junior at a small games company. Even if I take the attitude that I should use D as it is intended, instead of trying to write C++ by using D, it always felt unnecessarily obstructive to require me to make a temporary variable to avoid copying something simple like a vector or a matrix. It feels very restrictive when trying to express mathematical calculations in a concise manner. Thanks for writing that DIP, you have covered everything I would love to see in great detail with good examples! I honestly couldn't think of anything more that could be added.
Mar 27 2018
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 27.03.2018 20:14, Manu wrote:
 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
   (contribution appreciated)
 
 As far as I can tell, it's completely benign, it just eliminates the
 annoying edge cases when interacting with functions that take
 arguments by ref. There's no spill-over affect anywhere that I'm aware
 of, and if you can find a single wart, I definitely want to know about
 it.
???
 I've asked so many times for a technical destruction, nobody will
 present any opposition that is anything other than a rejection *in
 principle*. This is a holy war, not a technical one.
That's extremely unfair. It is just a bad idea to overload D const for this purpose. Remove the "const" requirement and I'm on board.
Mar 28 2018
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 28.03.2018 13:34, Timon Gehr wrote:
 On 27.03.2018 20:14, Manu wrote:
 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md 

   (contribution appreciated)

 As far as I can tell, it's completely benign, it just eliminates the
 annoying edge cases when interacting with functions that take
 arguments by ref. There's no spill-over affect anywhere that I'm aware
 of, and if you can find a single wart, I definitely want to know about
 it.
??? >> I've asked so many times for a technical destruction, nobody will
 present any opposition that is anything other than a rejection *in
 principle*. This is a holy war, not a technical one.
That's extremely unfair. It is just a bad idea to overload D const for this purpose. Remove the "const" requirement and I'm on board.
"The proposal could be amended to accept mutable ref's depending on the value-judgement balancing these 2 use cases. Sticking with const requires no such value judgement to be made at this time, and it's much easier to relax the spec in the future with emergence of evidence to do so." Just get it right the first time. "const" is a serious API restriction, and it shouldn't be forced on anyone, even intermittently until they figure out that it is too restrictive (as well as viral).
Mar 28 2018
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/28/18 7:50 AM, Timon Gehr wrote:
 "The proposal could be amended to accept mutable ref's depending on the 
 value-judgement balancing these 2 use cases. Sticking with const 
 requires no such value judgement to be made at this time, and it's much 
 easier to relax the spec in the future with emergence of evidence to do 
 so."
 
 Just get it right the first time. "const" is a serious API restriction, 
 and it shouldn't be forced on anyone, even intermittently until they 
 figure out that it is too restrictive (as well as viral).
A great way to move things forward here, Timon, is to write a pull request against the DIP with motivating text and examples.
Apr 01 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01.04.2018 19:20, Andrei Alexandrescu wrote:
 On 3/28/18 7:50 AM, Timon Gehr wrote:
 "The proposal could be amended to accept mutable ref's depending on 
 the value-judgement balancing these 2 use cases. Sticking with const 
 requires no such value judgement to be made at this time, and it's 
 much easier to relax the spec in the future with emergence of evidence 
 to do so."

 Just get it right the first time. "const" is a serious API 
 restriction, and it shouldn't be forced on anyone, even intermittently 
 until they figure out that it is too restrictive (as well as viral).
A great way to move things forward here, Timon, is to write a pull request against the DIP with motivating text and examples.
I agree, but unfortunately I have many other things on my plate right now. Add to this that there are six or seven other DIPs that I really ought to finish/write/implement/rebase. I'll try to get back to this soon. Here, I was trying to make sure that popular misconceptions do not gain more traction.
Apr 01 2018
parent Manu <turkeyman gmail.com> writes:
On 1 April 2018 at 11:55, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 01.04.2018 19:20, Andrei Alexandrescu wrote:
 On 3/28/18 7:50 AM, Timon Gehr wrote:
 "The proposal could be amended to accept mutable ref's depending on the
 value-judgement balancing these 2 use cases. Sticking with const requires no
 such value judgement to be made at this time, and it's much easier to relax
 the spec in the future with emergence of evidence to do so."

 Just get it right the first time. "const" is a serious API restriction,
 and it shouldn't be forced on anyone, even intermittently until they figure
 out that it is too restrictive (as well as viral).
A great way to move things forward here, Timon, is to write a pull request against the DIP with motivating text and examples.
I agree, but unfortunately I have many other things on my plate right now. Add to this that there are six or seven other DIPs that I really ought to finish/write/implement/rebase. I'll try to get back to this soon. Here, I was trying to make sure that popular misconceptions do not gain more traction.
I'm convinced. There are lots of reasons it should not only apply to const. I only originally went that way because it was more conservative, easier to relax, and because the stated reason why you can't _already_ do this thing is allegedly because people freak out at the idea that a function might return data into a temporary. 'const' prevents that from being a possibility, but unlike C++, there are significant useful cases for not restricting to const. The most exciting for me is pipeline programming (ie, UFCS chains), which are a major winning feature of D. They'll be more convenient among other things.
Apr 01 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 28 Mar. 2018 4:35 am, "Timon Gehr via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

On 27.03.2018 20:14, Manu wrote:

 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx
 -rval_to_ref.md
   (contribution appreciated)

 As far as I can tell, it's completely benign, it just eliminates the
 annoying edge cases when interacting with functions that take
 arguments by ref. There's no spill-over affect anywhere that I'm aware
 of, and if you can find a single wart, I definitely want to know about
 it.
??? I've asked so many times for a technical destruction, nobody will
 present any opposition that is anything other than a rejection *in
 principle*. This is a holy war, not a technical one.
That's extremely unfair. It is just a bad idea to overload D const for this purpose. Remove the "const" requirement and I'm on board. I discussed that in that document. I'm happy to remove const, but it requires a value judgement on the meaning of non-const in this case. It becomes controversial without const, but I'm personally happy to remove it if you can make The argument in favour. Can you give me some ideas where it would be useful?
Mar 28 2018
parent reply Kagamin <spam here.lot> writes:
On Wednesday, 28 March 2018 at 16:15:53 UTC, Manu wrote:
 I discussed that in that document. I'm happy to remove const, 
 but it requires a value judgement on the meaning of non-const 
 in this case. It becomes controversial without const, but I'm 
 personally happy to remove it if you can make The argument in 
 favour. Can you give me some ideas where it would be useful?
doesn't make sense because the output would be immediately 
discarded; such a function call given an rvalue as argument 
likely represents an accidental mistake on the users part, and 
we can catch that invalid code.
Most obvious: void Close(ref HANDLE h) { CloseHandle(h); h=INVALID_HANDLE; } If you want to take input argument, then of course mark it as input, but if not then not. Why the callee would need to care if the argument is rvalue or not? The only criticism against rvalue references I saw was when the reference outlives the temporary.
Mar 30 2018
parent Manu <turkeyman gmail.com> writes:
On 30 March 2018 at 02:47, Kagamin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 The only criticism against rvalue references I saw was when the
 reference outlives the temporary.
That's the only *technical* criticism I've ever heard too. Fortunately, D now has mechanisms for preventing that; D's safety story with respect to reference lifetime seems to be solid now. 'return ref', and enforcing ref pointers don't escape the callee cleared to the road to allow accepting temporaries safety.
Mar 30 2018
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 27.03.2018 20:14, Manu wrote:
 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
   (contribution appreciated)
"Temporary destruction Destruction of any temporaries occurs naturally at the end of the scope, as usual." This is actually unusual. --- import std.stdio; struct S{ ~this(){ writeln("destroyed!"); } } void foo(S s){} void main(){ foo(S()); writeln("end the scope"); } --- prints: destroyed! end the scope "Overload resolution ... This follows existing language rules. No change is proposed here." Yes, you _are_ proposing a change. Right now rvalues "prefer" the by-value overload because the ref overload _does not match at all_. Now you make it match both, so you are adding additional disambiguation rules. You need to be more explicit about those. Note that lvalues prefer the ref overload because the ref overload is more specialized. The new rule is the only instance where a less specialized overload is preferred. You also need to specify the interactions with matching levels (https://dlang.org/spec/function.html#function-overloading): E.g., your DIP is compatible with the following behavior: --- import std.stdio; struct S{} void fun(S){ writeln("A"); } void fun(ref const(S)){ writeln("B"); } void main(){ fun(S()); // calls A S s; fun(s); // calls A const(S) t; fun(t); // calls B fun(const(S)()); // calls B } --- The first example will cause friction when people try to add an explicit rvalue overload alongside a previous catch-it-all overload, the second example shows a breaking language change. You cannot "fix" the first example without introducing breaking language changes. The above code compiles and runs in current D. This just smells bad. Remove the "const" requirement.
Mar 28 2018
parent reply Manu <turkeyman gmail.com> writes:
On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 27.03.2018 20:14, Manu wrote:
 That's exactly what I've been saying. For like, 9 years..
 It looks like this:

 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
   (contribution appreciated)
"Temporary destruction Destruction of any temporaries occurs naturally at the end of the scope, as usual." This is actually unusual. --- import std.stdio; struct S{ ~this(){ writeln("destroyed!"); } } void foo(S s){} void main(){ foo(S()); writeln("end the scope"); } --- prints: destroyed! end the scope
Right, exactly... So, what's wrong?
 "Overload resolution
 ...
 This follows existing language rules. No change is proposed here."

 Yes, you _are_ proposing a change. Right now rvalues "prefer" the by-value
 overload because the ref overload _does not match at all_. Now you make it
 match both, so you are adding additional disambiguation rules. You need to
 be more explicit about those.
Oh right... yeah okay, I'll tweak the language.
 Note that lvalues prefer the ref overload because the ref overload is more
 specialized. The new rule is the only instance where a less specialized
 overload is preferred.
I've never heard any discussion involving the term 'specialised', or seen any definition where overloading prefers a "more specialised' version... is that a thing? Given: void f(int); void f(const(int)); f(10); That calls the 'int' one, but it could call either one... that's definitely not choosing a 'more specialised' match.
 You also need to specify the interactions with matching levels
 (https://dlang.org/spec/function.html#function-overloading):

 E.g., your DIP is compatible with the following behavior:

 ---
 import std.stdio;

 struct S{}
 void fun(S){ writeln("A"); }
 void fun(ref const(S)){ writeln("B"); }

 void main(){
     fun(S()); // calls A
     S s;
     fun(s); // calls A

     const(S) t;
     fun(t); // calls B
     fun(const(S)()); // calls B
 }
 ---

 The first example will cause friction when people try to add an explicit
 rvalue overload alongside a previous catch-it-all overload, the second
 example shows a breaking language change.

 You cannot "fix" the first example without introducing breaking language
 changes. The above code compiles and runs in current D.

 This just smells bad. Remove the "const" requirement.
This is very compelling reason to remove the const.
Mar 28 2018
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 28.03.2018 20:20, Manu wrote:
 On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 27.03.2018 20:14, Manu wrote:
 That's exactly what I've been saying. For like, 9 years..
 It looks like this:

 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
    (contribution appreciated)
"Temporary destruction Destruction of any temporaries occurs naturally at the end of the scope, as usual." This is actually unusual. ...
... So, what's wrong? ...
In my example, the temporary is destroyed after/at the end of the function call, not at the end of the scope. Re-reading the DIP, I think you meant the right thing, but the wording is a bit confusing. Maybe just clarify that "the scope" is the one your rewrite introduces implicitly, or explicitly state that the lifetime ends at the end of the function call.
 
 "Overload resolution
 ...
 Note that lvalues prefer the ref overload because the ref overload is more
 specialized. The new rule is the only instance where a less specialized
 overload is preferred.
I've never heard any discussion involving the term 'specialised', or seen any definition where overloading prefers a "more specialised' version... is that a thing? Given: void f(int); void f(const(int)); f(10); That calls the 'int' one, but it could call either one...
The overload resolution rules in D have four different matching levels: - exact match - match with type qualifier conversion - match with general implicit conversion - no match The matching level for one overload is the minimal matching level for any argument. In your example f(int) matches exactly, but f(const(int)) matches with type qualifier conversion, therefore f(int) is chosen as it is the unique function that matches best. Only if two overloads match with the same best level is specialization used. An overload A is more specialized than another overload B if we can call B with all arguments with which we can call A. As it is possible to call a by-value function with an lvalue or an rvalue, but ref cannot be called with an rvalue, ref is more specialized.
 that's definitely not choosing a 'more specialised' match.
 ...
Implicit conversions are ignored when checking for specialization so, yes, here both functions are equally specialized. However, f(int*) is more specialized than f(const(int)*): --- import std.stdio; void f(int* a){ writeln("A"); } void f(const(int)* b){ writeln("B"); } void main(){ f(new immutable(int)); // guess what this prints. :) } ---
Mar 28 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:
 On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 That's _if_ T is big and _if_ it even gets copied,
You've just described the exact anatomy of a ref function! You wouldn't write a function to receive T by ref UNLESS T was both big, and the function probably won't inline (therefore definitely copy), and that condition will be triggered by any of the list of reasons I've said a bunch of times (extern, dll, lib, virtual, etc). People don't just love writing ref (well, some people might), but they use it deliberately, typically in user-facing boundary API's for these exact reasons.
I know. I was arguing that those cases are uncommon and the API pain is therefore not too big of an issue. I'm pretty sure that you feel it more because of what you write in D.
 Right. I'm talking about deliberate use of ref... Or *existing*
 (deliberate) use of ref, as is the case in almost all my my 
 cases. The
 code already exists.
Right, and I was assuming (perhaps incorrectly) that this existing code was C++, hence me being on board with binding rvalues to const ref there.
 Only if you ARE moving, and not copying. D must deep copy too 
 if you
 actually copy.
 Your example assumes C++ doesn't have a move constructor. D has
 implicit move semantics, so you can only make an equivalent 
 comparison
 where C++ also defines the move constructor so the move case 
 doesn't
 pollute the ref comparison.
I wasn't assuming the lack of a move constructor. What I was saying is that passing by value in C++ will usually mean a copy, whereas in D it usually means a move.
 Also, irrespective of whether move semantics are performed 
 (eliding
 potential deep copying, as in your example), the binary memcpy 
 still
 has to be performed when handling values by-val, unless RVO 
 (we're not
 talking about return values), or inlining is able to eliminate 
 it.
Good point about memcpy.
 In C++'s case, it's not that references were deficient at 
 being references that C++ needed rval-references, it's that 
 references were deficient at being move-able.
There were deficient at being moveable because temporaries can bind to const T&.
... what? That's just not true at all. If temporaries couldn't bind to C++ ref, then you *definitely* wouldn't be able to move it, because you can guarantee that someone else owns the reference.
Precisely. That's how D works.
 rvalue references were introduced in C++ to capture the calls 
 with
 rvalues into a separate function call, exactly the same way as 
 the
 by-value overload will catch the rvalues in D (and perform an 
 implicit
 move).
Yes.
 It was impossible for C++ to implement D's implicit move 
 semantics
 when receiving by-value for reasons that have nothing to do with
 references; C++ allows interior pointers which breaks implicit 
 moving,
 C++ can overload default constructor which also breaks implicit 
 moving
 (no implicit way to reset the state of the prior owner).
I hadn't thought about the implications of interior pointers. Very good point.
 References have no interaction with move semantics.
Even given interior pointers, I disagree.
 But again, we're not talking about move semantics here, we're 
 just talking about references ;)
See comment above ;)
 I'd love to know what that would look like.
That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated)
I was unaware of this (or I forgot). After reading it I'm not sure of what corner cases might arise, but if I'm getting it right I think it could work.
 And as far as I
 can tell, it basically only affects me, because I do so much 
 work
 against established C++ code! >_<
That's entirely possible. I can use my fingers to count the number of times I've written `extern(C++)`. Atila
Mar 30 2018
next sibling parent Manu <turkeyman gmail.com> writes:
On 30 March 2018 at 02:06, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:
 On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:
 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 That's _if_ T is big and _if_ it even gets copied,
You've just described the exact anatomy of a ref function! You wouldn't write a function to receive T by ref UNLESS T was both big, and the function probably won't inline (therefore definitely copy), and that condition will be triggered by any of the list of reasons I've said a bunch of times (extern, dll, lib, virtual, etc). People don't just love writing ref (well, some people might), but they use it deliberately, typically in user-facing boundary API's for these exact reasons.
I know. I was arguing that those cases are uncommon and the API pain is therefore not too big of an issue. I'm pretty sure that you feel it more because of what you write in D.
Totally. But I think it's uncommon at least in-part because of this inconvenience. It's at least a little bit circular. Even you're saying above "yeah, just do by-val, it's *probably* insignificant"... you only say that because you've allowed this inconvenience to change your behaviour. And I think that's to be expected. Other reasons that it's not common: extern(C++) is not common. OOP (virtuals) in D are fairly uncommon. We don't OOP much in D. Closed-source (binary lib) distribution is uncommon... possibly unheard of? (yet!) DLL's are still problematic onvarious fronts, and I don't think they enjoy anywhere near as much use as they deserve. Most of these aren't desirable... we'd like to see more commercial (closed source?) users, DLL's should be more popular than they are... and more extern(C++) means more people using a key development investment in D. So yeah, I agree it's relatively uncommon if you don't interact with one of the niches where it tends to be common! :P
 Right. I'm talking about deliberate use of ref... Or *existing*
 (deliberate) use of ref, as is the case in almost all my my cases. The
 code already exists.
Right, and I was assuming (perhaps incorrectly) that this existing code was C++, hence me being on board with binding rvalues to const ref there.
Right. But I'm not a fan is just rearranging the edge cases by limiting it to a different set of cases and not applying it generally. This also interacts with meta constructions, and while there are ANY cases where calling doesn't just work as usual and it needs special case handling, meta will still need static-if's to handle those cases. We're just changing the criteria for the 'if'. By allowing fully symmetric function calling rules, only then does the noisy case-handling logic disappear from meta.
 Only if you ARE moving, and not copying. D must deep copy too if you
 actually copy.
 Your example assumes C++ doesn't have a move constructor. D has
 implicit move semantics, so you can only make an equivalent comparison
 where C++ also defines the move constructor so the move case doesn't
 pollute the ref comparison.
I wasn't assuming the lack of a move constructor. What I was saying is that passing by value in C++ will usually mean a copy, whereas in D it usually means a move.
Right. But we're not talking about move's, we're talking about NOT-move's (ie, preventing copies by passing by ref) ;) Despite the appearance that we might be talking about rvalues, we're actually talking about passing lvalues by ref... that's WHY you write a function to accept its args by ref; to prevent deep copies when passing lvalues. The issue is, it's super-common to call functions with rvalues, and the edge cases created by using ref are annoying and asymmetric, hence such functions should receive rvalues too.
 In C++'s case, it's not that references were deficient at being
 references that C++ needed rval-references, it's that references were
 deficient at being move-able.
There were deficient at being moveable because temporaries can bind to const T&.
... what? That's just not true at all. If temporaries couldn't bind to C++ ref, then you *definitely* wouldn't be able to move it, because you can guarantee that someone else owns the reference.
Precisely. That's how D works.
Sorry, I'm lost now. I don't understand your initial point. You created a relationship between ref's accepting rvalues, and the reason that C++ introduced rvalue-references. No such relationship exists... and I was trying to show that your reasoning was inverted.
 rvalue references were introduced in C++ to capture the calls with
 rvalues into a separate function call, exactly the same way as the
 by-value overload will catch the rvalues in D (and perform an implicit
 move).
Yes.
 References have no interaction with move semantics.
Even given interior pointers, I disagree.
Please explain how there's any relationship between ref functions and move semantics? C++/D's mechanism for catching the move case is different, but that still doesn't affect or interact with the ref case. ref accepting rvalues has absolutely no effect on move semantics, either theoretically in D, or practically in C++. It's a totally separate problem space.
 But again, we're not talking about move semantics here, we're just talking
 about references ;)
See comment above ;)
Likewise :P
 I'd love to know what that would look like.
That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated)
I was unaware of this (or I forgot). After reading it I'm not sure of what corner cases might arise, but if I'm getting it right I think it could work.
Right. So does that admission dismiss your points above where you suggest there's some reason that C++ ref functions accepting rvalues lead to T&&? As far as I can tell from your prior posts, your only resistance is the idea of a 'slippery slope' that leads to T&& appearing in D? I promise, I'm just as concerned that never happens as you :) If you can find a connection, I want to know about it. But I'm confident that no such thing exists..
 And as far as I
 can tell, it basically only affects me, because I do so much work
 against established C++ code! >_<
That's entirely possible. I can use my fingers to count the number of times I've written `extern(C++)`.
Exactly. I'd like to think that my set of use cases are not cases that we want to discourage however. I'd like to see more users like myself in the future...
Mar 30 2018
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 30.03.2018 11:06, Atila Neves wrote:
 
 Right, and I was assuming (perhaps incorrectly) that this existing code 
 was C++, hence me being on board with binding rvalues to const ref there.
That must not happen. D const and C++ const don't even mean the same thing, and now suddenly you will see people use extern(C++) just to get the binding of rvalues to ref. Add to this the confusing overload behavior if const signifies anything other than read-only.
Mar 31 2018
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/31/18 9:28 AM, Timon Gehr wrote:
 On 30.03.2018 11:06, Atila Neves wrote:
 Right, and I was assuming (perhaps incorrectly) that this existing 
 code was C++, hence me being on board with binding rvalues to const 
 ref there.
That must not happen. D const and C++ const don't even mean the same thing, and now suddenly you will see people use extern(C++) just to get the binding of rvalues to ref. Add to this the confusing overload behavior if const signifies anything other than read-only.
This is a very important aspect that must not be overlooked. Allow me to reiterate: the one way to get this going is convert the forum dialog into a team and a DIP. Andrei
Mar 31 2018
prev sibling parent Rubn <where is.this> writes:
On Tuesday, 27 March 2018 at 15:50:37 UTC, Atila Neves wrote:
 It's fine for references to
 just be references in D. We're not struggling to make 
 references
 move-able in D, that's not a thing, we already have move 
 semantics.
 Any extension of this conversation about references into C++
 rvalue-references (T&&) and or move-semantics are red-herrings.
 There's no such problem in D that needs to be resolved, and the
 existing solution is excellent.
If I'm reading you correctly (which I might not), you seem to be saying that there's a way forward in which: 1) D's move semantics aren't affected 2) No rvalue references are introduced 3) Temporaries can bind to ref const(T) I'd love to know what that would look like. Atila
Well currently if you only have this implemented: void foo(const ref Type); Type temp = Type(10); foo(temp); Where the hell are you going to do your move semantics? You can't do it anyways currently, it's completely meaningless cause you can't. void foo(Type); void foo(const ref Type); foo(Type(10)); Now we have move semantics with an additional definition. With the proposed change, nothing about that would change. A temporary is only passed to a const ref as a "last resort". If it can do a move instead, it will do the move. The only change that is desired is to make the first sample code above have nicer syntax. That's it, like in the first example you don't care about it being a temporary or not.
Mar 27 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote:
 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
 C++ T&& (Rvalue reference) -> D T
Not really, in C++ it is an actual reference and you get to choose which function actually does the move. In D it just does the copy when passed to the function.
It doesn't copy.
 So you can't do this in D.

 void bar(T&& t)
 {
     // actually move contents of T
 }

 void foo(T&& t)
 {
     bar(std::forward<T>(t)); // Can't do this in D without 
 making another actual copy cause it isn't a reference
 }
You can most definitely do this in D: void bar(T)(auto ref T t) { // T is a ref for lvalues, by value for rvalues } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } More to the point: import std.stdio; struct Foo { ubyte[] data; this(int n) { writeln("ctor n = ", n); data.length = n; } this(this) { writeln("postBlit n = ", data.length); data = data.dup; } } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } void bar(T)(auto ref T t) { writeln("bar: ", t.data[0]); } void main() { bar(Foo(10)); auto f = Foo(5); f.data[0] = 42; bar(f); } The output is: ctor n = 10 bar: 0 ctor n = 5 bar: 42 Notice the lack of "postBlit" in the output. No copies were made. In D, by value *does not* mean copy. And given that, contrary to C++, the compiler doesn't write the postBlit constructor for you, you'd only ever get copies if you implemented it yourself!
 What's a concrete example that you would be required to know 
 whether a const& is a temporary or not.
To know whether or not you can move instead of copy. If it's a temporary, you can move. If it's not, you have to copy. Since temporaries bind to const T& in C++, you might have a temporary, or you might have an lvalue. Since you don't know, you have to copy. To support move semantics, C++ got T&&, which lvalues can't bind to. So if you have a T&&, you know it's about to go away and a move is possible. In D, if it's ref then it can't be a temporary. If it's a value then it can, and it gets moved.
 I've come across a few pains of such. It make be easier to use 
 but it comes at a performance hit. In part binaries become huge 
 because of how "init" is implemented.

 struct StaticArray(T, size_t capacity)
 {
     size_t length;
     T[capacity] values;
 }

 Copying the above structure copies unnecessary data for any 
 move/copy operation. Eg when length = 0, it'll still copy 
 everything. This includes initialization.
This is that rare type for which moving is the same as copying. In that case (assuming it gets copied, see my reply to Manu), pass by ref. You won't be able to pass in temporaries, but I think that's a small price to pay for not having rvalue references. In this case specifically, I don't know why you wouldn't just slice it when passing to functions. Atila
Mar 27 2018
parent reply Rubn <where is.this> writes:
On Tuesday, 27 March 2018 at 07:33:12 UTC, Atila Neves wrote:
 On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote:
 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
 C++ T&& (Rvalue reference) -> D T
Not really, in C++ it is an actual reference and you get to choose which function actually does the move. In D it just does the copy when passed to the function.
It doesn't copy.
It copies the memory, so it does 2 memcpy's in the sense where as C++ only calls its move constructor once.
 So you can't do this in D.

 void bar(T&& t)
 {
     // actually move contents of T
 }

 void foo(T&& t)
 {
     bar(std::forward<T>(t)); // Can't do this in D without 
 making another actual copy cause it isn't a reference
 }
You can most definitely do this in D: void bar(T)(auto ref T t) { // T is a ref for lvalues, by value for rvalues } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } More to the point: import std.stdio; struct Foo { ubyte[] data; this(int n) { writeln("ctor n = ", n); data.length = n; } this(this) { writeln("postBlit n = ", data.length); data = data.dup; } } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } void bar(T)(auto ref T t) { writeln("bar: ", t.data[0]); } void main() { bar(Foo(10)); auto f = Foo(5); f.data[0] = 42; bar(f); } The output is: ctor n = 10 bar: 0 ctor n = 5 bar: 42 Notice the lack of "postBlit" in the output. No copies were made. In D, by value *does not* mean copy. And given that, contrary to C++, the compiler doesn't write the postBlit constructor for you, you'd only ever get copies if you implemented it yourself!
Well for starters your code is wrong. You are calling bar() instead of foo(). D has hidden implementation details, from your perspective it looks like it isn't doing any copying from the high-level viewpoint, but from the low-level viewpoint your object has been copied (moved memory) multiple times. struct Foo { ubyte[1024] data; } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } Foo gfoo; void bar(T)(auto ref T t) { import std.algorithm.mutation : move; move(t, gfoo); } void main() { foo(Foo(10)); } _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv: push rbp mov rbp, rsp sub rsp, 3104 lea rax, [rbp + 16] lea rdi, [rbp - 2048] lea rcx, [rbp - 1024] mov edx, 1024 mov rsi, rcx mov qword ptr [rbp - 2056], rdi mov rdi, rsi mov rsi, rax mov qword ptr [rbp - 2064], rcx call memcpy PLT <--------------------- hidden copy Then the other copy is in move() to gfoo. That hidden copy will happen for every additional function call you try to pass Foo through.
 What's a concrete example that you would be required to know 
 whether a const& is a temporary or not.
To know whether or not you can move instead of copy. If it's a temporary, you can move. If it's not, you have to copy. Since temporaries bind to const T& in C++, you might have a temporary, or you might have an lvalue. Since you don't know, you have to copy. To support move semantics, C++ got T&&, which lvalues can't bind to. So if you have a T&&, you know it's about to go away and a move is possible. In D, if it's ref then it can't be a temporary. If it's a value then it can, and it gets moved.
D already has move semantics, an easy solution to this is to just use another keyword. It doesn't have to bind to const ref to get what is desired: // what was suggested in the original DIP, since scope is being used for something else now void foo( temp ref value) { } Now you don't have this problem. You only get this behavior when you basically say you don't care whether it is a temporary or not. So what's your problem with it now ?
 I've come across a few pains of such. It make be easier to use 
 but it comes at a performance hit. In part binaries become 
 huge because of how "init" is implemented.

 struct StaticArray(T, size_t capacity)
 {
     size_t length;
     T[capacity] values;
 }

 Copying the above structure copies unnecessary data for any 
 move/copy operation. Eg when length = 0, it'll still copy 
 everything. This includes initialization.
This is that rare type for which moving is the same as copying. In that case (assuming it gets copied, see my reply to Manu), pass by ref. You won't be able to pass in temporaries, but I think that's a small price to pay for not having rvalue references. In this case specifically, I don't know why you wouldn't just slice it when passing to functions. Atila
This wasn't an example for rvalue references. This was an example illustrating the negative results of the current "pain-free" system. I have a 100 mb binary, 90 mb of that come from a single structure. I mean sure you don't really have to worry about implementing move constructors and such but it is far from being pain free, that's what that was suppose to illustrate.
Mar 27 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Mar 27, 2018 at 08:25:36PM +0000, Rubn via Digitalmars-d wrote:
[...]
 _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
   push rbp
   mov rbp, rsp
   sub rsp, 3104
   lea rax, [rbp + 16]
   lea rdi, [rbp - 2048]
   lea rcx, [rbp - 1024]
   mov edx, 1024
   mov rsi, rcx
   mov qword ptr [rbp - 2056], rdi
   mov rdi, rsi
   mov rsi, rax
   mov qword ptr [rbp - 2064], rcx
   call memcpy PLT    <--------------------- hidden copy
[...] Is this generated by dmd, or gdc/ldc? Generally, when it comes to performance issues, I don't even bother looking at dmd-generated code anymore. If the extra copying is still happening with gdc -O2 / ldc -O, then you have a point. Otherwise, it doesn't really say very much. T -- People tell me that I'm skeptical, but I don't believe them.
Mar 27 2018
parent reply Rubn <where is.this> writes:
On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote:
 On Tue, Mar 27, 2018 at 08:25:36PM +0000, Rubn via 
 Digitalmars-d wrote: [...]
 _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
   push rbp
   mov rbp, rsp
   sub rsp, 3104
   lea rax, [rbp + 16]
   lea rdi, [rbp - 2048]
   lea rcx, [rbp - 1024]
   mov edx, 1024
   mov rsi, rcx
   mov qword ptr [rbp - 2056], rdi
   mov rdi, rsi
   mov rsi, rax
   mov qword ptr [rbp - 2064], rcx
   call memcpy PLT    <--------------------- hidden copy
[...] Is this generated by dmd, or gdc/ldc? Generally, when it comes to performance issues, I don't even bother looking at dmd-generated code anymore. If the extra copying is still happening with gdc -O2 / ldc -O, then you have a point. Otherwise, it doesn't really say very much. T
It happens with LDC too, not sure how it would be able to know to do any kind of optimization like that unless it was able to inline every single function called into one function and be able to do optimize it from there. I don't imagine that'll be likely though.
Mar 27 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Mar 27, 2018 at 09:52:25PM +0000, Rubn via Digitalmars-d wrote:
 On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote:
 On Tue, Mar 27, 2018 at 08:25:36PM +0000, Rubn via Digitalmars-d wrote:
 [...]
 _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
   push rbp
   mov rbp, rsp
   sub rsp, 3104
   lea rax, [rbp + 16]
   lea rdi, [rbp - 2048]
   lea rcx, [rbp - 1024]
   mov edx, 1024
   mov rsi, rcx
   mov qword ptr [rbp - 2056], rdi
   mov rdi, rsi
   mov rsi, rax
   mov qword ptr [rbp - 2064], rcx
   call memcpy PLT    <--------------------- hidden copy
[...] Is this generated by dmd, or gdc/ldc? Generally, when it comes to performance issues, I don't even bother looking at dmd-generated code anymore. If the extra copying is still happening with gdc -O2 / ldc -O, then you have a point. Otherwise, it doesn't really say very much. T
It happens with LDC too, not sure how it would be able to know to do any kind of optimization like that unless it was able to inline every single function called into one function and be able to do optimize it from there. I don't imagine that'll be likely though.
You'll be surprised. Don't underestimate the power of modern optimizers. I've seen LDC do inlining that's so aggressive, that it essentially evaluated an entire series of function calls at compile-time (likely on the IR) and generated a single instruction to load the answer into the return register at runtime. :-D Of course, it still generated the individual functions, but those are never actually called at runtime. (On one occasion, this produced odd-looking "benchmark" results where the ldc executable computed the answer in exactly 0ms, whereas everyone else took a lot longer than that. :-D (Well, it was probably a few nanosecs while the CPU decoded and ran the instruction, but I don't think any benchmark could measure that!)) For your code example, you might want to look at the code generated for callers of the function, since when compiling individual functions in isolation, LDC is obligated to follow the ABI, which could include redundant copying. But if inlining was possible, it could generate very different code. T -- Dogs have owners ... cats have staff. -- Krista Casada
Mar 27 2018
prev sibling parent reply kinke <noone nowhere.com> writes:
On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:
 It happens with LDC too, not sure how it would be able to know 
 to do any kind of optimization like that unless it was able to 
 inline every single function called into one function and be 
 able to do optimize it from there. I don't imagine that'll be 
 likely though.
It does it in your code sample with `-O`, there's no call to bar and the foo() by-value arg is memcpy'd to the global. If you compile everything with LTO, your code and all 3rd-party libs as well as druntime/Phobos, LLVM is able to optimize the whole program as if it were inside a single gigantic 'object' file in LLVM bitcode IR, and is thus indeed theoretically able to inline *all* functions.
Mar 27 2018
next sibling parent reply Rubn <where is.this> writes:
On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote:
 On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:
 It happens with LDC too, not sure how it would be able to know 
 to do any kind of optimization like that unless it was able to 
 inline every single function called into one function and be 
 able to do optimize it from there. I don't imagine that'll be 
 likely though.
It does it in your code sample with `-O`, there's no call to bar and the foo() by-value arg is memcpy'd to the global. If you compile everything with LTO, your code and all 3rd-party libs as well as druntime/Phobos, LLVM is able to optimize the whole program as if it were inside a single gigantic 'object' file in LLVM bitcode IR, and is thus indeed theoretically able to inline *all* functions.
A bit off topic now but anyways: Well that example I posted didn't do anything, so it would optimize it out quite easily. The entire function was excluded essentially. Just adding a few writeln it isn't able to remove the function entirely anymore and can't optimize it out. Idk if you want to try some different options but flto didn't do anything for it. https://godbolt.org/g/bLdpnm import std.stdio : writeln; struct Foo { ubyte[1024] data; this(int a) { data[0] = cast(ubyte)a; } } void foo(T)(auto ref T t) { import std.functional: forward; writeln(gfoo.data[0]); bar(forward!t); writeln(gfoo.data[0]); } __gshared Foo gfoo; void bar(T)(auto ref T t) { import std.algorithm.mutation : move; writeln(gfoo.data[0]); move(t, gfoo); } void main() { foo(Foo(10)); }
Mar 27 2018
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote:
 Just adding a few writeln it isn't able to remove the function 
 entirely anymore and can't optimize it out.
Well writeln() here involves number -> string formatting, GC, I/O, template bloat... There are indeed superfluous memcpy's in your foo() there (although the forward and bar calls are still inlined), which after a quick glance seem to be LLVM optimizer shortcomings, the IR emitted by LDC looks fine. For an abitrary external function, it's all fine as it should be, boiling down to a single memcpy in foo() and a direct memset in main(): https://run.dlang.io/is/O1aeLK
Mar 27 2018
parent Rubn <where is.this> writes:
On Wednesday, 28 March 2018 at 00:56:29 UTC, kinke wrote:
 On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote:
 Just adding a few writeln it isn't able to remove the function 
 entirely anymore and can't optimize it out.
Well writeln() here involves number -> string formatting, GC, I/O, template bloat... There are indeed superfluous memcpy's in your foo() there (although the forward and bar calls are still inlined), which after a quick glance seem to be LLVM optimizer shortcomings, the IR emitted by LDC looks fine. For an abitrary external function, it's all fine as it should be, boiling down to a single memcpy in foo() and a direct memset in main(): https://run.dlang.io/is/O1aeLK
Well somethings wrong if writeln causes optimization to not occur, if that is the case then it'd be best to just use printf() instead. Anyways using small examples to show optimization is usually not what's going to happen in actual code. Functions are rarely that simple, and if adding a single writeln() to a call is enough to eliminate that optimization, I can only imagine what other little things do as well.
Mar 27 2018
prev sibling parent kinke <noone nowhere.com> writes:
On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote:
 On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:
 It happens with LDC too, not sure how it would be able to know 
 to do any kind of optimization like that unless it was able to 
 inline every single function called into one function and be 
 able to do optimize it from there. I don't imagine that'll be 
 likely though.
It does it in your code sample with `-O`, there's no call to bar and the foo() by-value arg is memcpy'd to the global.
For reference: https://run.dlang.io/is/2vDEXP Note that main() boils down to a `memset(&gfoo, 10, 1024); return 0;`: _Dmain: .cfi_startproc pushq %rax .Lcfi0: .cfi_def_cfa_offset 16 data16 leaq onlineapp.Foo onlineapp.gfoo TLSGD(%rip), %rdi data16 data16 rex64 callq __tls_get_addr PLT movl $10, %esi movl $1024, %edx movq %rax, %rdi callq memset PLT xorl %eax, %eax popq %rcx retq
Mar 27 2018
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 27.03.2018 22:25, Rubn wrote:
 
 D already has move semantics, an easy solution to this is to just use 
 another keyword. It doesn't have to bind to const ref to get what is 
 desired:
 
 // what was suggested in the original DIP, since scope is being used for 
 something else now
 void foo( temp ref value)
 {
 }
 
 Now you don't have this problem. You only get this behavior when you 
 basically say you don't care whether it is a temporary or not.
Another benefit of this solution is that the overload resolution rules are obvious. foo( temp ref T value) is less specialized than both foo(T value) and foo(ref T value). Manu: Consider this.
Mar 28 2018
parent Manu <turkeyman gmail.com> writes:
On 28 March 2018 at 05:38, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 27.03.2018 22:25, Rubn wrote:
 D already has move semantics, an easy solution to this is to just use
 another keyword. It doesn't have to bind to const ref to get what is
 desired:

 // what was suggested in the original DIP, since scope is being used for
 something else now
 void foo( temp ref value)
 {
 }

 Now you don't have this problem. You only get this behavior when you
 basically say you don't care whether it is a temporary or not.
Another benefit of this solution is that the overload resolution rules are obvious. foo( temp ref T value) is less specialized than both foo(T value) and foo(ref T value). Manu: Consider this.
This defeats the entire point to me. I want symmetrical calling code in all cases... the current edges are a massive pain in the arse. In the event of yet-another-attribute, then we just shift the set of edge cases onto that attribute instead, and it makes no practical difference in the end.
Mar 28 2018
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
 Explain how the situation if implemented would be any different 
 than
 the workaround?

 This seems even simpler than the pow thing to me.
 Rewrite:
     func(f());
 as:
     { auto __t0 = f(); func(__t0); }


 How is that worse than the code you have to write:
     T temp = f();
     T zero = 0;
     func(temp, zero);
I feel like this example wasn't really concrete enough for me. I wrote a version below that I think made it a little clearer for myself. ----- import std.stdio : writeln; struct Foo { int data; } int foo(Foo x) { writeln("here"); return x.data; } int foo(ref Foo x) { writeln("there"); return x.data; } void main() { auto x = Foo(5); auto y = foo(x); writeln(y); auto z = foo(Foo(5)); writeln(z); }
Mar 26 2018
prev sibling next sibling parent Kagamin <spam here.lot> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
 By contrast, people will NOT forgive the fact that they have to 
 change:

     func(f(x), f(y), f(z));

 to:

     T temp = f(x);
     T temp2 = f(y);
     T temp3 = f(z);
     func(temp, temp2, temp3);

 That's just hideous and in-defensible.

 A better story would be:

     func(f(x), f(y), f(z));
 =>
     func(x.f, y.f, z.f);
Another workaround: auto r(T)(T a) { struct R { T val; } return R(a); } void f(in ref int p); int main() { f(1.r.val); return 0; }
Mar 28 2018
prev sibling next sibling parent reply Dgame <r.schuett.1987 gmail.com> writes:
Just to be sure it does not got los: You know that you can avoid 
the temp/copy if you add one method to your struct, yes?

----
import std.stdio;

struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
}

void foo(ref const Big b) {
     writeln(b.name);
}

void main() {

     foo(b);

}
----

That works like a charm and avoids any need for rvalue 
references. Just add it as mixin template in Phobos or something 
like that.
Mar 29 2018
parent reply Rubn <where is.this> writes:
On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:
 Just to be sure it does not got los: You know that you can 
 avoid the temp/copy if you add one method to your struct, yes?

 ----
 import std.stdio;

 struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
 }

 void foo(ref const Big b) {
     writeln(b.name);
 }

 void main() {

     foo(b);

 }
 ----

 That works like a charm and avoids any need for rvalue 
 references. Just add it as mixin template in Phobos or 
 something like that.
Doesn't work with built-in types like float. Just adds bloat for operators like opBinary if you want that to be ref. foo((a.byRef + b.byRef * c.byRef).byRef) // vs foo(a + b * c); It's kind of funny all this talk about allowing temporaries to bind to refs being messy, yet you can already bind a temporary to a ref in a messy way using that.
Mar 29 2018
parent reply Dgame <r.schuett.1987 gmail.com> writes:
On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote:
 On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:
 Just to be sure it does not got los: You know that you can 
 avoid the temp/copy if you add one method to your struct, yes?

 ----
 import std.stdio;

 struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
 }

 void foo(ref const Big b) {
     writeln(b.name);
 }

 void main() {

     foo(b);

 }
 ----

 That works like a charm and avoids any need for rvalue 
 references. Just add it as mixin template in Phobos or 
 something like that.
Doesn't work with built-in types like float.
Why would you want to use a float as a rvalue reference?
 Just adds bloat for operators like opBinary if you want that to 
 be ref.

 foo((a.byRef + b.byRef * c.byRef).byRef)

 // vs

 foo(a + b * c);

 It's kind of funny all this talk about allowing temporaries to 
 bind to refs being messy, yet you can already bind a temporary 
 to a ref in a messy way using that.
Yeah, it is a bit messy. It is not perfect, but is does avoid any temp var! Let's look at a Vector2f example with the following opBinary: ---- auto opBinary(string op)(ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef); ---- Since opBinary needs to be a template, you can combine my solution with auto ref: ---- auto opBinary(string op)(auto ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef); ---- That is cleaner. :)
Mar 29 2018
parent Rubn <where is.this> writes:
On Thursday, 29 March 2018 at 20:22:47 UTC, Dgame wrote:
 On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote:
 On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:
 Just to be sure it does not got los: You know that you can 
 avoid the temp/copy if you add one method to your struct, yes?

 ----
 import std.stdio;

 struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
 }

 void foo(ref const Big b) {
     writeln(b.name);
 }

 void main() {

     foo(b);

 }
 ----

 That works like a charm and avoids any need for rvalue 
 references. Just add it as mixin template in Phobos or 
 something like that.
Doesn't work with built-in types like float.
Why would you want to use a float as a rvalue reference?
In templates to avoid template bloat with auto ref.
 Just adds bloat for operators like opBinary if you want that 
 to be ref.

 foo((a.byRef + b.byRef * c.byRef).byRef)

 // vs

 foo(a + b * c);

 It's kind of funny all this talk about allowing temporaries to 
 bind to refs being messy, yet you can already bind a temporary 
 to a ref in a messy way using that.
Yeah, it is a bit messy. It is not perfect, but is does avoid any temp var! Let's look at a Vector2f example with the following opBinary: ---- auto opBinary(string op)(ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef); ---- Since opBinary needs to be a template, you can combine my solution with auto ref: ---- auto opBinary(string op)(auto ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef); ---- That is cleaner. :)
Just adding more template bloat and it is still messy, it's not like we are trying to find a work around to a problem. A solution already exists, the cleaner syntax you'd get with rvalue references (along with better compatibility with C++ and other benefits) can't be beat.
Mar 29 2018
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:
 On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:
 On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d

 wrote:
 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
 My biggest concern in all of this is that I don't want to see ref
 start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants
 to
 avoid a copy.
That ship sailed ages ago: It's already unclear. If we want to fix that, we can fix it, but blocking rvalue ref does nothing for that cause.
Really? And how often does ref get used just to avoid copying? You can almost always look at a function, see that it accepts ref, and know that it's supposed to mutate the argument. Functions that want to avoid copying lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a number of folks would start using it all over the place to avoid copying. Right now, folks rarely use ref that way, because it becomes too annoying to call the function with an rvalue. So, while it might not be the case 100% of the time right now that ref is used with the purpose of mutating the argument, it almost always is. As such, you can pretty reliably look at a function signature and expect that if one of its parameters is ref, it's going to be mutating that argument. The function that accepts an argument by ref with no intention of mutating it is very much the exception, and I really don't want to see that change.
Interesting. Just understand that you're trading that feeling for a suite of edge cases though. Accepting asymmetric calling rules is a pretty big cost to pay for that 'nice thought'. That idea also dismisses the existence of the set of cases where ref is genuinely useful/correct. Your sentiment effectively puts those use cases into the position of 2nd-class citizens. I understand your sentiment, but I think the cost is not balanced, or even particularly fair with respect to users in those niche groups :/
I'm not arguing against having a way to indicate that a parameter accepts both rvalues and lvalues and that the rvalues get copied to an invisible variable so that they can be passed as an lvalue. I'm arguing against simply making ref have that behavior. Assuming that the details of how that worked internally didn't involve problematic stuff like the rvalue reference stuff that Andrei and Walter are so against, having an attribute such as rvalue to attach to a ref parameter to allow it to accept rvalues would be fine with me. I just don't want ref by itself to lose its current semantics, because that would dilute its meaning and increase its amiguity. I want to be able to look at a function signature, see ref without other qualifiers, and be reasonably certain that the function is supposed to be mutating that argument, whereas if ref by itself accepted rvalues, then we lose that. If an attribute were used to make it allow rvalues, then we wouldn't. - Jonathan M Davis
Mar 30 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:
 On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:
 On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d

 wrote:
 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
 My biggest concern in all of this is that I don't want to see ref
 start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants
 to
 avoid a copy.
That ship sailed ages ago: It's already unclear. If we want to fix that, we can fix it, but blocking rvalue ref does nothing for that cause.
Really? And how often does ref get used just to avoid copying? You can almost always look at a function, see that it accepts ref, and know that it's supposed to mutate the argument. Functions that want to avoid copying lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a number of folks would start using it all over the place to avoid copying. Right now, folks rarely use ref that way, because it becomes too annoying to call the function with an rvalue. So, while it might not be the case 100% of the time right now that ref is used with the purpose of mutating the argument, it almost always is. As such, you can pretty reliably look at a function signature and expect that if one of its parameters is ref, it's going to be mutating that argument. The function that accepts an argument by ref with no intention of mutating it is very much the exception, and I really don't want to see that change.
Interesting. Just understand that you're trading that feeling for a suite of edge cases though. Accepting asymmetric calling rules is a pretty big cost to pay for that 'nice thought'. That idea also dismisses the existence of the set of cases where ref is genuinely useful/correct. Your sentiment effectively puts those use cases into the position of 2nd-class citizens. I understand your sentiment, but I think the cost is not balanced, or even particularly fair with respect to users in those niche groups :/
I'm not arguing against having a way to indicate that a parameter accepts both rvalues and lvalues and that the rvalues get copied to an invisible variable so that they can be passed as an lvalue. I'm arguing against simply making ref have that behavior. Assuming that the details of how that worked internally didn't involve problematic stuff like the rvalue reference stuff that Andrei and Walter are so against, having an attribute such as rvalue to attach to a ref parameter to allow it to accept rvalues would be fine with me. I just don't want ref by itself to lose its current semantics, because that would dilute its meaning and increase its amiguity.
https://imgflip.com/i/27gjv9
 I want to be able to look at a function signature, see ref without other
 qualifiers, and be reasonably certain that the function is supposed to be
 mutating that argument, whereas if ref by itself accepted rvalues, then we
 lose that. If an attribute were used to make it allow rvalues, then we
 wouldn't.
I don't know any reason why someone wouldn't attribute the argument 'const' if it doesn't intend to write to it. Likewise 'return ref' if it's going to be modified and returned. I'm not sure your concern is actually a thing...?
Mar 30 2018
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:
 On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
 I want to be able to look at a function signature, see ref without other
 qualifiers, and be reasonably certain that the function is supposed to
 be
 mutating that argument, whereas if ref by itself accepted rvalues, then
 we lose that. If an attribute were used to make it allow rvalues, then
 we wouldn't.
I don't know any reason why someone wouldn't attribute the argument 'const' if it doesn't intend to write to it. Likewise 'return ref' if it's going to be modified and returned. I'm not sure your concern is actually a thing...?
const would do it, but given how restrictive const is in D, I don't see how it would be very reasonable to restrict passing rvalues to const. And honestly, I don't really want to encourage the use of const. It's fine if folks use it where it really works, and it's potentially even quite valuable there, but it seems like too often folks try to add const in places where it's just going to cause problems. It's particularly bad when folks try to add const to generic code - e.g. we recently had to revert a commit to std.random which worked with dynamic arrays but not other ranges because of const. And there have been other cases where folks have wanted to try to make stuff in Phobos "const-correct", which would cause problems. So, I'm not a big fan of the idea of doing anything that would make folks want to use const more. Now, if you can convince Walter and Andrei to allow const ref to accept rvalues, then fine. I think that that's definitely worse than an attribute specifically for that, given how limiting const is, but it wouldn't screw up normal ref in the process, which is what I'm most worried about here. So, I don't think that going with const would be the best solution to the problem, but it's far better than making ref in general accept rvalues. - Jonathan M Davis
Mar 30 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:
 On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
 I want to be able to look at a function signature, see ref without other
 qualifiers, and be reasonably certain that the function is supposed to
 be
 mutating that argument, whereas if ref by itself accepted rvalues, then
 we lose that. If an attribute were used to make it allow rvalues, then
 we wouldn't.
I don't know any reason why someone wouldn't attribute the argument 'const' if it doesn't intend to write to it. Likewise 'return ref' if it's going to be modified and returned. I'm not sure your concern is actually a thing...?
const would do it, but given how restrictive const is in D, I don't see how it would be very reasonable to restrict passing rvalues to const.
I'm actually coming around to the idea of not restricting it to const... But what I'm trying to say is, the number of cases where your principle will feel violated should be as infrequent as the cases where const is insufficient for 'reasons' ;) As far as I know, in usages that I've ever encountered, I can't imagine a case where you would feel violated :)
 And honestly, I don't really want to encourage the use of const. It's fine if
 folks use it where it really works, and it's potentially even quite valuable
 there, but it seems like too often folks try to add const in places where
 it's just going to cause problems. It's particularly bad when folks try to
 add const to generic code - e.g. we recently had to revert a commit to
 std.random which worked with dynamic arrays but not other ranges because of
 const. And there have been other cases where folks have wanted to try to
 make stuff in Phobos "const-correct", which would cause problems. So, I'm
 not a big fan of the idea of doing anything that would make folks want to
 use const more.
It's interesting... I recognise the general backlash against const. I personally just make myself a `struct Mutable(T) { ... }` which I use when I want to violate const ;) But if it turns out that const is useless, then we really need to reconsider the basic design of const >_< Like, what's the point? You're advocating active discouragement of const... why is there a feature which it's accepted is a loaded-gun? Handing it to people creates a high probability they'll shoot themselves in the feet.
 Now, if you can convince Walter and Andrei to allow const ref to accept
 rvalues, then fine. I think that that's definitely worse than an attribute
 specifically for that, given how limiting const is, but it wouldn't screw up
 normal ref in the process, which is what I'm most worried about here. So, I
 don't think that going with const would be the best solution to the problem,
 but it's far better than making ref in general accept rvalues.
Useful examples using 'return ref' have been presented, which I find quite compelling too. That's an interesting case of non-const ref. Safe to say I'm not convincing anyone of anything in any way other than DIP form. But I'd like to understand your concern better. You say it's about scanning an API and understanding some details from it based on seeing 'ref' written there... how does a function accepting an rvalue interact with your visibility of the API? Like, your criticism is with respect to understanding the API at a glance... I don't understand how this proposal interferes with that in any way? Sadly, I don't think I'll be able to make it to DConf this year... which is probably a reason for rejoice of literally everybody attending! :P It would be nice to workshop it in person though.
Mar 30 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/30/2018 6:03 PM, Manu wrote:
 Sadly, I don't think I'll be able to make it to DConf this year...
:-(
 which is probably a reason for rejoice of literally everybody
 attending! :P
You'll be missed.
 It would be nice to workshop it in person though.
Mar 30 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 30, 2018 18:52:38 Walter Bright via Digitalmars-d wrote:
 On 3/30/2018 6:03 PM, Manu wrote:
 Sadly, I don't think I'll be able to make it to DConf this year...
: :-( :
 which is probably a reason for rejoice of literally everybody
 attending! :P
You'll be missed.
My usual response to that is that it's better to be missed than to be hit. ;) But yes, it will be a shame if it's unable to come. - Jonathan M Davis
Mar 30 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 30 March 2018 at 18:52, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 3/30/2018 6:03 PM, Manu wrote:
 Sadly, I don't think I'll be able to make it to DConf this year...
:-(
 which is probably a reason for rejoice of literally everybody
 attending! :P
You'll be missed.
Thank your stars... I wouldn't be able to resist heckling everyone on this the whole time! :P
Mar 30 2018
prev sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 30, 2018 18:03:49 Manu via Digitalmars-d wrote:
 On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d
 And honestly, I don't really want to encourage the use of const. It's
 fine if folks use it where it really works, and it's potentially even
 quite valuable there, but it seems like too often folks try to add
 const in places where it's just going to cause problems. It's
 particularly bad when folks try to add const to generic code - e.g. we
 recently had to revert a commit to std.random which worked with dynamic
 arrays but not other ranges because of const. And there have been other
 cases where folks have wanted to try to make stuff in Phobos
 "const-correct", which would cause problems. So, I'm not a big fan of
 the idea of doing anything that would make folks want to use const
 more.
It's interesting... I recognise the general backlash against const. I personally just make myself a `struct Mutable(T) { ... }` which I use when I want to violate const ;)
Just don't do anything where you cast away const and then mutate. That violates the type system and results in undefined behavior. And depending on what the compiler chooses to do based on const, you could get some rather subtle and nasty bugs.
 But if it turns out that const is useless, then we really need to
 reconsider the basic design of const >_<
 Like, what's the point? You're advocating active discouragement of
 const... why is there a feature which it's accepted is a loaded-gun?
 Handing it to people creates a high probability they'll shoot
 themselves in the feet.
const makes some sense when you want to write code that accepts both mutable and immutable arguments, and there are times where it works perfectly fine to use const. I see no problem with using const in such situations. The problem is that there are a _lot_ cases where const simply can't be used in D, because you need to mutate _something_ involved, even if it's not something that's part of the "logical" state of the object - e.g. in D, you can't put a mutex in a type and then have it protect anything in a const member function, because locking the mutex would require mutating it, and D provides no backdoors like C++'s mutable. What's const is actually const. Also, const tends to interact very badly with generic code. Whatever API you're duck typing has to include const as part of it, or you can't use const - e.g. ranges can't be const, because they have to be mutated to be iterated, and because the range API does not require that properties like front or empty be const, no generic code can assume that even those work with const (and for many ranges, they _can_'t be const - especially when one range wraps another). In general, it can be quite difficult to even do something like have a tail-const range over a container. Sometimes, const can be made to work in more complex situations, but often it can't, and when it can't, it's often a royal pain. For non-generic code that can clearly treat an object as fully const without needing any backdoors, const is just fine, and it may even help prevent bugs. But for a _lot_ of code - especially idiomatic D code that does a lot with templates and ranges - const simply doesn't work. So, const can be used on some level, but anyone who tries to be "const-correct" in D like many of us usually try to do in C++ is in for a world of frustration. If you haven't, I'd suggest that you read this article I recently wrote on the topic: http://jmdavisprog.com/articles/why-const-sucks.html And here's the newsgroup thread discussing it: https://forum.dlang.org/thread/mailman.581.1520247475.3374.digitalmars-d-announce puremagic.com
 Now, if you can convince Walter and Andrei to allow const ref to accept
 rvalues, then fine. I think that that's definitely worse than an
 attribute specifically for that, given how limiting const is, but it
 wouldn't screw up normal ref in the process, which is what I'm most
 worried about here. So, I don't think that going with const would be
 the best solution to the problem, but it's far better than making ref
 in general accept rvalues.
Useful examples using 'return ref' have been presented, which I find quite compelling too. That's an interesting case of non-const ref. Safe to say I'm not convincing anyone of anything in any way other than DIP form.
Yes, at this point, convincing Walter or Andrei will require writing a DIP. A well-written proposal that clearly doesn't have the downsides that they're so against may have a chance with them, but without something that formally and clearly provides the arguments, I don't think that they're even going to pay much attention at this point. The topic has been debated to death previously, and without a DIP, it really doesn't matter what Walter and Andrei think, since at this point, they wouldn't introduce a language change like that without a DIP. So, they'll just do like Andrei did in response to this thread and tell you to write a DIP rather than be willing to discuss it further on its own. But no matter the topic, if it involves a major change to the language, since that now requires a DIP, there's not much point in arguing the matter with Walter or Andrei beforehand except to help iron out the DIP. Increasingly, they don't get involved in such discussions outside of the DIP process, because it eats up too much of their time.
 But I'd like to understand your concern better. You say it's about
 scanning an API and understanding some details from it based on seeing
 'ref' written there... how does a function accepting an rvalue
 interact with your visibility of the API?
 Like, your criticism is with respect to understanding the API at a
 glance... I don't understand how this proposal interferes with that in
 any way?
The only thing that interfers with that is making ref with no other qualifiers accept rvalues. If it's const or inout or immutable or something like rvalue, then you can look at the API and know that ref is on the parameter so that lvalues will not be copied, and if it accepts rvalues too (be by it copying them to invisible variables to pass as lvalues or whatever), then that may mean a performance hit due to the lack of a move, but you know that the function is not mutating the argument with the intention of giving the result to the caller. If it's const, then mutation wouldn't be possible, and if it's rvalue, then it simply wouldn't make sense for the function to be providing the result back to the caller, because that would only work with lvalues. rvalue would indicate that the function _could_ mutate the argument (so the restrictions imposed by const would not apply), and relying on the argument not being mutated would be asking for bugs, but it would clearly be ref for the purpose of avoiding copying lvalues and not because the purpose is to mutate the argument and provide the result. As long as naked ref does not accept rvalues, then you can look it at and reasonably assume that the function is intended to mutate the argument and provide the result to the caller. It also prevents accidentally passing an rvalue and thus avoids the bugs that would come from passing an rvalue to a function that's supposed to be mutating its arguments. As it stands, folks do still sometimes write functions which accept arguments by ref to avoid copying lvalues, but because of the annoyances with passing rvalues, most folks don't do that. And if the solution were to add rvalue to make ref accept rvalues, then anyone who wanted to use ref for efficiency reasons would use rvalue on the ref, and it would make it that much more reliable that a naked ref indicated that the function was supposed to mutate the argument. It also wouldn't introduce the problem of accidentally passing rvalues to ref parameters like making ref in general accept rvalues would. So, unless you're proposing that ref in general accept rvalues, I'm not necessarily criticizing what you're proposing. I do have reservations about using const instead of something like rvalue to indicate that rvalues should be accepted due to the restrictions on const, but I also don't care enough about being able to pass rvalues by ref to make a big stink about const ref and auto ref being the only way to accept rvalues without copying lvalues. I think that it's worth pointing out that making const part of the solution is problematic, but if you write a DIP that Walter and Andrei accept that uses const to indicate that ref will accept rvalues, then it's generally not going to cause problems for me, since I won't be using it enough for the restrictions on const to matter to me. So, I think that it's a worse solution, but a subpar solution to this problem only affects me with regards to the fact that I want D to be a great language (and thus would prefer not to have subpar solutions) and insofar as I run into other people's code that uses the technique. I might end up using it occasonally, but for the most part, the few cases where I'd care, I've been able to use auto ref to solve the problem. Most of that I write works just fine passing stuff by value (and if it doesn't, I tend to make the type a reference type to solve the problem), and I'm not likely to do much with calling extern(C++) functions outside of auto-generated code. So, while I acknowledge the problem and have no problem with it being solved, it simply isn't an issue that affects me much. As such, it will be that much more annoying if the solution makes existing parts of the language worse.
 Sadly, I don't think I'll be able to make it to DConf this year...
 which is probably a reason for rejoice of literally everybody
 attending! :P
 It would be nice to workshop it in person though.
LOL. I don't know that anyone would rejoice about it. If they really don't want to talk to you, they probably just won't talk to you. But it's not always easy to get away, and it's not necessarily cheap, so it's understandable if you can't come. Fortunately, I generally haven't had a problem with getting the time off to go. It's the cost that generally makes the trip problematic for me, though I've managed to make it works thus far, and this year, I'm speaking, so it won't be anywhere near as expensive for me as the last couple of years were. If you can't make it this year, then hopefully you'll be able to make it next year, wherever it happens to end up being. - Jonathan M Davis
Mar 30 2018