www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - rvalues -> ref (yup... again!)

reply Manu <turkeyman gmail.com> writes:
Forked from the x^^y thread...

On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 3/23/2018 11:09 AM, Manu wrote:

 [...]


 Rvalue references are not trivial and can have major unintended
 consequences. They're a rather ugly feature in C++, with weirdities. I doubt
 D will ever have them.


Can you please explain these 'weirdities'?
What are said "major unintended consequences"?
Explain how the situation if implemented would be any different than
the workaround?

This seems even simpler than the pow thing to me.
Rewrite:
    func(f());
as:
    { auto __t0 = f(); func(__t0); }


How is that worse than the code you have to write:
    T temp = f();
    T zero = 0;
    func(temp, zero);

This 'workaround' is really upsetting. It's ugly, visually noisy, very
annoying to implement over and over, and litters the local namespace
with rubbish. There's no good name for most of this stuff, they just
get named t0, t1, t2...

Surely you can see how this objectively makes code worse...?
This one more than anything has made my work trying to convince people
that D is cool very hard. Most things I can explain around, or say
it's a known issue and it'll be better soon. This one is kinda harder;
"yeah... that's like, working exactly as intended! I know, isn't D
cool!" ;)



 But at some level, D cannot replace C++ on a line-by-line basis. There's
 always going to be something different. If not in the core language, in the
 way the standard library works. If you're heavily using templates and stuff
 in C++, you're likely going to have to rethink how the code works to get it
 in D (or any other language).


I haven't experienced much friction on that front, or had it expressed
by new users. They're generally willing to humour that D has
differences in it's meta, probably because they understand C++ sucks
and drastic changes must be made. They generally complain about choice
of '!' for a few minutes and then move on.
By contrast, people will NOT forgive the fact that they have to change:

    func(f(x), f(y), f(z));

to:

    T temp = f(x);
    T temp2 = f(y);
    T temp3 = f(z);
    func(temp, temp2, temp3);

That's just hideous and in-defensible.

A better story would be:

    func(f(x), f(y), f(z));
=>
    func(x.f, y.f, z.f);

Instead of being outraged, they'd be further seduced. Sadly, that's
not our reality (yet).
What a missed opportunity! ;)



 For example, in my efforts translating C to D, the clumsy part is the
 metaprogramming in the C preprocessor. There's nothing there D cannot do,
 but it has to be redesigned. The result is much better, but translating by
 rote is simply impossible.


I don't often work with C, and I think it's been considered pretty
unsavoury to lean heavily on the preprocessor for a few decades now.
Even if I did, I probably wouldn't 'port' C, so much as interact with
it, and extern(C) works well.
I'm more focused on C++, and as such, my experiences differ significantly ;)

Reworking a field of preprocessor cruft is in quite a different space
than "reworking all calls to func()".
People are forgiving of a thing if they appreciate and understand it.



 Also, just try translating some of the code in Phobos to C++. It was tried
 to do ranges for C++, and the result was terrifying. (It worked, but that's
 about all that could be said for it.)


I've done comprehensive slices and ranges in C++. It works reasonably
well. Certainly, the worst outcome was that I significantly diminished
any arguments I had to switch to D...

Not sure what your point is though? You'll find no argument from me
that slice, ranges, algorithms are awesome, and one of D's main
events... but new users need to get far enough into the woods to reach
that point.
We need to make sure we're the least amount likely to repel them prior
to reaching that point as possible.

I had a lunch discussion with a colleague today; he believes my claims
about ranges/algorithms and stuff, but he has no feel for it, and
can't really grok just how ground breaking that stuff is... he admits
he needs some experience to comment, and he's actively checking it
out.
It's critical that he doesn't try to call a function that takes a ref
before he reaches that moment, because if he does, chances are he'll
get angry at me for wasting his time.
Mar 23 2018
next sibling parent reply Nick Sabalausky <a a.a> writes:
It never made any sense to me that there could be any problem 
with the compiler automatically creating a temporary hidden 
lvalue so a ref could be taken. If there IS any problem, I can 
only imagine it would be symptomatic of a different, larger 
problem.
Mar 23 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:

 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.


It can be a serious API problem if you can't look at ref and know that the
intention is that the original argument's value will be used and then
mutated (whereas with out, it will be mutated, but the original value won't
be used). So, having ref in general accept rvalues would be a huge problem.
However, that could be solved by having a different attribute indicate that
the idea is that the compiler will accept rvalues and create temporaries for
them if they're passed instead of an lvalue. Then, the situation is clear.

As for rvalue references in general, I can never remember what exactly the
problem is. IIRC, part of it relates to the fact that it makes it impossible
for the compiler to know whether it's dealing with an actual lvalue or not,
which has serious  safety implications, and in the case of C++, complicates
things considerably. Andrei has plenty of nasty things to say about how
rvalue references in C++ were a huge mistake. It's just that I can never
remember the arguments very well.

The use of scope with DIP 1000 may reduce the problem such that scope ref
could be made to work  safely (I don't know), but that by itself isn't
enough if you want it to be clear whether a function is supposed to be
mutating the argument or if it's just trying to avoid copying it. C++ solves
that problem by requiring const with rvalue references, but given how D's
const works, that really wouldn't make sense. So, we'd have to do something
else.

I get the impression that this issue is one where it seems like a small one
on the surface but that when you get into the guts of what it actually means
to implement it, it gets nasty.

I think that most of us have just take the approach of passing everything by
value unless the type is clearly too expensive to copy for that to make
sense, in which case, it's then passed by ref or auto ref, or we make it a
reference type. I don't know how much the problem with the lack of rvalue
references in D is a PR issue, because C++ programmers get annoyed about it
and deem D to be subpar as a result, and how much it's an actual performance
problem. I don't work in Manu's world, so I don't what really makes sense
there. For me, passing by value is usually a non-issue, and it's easy enough
to work around the problem in the rare cases where it is a problem. Clearly,
it's a PR issue in Manu's world, but it may also be a performance problem. I
don't know. Either way, it's clear that Manu and others like him think that
the fact that D doesn't have rvalue references is a serious deficiency.

- Jonathan M Davis
Mar 23 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/23/2018 07:46 PM, Jonathan M Davis wrote:

 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:

 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.


 
 It can be a serious API problem if you can't look at ref and know that the
 intention is that the original argument's value will be used and then
 mutated (whereas with out, it will be mutated, but the original value won't
 be used).


???. That's equally true when an lvalue is passed in.


 However, that could be solved by having a different attribute indicate that
 the idea is that the compiler will accept rvalues and create temporaries for
 them if they're passed instead of an lvalue. Then, the situation is clear.


Why require the callee's author to add boilerplate? Just do it for all 
ref params that are given an rvalue.


 As for rvalue references in general, I can never remember what exactly the
 problem is. IIRC, part of it relates to the fact that it makes it impossible
 for the compiler to know whether it's dealing with an actual lvalue or not,


As long as the compiler takes the advocated "automatic hidden temporary" 
approach, then it *is* an actual lvalue. We're talking nothing more than 
sugar here. Sugar for what we're already forced to do manually. There 
*cannot* be a technical problem here that isn't *already* a problem with 
the author manually creating a temp var.
Mar 23 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d 
wrote:

 On 03/23/2018 07:46 PM, Jonathan M Davis wrote:

 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d 




wrote:

 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.


 It can be a serious API problem if you can't look at ref and know that
 the intention is that the original argument's value will be used and
 then mutated (whereas with out, it will be mutated, but the original
 value won't be used).


 ???. That's equally true when an lvalue is passed in.


 However, that could be solved by having a different attribute indicate
 that the idea is that the compiler will accept rvalues and create
 temporaries for them if they're passed instead of an lvalue. Then, the
 situation is clear.


 Why require the callee's author to add boilerplate? Just do it for all
 ref params that are given an rvalue.


Because if the point of the function accepting its argument by ref is
because it's going to mutate the argument, then it makes no sense for it to
accept rvalues, whereas if the point of the function accepting its argument
by ref is to just avoid a copy, then it makes sense to accept both. It
should be clear from the API which is intended. As it stands, because a
function can't accept rvalues by ref, it's usually reasonable to assume that
a function accepts its argument by ref because it's mutating that argument
rather than simply because it's trying to avoid a copy. If ref suddenly
starts accepting rvalues, then we lose that.

With C++, you know the difference because, only const& accepts rvalues, and
whether it's const or some other attribute, I'd very much like that any ref
that accepts rvalues in D be marked with something to indicate that it does
rather than making ref do double-duty and make it unclear what it's there
for.

- Jonathan M Davis
Mar 24 2018
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:


 As it stands, because a function can't accept rvalues by ref, it's
 usually reasonable to assume that a function accepts its argument by ref
 because it's mutating that argument rather than simply because it's
 trying to avoid a copy. If ref suddenly starts accepting rvalues, then
 we lose that.


Any reason you can't simply use `ref` to imply 'modifies value' and 
`const ref` as 'passed by ref for performance reasons'?

-- 
Johannes
Mar 24 2018
parent Johannes Pfau <nospam example.com> writes:
Am Sat, 24 Mar 2018 17:10:53 +0000 schrieb Johannes Pfau:


 Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:
 

 As it stands, because a function can't accept rvalues by ref, it's
 usually reasonable to assume that a function accepts its argument by
 ref because it's mutating that argument rather than simply because it's
 trying to avoid a copy. If ref suddenly starts accepting rvalues, then
 we lose that.


 
 Any reason you can't simply use `ref` to imply 'modifies value' and
 `const ref` as 'passed by ref for performance reasons'?


Sorry, I see Manu already asked the same question.

-- 
Johannes
Mar 24 2018
prev sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/24/2018 03:03 AM, Jonathan M Davis wrote:

 On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d
 wrote:

 Why require the callee's author to add boilerplate? Just do it for all
 ref params that are given an rvalue.


 
 Because if the point of the function accepting its argument by ref is
 because it's going to mutate the argument, then it makes no sense for it to
 accept rvalues, 


1. That *isn't* always the core point of a function which takes a 
non-const ref argument.

2. It's the caller who decides whether or not the ref-result is needed, 
not the callee.

3. The frequent recurring complaints about no rvalue references are a 
testament that this is too common a use-case for the current "manually 
insert temporaries" workaround to be satisfactory.

4. If the whole point of disallowing rvalue references is to prevent 
accidents due to callers not realizing a param is being passed by ref 
(as it sounds like you're suggesting), then it's nothing but a broken 
half-solution, because it fails to provide that safety (and thus fails 
its own charter) when that same caller, once again not realizing a param 
is ref, passes an *lvalue* without expecting it to change. *If* the 
problem we want to solve here really is making sure a caller knows when 
a param is ref, we've failed, and the only way to actually accomplish it 

and raise an error when they don't.
Mar 24 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 00:04, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d
 wrote:

 On 03/23/2018 07:46 PM, Jonathan M Davis wrote:

 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d




 wrote:

 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.


 It can be a serious API problem if you can't look at ref and know that
 the intention is that the original argument's value will be used and
 then mutated (whereas with out, it will be mutated, but the original
 value won't be used).


 ???. That's equally true when an lvalue is passed in.


 However, that could be solved by having a different attribute indicate
 that the idea is that the compiler will accept rvalues and create
 temporaries for them if they're passed instead of an lvalue. Then, the
 situation is clear.


 Why require the callee's author to add boilerplate? Just do it for all
 ref params that are given an rvalue.


 Because if the point of the function accepting its argument by ref is
 because it's going to mutate the argument, then it makes no sense for it to
 accept rvalues, whereas if the point of the function accepting its argument
 by ref is to just avoid a copy, then it makes sense to accept both. It
 should be clear from the API which is intended.


Write const; it's as clear as day!
Mar 24 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 16:46, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:

 It never made any sense to me that there could be any problem
 with the compiler automatically creating a temporary hidden
 lvalue so a ref could be taken. If there IS any problem, I can
 only imagine it would be symptomatic of a different, larger
 problem.


 It can be a serious API problem if you can't look at ref and know that the
 intention is that the original argument's value will be used and then
 mutated (whereas with out, it will be mutated, but the original value won't
 be used).


Okay, let's read 'const ref' every time I say 'ref'. I thought that
would be fairly safe to assume. Sorry!



 However, that could be solved by having a different attribute indicate that
 the idea is that the compiler will accept rvalues and create temporaries for
 them if they're passed instead of an lvalue. Then, the situation is clear.


No, no attributes.. Just accept any argument to const-ref!
The function's not gonna change it.



 As for rvalue references in general, I can never remember what exactly the
 problem is. IIRC, part of it relates to the fact that it makes it impossible
 for the compiler to know whether it's dealing with an actual lvalue or not,
 which has serious  safety implications, and in the case of C++, complicates
 things considerably. Andrei has plenty of nasty things to say about how
 rvalue references in C++ were a huge mistake. It's just that I can never
 remember the arguments very well.

 The use of scope with DIP 1000 may reduce the problem such that scope ref
 could be made to work  safely (I don't know)


True, but if you follow that line of reasoning, then this case must
equally be banned:

T temp = f();
func(temp);

The  safety fear is that the pointer to the stack arg may escape, and
that's identical whether the argument is an explicit temp, or an
implicit one.



 but that by itself isn't
 enough if you want it to be clear whether a function is supposed to be
 mutating the argument


Functions that receive const args make it pretty clear that they don't
intend to mutate the arg.



 I get the impression that this issue is one where it seems like a small one
 on the surface but that when you get into the guts of what it actually means
 to implement it, it gets nasty.


It really doesn't... the compiler just make the same temp that I type
with my hands. That's literally all that it needs to do. I'd bet money
it's a one-paragraph change.



 Clearly, it's a PR issue in Manu's world, but it may also be a performance
problem. I
 don't know. Either way, it's clear that Manu and others like him think that
 the fact that D doesn't have rvalue references is a serious deficiency.


That's an understatement, but yes :P
We want to call C++ code. The D code shouldn't be objectively worse
than the C++ code we're trying to escape, otherwise we're undermining
our gravity towards D.
We can't have a situation that goes "I wrote this 3 line function in
D, but now it's 6 lines because I had to break args to temps, it's
objectively worse than the equivalent C++ function... why am I writing
it in D again?".
Mar 23 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 01:35, Manu wrote:

 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!


Absolutely not. It makes absolutely no sense to restrict rvalue 
references to const objects. (Recall that const is transitive and 
actually prevents mutation. This is not C++.)
Mar 23 2018
parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 24.03.2018 01:35, Manu wrote:

 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!



 Absolutely not. It makes absolutely no sense to restrict rvalue references
 to const objects. (Recall that const is transitive and actually prevents
 mutation. This is not C++.)


We're not talking about rvalue-references... were talking about
not-rvalue-references to temporaries.
Mar 23 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 04:14, Manu wrote:

 On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 24.03.2018 01:35, Manu wrote:

 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!



 Absolutely not. It makes absolutely no sense to restrict rvalue references
 to const objects. (Recall that const is transitive and actually prevents
 mutation. This is not C++.)


 
 We're not talking about rvalue-references... were talking about
 not-rvalue-references to temporaries.
 


Let me rephrase:

Absolutely not. It makes absolutely no sense to force const for 
not-rvalue-references to temporaries. (Recall that const is transitive 
and actually prevents mutation. This is not C++.)
Mar 23 2018
parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 20:19, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 24.03.2018 04:14, Manu wrote:

 On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 24.03.2018 01:35, Manu wrote:

 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!




 Absolutely not. It makes absolutely no sense to restrict rvalue
 references
 to const objects. (Recall that const is transitive and actually prevents
 mutation. This is not C++.)



 We're not talking about rvalue-references... were talking about
 not-rvalue-references to temporaries.


 Let me rephrase:

 Absolutely not. It makes absolutely no sense to force const for
 not-rvalue-references to temporaries. (Recall that const is transitive and
 actually prevents mutation. This is not C++.)


Yes, I know... why would you mutate an argument who's life doesn't
extend beyond the call? Any mutation is pointless.
Mar 23 2018
prev sibling next sibling parent reply MattCoder <nospam mail.com> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 ...
 By contrast, people will NOT forgive the fact that they have to 
 change:

     func(f(x), f(y), f(z));

 to:

     T temp = f(x);
     T temp2 = f(y);
     T temp3 = f(z);
     func(temp, temp2, temp3);
...


Or you could do this:

func(tx=f(x), ty=f(y), tz=f(z));

I know it will not solve your problem, but except for the 
explicit variables, it's almost like your first example.

Well, to be honest I still can't understand why would you want to 
pass a RValue as reference.

Matt.
Mar 23 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:

 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.


Well, it's frequently the case that you don't want to copy an object if you
don't have to - especially in the gaming world, where every ounce of
performance matters. If a function accepts an argument by ref, then no copy
is made, but then you have to pass an lvalue, making it really annoying to
call the function when what you have is an rvalue. On the other hand, if the
function doesn't accept its argument by ref, then you can pass an rvalue
(and it will be moved, making that efficient), but then lvalues get copied
when that's often not what you want. C++'s solution to this was rvalue
references. That way, as long as the parameter is const, it can accept both
rvalues and lvalues, and it won't copy unless it has to. The closest
analogue that D has to this is auto ref, which requires templates, which may
or may not be acceptable. In many cases, it works great, whereas in others,
it doesn't work at all (e.g. virtual functions), and it can result in
template bloat.

The whole situation is complicated by the fact that sometimes it's actually
faster to pass by value and copy the argument, even if it's an lvalue.
Passing by const& can often result in unnecessary copies if anywhere in the
chain passes the object by value, whereas if it's passed by value, the same
object can often be moved multiple times, avoiding any copies. It's my
understanding that prior to C++11, it was considered best practice to pass
by const& as much as possible to avoid copies but that after C++11 (which
added move constructors), it's often considered better to pass by value,
because then in many cases, the compiler can move the object instead of
copying it. So, C++ gives you control over which you do, and it's not
necessarily straightforward as to which you should use (though plenty of
older C++ programmers likely just use const& all over the place out of
habit).

D supports moving out of the box without move constructors, so it does a
good job of handling the cases where a move is most appropriate, but it
doesn't have rvalue references, so it doesn't have the same flexibility as
C++ when you want to pass something by reference. So, anyone looking to pass
stuff by reference all over the place (like Manu and his coworkers) is going
to find the lack of rvalue references in D to be really annoying.

- Jonathan M Davis
Mar 23 2018
parent reply MattCoder <nospam mail.com> writes:
On Friday, 23 March 2018 at 23:58:05 UTC, Jonathan M Davis wrote:

 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d 
 wrote:

 Well, to be honest I still can't understand why would you want 
 to pass a RValue as reference.


 Well, it's frequently the case that you don't want to copy an 
 object if you don't have to...
 - Jonathan M Davis


Well the concept it's OK. (Differences between passing by value 
vs reference, copy etc.). Except for the const thing in C++, 
because I don't know this language, and by the way thanks for 
explaining that.

Question:

In C++ the signature of the function which will receive the 
references like in this case, need to be "const ref" parameters, 
right? - If yes, then since it's const ref parameter, will not 
change the value passed, even if it's lvalue, right?

Matt.
Mar 23 2018
parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 17:48, MattCoder via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Question:

 In C++ the signature of the function which will receive the references like
 in this case, need to be "const ref" parameters, right? - If yes, then since
 it's const ref parameter, will not change the value passed, even if it's
 lvalue, right?


Right. It's so obvious isn't it ;)
Mar 23 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:

 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.


 Well, it's frequently the case that you don't want to copy an object if you
 don't have to - especially in the gaming world, where every ounce of
 performance matters. If a function accepts an argument by ref, then no copy
 is made, but then you have to pass an lvalue, making it really annoying to
 call the function when what you have is an rvalue. On the other hand, if the
 function doesn't accept its argument by ref, then you can pass an rvalue
 (and it will be moved, making that efficient), but then lvalues get copied
 when that's often not what you want. C++'s solution to this was rvalue
 references.


Ummm... rvalue-references are something completely different.
rval-ref's are C++'s solution to move semantics.
C++ just simply accepts rvalues passed to const& args. It makes a temp
and passes the ref, as you expect.



 That way, as long as the parameter is const, it can accept both
 rvalues and lvalues, and it won't copy unless it has to. The closest
 analogue that D has to this is auto ref, which requires templates, which may
 or may not be acceptable.


auto-ref == template function, which by definition is NOT an
extern(C++) function. auto-ref may also resolve to NOT a ref, which
means a move... data structures that are large (ie, a vector or
matrix) still have to copy a bunch of memory, even if the copy is
algorithmically 'cheap'.



 In many cases, it works great, whereas in others,
 it doesn't work at all (e.g. virtual functions), and it can result in
 template bloat.


And templates, that's another case where it fails.
auto-ref is something else unrelated to this topic, and it's useful in
a different set of cases for a different set of uses/reasons. It's got
nothing to do with this.



 The whole situation is complicated by the fact that sometimes it's actually
 faster to pass by value and copy the argument, even if it's an lvalue.


The api author wouldn't have made the arg a ref in that case.



 Passing by const& can often result in unnecessary copies if anywhere in the
 chain passes the object by value, whereas if it's passed by value, the same
 object can often be moved multiple times, avoiding any copies**.


**avoiding __calls to the copy constructor__. The memory is likely to
still be moved around a bunch of times.
The case you're thinking of is when values are *returned* by value, in
that case, copy elision is possible, and the result can be constructed
in place. That's a completely unrelated problem... you don't do
return-by-ref ;)



 It's my
 understanding that prior to C++11, it was considered best practice to pass
 by const& as much as possible to avoid copies but that after C++11 (which
 added move constructors), it's often considered better to pass by value,
 because then in many cases, the compiler can move the object instead of
 copying it**.


** Assuming the object is tiny, but has an expensive copy constructor.
In the case where an object is large (and has a primitive, or no copy
constructor) it doesn't change the situation; you still wanna pass a
big thing by ref in all cases; ie, vector/matrix.



 So, C++ gives you control over which you do, and it's not
 necessarily straightforward as to which you should use (though plenty of
 older C++ programmers likely just use const& all over the place out of
 habit).


D gives you the same set of options; except that passing args by ref
is a PITA in D, and ruins your code.
Mar 23 2018
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote:

 but that by itself isn't
 enough if you want it to be clear whether a function is supposed to be
 mutating the argument


 Functions that receive const args make it pretty clear that they don't
 intend to mutate the arg.


Yes, but with how restrictive const is in D, I have a very hard time
believing that it's going to work well to start using const ref much even if
it accepted rvalues.

- Jonathan M Davis
Mar 23 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote:

 On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:

 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:

 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.


 Well, it's frequently the case that you don't want to copy an object if
 you don't have to - especially in the gaming world, where every ounce
 of performance matters. If a function accepts an argument by ref, then
 no copy is made, but then you have to pass an lvalue, making it really
 annoying to call the function when what you have is an rvalue. On the
 other hand, if the function doesn't accept its argument by ref, then
 you can pass an rvalue (and it will be moved, making that efficient),
 but then lvalues get copied when that's often not what you want. C++'s
 solution to this was rvalue references.


 Ummm... rvalue-references are something completely different.
 rval-ref's are C++'s solution to move semantics.
 C++ just simply accepts rvalues passed to const& args. It makes a temp
 and passes the ref, as you expect.


It was my understanding that that _was_ an rvalue reference, and I remember
that Andrei was against const ref accepting rvalues in D due to issues with
rvalue references. It's quite possible that I misremember though, and I
certainly am not an expert on all of the issues with rvalues and references
in C++, and my C++ knowledge is getting increasingly rusty, since I don't
use C++ much anymore. In any case, I have a terrible time remembering
Andrei's exact arguments, but he feels very strongly about them, so anyone
looking to convince him is going to have a hard time of it.

My biggest concern in all of this is that I don't want to see ref start
accepting rvalues as has been occasionally discussed. It needs to be clear
when a function is accept an argument by ref because it's going to mutate
the object and when it's accepting by ref because it wants to avoid a copy.
The addition of const solves that problem for C++, but given how restrictive
const is in D, I doubt that much of anyone would ultimately be very happy
with using const ref in their code very often.


 In many cases, it works great, whereas in others,
 it doesn't work at all (e.g. virtual functions), and it can result in
 template bloat.


 And templates, that's another case where it fails.
 auto-ref is something else unrelated to this topic, and it's useful in
 a different set of cases for a different set of uses/reasons. It's got
 nothing to do with this.


auto ref has everything to do with a function accepting both rvalues and
lvalues without making a copy. auto ref is the solution that was introduced
into D to solve that very problem. It's just that it has limitations that
make it inappropriate in a number of cases, so you don't consider it a
solution to your problem.


 So, C++ gives you control over which you do, and it's not
 necessarily straightforward as to which you should use (though plenty of
 older C++ programmers likely just use const& all over the place out of
 habit).


 D gives you the same set of options; except that passing args by ref
 is a PITA in D, and ruins your code.


Which is why I said that D doesn't give you the same set of options.

- Jonathan M Davis
Mar 23 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/23/2018 09:06 PM, Jonathan M Davis wrote:

 
 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be clear
 when a function is accept an argument by ref because it's going to mutate
 the object and when it's accepting by ref because it wants to avoid a copy.
 


That ship sailed ages ago: It's already unclear. If we want to fix that, 
we can fix it, but blocking rvalue ref does nothing for that cause.
Mar 29 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d 
wrote:

 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:

 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants to
 avoid a copy.


 That ship sailed ages ago: It's already unclear. If we want to fix that,
 we can fix it, but blocking rvalue ref does nothing for that cause.


Really? And how often does ref get used just to avoid copying? You can
almost always look at a function, see that it accepts ref, and know that
it's supposed to mutate the argument. Functions that want to avoid copying
lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a
number of folks would start using it all over the place to avoid copying.
Right now, folks rarely use ref that way, because it becomes too annoying to
call the function with an rvalue. So, while it might not be the case 100% of
the time right now that ref is used with the purpose of mutating the
argument, it almost always is. As such, you can pretty reliably look at a
function signature and expect that if one of its parameters is ref, it's
going to be mutating that argument. The function that accepts an argument by
ref with no intention of mutating it is very much the exception, and I
really don't want to see that change.

- Jonathan M Davis
Mar 29 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d
 wrote:

 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:

 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants to
 avoid a copy.


 That ship sailed ages ago: It's already unclear. If we want to fix that,
 we can fix it, but blocking rvalue ref does nothing for that cause.


 Really? And how often does ref get used just to avoid copying? You can
 almost always look at a function, see that it accepts ref, and know that
 it's supposed to mutate the argument. Functions that want to avoid copying
 lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a
 number of folks would start using it all over the place to avoid copying.
 Right now, folks rarely use ref that way, because it becomes too annoying to
 call the function with an rvalue. So, while it might not be the case 100% of
 the time right now that ref is used with the purpose of mutating the
 argument, it almost always is. As such, you can pretty reliably look at a
 function signature and expect that if one of its parameters is ref, it's
 going to be mutating that argument. The function that accepts an argument by
 ref with no intention of mutating it is very much the exception, and I
 really don't want to see that change.


Interesting. Just understand that you're trading that feeling for a
suite of edge cases though. Accepting asymmetric calling rules is a
pretty big cost to pay for that 'nice thought'.
That idea also dismisses the existence of the set of cases where ref
is genuinely useful/correct. Your sentiment effectively puts those use
cases into the position of 2nd-class citizens.
I understand your sentiment, but I think the cost is not balanced, or
even particularly fair with respect to users in those niche groups :/
Mar 30 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 17:58, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote:

 but that by itself isn't
 enough if you want it to be clear whether a function is supposed to be
 mutating the argument


 Functions that receive const args make it pretty clear that they don't
 intend to mutate the arg.


 Yes, but with how restrictive const is in D, I have a very hard time
 believing that it's going to work well to start using const ref much even if
 it accepted rvalues.


This is an interesting point, but I don't think it changes the balance
in any way. Thinking of the majority of my anecdotal cases, I don't
think it would be a problem.
Something complex enough for const to be a problem likely doesn't
conform to this pattern.

Further, extern(C++) functions that receive const& args are 'const
ref' regardless of D's const semantics. So for extern(C++), which I
suspect is a high percentage of cases where this issue is significant
(because D doesn't naturally follow C++'s pattern so much anyway),
that point doesn't matter.
Mar 23 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 02:16, Manu wrote:

 This is an interesting point, but I don't think it changes the balance
 in any way. Thinking of the majority of my anecdotal cases, I don't
 think it would be a problem.
 Something complex enough for const to be a problem likely doesn't
 conform to this pattern.


Why aim for "it often works", when we want "it always works"? Forcing 
const upon people who want to pass rvalues by reference is just not good 
enough. It is bad language design.

Also I think the point about documenting mutation intent is moot, as 
rvalues can be receivers for method calls, which will _already_ pass an 
rvalue by reference no matter whether it intends to mutate it or not. We 
can require some special annotation for this behavior, but I'd be 
perfectly fine without it.
Mar 23 2018
parent reply Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 24.03.2018 02:16, Manu wrote:

 This is an interesting point, but I don't think it changes the balance
 in any way. Thinking of the majority of my anecdotal cases, I don't
 think it would be a problem.
 Something complex enough for const to be a problem likely doesn't
 conform to this pattern.



 Why aim for "it often works", when we want "it always works"? Forcing const
 upon people who want to pass rvalues by reference is just not good enough.
 It is bad language design.


I think you need to re-read the whole thread, and understand what
we're even talking about.
Nobody wants to pass rvalues by mutable-ref... that's completely
pointless, since it's an rvalue that will timeout immediately anyway.
Passing by const-ref is perfectly acceptable.

I suspect Jonathan's talking about classic D situations with const
like, I might pass by const-ref, but then I can't call a getter that
caches the result.
That's a classic problem with D's const, and that's not on debate
here. I don't think that has any impact on this proposal; people
understand what const means in D, and that's no different here than
anywhere else.



 Also I think the point about documenting mutation intent is moot, as rvalues
 can be receivers for method calls, which will _already_ pass an rvalue by
 reference no matter whether it intends to mutate it or not. We can require
 some special annotation for this behavior, but I'd be perfectly fine without
 it.


I have no idea what this paragraph means... can you elaborate further
what you're talking about?
Mar 23 2018
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 05:03, Manu wrote:

 On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 24.03.2018 02:16, Manu wrote:

 This is an interesting point, but I don't think it changes the balance
 in any way. Thinking of the majority of my anecdotal cases, I don't
 think it would be a problem.
 Something complex enough for const to be a problem likely doesn't
 conform to this pattern.



 Why aim for "it often works", when we want "it always works"? Forcing const
 upon people who want to pass rvalues by reference is just not good enough.
 It is bad language design.


 
 I think you need to re-read the whole thread, and understand what
 we're even talking about. ...


That will not be necessary. I wouldn't even have had to read it the 
first time. Those discussions always go the same way:

M: I wish we could pass rvalue arguments to ref parameters.
J: That would be terrible, as people would then pass rvalues as ref by 
accident and not see the mutation that the author of the function 
intended them to see.
M: Only do it for const ref parameters then.
T: No, this has nothing to do with const.

(M can be replaced by a variety of other letters; this is a somewhat 
common feature request.)


 Nobody wants to pass rvalues by mutable-ref... that's completely
 pointless, since it's an rvalue that will timeout immediately anyway.
 Passing by const-ref is perfectly acceptable.
 ...


Your temporary might have mutable indirections. Maybe you don't want to 
be forced to annotate your methods as const, limiting your future options.


 I suspect Jonathan's talking about classic D situations with const
 like, I might pass by const-ref, but then I can't call a getter that
 caches the result.
 That's a classic problem with D's const, and that's not on debate
 here. I don't think that has any impact on this proposal; people
 understand what const means in D, and that's no different here than
 anywhere else.
 ...


Your proposal _changes_ the meaning of const. I.e., "const does all it 
did previously and now it also allows rvalues to be passed to ref 
functions". This is bad, as one has little to do with the other, yet now 
you couple them. Programmers who want to pass rvalues as ref do not 
necessarily want to use D const on their objects.


 

 Also I think the point about documenting mutation intent is moot, as rvalues
 can be receivers for method calls, which will _already_ pass an rvalue by
 reference no matter whether it intends to mutate it or not. We can require
 some special annotation for this behavior, but I'd be perfectly fine without
 it.


 
 I have no idea what this paragraph means... can you elaborate further
 what you're talking about?
 


This works:

struct S{
     int x;
     void inc(){
         this.x++; // note: 'this' is passed by ref
     }
}

void main(){
     S().inc();
}

but this does not:

struct S{
     int x;
}
void inc(ref S self){
     self.x++; // note: 'self' is passed by ref
}

void main(){
     S().inc();
}

I.e. there is a special case where your rewrite is already applied. Note 
how "inc" cannot even be made const.

What I'm saying is that I don't really buy Jonathan's argument. 
Basically, you should just pass the correct arguments to functions, as 
you always need to do. If you cannot use the result of some mutation 
that you need to use, you will probably notice.


There are only three sensible ways to fix the problem:

1. Just allow rvalue arguments to bind to ref parameters. (My preferred 
solution, though it will make the overloading rules slightly more 
complicated.)

2. Add some _new_ annotation for ref parameters that signifies that you 
want the same treatment for them that the implicit 'this' reference 
gets. (A close second.)

3. Continue to require code bloat (auto ref) or manual boilerplate 
(overloads). (I'm not very fond of this option, but it does not require 
a language change.)
Mar 24 2018
next sibling parent reply kinke <noone nowhere.com> writes:
On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:

 What I'm saying is that I don't really buy Jonathan's argument. 
 Basically, you should just pass the correct arguments to 
 functions, as you always need to do. If you cannot use the 
 result of some mutation that you need to use, you will probably 
 notice.


I agree, but restricting it to const ref would be enough for 
almost all of my use cases. The MS C++ compiler just emits a 
warning when binding an rvalue to a mutable ref ('nonstandard 
extension used'), I'd find that absolutely viable for D too.


 There are only three sensible ways to fix the problem:

 1. Just allow rvalue arguments to bind to ref parameters. (My 
 preferred solution, though it will make the overloading rules 
 slightly more complicated.)


I always thought the main concern was potential escaping refs to 
the rvalue, which would be solvable by allowing rvalues to be 
bound to `scope ref` params only. That'd be my preferred choice 
as well.


 2. Add some _new_ annotation for ref parameters that signifies 
 that you want the same treatment for them that the implicit 
 'this' reference gets. (A close second.)


*Shudder*.


 3. Continue to require code bloat (auto ref) or manual 
 boilerplate (overloads). (I'm not very fond of this option, but 
 it does not require a language change.)


While `auto ref` seems to have worked out surprisingly well for 
code written in D, it doesn't solve the problem when interfacing 
with (many) external C++ functions taking structs (represented in 
D by structs as well) by (mostly const) ref. You're forced to 
declare lvalues for all of these args, uglifying the code 
substantially.
Mar 24 2018
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 24.03.2018 15:56, kinke wrote:

 On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:

 What I'm saying is that I don't really buy Jonathan's argument. 
 Basically, you should just pass the correct arguments to functions, as 
 you always need to do. If you cannot use the result of some mutation 
 that you need to use, you will probably notice.


 
 I agree, but restricting it to const ref would be enough for almost all 
 of my use cases. The MS C++ compiler just emits a warning when binding 
 an rvalue to a mutable ref ('nonstandard extension used'), I'd find that 
 absolutely viable for D too.
 ...


A warning is not viable. (There's no good way to fix it.)


 There are only three sensible ways to fix the problem:

 1. Just allow rvalue arguments to bind to ref parameters. (My 
 preferred solution, though it will make the overloading rules slightly 
 more complicated.)


 
 I always thought the main concern was potential escaping refs to the 
 rvalue, which would be solvable by allowing rvalues to be bound to 
 `scope ref` params only. That'd be my preferred choice as well.
 ...


There is no difference between escaping refs to an rvalue and escaping 
refs to a short-lived lvalue, as the callee has no idea where the 
address is coming from anyway. According to Walter, ref parameters are 
not supposed to be escaped, and  safe will enforce it.

Also, AFAIU, "scope" in "scope ref T" already applies to "T", not "ref".


 2. Add some _new_ annotation for ref parameters that signifies that 
 you want the same treatment for them that the implicit 'this' 
 reference gets. (A close second.)


 
 *Shudder*.
 ...


Well, it beats "const".


 3. Continue to require code bloat (auto ref) or manual boilerplate 
 (overloads). (I'm not very fond of this option, but it does not 
 require a language change.)


 
 While `auto ref` seems to have worked out surprisingly well for code 
 written in D, it doesn't solve the problem when interfacing with (many) 
 external C++ functions taking structs (represented in D by structs as 
 well) by (mostly const) ref. You're forced to declare lvalues for all of 
 these args, uglifying the code substantially.


You can add additional overloads on the D side. (This can even be 
automated using a string mixin.)
Mar 24 2018
parent kinke <noone nowhere.com> writes:
On Saturday, 24 March 2018 at 15:36:14 UTC, Timon Gehr wrote:

 On 24.03.2018 15:56, kinke wrote:

 I agree, but restricting it to const ref would be enough for 
 almost all of my use cases. The MS C++ compiler just emits a 
 warning when binding an rvalue to a mutable ref ('nonstandard 
 extension used'), I'd find that absolutely viable for D too.
 ...


 A warning is not viable. (There's no good way to fix it.)


As long as specific warnings cannot be suppressed via pragmas, 
one would need to predeclare the lvalue to get rid of it; fine 
IMHO for the, as I expect, very rare use cases.


 There is no difference between escaping refs to an rvalue and 
 escaping refs to a short-lived lvalue, as the callee has no 
 idea where the address is coming from anyway. According to 
 Walter, ref parameters are not supposed to be escaped, and 
  safe will enforce it.


Alright, the less keywords overhead, the better. :)


 You can add additional overloads on the D side. (This can even 
 be automated using a string mixin.)


Right I can, but I don't want to add 7 overloads for a C++ 
function taking 3 params by const ref. Even if autogenerated by 
some tool or fancy mixins, the code's legibility would suffer a 
lot. D's syntax is IMO one of its strongest selling points, and 
that shouldn't degrade when it comes to C(++) interop.
Mar 24 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 07:56, kinke via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:

 What I'm saying is that I don't really buy Jonathan's argument. Basically,
 you should just pass the correct arguments to functions, as you always need
 to do. If you cannot use the result of some mutation that you need to use,
 you will probably notice.



 I agree, but restricting it to const ref would be enough for almost all of
 my use cases. The MS C++ compiler just emits a warning when binding an
 rvalue to a mutable ref ('nonstandard extension used'), I'd find that
 absolutely viable for D too.


 There are only three sensible ways to fix the problem:

 1. Just allow rvalue arguments to bind to ref parameters. (My preferred
 solution, though it will make the overloading rules slightly more
 complicated.)



 I always thought the main concern was potential escaping refs to the rvalue,
 which would be solvable by allowing rvalues to be bound to `scope ref`
 params only. That'd be my preferred choice as well.


I touched on this.
It sounds reasonable at first glance. But the reasoning goes:
1. for safety reasons, we won't allow implicit stack temporaries to
pass to functions that may escape them.
2. support passing temporaries only to 'scope ref'
3. realise that the implicit temp created by an rvalue is identical to
the manually authored temp (or any other stack locals in the caller),
you find that the only reasonable option to satisfy the alleged safety
concern, is that all stack variables may never be passed to any
function that's not 'scope ref'.

Ie, you can't do this for safety reasons, because the exact same
reasoning would exclude all stack variables ever from being passed the
same way. The proposition is self-defeating.
I was attracted to this idea for a short while, until I realised it
was ridiculous. ;)



 3. Continue to require code bloat (auto ref) or manual boilerplate
 (overloads). (I'm not very fond of this option, but it does not require a
 language change.)



 While `auto ref` seems to have worked out surprisingly well for code written
 in D, it doesn't solve the problem when interfacing with (many) external C++
 functions taking structs (represented in D by structs as well) by (mostly
 const) ref. You're forced to declare lvalues for all of these args,
 uglifying the code substantially.


It's also not great for libraries. auto ref functions are template
functions... what if I want to export that function? I can't.
There are practical API and ABI reasons that you don't want every
function to be a template function.
Library authors carefully control which functions are 'real' functions
and which are templates, for a wide variety of reasons. Binary libs
are a thing. DLL's are a thing.
Mar 24 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 06:49, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 24.03.2018 05:03, Manu wrote:

 I have no idea what this paragraph means... can you elaborate further
 what you're talking about?


 This works:

 struct S{
     int x;
     void inc(){
         this.x++; // note: 'this' is passed by ref
     }
 }

 void main(){
     S().inc();
 }

 but this does not:

 struct S{
     int x;
 }
 void inc(ref S self){
     self.x++; // note: 'self' is passed by ref
 }

 void main(){
     S().inc();
 }

 I.e. there is a special case where your rewrite is already applied. Note how
 "inc" cannot even be made const.

 What I'm saying is that I don't really buy Jonathan's argument. Basically,
 you should just pass the correct arguments to functions, as you always need
 to do. If you cannot use the result of some mutation that you need to use,
 you will probably notice.


Your example demonstrates the exact reason why rvalue->ref is const&
in C++, and illegal in D though.
You mutate a temporary that times out at the end of the statement...
your statement is never assigned to anything, and has no effect.
If your statement is assigned to something, then you already have an
lvalue to pass to such a function that receives mutable ref.
Mar 24 2018
parent Nick Treleaven <nick geany.org> writes:
On Saturday, 24 March 2018 at 17:34:09 UTC, Manu wrote:

 You mutate a temporary that times out at the end of the 
 statement...
 your statement is never assigned to anything, and has no effect.


That is solved by having the ref function return its argument (so 
it can be chained):

struct S;
ref S modify(return ref S s);

S(data).modify.writeln;

This rvalue pattern would be disallowed if argument `s` was 
const. Why not make `modify` just return a copy then, maybe the 
optimizer could remove the copy? So that you can also use it with 
lvalues:

S s;
modify(s);
Mar 29 2018
prev sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 03/24/2018 12:03 AM, Manu wrote:

 Why aim for "it often works", when we want "it always works"? Forcing const
 upon people who want to pass rvalues by reference is just not good enough.
 It is bad language design.


 
 I think you need to re-read the whole thread, and understand what
 we're even talking about.
 Nobody wants to pass rvalues by mutable-ref... 


I do. The ban serves no useful purpose (at least not any purpose that D 
doesn't *already* fail at - like knowing at the callsite whether a param 
is intended to be mutated). And it would permit the "avoid a copy" 
benefits in a completely orthogonal way - *without* relying on the param 
fitting D's transitive const requirements.
Mar 29 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On 23 March 2018 at 18:06, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote:

 On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:

 On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:

 Well, to be honest I still can't understand why would you want to
 pass a RValue as reference.


 Well, it's frequently the case that you don't want to copy an object if
 you don't have to - especially in the gaming world, where every ounce
 of performance matters. If a function accepts an argument by ref, then
 no copy is made, but then you have to pass an lvalue, making it really
 annoying to call the function when what you have is an rvalue. On the
 other hand, if the function doesn't accept its argument by ref, then
 you can pass an rvalue (and it will be moved, making that efficient),
 but then lvalues get copied when that's often not what you want. C++'s
 solution to this was rvalue references.


 Ummm... rvalue-references are something completely different.
 rval-ref's are C++'s solution to move semantics.
 C++ just simply accepts rvalues passed to const& args. It makes a temp
 and passes the ref, as you expect.


 It was my understanding that that _was_ an rvalue reference, and I remember
 that Andrei was against const ref accepting rvalues in D due to issues with
 rvalue references.


C++ const& to 'rvalues' are just lvalue refs to temporaries, exactly
as I propose here.
rval-ref's are something completely different (all about move
semantics), and have nothing to do with this conversation.

There is a potentially interesting parallel conversation which
discusses how to interact with extern(C++) functions that receive
rvalue ref's, but that actually is a complex conversation, and no
simple answers exist.



 In any case, I have a terrible time remembering
 Andrei's exact arguments, but he feels very strongly about them, so anyone
 looking to convince him is going to have a hard time of it.


Fortunately, I'm not trying to make any sort of argument for rvalue-ref's in D.



 My biggest concern in all of this is that I don't want to see ref start
 accepting rvalues as has been occasionally discussed. It needs to be clear
 when a function is accept an argument by ref because it's going to mutate
 the object and when it's accepting by ref because it wants to avoid a copy.


It's not going to mutate the argument, because it's const.



 The addition of const solves that problem for C++, but given how restrictive
 const is in D, I doubt that much of anyone would ultimately be very happy
 with using const ref in their code very often.


I expect I'll be 100% satisfied. And if not, it'll be something very
close to 100%.
This pattern is quite unlikely to proliferate within D code natively
(because auto ref, D move semantics, classes-as-ref-types, and such),
but it's essential for interacting with C++.



 auto ref has everything to do with a function accepting both rvalues and
 lvalues without making a copy. auto ref is the solution that was introduced
 into D to solve that very problem.


Yeah, somehow that emerged from this conversation years ago. I
aggressively expressed at the time that I never accepted it as a
solution, because it's not.
It's got nothing to do with this issue, and I said at the time that
I'll be very annoyed if it starts getting raised in this context ;)



 It's just that it has limitations that
 make it inappropriate in a number of cases, so you don't consider it a
 solution to your problem.


It's orthogonal to this conversation. It's the ability for templates
to automate the ref-ness of args for calling efficiency.
It's something like scott myers 'universal references'; ie,
`template<typename T> void func(T&& arg)`. It's really got nothing to
do with this conversation.



 So, C++ gives you control over which you do, and it's not
 necessarily straightforward as to which you should use (though plenty of
 older C++ programmers likely just use const& all over the place out of
 habit).


 D gives you the same set of options; except that passing args by ref
 is a PITA in D, and ruins your code.


 Which is why I said that D doesn't give you the same set of options.


It does though; D just forces you to write the temps that should be
implicit by hand. There's no reason for this. It just makes code ugly,
people angry, and there is no advantage.
Mar 23 2018
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...

 On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On 3/23/2018 11:09 AM, Manu wrote:

 [...]


 Rvalue references are not trivial and can have major 
 unintended consequences. They're a rather ugly feature in C++, 
 with weirdities. I doubt D will ever have them.


 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
 Explain how the situation if implemented would be any different 
 than
 the workaround?

 This seems even simpler than the pow thing to me.
 Rewrite:
     func(f());
 as:
     { auto __t0 = f(); func(__t0); }


I understand what you want, but I'm struggling to understand why 
it's such a huge deal.

The reason you want to pass by reference is for performance, to 
avoid copying the data at the call boundary.

So there are 2 cases: an lvalue needs to be passed, or an rvalue 
needs to be passed.

1. The address of the lvalue is passed.

2. The rvalue is copied to a local, then the address of that 
local is passed.

So in the rvalue case, you're not getting the performance benefit 
of passing by reference, because you have to copy to a local 
anyway.

What I would do in D currently to get the same performance and 
API:

void foo(float[32] v) { foo(v); }
void foo(ref float[32] v) { ... }

or

void foo()(auto ref float[32] v) { ... }

What is so totally unacceptable about those solutions? I 
personally like the second because it scales better to multiple 
parameters. I know you have said it's not relevant and annoying 
that people bring up auto ref, but I dont' get how or why. It's 
exactly D's solution to the problem.

There's a little more work to be done when thinking about 
extern(C++) and/or virtual functions, but most code for most 
people isn't made of virtual extern(C++) functions that take 
large value types can't accept the cost of copying a few lvalues.
Mar 24 2018
next sibling parent MattCoder <nospam mail.com> writes:
On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote:

 I understand what you want, but I'm struggling to understand 
 why it's such a huge deal.
 ...
 What I would do in D currently to get the same performance and 
 API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }

 What is so totally unacceptable about those solutions?


I hope OP answers that, because that was what I tried to refer in 
my post.

As your example, it works like in C and I'd prefer to use it in 
my code, instead of passing a rvalue as reference.

Matt.
Mar 24 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 24 March 2018 at 04:57, John Colvin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...

 On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 3/23/2018 11:09 AM, Manu wrote:

 [...]



 Rvalue references are not trivial and can have major unintended
 consequences. They're a rather ugly feature in C++, with weirdities. I doubt
 D will ever have them.



 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
 Explain how the situation if implemented would be any different than
 the workaround?

 This seems even simpler than the pow thing to me.
 Rewrite:
     func(f());
 as:
     { auto __t0 = f(); func(__t0); }


 I understand what you want, but I'm struggling to understand why it's such a
 huge deal.


Because it makes this kind of D code that interacts with C++
objectively worse than C++, and there's no reason for it.
You can't say to someone who just frustrated-ly doubled their line
count by manually introducing a bunch of temporaries in a tiny
function that appears to do something so simple as 'call a function',
that "oh yeah, isn't it cool that you can't just call your functions
anymore! isn't D cool! we should switch to D right?"
It's embarrassing. I've been put in the position where I have to try
and 'explain' this feature quite some number of times... they usually
just give me 'the look'™; ya know, quietly wondering if I'm still
sane, and all I end up with is someone who's about 95% less convinced
that D is cool than they were 5 seconds beforehand.
What pisses me off is that's such a pointless thing to happen, because
this issue is so trivial!

In my experience, people are evaluating how D will materially impact
the exact same code they're already writing in C++. This is one of
those ways that they will be materially impacted, and it's almost
enough all on its own to cause people to dismiss the entire thing on
the spot.
Pretty much the best case at this phase is that the D code is exactly
the same as C++. If we can trim off a few parens here and there
(ufcs?), maybe remove some '::' operators (modules that don't suck),
that's a huge win.



 The reason you want to pass by reference is for performance, to avoid
 copying the data at the call boundary.

 So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be
 passed.

 1. The address of the lvalue is passed.

 2. The rvalue is copied to a local, then the address of that local is
 passed.

 So in the rvalue case, you're not getting the performance benefit of passing
 by reference, because you have to copy to a local anyway.

 What I would do in D currently to get the same performance and API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }


Can't be extern(C++), can't be virtual either (both is likely).
I said before; you're talking about Scott Meyers 'universal
references' as a language concept, and I'm just talking about calling
a function.



 but I dont' get how or why. It's exactly D's solution to the problem.


It doesn't solve the problem... it doesn't even address the problem.
You're talking about a totally different thing >_<



 There's a little more work to be done when thinking about extern(C++) and/or
 virtual functions, but most code for most people isn't made of virtual
 extern(C++) functions that take large value types can't accept the cost of
 copying a few lvalues.


Correct, as I said before; people are getting bent out of shape over a
thing that will likely never affect them!
This will likely have very little impact on normal D code because D
const, auto ref, D move semantics, etc. It will have large impact on
interaction with C++ code, and the impression D is able to make on
that set of users. They're an important target market.
Mar 24 2018
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 24 March 2018 at 17:30:35 UTC, Manu wrote:

 On 24 March 2018 at 04:57, John Colvin via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...

 On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On 3/23/2018 11:09 AM, Manu wrote:

 [...]



 Rvalue references are not trivial and can have major 
 unintended consequences. They're a rather ugly feature in 
 C++, with weirdities. I doubt D will ever have them.



 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
 Explain how the situation if implemented would be any 
 different than
 the workaround?

 This seems even simpler than the pow thing to me.
 Rewrite:
     func(f());
 as:
     { auto __t0 = f(); func(__t0); }


 I understand what you want, but I'm struggling to understand 
 why it's such a huge deal.


 Because it makes this kind of D code that interacts with C++
 objectively worse than C++, and there's no reason for it.
 You can't say to someone who just frustrated-ly doubled their 
 line
 count by manually introducing a bunch of temporaries in a tiny
 function that appears to do something so simple as 'call a 
 function',
 that "oh yeah, isn't it cool that you can't just call your 
 functions
 anymore! isn't D cool! we should switch to D right?"
 It's embarrassing. I've been put in the position where I have 
 to try
 and 'explain' this feature quite some number of times... they 
 usually
 just give me 'the look'™; ya know, quietly wondering if I'm 
 still
 sane, and all I end up with is someone who's about 95% less 
 convinced
 that D is cool than they were 5 seconds beforehand.
 What pisses me off is that's such a pointless thing to happen, 
 because
 this issue is so trivial!

 In my experience, people are evaluating how D will materially 
 impact
 the exact same code they're already writing in C++. This is one 
 of
 those ways that they will be materially impacted, and it's 
 almost
 enough all on its own to cause people to dismiss the entire 
 thing on
 the spot.
 Pretty much the best case at this phase is that the D code is 
 exactly
 the same as C++. If we can trim off a few parens here and there
 (ufcs?), maybe remove some '::' operators (modules that don't 
 suck),
 that's a huge win.



 The reason you want to pass by reference is for performance, 
 to avoid copying the data at the call boundary.

 So there are 2 cases: an lvalue needs to be passed, or an 
 rvalue needs to be passed.

 1. The address of the lvalue is passed.

 2. The rvalue is copied to a local, then the address of that 
 local is passed.

 So in the rvalue case, you're not getting the performance 
 benefit of passing by reference, because you have to copy to a 
 local anyway.

 What I would do in D currently to get the same performance and 
 API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }


 Can't be extern(C++), can't be virtual either (both is likely).
 I said before; you're talking about Scott Meyers 'universal
 references' as a language concept, and I'm just talking about 
 calling
 a function.



 but I dont' get how or why. It's exactly D's solution to the 
 problem.


 It doesn't solve the problem... it doesn't even address the 
 problem. You're talking about a totally different thing >_<


Auto ref allows the unnecessary copy to be avoided for lvalues 
and creates a temporary (as part of passing the value) for 
rvalues. It has downsides (virtual functions and extern(C++), but 
it does directly address the problem you're talking about, unless 
I have totally misunderstood you.

Here is a small proof of concept I made to demonstrate how easy 
it seems to be to use `auto ref` to call a C++ virtual const& 
function without incurring any more copies than would happen with 
the same calls from C++. I'm sure it could be improved a lot, but 
does the basic concept match what you would need?

// D source file:

/// mix this in to your extern(C++) class with a list of the 
functions where you
/// want to be able to pass rvalues to ref parameters.
auto rValueRefCalls(Funcs ...)()
{
     string ret;
     foreach (foo; Funcs)
         ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ 
`(Args ...)(auto ref Args args)
     if (__traits(compiles, (&this.` ~ __traits(identifier, foo) ~ 
`)(args)))
     {
         (&this.foo)(args);
     }`;

     return ret;
}

extern(C++)
{
     class A
     {
         void foo(const ref int v);
         mixin(rValueRefCalls!foo);
     }

     A makeA();
}

void main()
{
     int x = 3;
     auto a = makeA();
     a.foo(x);
     a.foo(3);
}


// C++ source file:

#include<cstdio>

class A
{
public:
     virtual void foo(const int& v)
     {
         printf("%d\n", v);
     }
};

A *makeA()
{
     return new A;
}
Mar 24 2018
next sibling parent reply Rubn <where is.this> writes:
On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:

 Auto ref allows the unnecessary copy to be avoided for lvalues 
 and creates a temporary (as part of passing the value) for 
 rvalues. It has downsides (virtual functions and extern(C++), 
 but it does directly address the problem you're talking about, 
 unless I have totally misunderstood you.


You are trading syntax bloat for binary bloat. Having 4 
parameters with auto ref can generate 16 different variants for 
the exact same function. If you are doing any sort of 
mathematical calculations you could cause a potential cache miss 
for calling the same exact function because one of your 
parameters didn't end up being the same type of value. You are 
causing all this bloat and potential slowdowns all to avoid 
having to do this:

float value0 = a + b;
float value1 = c + d;
float value2 = e + f;
someFunc(value0, value1, value2);

That's an inherent flaw in design. You obviously agree that there 
is a problem, but how can you justify "auto ref" being a "good" 
solution to the problem? It's a horrible one, it causes excessive 
bloat and potential slowdowns just to get a cleaner readable 
syntax. That shouldn't be a solution that is deemed acceptable.
Mar 24 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote:

 On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:

 Auto ref allows the unnecessary copy to be avoided for lvalues
 and creates a temporary (as part of passing the value) for
 rvalues. It has downsides (virtual functions and extern(C++),
 but it does directly address the problem you're talking about,
 unless I have totally misunderstood you.


 You are trading syntax bloat for binary bloat. Having 4
 parameters with auto ref can generate 16 different variants for
 the exact same function. If you are doing any sort of
 mathematical calculations you could cause a potential cache miss
 for calling the same exact function because one of your
 parameters didn't end up being the same type of value. You are
 causing all this bloat and potential slowdowns all to avoid
 having to do this:

 float value0 = a + b;
 float value1 = c + d;
 float value2 = e + f;
 someFunc(value0, value1, value2);

 That's an inherent flaw in design. You obviously agree that there
 is a problem, but how can you justify "auto ref" being a "good"
 solution to the problem? It's a horrible one, it causes excessive
 bloat and potential slowdowns just to get a cleaner readable
 syntax. That shouldn't be a solution that is deemed acceptable.


How good or bad it is depends on what you're doing and how many auto ref
parameters there are in your code. If you're using it occasionally, it's not
a problem at all, whereas if you're using it all over the place, you do get
a lot of template bloat.

Regardless, John's point was that auto ref solves the problem of being able
to call a function with both lvalues and rvalues without copying lvalues,
and he didn't understand why anyone was trying to argue that it doesn't do
that. And I agree with him on that point. It does not help with virtual
functions or extern(C++), and it creates a lot of template bloat if it's
used heavily, so there are downsides to it, but it _does_ solve the basic
problem of being able to call a function with both lvalues and rvalues
without copying the lvalues. It just doesn't solve it in a way that everyone
considers acceptable.

auto ref also helps with forwarding refness, so it's useful for more than
just avoiding copying lvalues, but the entire reason that auto ref was
originally added to the language was to solve this exact problem. And maybe
we need a different solution for some use cases (like virtual functions or
cases where the template bloat is deemed unacceptable), but auto ref is in
the language to solve this problem. So, much as at may make sense to argue
that it's not a good solution to this problem, it really doesn't make sense
to argue that it has nothing to do with it.

- Jonathan M Davis
Mar 24 2018
parent Rubn <where is.this> writes:
On Sunday, 25 March 2018 at 01:43:43 UTC, Jonathan M Davis wrote:

 On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote:

 On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:

 Auto ref allows the unnecessary copy to be avoided for 
 lvalues and creates a temporary (as part of passing the 
 value) for rvalues. It has downsides (virtual functions and 
 extern(C++), but it does directly address the problem you're 
 talking about, unless I have totally misunderstood you.


 You are trading syntax bloat for binary bloat. Having 4 
 parameters with auto ref can generate 16 different variants 
 for the exact same function. If you are doing any sort of 
 mathematical calculations you could cause a potential cache 
 miss for calling the same exact function because one of your 
 parameters didn't end up being the same type of value. You are 
 causing all this bloat and potential slowdowns all to avoid 
 having to do this:

 float value0 = a + b;
 float value1 = c + d;
 float value2 = e + f;
 someFunc(value0, value1, value2);

 That's an inherent flaw in design. You obviously agree that 
 there is a problem, but how can you justify "auto ref" being a 
 "good" solution to the problem? It's a horrible one, it causes 
 excessive bloat and potential slowdowns just to get a cleaner 
 readable syntax. That shouldn't be a solution that is deemed 
 acceptable.


 How good or bad it is depends on what you're doing and how many 
 auto ref parameters there are in your code. If you're using it 
 occasionally, it's not a problem at all, whereas if you're 
 using it all over the place, you do get a lot of template bloat.

 Regardless, John's point was that auto ref solves the problem 
 of being able to call a function with both lvalues and rvalues 
 without copying lvalues, and he didn't understand why anyone 
 was trying to argue that it doesn't do that. And I agree with 
 him on that point. It does not help with virtual functions or 
 extern(C++), and it creates a lot of template bloat if it's 
 used heavily, so there are downsides to it, but it _does_ solve 
 the basic problem of being able to call a function with both 
 lvalues and rvalues without copying the lvalues. It just 
 doesn't solve it in a way that everyone considers acceptable.


That's a horrible way to look at it. You can still technically 
drive a car with square wheels, it may not be a very comfortable 
or efficient ride, but it still __does__ solve the basic problem 
of moving the car. Because it works we shouldn't look at any 
other solution.


 auto ref also helps with forwarding refness, so it's useful for 
 more than just avoiding copying lvalues, but the entire reason 
 that auto ref was originally added to the language was to solve 
 this exact problem. And maybe we need a different solution for 
 some use cases (like virtual functions or cases where the 
 template bloat is deemed unacceptable), but auto ref is in the 
 language to solve this problem. So, much as at may make sense 
 to argue that it's not a good solution to this problem, it 
 really doesn't make sense to argue that it has nothing to do 
 with it.


I never said it has nothing to do with it. It's just a really 
horrible solution.

If you are going to add an exception just for two minor cases, at 
that point you might as well start looking at adding rvalue 
references, cause that's what it is going to lead to. It'd 
provided better integration with C++ and provide an 
__acceptable__ solution to the problem (not one that solves the 
problem in a roundabout square-wheel-like way) that doesn't 
create runtime bloat and slowdown just to have clean readable 
syntax.
Mar 24 2018
prev sibling parent Rubn <where is.this> writes:
On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:

 Here is a small proof of concept I made to demonstrate how easy 
 it seems to be to use `auto ref` to call a C++ virtual const& 
 function without incurring any more copies than would happen 
 with the same calls from C++. I'm sure it could be improved a 
 lot, but does the basic concept match what you would need?

 // D source file:

 /// mix this in to your extern(C++) class with a list of the 
 functions where you
 /// want to be able to pass rvalues to ref parameters.
 auto rValueRefCalls(Funcs ...)()
 {
     string ret;
     foreach (foo; Funcs)
         ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ 
 `(Args ...)(auto ref Args args)
     if (__traits(compiles, (&this.` ~ __traits(identifier, foo) 
 ~ `)(args)))
     {
         (&this.foo)(args);
     }`;

     return ret;
 }

 extern(C++)
 {
     class A
     {
         void foo(const ref int v);
         mixin(rValueRefCalls!foo);
     }

     A makeA();
 }

 void main()
 {
     int x = 3;
     auto a = makeA();
     a.foo(x);
     a.foo(3);
 }


 // C++ source file:

 #include<cstdio>

 class A
 {
 public:
     virtual void foo(const int& v)
     {
         printf("%d\n", v);
     }
 };

 A *makeA()
 {
     return new A;
 }


That isn't going to scale to the number of potential functions 
that are going to need this. It's going to cause slow compile 
speeds and create a bloated binary.
Mar 24 2018
prev sibling parent Rubn <where is.this> writes:
On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote:

 I understand what you want, but I'm struggling to understand 
 why it's such a huge deal.

 The reason you want to pass by reference is for performance, to 
 avoid copying the data at the call boundary.


It's pretty simple:

float foo() { ... }
ref float bar() { ... }

void someFunc(ref float);

someFunc(bar()); // ok

float temp = foo();
someFunc(temp); // Have to create a temporary anyways for the 
function to work
someFunc(foo()); // Compile error, need to use the hideous code 
above


So there really isn't any performance penalty cause if you want 
to call that function you are going to have to create a temporary 
variable anyways, but now you can't just call the function in one 
line. It requires multiple lines and a temporary variable name.

This becomes especially horrible for math libraries:

void someFunc(ref Vector3) { ... }

someFunc(a + b); // Can't do this

Vector3 temp = a + b;
someFunc(temp); // Have to do this

So there's no performance penalty for what he is requesting, but 
allows for a cleaner syntax to do the exact same thing.


 So there are 2 cases: an lvalue needs to be passed, or an 
 rvalue needs to be passed.

 1. The address of the lvalue is passed.

 2. The rvalue is copied to a local, then the address of that 
 local is passed.

 So in the rvalue case, you're not getting the performance 
 benefit of passing by reference, because you have to copy to a 
 local anyway.

 What I would do in D currently to get the same performance and 
 API:

 void foo(float[32] v) { foo(v); }
 void foo(ref float[32] v) { ... }

 or

 void foo()(auto ref float[32] v) { ... }

 What is so totally unacceptable about those solutions? I 
 personally like the second because it scales better to multiple 
 parameters. I know you have said it's not relevant and annoying 
 that people bring up auto ref, but I dont' get how or why. It's 
 exactly D's solution to the problem.


It doesn't scale better, that's part of the problem:

void foo()(auto ref MyType1, auto ref MyType2, auto ref MyType3, 
auto ref MyType4) { ... }

The above has the possibility of generating 16 different 
functions that basically all do the exact same thing. It creates 
excessive bloat, and now what if you want to take the address of 
the function? You can't cause you have to choose between one of 
the 16 variants.

This is template bloat at it's finest, except it isn't even doing 
anything useful. I can only imagine telling someone they have to 
code gen 16 identical functions just to be able to call a 
function without having to create useless temporary variables in 
the scope a function is being called in.
Mar 24 2018
prev sibling next sibling parent Dgame <r.schuett.1987 gmail.com> writes:
Here is what I've used if I had to: 
https://p0nce.github.io/d-idioms/#Rvalue-references:-Understanding-auto-ref-and-then-not-using-it
Mar 24 2018
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...
 <snip>


There are too many replies on this thread, addressing all the 
comments would take forever and pollute the thread itself. So 
forgive me if I say something that was covered already by someone 
else.

AFAIK being able to bind rvalues to `ref const(T)`, only makes 
sense when calling C++ functions that take `const T&` (especially 
since that is common). I have not yet heard any other use for 
them. I'd be in favour of allowing it _only_ for `extern(C++)` 
functions. Otherwise use `auto ref` or have overloads for 
pass-by-value and pass-by-ref.
I too, once a recent immigrant from the lands of C++, used to 
keep writing `ref const(T)`. I just pass by value now.

C++ T&& (forwarding reference) -> D auto ref T
C++ T&& (Rvalue reference) -> D T
C++ const T& -> D T
C++ T& -> D ref T

If replacing const T& with T chafes, I understand. I used to feel 
that way too. It's _possible_ that would incur a penalty in 
copying/moving, but IME the cost is either 0, negligible, or 
negative (!).

As mentioned above, if calling C++ code there's no choice about 
using T instead of const T&, so for pragmatic reasons that should 
be allowed. But only there, because...


 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?


Rvalue references. They exist because of being able to bind 
temporaries to const T& in C++, which means there's no way of 
knowing if your const T& was originally a temporary or not. To 
disambiguate C++11 introduced the type system horror that are 
rvalue references (which also do double-duty in enabling perfect 
forwarding!).

D doesn't have or need rvalue references _because_ of not 
allowing temporaries to bind to ref const(T). You get move 
semantics in D without the pain. That's a win in my book.

Atila
Mar 26 2018
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...
 <snip>


 There are too many replies on this thread, addressing all the 
 comments would take forever and pollute the thread itself. So 
 forgive me if I say something that was covered already by 
 someone else.

 AFAIK being able to bind rvalues to `ref const(T)`, only makes 
 sense when calling C++ functions that take `const T&` 
 (especially since that is common). I have not yet heard any 
 other use for them. I'd be in favour of allowing it _only_ for 
 `extern(C++)` functions. Otherwise use `auto ref` or have 
 overloads for pass-by-value and pass-by-ref.
 I too, once a recent immigrant from the lands of C++, used to 
 keep writing `ref const(T)`. I just pass by value now.

 C++ T&& (forwarding reference) -> D auto ref T
 C++ T&& (Rvalue reference) -> D T
 C++ const T& -> D T
 C++ T& -> D ref T

 If replacing const T& with T chafes, I understand. I used to 
 feel that way too. It's _possible_ that would incur a penalty 
 in copying/moving, but IME the cost is either 0, negligible, or 
 negative (!).


I'm tearing my remaining stubs of hair out trying to understand 
why memory copies (not talking about copy constructors) are 
needed when passing an rvalue to a non-ref function: 
https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy
Mar 26 2018
parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 11:13, John Colvin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...
 <snip>



 There are too many replies on this thread, addressing all the comments
 would take forever and pollute the thread itself. So forgive me if I say
 something that was covered already by someone else.

 AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when
 calling C++ functions that take `const T&` (especially since that is
 common). I have not yet heard any other use for them. I'd be in favour of
 allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or
 have overloads for pass-by-value and pass-by-ref.
 I too, once a recent immigrant from the lands of C++, used to keep writing
 `ref const(T)`. I just pass by value now.

 C++ T&& (forwarding reference) -> D auto ref T
 C++ T&& (Rvalue reference) -> D T
 C++ const T& -> D T
 C++ T& -> D ref T

 If replacing const T& with T chafes, I understand. I used to feel that way
 too. It's _possible_ that would incur a penalty in copying/moving, but IME
 the cost is either 0, negligible, or negative (!).



 I'm tearing my remaining stubs of hair out trying to understand why memory
 copies (not talking about copy constructors) are needed when passing an
 rvalue to a non-ref function:
 https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy


Passing rvalues to non-ref functions may elide a memory copy. Moves
can be very efficient.
But we're talking about ref functions right? Not not-ref functions...?
Mar 26 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...
 <snip>



 There are too many replies on this thread, addressing all the comments would
 take forever and pollute the thread itself. So forgive me if I say something
 that was covered already by someone else.

 AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when
 calling C++ functions that take `const T&` (especially since that is
 common). I have not yet heard any other use for them. I'd be in favour of
 allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or
 have overloads for pass-by-value and pass-by-ref.
 I too, once a recent immigrant from the lands of C++, used to keep writing
 `ref const(T)`. I just pass by value now.

 C++ T&& (forwarding reference) -> D auto ref T
 C++ T&& (Rvalue reference) -> D T
 C++ const T& -> D T


Yeah, no... T may be big. Copying a large thing sucks. Memory copying
is the slowest thing computers can do.
As an API author, exactly as in C++, you will make a judgement on a
case-by-case basis on this matter. It may be by-value, it may be by
const-ref. It depends on a bunch of things, and they are points for
consideration by the API author, not the user.


 C++ T& -> D ref T


I agree the other 3 cases are correct.
Mar 26 2018
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/26/2018 12:24 PM, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

 C++ const T& -> D T


 
 Yeah, no... T may be big. Copying a large thing sucks. Memory copying
 is the slowest thing computers can do.
 As an API author, exactly as in C++, you will make a judgement on a
 case-by-case basis on this matter. It may be by-value, it may be by
 const-ref. It depends on a bunch of things, and they are points for
 consideration by the API author, not the user.


Copying does suck, I agree. Consider the following:

     void foo(T t) { foo(t); } <= add this overload
     void foo(ref T t) { ... }
     T aaa();

     foo(aaa());

With inlining, I suspect we can get the compiler to not make any extra copies. 
It's not that different from NRVO. And as a marvy bonus, no weird semantic 
problems (as Atila mentioned).
Mar 26 2018
next sibling parent reply Rubn <where is.this> writes:
On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:

 On 3/26/2018 12:24 PM, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

 C++ const T& -> D T


 
 Yeah, no... T may be big. Copying a large thing sucks. Memory 
 copying
 is the slowest thing computers can do.
 As an API author, exactly as in C++, you will make a judgement 
 on a
 case-by-case basis on this matter. It may be by-value, it may 
 be by
 const-ref. It depends on a bunch of things, and they are 
 points for
 consideration by the API author, not the user.


 Copying does suck, I agree. Consider the following:

     void foo(T t) { foo(t); } <= add this overload
     void foo(ref T t) { ... }
     T aaa();

     foo(aaa());

 With inlining, I suspect we can get the compiler to not make 
 any extra copies. It's not that different from NRVO. And as a 
 marvy bonus, no weird semantic problems (as Atila mentioned).


How do you add this overload for the following?


void foo(ref T t) { ... }

void function(ref int) func = &foo;
int aaa();

func(aaa()); // err
Mar 26 2018
next sibling parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 16:21, Rubn via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:

 On 3/26/2018 12:24 PM, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

 C++ const T& -> D T



 Yeah, no... T may be big. Copying a large thing sucks. Memory copying
 is the slowest thing computers can do.
 As an API author, exactly as in C++, you will make a judgement on a
 case-by-case basis on this matter. It may be by-value, it may be by
 const-ref. It depends on a bunch of things, and they are points for
 consideration by the API author, not the user.



 Copying does suck, I agree. Consider the following:

     void foo(T t) { foo(t); } <= add this overload
     void foo(ref T t) { ... }
     T aaa();

     foo(aaa());

 With inlining, I suspect we can get the compiler to not make any extra
 copies. It's not that different from NRVO. And as a marvy bonus, no weird
 semantic problems (as Atila mentioned).



 How do you add this overload for the following?


 void foo(ref T t) { ... }

 void function(ref int) func = &foo;
 int aaa();

 func(aaa()); // err


Exactly.
Mar 26 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 19:25, Manu <turkeyman gmail.com> wrote:

 On 26 March 2018 at 16:21, Rubn via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:

 On 3/26/2018 12:24 PM, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

 C++ const T& -> D T



 Yeah, no... T may be big. Copying a large thing sucks. Memory copying
 is the slowest thing computers can do.
 As an API author, exactly as in C++, you will make a judgement on a
 case-by-case basis on this matter. It may be by-value, it may be by
 const-ref. It depends on a bunch of things, and they are points for
 consideration by the API author, not the user.



 Copying does suck, I agree. Consider the following:

     void foo(T t) { foo(t); } <= add this overload
     void foo(ref T t) { ... }
     T aaa();

     foo(aaa());

 With inlining, I suspect we can get the compiler to not make any extra
 copies. It's not that different from NRVO. And as a marvy bonus, no weird
 semantic problems (as Atila mentioned).



 How do you add this overload for the following?


 void foo(ref T t) { ... }

 void function(ref int) func = &foo;
 int aaa();

 func(aaa()); // err


 Exactly.


We're just kicking the can. And the only reason to do so is
ideological, as far as I can tell.
I want to hear an argument against... or any issue that's introduced
by allowing the implicit temporary?
Mar 26 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 15:48, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 3/26/2018 12:24 PM, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

 C++ const T& -> D T



 Yeah, no... T may be big. Copying a large thing sucks. Memory copying
 is the slowest thing computers can do.
 As an API author, exactly as in C++, you will make a judgement on a
 case-by-case basis on this matter. It may be by-value, it may be by
 const-ref. It depends on a bunch of things, and they are points for
 consideration by the API author, not the user.



 Copying does suck, I agree. Consider the following:

     void foo(T t) { foo(t); } <= add this overload
     void foo(ref T t) { ... }
     T aaa();

     foo(aaa());

 With inlining, I suspect we can get the compiler to not make any extra
 copies. It's not that different from NRVO. And as a marvy bonus, no weird
 semantic problems (as Atila mentioned).


It's a terrible experience to add 2^n overloads, just to abuse the
parameter list (as in your example) to create the temporary in an
implicit manner. The compiler can easily create the exact same
temporary without requiring a bunch of overloads to be declared.
What's the advantage of requiring that effort from the user, rather
than just doing it at the call site?
There's side effects to that hack too. Now there are overloads; so
taking function pointers becomes awkward (might be important in some
cases). If the interface has a binary boundary (static lib/dll) then
what does that look like with respect to inlining? Where do the
overloads go if the function is virtual?
I already know the answers to these questions, but the point is,
there's a whole lot more baggage introduced into the scene that just
doesn't need to be there.

So, while I agree that's an existing workaround, it kinda misses the
point. This thread isn't about inlining, it's about NOT inlining. You
don't use ref args unless you have a reason to, and that reason is
likely to have friction with that particular work-around.
Mar 26 2018
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...
 <snip>



 C++ T&& (forwarding reference) -> D auto ref T
 C++ T&& (Rvalue reference) -> D T
 C++ const T& -> D T


 Yeah, no... T may be big. Copying a large thing sucks. Memory 
 copying
 is the slowest thing computers can do.


That's _if_ T is big and _if_ it even gets copied, the 
combination of which I think happens very rarely. When that 
happens I think that the temporary isn't a big deal. This code:

struct Foo { int[1024] data; }
int byValue(Foo f) { return f.data[42]; }

Generates this assembly (ldc2 -O3, clang does the same for C++):

0000000000000000 <_D3foo7byValueFSQo3FooZi>:
    0:   8b 84 24 b0 00 00 00    mov    eax,DWORD PTR [rsp+0xb0]
    7:   c3                      ret

And I wrote a type for which a move and a copy are the same on 
purpose: in "real life" it's more likely that the memory will be 
dynamically allocated, probably held in a slice, and moved 
instead.

Are there cases in which there will be an expensive copy? Yes, 
then pass by ref/pointer. But measure first, and prefer to pass 
by value by default.

It's different in C++ - stick a std::vector or std::string in 
your struct and passing by value (usually) copies the dynamically 
allocated memory. In D it moves.


 As an API author, exactly as in C++, you will make a judgement 
 on a
 case-by-case basis on this matter. It may be by-value, it may 
 be by
 const-ref. It depends on a bunch of things, and they are points 
 for
 consideration by the API author, not the user.


You can still do that in D. There well may be a reason to pass by 
const ref. I'm just saying that there aren't that many, and in 
that in those rare cases a temporary is fine. Especially if the 
alternative are rvalue references.

Atila
Mar 27 2018
prev sibling parent reply Rubn <where is.this> writes:
On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

 C++ T&& (Rvalue reference) -> D T


Not really, in C++ it is an actual reference and you get to 
choose which function actually does the move. In D it just does 
the copy when passed to the function. So you can't do this in D.

void bar(T&& t)
{
     // actually move contents of T
}

void foo(T&& t)
{
     bar(std::forward<T>(t)); // Can't do this in D without making 
another actual copy cause it isn't a reference
}


 If replacing const T& with T chafes, I understand. I used to 
 feel that way too. It's _possible_ that would incur a penalty 
 in copying/moving, but IME the cost is either 0, negligible, or 
 negative (!).

 As mentioned above, if calling C++ code there's no choice about 
 using T instead of const T&, so for pragmatic reasons that 
 should be allowed. But only there, because...


 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?


 Rvalue references. They exist because of being able to bind 
 temporaries to const T& in C++, which means there's no way of 
 knowing if your const T& was originally a temporary or not. To 
 disambiguate C++11 introduced the type system horror that are 
 rvalue references (which also do double-duty in enabling 
 perfect forwarding!).


What's a concrete example that you would be required to know 
whether a const& is a temporary or not. I don't really see it 
otherwise. The solution everyone is saying to use as an 
alternative would be the literal case of how it would be 
implemented in the language.


 D doesn't have or need rvalue references _because_ of not 
 allowing temporaries to bind to ref const(T). You get move 
 semantics in D without the pain. That's a win in my book.

 Atila


I've come across a few pains of such. It make be easier to use 
but it comes at a performance hit. In part binaries become huge 
because of how "init" is implemented.

struct StaticArray(T, size_t capacity)
{
     size_t length;
     T[capacity] values;
}

Copying the above structure copies unnecessary data for any 
move/copy operation. Eg when length = 0, it'll still copy 
everything. This includes initialization.
Mar 26 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 26 March 2018 at 17:30, Rubn via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

 C++ T&& (Rvalue reference) -> D T



 Not really, in C++ it is an actual reference and you get to choose which
 function actually does the move. In D it just does the copy when passed to
 the function. So you can't do this in D.

 void bar(T&& t)
 {
     // actually move contents of T
 }

 void foo(T&& t)
 {
     bar(std::forward<T>(t)); // Can't do this in D without making another
 actual copy cause it isn't a reference
 }


You can still get there with D, if you assume the forwarding function
is simple enough to inline (and the code is public).
I was cautiously concerned about this for a long time, but I've never
made a noise about it. I've digested and resolved that's okay though
:)
There are very few cases where you will fail to achieve equivalent
efficiency, and I think D's amazingly simplified move semantics more
than compensate for any conceivable loss.



 If replacing const T& with T chafes, I understand. I used to feel that way
 too. It's _possible_ that would incur a penalty in copying/moving, but IME
 the cost is either 0, negligible, or negative (!).

 As mentioned above, if calling C++ code there's no choice about using T
 instead of const T&, so for pragmatic reasons that should be allowed. But
 only there, because...


 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?



 Rvalue references. They exist because of being able to bind temporaries to
 const T& in C++, which means there's no way of knowing if your const T& was
 originally a temporary or not. To disambiguate C++11 introduced the type
 system horror that are rvalue references (which also do double-duty in
 enabling perfect forwarding!).



 What's a concrete example that you would be required to know whether a
 const& is a temporary or not. I don't really see it otherwise. The solution
 everyone is saying to use as an alternative would be the literal case of how
 it would be implemented in the language.


He's trying to say that C++ introduced rvalue references because
normal references weren't able to allow for move semantics to exist.
It's a red-herring. D already has move semantics, they work well, and
they're not on trial here.

In C++'s case, it's not that references were deficient at being
references that C++ needed rval-references, it's that references were
deficient at being move-able.
That is not a problem that exists in D. It's fine for references to
just be references in D. We're not struggling to make references
move-able in D, that's not a thing, we already have move semantics.
Any extension of this conversation about references into C++
rvalue-references (T&&) and or move-semantics are red-herrings.
There's no such problem in D that needs to be resolved, and the
existing solution is excellent.
Mar 26 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 27 March 2018 at 02:41:12 UTC, Manu wrote:


 He's trying to say that C++ introduced rvalue references 
 because normal references weren't able to allow for move 
 semantics to exist. It's a red-herring. D already has move 
 semantics, they work well, and they're not on trial here.

 In C++'s case, it's not that references were deficient at being
 references that C++ needed rval-references, it's that 
 references were
 deficient at being move-able.


There were deficient at being moveable because temporaries can 
bind to const T&.


 That is not a problem that exists in D.


Because temporaries can't bind to ref const(T).


 It's fine for references to
 just be references in D. We're not struggling to make references
 move-able in D, that's not a thing, we already have move 
 semantics.
 Any extension of this conversation about references into C++
 rvalue-references (T&&) and or move-semantics are red-herrings.
 There's no such problem in D that needs to be resolved, and the
 existing solution is excellent.


If I'm reading you correctly (which I might not), you seem to be 
saying that there's a way forward in which:

1) D's move semantics aren't affected
2) No rvalue references are introduced
3) Temporaries can bind to ref const(T)

I'd love to know what that would look like.

Atila
Mar 27 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Forked from the x^^y thread...
 <snip>




 C++ T&& (forwarding reference) -> D auto ref T
 C++ T&& (Rvalue reference) -> D T
 C++ const T& -> D T



 Yeah, no... T may be big. Copying a large thing sucks. Memory copying
 is the slowest thing computers can do.



 That's _if_ T is big and _if_ it even gets copied,


You've just described the exact anatomy of a ref function!
You wouldn't write a function to receive T by ref UNLESS T was both
big, and the function probably won't inline (therefore definitely
copy), and that condition will be triggered by any of the list of
reasons I've said a bunch of times (extern, dll, lib, virtual, etc).
People don't just love writing ref (well, some people might), but they
use it deliberately, typically in user-facing boundary API's for these
exact reasons.



 the combination of which
 I think happens very rarely. When that happens I think that the temporary
 isn't a big deal. This code:

 struct Foo { int[1024] data; }
 int byValue(Foo f) { return f.data[42]; }

 Generates this assembly (ldc2 -O3, clang does the same for C++):

 0000000000000000 <_D3foo7byValueFSQo3FooZi>:
    0:   8b 84 24 b0 00 00 00    mov    eax,DWORD PTR [rsp+0xb0]
    7:   c3                      ret

 And I wrote a type for which a move and a copy are the same on purpose: in
 "real life" it's more likely that the memory will be dynamically allocated,
 probably held in a slice, and moved instead.

 Are there cases in which there will be an expensive copy? Yes, then pass by
 ref/pointer. But measure first, and prefer to pass by value by default.


Right. I'm talking about deliberate use of ref... Or *existing*
(deliberate) use of ref, as is the case in almost all my my cases. The
code already exists.
As you assess, use of ref in D is fairly rare. I've been looking for
cases where other people are inconvenienced by this... hard to find.
I suspect there are reasons for this. One of them is that this
inconvenience suppresses it; ie, you will choose not to use a ref even
when you might prefer to. Others include the fact that extern(C++)
isn't super popular, DLL's aren't popular, closed-source distributed
code is non popular, OOP is not popular, etc.



 It's different in C++ - stick a std::vector or std::string in your struct
 and passing by value (usually) copies the dynamically allocated memory. In D
 it moves.


Only if you ARE moving, and not copying. D must deep copy too if you
actually copy.
Your example assumes C++ doesn't have a move constructor. D has
implicit move semantics, so you can only make an equivalent comparison
where C++ also defines the move constructor so the move case doesn't
pollute the ref comparison.
Also, irrespective of whether move semantics are performed (eliding
potential deep copying, as in your example), the binary memcpy still
has to be performed when handling values by-val, unless RVO (we're not
talking about return values), or inlining is able to eliminate it.
Also, we're not talking about move semantics!



 As an API author, exactly as in C++, you will make a judgement on a
 case-by-case basis on this matter. It may be by-value, it may be by
 const-ref. It depends on a bunch of things, and they are points for
 consideration by the API author, not the user.



 You can still do that in D. There well may be a reason to pass by const ref.
 I'm just saying that there aren't that many, and in that in those rare cases
 a temporary is fine. Especially if the alternative are rvalue references.


We're not talking about rvalue references, we're talking about normal
references >_<



 He's trying to say that C++ introduced rvalue references because normal
 references weren't able to allow for move semantics to exist. It's a
 red-herring. D already has move semantics, they work well, and they're not
 on trial here.

 In C++'s case, it's not that references were deficient at being
 references that C++ needed rval-references, it's that references were
 deficient at being move-able.



 There were deficient at being moveable because temporaries can bind to const
 T&.


... what? That's just not true at all.
If temporaries couldn't bind to C++ ref, then you *definitely*
wouldn't be able to move it, because you can guarantee that someone
else owns the reference. You can't move references, under any
circumstances, in either language... and that's actually the whole
point of references.

rvalue references were introduced in C++ to capture the calls with
rvalues into a separate function call, exactly the same way as the
by-value overload will catch the rvalues in D (and perform an implicit
move).
It was impossible for C++ to implement D's implicit move semantics
when receiving by-value for reasons that have nothing to do with
references; C++ allows interior pointers which breaks implicit moving,
C++ can overload default constructor which also breaks implicit moving
(no implicit way to reset the state of the prior owner).
That's the reason that C++ was forced to introduce rvalue references,
rather than implement implicit move logic like in D.

It was the choice of D to ban interior pointers, and dis-allow
overloading the default struct constructor which supports D's implicit
move semantics. If D didn't have those 2 things, it would need
rvalue-ref's the same as C++ for the same reasons. References have no
interaction with move semantics.

But again, we're not talking about move semantics here, we're just
talking about references ;)



 That is not a problem that exists in D.



 Because temporaries can't bind to ref const(T).


No. This truly has nothing to do with it.
rvalues would continue to choose the by-val function, performing a
move whenever possible. All existing rules are correct as they are.



 It's fine for references to
 just be references in D. We're not struggling to make references
 move-able in D, that's not a thing, we already have move semantics.
 Any extension of this conversation about references into C++
 rvalue-references (T&&) and or move-semantics are red-herrings.
 There's no such problem in D that needs to be resolved, and the
 existing solution is excellent.



 If I'm reading you correctly (which I might not), you seem to be saying that
 there's a way forward in which:

 1) D's move semantics aren't affected
 2) No rvalue references are introduced
 3) Temporaries can bind to ref const(T)

 I'd love to know what that would look like.


That's exactly what I've been saying. For like, 9 years..
It looks like this:
https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
 (contribution appreciated)

As far as I can tell, it's completely benign, it just eliminates the
annoying edge cases when interacting with functions that take
arguments by ref. There's no spill-over affect anywhere that I'm aware
of, and if you can find a single wart, I definitely want to know about
it.
I've asked so many times for a technical destruction, nobody will
present any opposition that is anything other than a rejection *in
principle*. This is a holy war, not a technical one. And as far as I
can tell, it basically only affects me, because I do so much work
against established C++ code! >_<
Mar 27 2018
next sibling parent Peter Campbell <peter spcampbell.co.uk> writes:
On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:

 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
  (contribution appreciated)


I've followed this thread since it was made as this has been one 
of the very few disappointments of the language for me. I only 
tend to write game code and use D for hobbyist projects whilst 
using C++ full-time as a junior at a small games company. Even if 
I take the attitude that I should use D as it is intended, 
instead of trying to write C++ by using D, it always felt 
unnecessarily obstructive to require me to make a temporary 
variable to avoid copying something simple like a vector or a 
matrix. It feels very restrictive when trying to express 
mathematical calculations in a concise manner.

Thanks for writing that DIP, you have covered everything I would 
love to see in great detail with good examples! I honestly 
couldn't think of anything more that could be added.
Mar 27 2018
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 27.03.2018 20:14, Manu wrote:

 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
   (contribution appreciated)
 
 As far as I can tell, it's completely benign, it just eliminates the
 annoying edge cases when interacting with functions that take
 arguments by ref. There's no spill-over affect anywhere that I'm aware
 of, and if you can find a single wart, I definitely want to know about
 it.


???


 I've asked so many times for a technical destruction, nobody will
 present any opposition that is anything other than a rejection *in
 principle*. This is a holy war, not a technical one.


That's extremely unfair. It is just a bad idea to overload D const for 
this purpose. Remove the "const" requirement and I'm on board.
Mar 28 2018
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 28.03.2018 13:34, Timon Gehr wrote:

 On 27.03.2018 20:14, Manu wrote:

 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md 

   (contribution appreciated)

 As far as I can tell, it's completely benign, it just eliminates the
 annoying edge cases when interacting with functions that take
 arguments by ref. There's no spill-over affect anywhere that I'm aware
 of, and if you can find a single wart, I definitely want to know about
 it.


 
 ???
  >> I've asked so many times for a technical destruction, nobody will

 present any opposition that is anything other than a rejection *in
 principle*. This is a holy war, not a technical one.


 
 That's extremely unfair. It is just a bad idea to overload D const for 
 this purpose. Remove the "const" requirement and I'm on board.


"The proposal could be amended to accept mutable ref's depending on the 
value-judgement balancing these 2 use cases. Sticking with const 
requires no such value judgement to be made at this time, and it's much 
easier to relax the spec in the future with emergence of evidence to do so."

Just get it right the first time. "const" is a serious API restriction, 
and it shouldn't be forced on anyone, even intermittently until they 
figure out that it is too restrictive (as well as viral).
Mar 28 2018
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/28/18 7:50 AM, Timon Gehr wrote:

 "The proposal could be amended to accept mutable ref's depending on the 
 value-judgement balancing these 2 use cases. Sticking with const 
 requires no such value judgement to be made at this time, and it's much 
 easier to relax the spec in the future with emergence of evidence to do 
 so."
 
 Just get it right the first time. "const" is a serious API restriction, 
 and it shouldn't be forced on anyone, even intermittently until they 
 figure out that it is too restrictive (as well as viral).


A great way to move things forward here, Timon, is to write a pull 
request against the DIP with motivating text and examples.
Apr 01 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01.04.2018 19:20, Andrei Alexandrescu wrote:

 On 3/28/18 7:50 AM, Timon Gehr wrote:

 "The proposal could be amended to accept mutable ref's depending on 
 the value-judgement balancing these 2 use cases. Sticking with const 
 requires no such value judgement to be made at this time, and it's 
 much easier to relax the spec in the future with emergence of evidence 
 to do so."

 Just get it right the first time. "const" is a serious API 
 restriction, and it shouldn't be forced on anyone, even intermittently 
 until they figure out that it is too restrictive (as well as viral).


 
 A great way to move things forward here, Timon, is to write a pull 
 request against the DIP with motivating text and examples.


I agree, but unfortunately I have many other things on my plate right 
now. Add to this that there are six or seven other DIPs that I really 
ought to finish/write/implement/rebase. I'll try to get back to this 
soon. Here, I was trying to make sure that popular misconceptions do not 
gain more traction.
Apr 01 2018
parent Manu <turkeyman gmail.com> writes:
On 1 April 2018 at 11:55, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 01.04.2018 19:20, Andrei Alexandrescu wrote:

 On 3/28/18 7:50 AM, Timon Gehr wrote:

 "The proposal could be amended to accept mutable ref's depending on the
 value-judgement balancing these 2 use cases. Sticking with const requires no
 such value judgement to be made at this time, and it's much easier to relax
 the spec in the future with emergence of evidence to do so."

 Just get it right the first time. "const" is a serious API restriction,
 and it shouldn't be forced on anyone, even intermittently until they figure
 out that it is too restrictive (as well as viral).



 A great way to move things forward here, Timon, is to write a pull request
 against the DIP with motivating text and examples.



 I agree, but unfortunately I have many other things on my plate right now.
 Add to this that there are six or seven other DIPs that I really ought to
 finish/write/implement/rebase. I'll try to get back to this soon. Here, I
 was trying to make sure that popular misconceptions do not gain more
 traction.


I'm convinced. There are lots of reasons it should not only apply to const.
I only originally went that way because it was more conservative,
easier to relax, and because the stated reason why you can't _already_
do this thing is allegedly because people freak out at the idea that a
function might return data into a temporary. 'const' prevents that
from being a possibility, but unlike C++, there are significant useful
cases for not restricting to const.
The most exciting for me is pipeline programming (ie, UFCS chains),
which are a major winning feature of D. They'll be more convenient
among other things.
Apr 01 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 28 Mar. 2018 4:35 am, "Timon Gehr via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

On 27.03.2018 20:14, Manu wrote:


 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx
 -rval_to_ref.md
   (contribution appreciated)

 As far as I can tell, it's completely benign, it just eliminates the
 annoying edge cases when interacting with functions that take
 arguments by ref. There's no spill-over affect anywhere that I'm aware
 of, and if you can find a single wart, I definitely want to know about
 it.


???


I've asked so many times for a technical destruction, nobody will

 present any opposition that is anything other than a rejection *in
 principle*. This is a holy war, not a technical one.


That's extremely unfair. It is just a bad idea to overload D const for this
purpose. Remove the "const" requirement and I'm on board.


I discussed that in that document. I'm happy to remove const, but it
requires a value judgement on the meaning of non-const in this case. It
becomes controversial without const, but I'm personally happy to remove it
if you can make The argument in favour. Can you give me some ideas where it
would be useful?
Mar 28 2018
parent reply Kagamin <spam here.lot> writes:
On Wednesday, 28 March 2018 at 16:15:53 UTC, Manu wrote:

 I discussed that in that document. I'm happy to remove const, 
 but it requires a value judgement on the meaning of non-const 
 in this case. It becomes controversial without const, but I'm 
 personally happy to remove it if you can make The argument in 
 favour. Can you give me some ideas where it would be useful?



doesn't make sense because the output would be immediately 
discarded; such a function call given an rvalue as argument 
likely represents an accidental mistake on the users part, and 
we can catch that invalid code.


Most obvious:

void Close(ref HANDLE h)
{
   CloseHandle(h);
   h=INVALID_HANDLE;
}

If you want to take input argument, then of course mark it as 
input, but if not then not. Why the callee would need to care if 
the argument is rvalue or not? The only criticism against rvalue 
references I saw was when the reference outlives the temporary.
Mar 30 2018
parent Manu <turkeyman gmail.com> writes:
On 30 March 2018 at 02:47, Kagamin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 The only criticism against rvalue references I saw was when the
 reference outlives the temporary.


That's the only *technical* criticism I've ever heard too.
Fortunately, D now has mechanisms for preventing that; D's  safety
story with respect to reference lifetime seems to be solid now.
'return ref', and enforcing ref pointers don't escape the callee
cleared to the road to allow accepting temporaries safety.
Mar 30 2018
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 27.03.2018 20:14, Manu wrote:

 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
   (contribution appreciated)


"Temporary destruction
Destruction of any temporaries occurs naturally at the end of the scope, 
as usual."

This is actually unusual.

---
import std.stdio;
struct S{
     ~this(){
         writeln("destroyed!");
     }
}

void foo(S s){}

void main(){
     foo(S());
     writeln("end the scope");
}
---

prints:

destroyed!
end the scope



"Overload resolution
...
This follows existing language rules. No change is proposed here."

Yes, you _are_ proposing a change. Right now rvalues "prefer" the 
by-value overload because the ref overload _does not match at all_. Now 
you make it match both, so you are adding additional disambiguation 
rules. You need to be more explicit about those.

Note that lvalues prefer the ref overload because the ref overload is 
more specialized. The new rule is the only instance where a less 
specialized overload is preferred.


You also need to specify the interactions with matching levels 
(https://dlang.org/spec/function.html#function-overloading):

E.g., your DIP is compatible with the following behavior:

---
import std.stdio;

struct S{}
void fun(S){ writeln("A"); }
void fun(ref const(S)){ writeln("B"); }

void main(){
     fun(S()); // calls A
     S s;
     fun(s); // calls A

     const(S) t;
     fun(t); // calls B
     fun(const(S)()); // calls B
}
---


The first example will cause friction when people try to add an explicit 
rvalue overload alongside a previous catch-it-all overload, the second 
example shows a breaking language change.

You cannot "fix" the first example without introducing breaking language 
changes. The above code compiles and runs in current D.

This just smells bad. Remove the "const" requirement.
Mar 28 2018
parent reply Manu <turkeyman gmail.com> writes:
On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 27.03.2018 20:14, Manu wrote:

 That's exactly what I've been saying. For like, 9 years..
 It looks like this:

 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
   (contribution appreciated)



 "Temporary destruction
 Destruction of any temporaries occurs naturally at the end of the scope, as
 usual."

 This is actually unusual.

 ---
 import std.stdio;
 struct S{
     ~this(){
         writeln("destroyed!");
     }
 }

 void foo(S s){}

 void main(){
     foo(S());
     writeln("end the scope");
 }
 ---

 prints:

 destroyed!
 end the scope


Right, exactly...
So, what's wrong?



 "Overload resolution
 ...
 This follows existing language rules. No change is proposed here."

 Yes, you _are_ proposing a change. Right now rvalues "prefer" the by-value
 overload because the ref overload _does not match at all_. Now you make it
 match both, so you are adding additional disambiguation rules. You need to
 be more explicit about those.


Oh right... yeah okay, I'll tweak the language.



 Note that lvalues prefer the ref overload because the ref overload is more
 specialized. The new rule is the only instance where a less specialized
 overload is preferred.


I've never heard any discussion involving the term 'specialised', or
seen any definition where overloading prefers a "more specialised'
version... is that a thing?
Given:
  void f(int);
  void f(const(int));
  f(10);

That calls the 'int' one, but it could call either one... that's
definitely not choosing a 'more specialised' match.



 You also need to specify the interactions with matching levels
 (https://dlang.org/spec/function.html#function-overloading):

 E.g., your DIP is compatible with the following behavior:

 ---
 import std.stdio;

 struct S{}
 void fun(S){ writeln("A"); }
 void fun(ref const(S)){ writeln("B"); }

 void main(){
     fun(S()); // calls A
     S s;
     fun(s); // calls A

     const(S) t;
     fun(t); // calls B
     fun(const(S)()); // calls B
 }
 ---

 The first example will cause friction when people try to add an explicit
 rvalue overload alongside a previous catch-it-all overload, the second
 example shows a breaking language change.

 You cannot "fix" the first example without introducing breaking language
 changes. The above code compiles and runs in current D.

 This just smells bad. Remove the "const" requirement.


This is very compelling reason to remove the const.
Mar 28 2018
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 28.03.2018 20:20, Manu wrote:

 On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 27.03.2018 20:14, Manu wrote:

 That's exactly what I've been saying. For like, 9 years..
 It looks like this:

 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
    (contribution appreciated)



 "Temporary destruction
 Destruction of any temporaries occurs naturally at the end of the scope, as
 usual."

 This is actually unusual.
  ...


 ...
 So, what's wrong?
 ...


In my example, the temporary is destroyed after/at the end of the 
function call, not at the end of the scope.

Re-reading the DIP, I think you meant the right thing, but the wording 
is a bit confusing. Maybe just clarify that "the scope" is the one your 
rewrite introduces implicitly, or explicitly state that the lifetime 
ends at the end of the function call.


 

 "Overload resolution
 ...


 

 Note that lvalues prefer the ref overload because the ref overload is more
 specialized. The new rule is the only instance where a less specialized
 overload is preferred.


 
 I've never heard any discussion involving the term 'specialised', or
 seen any definition where overloading prefers a "more specialised'
 version... is that a thing?
 Given:
    void f(int);
    void f(const(int));
    f(10);
 
 That calls the 'int' one, but it could call either one...


The overload resolution rules in D have four different matching levels:

- exact match
- match with type qualifier conversion
- match with general implicit conversion
- no match

The matching level for one overload is the minimal matching level for 
any argument. In your example f(int) matches exactly, but f(const(int)) 
matches with type qualifier conversion, therefore f(int) is chosen as it 
is the unique function that matches best.

Only if two overloads match with the same best level is specialization 
used. An overload A is more specialized than another overload B if we 
can call B with all arguments with which we can call A. As it is 
possible to call a by-value function with an lvalue or an rvalue, but 
ref cannot be called with an rvalue, ref is more specialized.


 that's definitely not choosing a 'more specialised' match.
 ...


Implicit conversions are ignored when checking for specialization so, 
yes, here both functions are equally specialized. However, f(int*) is 
more specialized than f(const(int)*):

---
import std.stdio;

void f(int* a){
     writeln("A");
}
void f(const(int)* b){
     writeln("B");
}

void main(){
     f(new immutable(int)); // guess what this prints. :)
}
---
Mar 28 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:

 On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:











 That's _if_ T is big and _if_ it even gets copied,


 You've just described the exact anatomy of a ref function!
 You wouldn't write a function to receive T by ref UNLESS T was 
 both
 big, and the function probably won't inline (therefore 
 definitely
 copy), and that condition will be triggered by any of the list 
 of
 reasons I've said a bunch of times (extern, dll, lib, virtual, 
 etc).
 People don't just love writing ref (well, some people might), 
 but they
 use it deliberately, typically in user-facing boundary API's 
 for these
 exact reasons.


I know. I was arguing that those cases are uncommon and the API 
pain is therefore not too big of an issue. I'm pretty sure that 
you feel it more because of what you write in D.


 Right. I'm talking about deliberate use of ref... Or *existing*
 (deliberate) use of ref, as is the case in almost all my my 
 cases. The
 code already exists.


Right, and I was assuming (perhaps incorrectly) that this 
existing code was C++, hence me being on board with binding 
rvalues to const ref there.


 Only if you ARE moving, and not copying. D must deep copy too 
 if you
 actually copy.
 Your example assumes C++ doesn't have a move constructor. D has
 implicit move semantics, so you can only make an equivalent 
 comparison
 where C++ also defines the move constructor so the move case 
 doesn't
 pollute the ref comparison.


I wasn't assuming the lack of a move constructor. What I was 
saying is that passing by value in C++ will usually mean a copy, 
whereas in D it usually means a move.


 Also, irrespective of whether move semantics are performed 
 (eliding
 potential deep copying, as in your example), the binary memcpy 
 still
 has to be performed when handling values by-val, unless RVO 
 (we're not
 talking about return values), or inlining is able to eliminate 
 it.


Good point about memcpy.


 In C++'s case, it's not that references were deficient at 
 being references that C++ needed rval-references, it's that 
 references were deficient at being move-able.



 There were deficient at being moveable because temporaries can 
 bind to const T&.


 ... what? That's just not true at all.
 If temporaries couldn't bind to C++ ref, then you *definitely*
 wouldn't be able to move it, because you can guarantee that 
 someone
 else owns the reference.


Precisely. That's how D works.



 rvalue references were introduced in C++ to capture the calls 
 with
 rvalues into a separate function call, exactly the same way as 
 the
 by-value overload will catch the rvalues in D (and perform an 
 implicit
 move).


Yes.


 It was impossible for C++ to implement D's implicit move 
 semantics
 when receiving by-value for reasons that have nothing to do with
 references; C++ allows interior pointers which breaks implicit 
 moving,
 C++ can overload default constructor which also breaks implicit 
 moving
 (no implicit way to reset the state of the prior owner).


I hadn't thought about the implications of interior pointers. 
Very good point.


 References have no interaction with move semantics.


Even given interior pointers, I disagree.


 But again, we're not talking about move semantics here, we're 
 just talking about references ;)


See comment above ;)


 I'd love to know what that would look like.


 That's exactly what I've been saying. For like, 9 years..
 It looks like this:
 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
  (contribution appreciated)


I was unaware of this (or I forgot). After reading it I'm not 
sure of what corner cases might arise, but if I'm getting it 
right I think it could work.


 And as far as I
 can tell, it basically only affects me, because I do so much 
 work
 against established C++ code! >_<


That's entirely possible. I can use my fingers to count the 
number of times I've written `extern(C++)`.

Atila



 Mar 30 2018



 next sibling parent  Manu <turkeyman gmail.com>  writes:


On 30 March 2018 at 02:06, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:

 On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:











 That's _if_ T is big and _if_ it even gets copied,



 You've just described the exact anatomy of a ref function!
 You wouldn't write a function to receive T by ref UNLESS T was both
 big, and the function probably won't inline (therefore definitely
 copy), and that condition will be triggered by any of the list of
 reasons I've said a bunch of times (extern, dll, lib, virtual, etc).
 People don't just love writing ref (well, some people might), but they
 use it deliberately, typically in user-facing boundary API's for these
 exact reasons.



 I know. I was arguing that those cases are uncommon and the API pain is
 therefore not too big of an issue. I'm pretty sure that you feel it more
 because of what you write in D.


Totally.
But I think it's uncommon at least in-part because of this
inconvenience. It's at least a little bit circular. Even you're saying
above "yeah, just do by-val, it's *probably* insignificant"... you
only say that because you've allowed this inconvenience to change your
behaviour. And I think that's to be expected.
Other reasons that it's not common:
extern(C++) is not common.
OOP (virtuals) in D are fairly uncommon. We don't OOP much in D.
Closed-source (binary lib) distribution is uncommon... possibly
unheard of? (yet!)
DLL's are still problematic onvarious fronts, and I don't think they
enjoy anywhere near as much use as they deserve.

Most of these aren't desirable... we'd like to see more commercial
(closed source?) users, DLL's should be more popular than they are...
and more extern(C++) means more people using a key development
investment in D.
So yeah, I agree it's relatively uncommon if you don't interact with
one of the niches where it tends to be common! :P



 Right. I'm talking about deliberate use of ref... Or *existing*
 (deliberate) use of ref, as is the case in almost all my my cases. The
 code already exists.



 Right, and I was assuming (perhaps incorrectly) that this existing code was
 C++, hence me being on board with binding rvalues to const ref there.


Right. But I'm not a fan is just rearranging the edge cases by
limiting it to a different set of cases and not applying it generally.
This also interacts with meta constructions, and while there are ANY
cases where calling doesn't just work as usual and it needs special
case handling, meta will still need static-if's to handle those cases.
We're just changing the criteria for the 'if'.
By allowing fully symmetric function calling rules, only then does the
noisy case-handling logic disappear from meta.



 Only if you ARE moving, and not copying. D must deep copy too if you
 actually copy.
 Your example assumes C++ doesn't have a move constructor. D has
 implicit move semantics, so you can only make an equivalent comparison
 where C++ also defines the move constructor so the move case doesn't
 pollute the ref comparison.


 I wasn't assuming the lack of a move constructor. What I was saying is that
 passing by value in C++ will usually mean a copy, whereas in D it usually
 means a move.


Right. But we're not talking about move's, we're talking about
NOT-move's (ie, preventing copies by passing by ref) ;)
Despite the appearance that we might be talking about rvalues, we're
actually talking about passing lvalues by ref... that's WHY you write
a function to accept its args by ref; to prevent deep copies when
passing lvalues.
The issue is, it's super-common to call functions with rvalues, and
the edge cases created by using ref are annoying and asymmetric, hence
such functions should receive rvalues too.



 In C++'s case, it's not that references were deficient at being
 references that C++ needed rval-references, it's that references were
 deficient at being move-able.


 There were deficient at being moveable because temporaries can bind to
 const T&.


 ... what? That's just not true at all.
 If temporaries couldn't bind to C++ ref, then you *definitely*
 wouldn't be able to move it, because you can guarantee that someone
 else owns the reference.


 Precisely. That's how D works.


Sorry, I'm lost now. I don't understand your initial point. You
created a relationship between ref's accepting rvalues, and the reason
that C++ introduced rvalue-references.
No such relationship exists... and I was trying to show that your
reasoning was inverted.



 rvalue references were introduced in C++ to capture the calls with
 rvalues into a separate function call, exactly the same way as the
 by-value overload will catch the rvalues in D (and perform an implicit
 move).


 Yes.


 References have no interaction with move semantics.


 Even given interior pointers, I disagree.


Please explain how there's any relationship between ref functions and
move semantics?

C++/D's mechanism for catching the move case is different, but that
still doesn't affect or interact with the ref case.
ref accepting rvalues has absolutely no effect on move semantics,
either theoretically in D, or practically in C++. It's a totally
separate problem space.



 But again, we're not talking about move semantics here, we're just talking
 about references ;)


 See comment above ;)


Likewise :P



 I'd love to know what that would look like.



 That's exactly what I've been saying. For like, 9 years..
 It looks like this:

 https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
  (contribution appreciated)


 I was unaware of this (or I forgot). After reading it I'm not sure of what
 corner cases might arise, but if I'm getting it right I think it could work.


Right. So does that admission dismiss your points above where you
suggest there's some reason that C++ ref functions accepting rvalues
lead to T&&?
As far as I can tell from your prior posts, your only resistance is
the idea of a 'slippery slope' that leads to T&& appearing in D? I
promise, I'm just as concerned that never happens as you :)
If you can find a connection, I want to know about it. But I'm
confident that no such thing exists..



 And as far as I
 can tell, it basically only affects me, because I do so much work
 against established C++ code! >_<



 That's entirely possible. I can use my fingers to count the number of times
 I've written `extern(C++)`.


Exactly.
I'd like to think that my set of use cases are not cases that we want
to discourage however. I'd like to see more users like myself in the
future...



 Mar 30 2018



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 30.03.2018 11:06, Atila Neves wrote:

 
 Right, and I was assuming (perhaps incorrectly) that this existing code 
 was C++, hence me being on board with binding rvalues to const ref there.


That must not happen. D const and C++ const don't even mean the same 
thing, and now suddenly you will see people use extern(C++) just to get 
the binding of rvalues to ref. Add to this the confusing overload 
behavior if const signifies anything other than read-only.



 Mar 31 2018



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 3/31/18 9:28 AM, Timon Gehr wrote:

 On 30.03.2018 11:06, Atila Neves wrote:

 Right, and I was assuming (perhaps incorrectly) that this existing 
 code was C++, hence me being on board with binding rvalues to const 
 ref there.


 
 That must not happen. D const and C++ const don't even mean the same 
 thing, and now suddenly you will see people use extern(C++) just to get 
 the binding of rvalues to ref. Add to this the confusing overload 
 behavior if const signifies anything other than read-only.


This is a very important aspect that must not be overlooked.

Allow me to reiterate: the one way to get this going is convert the 
forum dialog into a team and a DIP.


Andrei



 Mar 31 2018



prev sibling  parent  Rubn <where is.this>  writes:


On Tuesday, 27 March 2018 at 15:50:37 UTC, Atila Neves wrote:

 It's fine for references to
 just be references in D. We're not struggling to make 
 references
 move-able in D, that's not a thing, we already have move 
 semantics.
 Any extension of this conversation about references into C++
 rvalue-references (T&&) and or move-semantics are red-herrings.
 There's no such problem in D that needs to be resolved, and the
 existing solution is excellent.


 If I'm reading you correctly (which I might not), you seem to 
 be saying that there's a way forward in which:

 1) D's move semantics aren't affected
 2) No rvalue references are introduced
 3) Temporaries can bind to ref const(T)

 I'd love to know what that would look like.

 Atila



Well currently if you only have this implemented:

void foo(const ref Type);

Type temp = Type(10);
foo(temp);

Where the hell are you going to do your move semantics? You can't 
do it anyways currently, it's completely meaningless cause you 
can't.

void foo(Type);
void foo(const ref Type);

foo(Type(10));

Now we have move semantics with an additional definition.


With the proposed change, nothing about that would change. A 
temporary is only passed to a const ref as a "last resort". If it 
can do a move instead, it will do the move. The only change that 
is desired is to make the first sample code above have nicer 
syntax. That's it, like in the first example you don't care about 
it being a temporary or not.



 Mar 27 2018



prev sibling  parent reply Atila Neves <atila.neves gmail.com>  writes:


On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote:

 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

 C++ T&& (Rvalue reference) -> D T


 Not really, in C++ it is an actual reference and you get to 
 choose which function actually does the move. In D it just does 
 the copy when passed to the function.


It doesn't copy.


 So you can't do this in D.

 void bar(T&& t)
 {
     // actually move contents of T
 }

 void foo(T&& t)
 {
     bar(std::forward<T>(t)); // Can't do this in D without 
 making another actual copy cause it isn't a reference
 }



You can most definitely do this in D:

void bar(T)(auto ref T t) {
     // T is a ref for lvalues, by value for rvalues
}

void foo(T)(auto ref T t) {
     import std.functional: forward;
     bar(forward!t);
}


More to the point:

import std.stdio;

struct Foo {
     ubyte[] data;

     this(int n) {
         writeln("ctor n = ", n);
         data.length = n;
     }

     this(this) {
         writeln("postBlit n = ", data.length);
         data = data.dup;
     }
}

void foo(T)(auto ref T t) {
     import std.functional: forward;
     bar(forward!t);
}

void bar(T)(auto ref T t) {
     writeln("bar: ", t.data[0]);
}

void main() {
     bar(Foo(10));
     auto f = Foo(5);
     f.data[0] = 42;
     bar(f);
}


The output is:

ctor n = 10
bar: 0
ctor n = 5
bar: 42

Notice the lack of "postBlit" in the output. No copies were made. 
In D, by value *does not* mean copy. And given that, contrary to 
C++, the compiler doesn't write the postBlit constructor for you, 
you'd only ever get copies if you implemented it yourself!


 What's a concrete example that you would be required to know 
 whether a const& is a temporary or not.


To know whether or not you can move instead of copy. If it's a 
temporary, you can move. If it's not, you have to copy. Since 
temporaries bind to const T& in C++, you might have a temporary, 
or you might have an lvalue. Since you don't know, you have to 
copy. To support move semantics, C++ got T&&, which lvalues can't 
bind to. So if you have a T&&, you know it's about to go away and 
a move is possible.

In D, if it's ref then it can't be a temporary. If it's a value 
then it can, and it gets moved.


 I've come across a few pains of such. It make be easier to use 
 but it comes at a performance hit. In part binaries become huge 
 because of how "init" is implemented.

 struct StaticArray(T, size_t capacity)
 {
     size_t length;
     T[capacity] values;
 }

 Copying the above structure copies unnecessary data for any 
 move/copy operation. Eg when length = 0, it'll still copy 
 everything. This includes initialization.


This is that rare type for which moving is the same as copying. 
In that case (assuming it gets copied, see my reply to Manu), 
pass by ref. You won't be able to pass in temporaries, but I 
think that's a small price to pay for not having rvalue 
references.

In this case specifically, I don't know why you wouldn't just 
slice it when passing to functions.

Atila



 Mar 27 2018



  parent reply Rubn <where is.this>  writes:


On Tuesday, 27 March 2018 at 07:33:12 UTC, Atila Neves wrote:

 On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote:

 On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

 C++ T&& (Rvalue reference) -> D T


 Not really, in C++ it is an actual reference and you get to 
 choose which function actually does the move. In D it just 
 does the copy when passed to the function.


 It doesn't copy.


It copies the memory, so it does 2 memcpy's in the sense where as 
C++ only calls its move constructor once.


 So you can't do this in D.

 void bar(T&& t)
 {
     // actually move contents of T
 }

 void foo(T&& t)
 {
     bar(std::forward<T>(t)); // Can't do this in D without 
 making another actual copy cause it isn't a reference
 }



 You can most definitely do this in D:

 void bar(T)(auto ref T t) {
     // T is a ref for lvalues, by value for rvalues
 }

 void foo(T)(auto ref T t) {
     import std.functional: forward;
     bar(forward!t);
 }


 More to the point:

 import std.stdio;

 struct Foo {
     ubyte[] data;

     this(int n) {
         writeln("ctor n = ", n);
         data.length = n;
     }

     this(this) {
         writeln("postBlit n = ", data.length);
         data = data.dup;
     }
 }

 void foo(T)(auto ref T t) {
     import std.functional: forward;
     bar(forward!t);
 }

 void bar(T)(auto ref T t) {
     writeln("bar: ", t.data[0]);
 }

 void main() {
     bar(Foo(10));
     auto f = Foo(5);
     f.data[0] = 42;
     bar(f);
 }


 The output is:

 ctor n = 10
 bar: 0
 ctor n = 5
 bar: 42

 Notice the lack of "postBlit" in the output. No copies were 
 made. In D, by value *does not* mean copy. And given that, 
 contrary to C++, the compiler doesn't write the postBlit 
 constructor for you, you'd only ever get copies if you 
 implemented it yourself!


Well for starters your code is wrong. You are calling bar() 
instead of foo().

D has hidden implementation details, from your perspective it 
looks like it isn't doing any copying from the high-level 
viewpoint, but from the low-level viewpoint your object has been 
copied (moved memory) multiple times.

struct Foo {
     ubyte[1024] data;
}

void foo(T)(auto ref T t) {
     import std.functional: forward;
     bar(forward!t);
}

Foo gfoo;

void bar(T)(auto ref T t) {
     import std.algorithm.mutation : move;
     move(t, gfoo);
}

void main() {
     foo(Foo(10));
}

_D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
   push rbp
   mov rbp, rsp
   sub rsp, 3104
   lea rax, [rbp + 16]
   lea rdi, [rbp - 2048]
   lea rcx, [rbp - 1024]
   mov edx, 1024
   mov rsi, rcx
   mov qword ptr [rbp - 2056], rdi
   mov rdi, rsi
   mov rsi, rax
   mov qword ptr [rbp - 2064], rcx
   call memcpy PLT    <--------------------- hidden copy


Then the other copy is in move() to gfoo. That hidden copy will 
happen for every additional function call you try to pass Foo 
through.



 What's a concrete example that you would be required to know 
 whether a const& is a temporary or not.


 To know whether or not you can move instead of copy. If it's a 
 temporary, you can move. If it's not, you have to copy. Since 
 temporaries bind to const T& in C++, you might have a 
 temporary, or you might have an lvalue. Since you don't know, 
 you have to copy. To support move semantics, C++ got T&&, which 
 lvalues can't bind to. So if you have a T&&, you know it's 
 about to go away and a move is possible.

 In D, if it's ref then it can't be a temporary. If it's a value 
 then it can, and it gets moved.


D already has move semantics, an easy solution to this is to just 
use another keyword. It doesn't have to bind to const ref to get 
what is desired:

// what was suggested in the original DIP, since scope is being 
used for something else now
void foo( temp ref value)
{
}

Now you don't have this problem. You only get this behavior when 
you basically say you don't care whether it is a temporary or not.

So what's your problem with it now ?


 I've come across a few pains of such. It make be easier to use 
 but it comes at a performance hit. In part binaries become 
 huge because of how "init" is implemented.

 struct StaticArray(T, size_t capacity)
 {
     size_t length;
     T[capacity] values;
 }

 Copying the above structure copies unnecessary data for any 
 move/copy operation. Eg when length = 0, it'll still copy 
 everything. This includes initialization.


 This is that rare type for which moving is the same as copying. 
 In that case (assuming it gets copied, see my reply to Manu), 
 pass by ref. You won't be able to pass in temporaries, but I 
 think that's a small price to pay for not having rvalue 
 references.

 In this case specifically, I don't know why you wouldn't just 
 slice it when passing to functions.

 Atila


This wasn't an example for rvalue references. This was an example 
illustrating the negative results of the current "pain-free" 
system. I have a 100 mb binary, 90 mb of that come from a single 
structure. I mean sure you don't really have to worry about 
implementing move constructors and such but it is far from being 
pain free, that's what that was suppose to illustrate.



 Mar 27 2018



 next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Tue, Mar 27, 2018 at 08:25:36PM +0000, Rubn via Digitalmars-d wrote:
[...]

 _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
   push rbp
   mov rbp, rsp
   sub rsp, 3104
   lea rax, [rbp + 16]
   lea rdi, [rbp - 2048]
   lea rcx, [rbp - 1024]
   mov edx, 1024
   mov rsi, rcx
   mov qword ptr [rbp - 2056], rdi
   mov rdi, rsi
   mov rsi, rax
   mov qword ptr [rbp - 2064], rcx
   call memcpy PLT    <--------------------- hidden copy


[...]

Is this generated by dmd, or gdc/ldc?

Generally, when it comes to performance issues, I don't even bother
looking at dmd-generated code anymore.  If the extra copying is still
happening with gdc -O2 / ldc -O, then you have a point. Otherwise, it
doesn't really say very much.


T

-- 
People tell me that I'm skeptical, but I don't believe them.



 Mar 27 2018



  parent reply Rubn <where is.this>  writes:


On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote:

 On Tue, Mar 27, 2018 at 08:25:36PM +0000, Rubn via 
 Digitalmars-d wrote: [...]

 _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
   push rbp
   mov rbp, rsp
   sub rsp, 3104
   lea rax, [rbp + 16]
   lea rdi, [rbp - 2048]
   lea rcx, [rbp - 1024]
   mov edx, 1024
   mov rsi, rcx
   mov qword ptr [rbp - 2056], rdi
   mov rdi, rsi
   mov rsi, rax
   mov qword ptr [rbp - 2064], rcx
   call memcpy PLT    <--------------------- hidden copy


 [...]

 Is this generated by dmd, or gdc/ldc?

 Generally, when it comes to performance issues, I don't even 
 bother looking at dmd-generated code anymore.  If the extra 
 copying is still happening with gdc -O2 / ldc -O, then you have 
 a point. Otherwise, it doesn't really say very much.


 T


It happens with LDC too, not sure how it would be able to know to 
do any kind of optimization like that unless it was able to 
inline every single function called into one function and be able 
to do optimize it from there. I don't imagine that'll be likely 
though.



 Mar 27 2018



 next sibling parent  "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Tue, Mar 27, 2018 at 09:52:25PM +0000, Rubn via Digitalmars-d wrote:

 On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote:

 On Tue, Mar 27, 2018 at 08:25:36PM +0000, Rubn via Digitalmars-d wrote:
 [...]

 _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
   push rbp
   mov rbp, rsp
   sub rsp, 3104
   lea rax, [rbp + 16]
   lea rdi, [rbp - 2048]
   lea rcx, [rbp - 1024]
   mov edx, 1024
   mov rsi, rcx
   mov qword ptr [rbp - 2056], rdi
   mov rdi, rsi
   mov rsi, rax
   mov qword ptr [rbp - 2064], rcx
   call memcpy PLT    <--------------------- hidden copy


 [...]
 
 Is this generated by dmd, or gdc/ldc?
 
 Generally, when it comes to performance issues, I don't even bother
 looking at dmd-generated code anymore.  If the extra copying is
 still happening with gdc -O2 / ldc -O, then you have a point.
 Otherwise, it doesn't really say very much.
 
 
 T


 
 It happens with LDC too, not sure how it would be able to know to do
 any kind of optimization like that unless it was able to inline every
 single function called into one function and be able to do optimize it
 from there.  I don't imagine that'll be likely though.


You'll be surprised.  Don't underestimate the power of modern
optimizers.  I've seen LDC do inlining that's so aggressive, that it
essentially evaluated an entire series of function calls at compile-time
(likely on the IR) and generated a single instruction to load the answer
into the return register at runtime. :-D  Of course, it still generated
the individual functions, but those are never actually called at
runtime.

(On one occasion, this produced odd-looking "benchmark" results where the
ldc executable computed the answer in exactly 0ms, whereas everyone else
took a lot longer than that. :-D  (Well, it was probably a few nanosecs
while the CPU decoded and ran the instruction, but I don't think any
benchmark could measure that!))

For your code example, you might want to look at the code generated for
callers of the function, since when compiling individual functions in
isolation, LDC is obligated to follow the ABI, which could include
redundant copying. But if inlining was possible, it could generate very
different code.


T

-- 
Dogs have owners ... cats have staff. -- Krista Casada



 Mar 27 2018



prev sibling  parent reply kinke <noone nowhere.com>  writes:


On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:

 It happens with LDC too, not sure how it would be able to know 
 to do any kind of optimization like that unless it was able to 
 inline every single function called into one function and be 
 able to do optimize it from there. I don't imagine that'll be 
 likely though.


It does it in your code sample with `-O`, there's no call to bar 
and the foo() by-value arg is memcpy'd to the global.

If you compile everything with LTO, your code and all 3rd-party 
libs as well as druntime/Phobos, LLVM is able to optimize the 
whole program as if it were inside a single gigantic 'object' 
file in LLVM bitcode IR, and is thus indeed theoretically able to 
inline *all* functions.



 Mar 27 2018



 next sibling parent reply Rubn <where is.this>  writes:


On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote:

 On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:

 It happens with LDC too, not sure how it would be able to know 
 to do any kind of optimization like that unless it was able to 
 inline every single function called into one function and be 
 able to do optimize it from there. I don't imagine that'll be 
 likely though.


 It does it in your code sample with `-O`, there's no call to 
 bar and the foo() by-value arg is memcpy'd to the global.

 If you compile everything with LTO, your code and all 3rd-party 
 libs as well as druntime/Phobos, LLVM is able to optimize the 
 whole program as if it were inside a single gigantic 'object' 
 file in LLVM bitcode IR, and is thus indeed theoretically able 
 to inline *all* functions.


A bit off topic now but anyways:

Well that example I posted didn't do anything, so it would 
optimize it out quite easily. The entire function was excluded 
essentially. Just adding a few writeln it isn't able to remove 
the function entirely anymore and can't optimize it out. Idk if 
you want to try some different options but flto didn't do 
anything for it.

https://godbolt.org/g/bLdpnm

import std.stdio : writeln;

struct Foo {
     ubyte[1024] data;

     this(int a)
     {
         data[0] = cast(ubyte)a;
     }
}

void foo(T)(auto ref T t) {
     import std.functional: forward;
     writeln(gfoo.data[0]);
     bar(forward!t);
     writeln(gfoo.data[0]);
}

__gshared Foo gfoo;

void bar(T)(auto ref T t) {
     import std.algorithm.mutation : move;
     writeln(gfoo.data[0]);
     move(t, gfoo);
}

void main() {
     foo(Foo(10));
}



 Mar 27 2018



  parent reply kinke <noone nowhere.com>  writes:


On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote:

 Just adding a few writeln it isn't able to remove the function 
 entirely anymore and can't optimize it out.


Well writeln() here involves number -> string formatting, GC, 
I/O, template bloat... There are indeed superfluous memcpy's in 
your foo() there (although the forward and bar calls are still 
inlined), which after a quick glance seem to be LLVM optimizer 
shortcomings, the IR emitted by LDC looks fine.
For an abitrary external function, it's all fine as it should be, 
boiling down to a single memcpy in foo() and a direct memset in 
main(): https://run.dlang.io/is/O1aeLK



 Mar 27 2018



  parent  Rubn <where is.this>  writes:


On Wednesday, 28 March 2018 at 00:56:29 UTC, kinke wrote:

 On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote:

 Just adding a few writeln it isn't able to remove the function 
 entirely anymore and can't optimize it out.


 Well writeln() here involves number -> string formatting, GC, 
 I/O, template bloat... There are indeed superfluous memcpy's in 
 your foo() there (although the forward and bar calls are still 
 inlined), which after a quick glance seem to be LLVM optimizer 
 shortcomings, the IR emitted by LDC looks fine.
 For an abitrary external function, it's all fine as it should 
 be, boiling down to a single memcpy in foo() and a direct 
 memset in main(): https://run.dlang.io/is/O1aeLK


Well somethings wrong if writeln causes optimization to not 
occur, if that is the case then it'd be best to just use printf() 
instead. Anyways using small examples to show optimization is 
usually not what's going to happen in actual code. Functions are 
rarely that simple, and if adding a single writeln() to a call is 
enough to eliminate that optimization, I can only imagine what 
other little things do as well.



 Mar 27 2018



prev sibling  parent  kinke <noone nowhere.com>  writes:


On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote:

 On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:

 It happens with LDC too, not sure how it would be able to know 
 to do any kind of optimization like that unless it was able to 
 inline every single function called into one function and be 
 able to do optimize it from there. I don't imagine that'll be 
 likely though.


 It does it in your code sample with `-O`, there's no call to 
 bar and the foo() by-value arg is memcpy'd to the global.


For reference: https://run.dlang.io/is/2vDEXP
Note that main() boils down to a `memset(&gfoo, 10, 1024); return 
0;`:

_Dmain:
	.cfi_startproc
	pushq	%rax
.Lcfi0:
	.cfi_def_cfa_offset 16
	data16
	leaq	onlineapp.Foo onlineapp.gfoo TLSGD(%rip), %rdi
	data16
	data16
	rex64
	callq	__tls_get_addr PLT
	movl	$10, %esi
	movl	$1024, %edx
	movq	%rax, %rdi
	callq	memset PLT
	xorl	%eax, %eax
	popq	%rcx
	retq



 Mar 27 2018



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 27.03.2018 22:25, Rubn wrote:

 
 D already has move semantics, an easy solution to this is to just use 
 another keyword. It doesn't have to bind to const ref to get what is 
 desired:
 
 // what was suggested in the original DIP, since scope is being used for 
 something else now
 void foo( temp ref value)
 {
 }
 
 Now you don't have this problem. You only get this behavior when you 
 basically say you don't care whether it is a temporary or not.


Another benefit of this solution is that the overload resolution rules 
are obvious. foo( temp ref T value) is less specialized than both foo(T 
value) and foo(ref T value).

 Manu: Consider this.



 Mar 28 2018



  parent  Manu <turkeyman gmail.com>  writes:


On 28 March 2018 at 05:38, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 27.03.2018 22:25, Rubn wrote:

 D already has move semantics, an easy solution to this is to just use
 another keyword. It doesn't have to bind to const ref to get what is
 desired:

 // what was suggested in the original DIP, since scope is being used for
 something else now
 void foo( temp ref value)
 {
 }

 Now you don't have this problem. You only get this behavior when you
 basically say you don't care whether it is a temporary or not.


 Another benefit of this solution is that the overload resolution rules are
 obvious. foo( temp ref T value) is less specialized than both foo(T value)
 and foo(ref T value).

  Manu: Consider this.


This defeats the entire point to me. I want symmetrical calling code
in all cases... the current edges are a massive pain in the arse.
In the event of yet-another-attribute, then we just shift the set of
edge cases onto that attribute instead, and it makes no practical
difference in the end.



 Mar 28 2018



prev sibling next sibling parent  jmh530 <john.michael.hall gmail.com>  writes:


On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 Can you please explain these 'weirdities'?
 What are said "major unintended consequences"?
 Explain how the situation if implemented would be any different 
 than
 the workaround?

 This seems even simpler than the pow thing to me.
 Rewrite:
     func(f());
 as:
     { auto __t0 = f(); func(__t0); }


 How is that worse than the code you have to write:
     T temp = f();
     T zero = 0;
     func(temp, zero);


I feel like this example wasn't really concrete enough for me. I 
wrote a version below that I think made it a little clearer for 
myself.

-----

import std.stdio : writeln;

struct Foo
{
     int data;
}

int foo(Foo x)
{
     writeln("here");
     return x.data;
}

int foo(ref Foo x)
{
     writeln("there");
     return x.data;
}


void main()
{
     auto x = Foo(5);
     auto y = foo(x);
     writeln(y);

     auto z = foo(Foo(5));
     writeln(z);
}



 Mar 26 2018



prev sibling next sibling parent  Kagamin <spam here.lot>  writes:


On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

 By contrast, people will NOT forgive the fact that they have to 
 change:

     func(f(x), f(y), f(z));

 to:

     T temp = f(x);
     T temp2 = f(y);
     T temp3 = f(z);
     func(temp, temp2, temp3);

 That's just hideous and in-defensible.

 A better story would be:

     func(f(x), f(y), f(z));
 =>
     func(x.f, y.f, z.f);


Another workaround:

auto r(T)(T a)
{
     struct R { T val; }
     return R(a);
}
void f(in ref int p);
int main()
{
     f(1.r.val);
     return 0;
}



 Mar 28 2018



prev sibling next sibling parent reply Dgame <r.schuett.1987 gmail.com>  writes:


Just to be sure it does not got los: You know that you can avoid 
the temp/copy if you add one method to your struct, yes?

----
import std.stdio;

struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
}

void foo(ref const Big b) {
     writeln(b.name);
}

void main() {

     foo(b);

}
----

That works like a charm and avoids any need for rvalue 
references. Just add it as mixin template in Phobos or something 
like that.



 Mar 29 2018



  parent reply Rubn <where is.this>  writes:


On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:

 Just to be sure it does not got los: You know that you can 
 avoid the temp/copy if you add one method to your struct, yes?

 ----
 import std.stdio;

 struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
 }

 void foo(ref const Big b) {
     writeln(b.name);
 }

 void main() {

     foo(b);

 }
 ----

 That works like a charm and avoids any need for rvalue 
 references. Just add it as mixin template in Phobos or 
 something like that.


Doesn't work with built-in types like float.

Just adds bloat for operators like opBinary if you want that to 
be ref.

foo((a.byRef + b.byRef * c.byRef).byRef)

// vs

foo(a + b * c);

It's kind of funny all this talk about allowing temporaries to 
bind to refs being messy, yet you can already bind a temporary to 
a ref in a messy way using that.



 Mar 29 2018



  parent reply Dgame <r.schuett.1987 gmail.com>  writes:


On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote:

 On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:

 Just to be sure it does not got los: You know that you can 
 avoid the temp/copy if you add one method to your struct, yes?

 ----
 import std.stdio;

 struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
 }

 void foo(ref const Big b) {
     writeln(b.name);
 }

 void main() {

     foo(b);

 }
 ----

 That works like a charm and avoids any need for rvalue 
 references. Just add it as mixin template in Phobos or 
 something like that.


 Doesn't work with built-in types like float.


Why would you want to use a float as a rvalue reference?


 Just adds bloat for operators like opBinary if you want that to 
 be ref.

 foo((a.byRef + b.byRef * c.byRef).byRef)

 // vs

 foo(a + b * c);

 It's kind of funny all this talk about allowing temporaries to 
 bind to refs being messy, yet you can already bind a temporary 
 to a ref in a messy way using that.


Yeah, it is a bit messy. It is not perfect, but is does avoid any 
temp var!
Let's look at a Vector2f example with the following opBinary:

----
auto opBinary(string op)(ref const Vector2f v) {
     return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" 
~ op ~ "v.y"));
}

void foo(ref const Vector2f v) {
     writeln(v.x, ':', v.y);
}

foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef);
----

Since opBinary needs to be a template, you can combine my 
solution with auto ref:

----
auto opBinary(string op)(auto ref const Vector2f v) {
     return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" 
~ op ~ "v.y"));
}

void foo(ref const Vector2f v) {
     writeln(v.x, ':', v.y);
}

foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef);
----

That is cleaner. :)



 Mar 29 2018



  parent  Rubn <where is.this>  writes:


On Thursday, 29 March 2018 at 20:22:47 UTC, Dgame wrote:

 On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote:

 On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:

 Just to be sure it does not got los: You know that you can 
 avoid the temp/copy if you add one method to your struct, yes?

 ----
 import std.stdio;

 struct Big {
     string name;
     float[1000] values;

     this(string name) {
         this.name = name;
     }

      disable
     this(this);

     ref auto byRef() inout {
         return this;
     }
 }

 void foo(ref const Big b) {
     writeln(b.name);
 }

 void main() {

     foo(b);

 }
 ----

 That works like a charm and avoids any need for rvalue 
 references. Just add it as mixin template in Phobos or 
 something like that.


 Doesn't work with built-in types like float.


 Why would you want to use a float as a rvalue reference?


In templates to avoid template bloat with auto ref.


 Just adds bloat for operators like opBinary if you want that 
 to be ref.

 foo((a.byRef + b.byRef * c.byRef).byRef)

 // vs

 foo(a + b * c);

 It's kind of funny all this talk about allowing temporaries to 
 bind to refs being messy, yet you can already bind a temporary 
 to a ref in a messy way using that.


 Yeah, it is a bit messy. It is not perfect, but is does avoid 
 any temp var!
 Let's look at a Vector2f example with the following opBinary:

 ----
 auto opBinary(string op)(ref const Vector2f v) {
     return Vector2f(mixin("this.x" ~ op ~ "v.x"), 
 mixin("this.y" ~ op ~ "v.y"));
 }

 void foo(ref const Vector2f v) {
     writeln(v.x, ':', v.y);
 }

 foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef);
 ----

 Since opBinary needs to be a template, you can combine my 
 solution with auto ref:

 ----
 auto opBinary(string op)(auto ref const Vector2f v) {
     return Vector2f(mixin("this.x" ~ op ~ "v.x"), 
 mixin("this.y" ~ op ~ "v.y"));
 }

 void foo(ref const Vector2f v) {
     writeln(v.x, ':', v.y);
 }

 foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef);
 ----

 That is cleaner. :)


Just adding more template bloat and it is still messy, it's not 
like we are trying to find a work around to a problem. A solution 
already exists, the cleaner syntax you'd get with rvalue 
references (along with better compatibility with C++ and other 
benefits) can't be beat.



 Mar 29 2018



prev sibling next sibling parent  Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:

 On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:

 On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d

 wrote:

 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:

 My biggest concern in all of this is that I don't want to see ref
 start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants
 to
 avoid a copy.


 That ship sailed ages ago: It's already unclear. If we want to fix
 that,
 we can fix it, but blocking rvalue ref does nothing for that cause.


 Really? And how often does ref get used just to avoid copying? You can
 almost always look at a function, see that it accepts ref, and know that
 it's supposed to mutate the argument. Functions that want to avoid
 copying lvalues usually use auto ref, not ref, whereas if ref accepted
 rvalues, a number of folks would start using it all over the place to
 avoid copying. Right now, folks rarely use ref that way, because it
 becomes too annoying to call the function with an rvalue. So, while it
 might not be the case 100% of the time right now that ref is used with
 the purpose of mutating the argument, it almost always is. As such, you
 can pretty reliably look at a function signature and expect that if one
 of its parameters is ref, it's going to be mutating that argument. The
 function that accepts an argument by ref with no intention of mutating
 it is very much the exception, and I really don't want to see that
 change.


 Interesting. Just understand that you're trading that feeling for a
 suite of edge cases though. Accepting asymmetric calling rules is a
 pretty big cost to pay for that 'nice thought'.
 That idea also dismisses the existence of the set of cases where ref
 is genuinely useful/correct. Your sentiment effectively puts those use
 cases into the position of 2nd-class citizens.
 I understand your sentiment, but I think the cost is not balanced, or
 even particularly fair with respect to users in those niche groups :/


I'm not arguing against having a way to indicate that a parameter accepts
both rvalues and lvalues and that the rvalues get copied to an invisible
variable so that they can be passed as an lvalue. I'm arguing against simply
making ref have that behavior. Assuming that the details of how that worked
internally didn't involve problematic stuff like the rvalue reference stuff
that Andrei and Walter are so against, having an attribute such as  rvalue
to attach to a ref parameter to allow it to accept rvalues would be fine
with me. I just don't want ref by itself to lose its current semantics,
because that would dilute its meaning and increase its amiguity.

I want to be able to look at a function signature, see ref without other
qualifiers, and be reasonably certain that the function is supposed to be
mutating that argument, whereas if ref by itself accepted rvalues, then we
lose that. If an attribute were used to make it allow rvalues, then we
wouldn't.

- Jonathan M Davis



 Mar 30 2018



prev sibling next sibling parent  Manu <turkeyman gmail.com>  writes:


On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:

 On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:

 On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d

 wrote:

 On 03/23/2018 09:06 PM, Jonathan M Davis wrote:

 My biggest concern in all of this is that I don't want to see ref
 start
 accepting rvalues as has been occasionally discussed. It needs to be
 clear when a function is accept an argument by ref because it's going
 to mutate the object and when it's accepting by ref because it wants
 to
 avoid a copy.


 That ship sailed ages ago: It's already unclear. If we want to fix
 that,
 we can fix it, but blocking rvalue ref does nothing for that cause.


 Really? And how often does ref get used just to avoid copying? You can
 almost always look at a function, see that it accepts ref, and know that
 it's supposed to mutate the argument. Functions that want to avoid
 copying lvalues usually use auto ref, not ref, whereas if ref accepted
 rvalues, a number of folks would start using it all over the place to
 avoid copying. Right now, folks rarely use ref that way, because it
 becomes too annoying to call the function with an rvalue. So, while it
 might not be the case 100% of the time right now that ref is used with
 the purpose of mutating the argument, it almost always is. As such, you
 can pretty reliably look at a function signature and expect that if one
 of its parameters is ref, it's going to be mutating that argument. The
 function that accepts an argument by ref with no intention of mutating
 it is very much the exception, and I really don't want to see that
 change.


 Interesting. Just understand that you're trading that feeling for a
 suite of edge cases though. Accepting asymmetric calling rules is a
 pretty big cost to pay for that 'nice thought'.
 That idea also dismisses the existence of the set of cases where ref
 is genuinely useful/correct. Your sentiment effectively puts those use
 cases into the position of 2nd-class citizens.
 I understand your sentiment, but I think the cost is not balanced, or
 even particularly fair with respect to users in those niche groups :/


 I'm not arguing against having a way to indicate that a parameter accepts
 both rvalues and lvalues and that the rvalues get copied to an invisible
 variable so that they can be passed as an lvalue. I'm arguing against simply
 making ref have that behavior. Assuming that the details of how that worked
 internally didn't involve problematic stuff like the rvalue reference stuff
 that Andrei and Walter are so against, having an attribute such as  rvalue
 to attach to a ref parameter to allow it to accept rvalues would be fine
 with me. I just don't want ref by itself to lose its current semantics,
 because that would dilute its meaning and increase its amiguity.


https://imgflip.com/i/27gjv9



 I want to be able to look at a function signature, see ref without other
 qualifiers, and be reasonably certain that the function is supposed to be
 mutating that argument, whereas if ref by itself accepted rvalues, then we
 lose that. If an attribute were used to make it allow rvalues, then we
 wouldn't.


I don't know any reason why someone wouldn't attribute the argument
'const' if it doesn't intend to write to it. Likewise 'return ref' if
it's going to be modified and returned.
I'm not sure your concern is actually a thing...?



 Mar 30 2018



prev sibling next sibling parent  Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:

 On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d

 I want to be able to look at a function signature, see ref without other
 qualifiers, and be reasonably certain that the function is supposed to
 be
 mutating that argument, whereas if ref by itself accepted rvalues, then
 we lose that. If an attribute were used to make it allow rvalues, then
 we wouldn't.


 I don't know any reason why someone wouldn't attribute the argument
 'const' if it doesn't intend to write to it. Likewise 'return ref' if
 it's going to be modified and returned.
 I'm not sure your concern is actually a thing...?


const would do it, but given how restrictive const is in D, I don't see how
it would be very reasonable to restrict passing rvalues to const. And
honestly, I don't really want to encourage the use of const. It's fine if
folks use it where it really works, and it's potentially even quite valuable
there, but it seems like too often folks try to add const in places where
it's just going to cause problems. It's particularly bad when folks try to
add const to generic code - e.g. we recently had to revert a commit to
std.random which worked with dynamic arrays but not other ranges because of
const. And there have been other cases where folks have wanted to try to
make stuff in Phobos "const-correct", which would cause problems. So, I'm
not a big fan of the idea of doing anything that would make folks want to
use const more.

Now, if you can convince Walter and Andrei to allow const ref to accept
rvalues, then fine. I think that that's definitely worse than an attribute
specifically for that, given how limiting const is, but it wouldn't screw up
normal ref in the process, which is what I'm most worried about here. So, I
don't think that going with const would be the best solution to the problem,
but it's far better than making ref in general accept rvalues.

- Jonathan M Davis



 Mar 30 2018



prev sibling next sibling parent reply Manu <turkeyman gmail.com>  writes:


On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:

 On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d

 I want to be able to look at a function signature, see ref without other
 qualifiers, and be reasonably certain that the function is supposed to
 be
 mutating that argument, whereas if ref by itself accepted rvalues, then
 we lose that. If an attribute were used to make it allow rvalues, then
 we wouldn't.


 I don't know any reason why someone wouldn't attribute the argument
 'const' if it doesn't intend to write to it. Likewise 'return ref' if
 it's going to be modified and returned.
 I'm not sure your concern is actually a thing...?


 const would do it, but given how restrictive const is in D, I don't see how
 it would be very reasonable to restrict passing rvalues to const.


I'm actually coming around to the idea of not restricting it to const...
But what I'm trying to say is, the number of cases where your
principle will feel violated should be as infrequent as the cases
where const is insufficient for 'reasons' ;)
As far as I know, in usages that I've ever encountered, I can't
imagine a case where you would feel violated :)



 And honestly, I don't really want to encourage the use of const. It's fine if
 folks use it where it really works, and it's potentially even quite valuable
 there, but it seems like too often folks try to add const in places where
 it's just going to cause problems. It's particularly bad when folks try to
 add const to generic code - e.g. we recently had to revert a commit to
 std.random which worked with dynamic arrays but not other ranges because of
 const. And there have been other cases where folks have wanted to try to
 make stuff in Phobos "const-correct", which would cause problems. So, I'm
 not a big fan of the idea of doing anything that would make folks want to
 use const more.


It's interesting... I recognise the general backlash against const. I
personally just make myself a `struct Mutable(T) { ... }` which I use
when I want to violate const ;)
But if it turns out that const is useless, then we really need to
reconsider the basic design of const >_<
Like, what's the point? You're advocating active discouragement of
const... why is there a feature which it's accepted is a loaded-gun?
Handing it to people creates a high probability they'll shoot
themselves in the feet.



 Now, if you can convince Walter and Andrei to allow const ref to accept
 rvalues, then fine. I think that that's definitely worse than an attribute
 specifically for that, given how limiting const is, but it wouldn't screw up
 normal ref in the process, which is what I'm most worried about here. So, I
 don't think that going with const would be the best solution to the problem,
 but it's far better than making ref in general accept rvalues.


Useful examples using 'return ref' have been presented, which I find
quite compelling too. That's an interesting case of non-const ref.
Safe to say I'm not convincing anyone of anything in any way other
than DIP form.


But I'd like to understand your concern better. You say it's about
scanning an API and understanding some details from it based on seeing
'ref' written there... how does a function accepting an rvalue
interact with your visibility of the API?
Like, your criticism is with respect to understanding the API at a
glance... I don't understand how this proposal interferes with that in
any way?

Sadly, I don't think I'll be able to make it to DConf this year...
which is probably a reason for rejoice of literally everybody
attending! :P
It would be nice to workshop it in person though.



 Mar 30 2018



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 3/30/2018 6:03 PM, Manu wrote:

 Sadly, I don't think I'll be able to make it to DConf this year...


:-(


 which is probably a reason for rejoice of literally everybody
 attending! :P


You'll be missed.


 It would be nice to workshop it in person though.





 Mar 30 2018



 next sibling parent  Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Friday, March 30, 2018 18:52:38 Walter Bright via Digitalmars-d wrote:

 On 3/30/2018 6:03 PM, Manu wrote:

 Sadly, I don't think I'll be able to make it to DConf this year...


 :
 :-(
 :

 which is probably a reason for rejoice of literally everybody
 attending! :P


 You'll be missed.


My usual response to that is that it's better to be missed than to be hit.
;)

But yes, it will be a shame if it's unable to come.

- Jonathan M Davis



 Mar 30 2018



prev sibling  parent  Manu <turkeyman gmail.com>  writes:


On 30 March 2018 at 18:52, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 3/30/2018 6:03 PM, Manu wrote:

 Sadly, I don't think I'll be able to make it to DConf this year...



 :-(


 which is probably a reason for rejoice of literally everybody
 attending! :P



 You'll be missed.


Thank your stars... I wouldn't be able to resist heckling everyone on
this the whole time! :P



 Mar 30 2018



prev sibling  parent  Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Friday, March 30, 2018 18:03:49 Manu via Digitalmars-d wrote:

 On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d

 And honestly, I don't really want to encourage the use of const. It's
 fine if folks use it where it really works, and it's potentially even
 quite valuable there, but it seems like too often folks try to add
 const in places where it's just going to cause problems. It's
 particularly bad when folks try to add const to generic code - e.g. we
 recently had to revert a commit to std.random which worked with dynamic
 arrays but not other ranges because of const. And there have been other
 cases where folks have wanted to try to make stuff in Phobos
 "const-correct", which would cause problems. So, I'm not a big fan of
 the idea of doing anything that would make folks want to use const
 more.


 It's interesting... I recognise the general backlash against const. I
 personally just make myself a `struct Mutable(T) { ... }` which I use
 when I want to violate const ;)


Just don't do anything where you cast away const and then mutate. That
violates the type system and results in undefined behavior. And depending on
what the compiler chooses to do based on const, you could get some rather
subtle and nasty bugs.


 But if it turns out that const is useless, then we really need to
 reconsider the basic design of const >_<
 Like, what's the point? You're advocating active discouragement of
 const... why is there a feature which it's accepted is a loaded-gun?
 Handing it to people creates a high probability they'll shoot
 themselves in the feet.


const makes some sense when you want to write code that accepts both mutable
and immutable arguments, and there are times where it works perfectly fine
to use const. I see no problem with using const in such situations. The
problem is that there are a _lot_ cases where const simply can't be used in
D, because you need to mutate _something_ involved, even if it's not
something that's part of the "logical" state of the object - e.g. in D, you
can't put a mutex in a type and then have it protect anything in a const
member function, because locking the mutex would require mutating it, and D
provides no backdoors like C++'s mutable. What's const is actually const.

Also, const tends to interact very badly with generic code. Whatever API
you're duck typing has to include const as part of it, or you can't use
const - e.g. ranges can't be const, because they have to be mutated to be
iterated, and because the range API does not require that properties like
front or empty be const, no generic code can assume that even those work
with const (and for many ranges, they _can_'t be const - especially when one
range wraps another). In general, it can be quite difficult to even do
something like have a tail-const range over a container. Sometimes, const
can be made to work in more complex situations, but often it can't, and when
it can't, it's often a royal pain.

For non-generic code that can clearly treat an object as fully const without
needing any backdoors, const is just fine, and it may even help prevent
bugs. But for a _lot_ of code - especially idiomatic D code that does a lot
with templates and ranges - const simply doesn't work.

So, const can be used on some level, but anyone who tries to be
"const-correct" in D like many of us usually try to do in C++ is in for a
world of frustration.

If you haven't, I'd suggest that you read this article I recently wrote on
the topic:

http://jmdavisprog.com/articles/why-const-sucks.html

And here's the newsgroup thread discussing it:

https://forum.dlang.org/thread/mailman.581.1520247475.3374.digitalmars-d-announce puremagic.com


 Now, if you can convince Walter and Andrei to allow const ref to accept
 rvalues, then fine. I think that that's definitely worse than an
 attribute specifically for that, given how limiting const is, but it
 wouldn't screw up normal ref in the process, which is what I'm most
 worried about here. So, I don't think that going with const would be
 the best solution to the problem, but it's far better than making ref
 in general accept rvalues.


 Useful examples using 'return ref' have been presented, which I find
 quite compelling too. That's an interesting case of non-const ref.
 Safe to say I'm not convincing anyone of anything in any way other
 than DIP form.


Yes, at this point, convincing Walter or Andrei will require writing a DIP.
A well-written proposal that clearly doesn't have the downsides that they're
so against may have a chance with them, but without something that formally
and clearly provides the arguments, I don't think that they're even going to
pay much attention at this point. The topic has been debated to death
previously, and without a DIP, it really doesn't matter what Walter and
Andrei think, since at this point, they wouldn't introduce a language change
like that without a DIP. So, they'll just do like Andrei did in response to
this thread and tell you to write a DIP rather than be willing to discuss it
further on its own. But no matter the topic, if it involves a major change
to the language, since that now requires a DIP, there's not much point in
arguing the matter with Walter or Andrei beforehand except to help iron out
the DIP. Increasingly, they don't get involved in such discussions outside
of the DIP process, because it eats up too much of their time.


 But I'd like to understand your concern better. You say it's about
 scanning an API and understanding some details from it based on seeing
 'ref' written there... how does a function accepting an rvalue
 interact with your visibility of the API?
 Like, your criticism is with respect to understanding the API at a
 glance... I don't understand how this proposal interferes with that in
 any way?


The only thing that interfers with that is making ref with no other
qualifiers accept rvalues. If it's const or inout or immutable or something
like  rvalue, then you can look at the API and know that ref is on the
parameter so that lvalues will not be copied, and if it accepts rvalues too
(be by it copying them to invisible variables to pass as lvalues or
whatever), then that may mean a performance hit due to the lack of a move,
but you know that the function is not mutating the argument with the
intention of giving the result to the caller. If it's const, then mutation
wouldn't be possible, and if it's  rvalue, then it simply wouldn't make
sense for the function to be providing the result back to the caller,
because that would only work with lvalues.  rvalue would indicate that the
function _could_ mutate the argument (so the restrictions imposed by const
would not apply), and relying on the argument not being mutated would be
asking for bugs, but it would clearly be ref for the purpose of avoiding
copying lvalues and not because the purpose is to mutate the argument and
provide the result.

As long as naked ref does not accept rvalues, then you can look it at and
reasonably assume that the function is intended to mutate the argument and
provide the result to the caller. It also prevents accidentally passing an
rvalue and thus avoids the bugs that would come from passing an rvalue to a
function that's supposed to be mutating its arguments.

As it stands, folks do still sometimes write functions which accept
arguments by ref to avoid copying lvalues, but because of the annoyances
with passing rvalues, most folks don't do that. And if the solution were to
add  rvalue to make ref accept rvalues, then anyone who wanted to use ref
for efficiency reasons would use  rvalue on the ref, and it would make it
that much more reliable that a naked ref indicated that the function was
supposed to mutate the argument. It also wouldn't introduce the problem of
accidentally passing rvalues to ref parameters like making ref in general
accept rvalues would.

So, unless you're proposing that ref in general accept rvalues, I'm not
necessarily criticizing what you're proposing. I do have reservations about
using const instead of something like  rvalue to indicate that rvalues
should be accepted due to the restrictions on const, but I also don't care
enough about being able to pass rvalues by ref to make a big stink about
const ref and auto ref being the only way to accept rvalues without copying
lvalues. I think that it's worth pointing out that making const part of the
solution is problematic, but if you write a DIP that Walter and Andrei
accept that uses const to indicate that ref will accept rvalues, then it's
generally not going to cause problems for me, since I won't be using it
enough for the restrictions on const to matter to me. So, I think that it's
a worse solution, but a subpar solution to this problem only affects me with
regards to the fact that I want D to be a great language (and thus would
prefer not to have subpar solutions) and insofar as I run into other
people's code that uses the technique. I might end up using it occasonally,
but for the most part, the few cases where I'd care, I've been able to use
auto ref to solve the problem.

Most of that I write works just fine passing stuff by value (and if it
doesn't, I tend to make the type a reference type to solve the problem), and
I'm not likely to do much with calling extern(C++) functions outside of
auto-generated code. So, while I acknowledge the problem and have no problem
with it being solved, it simply isn't an issue that affects me much. As
such, it will be that much more annoying if the solution makes existing
parts of the language worse.


 Sadly, I don't think I'll be able to make it to DConf this year...
 which is probably a reason for rejoice of literally everybody
 attending! :P
 It would be nice to workshop it in person though.


LOL. I don't know that anyone would rejoice about it. If they really don't
want to talk to you, they probably just won't talk to you.

But it's not always easy to get away, and it's not necessarily cheap, so
it's understandable if you can't come. Fortunately, I generally haven't had
a problem with getting the time off to go. It's the cost that generally
makes the trip problematic for me, though I've managed to make it works thus
far, and this year, I'm speaking, so it won't be anywhere near as expensive
for me as the last couple of years were.

If you can't make it this year, then hopefully you'll be able to make it
next year, wherever it happens to end up being.

- Jonathan M Davis



 Mar 30 2018