www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - const ref parameters and r-value references

reply "Mark Isaacson" <turck11 hotmail.com> writes:
I'm in the process of learning/practicing D and I noticed 
something that seems peculiar coming from a C++ background:

If I compile and run:

void fun(const ref int x) {
   //Stuff
}

unittest {
   fun(5); //Error! Does not compile
}

I get the specified error in my unit test. I understand that the 
cause is that I've attempted to bind ref to an r-value, what's 
curious is that in C++, the compiler realizes that this is a 
non-issue because of 'const' and just 'makes it work'. Is there a 
rationale behind why D does not do this? Is there a way to write 
'fun' such that it avoids copies but still pledges 
const-correctness while also allowing r-values to be passed in?

Thanks in advance!
May 02 2014
next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 2 May 2014 at 08:17:09 UTC, Mark Isaacson wrote:

 I'm in the process of learning/practicing D and I noticed 
 something that seems peculiar coming from a C++ background:

 If I compile and run:

 void fun(const ref int x) {
   //Stuff
 }

 unittest {
   fun(5); //Error! Does not compile
 }

 I get the specified error in my unit test. I understand that 
 the cause is that I've attempted to bind ref to an r-value, 
 what's curious is that in C++, the compiler realizes that this 
 is a non-issue because of 'const' and just 'makes it work'. Is 
 there a rationale behind why D does not do this? Is there a way 
 to write 'fun' such that it avoids copies but still pledges 
 const-correctness while also allowing r-values to be passed in?


There is `auto ref`, but it only works for templates and is 
somewhat different:

     void fun()(auto ref const int x) {
         // Stuff
     }

     unittest {
         fun(5);    // pass by value
         int a = 5;
         fun(a);    // pass by ref
     }

It generates two functions, with and without ref respectively.

Allowing rvalues to bind to ref (not only const) has been 
discussed on several occasions, but I don't remember the outcome. 
Here is one discussion:

http://forum.dlang.org/thread/ntsyfhesnywfxvzbemwc forum.dlang.org
May 02 2014
parent reply "Mark Isaacson" <turck11 hotmail.com> writes:
Auto ref parameters seem to be just what I need. Thanks! I'd 
still be curious if anyone has additional information regarding 
the rationale at play (I'm spoiled, reading TDPL and having each 
decision explained in text).
May 02 2014
next sibling parent "Meta" <jared771 gmail.com> writes:
On Friday, 2 May 2014 at 21:29:51 UTC, Mark Isaacson wrote:

 Auto ref parameters seem to be just what I need. Thanks! I'd 
 still be curious if anyone has additional information regarding 
 the rationale at play (I'm spoiled, reading TDPL and having 
 each decision explained in text).


The C++ way was deemed not good enough, but nobody knows how to 
do it better.
May 02 2014
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 2 May 2014 at 21:29:51 UTC, Mark Isaacson wrote:

 Auto ref parameters seem to be just what I need. Thanks! I'd 
 still be curious if anyone has additional information regarding 
 the rationale at play (I'm spoiled, reading TDPL and having 
 each decision explained in text).


I had the impression that your goal was to avoid copying. I 
didn't write it explicitly, but `auto ref` doesn't do that. It 
just allows you to avoid writing the same function twice, once 
with ref, and once without. It is essentially equivalent to this:

     void fun(ref const int x) {
         // Stuff
     }

     void fun(const int x) {
         // Stuff
     }

This means that you will still get a (bit-wise) copy if you pass 
in an r-value. But semantically, this is a move, not a copy, so 
it is potentially cheaper than a copy (if your type has an 
expensive postblit). You can see this from the following little 
test program:

     import std.stdio;

     struct S {
         this(this) {
             writefln("postblit %x", &this);
         }

         ~this() {
             writefln("destructor %x", &this);
         }
     }

     void fun()(auto ref const S x) {
         writefln("fun: &x = %x", &x);
     }

     void main() {
         writeln("passing lvalue ...");
         S s;
         fun(s);
         writeln("copying struct ...");
         auto x = s;
         writeln("passing rvalue ...");
         fun(S());
         writeln("done");
     }

As you can see, no postblit is called when the struct is passed 
by value:

     passing lvalue ...
     fun: &x = 7ffffa8c73b8
     copying struct ...
     postblit 7ffffa8c73b9
     passing rvalue ...
     fun: &x = 7ffffa8c7370
     destructor 7ffffa8c7370
     done
     destructor 7ffffa8c73b9
     destructor 7ffffa8c73b8
May 04 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05/04/2014 12:58 PM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 This means that you will still get a (bit-wise) copy if you pass in an
 r-value. But semantically, this is a move, not a copy, so it is
 potentially cheaper than a copy (if your type has an expensive postblit).


It can be constructed in-place.
May 04 2014
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 4 May 2014 at 11:15:59 UTC, Timon Gehr wrote:

 On 05/04/2014 12:58 PM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 This means that you will still get a (bit-wise) copy if you 
 pass in an
 r-value. But semantically, this is a move, not a copy, so it is
 potentially cheaper than a copy (if your type has an expensive 
 postblit).


 It can be constructed in-place.


Doesn't that depend on the ABI?
May 04 2014
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Fri, 02 May 2014 08:17:06 +0000
Mark Isaacson via Digitalmars-d-learn
<digitalmars-d-learn puremagic.com> wrote:


 I'm in the process of learning/practicing D and I noticed
 something that seems peculiar coming from a C++ background:

 If I compile and run:

 void fun(const ref int x) {
    //Stuff
 }

 unittest {
    fun(5); //Error! Does not compile
 }

 I get the specified error in my unit test. I understand that the
 cause is that I've attempted to bind ref to an r-value, what's
 curious is that in C++, the compiler realizes that this is a
 non-issue because of 'const' and just 'makes it work'. Is there a
 rationale behind why D does not do this? Is there a way to write
 'fun' such that it avoids copies but still pledges
 const-correctness while also allowing r-values to be passed in?


By design, in D, ref only accepts lvalues. Unlike in C++, constness has no
effect on that. IIRC, the reasons have something to do with being able to tell
whether the argument is indeed an lvalue or not as well as there being
implementation issues with the fact that const T& foo in C++ doesn't necessary
have a variable for foo to refer to. I don't think that I've ever entirely
understood the rationale behind it, but Andrei is quite adamant on the matter,
and it's been argued over quite a few times. I don't know whether D's choice
on the matter is right or not, but it's not changing at this point.
Regardless, the problem that it generates is the fact that we don't have a
construct which does what C++'s const T& does - i.e. indicate that you want
to accept both lvalues and rvalues without making a copy.

Andrei suggested auto ref to fix this problem, and Walter implemented it, but
he misunderstood what Andrei had meant, so the result was a template-only
solution. If you declare

auto foo(T)(auto ref T bar) {...}

then when you call foo with an lvalue, foo will be instantiated with bar being
a ref T, whereas if foo is called with an rvalue, it will be instintiated with
bar being a T. In either case, no copy will take place. However, it requires
that foo be templated, and it results in a combinatorial explosion of template
instantiations as more auto ref parameters are added, and the function is used
with various combinations of lvalues and rvalues.

The alternative is to declare each overload yourself:

auto foo(ref T bar) {...}
auto foo(T bar) {...}

That doesn't require the function to be templated, and it works for one, maybe
two function parameters, but you have the same combinatorial explosion of
function declarations as you had with template instantiations with auto ref -
except now you're declaring them all explicitly yourself instead of the
compiler generating them for you.

What has been suggested is that we have a way to mark a non-templated function
as accepting both lvalues and rvalues - e.g.

auto foo(NewRefThingy T bar) {...}

and what it would do is make it so that underneath the hood, foo would
actually be

auto foo(ref T bar) {...}

but instead of giving an error when you pass it an rvalue, it would do the
equivalent of

auto temp = returnsRValue();
foo(temp);

so that foo would have an rvalue. You still wouldn't get any copying
happening, you'd only have to declare one function, and you wouldn't get a
combinatorial explosion of function declarations or template instantiations.
I believe that that is essentially what Andrei originally intended for auto
ref to be.

The problem is that we don't want to introduce yet another attribute to do
this. We could reuse auto ref for it, so you'd do

auto foo(auto ref T bar) {...}

but that either means that we redefine what auto ref does with templated
functions (which would be a problem, because it's actually useful there for
other purposes, because it's the closest thing that we have to perfect
forwarding at this point), or we make it so that auto ref does something
different with normal functions than it does with template functions, which
could be confusing, and you might actually like to be able to use the
non-templated auto ref solution with templated functions. It should be
possible _some_ of the time for the compiler to determine that it can optimize
the templated version into the non-templated one (i.e. when it can determine
that the forwarding capabilities of thet templated auto ref aren't used), but
it's not clear how well that will work, or whether it's an acceptable
solution.

Walter has suggested that we just redefine ref itself to do what I just
described rather than using auto ref or defining a new attribute. However,
both Andrei and I argued with him quite a bit over that, because that makes it
so that you can't tell whether a ref argument is intended to mutate what's
being passed in, or whether it's just an optimization (and you can't just use
const in all of the situations where you don't want the mutation, because D's
const is far more restrictive than C++'s const). Others agree with us, and
there are probably some that agree with Walter, but I don't remember at this
point. And Manu was arguing for something with scope ref to solve the problem,
though I'm not sure that he had much support on that one, and I can't remember
the details at this point. It involved having scope return types and altered
the meaning of scope slightly (though it was at least theoretically in the
spirit of what scope is supposed to be). But regradless of the merits of his
suggestion, Andrei was flat-out against it, because having scope ref would
essentially be creating a new attribute, which is unacceptable at this point.

In any case, we discussed it quite a bit at dconf last year and in the
newsgroup shortly thereafter, but we never came to a decision on what to do.
We have what probably should be the solution underneat the hood, just not a
way to express it in the language yet.

What _did_ come out of dconf last year with regards to ref was a means of
making it fully  safe like it's supposed to be by adding some runtime checks
that would only need to be used under circumstances when the compiler couldn't
guarantee that ref wasn't referring to a variable on the stack. But even that
hasn't been implemented yet AFAIK - probably in part because while those of us
in the discussion at dconf agreed on it, some of the folks in the newsgroup
weren't quite so hot on the idea.

So, yes, this is something that we should have solved by now, and we have
some good ideas on the subject, but we just haven't managed to get our act
together on the matter yet.

- Jonathan M Davis
May 04 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:


 Andrei suggested auto ref to fix this problem, and Walter 
 implemented it, but he misunderstood what Andrei had meant,


I missed this detail of the story :-)



 Walter has suggested that we just redefine ref itself to do 
 what I just
 described rather than using auto ref or defining a new 
 attribute. However,
 both Andrei and I argued with him quite a bit over that, 
 because that makes it
 so that you can't tell whether a ref argument is intended to 
 mutate what's
 being passed in, or whether it's just an optimization


I think Ada was designed like that, and then have had to fix the 
language. So it's not a good idea.

Bye,
bearophile
May 04 2014
parent reply "Mark Isaacson" <turck11 hotmail.com> writes:
Thanks for the insights! I suppose we'll get a chance to see 
where things stand at this year's dconf.

It's quite interesting that D's concept of r-values seems less 
developed than C++. Here's hoping that that only results in a 
better thought out solution.
May 04 2014
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Sun, 04 May 2014 19:08:27 +0000
Mark Isaacson via Digitalmars-d-learn
<digitalmars-d-learn puremagic.com> wrote:


 Thanks for the insights! I suppose we'll get a chance to see
 where things stand at this year's dconf.

 It's quite interesting that D's concept of r-values seems less
 developed than C++. Here's hoping that that only results in a
 better thought out solution.


Well, IIRC, rvalue-references are exactly what exactly what Andrei wants to
avoid due the large number of complications that the introduce to the
language. Ultimately, we want a solution in D that is simpler but still does
the job. We have simpler, but haven't quite sorted out the "still does the
job" part. I expect that we'll get there eventually, but we really should have
gotten there long before now and haven't.

- Jonathan M Davis
May 04 2014