www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP 1016--ref T accepts r-values--Community Review Round 1

reply Mike Parker <aldacron gmail.com> writes:
This is the feedback thread for the first round of Community 
Review for DIP 1016, "ref T accepts r-values":

https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md

All review-related feedback on and discussion of the DIP should 
occur in this thread. The review period will end at 11:59 PM ET 
on August 3, or when I make a post declaring it complete.

At the end of Round 1, if further review is deemed necessary, the 
DIP will be scheduled for another round. Otherwise, it will be 
queued for the Final Review and Formal Assessment by the language 
maintainers.

Please familiarize yourself with the documentation for the 
Community Review before participating.

https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

Thanks in advance to all who participate.
Jul 19 2018
next sibling parent reply Dominikus Dittes Scherkl <dominikus.scherkl continental-corporation.com> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md


+1

The DIP reduces bloat, doesn't brake existing code, and I can't 
see downsides.
Jul 20 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 20 July 2018 at 08:12:00 UTC, Dominikus Dittes Scherkl 
wrote:

 On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md


 +1

 The DIP reduces bloat, doesn't brake existing code, and I can't 
 see downsides.


Couldn't have put it better.
Jul 20 2018
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values"


I'd prefer a solution which allows one to make an invisible temp 
manually without making a new statement or a new symbol name. By 
appending something (like .byRef or byRef!long, the latter making 
an implicit type conversion) after a rvalue statement. This would 
still prevent things like byRefFunction(aVariable.incremented) 
when incremented copies the value without the user excepting it.

But If the options are either your solution or the present 
situation, yours is still better. Thanks for the effort to make 
it... I believe Manu will be pleased.
Jul 20 2018
next sibling parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 Thanks for the effort to make it... I believe Manu will be 
 pleased.


Manu is the one who wrote the DIP :-)
Jul 20 2018
parent Dukc <ajieskola gmail.com> writes:
On Friday, 20 July 2018 at 09:24:19 UTC, Bastiaan Veelo wrote:

 On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 Thanks for the effort to make it... I believe Manu will be 
 pleased.


 Manu is the one who wrote the DIP :-)


Though it was Mike :-)
Jul 20 2018
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


That can't work: either it returns an expired stack temporary 
(*very* bad), or allocates with no way to deallocate (bad).
Jul 20 2018
next sibling parent reply Dgame <r.schuett.1987 gmail.com> writes:
On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson wrote:

 On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


 That can't work: either it returns an expired stack temporary 
 (*very* bad), or allocates with no way to deallocate (bad).


What about something like this?

----
import std.stdio;

ref T byRef(T)(T value) {
     static T _val = void;
     _val = value;

     return _val;
}

void foo(ref int a) {
     writeln("A = ", a);
}

void main() {
     foo(42.byRef);
     foo(23.byRef);
}
----
Jul 20 2018
next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 20 July 2018 at 10:08:03 UTC, Dgame wrote:

 On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson wrote:

 On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


 That can't work: either it returns an expired stack temporary 
 (*very* bad), or allocates with no way to deallocate (bad).


 What about something like this?

 ----
 import std.stdio;

 ref T byRef(T)(T value) {
     static T _val = void;
     _val = value;

     return _val;
 }

 void foo(ref int a) {
     writeln("A = ", a);
 }

 void main() {
     foo(42.byRef);
     foo(23.byRef);
 }
 ----


That can't work, consider e.g.:

foo(10.byRef, 20.byRef); // calls foo with (20, 20)

https://run.dlang.io/is/lazeu2
Jul 20 2018
parent reply Dgame <r.schuett.1987 gmail.com> writes:
On Friday, 20 July 2018 at 10:31:48 UTC, Seb wrote:

 On Friday, 20 July 2018 at 10:08:03 UTC, Dgame wrote:

 On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson wrote:

 On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


 That can't work: either it returns an expired stack temporary 
 (*very* bad), or allocates with no way to deallocate (bad).


 What about something like this?

 ----
 import std.stdio;

 ref T byRef(T)(T value) {
     static T _val = void;
     _val = value;

     return _val;
 }

 void foo(ref int a) {
     writeln("A = ", a);
 }

 void main() {
     foo(42.byRef);
     foo(23.byRef);
 }
 ----


 That can't work, consider e.g.:

 foo(10.byRef, 20.byRef); // calls foo with (20, 20)

 https://run.dlang.io/is/lazeu2


True.. But this could work (but is way more uglier): 
https://run.dlang.io/is/rKs2yQ
Jul 20 2018
parent Seb <seb wilzba.ch> writes:
On Friday, 20 July 2018 at 10:43:54 UTC, Dgame wrote:

 On Friday, 20 July 2018 at 10:31:48 UTC, Seb wrote:

 On Friday, 20 July 2018 at 10:08:03 UTC, Dgame wrote:

 On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson 
 wrote:

 On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


 That can't work: either it returns an expired stack 
 temporary (*very* bad), or allocates with no way to 
 deallocate (bad).


 What about something like this?

 ----
 import std.stdio;

 ref T byRef(T)(T value) {
     static T _val = void;
     _val = value;

     return _val;
 }

 void foo(ref int a) {
     writeln("A = ", a);
 }

 void main() {
     foo(42.byRef);
     foo(23.byRef);
 }
 ----


 That can't work, consider e.g.:

 foo(10.byRef, 20.byRef); // calls foo with (20, 20)

 https://run.dlang.io/is/lazeu2


 True.. But this could work (but is way more uglier): 
 https://run.dlang.io/is/rKs2yQ



Putting the missing syntax sugar aside and performance overhead 
due to unneeded copies, this won't work either.
Consider for example that A could be a non-copyable type:

---
struct A {
     int i;
      disable this(this);
}
---

With this DIP it should be possible to do `foo(A(1))` because 
only a reference is passed around.
Jul 20 2018
prev sibling next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 20 July 2018 at 10:08:03 UTC, Dgame wrote:

 On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson wrote:

 On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


 That can't work: either it returns an expired stack temporary 
 (*very* bad), or allocates with no way to deallocate (bad).


 What about something like this?

 ----
 import std.stdio;

 ref T byRef(T)(T value) {
     static T _val = void;
     _val = value;

     return _val;
 }

 void foo(ref int a) {
     writeln("A = ", a);
 }

 void main() {
     foo(42.byRef);
     foo(23.byRef);
 }
 ----


Perhaps semantically correct, but practically a non-starter, due 
to the performance implications of loading and storing to 
thread-local storage and by extension the severe impact on 
compiler optimizations. And also syntactically ugly, even if you 
shorten it to something like `1.r`.

Anyway, I think the DIP is the best way forward - just remove the 
stupid and unnecessary restriction.
Jul 20 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 03:10, Dgame via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson wrote:

 On Friday, 20 July 2018 at 09:03:18 UTC, Dukc wrote:

 appending something (like .byRef or byRef!long, the latter
 making an implicit type conversion)


 That can't work: either it returns an expired stack temporary
 (*very* bad), or allocates with no way to deallocate (bad).


 What about something like this?

 ----
 import std.stdio;

 ref T byRef(T)(T value) {
      static T _val = void;
      _val = value;

      return _val;
 }

 void foo(ref int a) {
      writeln("A = ", a);
 }

 void main() {
      foo(42.byRef);
      foo(23.byRef);
 }
 ----


Substitutes one class of edge case with a different style of edge
case. This general approach doesn't solve my biggest bugbear.
Jul 20 2018
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


 That can't work: either it returns an expired stack temporary 
 (*very* bad), or allocates with no way to deallocate (bad).


How so? It could be made it act exactly as if the temporary was 
made just before the function call, meaning the lifetime would 
end at the end of current scope.

Of course, this required compiler magic. A library solution would 
have exactly the limits you said.
Jul 20 2018
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 20 July 2018 at 16:39:46 UTC, Dukc wrote:

 On Friday, 20 July 2018 at 09:39:47 UTC, Nicholas Wilson wrote:

 appending something (like .byRef or byRef!long, the latter 
 making an implicit type conversion)


 That can't work: either it returns an expired stack temporary 
 (*very* bad), or allocates with no way to deallocate (bad).


 How so? It could be made it act exactly as if the temporary was 
 made just before the function call, meaning the lifetime would 
 end at the end of current scope.


... which is exactly what this DIP proposes ...


 Of course, this required compiler magic. A library solution 
 would have exactly the limits you said.


... and why its a DIP, and not a phobos PR.
Jul 20 2018
parent Dukc <ajieskola gmail.com> writes:
On Friday, 20 July 2018 at 23:19:08 UTC, Nicholas Wilson wrote:

 On Friday, 20 July 2018 at 16:39:46 UTC, Dukc wrote:

 How so? It could be made it act exactly as if the temporary 
 was made just before the function call, meaning the lifetime 
 would end at the end of current scope.


 ... which is exactly what this DIP proposes ...


... except the compiler would do that only when appending .byRef.


 Of course, this required compiler magic. A library solution 
 would have exactly the limits you said.


 ... and why its a DIP, and not a phobos PR.


With that I agree.
Jul 21 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 02:05, Dukc via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community
 Review for DIP 1016, "ref T accepts r-values"


 I'd prefer a solution which allows one to make an invisible temp
 manually without making a new statement or a new symbol name. By
 appending something (like .byRef or byRef!long, the latter making
 an implicit type conversion) after a rvalue statement. This would
 still prevent things like byRefFunction(aVariable.incremented)
 when incremented copies the value without the user excepting it.


This undermines one of the core reasons for the DIP; you're just
substituting one awkward edge case with a different awkward edge case.
Meta code is much simpler and less error-prone when there aren't
syntactic edge cases associated with interacting with ref args, and as
soon as they exist, library authors write some template without such
explicit support for reference args (perhaps they never thought of
it), and the end-user experiences nasty surprises.


 But If the options are either your solution or the present
 situation, yours is still better. Thanks for the effort to make
 it... I believe Manu will be pleased.


I'll be pleased if it doesn't get rejected on principle ;)
Jul 20 2018
prev sibling next sibling parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md


The only difficulty I had was reading

 An example may be some meta that reflects or receives a 
 function by alias.


My difficulties are with the word “meta” used as a noun and what 
“reflecting a function” means. I think I would have worded it as

 An example may be some template with an ‘alias’ parameter that 
 is instantiated on a function.


Otherwise, good work, I think!
Jul 20 2018
parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 02:25, Bastiaan Veelo via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md


 The only difficulty I had was reading

 An example may be some meta that reflects or receives a
 function by alias.


 My difficulties are with the word “meta” used as a noun and what
 “reflecting a function” means. I think I would have worded it as

 An example may be some template with an ‘alias’ parameter that
 is instantiated on a function.




I use the term 'meta' because it's not so clear how the construction
emerges; it is often not a single template, because that's relatively
easy to control, but rather some construction or composition with
components from 3rd party libs which you don't control. (I've made
that point elsewhere)
I use 'meta' to generally refer to any such machinery in any form.
Jul 20 2018
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on August 3, or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the Final Review and Formal Assessment by the 
 language maintainers.

 Please familiarize yourself with the documentation for the 
 Community Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


First of all, I'm very glad to see Manu's persistence turn to 
fruition and by incorporating all the feedback so far, I think we 
have a very solid proposal.

One question:


 It is important that `T` be defined as the argument type [..]


I think this should say that `T` is defined as the *parameter* 
type. This is what I would expect to happen for `void fun(ref int 
x, ref long y, ref double z)`:

```
fun(short(1), byte(2), int(3));
```

=>

```
{
     int    __fun_temp0 = void;
     long   __fun_temp1 = void;
     double __fun_temp2 = void;

     fun(
         __fun_temp0 := short(1),
         __fun_temp1 := byte(2),
         __fun_temp2 := int(3)
     );
}
```

One other subtle point is that the order of declaration of the 
temporaries determines their lifetime, so we must be careful with 
that. In fact I think that we should define it so declaration and 
initialization are not separated and simply make the order of 
declaration match the function argument evaluation order.
Jul 20 2018
parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 03:55, Petar via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md

 All review-related feedback on and discussion of the DIP should
 occur in this thread. The review period will end at 11:59 PM ET
 on August 3, or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary,
 the DIP will be scheduled for another round. Otherwise, it will
 be queued for the Final Review and Formal Assessment by the
 language maintainers.

 Please familiarize yourself with the documentation for the
 Community Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


 First of all, I'm very glad to see Manu's persistence turn to
 fruition and by incorporating all the feedback so far, I think we
 have a very solid proposal.

 One question:


 It is important that `T` be defined as the argument type [..]


 I think this should say that `T` is defined as the *parameter*
 type.


You're exactly right!
I'll fix it.

This is what I would expect to happen for `void fun(ref int

 x, ref long y, ref double z)`:

 ```
 fun(short(1), byte(2), int(3));
 ```

 =>

 ```
 {
      int    __fun_temp0 = void;
      long   __fun_temp1 = void;
      double __fun_temp2 = void;

      fun(
          __fun_temp0 := short(1),
          __fun_temp1 := byte(2),
          __fun_temp2 := int(3)
      );
 }
 ```


Precisely. I just used the wrong word!



 One other subtle point is that the order of declaration of the
 temporaries determines their lifetime, so we must be careful with
 that. In fact I think that we should define it so declaration and
 initialization are not separated and simply make the order of
 declaration match the function argument evaluation order.


Consider that we were calling a normal function:
  fun(T(x, y), gun(10, obj.prop), makeThing());

ref aside, the initialisation order of the temporaries should exactly
match the initialisation and/or evaluation order of args to regular
functions.

For instance, with an expression (x and y are objects):
    T x, y;
    fun(x + y)  -- becomes --> fun(T(x + y))
Ie, the construction should be incorporated into the usual argument
evaluation expression, and evaluated at the normal time.

I'm not sure if that evaluation order is spec-ed, or just
implementation defined?
Whichever it is, we much match that.

Can you propose text I could use to this effect? Add a PR with that
sentence if you like?
Jul 20 2018
prev sibling next sibling parent Timoses <timosesu gmail.com> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 Thanks in advance to all who participate.


"It has been noted that is it possible"
switch: it <-> is
Jul 20 2018
prev sibling next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on August 3, or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the Final Review and Formal Assessment by the 
 language maintainers.

 Please familiarize yourself with the documentation for the 
 Community Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


I like this too. Solid work!

One minor point: the DIP could mentioned structs with disabled 
postblits (it currently only tangentially touches it by stating 
that no copy operation should happen).
It would be great to make sure that this will work:

---
struct A
{
     int i;
      disable this(this);
}

void fun(ref A x);
void test()
{
     fun(A(42));
}
---
Jul 20 2018
parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 04:50, Seb via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md

 All review-related feedback on and discussion of the DIP should
 occur in this thread. The review period will end at 11:59 PM ET
 on August 3, or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary,
 the DIP will be scheduled for another round. Otherwise, it will
 be queued for the Final Review and Formal Assessment by the
 language maintainers.

 Please familiarize yourself with the documentation for the
 Community Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


 I like this too. Solid work!

 One minor point: the DIP could mentioned structs with disabled
 postblits (it currently only tangentially touches it by stating
 that no copy operation should happen).
 It would be great to make sure that this will work:

 ---
 struct A
 {
      int i;
       disable this(this);
 }

 void fun(ref A x);
 void test()
 {
      fun(A(42));
 }
 ---


Can I get a second opinion on this?
I think it's implicit that this works (ref args do not copy), and I
don't think an unrelated thing (copy construction) needs to be made
explicit here.
Jul 20 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/20/18 1:16 AM, Mike Parker wrote:

 This is the feedback thread for the first round of Community Review for 
 DIP 1016, "ref T accepts r-values":
 
 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e294
bd7/DIPs/DIP1016.md 
 
 
 All review-related feedback on and discussion of the DIP should occur in 
 this thread. The review period will end at 11:59 PM ET on August 3, or 
 when I make a post declaring it complete.
 
 At the end of Round 1, if further review is deemed necessary, the DIP 
 will be scheduled for another round. Otherwise, it will be queued for 
 the Final Review and Formal Assessment by the language maintainers.
 
 Please familiarize yourself with the documentation for the Community 
 Review before participating.
 
 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review
 
 Thanks in advance to all who participate.


Looks pretty good to me.

There is very little discussion of the original problem (the reason why 
ref does not bind to rvalues). I don't know if this will satisfy Walter.

A couple points to make there:

1. `this` has been binding as ref to rvalues since structs received 
`this` by ref (yes, they were pointers at one point), and there really 
have not been averse effects.

2. The case is made to allow return ref pipeline functionality, but does 
not address the other cases. A strawman to discuss is a function which 
takes a value-type by reference, and returns void. It's possible the 
function squirrels away a copy of the value somewhere, but unlikely (why 
would it take it by ref in that case?). I'd say 99% of the time the 
point is to modify the parameter for use after the function ends. But in 
the rvalue case, you will lose it immediately. So in effect a call that 
seemingly has some effect will have none.

I would point out that 1 here serves as an example of why 2 isn't 
critical -- you can easily get into this exact situation WITHOUT the DIP 
if the function in question is a member function. And people have 
stumbled across such issues, fixed the problems, and moved on. I would 
say from personal experience the occurrence of such bugs is pretty rare.

I would also point out that the issues with generic code that have been 
pointed out already apply here as well -- generic code might NOT CARE if 
the function has any effect, but having to do metacrobatics to call it 
the "right way" or avoid calling it on purpose would be painful to write 
(as has been demonstrated).

One further point, on the default arguments: you can have a default 
argument be ref, so make sure that section does not read like it 
*always* has to create a temporary.

-Steve
Jul 20 2018
parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 05:40, Steven Schveighoffer via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 7/20/18 1:16 AM, Mike Parker wrote:

 This is the feedback thread for the first round of Community Review for
 DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md


 All review-related feedback on and discussion of the DIP should occur in
 this thread. The review period will end at 11:59 PM ET on August 3, or
 when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the DIP
 will be scheduled for another round. Otherwise, it will be queued for
 the Final Review and Formal Assessment by the language maintainers.

 Please familiarize yourself with the documentation for the Community
 Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


 Looks pretty good to me.

 There is very little discussion of the original problem (the reason why
 ref does not bind to rvalues). I don't know if this will satisfy Walter.


I'm not aware of any substantial discussion?
I understand the design is to prevent accidental disposal of function results.


 A couple points to make there:

 1. `this` has been binding as ref to rvalues since structs received
 `this` by ref (yes, they were pointers at one point), and there really
 have not been averse effects.


Interesting. I'm not sure how to write this in a sentence, can you
produce a patch or PR?


 2. The case is made to allow return ref pipeline functionality, but does
 not address the other cases. A strawman to discuss is a function which
 takes a value-type by reference, and returns void. It's possible the
 function squirrels away a copy of the value somewhere, but unlikely (why
 would it take it by ref in that case?). I'd say 99% of the time the
 point is to modify the parameter for use after the function ends. But in
 the rvalue case, you will lose it immediately. So in effect a call that
 seemingly has some effect will have none.


Do you really think words on this are valuable? It's already a very long DIP.
There is comment that the original design was intended to prevent
accidental loss of function results, and the rest of the DIP tries to
establish that that decision is out of date.
I think the discussion you are asking for is there... perhaps you can
suggest how you'd be satisfied here?


 I would point out that 1 here serves as an example of why 2 isn't
 critical -- you can easily get into this exact situation WITHOUT the DIP
 if the function in question is a member function. And people have
 stumbled across such issues, fixed the problems, and moved on. I would
 say from personal experience the occurrence of such bugs is pretty rare.

 I would also point out that the issues with generic code that have been
 pointed out already apply here as well -- generic code might NOT CARE if
 the function has any effect, but having to do metacrobatics to call it
 the "right way" or avoid calling it on purpose would be painful to write
 (as has been demonstrated).


I understand your points. I'm not sure I have a sense of the text
you'd like to read though.
If you'd like to suggest a specific patch, or PR, that'd be get me on
the same wavelength :)


 One further point, on the default arguments: you can have a default
 argument be ref, so make sure that section does not read like it
 *always* has to create a temporary.


Got it.
Jul 20 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 05:16:53 Mike Parker via Digitalmars-d wrote:

 This is the feedback thread for the first round of Community
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd
 7/DIPs/DIP1016.md

 All review-related feedback on and discussion of the DIP should
 occur in this thread. The review period will end at 11:59 PM ET
 on August 3, or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the
 DIP will be scheduled for another round. Otherwise, it will be
 queued for the Final Review and Formal Assessment by the language
 maintainers.

 Please familiarize yourself with the documentation for the
 Community Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


I am completely against allowing ref to accept rvalues without some sort of
attribute indicating that it should be allowed to (e.g.  rvalue ref).
Allowing ref to accept rvalues goes completely against the idea that ref is
for passing an object so that it can be mutated and have its result affect
the caller. With this DIP, we'd likely start seeing folks using ref all over
the place even when it has nothing to do with having the function mutating
its arguments, and that's not only error-prone, but it obfuscates what ref
was originally intended for.

auto ref was already introduced to solve this problem. The problem of course
is that it requires that the function be templated, and while that's often
desirable, it's not always a viable option (e.g. it doesn't work with
virtual functions). So, I'm fine with adding something akin to auto ref
which is intended to solve the non-templated case with semantics similar to
those described in the DIP, but I think that it would be a huge mistake to
make normal ref accept rvalues.

IMHO, having it be an error to pass an rvalue by ref is often a very
valuable behavior, and I do _not_ want to lose that. I would be fine if this
DIP proposed an attribute such as  rvalue to enable ref to accept rvalues,
but I very much hope that this DIP is not accepted so long as it allows ref
to accept rvalues without such an attribute.

- Jonathan M Davis
Jul 20 2018
next sibling parent reply Daniel N <no public.email> writes:
On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis wrote:

 auto ref was already introduced to solve this problem. The 
 problem of course is that it requires that the function be 
 templated, and while that's often desirable, it's not always a 
 viable option (e.g. it doesn't work with virtual functions). 
 So, I'm fine with adding something akin to auto ref which is 
 intended to solve the non-templated case with semantics similar 
 to those described in the DIP, but I think that it would be a 
 huge mistake to make normal ref accept rvalues.


I'm in favour of this DIP.

However, after this DIP... what is the point of the old "auto 
ref"? If you want to turn your function into a template it's 
trivial.

What if the current semantics of "auto ref" was redefined to what 
this DIP proposes, i.e. non-templated rvalue ref.
Jul 20 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 15:18:33 Daniel N via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis wrote:

 auto ref was already introduced to solve this problem. The
 problem of course is that it requires that the function be
 templated, and while that's often desirable, it's not always a
 viable option (e.g. it doesn't work with virtual functions).
 So, I'm fine with adding something akin to auto ref which is
 intended to solve the non-templated case with semantics similar
 to those described in the DIP, but I think that it would be a
 huge mistake to make normal ref accept rvalues.


 I'm in favour of this DIP.

 However, after this DIP... what is the point of the old "auto
 ref"? If you want to turn your function into a template it's
 trivial.

 What if the current semantics of "auto ref" was redefined to what
 this DIP proposes, i.e. non-templated rvalue ref.


When auto ref is used, the refness of the arguments is forwarded, which can
be critical for stuff like emplace. It also avoids any temporaries, because
if you pass an rvalue, it just instantiates the template so that the
parameter isn't ref, resulting in a move, which is almost certainly more
efficent than what this DIP proposes. So, we would definitely not want to
change what auto ref means for templated functions.

We could choose to implement auto ref for non-templated functions using the
semantics proposed for ref in this DIP, but that would mean that auto ref
did different things for templated and non-templated code, which would be
connfusing and potentially problematic when trying to do something like
templatize an existing function. Also, it would mean that templates could
not use the new semantics, whereas if we added a new attribute such as
 rvalue, then both templated and non-templated functions could have
functions accept rvalues by ref using temporaries in those cases that the
programmer wants that behavior instead of auto ref (e.g. to reduce template
bloat).

- Jonathan M Davis
Jul 20 2018
prev sibling next sibling parent reply meppl <mephisto nordhoff-online.de> writes:
On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 05:16:53 Mike Parker via Digitalmars-d 
 wrote:

 ...


 ...
 Allowing ref to accept rvalues goes completely against the idea 
 that ref is for passing an object so that it can be mutated and 
 have its result affect the caller. With this DIP, we'd likely 
 start seeing folks using ref all over the place even when it 
 has nothing to do with having the function mutating its 
 arguments, and that's not only error-prone, but it obfuscates 
 what ref was originally intended for.
 ...


So, if `immutable` would be the default in D, you would be okay 
with "DIP 1016"?  Because then people would have to write `ref 
mutable` for mutation
Jul 20 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 15:50:29 meppl via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 05:16:53 Mike Parker via Digitalmars-d

 wrote:

 ...


 ...
 Allowing ref to accept rvalues goes completely against the idea
 that ref is for passing an object so that it can be mutated and
 have its result affect the caller. With this DIP, we'd likely
 start seeing folks using ref all over the place even when it
 has nothing to do with having the function mutating its
 arguments, and that's not only error-prone, but it obfuscates
 what ref was originally intended for.
 ...


 So, if `immutable` would be the default in D, you would be okay
 with "DIP 1016"?  Because then people would have to write `ref
 mutable` for mutation


No. I don't see why that would help at all.

Honestly, if D had immutable by default, I'd probably quit D, because it
would make the language hell to use. Some things make sense as immutable but
most don't. If I wanted that kind of strait jacket, I'd use a language like
Haskell.

But regardless, even if I could put up with a such a default, it wouldn't
help, because the use case that Manu is trying to solve would be using ref
mutable just like the use cases that ref is normally used for now. There
needs to be a distinction between the case where ref is used because the
intent is to mutate the object and the case where the intent is to avoid
having to copy lvalues and mutation is acceptable but not the goal. C++
solves this by using const, since once const is used, mutation is no longer
an issue, so the refness is clearly to avoid copying, but with how
restrictive const is in D, it probably wouldn't solve much, because many use
cases couldn't use const. We really do need a mutable equivalent to C++'s
const&. But Manu's solution unnecesarily destroys the dinstinction between
ref being used as means to mutate the argument and ref being used to avoid
copying the argument. Since we can't use const for that, we really need a
new attribute.

- Jonathan M Davis
Jul 20 2018
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Fri, 20 Jul 2018 10:33:56 -0600
schrieb Jonathan M Davis <newsgroup.d jmdavisprog.com>:


 On Friday, July 20, 2018 15:50:29 meppl via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis wrote:  

 On Friday, July 20, 2018 05:16:53 Mike Parker via Digitalmars-d

 wrote:  

 ...  


 ...
 Allowing ref to accept rvalues goes completely against the idea
 that ref is for passing an object so that it can be mutated and
 have its result affect the caller. With this DIP, we'd likely
 start seeing folks using ref all over the place even when it
 has nothing to do with having the function mutating its
 arguments, and that's not only error-prone, but it obfuscates
 what ref was originally intended for.
 ...  


 So, if `immutable` would be the default in D, you would be okay
 with "DIP 1016"?  Because then people would have to write `ref
 mutable` for mutation  


 
 No. I don't see why that would help at all.
 
 Honestly, if D had immutable by default, I'd probably quit D, because it
 would make the language hell to use. Some things make sense as immutable but
 most don't. If I wanted that kind of strait jacket, I'd use a language like
 Haskell.
 
 But regardless, even if I could put up with a such a default, it wouldn't
 help, because the use case that Manu is trying to solve would be using ref
 mutable just like the use cases that ref is normally used for now. There
 needs to be a distinction between the case where ref is used because the
 intent is to mutate the object and the case where the intent is to avoid
 having to copy lvalues and mutation is acceptable but not the goal. C++
 solves this by using const, since once const is used, mutation is no longer
 an issue, so the refness is clearly to avoid copying, but with how
 restrictive const is in D, it probably wouldn't solve much, because many use
 cases couldn't use const. We really do need a mutable equivalent to C++'s
 const&. But Manu's solution unnecesarily destroys the dinstinction between
 ref being used as means to mutate the argument and ref being used to avoid
 copying the argument. Since we can't use const for that, we really need a
 new attribute.
 
 - Jonathan M Davis


I understand the distinction you make, but I don't feel
strongly about it. Vector math can use const in D just fine as
there are no indirections involved, so my functions would have
`ref const` arguments all over the place to discriminate
between "for performance" and "for mutation". That said, I'm
not opposed to some other keyword or  tribute either.

-- 
Marco
Jul 22 2018
prev sibling parent reply aliak <something something.com> writes:
On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 05:16:53 Mike Parker via Digitalmars-d 
 wrote:

 [...]


 I am completely against allowing ref to accept rvalues without 
 some sort of attribute indicating that it should be allowed to 
 (e.g.  rvalue ref). Allowing ref to accept rvalues goes 
 completely against the idea that ref is for passing an object 
 so that it can be mutated and have its result affect the 
 caller. With this DIP, we'd likely start seeing folks using ref 
 all over the place even when it has nothing to do with having 
 the function mutating its arguments, and that's not only 
 error-prone, but it obfuscates what ref was originally intended 
 for.

 [...]


I kinda agree with this, just not so strongly I think.

But as for a UDA, maybe  implicit from the copy constructor DIP 
can be reused here?

void f( implicit ref A a) {}

On a side note, I'm not familiar with the talks that've gone 
around the keyword "in", and I see there's a deprecation issue on 
bugzilla [1]. Can "in" be added here? "in ref" or just repurposed 
completely?

Cheers,
- Ali

[1] https://issues.dlang.org/show_bug.cgi?id=18604
Jul 20 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 16:42:54 aliak via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 05:16:53 Mike Parker via Digitalmars-d

 wrote:

 [...]


 I am completely against allowing ref to accept rvalues without
 some sort of attribute indicating that it should be allowed to
 (e.g.  rvalue ref). Allowing ref to accept rvalues goes
 completely against the idea that ref is for passing an object
 so that it can be mutated and have its result affect the
 caller. With this DIP, we'd likely start seeing folks using ref
 all over the place even when it has nothing to do with having
 the function mutating its arguments, and that's not only
 error-prone, but it obfuscates what ref was originally intended
 for.

 [...]


 I kinda agree with this, just not so strongly I think.

 But as for a UDA, maybe  implicit from the copy constructor DIP
 can be reused here?

 void f( implicit ref A a) {}


I don't know. Maybe. I'd certainly prefer  rvalue, since it would be clearer
(and slightly shorter for that matter), and I don't really agree with copy
constructors requiring  implicit anyway. But at the moment, I don't see a
technical reason why the attribute couldn't be reused.


 On a side note, I'm not familiar with the talks that've gone
 around the keyword "in", and I see there's a deprecation issue on
 bugzilla [1]. Can "in" be added here? "in ref" or just repurposed
 completely?


in was originally supposed to be const scope, but it's really only ever been
const (except for with delegates), since scope hasn't ever done anything for
anything other than delegates, and it was never really defined what scope
was supposed to do exactly - just the general idea that it was supposed to
prevent escaping, not what that really meant in practice or what exactly it
didn't allow to escape. DIP 1000 actually defines what scope does in detail
and makes it work for far more than delegates. However, if in were then to
actually mean const scope as originally intended, then a large percentage of
code would break if it used -dip1000 (and would of course break once
-dip1000 becomes the default), so in has officially become just const.
However, some folks are quite unhappy about that, and -dip1000 breaks tons
of stuff anyway, so arguably, we should just allow in to mean const scope
and have it break whatever it breaks. So, what's actually going to happen
with that is unclear - especially since we really don't have a clean
migration path to making -dip1000 the default anyway, But for now, in just
means const.

Regardless, in has never had anything to do with ref. So, making it imply
ref now would definitely break code, and since in has always implied const,
if we were talking about using in for any of this, we might as well just go
with const ref like C++ does, though given how restrictive D's const is,
that's arguably a very poor solution.

- Jonathan M Davis
Jul 20 2018
parent reply Daniel N <no public.email> writes:
On Friday, 20 July 2018 at 17:02:04 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 16:42:54 aliak via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis

 But as for a UDA, maybe  implicit from the copy constructor 
 DIP can be reused here?

 void f( implicit ref A a) {}


 I don't know. Maybe. I'd certainly prefer  rvalue, since it 
 would be clearer (and slightly shorter for that matter), and I 
 don't really agree with copy constructors requiring  implicit 
 anyway. But at the moment, I don't see a technical reason why 
 the attribute couldn't be reused.


I find the DIP addresses this in a cleaner way with opt-out.
1) Avoids adding more UDA:s
2) It's probably more common to wish to use this feature than the 
opposite, i.e. by taking the opt-out route we can significantly 
reduce  uda clutter.

See DIP:

 void lval_only(int x)  disable;
 void lval_only(ref int x);

 int x = 10;
 lval_only(x);  // ok: choose by-ref
 lval_only(10); // error: literal matches by-val, which is 
  disabled
Jul 20 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 17:25:25 Daniel N via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 17:02:04 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 16:42:54 aliak via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis

 But as for a UDA, maybe  implicit from the copy constructor
 DIP can be reused here?

 void f( implicit ref A a) {}


 I don't know. Maybe. I'd certainly prefer  rvalue, since it
 would be clearer (and slightly shorter for that matter), and I
 don't really agree with copy constructors requiring  implicit
 anyway. But at the moment, I don't see a technical reason why
 the attribute couldn't be reused.


 I find the DIP addresses this in a cleaner way with opt-out.
 1) Avoids adding more UDA:s
 2) It's probably more common to wish to use this feature than the
 opposite, i.e. by taking the opt-out route we can significantly
 reduce  uda clutter.

 See DIP:

 void lval_only(int x)  disable;
 void lval_only(ref int x);

 int x = 10;
 lval_only(x);  // ok: choose by-ref
 lval_only(10); // error: literal matches by-val, which is
  disabled




It changes the semantics of existing code, and honestly, it seems ridiculous
to me to have to declare a separate function to make ref work the way that
it's always worked.

And while I'm sure that if this is implemented, some folks will rush out and
start slapping ref all over the place (or  rvalue ref if it were actually
done with a new attribute like it should be IMHO), the reality of the matter
is that whether creating a temporary to pass rvalues by ref or simply having
the function accept everything by value is better is not straightforward. In
many cases, passing by value is more efficient - especially when lots of
rvalues are involved and relatively few lvalues. Even C++ has changed its
tune on this as far as best practices go. With C++98, it was considered best
practice to use const& all over the place, whereas once they added move
constructors, the situation became much more complicated, and as I
understand it, it's now often considered best practice to pass by value
unless you find that you need to do otherwise, because using const& tends to
prevent moves and often results in extra copies being made. So, while being
able to pass rvalues by ref may be useful in many cases, using  rvalue ref
all over the place is unlikely to be a good idea in general. So, I don't
think that it's at all safe to say that allowing ref accept rvalues by ref
is going to reduce the amount of UDA clutter in the average D program -
especially when  rvalue would mean just adding an attribute, whereas what
the DIP proposes involves having to declare an extra overload just to
disable it.

Either way, what this DIP proposes means that existing uses of ref will need
to be changed - and in an extremely verbose way - in order to get back their
current behavior.

- Jonathan M Davis
Jul 20 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 11:05, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, July 20, 2018 17:25:25 Daniel N via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 17:02:04 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 16:42:54 aliak via Digitalmars-d wrote:

 On Friday, 20 July 2018 at 13:21:11 UTC, Jonathan M Davis

 But as for a UDA, maybe  implicit from the copy constructor
 DIP can be reused here?

 void f( implicit ref A a) {}


 I don't know. Maybe. I'd certainly prefer  rvalue, since it
 would be clearer (and slightly shorter for that matter), and I
 don't really agree with copy constructors requiring  implicit
 anyway. But at the moment, I don't see a technical reason why
 the attribute couldn't be reused.


 I find the DIP addresses this in a cleaner way with opt-out.
 1) Avoids adding more UDA:s
 2) It's probably more common to wish to use this feature than the
 opposite, i.e. by taking the opt-out route we can significantly
 reduce  uda clutter.

 See DIP:

 void lval_only(int x)  disable;
 void lval_only(ref int x);

 int x = 10;
 lval_only(x);  // ok: choose by-ref
 lval_only(10); // error: literal matches by-val, which is
  disabled




 It changes the semantics of existing code,


No *existing* code 'accidentally' dismissed results of ref args. It
doesn't actually change semantics of existing code. It changes
semantics of code you could potentially write in the future.


 and honestly, it seems ridiculous
 to me to have to declare a separate function to make ref work the way that
 it's always worked.


This is exactly what  disable is designed and good for.
This feels like an excellent and intuitive choice to me. I think this
is an improvement by itself, irrespective of this DIP.


 And while I'm sure that if this is implemented, some folks will rush out and
 start slapping ref all over the place (or  rvalue ref if it were actually
 done with a new attribute like it should be IMHO), the reality of the matter
 is that whether creating a temporary to pass rvalues by ref or simply having
 the function accept everything by value is better is not straightforward. In
 many cases, passing by value is more efficient - especially when lots of
 rvalues are involved and relatively few lvalues. Even C++ has changed its
 tune on this as far as best practices go. With C++98, it was considered best
 practice to use const& all over the place, whereas once they added move
 constructors, the situation became much more complicated, and as I
 understand it, it's now often considered best practice to pass by value
 unless you find that you need to do otherwise, because using const& tends to
 prevent moves and often results in extra copies being made.


This DIP isn't a discussion on efficiency. 'best practise' is a
conversation that will continue to be ongoing (and is highly context
sensitive). Fortunately, D has a more mature pre-disposition relative
to C++ on this matter, so I expect the explosion of ref you predict
will not be so explosive as you imagine making direct comparison to
C++.
The point this DIP makes is that, choice to use 'ref' is a decision
_for the API author_, and for reasons that context sensitive.

Whether ref should or shouldn't be used is not your prescription to
make, and while there may be discussions about efficiency, I think
there are many more cases where the choice has nothing to do with
efficiency.
In many cases, it's already prescribed by 3rd party, and you have no choice.


 So, while being
 able to pass rvalues by ref may be useful in many cases, using  rvalue ref
 all over the place is unlikely to be a good idea in general. So, I don't
 think that it's at all safe to say that allowing ref accept rvalues by ref
 is going to reduce the amount of UDA clutter in the average D program -
 especially when  rvalue would mean just adding an attribute, whereas what
 the DIP proposes involves having to declare an extra overload just to
 disable it.


I think disabling it is going to be the overwhelmingly niche case, but
it's good that you can still do it and be satisfied if that's what you
want to do.

Can you link me to a single line of your own code where you've used
this pattern?


 Either way, what this DIP proposes means that existing uses of ref will need
 to be changed - and in an extremely verbose way - in order to get back their
 current behavior.


Their currently behaviour is mostly wrong, and the exact thing I'm
trying to fix though.
Jul 20 2018
prev sibling next sibling parent Radu <rad.racariu gmail.com> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on August 3, or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the Final Review and Formal Assessment by the 
 language maintainers.

 Please familiarize yourself with the documentation for the 
 Community Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


I like this proposal. It fits nicely in the C++ interface story 
and clears the complexity of defining non-template APIs that work 
with (const) ref inputs.
Kudos for Manu for pushing this through.
Jul 20 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 06:21, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, July 20, 2018 05:16:53 Mike Parker via Digitalmars-d wrote:

 This is the feedback thread for the first round of Community
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd
 7/DIPs/DIP1016.md

 All review-related feedback on and discussion of the DIP should
 occur in this thread. The review period will end at 11:59 PM ET
 on August 3, or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the
 DIP will be scheduled for another round. Otherwise, it will be
 queued for the Final Review and Formal Assessment by the language
 maintainers.

 Please familiarize yourself with the documentation for the
 Community Review before participating.

 https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

 Thanks in advance to all who participate.


 I am completely against allowing ref to accept rvalues without some sort of
 attribute indicating that it should be allowed to (e.g.  rvalue ref).


I sincerely hope you're in the minority :(


 Allowing ref to accept rvalues goes completely against the idea that ref is
 for passing an object so that it can be mutated and have its result affect
 the caller.


That is *one use* for ref. I produced text to the effect of changing
your mental model and such assertions.

"This DIP proposes that we reconsider the choice to receive an
argument by value or by reference as a detail that the API *author*
selects with respect to criteria relevant to their project or domain."

Could I improve communication of that sentiment?


 With this DIP, we'd likely start seeing folks using ref all over
 the place even when it has nothing to do with having the function mutating
 its arguments,


Absolutely. 'ref' is arbitrarily suppressed in today's language.


 and that's not only error-prone, but it obfuscates what ref
 was originally intended for.


Ref's original 'intent' was narrower than it's practical use. We're
specifically trying to relax the restrictions to respect that reality.


 auto ref was already introduced to solve this problem. The problem of course
 is that it requires that the function be templated, and while that's often
 desirable, it's not always a viable option (e.g. it doesn't work with
 virtual functions). So, I'm fine with adding something akin to auto ref
 which is intended to solve the non-templated case with semantics similar to
 those described in the DIP, but I think that it would be a huge mistake to
 make normal ref accept rvalues.


If the function uses ref to return results, many such cases use can 'out'.
Functions that mutate the argument to return results aren't
automatically a dismissal of results; perhaps the function is impure,
or it receives multiple args (and one is uninteresting).
It is legitimate to discard the outputs of any function, whether you
ignore the functions return value OR whether you ignore results output
via ref.
Ignoring the return value raises the same argument, but you don't have
this reaction in that case.

It's too strict to frustrate all other useful applications of ref
because of a strict application of this arbitrary rule.


 IMHO, having it be an error to pass an rvalue by ref is often a very
 valuable behavior, and I do _not_ want to lose that. I would be fine if this
 DIP proposed an attribute such as  rvalue to enable ref to accept rvalues,
 but I very much hope that this DIP is not accepted so long as it allows ref
 to accept rvalues without such an attribute.


This DIP proposes how to retain your desired
passing-rvalue-is-an-error feature, and also demonstrates the
symmetrical mechanism for lvalues too.
Jul 20 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 11:50:41 Manu via Digitalmars-d wrote:

 I am completely against allowing ref to accept rvalues without some sort
 of attribute indicating that it should be allowed to (e.g.  rvalue
 ref).


 I sincerely hope you're in the minority :(


 Allowing ref to accept rvalues goes completely against the idea that ref
 is for passing an object so that it can be mutated and have its result
 affect the caller.


 That is *one use* for ref. I produced text to the effect of changing
 your mental model and such assertions.


It's the primary use for ref as it stands given that ref does not currently
accept rvalues, and personally, when I use ref on a function, it's a very
specific API decision where it really does not make sense to accept lvalues.
It's also how plenty of other folks use ref (e.g. it's generally how Phobo
uses ref). This DIP doesn't change any of that. It just makes it harder in
favor of providing a way to pass rvalues by ref. IMHO, those two use cases
are distinct and should be distinct in the API. Using  disable as the DIP
suggests is ridiculously verbose in comparison to how ref currently works,
and it would require updating many existing uses of ref just to avoid the
bugs caused by accepting rvalues. IMHO, it makes far more sense to require
an explicit attribute to indicate that a parameter accepts rvalues. It's
less error-prone, less verbose, and doesn't cause problems for existing
code.


 "This DIP proposes that we reconsider the choice to receive an
 argument by value or by reference as a detail that the API *author*
 selects with respect to criteria relevant to their project or domain."

 Could I improve communication of that sentiment?


The reality of the matter is that it's always going to be up to the API
author on some level - e.g. even if the proposed changes were implemented,
there's still the question of whether a function's parameter should be
marked with ref or not, and arguably, in general, it really shouldn't be,
because it destroys the compiler's ability to move. Yes, by either allowing
ref to accept rvalues or by adding an attribute like  rvalue to make ref
accept rvalues, it's then up to the caller as to whether they pass an lvalue
or rvalue, but it's still up to the API designer to decide how values are
passed, and in some cases, it really does matter whether rvalues are
accepted or not.

- Jonathan M Davis
Jul 20 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/20/18 3:36 PM, Jonathan M Davis wrote:

 
 The reality of the matter is that it's always going to be up to the API
 author on some level - e.g. even if the proposed changes were implemented,
 there's still the question of whether a function's parameter should be
 marked with ref or not, and arguably, in general, it really shouldn't be,
 because it destroys the compiler's ability to move. Yes, by either allowing
 ref to accept rvalues or by adding an attribute like  rvalue to make ref
 accept rvalues, it's then up to the caller as to whether they pass an lvalue
 or rvalue, but it's still up to the API designer to decide how values are
 passed, and in some cases, it really does matter whether rvalues are
 accepted or not.


This has been accepted, pretty much always:

struct S
{
     int x;
     void opOpAssign(string op : "+")(int val) { x += val; }
}

struct Y
{
     S s;
     S foo() { return s; }
}

void main()
{
     import std.stdio;
     Y y;
     y.s += 5;
     writeln(y.s.x); // 5
     y.foo += 5;
     writeln(y.foo.x); // still 5
}

Yet, there has been no major catastrophe, no falling of the sky, no 
errors that I can ever think of that weren't caught because it obviously 
didn't do what it was supposed to. Note also, THERE IS NO WAY TO DISABLE 
IT! If you define operators on a type, you can call them on rvalues. Always.

This is one of the 2 reasons I think that this DIP is OK -- experience 
shows us that the allowance of binding `this` to rvalues hasn't caused 
problems.

The other reason is to avoid having to do acrobatics with static if in 
order to properly accept both rvalues and lvalues. Yes, you can use auto 
ref, but there are many cases where it's not desirable to use templates. 
Those are pretty well outlined in the DIP.

-Steve
Jul 22 2018
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 12:21:42 Manu via Digitalmars-d wrote:

 I think disabling it is going to be the overwhelmingly niche case, but
 it's good that you can still do it and be satisfied if that's what you
 want to do.

 Can you link me to a single line of your own code where you've used
 this pattern?


What, having ref mutate its argument and that being its entire purpose for
being there? How about almost every time I have ever used ref ever. e.g.
this is exactly what some functions in dxml do, and it would be just plain
wrong to pass an rvalue - e.g. the overload of parseDOM that operates on an
EntityRange:



or a function like writeTaggedText



which writes to the XMLWriter that it's given. Having either of those accept
rvalues would just cause bugs.

In the few cases where it seems appropriate to accept both rvalues and
lvalues without copying either, I can almost always use auto ref, because I
rarely use classes. I agree that being able to accept lvalues without
copying them would be useful for non-templated functions as well, but I do
not at all agree that that is how ref should work in general. IMHO, as with
auto ref, it really should be marked with an attribute to indicate that it
is purposefully accepting rvalues. I would not consider it a niche case at
all for ref to not accept rvalues.


 Either way, what this DIP proposes means that existing uses of ref will
 need to be changed - and in an extremely verbose way - in order to get
 back their current behavior.


 Their currently behaviour is mostly wrong, and the exact thing I'm
 trying to fix though.


Why would the current behviour be mostly wrong? If the purpose of using ref
is to accept the current value of a variable and mutate it such that the
argument is mutated, then the current behaviour is _exactly_ what's
desirable, and it prevents bugs.

- Jonathan M Davis
Jul 20 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 12:36, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, July 20, 2018 11:50:41 Manu via Digitalmars-d wrote:

 I am completely against allowing ref to accept rvalues without some sort
 of attribute indicating that it should be allowed to (e.g.  rvalue
 ref).


 I sincerely hope you're in the minority :(


 Allowing ref to accept rvalues goes completely against the idea that ref
 is for passing an object so that it can be mutated and have its result
 affect the caller.


 That is *one use* for ref. I produced text to the effect of changing
 your mental model and such assertions.


 It's the primary use for ref as it stands given that ref does not currently
 accept rvalues,


I'm not sure this is true.
I encounter ref constantly. It's just awkward and uncomfortable to use
rvalues => it's awkward and uncomfortable to use ref.
And I find it's comparatively rarely used for the thing you say.

I think 'primary' use is not what you say, and it's just that we all
suffer under the current rules.
Those that don't understand my pain probably just don't interact with
ref much at all. You can't have a strong opinion on this matter if you
rarely interact with ref.

I occasionally use it for the thing you say, but that's definitely a
minority case, and even in that case, I this rule has no affect on the
problem.
I have *never ever, in my life* 'accidentally' discarded an output
that I needed to use by passing an rvalue to an output-ing ref.

Think about it, that doesn't even structurally make sense.
If the function returns results via a ref, then the only way to refer
to the results it to refer to the lvalue that I passed in.
If I passed an rvalue, then it would be impossible to refer to the
results, and I would be unable to write the next line of code.

The compiler doesn't need to 'help' me here, because I literally can't
progress if I accidentally passed an rvalue.
If I successfully did pass an rvalue and dismiss the results, then
it's implied by the fact my code compiles and runs that I didn't need
the result; in that case, is no different than ignoring a return value
(which is perfectly valid and acceptable practise).

I'm sure it's *possible* to construct a case where you exhibit an
undesirable effect, but we're getting into such unlikely and obviously
contrived territory, that I don't see how any reasonable person could
find that the hindrance caused in all other cases by the current
design is in balance.


 and personally, when I use ref on a function, it's a very
 specific API decision where it really does not make sense to accept lvalues.
 It's also how plenty of other folks use ref (e.g. it's generally how Phobo
 uses ref). This DIP doesn't change any of that. It just makes it harder in
 favor of providing a way to pass rvalues by ref.


Sorry... what is this DIP making harder?


 IMHO, those two use cases
 are distinct and should be distinct in the API. Using  disable as the DIP
 suggests is ridiculously verbose in comparison to how ref currently works,
 and it would require updating many existing uses of ref just to avoid the
 bugs caused by accepting rvalues.


I think the 'bug' is implicitly resolved by the fact that you can't
make use of the results if you pass an rvalue anyway.
Like, there's no conceivable situation where a user thought they
authored their code correctly, but passing an rvalue by accident
allowed their code to compile.

Consider:
  void getOutput(ref T x);
  void doStuff(T x);

  T output;
  getOutput(output); // returns output
  doStuff(output); // consumes output

This is the 'bug' scenario:
  getOutput(makeT()); // returns output to rvalue
  doStuff( ...? ); // what do you even type here? the 'bug' resolves
itself implicitly by the programmer having to type something in this
gap


 IMHO, it makes far more sense to require
 an explicit attribute to indicate that a parameter accepts rvalues. It's
 less error-prone, less verbose, and doesn't cause problems for existing
 code.


Adding yet-another-attribute is another form of swapping one kind of
metacrobatics (to lift Schveighoffer's term) with a different kind,
and then this DIP is self-defeating.
If you want to synth a forwarding function, or something in that
domain, you now need to detect and mirror another ` rvalue` attribute
(guaranteed text mixin)... maybe this leads to `auto  rvalue`?

It's chaos.
Jul 20 2018
prev sibling next sibling parent Elie Morisse <syniurge gmail.com> writes:
On Friday, 20 July 2018 at 05:16:53 UTC, Mike Parker wrote:

 This is the feedback thread for the first round of Community 
 Review for DIP 1016, "ref T accepts r-values":

 https://github.com/dlang/DIPs/blob/725541d69149bc85a9492f7df07360f8e2948bd7/DIPs/DIP1016.md


Yay! Thank you Manu for taking the time to write the DIP, and I 
greatly hope that it will get accepted and implemented quickly.

Code interacting with C++ libraries will become infinitely more 
readable, and from the endless past discussions on the topic no 
real drawback ever emerged, in my opinion.
Jul 20 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 12:51, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, July 20, 2018 12:21:42 Manu via Digitalmars-d wrote:

 I think disabling it is going to be the overwhelmingly niche case, but
 it's good that you can still do it and be satisfied if that's what you
 want to do.

 Can you link me to a single line of your own code where you've used
 this pattern?


 What, having ref mutate its argument and that being its entire purpose for
 being there? How about almost every time I have ever used ref ever. e.g.
 this is exactly what some functions in dxml do, and it would be just plain
 wrong to pass an rvalue - e.g. the overload of parseDOM that operates on an
 EntityRange:



 or a function like writeTaggedText



 which writes to the XMLWriter that it's given. Having either of those accept
 rvalues would just cause bugs.

 In the few cases where it seems appropriate to accept both rvalues and
 lvalues without copying either, I can almost always use auto ref, because I
 rarely use classes. I agree that being able to accept lvalues without
 copying them would be useful for non-templated functions as well, but I do
 not at all agree that that is how ref should work in general. IMHO, as with
 auto ref, it really should be marked with an attribute to indicate that it
 is purposefully accepting rvalues. I would not consider it a niche case at
 all for ref to not accept rvalues.


Okay, this is such the kind of niche use case I'm describing, that the
documentation goes out of it's way to explain to the user the reason
why 'ref' may be used:

"Note that default initialization, copying, and assignment are
disabled for XMLWriter. This is because XMLWriter is essentially a
reference type, but in many cases, it doesn't need to be passed
around, and forcing it to be allocated on the heap in order to be a
reference type seemed like an unnecessary heap allocation. So, it's a
struct with default initialization, copying, and assignment disabled
so that like a reference type, it will not be copied or overwritten.
Code that needs to pass it around can pass it by ref or use the
constructor to explicitly allocate it on the heap and then pass around
the resulting pointer."

This demonstrates to me that the explicit application of  disable in
this case (and cases like it) is absolutely acceptable. It's more than
acceptable; it's entirely preferable to make the API just as explicit
as the documentation.

Secondly, given this API, I can't imagine a valid construction where
the user accidentally provided an rvalue, and the program still built
and runs (with bugs).
What conceivable bug is the rule preventing here? If the user
accidentally provided an rvalue and the function did nothing... what
would the program even do? What's the 'bug'? How would the programmer
call 'save' to write the xml to disk or whatever when the variable was
never declared anywhere. The bug prevents itself in other ways.

This link is self-defeating. Can you find more?


 Either way, what this DIP proposes means that existing uses of ref will
 need to be changed - and in an extremely verbose way - in order to get
 back their current behavior.


 Their currently behaviour is mostly wrong, and the exact thing I'm
 trying to fix though.


 Why would the current behviour be mostly wrong? If the purpose of using ref
 is to accept the current value of a variable and mutate it such that the
 argument is mutated,


That's not "the purpose of using ref".
That is _one use_ of ref, and in my world, it's comparatively rare.


 then the current behaviour is _exactly_ what's
 desirable, and it prevents bugs.


The currently behaviour is absolutely not desired by every other
practical use of 'ref'.

I'd still like to know an example of the kinds of bugs it's preventing
that wouldn't be prevented implicitly in other ways?
Jul 20 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 14:35:57 Manu via Digitalmars-d wrote:

 Their currently behaviour is mostly wrong, and the exact thing I'm
 trying to fix though.


 Why would the current behviour be mostly wrong? If the purpose of using
 ref is to accept the current value of a variable and mutate it such
 that the argument is mutated,


 That's not "the purpose of using ref".
 That is _one use_ of ref, and in my world, it's comparatively rare.


Comparatively rare? It's exactly what most functions using output ranges
need to do. For many output ranges, if the function copies the range instead
of copying, then you have a serious bug, because then the data that gets put
to the output range inside the function does not affect the function
argument - or worse, if it's a pseudo-reference type, it can partially
mutate the argument, putting it in an invalid state. So, unless a function
that accepts an output range is supposed to be taking ownership of it, the
function needs to mark the output range parameter with ref, and it would be
a bug for it to accept an rvalue.

If anyone screws up and does something like return an output range from a
function without that function returning ref and then passes it to a
function that is intended to operate on an output range, they would
currently get an error, whereas with this DIP, they would instead get subtle
bugs with their severity depending on what type of ouput range it was and
what the code was doing with it.

And yes, in most cases, programmers would not pass something other than a
variable to a function that is clearly designed to take its argument by ref
and mutate it, but with the heavy use of properties in a lot of D code, it
can be trivial to operate on something that looks like an lvalue but isn't.
Right now, such code gets caught, because it's an error to pass an rvalue to
a ref parameter, but with this DIP, that would no longer be true. The
situation gets even worse when you consider code maintenance issues like
when someone initially has something be a variable and then later changes it
to a property function and doesn't make it return by ref (and property
functions usually don't return by ref). With this DIP, there could easily be
silent code breakage as a result, whereas such breakage would not be silent
with the current semantics for ref.

Honestly, I don't see how accepting rvalues by ref is anything but
error-prone for any case where the ref is there because the argument is
supposed to be mutated and then be used after the function call. And while
you may very well want to use ref simply to avoid copying - and may do that
frequently right now in spite of that being a pain with rvalues - I have
exactly the opposite experience that you seem to with regards to how ref is
used. In my experience, using ref to avoid copies is rare, and using it to
have the argument be mutated is what's common.

So, while I sympathize with you about making it more user-friendly to pass
rvalues by ref in the case where you just want to avoid copies, I really
think that it merits its own attribute rather than messing up ref for what I
would have considered the common case.

- Jonathan M Davis
Jul 20 2018
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 20 July 2018 at 23:35:46 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 14:35:57 Manu via Digitalmars-d wrote:
 Comparatively rare? It's exactly what most functions using 
 output ranges need to do. For many output ranges, if the 
 function copies the range instead of copying, then you have a 
 serious bug, because then the data that gets put to the output 
 range inside the function does not affect the function argument 
 - or worse, if it's a pseudo-reference type, it can partially 
 mutate the argument, putting it in an invalid state. So, unless 
 a function that accepts an output range is supposed to be 
 taking ownership of it, the function needs to mark the output 
 range parameter with ref, and it would be a bug for it to 
 accept an rvalue.

 If anyone screws up and does something like return an output 
 range from a function without that function returning ref and 
 then passes it to a function that is intended to operate on an 
 output range, they would currently get an error, whereas with 
 this DIP, they would instead get subtle bugs with their 
 severity depending on what type of ouput range it was and what 
 the code was doing with it.


This is an interesting case not completely covered by Manu's 
prior reasoning (that you can't muck it up because you need to 
use it after e.g. appender). But for the case that you would not 
need it later either you already have it as a lvalue (e.g. from a 
function parameter) or its some kind of singleton (e.g. nullSink 
or put's to a global variable).


 And yes, in most cases, programmers would not pass something 
 other than a variable to a function that is clearly designed to 
 take its argument by ref and mutate it, but with the heavy use 
 of properties in a lot of D code, it can be trivial to operate 
 on something that looks like an lvalue but isn't. Right now, 
 such code gets caught, because it's an error to pass an rvalue 
 to a ref parameter, but with this DIP, that would no longer be 
 true. The situation gets even worse when you consider code 
 maintenance issues like when someone initially has something be 
 a variable and then later changes it to a property function and 
 doesn't make it return by ref (and property functions usually 
 don't return by ref). With this DIP, there could easily be 
 silent code breakage as a result, whereas such breakage would 
 not be silent with the current semantics for ref.

 Honestly, I don't see how accepting rvalues by ref is anything 
 but error-prone for any case where the ref is there because the 
 argument is supposed to be mutated and then be used after the 
 function call. And while you may very well want to use ref 
 simply to avoid copying - and may do that frequently right now 
 in spite of that being a pain with rvalues - I have exactly the 
 opposite experience that you seem to with regards to how ref is 
 used. In my experience, using ref to avoid copies is rare, and 
 using it to have the argument be mutated is what's common.


So this problem is restricted output range  properties that 
somehow don't return by ref, aren't intended to be used after 
having data written to them _and_ aren't singleton like?
Jul 20 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, July 21, 2018 00:10:09 Nicholas Wilson via Digitalmars-d wrote:

 So this problem is restricted output range  properties that
 somehow don't return by ref, aren't intended to be used after
 having data written to them _and_ aren't singleton like?


It's a problem for any function that accepts by ref with the intention of
mutating the argument rather than to avoid copying, and something other than
an lvalue is passed. Output ranges are just a particularly common use case.
And property functions are just an easy way to think that you're passing an
lvalue when you're not. For that matter, it's an easy mistake to make with
lots of functions in D, because parens are unnecessary when there are no
function arguments or when the only function argument is passed via UFCS.
And of course, if someone doesn't read the docs carefully enough (which
happens far too often), there are plenty of problems that someone could run
into where they pass an rvalue to a function designed to operate on an
lvalue. Right now, ref catches any and all mistakes where someone tries to
pass an rvalue by ref, but if ref accepts rvalues, it won't catch any of
them.

- Jonathan M Davis
Jul 20 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 17:33, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, July 21, 2018 00:10:09 Nicholas Wilson via Digitalmars-d wrote:

 So this problem is restricted output range  properties that
 somehow don't return by ref, aren't intended to be used after
 having data written to them _and_ aren't singleton like?


 It's a problem for any function that accepts by ref with the intention of
 mutating the argument rather than to avoid copying, and something other than
 an lvalue is passed. Output ranges are just a particularly common use case.


They are. But it's particularly hard to imagine a non-contrived
situation where the compile error would alert the user to a bug that
they wouldn't naturally discover in some other way (likely, the very
next line).
Such cases are so few, that I can't find that they balance the broad
value in this DIP.
Further, if you are writing code where this is at unusually high-risk
for some reason, use the  disable technique.


 And property functions are just an easy way to think that you're passing an
 lvalue when you're not. For that matter, it's an easy mistake to make with
 lots of functions in D, because parens are unnecessary when there are no
 function arguments or when the only function argument is passed via UFCS.
 And of course, if someone doesn't read the docs carefully enough (which
 happens far too often), there are plenty of problems that someone could run
 into where they pass an rvalue to a function designed to operate on an
 lvalue.


This is just saying stuff.

I think you're describing now a bug where a function returns an
lvalue, but it was meant to return an rvalue.
The user then uses that buggy function in conjunction with ref, and
the output is lost (I still can't quite imagine the scenario, given
the restrictive criteria that Nicholas highlighted earlier).

1. The function that's buggy is not the function that involves ref.
2. You're expecting the function that involves ref to catch a bug in
an unrelated function via the existing ref semantics. I think you're
exceeding reason.
3. While it _might_ catch your issue, the buggy function is still
buggy, and may manifest the bug in any number of other far more likely
ways:
   - it may not be involved in a call to ref (and therefore bug not
caught that way)
   - the accidentally-by-val return value may be accessed directly, or
via a method

You're describing an unrelated bug, and it's not ref's job to catch it.
At best, it's just a matter of luck that ref's existing semantic was
able to prevent a bug like you describe, but I think such a
responsibility it well outside of the design's charter.


 Right now, ref catches any and all mistakes where someone tries to
 pass an rvalue by ref, but if ref accepts rvalues, it won't catch any of
 them.


I'm still yet to identify a single realistic scenario where such a
mistake is likely.
I'm sure it's possible that some situations exist, but they're
evidently rare, and if you do identify such a particularly high risk
situation, use the  disable technique. It makes the at-risk construct
beautifully explicit in its intended design.

Adding an  rvalue attribute eliminates the primary value of this DIP.
Situation with meta would become substantially worse.
Meta case-handling which falls back to text-mixins (necessary when
attributes outside the type system appear) is the very worst kind of
'metacrobatics'.
Jul 20 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 18:02, Manu <turkeyman gmail.com> wrote:

 [...]

 I think you're describing now a bug where a function returns an
 lvalue, but it was meant to return an rvalue.


Sorry, wrong way around! I meant to say:
I think you're describing now a bug where a function returns an
rvalue, but it was meant to return an lvalue (ie, a ref).
Jul 20 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 18:04:26 Manu via Digitalmars-d wrote:

 On Fri, 20 Jul 2018 at 18:02, Manu <turkeyman gmail.com> wrote:

 [...]

 I think you're describing now a bug where a function returns an
 lvalue, but it was meant to return an rvalue.


 Sorry, wrong way around! I meant to say:
 I think you're describing now a bug where a function returns an
 rvalue, but it was meant to return an lvalue (ie, a ref).


The function returning an rvalue isn't necessarily a bug. It's the fact that
it was then used in conjunction with a function that accepts its argument by
ref in order to mutate it. If a function accepts its argument by ref in
order to mutate it, then it's a but for it to be given an rvalue regardless
of whether the rvalue comes from. It's just that some cases are more
obviously wrong than others (e.g. passing foo + bad might be obviously
wrong, whereas passing foo.bar may be wrong but look correct).

- Jonathan M Davis
Jul 20 2018
parent Jim Balter <Jim Balter.name> writes:
On Saturday, 21 July 2018 at 01:17:40 UTC, Jonathan M Davis wrote:

 On Friday, July 20, 2018 18:04:26 Manu via Digitalmars-d wrote:

 On Fri, 20 Jul 2018 at 18:02, Manu <turkeyman gmail.com> wrote:

 [...]

 I think you're describing now a bug where a function returns 
 an lvalue, but it was meant to return an rvalue.


 Sorry, wrong way around! I meant to say:
 I think you're describing now a bug where a function returns an
 rvalue, but it was meant to return an lvalue (ie, a ref).


 The function returning an rvalue isn't necessarily a bug. It's 
 the fact that it was then used in conjunction with a function 
 that accepts its argument by ref in order to mutate it. If a 
 function accepts its argument by ref in order to mutate it, 
 then it's a but for it to be given an rvalue regardless of 
 whether the rvalue comes from. It's just that some cases are 
 more obviously wrong than others (e.g. passing foo + bad might 
 be obviously wrong, whereas passing foo.bar may be wrong but 
 look correct).

 - Jonathan M Davis


If the value returned by the function is not supposed to be 
mutable, then the fact that the function taking the ref parameter 
doesn't mutate it is not a bug. If it is supposed to be mutable, 
then there's an unrelated bug in the function that returns the 
value, which just happens to get caught by the current ref 
parameter restriction. The only other sort of bug is where you 
have a value that isn't supposed to be mutated and you have no 
intention of mutating it, and yet you pass it to a ref function 
parameter the express purpose of which is to mutate it, and then 
you don't use the result of that mutation ... but it's hard to 
even see that as a bug, just a pointless exercise, and it's hard 
to come up with a likely scenario where this would happen 
accidentally.

The situation is similar with a property that either isn't 
supposed to be mutable and you don't expect to mutate it and 
don't use its changed value yet pass it to a ref parameter the 
express purpose of which is to mutate it, or the property is 
supposed to be mutable but isn't and the current ref parameter 
restriction just happens to catch that unrelated bug.

I've read this exchange carefully and so far I agree with Manu's 
reasoning and the value of the DIP as it stands, and I'm not in 
favor of requiring  rvalue, as that negates much of the intent. 
However, I'm a D neophyte so I could well be missing something.
Jul 25 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 18:17, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, July 20, 2018 18:04:26 Manu via Digitalmars-d wrote:

 On Fri, 20 Jul 2018 at 18:02, Manu <turkeyman gmail.com> wrote:

 [...]

 I think you're describing now a bug where a function returns an
 lvalue, but it was meant to return an rvalue.


 Sorry, wrong way around! I meant to say:
 I think you're describing now a bug where a function returns an
 rvalue, but it was meant to return an lvalue (ie, a ref).


 The function returning an rvalue isn't necessarily a bug. It's the fact that
 it was then used in conjunction with a function that accepts its argument by
 ref in order to mutate it.


It sounded like you were specifically describing something like a
property that was used to access an output range that was a member of
some other object.
Returning by-val was the 'bug' that made a copy of the output range
instead of returning a reference to it as was intended.

Was that not the scenario you were presenting?


 If a function accepts its argument by ref in
 order to mutate it, then it's a bug for it to be given an rvalue regardless
 of whether the rvalue comes from.


I don't think that's true though. It's just as much a 'bug' to ignore
such output as it is to ignore the return value of a function. (I
shamelessly ignore return values all the time!)
If the function output is not used, then it's not used.

In most cases of course, the result is used, and the programmer would
notice that they have no variable to refer to when they attempt to
refer to the result on the very next line.


 It's just that some cases are more
 obviously wrong than others (e.g. passing foo + bad might be obviously
 wrong, whereas passing foo.bar may be wrong but look correct).


Right, and I think in this example, foo.bar is to be interpreted such
that bar is a property, and it might accidentally return by-val
instead of by-ref?
In that case, 'bar' is the bug. What could it possibly mean for a
property to return an output range by-value?

It's not ref's charter to catch such a bug, and it's much more likely
to manifest in any number of other scenarios rather than an
interaction with ref.
I can't see how this is a compelling reason to dismiss all the
advantages of this DIP in favour of keeping the current semantic.
Jul 20 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, July 20, 2018 19:13:00 Manu via Digitalmars-d wrote:

 I can't see how this is a compelling reason to dismiss all the
 advantages of this DIP in favour of keeping the current semantic.


Honestly, I think we're just coming from points of view that are too
different. IMHO, the core use case for ref is for a function to mutate an
argument and have that result progagate to the argument, and having ref
accept rvalues is not only counter to that, but it risks bugs that are
currently impossible. I think that having a way to accept rvalues by ref for
functions where you want to avoid copying is potentially useful but not
particularly critical. On the other hand, you seem to see little or no value
in having parameters that are intended to only accept lvalues and see great
value in having functions that accept rvalues by ref in order to avoid
copying.

- Jonathan M Davis
Jul 20 2018
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 21 July 2018 at 04:09:25 UTC, Jonathan M Davis wrote:

 Honestly, I think we're just coming from points of view that 
 are too different. IMHO, the core use case for ref is for a 
 function to mutate an argument and have that result progagate 
 to the argument,


I think the critical point here is that the the caller is free to 
ignore the refness of the arg by creating a temporary and not 
using it further.


 and having ref accept rvalues is not only counter to that, but 
 it risks bugs that are currently impossible.


A class of bugs I believe to be self-evident and incredibly 
sparse.


I think that having a way to accept rvalues by ref
 for functions where you want to avoid copying is potentially 
 useful but not particularly critical. On the other hand, you 
 seem to see little or no value in having parameters that are 
 intended to only accept lvalues


As the API author you can  disable the rvalue overload, but 
nothing you do can stop the caller supplying an unused temporary 
to the ref argument. Yes you can discourage them with 
docs/ disabled overload sets/ whatever, but you can't stop them. 
That is no different to today. This DIP may lessen your ability 
to discourage them from misusing it, but you can't stop a 
stubborn idiot, and that is not the audience D is targeting.


 and see great value in having functions that accept rvalues by 
 ref in order to avoid copying.


It is not just the avoiding copying, if it were I'm not sure I'd 
support it. For me the greatest benefit is the increase in 
readability due to not having useless temporaries everywhere in 
ref heavy code (that may not be under API user's control).
Jul 20 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Fri, 20 Jul 2018 at 22:45, Nicholas Wilson via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, 21 July 2018 at 04:09:25 UTC, Jonathan M Davis wrote:

 Honestly, I think we're just coming from points of view that
 are too different. IMHO, the core use case for ref is for a
 function to mutate an argument and have that result progagate
 to the argument,


 I think the critical point here is that the the caller is free to
 ignore the refness of the arg by creating a temporary and not
 using it further.


 and having ref accept rvalues is not only counter to that, but
 it risks bugs that are currently impossible.


 A class of bugs I believe to be self-evident and incredibly
 sparse.


I think that having a way to accept rvalues by ref
 for functions where you want to avoid copying is potentially
 useful but not particularly critical. On the other hand, you
 seem to see little or no value in having parameters that are
 intended to only accept lvalues


 As the API author you can  disable the rvalue overload, but
 nothing you do can stop the caller supplying an unused temporary
 to the ref argument. Yes you can discourage them with
 docs/ disabled overload sets/ whatever, but you can't stop them.
 That is no different to today. This DIP may lessen your ability
 to discourage them from misusing it, but you can't stop a
 stubborn idiot, and that is not the audience D is targeting.


 and see great value in having functions that accept rvalues by
 ref in order to avoid copying.


 It is not just the avoiding copying, if it were I'm not sure I'd
 support it. For me the greatest benefit is the increase in
 readability due to not having useless temporaries everywhere in
 ref heavy code (that may not be under API user's control).


For me, the biggest win is reducing the amount of broken meta because
authors didn't think about ref up front (or ever).
Handling heaps of edge cases in complex meta really sucks; it's hard
to get right, and it's hard to test for. Most people NEVER test
anything with ref.
Removing unnecessary and awkward edge cases that result in brittle
code is always a good thing!

Also removing frustrating bloat and badly named temporaries (t, t1,
t2, etc) from places they never should have existed is a massive win.
Calling into C++, and via polymorphic interfaces also gain
disproportionate advantage.
Jul 20 2018
next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
On Saturday, 21 July 2018 at 05:59:37 UTC, Manu wrote:

 [...]


Let me give a concrete example of one of the situations Jonathan 
is describing. Consider the following code:


struct Secret
{
public:
     string key;
}

/* ... */

genrateRandomKey(ref string key) {
     key = /*  some code to actually generate the key */
}

Secret secret;
generateRandomKey(secret.key);


Now somebody else working on the project who sees the definition 
of Secret might think "Having public access to member variables 
is bad, so lets use property methods instead. This even allows us 
to do some contract checks!" and he changes the definition of 
Secret to the following:


struct Secret
{
private:
     string _key;

public:
     string key()  property const
     {
         return this._key;
     }

     void key(string key)  property
     in
     {
         assert(key.length == 256)
     }
     do
     {
         this._key = key;
     }
}


Now with DIP 1016, the use of generateRandomKey silently "fails", 
i.e. secret._key will not be changed by it, which in this case is 
a big problem as the key is still default initialized!

Of course one might argue that genrateRandomKey should not take 
its argument by reference and rather just return the key instead. 
But in my experience, there is quite a lot of code like this out 
there (e.g. in order to avoid copying, string is probably not the 
best example here...).

In one of your earlier answers you argued, that in cases like 
this, the  property function should probably have returned by 
reference and that not doing so is the real bug. Do you also 
think this is true in this case? I don't think so. One reason is 
for example, that you can not put contracts on your setter 
 property method then...

In my opinion, there is no bug with the definition of the 
 property methods here. The bug only arises through interactions 
with functions which take its parameters by reference.

Do you think this example in contrived? If yes, why?
Jul 21 2018
next sibling parent Johannes Loher <johannes.loher fg4f.de> writes:
On Saturday, 21 July 2018 at 07:10:51 UTC, Johannes Loher wrote:

 [...]


By the way, this does not mean I am totally against DIP 1016. It 
has a lot of benefits in my opinion. But I also think that 
Jonathan has a very valid point which definitely needs to be 
considered.
Jul 21 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Sat, 21 Jul 2018 at 00:15, Johannes Loher via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, 21 July 2018 at 05:59:37 UTC, Manu wrote:

 [...]


 Let me give a concrete example of one of the situations Jonathan
 is describing. Consider the following code:


 struct Secret
 {
 public:
      string key;
 }

 /* ... */

 genrateRandomKey(ref string key) {
      key = /*  some code to actually generate the key */
 }

 Secret secret;
 generateRandomKey(secret.key);


 Now somebody else working on the project who sees the definition
 of Secret might think "Having public access to member variables
 is bad, so lets use property methods instead. This even allows us
 to do some contract checks!" and he changes the definition of
 Secret to the following:


 struct Secret
 {
 private:
      string _key;

 public:
      string key()  property const
      {
          return this._key;
      }

      void key(string key)  property
      in
      {
          assert(key.length == 256)
      }
      do
      {
          this._key = key;
      }
 }


 Now with DIP 1016, the use of generateRandomKey silently "fails",
 i.e. secret._key will not be changed by it, which in this case is
 a big problem as the key is still default initialized!

 Of course one might argue that genrateRandomKey should not take
 its argument by reference and rather just return the key instead.
 But in my experience, there is quite a lot of code like this out
 there (e.g. in order to avoid copying, string is probably not the
 best example here...).

 In one of your earlier answers you argued, that in cases like
 this, the  property function should probably have returned by
 reference and that not doing so is the real bug. Do you also
 think this is true in this case? I don't think so. One reason is
 for example, that you can not put contracts on your setter
  property method then...

 In my opinion, there is no bug with the definition of the
  property methods here. The bug only arises through interactions
 with functions which take its parameters by reference.

 Do you think this example in contrived? If yes, why?


So, to be clear; the 'gotcha' moment as proposed is this:
  1. Function mutates an input received by ref.
  2. Existing code is structured so the function is called with a
member of some lvalue.
  3. Member is _changed_ to be an accessor property for some reason
*and* the property returns the value by-val.
  4. Gotcha!

It is definitely pretty contrived.

1. The function in this scenario is clearly an 'out' parameter, so it
should use 'out', not ref.
2. A function like that would almost certainly return its result, not
mutate an argument. Using ref this way is a poor choice and subverts
move semantics.
3. Scenario depends on introduction of a getter, but any getter
property that returns a member by-val is almost certainly a primitive
type. A getter for a struct member would necessarily return a ref, or
it would experience large copy semantics every time the get is called!
4. A struct-type getter that returns by-val exhibits this gotcha in a
variety of ways; you 'get' the member (a by-val copy), then mutate a
member in any way, (ie, call a method), and you've accidentally
modified the copy, not the source value! (`ref` is not the bug)
  - note, in all other constructions of this same 'bug', existing
language semantics find it acceptable to silently accept the
accidental mutation of an expiring rvalue. Nobody is losing sleep at
night.
5. It's super-unlikely a function returns a primitive type via a
mutating parameter; primitive results would virtually always be
`return`ed, so instances of this pattern are only meaningfully
applicable to structs (see above).
6. Failing all that, I have accepted prior that there may exist
legitimate occurrences, but when you filter out all the cases above,
what is left? It's super rare at very least, and the scenario depends
on 2 things: code is CHANGED to use properties that return by-val, and
mutating function is niche enough that it doesn't fall into any cases
above. I don't know what that case is; at this stage, it's still
hypothetical.
Recommend: use the  disable technique defensively if you suspect the
use case is likely to be used incorrectly.

As I've said prior; this case is quite contrived, and it's clear that
the bug is actually in the property, not the use of 'ref'.
The accidental mutation of an rvalue as suggested can occur in a
variety of ways - only one of which is using ref.

The same 'bug' expressed in a simpler and more likely way:

// a struct that shall be the member
struct M {
  int x;
  void mutate() { ++x; }
}

// the original (working) code
struct S {
  M member;
}
S s;
s.member.mutate();

// the user adds a property in place of an existing member, but the
property incorrectly returns by-val
struct S {
  private M _m;
  M member() { return _m; }
}
S s;
s.member.mutate(); // <- oops, mutated an rvalue!

There are countless ways you can construct the same bug. ref doesn't
contact this problem in a general way, so a solution to this class of
problem shouldn't be ref's responsibility.
It may be possible to argue that the existing ref semantics may
provide defense against a very small surface area of this class of
bug, but I'll never find that narrow advantage weighs favourably
against all the other advantages of this DIP.


...all that said, we understand that there is value in inhibiting
calls with rvalues in some cases. We address this in a very nice way
with  disable, which is also nicely symmetrical such that the
limitation may by applied to rval or lval's.



 By the way, this does not mean I am totally against DIP 1016. It
 has a lot of benefits in my opinion. But I also think that
 Jonathan has a very valid point which definitely needs to be
 considered.


Trust me, I've been considering Jonathan's case painstakingly for 10 years!!
I think it's a contrived, and fairly weak argument. If you consider
the problem as a *class of problem*, which it is, I don't understand
how the existing rule can be so zealously defended in the face of a
bunch of advantages, when all other constructions of the exact same
problem are silently allowed, and literally nobody complains about
them ever!
Jul 21 2018
next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
On Saturday, 21 July 2018 at 08:59:39 UTC, Manu wrote:

 4. A struct-type getter that returns by-val exhibits this 
 gotcha in a
 variety of ways; you 'get' the member (a by-val copy), then 
 mutate a
 member in any way, (ie, call a method), and you've accidentally
 modified the copy, not the source value! (`ref` is not the bug)
   - note, in all other constructions of this same 'bug', 
 existing
 language semantics find it acceptable to silently accept the
 accidental mutation of an expiring rvalue. Nobody is losing 
 sleep at
 night.


This argument kind of convinced me. I also just realized, there 
is another such situation which is very similar to functions 
taking ref parameters, where references to rvalues are actually 
allowed: foreach loops. The following code is actually valid, 
which kind of surprised me:

foreach(ref e; iota(0, 10))
{
     e += 1;
}

So DIP 1016 actually seems to make the language more consistent 
in that regard.

I also carefully reread the DIP and found some minor issues:

- "It has been noted that is it possible to perceive the current 
usage of":
it should be "[...] it is [...]

- Somebody already mentioned this, but this sections sounds 
confusing, please find a better wording: "An example may be some 
meta that reflects or receives a function by alias." Also you 
seem to be using "meta" as a noun many times. I'm not totally 
sure, but I don't think it actually is a noun and for the least, 
It is very uncommon as a noun which makes understanding the 
corresponding sections much harder. For example I don't 
understand what is actually meant by "brittle meta" in a later 
section.
Jul 22 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Sun, 22 Jul 2018 at 01:00, Johannes Loher via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, 21 July 2018 at 08:59:39 UTC, Manu wrote:

 4. A struct-type getter that returns by-val exhibits this
 gotcha in a
 variety of ways; you 'get' the member (a by-val copy), then
 mutate a
 member in any way, (ie, call a method), and you've accidentally
 modified the copy, not the source value! (`ref` is not the bug)
   - note, in all other constructions of this same 'bug',
 existing
 language semantics find it acceptable to silently accept the
 accidental mutation of an expiring rvalue. Nobody is losing
 sleep at
 night.


 This argument kind of convinced me. I also just realized, there
 is another such situation which is very similar to functions
 taking ref parameters, where references to rvalues are actually
 allowed: foreach loops. The following code is actually valid,
 which kind of surprised me:

 foreach(ref e; iota(0, 10))
 {
      e += 1;
 }

 So DIP 1016 actually seems to make the language more consistent
 in that regard.


Hooray! Perhaps my cause is not so hopeless after all these years :)

Yes, there are countless possible constructions of this 'issue'.
Here's my personal favourite:
  T().mutate();

;)

It's not 'ref's job to address this issue.


 I also carefully reread the DIP and found some minor issues:

 - "It has been noted that is it possible to perceive the current
 usage of":
 it should be "[...] it is [...]


There's a PR that nobody has merged that fixes some bugs.


 - Somebody already mentioned this, but this sections sounds
 confusing, please find a better wording: "An example may be some
 meta that reflects or receives a function by alias." Also you
 seem to be using "meta" as a noun many times. I'm not totally
 sure, but I don't think it actually is a noun and for the least,
 It is very uncommon as a noun which makes understanding the
 corresponding sections much harder. For example I don't
 understand what is actually meant by "brittle meta" in a later
 section.


I use the term 'meta' (as in meta-programming) to refer to
compile-time constructions.
I don't tend to say "a template", because many problematic
constructions are compositions, and then consider mixin; not
'template's.
I feel like 'meta' is the simplest accepted term for "compile time
machinery". I'm happy to change my language if it's so confusing. What
should I write?
Jul 22 2018
parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Sunday, 22 July 2018 at 08:53:53 UTC, Manu wrote:

 On Sun, 22 Jul 2018 at 01:00, Johannes Loher via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 - Somebody already mentioned this, but this sections sounds 
 confusing, please find a better wording: "An example may be 
 some meta that reflects or receives a function by alias." Also 
 you seem to be using "meta" as a noun many times. I'm not 
 totally sure, but I don't think it actually is a noun and for 
 the least, It is very uncommon as a noun which makes 
 understanding the corresponding sections much harder. For 
 example I don't understand what is actually meant by "brittle 
 meta" in a later section.


 I use the term 'meta' (as in meta-programming) to refer to
 compile-time constructions.
 I don't tend to say "a template", because many problematic
 constructions are compositions, and then consider mixin; not
 'template's.
 I feel like 'meta' is the simplest accepted term for "compile 
 time
 machinery". I'm happy to change my language if it's so 
 confusing. What
 should I write?


Both these terms (CT [construction|machinery]) I find more 
descriptive than just “meta”. If you want to use “meta” then I’d 
suggest “meta code” to differentiate it from meta data, meta key 
or meta proposal (which is this: a proposal on a proposal) :-)

-Bastiaan
Jul 22 2018
prev sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, July 22, 2018 01:53:53 Manu via Digitalmars-d wrote:

 I use the term 'meta' (as in meta-programming) to refer to
 compile-time constructions.
 I don't tend to say "a template", because many problematic
 constructions are compositions, and then consider mixin; not
 'template's.
 I feel like 'meta' is the simplest accepted term for "compile time
 machinery". I'm happy to change my language if it's so confusing. What
 should I write?


Then say meta-programming, not just meta. What you mean by meta can
certainly be inferred by the context, but I don't think that it's generally
accepted that meta by itself necesarily means meta-programming when
discussing D. Even then, I think that it's more common to say template
meta-programming than meta-programming by itself, but meta-programming by
itself is much more descriptive than just meta.

- Jonathan M Davis
Jul 22 2018
prev sibling next sibling parent ShadoLight <ettienne.gilbert gmail.com> writes:
On Saturday, 21 July 2018 at 08:59:39 UTC, Manu wrote:

 On Sat, 21 Jul 2018 at 00:15, Johannes Loher via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Saturday, 21 July 2018 at 05:59:37 UTC, Manu wrote:

 [...]


 Let me give a concrete example of one of the situations 
 Jonathan is describing. Consider the following code:




[Snip]


 Do you think this example in contrived? If yes, why?




[Snip]


 There are countless ways you can construct the same bug. ref 
 doesn't
 contact this problem in a general way, so a solution to this 
 class of
 problem shouldn't be ref's responsibility.


[Snip]


 ... I don't understand how the existing rule can be so 
 zealously defended in the face of a
 bunch of advantages, when all other constructions of the exact 
 same
 problem are silently allowed, and literally nobody complains 
 about them ever!


+1000

Very well and elegantly argued Manu.

I also notice that nobody that opposes this DIP has bothered to 
address the inconsistency that you raised above, i.e. the current 
acceptance of the same behaviour in other constructions, but 
somehow oppose this DIP for the exact same behaviour.
Jul 25 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 21 July 2018 at 08:59:39 UTC, Manu wrote:

 On Sat, 21 Jul 2018 at 00:15, Johannes Loher via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Saturday, 21 July 2018 at 05:59:37 UTC, Manu wrote:

 [...]


 Let me give a concrete example of one of the situations 
 Jonathan is describing. Consider the following code:


 struct Secret
 {
 public:
      string key;
 }

 /* ... */

 genrateRandomKey(ref string key) {
      key = /*  some code to actually generate the key */
 }

 Secret secret;
 generateRandomKey(secret.key);


 Now somebody else working on the project who sees the 
 definition of Secret might think "Having public access to 
 member variables is bad, so lets use property methods instead. 
 This even allows us to do some contract checks!" and he 
 changes the definition of Secret to the following:


 struct Secret
 {
 private:
      string _key;

 public:
      string key()  property const
      {
          return this._key;
      }

      void key(string key)  property
      in
      {
          assert(key.length == 256)
      }
      do
      {
          this._key = key;
      }
 }


 Now with DIP 1016, the use of generateRandomKey silently 
 "fails", i.e. secret._key will not be changed by it, which in 
 this case is a big problem as the key is still default 
 initialized!

 Of course one might argue that genrateRandomKey should not 
 take its argument by reference and rather just return the key 
 instead. But in my experience, there is quite a lot of code 
 like this out there (e.g. in order to avoid copying, string is 
 probably not the best example here...).

 In one of your earlier answers you argued, that in cases like 
 this, the  property function should probably have returned by 
 reference and that not doing so is the real bug. Do you also 
 think this is true in this case? I don't think so. One reason 
 is for example, that you can not put contracts on your setter 
  property method then...

 In my opinion, there is no bug with the definition of the 
  property methods here. The bug only arises through 
 interactions with functions which take its parameters by 
 reference.

 Do you think this example in contrived? If yes, why?


 So, to be clear; the 'gotcha' moment as proposed is this:
   1. Function mutates an input received by ref.
   2. Existing code is structured so the function is called with 
 a
 member of some lvalue.
   3. Member is _changed_ to be an accessor property for some 
 reason
 *and* the property returns the value by-val.
   4. Gotcha!

 It is definitely pretty contrived.

 1. The function in this scenario is clearly an 'out' parameter, 
 so it
 should use 'out', not ref.
 2. A function like that would almost certainly return its 
 result, not
 mutate an argument. Using ref this way is a poor choice and 
 subverts
 move semantics.
 3. Scenario depends on introduction of a getter, but any getter
 property that returns a member by-val is almost certainly a 
 primitive
 type. A getter for a struct member would necessarily return a 
 ref, or
 it would experience large copy semantics every time the get is 
 called!


This is assuming anything that isn't a primitive is a large 
struct that is copied, which, in my experience, is rarely the 
case.

I don't recall ever implementing a getter in D that returns by 
ref. I'd consider that a code smell in pretty much every 
language, allowing mutation from the outside. For context, I 
think that getters are a faint code smell and that setters always 
stink.

All of this to say that I disagree that getters will usually 
return by ref unless the return type is a primitive. That's not 
how I write code and I don't remember encountering this in the 
wild.


 4. A struct-type getter that returns by-val exhibits this 
 gotcha in a
 variety of ways; you 'get' the member (a by-val copy), then 
 mutate a
 member in any way, (ie, call a method), and you've accidentally
 modified the copy, not the source value!


I have trouble understanding why anyone would expect to mutate 
the original object without first consulting the API to see if 
was returned by ref. I'd never expect that to happen. I don't 
think that's a bug because I'd never expect things to work that 
way. Nearly all of my variables and function parameters are const 
anyway. Maybe my brain is weird. All I know is that I'd never 
encounter this and consider it a bug. To me not mutating my 
object is a feature.


 (`ref` is not the bug)
   - note, in all other constructions of this same 'bug', 
 existing
 language semantics find it acceptable to silently accept the
 accidental mutation of an expiring rvalue. Nobody is losing 
 sleep at
 night.


That's because T().mutate() is obviously not going to do 
anything. Nobody would expect it to.


 The same 'bug' expressed in a simpler and more likely way:

 // a struct that shall be the member
 struct M {
   int x;
   void mutate() { ++x; }
 }

 // the original (working) code
 struct S {
   M member;
 }
 S s;
 s.member.mutate();


It'd take roughly 5s between me seeing this in code review and 
typing the words "Law of Demeter violation". To me that's TRWTF.


 ...all that said, we understand that there is value in 
 inhibiting calls with rvalues in some cases. We address this in 
 a very nice way with  disable, which is also nicely symmetrical 
 such that the limitation may by applied to rval or lval's.


I like using  disable this way. It's unclear to me the impact on 
existing code that doesn't already have a  disable since it 
wasn't needed before.

I'm not against the DIP - I think easier interop with C++ is a 
good thing and this would help it. I have to think a bit more 
about the points Jonathan has brought up, because it sounds like 
there's a possibility that bugs might be introduced if the DIP 
goes through, at least as-is. I'm not sure.

Atila
Jul 25 2018
next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, 25 Jul 2018 at 10:45, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 ...all that said, we understand that there is value in
 inhibiting calls with rvalues in some cases. We address this in
 a very nice way with  disable, which is also nicely symmetrical
 such that the limitation may by applied to rval or lval's.


 I like using  disable this way. It's unclear to me the impact on
 existing code that doesn't already have a  disable since it
 wasn't needed before.

 I'm not against the DIP - I think easier interop with C++ is a
 good thing and this would help it. I have to think a bit more
 about the points Jonathan has brought up, because it sounds like
 there's a possibility that bugs might be introduced if the DIP
 goes through, at least as-is. I'm not sure.


FWIW; I presented a further solution for the property case, which I
think is a good improvement for properties in general (it will address
other constructions of this same issue that aren't in conjunction with
ref).
It addresses the issue Jonathan raised in the domain where the problem
exists, and leaves unrelated problems outside of this DIP's problem
space.
Jul 25 2018
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, 25 Jul 2018 at 10:45, Atila Neves via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, 21 July 2018 at 08:59:39 UTC, Manu wrote:

 [...]
 3. Scenario depends on introduction of a getter, but any getter
 property that returns a member by-val is almost certainly a
 primitive
 type. A getter for a struct member would necessarily return a
 ref, or
 it would experience large copy semantics every time the get is
 called!


 This is assuming anything that isn't a primitive is a large
 struct that is copied, which, in my experience, is rarely the
 case.


But my point is, that exact reasoning extends to the hypothetical ref
argument as the return value.
If a property function sees fit to return by-value (ie, struct is
small), then a function receiving that object will also receive the
argument by-value.

This will be the usual pattern. When a type becomes large enough to
want to pass it by ref, any property that returns one will also most
likely return by ref.
I'm trying to say that, a mismatch is naturally unlikely to occur. And
very unlikely occur *by accident*; it would be a design intent.


 I don't recall ever implementing a getter in D that returns by
 ref. I'd consider that a code smell in pretty much every
 language, allowing mutation from the outside.


Such a getter would likely return const ref, but discussions about
proper use of 'const' are not on topic here.


 For context, I
 think that getters are a faint code smell and that setters always
 stink.


So, you're reducing the power of the properties argument in principle?


 All of this to say that I disagree that getters will usually
 return by ref unless the return type is a primitive. That's not
 how I write code and I don't remember encountering this in the
 wild.


Agreed. My above amendment should be a more accurate tendency, which
is what I was trying to express, but in too-few words.


 (`ref` is not the bug)
   - note, in all other constructions of this same 'bug',
 existing
 language semantics find it acceptable to silently accept the
 accidental mutation of an expiring rvalue. Nobody is losing
 sleep at
 night.


 That's because T().mutate() is obviously not going to do
 anything. Nobody would expect it to.


Of course `T().mutate()` is obvious, but `s.member.mutate()` (where
member is property) might be misunderstood to do something. I don't
see how the exact set of arguments being applied to ref don't apply
here (and many other possible cases).


 The same 'bug' expressed in a simpler and more likely way:

 // a struct that shall be the member
 struct M {
   int x;
   void mutate() { ++x; }
 }

 // the original (working) code
 struct S {
   M member;
 }
 S s;
 s.member.mutate();


 It'd take roughly 5s between me seeing this in code review and
 typing the words "Law of Demeter violation". To me that's TRWTF.


I don't understand what you're saying here?
I think this case is equally hard to spot as the
passing-property-as-ref-arg case.

Either way, the solution to this whole branch of discussion lies with
property, not with ref (and I've given some ideas).
Jul 25 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 25 July 2018 at 18:37:55 UTC, Manu wrote:

 On Wed, 25 Jul 2018 at 10:45, Atila Neves via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Saturday, 21 July 2018 at 08:59:39 UTC, Manu wrote:





 But my point is, that exact reasoning extends to the 
 hypothetical ref
 argument as the return value.
 If a property function sees fit to return by-value (ie, struct 
 is
 small), then a function receiving that object will also receive 
 the
 argument by-value.


That's a good point. I mean, with code not under one's control it 
could happen otherwise, but most likely not.


 This will be the usual pattern. When a type becomes large 
 enough to
 want to pass it by ref, any property that returns one will also 
 most
 likely return by ref.


Yep.


 For context, I
 think that getters are a faint code smell and that setters 
 always
 stink.


 So, you're reducing the power of the properties argument in 
 principle?


Yes, as long as it's code I write or review.



 S s;
 s.member.mutate();


 It'd take roughly 5s between me seeing this in code review and 
 typing the words "Law of Demeter violation". To me that's 
 TRWTF.


 I don't understand what you're saying here?
 I think this case is equally hard to spot as the
 passing-property-as-ref-arg case.


That unless it's a UFCS chain there shouldn't be more than one 
dot. One should attempt to not call functions on returned 
members. It should be `s.mutate()` where `member` is aliased this 
or has `mutate` manually forwarded.

That is: don't "reach in" to objects, it's bad design anyway and 
the bug doesn't exist if you don't.
Jul 26 2018
prev sibling parent reply Johnatune <s s.cum> writes:
I was for this back when it was only for 'const ref' but that 
somehow changed to just ref. Which I think is a mistake. Yes D's 
const is broken and useless, but I don't think that's a reason to 
introduce difficult to locate bugs with the addition of this 
feature. There's not a simple way to locate where code might be 
breaking if a temporary value is being passed to a ref function 
when it wasn't intended. Right now this is a compiler error, even 
in C++. But with this proposed change it might take a lengthy 
amount of time in the debugger trying to understand what is 
happening.

Yes I want rvalue refs, just not like this.
Jul 21 2018
parent Manu <turkeyman gmail.com> writes:
On Sat., 21 Jul. 2018, 12:30 pm Johnatune via Digitalmars-d, <
digitalmars-d puremagic.com> wrote:


 I was for this back when it was only for 'const ref' but that
 somehow changed to just ref. Which I think is a mistake. Yes D's
 const is broken and useless, but I don't think that's a reason to
 introduce difficult to locate bugs with the addition of this
 feature. There's not a simple way to locate where code might be
 breaking if a temporary value is being passed to a ref function
 when it wasn't intended. Right now this is a compiler error, even
 in C++. But with this proposed change it might take a lengthy
 amount of time in the debugger trying to understand what is
 happening.

 Yes I want rvalue refs, just not like this.


It's like you didn't read the DIP.

I was initially very hesitant as you, but I was motivated to remove the
const restriction when I realised that usefulness of the pattern far
exceeds what you're familiar with in C++.
While interacting with interfaces similar to C++ is one motivating use case
(particularly for myself), I realised that pipeline programming and 'return
ref' change the game.
They are distinctly "D" coding styles, and this issue is a cause of
friction with respect to one of D's greatest success stories.
Such use with pipelines and 'return ref' imply that mutable ref also
supported.

There is indeed also the restrictive-const issue, and the current
recommendation is to not over-use const. From that perspecetive, this DIP
is in-line with current recommendations regarding const; it is obviously
naturally supported, but not mandated.

Finally, I have really thought about the problem case that everyone is so
terrified about, and I am still trying to visualise real-world situations
where that could arise. Please read the replies to Jonathan's sub-thread...
that has all the detail on that.

TLDR:
  - The fear in question relates to a general class of problem and not
specifically related to 'ref' - that's just one of very many ways to expose
the 'accidentally modified an rvalue' issue.
  - I don't believe it's within 'ref's responsibility domain to fight that
class of problem, particularly at the expense of the other advantages we're
inhibiting (proposed in thie DIP).
  - We're still trying to identify probably real-world examples.
  - Your fear is mostly invalid. And even if it's not, the DIP proposes a
very nice and principled mechanism to retain "lval-only please" API design.






 Jul 21 2018



prev sibling  parent reply Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Saturday, 21 July 2018 at 05:40:24 UTC, Nicholas Wilson wrote:


 It is not just the avoiding copying, if it were I'm not sure 
 I'd support it. For me the greatest benefit is the increase in 
 readability due to not having useless temporaries everywhere in 
 ref heavy code (that may not be under API user's control).


Explicit is better than implicit.
(The Zen of Python)

Frankly speaking, my feeling is that D is becoming a horrible 
mess for the programmer...

And, BTW, I'm totally with Jonathan also on  implicit for copy 
ctor... I really really don't like it, another nail in the coffin 
of a beauty, symmetry, and intuitive syntax.

I'm starting to think that only a D3, with a lot of thing 
reorganised without the obsession of breaking changes can safe 
that beautiful language: Python was _almost_ killed in the 2-3 
transaction, but kudos to Guido and the core time, it resurrected 
more strong than ever.

/Paolo



 Jul 21 2018



 next sibling parent reply JohnB <john john.com>  writes:


On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi wrote:

 Frankly speaking, my feeling is that D is becoming a horrible 
 mess for the programmer...



 I'm starting to think that only a D3, with a lot of thing 
 reorganised without the obsession of breaking changes can safe 
 that beautiful language...


If a transition from D to D2 is one cause of that mess, why a new 
transition wouldn't continue that mess.

And yes as new fellow playing with this language since 2012, I 
think it's becoming more like a C++ in madness.

John.



 Jul 21 2018



  parent  Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Saturday, 21 July 2018 at 12:54:28 UTC, JohnB wrote:

 On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi 
 wrote:

 Frankly speaking, my feeling is that D is becoming a horrible 
 mess for the programmer...



 I'm starting to think that only a D3, with a lot of thing 
 reorganised without the obsession of breaking changes can safe 
 that beautiful language...


 If a transition from D to D2 is one cause of that mess, why a 
 new transition wouldn't continue that mess.

 And yes as new fellow playing with this language since 2012, I 
 think it's becoming more like a C++ in madness.

 John.


Not to talk about the recently revived rcstring, that by default 
are not a range... guess it...

/P



 Jul 21 2018



prev sibling  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi wrote:


 Frankly speaking, my feeling is that D is becoming a horrible 
 mess for the programmer...



 /Paolo


How!? Please Explain. You need to demonstrate evidence instead of 
appeal to emotional fallacy by resorting to "feels".

-Alexander



 Jul 21 2018



  parent reply Marco Leise <Marco.Leise gmx.de>  writes:


Am Sat, 21 Jul 2018 19:22:05 +0000
schrieb 12345swordy <alexanderheistermann gmail.com>:


 On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi wrote:
 

 Frankly speaking, my feeling is that D is becoming a horrible 
 mess for the programmer...  


 

 /Paolo  


 How!? Please Explain. You need to demonstrate evidence instead of 
 appeal to emotional fallacy by resorting to "feels".
 
 -Alexander


The DIP increases consistency recalling that rvalues are
accepted:

- for the implicit 'this' parameter in methods
- in foreach loop variables declared as ref

No more special rules: rvalues are implicitly promoted to
lvalues where needed. The feeling probably comes from the
inevitable realization that the community is pluralistic and
Dlang acquired a lot of features that go towards someone else's
vision for a good PL. Some want a relaxed stance towards
breaking change, some want C++ or ObjC compatibility, some
want to know what assembly a piece of code compiles to or have
soft realtime constraints that don't work with a system
language's mark&sweep GC. Is D2 messier than D1? Sure it is,
and it caters to more use cases, too. As soon as you
substantiate what exact feature is adding to the horrible
mess, someone (often a group) will jump to defend it, because
they have a good use case or two.
It is kind of ironic that in order to do better than C++ you
have to support most of what modern C++ compilers offer and end
up having tons of unrelated features that make the language
just as bloated as C++ after a decade of community feedback.
It is a system PL. I think it needs to be this way and is a
lot cleaner with basic data types and more expressive still,
lacking a lot of C++'s legacy.

-- 
Marco



 Jul 24 2018



  parent reply Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Wednesday, 25 July 2018 at 02:21:18 UTC, Marco Leise wrote:

 Am Sat, 21 Jul 2018 19:22:05 +0000
 schrieb 12345swordy <alexanderheistermann gmail.com>:


 On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi 
 wrote:
 

 Frankly speaking, my feeling is that D is becoming a 
 horrible mess for the programmer...


 

 /Paolo


 How!? Please Explain. You need to demonstrate evidence instead 
 of appeal to emotional fallacy by resorting to "feels".
 
 -Alexander


 The DIP increases consistency recalling that rvalues are 
 accepted:

 - for the implicit 'this' parameter in methods
 - in foreach loop variables declared as ref

 No more special rules: rvalues are implicitly promoted to
 lvalues where needed.


That's correct, but 'this' is already a special guest in C++ 
style PL, with its own special rules. The support for ref 
variable in foreach loop.... can be removed (yup!), or made 
stricter.. no more inconsistency.


 The feeling probably comes from the
 inevitable realization that the community is pluralistic and
 Dlang acquired a lot of features that go towards someone else's
 vision for a good PL. Some want a relaxed stance towards
 breaking change, some want C++ or ObjC compatibility, some
 want to know what assembly a piece of code compiles to or have
 soft realtime constraints that don't work with a system
 language's mark&sweep GC. Is D2 messier than D1? Sure it is,
 and it caters to more use cases, too. As soon as you
 substantiate what exact feature is adding to the horrible
 mess, someone (often a group) will jump to defend it, because
 they have a good use case or two.


Yep, and I'm conscious to be often a pessimistic, lurking guy 
here in the forum... :-/


 It is kind of ironic that in order to do better than C++ you
 have to support most of what modern C++ compilers offer and end
 up having tons of unrelated features that make the language
 just as bloated as C++ after a decade of community feedback.
 It is a system PL. I think it needs to be this way and is a
 lot cleaner with basic data types and more expressive still,
 lacking a lot of C++'s legacy.


There's a tension between Walter effort in turning D as a 
suitable language for memory correctness, and a good candidate 
for writing 'bug free rock solid software fast' and the 
continuous addition of features like this.

Joke aside, I'm still on Jonathan side on this.

Finally, sorry to use often the term 'feeling', and sorry for not 
being constructive: but really is a 'feeling'... I don't pretend 
to be right. So no problems in just ignoring that

  :-P

Cheers,
Paolo



 Jul 25 2018



  parent reply Manu <turkeyman gmail.com>  writes:


On Wed., 25 Jul. 2018, 12:30 am Paolo Invernizzi via Digitalmars-d, <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 25 July 2018 at 02:21:18 UTC, Marco Leise wrote:

 Am Sat, 21 Jul 2018 19:22:05 +0000
 schrieb 12345swordy <alexanderheistermann gmail.com>:


 On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi
 wrote:


 Frankly speaking, my feeling is that D is becoming a
 horrible mess for the programmer...



 /Paolo


 How!? Please Explain. You need to demonstrate evidence instead
 of appeal to emotional fallacy by resorting to "feels".

 -Alexander


 The DIP increases consistency recalling that rvalues are
 accepted:

 - for the implicit 'this' parameter in methods
 - in foreach loop variables declared as ref

 No more special rules: rvalues are implicitly promoted to
 lvalues where needed.


 That's correct, but 'this' is already a special guest in C++
 style PL, with its own special rules. The support for ref
 variable in foreach loop.... can be removed (yup!), or made
 stricter.. no more inconsistency.


With UFCS as a super popular feature of D, 'this' is not really much of a
special guest at all.
It's just as much the first argument of a function as the first argument of
*any* UFCS call.



 It is kind of ironic that in order to do better than C++ you

 have to support most of what modern C++ compilers offer and end
 up having tons of unrelated features that make the language
 just as bloated as C++ after a decade of community feedback.
 It is a system PL. I think it needs to be this way and is a
 lot cleaner with basic data types and more expressive still,
 lacking a lot of C++'s legacy.


 There's a tension between Walter effort in turning D as a
 suitable language for memory correctness, and a good candidate
 for writing 'bug free rock solid software fast' and the
 continuous addition of features like this.


This isn't 'a feature' so much as lifting a restriction for the sake of a
bunch of uniformity and simplification.
I can't really see how you can find that disagreeable from your apparent
position...


Joke aside, I'm still on Jonathan side on this.

 Finally, sorry to use often the term 'feeling', and sorry for not
 being constructive: but really is a 'feeling'... I don't pretend
 to be right. So no problems in just ignoring that


It upsets me when people present strong opinions about this who literally
have no horse in the race. This is only really meaningful, and only affects
you if it actually affects you... It's clearly not important to you, or you
wouldn't be basing your opinion on *I kinda feel...*

Jonathan's argument is similar. He's worried about something that this
thread has tried and failed to determine exactly what is.
Meanwhile I think we have determined that the presumed practical trouble
cases are even less that I suspected up front.



 Jul 25 2018



 next sibling parent reply Jim Balter <Jim Balter.name>  writes:


On Wednesday, 25 July 2018 at 08:34:30 UTC, Manu wrote:
[snip]


 It upsets me when people present strong opinions about this who 
 literally have no horse in the race. This is only really 
 meaningful, and only affects you if it actually affects you... 
 It's clearly not important to you, or you wouldn't be basing 
 your opinion on *I kinda feel...*

 Jonathan's argument is similar. He's worried about something 
 that this
 thread has tried and failed to determine exactly what is.


I don't think that's fair. He has been quite specific about his 
concern and the kind of situations where there would be degraded 
behavior, and it clearly *is* important to him, and he certainly 
has a horse in the race. But I believe you are correct that those 
are cases where there's some unrelated bug that the ref parameter 
restriction just happens to catch, and that's not a good enough 
argument for keeping the restriction.


 Meanwhile I think we have determined that the presumed 
 practical trouble
 cases are even less that I suspected up front.


That's surprising; I didn't realize that you suspected practical 
trouble cases.



 Jul 25 2018



  parent  Manu <turkeyman gmail.com>  writes:


On Wed, 25 Jul 2018 at 04:50, Jim Balter via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 25 July 2018 at 08:34:30 UTC, Manu wrote:
 [snip]


 Meanwhile I think we have determined that the presumed
 practical trouble
 cases are even less that I suspected up front.


 That's surprising; I didn't realize that you suspected practical
 trouble cases.


My initial draft was written for 'ref const T ', and that was a
conservative choice because I felt the same unsubstantiated fear.
I think it's like this; there is a presumption that the feature was
made that way for a reason... so it *must* be protecting us against
something that it was designed to protect us against, right?

Others argued to remove the 'const' from my proposal, and then the
more I thought on that, and followed various experiments through, I
realised that my fears were unsubstantiated (I couldn't dream up
legitimate problem cases), and that mutable ref brought significant
additional value which actually strengthen the DIP substantially.

I still suspect it's possible to contrive a case where a bug may be
caught by the existing mechanic, but what we have determined by
following some of these cases presented here is that they're far less
likely than even I initially _imagined_ (yes, still working from the
same unsubstantiated fear), or that the bugs are actually unrelated
issues which should be addressed separately.


For instance, I can imagine a DIP to address the property concern that
we have identified here:

** Mutable-but-also-byval properties (ie, a property with a by-val
getter and also a setter) do not behave like a user expects when
supplied as ref arguments.
** Situation: a by-val getter passes an rval to a ref arg, which may
be mutated, and the results are lost.
** Propose: after the function call concludes, call the properties
setter, supplying the potentially mutated rval that the getter
returned.

This very simple semantic will cause non-ref properties to function
correctly even in the context of by-val getting and ref.
I actually think this is a very elegant solution to a wider class of
problem with properties.

...but this comment is dangerous. I REALLY don't want this point to
lead off on a tangent ;)
*considers deleting post... but clicks send anyway...*



 Jul 25 2018



prev sibling  parent reply Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Wednesday, 25 July 2018 at 08:34:30 UTC, Manu wrote:


 With UFCS as a super popular feature of D, 'this' is not really 
 much of a
 special guest at all.
 It's just as much the first argument of a function as the first 
 argument of
 *any* UFCS call.


Guido van Rossum has raised an objection on that a couple of 
decades ago...


 There's a tension between Walter effort in turning D as a 
 suitable language for memory correctness, and a good candidate 
 for writing 'bug free rock solid software fast' and the 
 continuous addition of features like this.


 This isn't 'a feature' so much as lifting a restriction for the 
 sake of a
 bunch of uniformity and simplification.
 I can't really see how you can find that disagreeable from your 
 apparent position...


That proposal is a 'Syntactic Sugar' feature, that simply hide 
what normally need to be explicitly coded: proved a temp rvalue, 
pass it to a callable taking ref. What you call 'simplification', 
I call it 'obfuscation'; what you call uniformity I call trying 
to spread a well justified restriction...


 Finally, sorry to use often the term 'feeling', and sorry for 
 not being constructive: but really is a 'feeling'... I don't 
 pretend to be right. So no problems in just ignoring that


 It upsets me when people present strong opinions about this who 
 literally have no horse in the race. This is only really 
 meaningful, and only affects you if it actually affects you... 
 It's clearly not important to you, or you wouldn't be basing 
 your opinion on *I kinda feel...*

 Jonathan's argument is similar. He's worried about something 
 that this
 thread has tried and failed to determine exactly what is.
 Meanwhile I think we have determined that the presumed 
 practical trouble cases are even less that I suspected up front.


Experience, in programming, has value: Walter is famous for his 
anecdotes on that.

D2 has already a lot of problematic stuff to solve, I not buy 
adding more (yes) features for the sake of an hypothetical 
'simplification' of what it's already possibile.

Just to be clear, I'm also against all the new proposed addition 
for, named parameter, new struct initialisation and so on....

No pun, really :-P

/Paolo



 Jul 25 2018



 next sibling parent  rikki cattermole <rikki cattermole.co.nz>  writes:


On 26/07/2018 12:40 AM, Paolo Invernizzi wrote:

 Just to be clear, I'm also against all the new proposed addition for, 
 named parameter, new struct initialisation and so on....


You'll go giddy once you hear about what I have planned after named 
parameters ;)

But seriously tho, just because a pattern of code can be done purely in 
library, doesn't mean it can't be done entirely better with a nice 
simple language feature which makes it considerably easier to learn and 
understand.

We do have to be careful, that we only try to go after features which 
are fairly well proven to our existing code and not for a theoretical 
improvement.



 Jul 25 2018



prev sibling  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 25 July 2018 at 12:40:16 UTC, Paolo Invernizzi 
wrote:


 That proposal is a 'Syntactic Sugar' feature, that simply hide 
 what normally need to be explicitly coded: proved a temp 
 rvalue, pass it to a callable taking ref. What you call 
 'simplification', I call it 'obfuscation'; what you call 
 uniformity I call trying to spread a well justified 
 restriction...
 /Paolo


A restriction which causes pointless redundant code for the 
caller who doesn't always have source code access. If my old 
teacher assistant taught me anything it is this: Redundant code 
is bad. You are literately forcing the programmer to create tmp 
variables that risk the possibility of being shadowed or worse, 
having its value change.

Your manual solution suggestion have it own set of problems. You 
might as well argue against the foreach statement, because its 
"obfuscation".

-Alexander



 Jul 25 2018



  parent reply Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Wednesday, 25 July 2018 at 13:36:38 UTC, 12345swordy wrote:

 On Wednesday, 25 July 2018 at 12:40:16 UTC, Paolo Invernizzi 
 wrote:


 That proposal is a 'Syntactic Sugar' feature, that simply hide 
 what normally need to be explicitly coded: proved a temp 
 rvalue, pass it to a callable taking ref. What you call 
 'simplification', I call it 'obfuscation'; what you call 
 uniformity I call trying to spread a well justified 
 restriction...
 /Paolo


 A restriction which causes pointless redundant code for the 
 caller who doesn't always have source code access. If my old 
 teacher assistant taught me anything it is this: Redundant code 
 is bad. You are literately forcing the programmer to create tmp 
 variables that risk the possibility of being shadowed or worse, 
 having its value change.


That's an opinion, naturally.

What it's "pointless redundant" for you, it is let's "let's force 
the programmer to think about what he is doing, passing an rvalue 
by ref" for me, at least.

At best, is "let's catch early some bugs (caused by other 
problems as Manu pointed out)", but as Jonathan states.


 Your manual solution suggestion have it own set of problems.


Set of problems as automatic promotion or conversion, as decades 
of problems with unsigned/signed have proved...


 You might as well argue against the foreach statement, because 
 its "obfuscation"


There's not a magic conversion between apples and oranges in a 
foreach loop... ref value apart.

/Paolo



 Jul 25 2018



  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 25 July 2018 at 16:43:38 UTC, Paolo Invernizzi 
wrote:


 That's an opinion, naturally.


No I am expressing an argument not an opinion.


 "let's force the programmer to think about what he is doing, 
 passing an rvalue by ref"


Nonsense, you have shown no evidence that they don't know what 
they are doing when making a automatic conversion. You might as 
well argue against the existence of var.


 At best, is "let's catch early some bugs (caused by other 
 problems as Manu pointed out)".


He also pointed it is own class of problems, as it can be 
replicated without ref.


 Set of problems as automatic promotion or conversion, as 
 decades of problems with unsigned/signed have proved...


False Equivalence. We are not discussing numeric overflows here.


 There's not a magic conversion between apples and oranges in a 
 foreach loop... ref value apart.


https://dlang.org/spec/type.html#usual-arithmetic-conversions
You where saying?


-Alexander



 Jul 25 2018



  parent reply Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Wednesday, 25 July 2018 at 17:52:00 UTC, 12345swordy wrote:

 On Wednesday, 25 July 2018 at 16:43:38 UTC, Paolo Invernizzi 
 wrote:


 That's an opinion, naturally.


 No I am expressing an argument not an opinion.


I don't know what vocabulary you are used to consult, but your 
'pointless' it's a plain and simple opinion. To me it's not 
pointless at all.


 "let's force the programmer to think about what he is doing, 
 passing an rvalue by ref"





 Nonsense, you have shown no evidence that they don't know what 
 they are doing when making a automatic conversion. You might as 
 well argue against the existence of var.


Actually, by definition, every bug is made by a programmer that 
THINK to know what he is doing... no? Aren't you. going a little 
too far in  judging?


 At best, is "let's catch early some bugs (caused by other 
 problems as Manu pointed out)".


 He also pointed it is own class of problems, as it can be 
 replicated without ref.


An so? Jonathan argumentation and mine is that are are. losing a 
way to catch such bugs earlier.


 Set of problems as automatic promotion or conversion, as 
 decades of problems with unsigned/signed have proved...


 False Equivalence. We are not discussing numeric overflows here.


It's not a false equivalence fallacy: all the discussion is about 
IMPLICIT conversion or rvalues to lvalues... your argumentation 
smell a little about strawmen (eheh)


 There's not a magic conversion between apples and oranges in a 
 foreach loop... ref value apart.


 https://dlang.org/spec/type.html#usual-arithmetic-conversions
 You where saying?


I'm saying that a foreach statement is easily lowered mentally in 
a for statement, and that implicitly converting between rvalue 
and lvalue is entirely another beast.

I will stop here... btw



 Jul 25 2018



  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 25 July 2018 at 19:55:50 UTC, Paolo Invernizzi 
wrote:


 I don't know what vocabulary you are used to consult,


ad hominem attacks is not an argument


 Actually, by definition, every bug is made by a programmer that 
 THINK to know what he is doing... no?


Avoiding burden of proof by shifting goal post.

 Aren't you. going a little too far in  judging?


loaded question


 An so?


He has explain the point in detail, go back and read his post.


 It's not a false equivalence fallacy: all the discussion is 
 about IMPLICIT conversion or rvalues to lvalues.


Yes it is, the issues regarding rvalue/lvalue conversion is not 
the same issues regarding the unsigned/signed conversion.

Alexander



 Jul 25 2018



  parent reply Manu <turkeyman gmail.com>  writes:


On Wed, 25 Jul 2018 at 13:55, 12345swordy via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 It's not a false equivalence fallacy: all the discussion is
 about IMPLICIT conversion or rvalues to lvalues.


 Yes it is, the issues regarding rvalue/lvalue conversion is not
 the same issues regarding the unsigned/signed conversion.


I don't want to encourage this tangent, but I do want to say; there's
no proposal of rvalue -> lvalue *conversion*.
The proposal is "ref accepts rvalues". There's no 'conversion'
anywhere in sight. That's not on the menu.



 Jul 25 2018



 next sibling parent  aliak <something something.com>  writes:


On Wednesday, 25 July 2018 at 21:55:00 UTC, Manu wrote:

 On Wed, 25 Jul 2018 at 13:55, 12345swordy via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 It's not a false equivalence fallacy: all the discussion is 
 about IMPLICIT conversion or rvalues to lvalues.


 Yes it is, the issues regarding rvalue/lvalue conversion is 
 not the same issues regarding the unsigned/signed conversion.


 I don't want to encourage this tangent, but I do want to say; 
 there's
 no proposal of rvalue -> lvalue *conversion*.
 The proposal is "ref accepts rvalues". There's no 'conversion'
 anywhere in sight. That's not on the menu.


Semantics? Call it a transformation. But it is an implicit 
changing of semantics.

Guess you encouraged it :p



 Jul 25 2018



prev sibling  parent reply Vijay Nayar <madric gmail.com>  writes:


On Wednesday, 25 July 2018 at 21:55:00 UTC, Manu wrote:

 On Wed, 25 Jul 2018 at 13:55, 12345swordy via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 It's not a false equivalence fallacy: all the discussion is 
 about IMPLICIT conversion or rvalues to lvalues.


 Yes it is, the issues regarding rvalue/lvalue conversion is 
 not the same issues regarding the unsigned/signed conversion.


 I don't want to encourage this tangent, but I do want to say; 
 there's
 no proposal of rvalue -> lvalue *conversion*.
 The proposal is "ref accepts rvalues". There's no 'conversion'
 anywhere in sight. That's not on the menu.


I know this is kinda out of the blue, but I really like to use 
"in" and "out" to clarify my function interfaces.  "out" seems to 
imply "ref", but it also initializes the value to its .init value.

Personally I don't get much mileage out of "out" assigning the 
".init" value.  The same keyword though could be used to satisfy 
both major use cases.

Maybe "ref" could accept rvalues as per the DIP, but "out ref" 
would not, allowing error checking for those who truly intend the 
argument to be used to store output (and this comes up a lot when 
a method needs to return multiple results).



 Nov 08 2018



  parent reply Manu <turkeyman gmail.com>  writes:


On Thu, Nov 8, 2018 at 3:10 AM Vijay Nayar via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 25 July 2018 at 21:55:00 UTC, Manu wrote:

 On Wed, 25 Jul 2018 at 13:55, 12345swordy via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 It's not a false equivalence fallacy: all the discussion is
 about IMPLICIT conversion or rvalues to lvalues.


 Yes it is, the issues regarding rvalue/lvalue conversion is
 not the same issues regarding the unsigned/signed conversion.


 I don't want to encourage this tangent, but I do want to say;
 there's
 no proposal of rvalue -> lvalue *conversion*.
 The proposal is "ref accepts rvalues". There's no 'conversion'
 anywhere in sight. That's not on the menu.


 I know this is kinda out of the blue, but I really like to use
 "in" and "out" to clarify my function interfaces.  "out" seems to
 imply "ref", but it also initializes the value to its .init value.

 Personally I don't get much mileage out of "out" assigning the
 ".init" value.  The same keyword though could be used to satisfy
 both major use cases.

 Maybe "ref" could accept rvalues as per the DIP, but "out ref"
 would not, allowing error checking for those who truly intend the
 argument to be used to store output (and this comes up a lot when
 a method needs to return multiple results).


I think it's reasonable that `out` could reject rvalues.



 Nov 08 2018



  parent reply Q. Schroll <qs.il.paperinik gmail.com>  writes:


On Friday, 9 November 2018 at 02:25:08 UTC, Manu wrote:

 I think it's reasonable that `out` could reject rvalues.


I have a question about lifetime. Consider your example

   ref int gun(return ref int y);

The current state `ref` also means that the return value of `gun` 
has an address.

Using `gun` on an rvalue, it makes an invisible variable in the 
caller's scope.

Will

   int* p = &(gun(10));

compile and `p` be referencing the invisible variable?



 Nov 09 2018



  parent reply Manu <turkeyman gmail.com>  writes:


On Fri, Nov 9, 2018 at 10:15 AM Q. Schroll via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Friday, 9 November 2018 at 02:25:08 UTC, Manu wrote:

 I think it's reasonable that `out` could reject rvalues.


 I have a question about lifetime. Consider your example

    ref int gun(return ref int y);

 The current state `ref` also means that the return value of `gun`
 has an address.

 Using `gun` on an rvalue, it makes an invisible variable in the
 caller's scope.

 Will

    int* p = &(gun(10));

 compile and `p` be referencing the invisible variable?


That's not  safe, for obvious reasons.



 Nov 09 2018



  parent reply Neia Neutuladh <neia ikeran.org>  writes:


On Fri, 09 Nov 2018 21:56:09 -0800, Manu wrote:

 That's not  safe, for obvious reasons.


I see three different ways of implementing DIP 1016, and exactly what's 
 safe differs in each.

The compiler can add in the temporary variable declaration at the start of 
the current scope, and the following would compile:

    int* gun(return ref p) { return &p; }
    void main()
    {
      // compiler adds: int __tmp = 10;
      int* p;
      // compiler converts to: p = gun(__tmp);
      p = gun(10);
    }

Or the compiler can add a new scope just for the temporary variable, 
entirely preventing escaping. It would be lowered to:

    int* p;
    {
      int __tmp = 10;
      p = gun(__tmp);
    }

And that wouldn't fly, obviously.

Or it could add the temporary variable just before the current statement, 
so you could write:

    writeln("hi");
    int* p = gun(10);

    // compiler sees:
    writeln("hi");
    int __tmp = 10;
    int* p;
    p = gun(__tmp);

But if you tried declaring the pointer first, you'd get:

    writeln("hi");
    int* p;
    p = gun(10);

    // compiler sees:
    writeln("hi");
    int* p;
    int __tmp = 10;
    p = gun(__tmp);

And -dip1000 requires the pointed-to value to come into existence before 
the pointer, so this doesn't work.



 Nov 09 2018



  parent  Manu <turkeyman gmail.com>  writes:


On Fri, Nov 9, 2018 at 11:00 PM Neia Neutuladh via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Or the compiler can add a new scope just for the temporary variable,
 entirely preventing escaping. It would be lowered to:

     int* p;
     {
       int __tmp = 10;
       p = gun(__tmp);
     }

 And that wouldn't fly, obviously.


This is the proper lowering, as detailed extensively in the DIP.
This operation is invalid, it's clearly an escaping local, and the
normal compiler semantics apply; there are no special rules prescribed
to this case:

void fun()  safe
{
    int* p;
    {
        int __tmp = 10;
        p = &__tmp;
    }
}

error : cannot take address of local `__tmp` in ` safe` function `fun`



 Nov 09 2018



prev sibling  parent  kinke <noone nowhere.com>  writes:


Thanks a lot, Manu, I'm a huge fan of this.

Wrt. binding rvalues to mutable refs, we could introduce 
something like `-transition=rval_to_mutable_ref` to have the 
compiler list all matching call sites.

Wrt. `auto ref`, I'd very much like to see its semantics change 
to 'pass this argument in the most efficient way' (depending on 
type and ABI, not just lvalue-ness of argument) at some point in 
the future.



 Jul 21 2018