www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - DIP 1016--ref T accepts r-values--Formal Assessment

reply Mike Parker <aldacron gmail.com> writes:
Walter and Andrei have declined to accept DIP 1016, "ref T 
accepts r-values", on the grounds that it has two fundamental 
flaws that would open holes in the language. They are not opposed 
to the feature in principle and suggested that a proposal that 
closes those holes and covers all the bases will have a higher 
chance of getting accepted.

You can read a summary of the Formal Assessment at the bottom of 
the document:

https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md
Jan 23 2019
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T 
 accepts r-values", on the grounds that it has two fundamental 
 flaws that would open holes in the language. They are not 
 opposed to the feature in principle and suggested that a 
 proposal that closes those holes and covers all the bases will 
 have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom 
 of the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md



 fun(10)
 ==>
 {
  T __temp0 = void;
  fun(__temp0 := 10);
 }


 The first problem the Language Maintainers identified with this 
 approach is that the rewrite is from an expression to a 
 statement, rendering it invalid.
 The expression should be rewritten as an expression to clarify 
 how it behaves in larger expressions.


But it does, or at least gives an example of: 
https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md#function-calls-as-arguments

The statement the expression is part of is "enscoped" and the 
temporaries live in the scope of that statement.
Jan 24 2019
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T 
 accepts r-values", on the grounds that it has two fundamental 
 flaws that would open holes in the language. They are not 
 opposed to the feature in principle and suggested that a 
 proposal that closes those holes and covers all the bases will 
 have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom 
 of the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md



 The second problem is the use of := (which the DIP Author 
 defines as representing "the initial construction, and not a 
 copy operation as would be expected if this code were written 
 with an = expression"). This approach shows its deficiencies in 
 the multiple arguments case; if the first constructor throws an 
 exception, all remaining values will be destroyed in the void 
 state as they never have the chance to become initialized.


Although not specified by the DIP, I think this could be easily 
remedied by saying that the order of construction is the same as 
if the temporaries were not bound to ref, i.e.

---
struct A {~this();} struct B{ ~this();}
A a();
B b();

void f(A a, B b);
void g(ref A a, ref B b);

f(a(),b());  //(1)
g(a(),b()); //(2)
---

and a() or b() may throw (and are pure), that (1) and (2) exhibit 
the same exception/destructor semantics.

Which is what I would expect to happen under no given 
specification (although specifying it is a good idea!). To put it 
another way, they should be constructed in place in the order 
they are used, and given they are a temporary, they can only be 
used once.
Jan 24 2019
parent reply kinke <noone nowhere.com> writes:
On Thursday, 24 January 2019 at 09:04:41 UTC, Nicholas Wilson 
wrote:

 On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 The second problem is the use of := (which the DIP Author 
 defines as representing "the initial construction, and not a 
 copy operation as would be expected if this code were written 
 with an = expression"). This approach shows its deficiencies 
 in the multiple arguments case; if the first constructor 
 throws an exception, all remaining values will be destroyed in 
 the void state as they never have the chance to become 
 initialized.


 Although not specified by the DIP, I think this could be easily 
 remedied by saying that the order of construction is the same 
 as if the temporaries were not bound to ref, i.e.

 ---
 struct A {~this();} struct B{ ~this();}
 A a();
 B b();

 void f(A a, B b);
 void g(ref A a, ref B b);

 f(a(),b());  //(1)
 g(a(),b()); //(2)
 ---

 and a() or b() may throw (and are pure), that (1) and (2) 
 exhibit the same exception/destructor semantics.


Describing this stuff in detail (rewritten expression?!), isn't 
trivial and requires knowledge about how calls and 
construction/destruction of argument expressions works.
E.g., the f() call in the code above is lowered to (-vcg-ast):

(bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = b();)) 
, __gate = true , f(__pfx, __pfy);

Here, (only seemingly unused) temporary `__gate` is used to 
control the destruction of temporaries with dtors (here just the 
`__pfx` arg, as `__pfy` is a special case [no potentially 
throwing later argument expressions...]) at the caller side 
(false => destruct; true => skip destruction, as it has been 
moved successfully to the callee, which destructed it). The dtor 
expressions of these temporaries (e.g., `__gate || __pfx.~this()` 
for `__pfx`) aren't visible with `-vg-ast`.

With this DIP, *all* rvalues passed by ref must be lowered to 
temporaries. In case they require destruction, the only 
difference wrt. the by-value case is that they are *always* 
destructed by the caller (after the call, or if an exception is 
thrown while they are in scope), i.e., their destruction isn't 
controlled by `__gate`.
Jan 24 2019
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 24 January 2019 at 21:03:29 UTC, kinke wrote:

 Describing this stuff in detail (rewritten expression?!), isn't 
 trivial and requires knowledge about how calls and 
 construction/destruction of argument expressions works.
 E.g., the f() call in the code above is lowered to (-vcg-ast):

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = 
 b();)) , __gate = true , f(__pfx, __pfy);

 Here, (only seemingly unused) temporary `__gate` is used to 
 control the destruction of temporaries with dtors (here just 
 the `__pfx` arg, as `__pfy` is a special case [no potentially 
 throwing later argument expressions...]) at the caller side 
 (false => destruct; true => skip destruction, as it has been 
 moved successfully to the callee, which destructed it). The 
 dtor expressions of these temporaries (e.g., `__gate || 
 __pfx.~this()` for `__pfx`) aren't visible with `-vg-ast`.

 With this DIP, *all* rvalues passed by ref must be lowered to 
 temporaries. In case they require destruction, the only 
 difference wrt. the by-value case is that they are *always* 
 destructed by the caller (after the call, or if an exception is 
 thrown while they are in scope), i.e., their destruction isn't 
 controlled by `__gate`.


Exactly, doing something that could result in a different outcome 
would be a disaster.
Jan 24 2019
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 1:05 PM kinke via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 24 January 2019 at 09:04:41 UTC, Nicholas Wilson
 wrote:

 On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 The second problem is the use of := (which the DIP Author
 defines as representing "the initial construction, and not a
 copy operation as would be expected if this code were written
 with an = expression"). This approach shows its deficiencies
 in the multiple arguments case; if the first constructor
 throws an exception, all remaining values will be destroyed in
 the void state as they never have the chance to become
 initialized.


 Although not specified by the DIP, I think this could be easily
 remedied by saying that the order of construction is the same
 as if the temporaries were not bound to ref, i.e.

 ---
 struct A {~this();} struct B{ ~this();}
 A a();
 B b();

 void f(A a, B b);
 void g(ref A a, ref B b);

 f(a(),b());  //(1)
 g(a(),b()); //(2)
 ---

 and a() or b() may throw (and are pure), that (1) and (2)
 exhibit the same exception/destructor semantics.


 Describing this stuff in detail (rewritten expression?!), isn't
 trivial and requires knowledge about how calls and
 construction/destruction of argument expressions works.


Sure, it's not 'trivial', but it is 'simple' in that it's isolated,
and it only affects the part of the DIP that defines the rewrite
semantic. It doesn't lead to "practically a completely differnet DIP"
as was suggested.
Changing the detail of the rewrite such that it has the proper effect
required by the surrounding text and handles exceptions correctly can
probably be done in such a way that not a single line of text requires
any changes.
Jan 24 2019
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = b();)) , __gate =
true 
 , f(__pfx, __pfy);


There must be an individual gate for each of __pfx and pfy. With the rewrite 
above, if b() throws then _pfx won't be destructed.

(All this rigamarole is part of why exception handling isn't free, even in the 
happy case.)
Jan 24 2019
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright wrote:

 On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = 
 b();)) , __gate = true , f(__pfx, __pfy);


 There must be an individual gate for each of __pfx and pfy. 
 With the rewrite above, if b() throws then _pfx won't be 
 destructed.

 (All this rigamarole is part of why exception handling isn't 
 free, even in the happy case.)


Indeed, and thats why it should be have exactly the same as if 
there were no `ref` involved, except for the not copying stuff. 
Behaving and differently is, a) wrong and, b) going to cause a 
lot of confusion.
Jan 24 2019
prev sibling parent reply kinke <noone nowhere.com> writes:
On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright wrote:

 On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = 
 b();)) , __gate = true , f(__pfx, __pfy);


 There must be an individual gate for each of __pfx and pfy. 
 With the rewrite above, if b() throws then _pfx won't be 
 destructed.


There is no individual gate, there's just one to rule the 
caller-destruction of all temporaries. That's the current state, 
and there's no need for that to change. I was trying to say that 
a rewrite as expression, as requested as part of the assessment, 
clearly isn't enough, as the dtor expressions aren't visible this 
way, and neither is the scoping (when the dtor expression of 
`__pfx` comes into play etc.).
Jan 25 2019
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/25/19 5:57 AM, kinke wrote:

 On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright wrote:

 On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = b();)) , 
 __gate = true , f(__pfx, __pfy);


 There must be an individual gate for each of __pfx and pfy. With the 
 rewrite above, if b() throws then _pfx won't be destructed.


 
 There is no individual gate, there's just one to rule the 
 caller-destruction of all temporaries. That's the current state, and 
 there's no need for that to change. I was trying to say that a rewrite 
 as expression, as requested as part of the assessment, clearly isn't 
 enough, as the dtor expressions aren't visible this way, and neither is 
 the scoping (when the dtor expression of `__pfx` comes into play etc.).


I think the point of the DIP is not to lower expressions. It makes no 
sense to, they have to be statements (just like all temporaries live 
until the end of a statement).

-Steve
Jan 25 2019
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/25/2019 2:57 AM, kinke wrote:

 On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright wrote:

 On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = b();)) , __gate = 
 true , f(__pfx, __pfy);


 There must be an individual gate for each of __pfx and pfy. With the rewrite 
 above, if b() throws then _pfx won't be destructed.


 
 There is no individual gate, there's just one to rule the caller-destruction
of 
 all temporaries.


What happens, then, when b() throws?
Jan 25 2019
parent reply kinke <noone nowhere.com> writes:
On Friday, 25 January 2019 at 19:08:55 UTC, Walter Bright wrote:

 On 1/25/2019 2:57 AM, kinke wrote:

 On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright 
 wrote:

 On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = 
 b();)) , __gate = true , f(__pfx, __pfy);


 There must be an individual gate for each of __pfx and pfy. 
 With the rewrite above, if b() throws then _pfx won't be 
 destructed.


 
 There is no individual gate, there's just one to rule the 
 caller-destruction of all temporaries.


 What happens, then, when b() throws?


`__pfx` goes out of scope, and is dtor expression 
(cleanup/finally) is run as part of stack unwinding. Rewritten as 
block statement:

{
   bool __gate = false;
   A __pfx = a();
   try {
     B __pfy = b(); // may throw
     __gate = true;
     return f(__pfx, __pfy); // move args to callee
   } finally {
     __gate || __pfx.__xdtor(); // only destruct if not moved to 
callee
   }
}

With this DIP, the g() call (both rvalue args passed by ref) 
would now become:

{
   A __pfx = a();
   try {
     B __pfy = b(); // may throw
     try {
       return g(__pfx, __pfy); // pass by ref
     } finally {
       __pfy.__xdtor(); // always destructed by caller
     }
   } finally {
     __pfx.__xdtor();
   }
}
Jan 25 2019
parent reply Neia Neutuladh <neia ikeran.org> writes:
On Fri, 25 Jan 2019 23:08:52 +0000, kinke wrote:


 On Friday, 25 January 2019 at 19:08:55 UTC, Walter Bright wrote:

 On 1/25/2019 2:57 AM, kinke wrote:

 On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright wrote:

 On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy =
 b();)) , __gate = true , f(__pfx, __pfy);


 There must be an individual gate for each of __pfx and pfy.
 With the rewrite above, if b() throws then _pfx won't be destructed.


 
 There is no individual gate, there's just one to rule the
 caller-destruction of all temporaries.


 What happens, then, when b() throws?


 
 `__pfx` goes out of scope, and is dtor expression (cleanup/finally) is
 run as part of stack unwinding. Rewritten as block statement:


And nested calls are serialized as you'd expect:

int foo(ref S i, ref S j);
S bar(ref S i, ref S j);
S someRvalue(int i);

foo(
    bar(someRvalue(1), someRvalue(2)),
    someRvalue(4));

// translates to something like:
{
    bool __gate1 = false;
    S __tmp1 = void;
    S __tmp2 = void;
    S __tmp3 = void;
    __tmp1 = someRvalue(1);
    try
    {
        __tmp2 = someRvalue(2);
        __gate1 = true;
        __tmp3 = bar(__tmp1, __tmp2);
    }
    finally
    {
        if (!__gate1) __tmp1.__xdtor();
    }
    S __tmp4 = void;
    bool __gate2 = false;
    try
    {
        __tmp4 = someRvalue(4);
        __gate2 = true;
        return foo(__tmp3, __tmp4);
    }
    finally
    {
        if (!__gate2)
        {
            __tmp3.__xdtor();
        }
    }
}
Jan 25 2019
parent reply Manu <turkeyman gmail.com> writes:
On Fri, Jan 25, 2019 at 4:20 PM Neia Neutuladh via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Fri, 25 Jan 2019 23:08:52 +0000, kinke wrote:


 On Friday, 25 January 2019 at 19:08:55 UTC, Walter Bright wrote:

 On 1/25/2019 2:57 AM, kinke wrote:

 On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright wrote:

 On 1/24/2019 1:03 PM, kinke wrote:

 (bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy =
 b();)) , __gate = true , f(__pfx, __pfy);


 There must be an individual gate for each of __pfx and pfy.
 With the rewrite above, if b() throws then _pfx won't be destructed.


 There is no individual gate, there's just one to rule the
 caller-destruction of all temporaries.


 What happens, then, when b() throws?


 `__pfx` goes out of scope, and is dtor expression (cleanup/finally) is
 run as part of stack unwinding. Rewritten as block statement:


 And nested calls are serialized as you'd expect:

 int foo(ref S i, ref S j);
 S bar(ref S i, ref S j);
 S someRvalue(int i);

 foo(
     bar(someRvalue(1), someRvalue(2)),
     someRvalue(4));

 // translates to something like:
 {
     bool __gate1 = false;
     S __tmp1 = void;
     S __tmp2 = void;
     S __tmp3 = void;
     __tmp1 = someRvalue(1);
     try
     {
         __tmp2 = someRvalue(2);
         __gate1 = true;
         __tmp3 = bar(__tmp1, __tmp2);
     }
     finally
     {
         if (!__gate1) __tmp1.__xdtor();
     }
     S __tmp4 = void;
     bool __gate2 = false;
     try
     {
         __tmp4 = someRvalue(4);
         __gate2 = true;
         return foo(__tmp3, __tmp4);
     }
     finally
     {
         if (!__gate2)
         {
             __tmp3.__xdtor();
         }
     }
 }


Is this fine?

Given above example:

int foo(ref S i, ref S j);
S bar(ref S i, ref S j);
S someRvalue(int i);

foo(
    bar(someRvalue(1), someRvalue(2)),
    someRvalue(4));

===>

{
  S __tmp0 = someRvalue(1);
  S __tmp1 = someRvalue(2);
  S __tmp2 = bar(__tmp0, __tmp1);
  S __tmp3 = someRvalue(4);
  foo(__tmp2, __tmp3);
}

Removing the `void` stuff end expanding such that the declaration +
initialisation is at the appropriate moments; any function can throw
normally, and the unwind works naturally?
Jan 25 2019
parent reply Neia Neutuladh <neia ikeran.org> writes:
On Fri, 25 Jan 2019 18:14:56 -0800, Manu wrote:

 Removing the `void` stuff end expanding such that the declaration +
 initialisation is at the appropriate moments; any function can throw
 normally, and the unwind works naturally?


The contention was that, if the arguments are constructed properly, 
ownership is given to the called function, which is responsible for 
calling destructors. But if the argument construction fails, the caller is 
responsible for calling destructors.

I'm not sure what the point of that was. The called function doesn't own 
its parameters and shouldn't ever call destructors. So now I'm confused.
Jan 25 2019
parent Manu <turkeyman gmail.com> writes:
On Fri, Jan 25, 2019 at 6:50 PM Neia Neutuladh via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Fri, 25 Jan 2019 18:14:56 -0800, Manu wrote:

 Removing the `void` stuff end expanding such that the declaration +
 initialisation is at the appropriate moments; any function can throw
 normally, and the unwind works naturally?


 The contention was that, if the arguments are constructed properly,
 ownership is given to the called function, which is responsible for
 calling destructors.


No, that was never the intent, and certainly not written anywhere.
Ownership is assigned the the calling scope that we introduce
surrounding the statement. That's where the temporaries declared; I
didn't consider that ownership unclear.


 I'm not sure what the point of that was. The called function doesn't own
 its parameters and shouldn't ever call destructors. So now I'm confused.


Correct. You're not confused. The callee does NOT own ref parameters.
Jan 25 2019
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T 
 accepts r-values", on the grounds that it has two fundamental 
 flaws that would open holes in the language. They are not 
 opposed to the feature in principle and suggested that a 
 proposal that closes those holes and covers all the bases will 
 have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom 
 of the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md



 void atomicIncrement(ref shared long x);
 atomicIncrement(myInt);


Raises a good point, not covered by  disable where the intent is 
to modify it and modifying a temporary is wrong. `out ref` 
perhaps?
Jan 24 2019
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 1:25 AM Nicholas Wilson via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T
 accepts r-values", on the grounds that it has two fundamental
 flaws that would open holes in the language. They are not
 opposed to the feature in principle and suggested that a
 proposal that closes those holes and covers all the bases will
 have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom
 of the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md



 void atomicIncrement(ref shared long x);
 atomicIncrement(myInt);


 Raises a good point, not covered by  disable where the intent is
 to modify it and modifying a temporary is wrong. `out ref`
 perhaps?


Actually, this was discussed, but somehow this piece of discussion
didn't get folded back it to the DIP. It is mentioned in the
'Community Review Round 1' digest though.

We discussed and concluded that one mechanism to mitigate this issue
was already readily available, and it's just that 'out' gains a much
greater sense of identity (which is actually a positive side-effect if
you ask me!).
You have a stronger motivation to use 'out' appropriately, because it
can issue compile errors if you accidentally supply an rvalue.

That doesn't address the specific `atomicIncrement` case here, but now
we're in VERY niche territory; we analysed a lot of cases, and
concluded that such cases were relatively few, and other choices exist
to mitigate those cases.
There are cases that want to do mutation to rvalues (like in pipeline
functions), and then most cases can use 'out' instead. Remaining cases
are quite hard to find, and in this particular case, I'd suggest that
`atomicIncrement`, a very low-level implementation-detail function,
should just receive a pointer.
Jan 24 2019
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 24 January 2019 at 09:49:14 UTC, Manu wrote:

 On Thu, Jan 24, 2019 at 1:25 AM Nicholas Wilson via We 
 discussed and concluded that one mechanism to mitigate this 
 issue
 was already readily available, and it's just that 'out' gains a 
 much
 greater sense of identity (which is actually a positive 
 side-effect if
 you ask me!).
 You have a stronger motivation to use 'out' appropriately, 
 because it
 can issue compile errors if you accidentally supply an rvalue.


True I forgot about that.


 That doesn't address the specific `atomicIncrement` case here, 
 but now
 we're in VERY niche territory; we analysed a lot of cases, and
 concluded that such cases were relatively few, and other 
 choices exist
 to mitigate those cases.
 There are cases that want to do mutation to rvalues (like in 
 pipeline
 functions), and then most cases can use 'out' instead. 
 Remaining cases
 are quite hard to find, and in this particular case, I'd 
 suggest that
 `atomicIncrement`, a very low-level implementation-detail 
 function,
 should just receive a pointer.


Probably.

Oh well, I'll add this to the long list of things to make sure is 
covered at dconf by the foundation.
Jan 24 2019
prev sibling parent reply kinke <noone nowhere.com> writes:
On Thursday, 24 January 2019 at 09:49:14 UTC, Manu wrote:

 We discussed and concluded that one mechanism to mitigate this 
 issue
 was already readily available, and it's just that 'out' gains a 
 much
 greater sense of identity (which is actually a positive 
 side-effect if
 you ask me!).
 You have a stronger motivation to use 'out' appropriately, 
 because it
 can issue compile errors if you accidentally supply an rvalue.


`out` with current semantics cannot be used as drop-in 
replacement for shared in-/output ref params, as `out` params are 
default-initialized on entry. Ignoring backwards compatibility 
for a second, I think getting rid of that would actually be 
beneficial (most args are probably already default-initialized by 
the callee in the line above the call...) - and I'd prefer an 

the side effect clearly visible.

I'd have otherwise proposed a ` noRVal` param UDA, but redefining 
`out` is too tempting indeed. ;)
Jan 24 2019
next sibling parent kinke <noone nowhere.com> writes:
On Thursday, 24 January 2019 at 20:01:45 UTC, kinke wrote:

 (most args are probably already default-initialized by the 
 callee in the line above the call...)


Should be 'by the callER' of course.
Jan 24 2019
prev sibling next sibling parent reply Rubn <where is.this> writes:
On Thursday, 24 January 2019 at 20:01:45 UTC, kinke wrote:

 On Thursday, 24 January 2019 at 09:49:14 UTC, Manu wrote:

 We discussed and concluded that one mechanism to mitigate this 
 issue
 was already readily available, and it's just that 'out' gains 
 a much
 greater sense of identity (which is actually a positive 
 side-effect if
 you ask me!).
 You have a stronger motivation to use 'out' appropriately, 
 because it
 can issue compile errors if you accidentally supply an rvalue.


 `out` with current semantics cannot be used as drop-in 
 replacement for shared in-/output ref params, as `out` params 
 are default-initialized on entry. Ignoring backwards 
 compatibility for a second, I think getting rid of that would 
 actually be beneficial (most args are probably already 
 default-initialized by the callee in the line above the 
 call...) - and I'd prefer an explicitly required `out` at the 


 I'd have otherwise proposed a ` noRVal` param UDA, but 
 redefining `out` is too tempting indeed. ;)


Yah I'd like this approach way better and it could be an error if 
the function didn't assign the variable a value instead. Rather 
than just default initializing it.

void test(out int value) {
     // error: value must be assigned a value
}

In the event the value passed isn't initialized. Can be dangerous 
if it is a struct with a destructor though. If they void 
initialized it, but I guess that's a problem with the current 
implementation as well.
Jan 24 2019
parent kinke <noone nowhere.com> writes:
On Thursday, 24 January 2019 at 21:57:57 UTC, Rubn wrote:

 it could be an error if the function didn't assign the variable 
 a value instead


I don't like that, conditional mutations like this should work 
too:

void conditionalIncrement(bool condition, out long value) {
   if (condition)
     ++value;
}

conditionalIncrement(true, 10); // not allowed, rvalue

long value = 10;
conditionalIncrement(true, out value); // fine
Jan 24 2019
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 12:05 PM kinke via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 24 January 2019 at 09:49:14 UTC, Manu wrote:

 We discussed and concluded that one mechanism to mitigate this
 issue
 was already readily available, and it's just that 'out' gains a
 much
 greater sense of identity (which is actually a positive
 side-effect if
 you ask me!).
 You have a stronger motivation to use 'out' appropriately,
 because it
 can issue compile errors if you accidentally supply an rvalue.


 `out` with current semantics cannot be used as drop-in
 replacement for shared in-/output ref params, as `out` params are
 default-initialized on entry.


Shared in/out functions are very rare by contrast to out parameters.


 Ignoring backwards compatibility
 for a second, I think getting rid of that would actually be
 beneficial (most args are probably already default-initialized by
 the callee in the line above the call...) - and I'd prefer an

 the side effect clearly visible.

 I'd have otherwise proposed a ` noRVal` param UDA, but redefining
 `out` is too tempting indeed. ;)


Maybe... but there are satisfying options for basically any case we
could imagine; and worst case, use a pointer rather than ref.
Adding stuff like  norval feels heavy-handed, and I personally judge
this issue as being severe enough to warrant that baggage.

What are some legit cases where, assuming a world where we want to
avoid naked ref in cases where we want to receive compile errors when
users pass rvalues, aren't satisfied by other options?
Jan 24 2019
parent reply kinke <noone nowhere.com> writes:
On Thursday, 24 January 2019 at 22:38:01 UTC, Manu wrote:

 Shared in/out functions are very rare by contrast to out 
 parameters.


The code I write is the exact opposite of your perception - some 
occasional side-effect-mutations of params, and almost no stuff 
'returned' as out params.


 What are some legit cases where, assuming a world where we want 
 to avoid naked ref in cases where we want to receive compile 
 errors when users pass rvalues, aren't satisfied by other 
 options?


Proposed `out` semantics:
---
void increment(out long value) { ++value; }
increment(out value);
---

vs. pointer version with current `out` semantics:
---
void increment(long* pValue) { ++(*pValue); }
increment(&value);
---

The pointer workaround is both ugly (C) and unsafe (you can pass 
null).
Jan 24 2019
parent reply Rubn <where is.this> writes:
On Thursday, 24 January 2019 at 23:18:11 UTC, kinke wrote:

 Proposed `out` semantics:
 ---
 void increment(out long value) { ++value; }
 increment(out value);
 ---

 vs. pointer version with current `out` semantics:
 ---
 void increment(long* pValue) { ++(*pValue); }
 increment(&value);
 ---

 The pointer workaround is both ugly (C) and unsafe (you can 
 pass null).


 safe void safestFunction() {
     int* ptr;
     increment(out *ptr); // can also pass null to ref/out even in 
 safe
}

It's probably going to be a hard sell to change the behavior of 
out now as well. It'd break quite a bit of code I think, did a 
search through druntime and phobos and quite a few functions use 
it. Maybe user code uses it less, I know I never use it.
Jan 24 2019
parent Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 3:50 PM Rubn via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 24 January 2019 at 23:18:11 UTC, kinke wrote:

 Proposed `out` semantics:
 ---
 void increment(out long value) { ++value; }
 increment(out value);
 ---

 vs. pointer version with current `out` semantics:
 ---
 void increment(long* pValue) { ++(*pValue); }
 increment(&value);
 ---

 The pointer workaround is both ugly (C) and unsafe (you can
 pass null).


  safe void safestFunction() {
      int* ptr;
      increment(out *ptr); // can also pass null to ref/out even in
  safe
 }

 It's probably going to be a hard sell to change the behavior of
 out now as well. It'd break quite a bit of code I think, did a
 search through druntime and phobos and quite a few functions use
 it. Maybe user code uses it less, I know I never use it.


I think any issues with `out` are tangential though. `out` does 'work'
right now, and it's a valid way to address the concern with respect to
one broad use case for ref.
Theoretically, `out` is the right solution for that particular class
of calling contexts, and any further issued with `out` should be taken
as a separate issue/discussion.
Jan 24 2019
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/24/2019 12:01 PM, kinke wrote:

 `out` params are default-initialized on entry. 
 Ignoring backwards compatibility for a second, I think getting rid of that
would 
 actually be beneficial (most args are probably already default-initialized by 
 the callee in the line above the call...)


The compiler should elide the default-initializations before the call (although 
it currently does not).

The 'out' comes from IDL (Interface Definition Language) and by using out 
parameters it is directly convertible to/from IDL.

Even earlier, 'out' comes from Ada (!)

 From an efficiency standpoint, having the initialization occur in the function 
is better than all that duplicated initialization code in all the callers.
Jan 24 2019
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 1/24/19 3:01 PM, kinke wrote:

 On Thursday, 24 January 2019 at 09:49:14 UTC, Manu wrote:

 We discussed and concluded that one mechanism to mitigate this issue
 was already readily available, and it's just that 'out' gains a much
 greater sense of identity (which is actually a positive side-effect if
 you ask me!).
 You have a stronger motivation to use 'out' appropriately, because it
 can issue compile errors if you accidentally supply an rvalue.


 
 `out` with current semantics cannot be used as drop-in replacement for 
 shared in-/output ref params, as `out` params are default-initialized on 
 entry. Ignoring backwards compatibility for a second, I think getting 
 rid of that would actually be beneficial (most args are probably already 
 default-initialized by the callee in the line above the call...) - and 

 make the side effect clearly visible.
 
 I'd have otherwise proposed a ` noRVal` param UDA, but redefining `out` 
 is too tempting indeed. ;)


It seems to me that a proposal adding the " rvalue" attribute in 
function signatures to each parameter that would accept either an rvalue 
or an lvalue would be easy to argue.

- No exposing existing APIs to wrong uses
- The function's writer makes the decision ("I'm fine with this function 
taking an rvalue")
- Appears in the function's documentation
- Syntax is light and localized where it belongs
- Scales well with number of parameters
- Transparent to callers

Whether existing keyword combinations ("in", "out", "ref" etc) could be 
used is a secondary point.

The advantage is there's a simple and clear path forward for API 
definition and use.


Andrei
Jan 28 2019
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 1/28/19 1:00 PM, Andrei Alexandrescu wrote:

 On 1/24/19 3:01 PM, kinke wrote:

 On Thursday, 24 January 2019 at 09:49:14 UTC, Manu wrote:

 We discussed and concluded that one mechanism to mitigate this issue
 was already readily available, and it's just that 'out' gains a much
 greater sense of identity (which is actually a positive side-effect if
 you ask me!).
 You have a stronger motivation to use 'out' appropriately, because it
 can issue compile errors if you accidentally supply an rvalue.


 `out` with current semantics cannot be used as drop-in replacement for 
 shared in-/output ref params, as `out` params are default-initialized 
 on entry. Ignoring backwards compatibility for a second, I think 
 getting rid of that would actually be beneficial (most args are 
 probably already default-initialized by the callee in the line above 
 the call...) - and I'd prefer an explicitly required `out` at the call 


 I'd have otherwise proposed a ` noRVal` param UDA, but redefining 
 `out` is too tempting indeed. ;)


 
 It seems to me that a proposal adding the " rvalue" attribute in 
 function signatures to each parameter that would accept either an rvalue 
 or an lvalue would be easy to argue.
 
 - No exposing existing APIs to wrong uses
 - The function's writer makes the decision ("I'm fine with this function 
 taking an rvalue")
 - Appears in the function's documentation
 - Syntax is light and localized where it belongs
 - Scales well with number of parameters
 - Transparent to callers
 
 Whether existing keyword combinations ("in", "out", "ref" etc) could be 
 used is a secondary point.
 
 The advantage is there's a simple and clear path forward for API 
 definition and use.
 
 
 Andrei


One more thought.

The main danger is restricted to a specific conversion: lvalue of type T 
is converted to ref of type U. That way both the caller and the function 
writer believe the value gets updated, when in fact it doesn't. Consider:

real modf(real x, ref real i);

Stores integral part in i, returns the fractional part. At this point 
there are two liabilities:

1. User passes the wrong parameter type:

double integral;
double frac = modf(x, integral);
// oops, integral is always NaN

The function silently converts integral from double to real and passes 
the resulting temporary into the function. The temporary is filled and 
lost, leaving user's value unchanged.

2. The API gets changed:

// Fine, let's use double
real modf(real x, ref double i);

At this point all correct callers are silently broken - everybody who 
correctly used a real for the integral part now has their call broken 
(real implicitly converts to a double temporary, and the change does not 
propagate to the user's value).

(If the example looks familiar it may be because of 
https://dlang.org/library/std/math/modf.html.)

So it seems that the real problem is that the participants wrongly 
believe an lvalue is updated.

But let's say the caller genuinely doesn't care about the integral part. 
To do so is awkward:

real unused;
double frac = modf(x, unused);

That code isn't any better or less dangerous than:

double frac = modf(x, double());

Here the user created willingly created an unnamed temporary of type 
double. Given that there's no doubt the user is not interested in that 
value after the call, the compiler could (in a proposed semantics) allow 
the conversion of the unnamed temporary to ref.

TL;DR: it could be argued that the only dangerous conversions are lvalue 
-> temp rvalue -> ref, so only disable those. The conversion rvalue -> 
temp rvalue -> ref is not dangerous because the starting value on the 
caller side could not be inspected after the call anyway.


Andrei
Jan 28 2019
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/28/19 2:58 PM, Andrei Alexandrescu wrote:

 TL;DR: it could be argued that the only dangerous conversions are lvalue 
 -> temp rvalue -> ref, so only disable those. The conversion rvalue -> 
 temp rvalue -> ref is not dangerous because the starting value on the 
 caller side could not be inspected after the call anyway.


I agree with you. It's one thing for the caller to pass in an rvalue 
that can clearly no longer be accessed. It's another thing to pass in an 
lvalue and have it "update" a temporary instead.

I already see this kind of bug all the time with alias this.

-Steve
Jan 28 2019
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/28/19 5:23 PM, Steven Schveighoffer wrote:

 I already see this kind of bug all the time with alias this.


Can you please post more detail? It may be of relevance to future work.
Jan 28 2019
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/28/19 9:15 PM, Andrei Alexandrescu wrote:

 On 1/28/19 5:23 PM, Steven Schveighoffer wrote:

 I already see this kind of bug all the time with alias this.


 
 Can you please post more detail? It may be of relevance to future work.


Any time you have the alias this, then you can get confused when calling 
a function expecting it to be sent as the original type, but the alias 
this is used instead. In cases where both are seemingly accepted by 
overloads, then it can be confusing which overload is used. Sometimes 
it's as simple as the overload that you were expecting to be called 
won't compile or is deselected by constraints. But the code still 
compiles and runs, just does something different from what you expected.

I meant that the effect is similar to what you were noting.

-Steve
Jan 28 2019
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Mon, Jan 28, 2019 at 12:00 PM Andrei Alexandrescu via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/28/19 1:00 PM, Andrei Alexandrescu wrote:

 On 1/24/19 3:01 PM, kinke wrote:

 On Thursday, 24 January 2019 at 09:49:14 UTC, Manu wrote:

 We discussed and concluded that one mechanism to mitigate this issue
 was already readily available, and it's just that 'out' gains a much
 greater sense of identity (which is actually a positive side-effect if
 you ask me!).
 You have a stronger motivation to use 'out' appropriately, because it
 can issue compile errors if you accidentally supply an rvalue.


 `out` with current semantics cannot be used as drop-in replacement for
 shared in-/output ref params, as `out` params are default-initialized
 on entry. Ignoring backwards compatibility for a second, I think
 getting rid of that would actually be beneficial (most args are
 probably already default-initialized by the callee in the line above
 the call...) - and I'd prefer an explicitly required `out` at the call


 I'd have otherwise proposed a ` noRVal` param UDA, but redefining
 `out` is too tempting indeed. ;)


 It seems to me that a proposal adding the " rvalue" attribute in
 function signatures to each parameter that would accept either an rvalue
 or an lvalue would be easy to argue.

 - No exposing existing APIs to wrong uses
 - The function's writer makes the decision ("I'm fine with this function
 taking an rvalue")
 - Appears in the function's documentation
 - Syntax is light and localized where it belongs
 - Scales well with number of parameters
 - Transparent to callers

 Whether existing keyword combinations ("in", "out", "ref" etc) could be
 used is a secondary point.

 The advantage is there's a simple and clear path forward for API
 definition and use.


 Andrei


 One more thought.

 The main danger is restricted to a specific conversion: lvalue of type T
 is converted to ref of type U. That way both the caller and the function
 writer believe the value gets updated, when in fact it doesn't. Consider:

 real modf(real x, ref real i);

 Stores integral part in i, returns the fractional part. At this point
 there are two liabilities:

 1. User passes the wrong parameter type:

 double integral;
 double frac = modf(x, integral);
 // oops, integral is always NaN

 The function silently converts integral from double to real and passes
 the resulting temporary into the function. The temporary is filled and
 lost, leaving user's value unchanged.

 2. The API gets changed:

 // Fine, let's use double
 real modf(real x, ref double i);

 At this point all correct callers are silently broken - everybody who
 correctly used a real for the integral part now has their call broken
 (real implicitly converts to a double temporary, and the change does not
 propagate to the user's value).

 (If the example looks familiar it may be because of
 https://dlang.org/library/std/math/modf.html.)

 So it seems that the real problem is that the participants wrongly
 believe an lvalue is updated.

 But let's say the caller genuinely doesn't care about the integral part.
 To do so is awkward:

 real unused;
 double frac = modf(x, unused);

 That code isn't any better or less dangerous than:

 double frac = modf(x, double());

 Here the user created willingly created an unnamed temporary of type
 double. Given that there's no doubt the user is not interested in that
 value after the call, the compiler could (in a proposed semantics) allow
 the conversion of the unnamed temporary to ref.

 TL;DR: it could be argued that the only dangerous conversions are lvalue
 -> temp rvalue -> ref, so only disable those. The conversion rvalue ->
 temp rvalue -> ref is not dangerous because the starting value on the
 caller side could not be inspected after the call anyway.


I started reading this post, and I was compelled to reply with this
same response, and then I realised you got there yourself.
I understand your concern, and it has actually been discussed lightly,
but regardless, you'll find that the issue you describe is not
suggested anywhere in this DIP.
This DIP is about passing rvalues to ref... so the issue you describe
passing lvalues to ref does not apply here.
There is no suggestion to change lvalue rules anywhere in this DIP.
Jan 28 2019
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/29/19 1:01 AM, Manu wrote:

 This DIP is about passing rvalues to ref... so the issue you describe
 passing lvalues to ref does not apply here.
 There is no suggestion to change lvalue rules anywhere in this DIP.


The problem is with rvalues resulting as temporaries from lvalues. As in:

void bump(ref int x) { ++x; }
short y = 42;
bump(y);
assert(y == 43); // oops

This is a smoking gun. If DIP 1016 does not propose allowing the code 
above, it has caused a gross misunderstanding. Similarly, you mention in 
a different response:


 4. "Under DIP 1016, a call with any T[] will silently "succeed" by
 converting the slice to void[]"  <--  Do you mean "... with any T[]
 rvalue ..."? What would be the aim of that call? Can you suggest a
 particularly sinister construction?


If your intent was to NOT allow rvalues resulting from conversions, it 
didn't come through at all. On the contrary, some examples suggest DIP 
1016 _does_ allow it, as in this example:

========
This inconvenience extends broadly to every manner of rvalue passed to 
functions, including:

...
fun(my_short); // implicit type conversions (ie, short->int promotion)
========

The reader assumes since the DIP characterizes passing a short lvalue to 
a ref int is an inconvenience, the DIP has set out to resolve it.

The lowering rules proposed by DIP 1006 prescribe the following:

void bump(ref int x) { ++x; }
short y = 42;
bump(y);
===>
void bump(ref int x) { ++x; }
short y = 42;
{
     int __temp0 = y;
     bump(__temp0);
}

So... yes, lvalues do (undesirably) get converted to rvalues of a 
different type according to the DIP.

Are you sure the DIP expresses what you are trying to accomplish?


Andrei
Jan 29 2019
prev sibling parent kinke <noone nowhere.com> writes:
On Monday, 28 January 2019 at 19:58:24 UTC, Andrei Alexandrescu 
wrote:

 On 1/28/19 1:00 PM, Andrei Alexandrescu wrote:

 It seems to me that a proposal adding the " rvalue" attribute 
 in function signatures to each parameter that would accept 
 either an rvalue or an lvalue would be easy to argue.
 
 - No exposing existing APIs to wrong uses
 - The function's writer makes the decision ("I'm fine with 
 this function taking an rvalue")
 - Appears in the function's documentation
 - Syntax is light and localized where it belongs
 - Scales well with number of parameters
 - Transparent to callers




The inverse, the  norval attribute, would feature all but the 
first advantages, at least once people get used to normal ref 
allowing (some) rvalues too. I can't think of many good reasons 
for a callee to disallow rvalues (with matching type at least); 
your example wrt. interlockedIncrement() is a good one, as 
synchronization overhead clearly isn't required for an rvalue. 
But my usages of (mutable) ref are >99% of the time something 
like:

Header readHeader(ref Stream stream)
{
     // read from & advance stream, return parsed header
}

where I'd like to be able put everything into one line if trivial 
(stream not required later on):

readHeader(Stream("file")).doSomething();

(And const ref is mostly used for performance with bigger 
structs, where rvalues are absolutely fine).

TLDR: I'd guess that I'd need to type ` rvalue` about 1000 times 
as often as ` norval`. I'm not kidding.


 TL;DR: it could be argued that the only dangerous conversions 
 are lvalue -> temp rvalue -> ref, so only disable those. The 
 conversion rvalue -> temp rvalue -> ref is not dangerous 
 because the starting value on the caller side could not be 
 inspected after the call anyway.


I agree that the DIP needs to be clearer wrt. rvalues resulting 
from implicit conversions. I also agree with the above point. I'm 
just not sure we need to allow implicit conversions from rvalue 
argument expressions at all. To minimize the changes for overload 
resolution, it might be enough to allow rvalue argument 
expressions of matching type only, i.e., the types which are 
allowed for current ref semantics. After all, I don't plan to use 
the proposed ref semantics for primitive types where that might 
come in handy (something silly like `modf(x, 123.456)` if the 
second param is a `ref real`), and explicit casts would IMO be an 
acceptable price to pay for safety and visibility in the 
remaining cases.
Jan 30 2019
prev sibling parent Rubn <where is.this> writes:
On Thursday, 24 January 2019 at 09:24:19 UTC, Nicholas Wilson 
wrote:

 On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T 
 accepts r-values", on the grounds that it has two fundamental 
 flaws that would open holes in the language. They are not 
 opposed to the feature in principle and suggested that a 
 proposal that closes those holes and covers all the bases will 
 have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom 
 of the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md



 void atomicIncrement(ref shared long x);
 atomicIncrement(myInt);


 Raises a good point, not covered by  disable where the intent 
 is to modify it and modifying a temporary is wrong. `out ref` 
 perhaps?


Why isn't it covered by  disable ?
Jan 24 2019
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T 
 accepts r-values", on the grounds that it has two fundamental 
 flaws that would open holes in the language. They are not 
 opposed to the feature in principle and suggested that a 
 proposal that closes those holes and covers all the bases will 
 have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom 
 of the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md



 [Reasons]. As such, they suggest that a new DIP be drafted from 
 scratch, and that a stronger proposal would have a better 
 chance of acceptance.


Given the simplicity of the fixes for the problems identified I 
really don't think thats the best way forward. One round of forum 
review + formal reassessment ought to be enough.
Jan 24 2019
parent 12345swordy <alexanderheistermann gmail.com> writes:
On Thursday, 24 January 2019 at 09:47:46 UTC, Nicholas Wilson 
wrote:

 Given the simplicity of the fixes for the problems identified I 
 really don't think thats the best way forward. One round of 
 forum review + formal reassessment ought to be enough.


Yes, please! Let not restart the whole dip process for this! The 
DIP is very lengthy as it currently is, and just encourage the 
notation of "where good work goes to die" if this have to go 
through the process all over again.

-Alex
Jan 24 2019
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
Hooray!
Who didn't see that coming 10 years off!

I understand the criticism, the most critical issue seems relatively
trivial and easy to patch (initialisation order), but this comment is
a problem for me "However, this proposal does not maintain the
distinction and instead conflates the two cases".
It's not like that's an accident, or I didn't think of this. The whole
point of this proposal is to lift an arbitrary restriction, thereby
achieving the 2 use cases stated on the opening paragraph, and
increasing language uniformity by removing edges.
"Maintaining the distinction" assumes an approach that is not lifting
one restriction and thereby creating greater uniformity in the
language, but rather, creating 2 more distinct sources of rules, and
that's not attractive as a solution space at all. We've talked about
solutions to this problem that involve adding more rules extensively,
and they're not satisfying.
So it's not because I'm an idiot that I conflated them, it's because
that's literally the point.

Then there's this one:
"The first problem the Language Maintainers identified with this
approach is that the rewrite is from an expression to a statement,
rendering it invalid"
And then present this snippet:
```
fun(10)
==>
{
  T __temp0 = void;
  fun(__temp0 := 10);
}
```
I'm upset by this comment, because it's almost like a deliberate
misrepresentation...
Not only did I never suggest anywhere that the intent is to translate
an expression into a statement, but the following examples demonstrate
why it's expanded that way with more complex compound statements, ie:
```
void fun(ref int x, ref int y);
int gun(ref int x);

fun(10, gun(20));  // <- clearly a statement
```
But even the simplest form introduced first (and presented as
evidence) DOES HAVE A SEMICOLON on the end in my DIP, but removed from
the review?
So I can't agree with this comment: "The expression should be
rewritten as an expression to clarify how it behaves in larger
expressions."; this whole thing is about statements, not expressions.
Misunderstanding that is to misunderstand the proposal entirely.

So my feeling is that this initial point "First, reasoning about the
proposed semantics is futile, as they are built upon a flawed
foundation" is just plain invalid, the suggestion I'm talking about
expressions is incorrect.
And that invalidates this second point "Second,reasoning about the
consequences of potentially revised semantics is equivalent to
considering a new DIP."
I think it's possible to consider this DIP, but the suggestion is that
something other than what I wrote is what has been considered.

Anyway, I can only blame myself for somehow failing to communicate
these details, although I don't know how I managed that.

The criticism about initialisation ordering is absolutely fair, and I
suspect it requires a small amount of tweaking (perhaps initialising
eagerly, or rather just expanding the code and ordering/sequencing the
expansion a little more carefully), to address the
throwing-an-exception mid-statement issue, and not an entirely
different DIP written by someone competent like Timon.

I think all of that is mostly just fluff to lend weight to this
critical sentence "The Language Maintainers insist that any proposal
allowing ref parameters to accept rvalue references must clearly
define how functions which make use of ref for side effects will not
accept rvalues."
And that's 'just like, your opinion man'. A key point is for functions
that do side-effects to receive rvalues too, and this was arrived at
after much community discussion. We identified real cases where we
*want* functions that receive rvalues to do side-effects,
specifically, UFCS chains are frequently broken by need to capture an
lvalue mid-steam.
Pipeline programming is a big deal, and improving that experience was
a motivator for this proposal.

This process is pretty unsatisfying, because it ships off to a
black-box committee, who were apparently able to misunderstand the
substance of the proposal and then not seek clarification, and despite
the only legitimate issue from my perspective being easily corrected,
it's been suggested to start a whole new DIP.

On Wed, Jan 23, 2019 at 11:20 PM Mike Parker via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T
 accepts r-values", on the grounds that it has two fundamental
 flaws that would open holes in the language. They are not opposed
 to the feature in principle and suggested that a proposal that
 closes those holes and covers all the bases will have a higher
 chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom of
 the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md
Jan 24 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/24/2019 1:31 AM, Manu wrote:

 This process is pretty unsatisfying, because it ships off to a
 black-box committee, who were apparently able to misunderstand the
 substance of the proposal and then not seek clarification, and despite
 the only legitimate issue from my perspective being easily corrected,
 it's been suggested to start a whole new DIP.


It's no problem if you want to rework the existing text, just submit it as a
new 
DIP.
Jan 24 2019
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 3:45 PM Walter Bright via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/24/2019 1:31 AM, Manu wrote:

 This process is pretty unsatisfying, because it ships off to a
 black-box committee, who were apparently able to misunderstand the
 substance of the proposal and then not seek clarification, and despite
 the only legitimate issue from my perspective being easily corrected,
 it's been suggested to start a whole new DIP.


 It's no problem if you want to rework the existing text, just submit it as a
new
 DIP.


This process has a long and deep pipe, why should it be a new DIP?
There's nothing from the rejection text that would motivate me to
change any words... Is it that you reject it 'in principle'? If so,
there's nothing I can ever do about that.
This took a substantial amount of my life, and you could have sought
clarification, or a revision before a rejection.
The rejection appears to be premised by misunderstanding more than
anything, and a one very real (but isolated) technical issue that I
believe can be corrected readily enough without affecting the
surrounding text.

The only improvement I could make is to better fold the discussion
from the community review into the core text, but it's not like that
digest wasn't already right there during consideration.

I have no idea how you guys managed to edit and re-frame my DIP as
applying to expressions? You removed the semicolons from the
statements, and then told me I had no idea what I was doing, mixing
expressions with statements that way... why did you do that?
Jan 24 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
No, it is not rejected in principle. Finding serious errors in it on the eve of 
approval is disappointing, and is not auspicious for being in a hurry to
approve it.

For example, I spent a lot of time working on ARC, and was all set to move 
forward with it when Timon stepped in and showed it was fundamentally memory 
unsafe. I couldn't come up with a reasonable solution. I'm grateful that Timon 
saved me from much further wasted work and embarrassment.

Rvalue references are not a simple problem (although they appear to be). I do 
believe the problems with it are solvable, but it is a bit unfair to the 
implementor to dump an incomplete spec on him and have him fill in the gaps.
The 
statement thing is a "do what I meant, not what I wrote" example, and DIPs need 
to be better than that. You're leaving him to design where the temporaries go, 
where the gates go, and ensure everything is properly exception safe.

I know it's frustrating for you, but it's worse if an rvalue-ref implementation 
is implemented, shipped, and heralded, and then turns out to be very broken.
Jan 24 2019
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 25 January 2019 at 07:33:02 UTC, Walter Bright wrote:

 No, it is not rejected in principle.


Good.


 Finding serious errors in it on the eve of approval is 
 disappointing,
 and is not  auspicious for being in a hurry to approve it.


Praytell, what serious errors? Also you should heed your own 
advice DIP1017 and send it back to draft.


 Rvalue references are not a simple problem (although they 
 appear to be).


Please, do enlighten us..


 The statement thing is a "do what I meant, not what I wrote" 
 example,


_You_ removed the semicolons. The DIP applying to statements was 
at worst implied by their use, and frankly quite obvious, how 
else could they transcend multiple expressions?



 You're leaving him to design where the temporaries go, where 
 the gates go, and ensure everything is properly exception safe.


You could have at least had the decency to notify Manu. As noted 
elsewhere this problem is nowhere even close to DIP breaking, and 
I'm certain he could have resolved that.
Jan 25 2019
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 11:35 PM Walter Bright via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 No, it is not rejected in principle. Finding serious errors in it on the eve of
 approval is disappointing, and is not auspicious for being in a hurry to
approve it.


I'm very clearly NOT in a hurry here. We've been sitting on this for 10 years.
What's weird is that you closed the door on iteration, and instead
suggested I should write a new one with someone competent.

More strangely, part of your feedback is broken, it appears you've
reviewed and made assessment against code and text that's just not
there, and you've neglected to respond to those points several times
now.
The error you found seems entirely revision-worthy rather than
rejection-worthy. Your comments about treating expressions as
statements is just wrong, and from that point on where you've
mis-interpreted something so fundamental to the DIP, I don't think
it's possible to trust any outcome from your 'formal assessment'.

I appreciate that you identified the exception issue, we'll fix it,
but I think you need to reconsider the formal rejection.


 but it is a bit unfair to the
 implementor to dump an incomplete spec on him and have him fill in the gaps


How is that the intent? We can get the rewrite semantics right with an
iteration.
So is it rejected on that premise? I don't understand how re-reading
it with satisfactory rewrite logic going to change your assessment of
the DIP in general? No surrounding text would change, and assuming
that the rewrite is corrected, then do you just find a different
reason to reject it?
If so, then that needs to be the reason for the rejection, and not the
error in the rewrite.

I presume the real reason for rejection is this part:
"They say that with the current semantics, this function only operates
on long values as it should. With the proposed semantics, the call
will accept all shared integral types. Any similar proposal must
address this hole in order to be accepted."

But to make that criticism is to miss the point entirely. The point is
to do the thing you say needs to be addressed... and a whole bunch of
techniques to motivate the compiler to emit desirable compile errors
can be deployed in various circumstances. None of them are
particularly unpleasant or awkward.
TL;DR: use `out`, use `*`, use  disable, use const. These can and
should all be deployed appropriately anyway.

Is that the reason it was rejected? If so, then I can't fix that by
rewriting the DIP, that *is* the DIP.
If you're not persuaded by the advantages, and that (rather extensive)
set of tools to mitigate the side effects you're concerned about, then
that's really more of an opinion than a technical rejection.
I guess you're entitled to say "I don't like it, and I reject it
because I don't like it", but you have to *say* that, and not make up
some other stuff.


 The statement thing is a "do what I meant, not what I wrote" example, and DIPs
need
 to be better than that. You're leaving him to design where the temporaries go,
 where the gates go, and ensure everything is properly exception safe.


I agree, we'll fix the temporaries; but getting that correct is a
revision surely. There's no spec to change there.
The criticism talking about rewriting expressions as statements is
still mysterious to me. I don't understand how a rejection can be
presented based on an incorrect reading of the DIP... and how am I
supposed to accept the rejection text containing those criticisms when
the criticisms don't address what's written?
You had to change the code (removing the semicolon from the statement)
to make the claim that I was rewriting expressions as statements, and
I honestly have no idea why you did that?

Anyway...
Jan 25 2019
prev sibling parent Dein <dwin d.com> writes:
On Friday, 25 January 2019 at 07:33:02 UTC, Walter Bright wrote:

 No, it is not rejected in principle. Finding serious errors in 
 it on the eve of approval is disappointing, and is not 
 auspicious for being in a hurry to approve it.

 For example, I spent a lot of time working on ARC, and was all 
 set to move forward with it when Timon stepped in and showed it 
 was fundamentally memory unsafe. I couldn't come up with a 
 reasonable solution. I'm grateful that Timon saved me from much 
 further wasted work and embarrassment.

 Rvalue references are not a simple problem (although they 
 appear to be). I do believe the problems with it are solvable, 
 but it is a bit unfair to the implementor to dump an incomplete 
 spec on him and have him fill in the gaps. The statement thing 
 is a "do what I meant, not what I wrote" example, and DIPs need 
 to be better than that. You're leaving him to design where the 
 temporaries go, where the gates go, and ensure everything is 
 properly exception safe.

 I know it's frustrating for you, but it's worse if an 
 rvalue-ref implementation is implemented, shipped, and 
 heralded, and then turns out to be very broken.


Bit ironic considering how some DIPs are being handled. Changes 
for dip 1000 are already being integrated in Phobos yet the dip 
is still only a draft and looks like part of it still needs to be 
rewritten. If you needed to iron out details by implementing it, 
then that should have been done on your own fork like everyone 
else instead of adding experimental code into master.
Jan 25 2019
prev sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Thursday, 24 January 2019 at 23:43:21 UTC, Walter Bright wrote:

 It's no problem if you want to rework the existing text, just 
 submit it as a new DIP.


And wait for another 180+ days for a fix? Come on dude, can you 
understand the frustration being display here?
Jan 24 2019
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/24/2019 4:31 PM, 12345swordy wrote:

 And wait for another 180+ days for a fix? Come on dude, can you understand the 
 frustration being display here?


Of course it's frustrating. On the other hand, we've had a lot of problems 
stemming from implementing features without thoroughly understanding them.

Rvalue references have a lot of subtleties to them, and we should not rush into 
it, especially since these issues only turned up at the last minute.
Jan 24 2019
next sibling parent Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 6:35 PM Walter Bright via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/24/2019 4:31 PM, 12345swordy wrote:

 And wait for another 180+ days for a fix? Come on dude, can you understand the
 frustration being display here?


 Of course it's frustrating. On the other hand, we've had a lot of problems
 stemming from implementing features without thoroughly understanding them.

 Rvalue references have a lot of subtleties to them, and we should not rush into
 it, especially since these issues only turned up at the last minute.


Which issues? The initialization order issue? That's relatively
trivial, isolated, and doesn't change the substance of the proposal in
any way (unless a working rewrite is impossible, which I'm confident
is not the case).
The rest of your criticisms certainly did not 'turn up at last
minute', they were extensively discussed, and discussion material is
available, and present in the community review summary.

And then there's the weird expression vs statement comments, which are
bizarre, because you literally had to modify my code snippets
(removing the semicolons) to read it that way... I can't accept that
feedback, that just demonstrates a mis-reading of the DIP. If the DIP
could be misunderstood that way, then that's surely revision-worthy,
not throw-it-out-and-start-over worthy, and it has nothing to say
about the substance of the design.
Jan 24 2019
prev sibling parent Manu <turkeyman gmail.com> writes:
On Thu, Jan 24, 2019 at 6:35 PM Walter Bright via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/24/2019 4:31 PM, 12345swordy wrote:

 And wait for another 180+ days for a fix? Come on dude, can you understand the
 frustration being display here?


 Of course it's frustrating. On the other hand, we've had a lot of problems
 stemming from implementing features without thoroughly understanding them.

 Rvalue references have a lot of subtleties to them, and we should not rush into
 it, especially since these issues only turned up at the last minute.


"Rush"? We've literally been debating this since my first post on this
forum... like, 10 years ago.
It's the issue I specifically joined this forum to complain about.
Jan 24 2019
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 25 January 2019 at 00:31:50 UTC, 12345swordy wrote:

 On Thursday, 24 January 2019 at 23:43:21 UTC, Walter Bright 
 wrote:

 It's no problem if you want to rework the existing text, just 
 submit it as a new DIP.


 And wait for another 180+ days for a fix? Come on dude, can you 
 understand the frustration being display here?


This will be discussed at DConf at the DLF meeting, which is only 
75ish days, along with all other DIPs that are not yet 
implemented, under review (incl. draft) or recently rejected and 
a whole host of other topics, so that we can finally get some 
process direction and vision happening.
Jan 24 2019
prev sibling next sibling parent Dgame <r.schuett.1987 gmail.com> writes:
On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

fun(10)
 ==>
 {
  T __temp0 = void;
  fun(__temp0 := 10);
 }


 The first problem the Language Maintainers identified with this 
 approach is that the rewrite is from an expression to a 
 statement, rendering it invalid.
 The expression should be rewritten as an expression to clarify 
 how it behaves in larger expressions.


Couldn't that just be rewritten as something like

fun(tuple(10).expand);

?
Jan 24 2019
prev sibling next sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T 
 accepts r-values", on the grounds that it has two fundamental 
 flaws that would open holes in the language. They are not 
 opposed to the feature in principle and suggested that a 
 proposal that closes those holes and covers all the bases will 
 have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom 
 of the document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md


Welp, one of the features of the implicit convertion dip that I 
am going to write depend on this, and it is quite disapointed to 
see that it is rejected without giving manu a chance for 
clarification and providing a solution without restarting the dip 
process again.
Jan 24 2019
prev sibling next sibling parent reply Elie Morisse <syniurge gmail.com> writes:
On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md



 Here, the DIP Author clearly expresses two reasons why a 
 programmer may choose to declare a function to accept ref 
 arguments. The Language Maintainers see this as the core of the 
 proposal and would expect the distinction between the two cases 
 to be maintained throughout. However, this proposal does not 
 maintain the distinction and instead conflates the two cases. 
 The Language Maintainers insist that any proposal allowing ref 
 parameters to accept rvalue references must clearly define how 
 functions which make use of ref for side effects will not 
 accept rvalues.


Sorry, but that's just irrelevant / missing the point.

On Thursday, 24 January 2019 at 09:31:41 UTC, Manu wrote:

 Hooray!
 Who didn't see that coming 10 years off!


I didn't see that coming and I'm deeply frustrated and 
disappointed by this review and rejection.
Jan 24 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/24/2019 4:21 PM, Elie Morisse wrote:

 I didn't see that coming and I'm deeply frustrated and disappointed by this 
 review and rejection.


On the contrary. It is good to find conceptual errors before implementing it. 
They're a LOT cheaper to fix earlier rather than when it is in the field.
Jan 24 2019
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 25 January 2019 at 07:02:47 UTC, Walter Bright wrote:

 On 1/24/2019 4:21 PM, Elie Morisse wrote:

 I didn't see that coming and I'm deeply frustrated and 
 disappointed by this review and rejection.


 On the contrary. It is good to find conceptual errors before 
 implementing it.


You mean the conceptual errors you made when reviewing it? Like:

That the conflation of pass by reference to avoid copying and 
mutation is not only deliberate but also mitigated by  disable.

That the DIP applies to statements, not expressions.

That the construction order issue is trivially fixable, by 
specifying the same behaviour as the non ref case modulo ref.
Jan 24 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/24/2019 11:53 PM, Nicholas Wilson wrote:

 That the conflation of pass by reference to avoid copying and mutation is not 
 only deliberate but also mitigated by  disable.


The first oddity about  disable is it is attached to the foo(int), not the 
foo(ref int). If I wanted to know if foo(ref int) takes rvalue references, I'd 
have to go looking for the existence of another function. This is one of those 
cases where it's hard to prove a negative, as other functions can be introduced 
by mixins. This is a strong usability negative.

Next, the  disable applies to the entire parameter list. However, overload 
selection is done by looking at each parameter. The DIP says:

"The DIP author responded that ideas to improve this are welcome, but that he 
cannot imagine a use case."

I can guarantee that the use case of more than one reference parameter will
come 
up. The workarounds the DIP suggests are simply awful.

Let's look at what C++ does for rvalue references:

     http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2027.html

The syntax is attached to the parameter declaration of the function it applies 
to, not some other function, and not every parameter:

     int foo(T&& t);  // C++ rvalue ref

There are no weird workarounds, at least for that aspect. There are indeed 
unlikable things about the C++ rules, but the DIP needs to pay more attention
to 
how C++ does this, and justify why D differs. Particularly because D will
likely 
have to have some mechanism of ABI compatibility with C++ functions that take 
rvalue references.

This is not a small problem.

A further problem is implicit conversions, which the DIP ignores by only
talking 
about ints.

     void bar(int);
     void foo(ref int);
     enum short s = 10;
     bar(s); // compiles
     foo(s); // currently fails to compile

Should `s` be promoted to an int temporary, then pass the temporary by 
reference? I can find no guidance in the DIP. What if `s` is a uint (i.e. the 
implicit conversion is a type paint and involves no temporary)?

Here's a discussion of Rust and rvalue references which may offer insight:

https://www.reddit.com/r/rust/comments/3ko5pm/explaining_cs_rvalue_references_from_a_rust/



 That the DIP applies to statements, not expressions.


The DIP should not invent its own syntax, give no explanation of it, and have 
the reader guess. (It did explain the :=, but not the use of { } and 
statements.) And, even if one did a mental rewrite, the semantics of the 
statement version are simply wrong. (For example, if 'fun' was actually a 
function pointer returned by another function, and that other function threw an 
exception - then the destructor would be run on an uninitialized variable.)



 That the construction order issue is trivially fixable, by specifying the same 
 behaviour as the non ref case modulo ref.


It should never have gotten this far without giving a precise explanation of
how 
exception safety is achieved when faced with multiple parameters. In the past 
I've done a lot of work on exception safety, and it isn't trivial once one goes 
beyond trivial cases.

All that criticism aside, I'd like to see rvalue references in D. But the DIP 
needs significant work.
Jan 25 2019
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 25 January 2019 at 11:56:58 UTC, Walter Bright wrote:

 It should never have gotten this far without giving a precise 
 explanation of how exception safety is achieved when faced with 
 multiple parameters.


The pot calling the kettle black. DIP1000? DIP1017?

Again, all that requires is a minor revision. It is not DIP 
breaking.
Jan 25 2019
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 25 January 2019 at 12:03:36 UTC, Nicholas Wilson wrote:

 On Friday, 25 January 2019 at 11:56:58 UTC, Walter Bright wrote:

 It should never have gotten this far without giving a precise 
 explanation of how exception safety is achieved when faced 
 with multiple parameters.


 The pot calling the kettle black. DIP1000? DIP1017?


Or DIP1008: https://issues.dlang.org/show_bug.cgi?id=19463
Jan 25 2019
prev sibling next sibling parent Olivier FAURE <couteaubleu gmail.com> writes:
On Friday, 25 January 2019 at 11:56:58 UTC, Walter Bright wrote:

 All that criticism aside, I'd like to see rvalue references in 
 D. But the DIP needs significant work.


I haven't participated to writing this DIP, but I personally 
appreciate this level of feedback.

I think it would have been more appreciated if the original 
answer had that level of detail.

Otherwise, I don't think this should be an all-or-nothing 
situation. It would make sense to bump the DIP back one stage or 
two for minor adjustments, and if the author decide that they 
need to make major changes to get past the problems you mention, 
require these changes to go through the entire process again 
(through another DIP).
Jan 25 2019
prev sibling next sibling parent reply Rubn <where is.this> writes:
On Friday, 25 January 2019 at 11:56:58 UTC, Walter Bright wrote:

 On 1/24/2019 11:53 PM, Nicholas Wilson wrote:

 That the conflation of pass by reference to avoid copying and 
 mutation is not only deliberate but also mitigated by  disable.


 The first oddity about  disable is it is attached to the 
 foo(int), not the foo(ref int). If I wanted to know if foo(ref 
 int) takes rvalue references, I'd have to go looking for the 
 existence of another function. This is one of those cases where 
 it's hard to prove a negative, as other functions can be 
 introduced by mixins. This is a strong usability negative.

 Next, the  disable applies to the entire parameter list. 
 However, overload selection is done by looking at each 
 parameter. The DIP says:

 "The DIP author responded that ideas to improve this are 
 welcome, but that he cannot imagine a use case."

 I can guarantee that the use case of more than one reference 
 parameter will come up. The workarounds the DIP suggests are 
 simply awful.

 Let's look at what C++ does for rvalue references:

     
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2027.html

 The syntax is attached to the parameter declaration of the 
 function it applies to, not some other function, and not every 
 parameter:

     int foo(T&& t);  // C++ rvalue ref

 There are no weird workarounds, at least for that aspect. There 
 are indeed unlikable things about the C++ rules, but the DIP 
 needs to pay more attention to how C++ does this, and justify 
 why D differs. Particularly because D will likely have to have 
 some mechanism of ABI compatibility with C++ functions that 
 take rvalue references.

 This is not a small problem.

 A further problem is implicit conversions, which the DIP 
 ignores by only talking about ints.

     void bar(int);
     void foo(ref int);
     enum short s = 10;
     bar(s); // compiles
     foo(s); // currently fails to compile

 Should `s` be promoted to an int temporary, then pass the 
 temporary by reference? I can find no guidance in the DIP. What 
 if `s` is a uint (i.e. the implicit conversion is a type paint 
 and involves no temporary)?

 Here's a discussion of Rust and rvalue references which may 
 offer insight:

 https://www.reddit.com/r/rust/comments/3ko5pm/explaining_cs_rvalue_references_from_a_rust/



 That the DIP applies to statements, not expressions.


 The DIP should not invent its own syntax, give no explanation 
 of it, and have the reader guess. (It did explain the :=, but 
 not the use of { } and statements.) And, even if one did a 
 mental rewrite, the semantics of the statement version are 
 simply wrong. (For example, if 'fun' was actually a function 
 pointer returned by another function, and that other function 
 threw an exception - then the destructor would be run on an 
 uninitialized variable.)



 That the construction order issue is trivially fixable, by 
 specifying the same behaviour as the non ref case modulo ref.


 It should never have gotten this far without giving a precise 
 explanation of how exception safety is achieved when faced with 
 multiple parameters. In the past I've done a lot of work on 
 exception safety, and it isn't trivial once one goes beyond 
 trivial cases.

 All that criticism aside, I'd like to see rvalue references in 
 D. But the DIP needs significant work.


For future reference, this is what a formal review should be. I'd 
also rather your exact words than some summarization of them.
Jan 25 2019
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 26/01/2019 10:00 AM, Rubn wrote:

 On Friday, 25 January 2019 at 11:56:58 UTC, Walter Bright wrote:

 On 1/24/2019 11:53 PM, Nicholas Wilson wrote:

 That the conflation of pass by reference to avoid copying and 
 mutation is not only deliberate but also mitigated by  disable.


 The first oddity about  disable is it is attached to the foo(int), not 
 the foo(ref int). If I wanted to know if foo(ref int) takes rvalue 
 references, I'd have to go looking for the existence of another 
 function. This is one of those cases where it's hard to prove a 
 negative, as other functions can be introduced by mixins. This is a 
 strong usability negative.

 Next, the  disable applies to the entire parameter list. However, 
 overload selection is done by looking at each parameter. The DIP says:

 "The DIP author responded that ideas to improve this are welcome, but 
 that he cannot imagine a use case."

 I can guarantee that the use case of more than one reference parameter 
 will come up. The workarounds the DIP suggests are simply awful.

 Let's look at what C++ does for rvalue references:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2027.html

 The syntax is attached to the parameter declaration of the function it 
 applies to, not some other function, and not every parameter:

     int foo(T&& t);  // C++ rvalue ref

 There are no weird workarounds, at least for that aspect. There are 
 indeed unlikable things about the C++ rules, but the DIP needs to pay 
 more attention to how C++ does this, and justify why D differs. 
 Particularly because D will likely have to have some mechanism of ABI 
 compatibility with C++ functions that take rvalue references.

 This is not a small problem.

 A further problem is implicit conversions, which the DIP ignores by 
 only talking about ints.

     void bar(int);
     void foo(ref int);
     enum short s = 10;
     bar(s); // compiles
     foo(s); // currently fails to compile

 Should `s` be promoted to an int temporary, then pass the temporary by 
 reference? I can find no guidance in the DIP. What if `s` is a uint 
 (i.e. the implicit conversion is a type paint and involves no temporary)?

 Here's a discussion of Rust and rvalue references which may offer 
 insight:

 https://www.reddit.com/r/rust/comments/3ko5pm/explaining_cs_rvalue_ref
rences_from_a_rust/ 




 That the DIP applies to statements, not expressions.


 The DIP should not invent its own syntax, give no explanation of it, 
 and have the reader guess. (It did explain the :=, but not the use of 
 { } and statements.) And, even if one did a mental rewrite, the 
 semantics of the statement version are simply wrong. (For example, if 
 'fun' was actually a function pointer returned by another function, 
 and that other function threw an exception - then the destructor would 
 be run on an uninitialized variable.)



 That the construction order issue is trivially fixable, by specifying 
 the same behaviour as the non ref case modulo ref.


 It should never have gotten this far without giving a precise 
 explanation of how exception safety is achieved when faced with 
 multiple parameters. In the past I've done a lot of work on exception 
 safety, and it isn't trivial once one goes beyond trivial cases.

 All that criticism aside, I'd like to see rvalue references in D. But 
 the DIP needs significant work.


 
 For future reference, this is what a formal review should be. I'd also 
 rather your exact words than some summarization of them.


So in other words, a formal review should include somebody acting as an 
'interviewer' prompting questions. If that is the case, I do think it 
would be a good idea.
Jan 25 2019
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Fri, Jan 25, 2019 at 4:00 AM Walter Bright via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/24/2019 11:53 PM, Nicholas Wilson wrote:

 That the conflation of pass by reference to avoid copying and mutation is not
 only deliberate but also mitigated by  disable.


 The first oddity about  disable is it is attached to the foo(int), not the
 foo(ref int). If I wanted to know if foo(ref int) takes rvalue references,


And right here, I can see our fundamental difference of perspective...

I never said anything about 'rvalue references', and I never meant
anything like that; at least, not in the C++ sense, which you seem to
be alluding to.
In C++, rval references are syntactically distinct and identifiable as
such, for the purposes of implementing move semantics.

If we want to talk about "rvalue references", then we need to be
having a *completely* different conversation.
That said, I'm not sure why you've raised this matter, since it's not
written anywhere in the DIP.

What I'm talking about is "not-rvalue-references accepting rvalues",
which if you want to transpose into C++ terms, is like `const T&`.


 There are indeed
 unlikable things about the C++ rules, but the DIP needs to pay more attention
to
 how C++ does this, and justify why D differs. Particularly because D will
likely
 have to have some mechanism of ABI compatibility with C++ functions that take
 rvalue references.


I'm not paying attention to C++ T&& rules, because this DIP has
nothing to do with T&&, and there would be no allusion to connecting
this to a T&& method. Again, I find that to be a very interesting
topic of conversation, but it has nothing to do with this DIP.


 [...]
 Should `s` be promoted to an int temporary, then pass the temporary by
 reference? I can find no guidance in the DIP. What if `s` is a uint (i.e. the
 implicit conversion is a type paint and involves no temporary)?


As per the DIP; yes, that is the point.
The text you seek is written: "[...]. The user should not experience
edge cases, or differences in functionality when calling fun(int x) vs
fun(ref int x)."
That text appears at least 2 times through the document as the stated goal.

Don't accept naked ref unless you want these semantics. There is a
suite of tools offered to use where this behaviour is undesirable.
Naked `ref` doesn't do anything particularly interesting in the
language today that's not *identical* semantically to using a pointer
and adding a single '&' character at the callsite. This DIP attempts
to make `ref` interesting and useful as a feature in its own right.
In discussions designing this thing, I've come to appreciate the UFCS
advantages as the most compelling opportunity, among all the other
things that burn me almost practically every time I write D code.


 The DIP should not invent its own syntax


I removed it, and replaced it with simpler code (that I think is
exception-correct) in my prior post here. It's also a super-trivial
amendment.


 It should never have gotten this far without giving a precise explanation of
how


exception safety is achieved when faced with multiple parameters.

I apologise. I've never used exceptions in any code I've ever written,
so it's pretty easy for me to overlook that detail.
Nobody else that did the community reviews flagged it, and that
includes you and Andrei, as members of the community.


 All that criticism aside, I'd like to see rvalue references in D. But the DIP
 needs significant work.


This is *NOT* an "rvalue-references" DIP; this is a "references" DIP.
If you want to see an rvalue references DIP, I agree that's a
completely different development, and it's also interesting to me... I
had *absolutely no idea* that an rvalue-references DIP was welcome. I
thought D was somewhat aggressively proud of the fact that we don't
have rvalue-references... apparently I took the wrong impression.

That said, this remains infinitely more important to me than an
rvalue-references DIP. It's been killing me for 10 years, and I'm
personally yet to feel hindered by our lack of rvalue-reference
support.
Jan 25 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/25/2019 7:44 PM, Manu wrote:

 I never said anything about 'rvalue references',


The DIP mentions them several times in the "forum threads" section. I see you 
want to distinguish the DIP from that; I recommend a section clearing that up.

However, my points about the serious problems with  disable syntax remain.

A section comparing with the C++ solution is necessary as well, more than the 
one sentence dismissal. For example, how C++ deals with the:

     void foo(const int x);
     void foo(const int& x);

situation needs to be understood and compared. Failing to understand it can
lead 
to serious oversights. For example, C++ doesn't require an  disable syntax to 
make it work.



 [...]
 Should `s` be promoted to an int temporary, then pass the temporary by
 reference? I can find no guidance in the DIP. What if `s` is a uint (i.e. the
 implicit conversion is a type paint and involves no temporary)?


 As per the DIP; yes, that is the point.
 The text you seek is written: "[...]. The user should not experience
 edge cases, or differences in functionality when calling fun(int x) vs
 fun(ref int x)."


I don't see how that addresses implicit type conversion at all.



 Don't accept naked ref unless you want these semantics. There is a
 suite of tools offered to use where this behaviour is undesirable.
 Naked `ref` doesn't do anything particularly interesting in the
 language today that's not *identical* semantically to using a pointer
 and adding a single '&' character at the callsite.


It's not good enough. The DIP needs to specifically address what happens with 
implicit conversions. The reader should not be left wondering about what is 
implied. I often read a spec and think yeah, yeah, of course it must be that 
way. But it is spelled out in the spec, and reading it gives me confidence that 
I'm understanding the semantics, and it gives me confidence that whoever wrote 
the spec understood it.

(Of course, writing out the implications sometimes causes the writer to realize 
he didn't actually understand it at all.)

Furthermore, D has these match levels:

     1. exact
     2. const
     3. conversion
     4. no match

If there are two or more matches at the same level, the decision is made based 
on partial ordering. How does adding the new ref/value overloading fit into
that?



 It should never have gotten this far without giving a precise explanation of
how


 exception safety is achieved when faced with multiple parameters.
 
 I apologise. I've never used exceptions in any code I've ever written,
 so it's pretty easy for me to overlook that detail.


It's so, so easy to get that wrong. C++ benefits from decades of compiler bug 
fixes with that.



 Nobody else that did the community reviews flagged it,


That's unfortunately right. Note that 'alias this' was approved and
implemented, 
and then multiple serious conceptual problems have appeared with it. I don't 
want a repeat of that.



 and that includes you and Andrei, as members of the community.


The idea was that Andrei & I wouldn't get too involved in the DIPs until they 
are vetted by the community. I.e. delegation.



 That said, this remains infinitely more important to me than an
 rvalue-references DIP. It's been killing me for 10 years, and I'm
 personally yet to feel hindered by our lack of rvalue-reference
 support.


I look forward to a much improved DIP from you (and anyone else who wishes to 
help you out with the work!).
Jan 25 2019
next sibling parent reply Rubn <where is.this> writes:
On Saturday, 26 January 2019 at 06:15:22 UTC, Walter Bright wrote:

 On 1/25/2019 7:44 PM, Manu wrote:

 I never said anything about 'rvalue references',


 The DIP mentions them several times in the "forum threads" 
 section. I see you want to distinguish the DIP from that; I 
 recommend a section clearing that up.

 However, my points about the serious problems with  disable 
 syntax remain.

 A section comparing with the C++ solution is necessary as well, 
 more than the one sentence dismissal. For example, how C++ 
 deals with the:

     void foo(const int x);
     void foo(const int& x);

 situation needs to be understood and compared. Failing to 
 understand it can lead to serious oversights. For example, C++ 
 doesn't require an  disable syntax to make it work.



 [...]
 Should `s` be promoted to an int temporary, then pass the 
 temporary by
 reference? I can find no guidance in the DIP. What if `s` is 
 a uint (i.e. the
 implicit conversion is a type paint and involves no 
 temporary)?


 As per the DIP; yes, that is the point.
 The text you seek is written: "[...]. The user should not 
 experience
 edge cases, or differences in functionality when calling 
 fun(int x) vs
 fun(ref int x)."


 I don't see how that addresses implicit type conversion at all.


Anything that could be implicitly converted to use foo(int) can 
be implicitly converted to pass a ref to the temporary that was 
implicitly converted to int into foo(ref int). No rules change in 
this regard. If you don't see how this address type conversion 
perhaps a code sample might help? The one that was given with 
short:

void foo(ref int);
void bar(int);

bar( short(10) ); // is ok
foo( short(10) ); // expected to be ok short->int ; ref to temp 
passed to foo

Just as bar(int) can be passed a short(10), foo(ref int) can be 
passed a reference to the temporary that was created as well.



 Don't accept naked ref unless you want these semantics. There 
 is a
 suite of tools offered to use where this behaviour is 
 undesirable.
 Naked `ref` doesn't do anything particularly interesting in the
 language today that's not *identical* semantically to using a 
 pointer
 and adding a single '&' character at the callsite.


 It's not good enough. The DIP needs to specifically address 
 what happens with implicit conversions. The reader should not 
 be left wondering about what is implied. I often read a spec 
 and think yeah, yeah, of course it must be that way. But it is 
 spelled out in the spec, and reading it gives me confidence 
 that I'm understanding the semantics, and it gives me 
 confidence that whoever wrote the spec understood it.

 (Of course, writing out the implications sometimes causes the 
 writer to realize he didn't actually understand it at all.)

 Furthermore, D has these match levels:

     1. exact
     2. const
     3. conversion
     4. no match

 If there are two or more matches at the same level, the 
 decision is made based on partial ordering. How does adding the 
 new ref/value overloading fit into that?


The DIP goes over this, though not in a lot of detail. All the 
same rules apply as with the current implementation. Where there 
would be a compiler error trying to pass an rvalue would instead 
forward the value.

Effectively what is being implemented is the following (for type 
matching only):

    void foo( ref int );
    void foo( int value ) { foo( value ); }

Anything that would have been passed to foo(int) is passed to 
foo(ref int) as a reference to a temporary instead. No rules are 
changed in this regard for matching, all the same rules apply (as 
stated in the DIP). It's pretty clear, unless you can give a 
specific problem faced where this doesn't hold? D is pretty 
strict to ensure rvalues aren't passed to ref's and that's what 
makes this relatively simple to implement without changing 
matching rules.
Jan 26 2019
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/26/2019 8:28 AM, Rubn wrote:

 [...]


The point is, the DIP needs to spell this out in an organized and complete 
fashion, like any proper spec does.

We all want a better specified language, let's make it happen.
Jan 26 2019
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Fri, Jan 25, 2019 at 10:20 PM Walter Bright via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/25/2019 7:44 PM, Manu wrote:

 I never said anything about 'rvalue references',


 The DIP mentions them several times in the "forum threads" section. I see you
 want to distinguish the DIP from that; I recommend a section clearing that up.

 However, my points about the serious problems with  disable syntax remain.


I think the ` disable` semantic is correct; I understand your
criticism that you have to search for the negative to understand the
restruction, but that perspective arises presumably from a presumption
that you want to explicity state inclusion, which is the opposite of
the intent.
The goal is to state exclusion, we are *adding* restrictions (ie,
removing potential calls) from the default more-inclusive behaviour,
and from that perspective, ` disable` is in the proper place.


 A section comparing with the C++ solution is necessary as well, more than the
 one sentence dismissal. For example, how C++ deals with the:

      void foo(const int x);
      void foo(const int& x);

 situation needs to be understood and compared. Failing to understand it can
lead
 to serious oversights. For example, C++ doesn't require an  disable syntax to
 make it work.


C++ doesn't desire a  disable semantic, it just works as described in this DIP.

Eg:

```c++
void fun(const int& x) {}
void test()
{
    fun(10);
    fun(short(10)); // <- no problem!
}
```

It's the dlang critics of this functionality that demand explicit
controls on functions accepting one kind or the other.
I personally see no value in all that noise, but I added it in due to
popular demand.


 [...]
 Should `s` be promoted to an int temporary, then pass the temporary by
 reference? I can find no guidance in the DIP. What if `s` is a uint (i.e. the
 implicit conversion is a type paint and involves no temporary)?


 As per the DIP; yes, that is the point.
 The text you seek is written: "[...]. The user should not experience
 edge cases, or differences in functionality when calling fun(int x) vs
 fun(ref int x)."


 I don't see how that addresses implicit type conversion at all.


It explicitly permits it as one of the goals of the DIP. Uniformity in
function calling is one of the main goals here.


 Don't accept naked ref unless you want these semantics. There is a
 suite of tools offered to use where this behaviour is undesirable.
 Naked `ref` doesn't do anything particularly interesting in the
 language today that's not *identical* semantically to using a pointer
 and adding a single '&' character at the callsite.


 It's not good enough. The DIP needs to specifically address what happens with
 implicit conversions. The reader should not be left wondering about what is
 implied.


As I said above, it couldn't be stated more clearly in the DIP; it is
very explicitly permitted, and stated that "the user should not
experience any difference in calling semantics when using ref".


 I often read a spec and think yeah, yeah, of course it must be that
 way. But it is spelled out in the spec, and reading it gives me confidence that
 I'm understanding the semantics, and it gives me confidence that whoever wrote
 the spec understood it.


Okay, but it is spelled out. How could I make it clearer?


 (Of course, writing out the implications sometimes causes the writer to realize
 he didn't actually understand it at all.)

 Furthermore, D has these match levels:

      1. exact
      2. const
      3. conversion
      4. no match

 If there are two or more matches at the same level, the decision is made based
 on partial ordering. How does adding the new ref/value overloading fit into
that?


I haven't described this well. I can try and improve this.
Where can I find these existing rules detailed comprehensively? I have
never seen them mentioned in the dlang language reference.
It's hard for me to speak in these terms, when I've never seen any
text in the language spec that does so.

Note; this criticism was nowhere to be found in your rejection text,
and it would have been trivial during community reviews to make this
note.
I feel like this is a mostly simple revision to make.


 It should never have gotten this far without giving a precise explanation of
how


 exception safety is achieved when faced with multiple parameters.

 I apologise. I've never used exceptions in any code I've ever written,
 so it's pretty easy for me to overlook that detail.


 It's so, so easy to get that wrong. C++ benefits from decades of compiler bug
 fixes with that.


I think my revision is water-tight (up a few posts). If the new
rewrite were written by hand and had some problem there, then there's
a serious problem with core language.


 Nobody else that did the community reviews flagged it,


 That's unfortunately right. Note that 'alias this' was approved and
implemented,
 and then multiple serious conceptual problems have appeared with it. I don't
 want a repeat of that.


I couldn't have used D to solve so many problems over the years
without `alias this`. If we refuse struct inheritence, then without
`alias this`, we're screwed.


 and that includes you and Andrei, as members of the community.


 The idea was that Andrei & I wouldn't get too involved in the DIPs until they
 are vetted by the community. I.e. delegation.


You're still members of the community. If something slipped through
community review that you could have easily spotted, then the strategy
you describe is no good.
If you had have spotted it 4-5 months ago, I would have wasted a LOT
less time... and now you're asking me to sign up for another few
hundred days of do-over from square-1.
I'd suggest that feedback at community-review stage would have been
extremely valuable.


 That said, this remains infinitely more important to me than an
 rvalue-references DIP. It's been killing me for 10 years, and I'm
 personally yet to feel hindered by our lack of rvalue-reference
 support.


 I look forward to a much improved DIP from you (and anyone else who wishes to
 help you out with the work!).


That's good, but telling me to start over rather than revise is
unbelievably annoying and de-motivating.

I'd like you and Andrei to revise the response at the end of the
rejected DIP to remove the text that is erroneous. It's not fair to
present a rejection where a good portion of the detailed reasoning is
just plain wrong.
By my reading, there will only be one paragraph left, and then the
motivation for a hard rejection will look very different.
Jan 28 2019
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/28/2019 10:10 PM, Manu wrote:

 Furthermore, D has these match levels:

       1. exact
       2. const
       3. conversion
       4. no match

 If there are two or more matches at the same level, the decision is made based
 on partial ordering. How does adding the new ref/value overloading fit into
that?


 
 I haven't described this well. I can try and improve this.
 Where can I find these existing rules detailed comprehensively? I have
 never seen them mentioned in the dlang language reference.


It's where it should be, under "Function Overloading":

https://dlang.org/spec/function.html#function-overloading



 If you had have spotted it 4-5 months ago, I would have wasted a LOT
 less time... and now you're asking me to sign up for another few
 hundred days of do-over from square-1.


It's pretty clear that just writing the DIP myself will take me less time than 
point-by-point endlessly arguing about it. But it will take more calendar time, 
because all this stuff falls to me. It's why I get paid the big bucks.
Jan 29 2019
prev sibling parent Manu <turkeyman gmail.com> writes:
On Fri, Jan 25, 2019 at 7:44 PM Manu <turkeyman gmail.com> wrote:

 On Fri, Jan 25, 2019 at 4:00 AM Walter Bright via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 The DIP should not invent its own syntax


 I removed it, and replaced it with simpler code (that I think is
 exception-correct) in my prior post here. It's also a super-trivial
 amendment.


Incidentally, the reason I invented a syntax in this DIP, was because
we have no initialisation syntax in D, despite the language clearly
having the ability to initialise values (when they're declared); we
have an amazingly complex and awkward library implementation of
`emplace`, which is pretty embarrassing really.
The fact that I needed to invent a syntax to perform an initialisation
is a very serious problem in its own right.

But forget about that; I removed the need to express initialisation
from the rewrite.
Jan 25 2019
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:
On 1/24/19 2:18 AM, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T accepts 
 r-values", on the grounds that it has two fundamental flaws that would 
 open holes in the language. They are not opposed to the feature in 
 principle and suggested that a proposal that closes those holes and 
 covers all the bases will have a higher chance of getting accepted.
 
 You can read a summary of the Formal Assessment at the bottom of the 
 document:
 
 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md


Hi everyone, I've followed the responses to this, some conveying 
frustration about the decision and some about the review process itself. 
As the person who carried a significant part of the review, allow me to 
share a few thoughts of possible interest.

* Fundamentally: a DIP should stand on its own and be judged on its own 
merit, regardless of rhetoric surrounding it, unstated assumptions, or 
trends of opinion in the forums. There has been a bit of material in 
this forum discussion that should have been argued properly as a part of 
the DIP itself.

* The misinterpretation of the rewrite (expression -> statement vs. 
statement -> statement) is mine, apologies. (It does not influence our 
decision and should not be construed as an essential aspect of the 
review.) The mistake was caused by the informality of the DIP, which 
shows rewrites as a few simplistic examples instead of a general rewrite 
rule. Function calls are expressions, so I naturally assumed the path 
would be to start with the function call expression. Formulating a 
general rule as a statement rewrite is possible but not easy and fraught 
with peril, as discussion in this thread has shown. I very much 
recommend going the expression route (e.g. with the help of lambdas) 
because that makes it very easy to expand to arbitrarily complex 
expressions involving function calls. Clarifying what temporaries get 
names and when in a complex expression is considerably more difficult 
(probably not impossible but why suffer).

* Arguments of the form: "You say DIP 1016 is bad, but look at how bad 
DIP XYZ is!" are great when directed at the poor quality of DIP XYZ. 
They are NOT good arguments in favor of DIP 1016.

* Arguments of the form "Functions that take ref parameters just for 
changing them are really niche anyway" should be properly made in the 
DIP, not in the forums and assumed without stating in the DIP. Again, 
what's being evaluated is "DIP" not "DIP + surrounding rhetoric". A good 
argument would be e.g. analyzing a number of libraries and assess that 
e.g. 91% uses of ref is for efficiency purposes, 3% is unclear, and only 
6% is for side-effect purpose. All preexisting code using ref parameters 
written under the current rule assumes that only lvalues will be bound 
to them. A subset of these functions take by ref for changing them only. 
The DIP should explain why that's not a problem, or if it is one it is a 
small problem, etc. My point is - the DIP should _approach_ the matter 
and build an argument about it. One more example from preexisting code 
for illustration, from the standard library:

// in the allocators API
bool expand(ref void[] b, size_t delta);
bool reallocate(ref void[] b, size_t s);

These primitives modify their first argument in essential ways. The 
intent is to fill b with the new slice resulted after 
expansion/reallocation. Under the current rules, calling these 
primitives is cumbersome, but usefully so because the processing done 
requires extra care if typed data is being reallocated. Under DIP 1016, 
a call with any T[] will silently "succeed" by converting the slice to 
void[], passing the temporary to expand/reallocate, then return as if 
all is well - yet the original slice has not been changed. The DIP 
should create a salient argument regarding these situations (and not 
only this example, but the entire class). It could perhaps argue that:

- Such code is bad to start with, and should not have been written.
- Such code is so rare, we can take the hit. We then have a 
recommendation for library writers on how to amend their codebase (use 
 disable or some other mechanisms).
- The advantages greatly outweigh this problem.
- The bugs caused are minor easy to find.
- ...

Point being: the matter, again should be _addressed_ by the DIP.

* Regarding our recommendation that the proposal is resubmited as a 
distinct DIP as opposed to a patch on the existing DIP: this was not 
embracing bureaucracy. Instead, we considered that the DIP was too poor 
to be easily modified into a strong proposal, and recommended that it be 
rewritten simply because it would be easier and would engender a 
stronger DIP.

* Regarding the argument "why not make this an iterative process where 
concerns are raised and incrementally addressed?" We modeled the DIP 
process after similar processes - conference papers, journal papers, 
proposals in other languages. There is a proposal by one or more 
responsibles, perfected by a community review, and submitted for review. 
This encourages building a strong proposal - as strong as can be - prior 
to submission. Washing that down to a negotiation between the proposers 
and the reviewers leads to a "worst acceptable proposal" state of 
affairs in which proposers are incentivized to submit the least-effort 
proposal, reactively change it as issues are raised by reviewers. As 
anyone who has submitted a conference paper, that's not how it works, 
and even if the process is highly frustrating (yes, reviewers in so many 
cases misunderstand parts of the paper...) it does lead to strong work. 
There are cases in which papers are "accepted with amends" - those are 
strong submissions that have a few issues that are easily fixable. With 
apologies, we do not consider this DIP to be in that category.

This result was frustrating and disheartening on our side, too: a 
stronger DIP should have resulted after all these years. I encourage 
interested people to make a DIP that is scientifically-argued, clearly 
formalized, and provides a thorough analysis of the consequences of the 
proposed design.


Hope this helps,

Andrei
Jan 28 2019
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Mon, Jan 28, 2019 at 9:25 AM Andrei Alexandrescu via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/24/19 2:18 AM, Mike Parker wrote:

 Walter and Andrei have declined to accept DIP 1016, "ref T accepts
 r-values", on the grounds that it has two fundamental flaws that would
 open holes in the language. They are not opposed to the feature in
 principle and suggested that a proposal that closes those holes and
 covers all the bases will have a higher chance of getting accepted.

 You can read a summary of the Formal Assessment at the bottom of the
 document:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md


 Hi everyone, I've followed the responses to this, some conveying
 frustration about the decision and some about the review process itself.
 As the person who carried a significant part of the review, allow me to
 share a few thoughts of possible interest.

 * Fundamentally: a DIP should stand on its own and be judged on its own
 merit, regardless of rhetoric surrounding it, unstated assumptions, or
 trends of opinion in the forums. There has been a bit of material in
 this forum discussion that should have been argued properly as a part of
 the DIP itself.

 * The misinterpretation of the rewrite (expression -> statement vs.
 statement -> statement) is mine, apologies. (It does not influence our
 decision and should not be construed as an essential aspect of the
 review.) The mistake was caused by the informality of the DIP, which
 shows rewrites as a few simplistic examples instead of a general rewrite
 rule. Function calls are expressions, so I naturally assumed the path
 would be to start with the function call expression. Formulating a
 general rule as a statement rewrite is possible but not easy and fraught
 with peril, as discussion in this thread has shown. I very much
 recommend going the expression route (e.g. with the help of lambdas)
 because that makes it very easy to expand to arbitrarily complex
 expressions involving function calls. Clarifying what temporaries get
 names and when in a complex expression is considerably more difficult
 (probably not impossible but why suffer).

 * Arguments of the form: "You say DIP 1016 is bad, but look at how bad
 DIP XYZ is!" are great when directed at the poor quality of DIP XYZ.
 They are NOT good arguments in favor of DIP 1016.

 * Arguments of the form "Functions that take ref parameters just for
 changing them are really niche anyway" should be properly made in the
 DIP, not in the forums and assumed without stating in the DIP. Again,
 what's being evaluated is "DIP" not "DIP + surrounding rhetoric". A good
 argument would be e.g. analyzing a number of libraries and assess that
 e.g. 91% uses of ref is for efficiency purposes, 3% is unclear, and only
 6% is for side-effect purpose. All preexisting code using ref parameters
 written under the current rule assumes that only lvalues will be bound
 to them. A subset of these functions take by ref for changing them only.
 The DIP should explain why that's not a problem, or if it is one it is a
 small problem, etc. My point is - the DIP should _approach_ the matter
 and build an argument about it. One more example from preexisting code
 for illustration, from the standard library:

 // in the allocators API
 bool expand(ref void[] b, size_t delta);
 bool reallocate(ref void[] b, size_t s);

 These primitives modify their first argument in essential ways. The
 intent is to fill b with the new slice resulted after
 expansion/reallocation. Under the current rules, calling these
 primitives is cumbersome, but usefully so because the processing done
 requires extra care if typed data is being reallocated. Under DIP 1016,
 a call with any T[] will silently "succeed" by converting the slice to
 void[], passing the temporary to expand/reallocate, then return as if
 all is well - yet the original slice has not been changed. The DIP
 should create a salient argument regarding these situations (and not
 only this example, but the entire class). It could perhaps argue that:

 - Such code is bad to start with, and should not have been written.
 - Such code is so rare, we can take the hit. We then have a
 recommendation for library writers on how to amend their codebase (use
  disable or some other mechanisms).
 - The advantages greatly outweigh this problem.
 - The bugs caused are minor easy to find.
 - ...

 Point being: the matter, again should be _addressed_ by the DIP.

 * Regarding our recommendation that the proposal is resubmited as a
 distinct DIP as opposed to a patch on the existing DIP: this was not
 embracing bureaucracy. Instead, we considered that the DIP was too poor
 to be easily modified into a strong proposal, and recommended that it be
 rewritten simply because it would be easier and would engender a
 stronger DIP.

 * Regarding the argument "why not make this an iterative process where
 concerns are raised and incrementally addressed?" We modeled the DIP
 process after similar processes - conference papers, journal papers,
 proposals in other languages. There is a proposal by one or more
 responsibles, perfected by a community review, and submitted for review.
 This encourages building a strong proposal - as strong as can be - prior
 to submission. Washing that down to a negotiation between the proposers
 and the reviewers leads to a "worst acceptable proposal" state of
 affairs in which proposers are incentivized to submit the least-effort
 proposal, reactively change it as issues are raised by reviewers. As
 anyone who has submitted a conference paper, that's not how it works,
 and even if the process is highly frustrating (yes, reviewers in so many
 cases misunderstand parts of the paper...) it does lead to strong work.
 There are cases in which papers are "accepted with amends" - those are
 strong submissions that have a few issues that are easily fixable. With
 apologies, we do not consider this DIP to be in that category.

 This result was frustrating and disheartening on our side, too: a
 stronger DIP should have resulted after all these years. I encourage
 interested people to make a DIP that is scientifically-argued, clearly
 formalized, and provides a thorough analysis of the consequences of the
 proposed design.


 Hope this helps,

 Andrei


A few things in here...

1. All of this is more useful criticism than the official and final
criticism affixed to the rejection, which when revised to remove the
incorrect criticisms, is basically left with the text "The Language
Maintainers found other issues with the proposal, most of which may
have been remedied through simple revision"
2. All of this criticism could have been given at any point in the
past half a year or so prior to submission, and that would have been
appreciated, rather than wasting our time.
3. "It does not influence our decision and should not be construed as
an essential aspect of the review"  <--  Then why did it feature as
one of just 3 core criticism in the rejection text? And supplied as
one of the 2 reasons that could not "have been remedied through simple
revision".
4. "Under DIP 1016, a call with any T[] will silently "succeed" by
converting the slice to void[]"  <--  Do you mean "... with any T[]
rvalue ..."? What would be the aim of that call? Can you suggest a
particularly sinister construction?
5. "and recommended that it be rewritten simply because it would be
easier and would engender a stronger DIP."  <--  I wrote the DIP I
wrote... your official feedback affixed to the bottom of the DIP was
pretty much entirely unhelpful, almost offensively so. I would just
write the same DIP if I started again. I genuinely hope someone can be
bothered to do this. After 10 years on this, I think I'm over it.
6. "This result was frustrating and disheartening on our side, too: a
stronger DIP ..."  <--  I'm sorry you hated it. You could have
reviewed it at any point, made suggestions at any point, written it
yourself, or encouraged someone competent to do it.
7. Your general tone is superior, as usual.

If it's so terrible as you say, after having survived community
treatment and approval, then I suggest there's a serious problem in
process. As you've already made clear, I'm not qualified to address
this.


 Hope this helps,


Sure.
Jan 29 2019
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 29 January 2019 at 08:35:11 UTC, Manu wrote:

 4. "Under DIP 1016, a call with any T[] will silently "succeed" 
  by
 converting the slice to void[]"  <--  Do you mean "... with any 
 T[]  rvalue ..."? What would be the aim of that call? Can you 
 suggest a particularly sinister construction?


I _think_ what is meant is:

void[] allocate(size_t size);
bool reallocate(ref void[] b, size_t s);
void deallocate(ref void[]);

T[] arr = allocate(42);
arr.reallocate(8192); // reallocates a temporary as T[] is cast 
to void
arr.deallocate(); // double free
Jan 29 2019
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/29/19 6:38 AM, Nicholas Wilson wrote:

 On Tuesday, 29 January 2019 at 08:35:11 UTC, Manu wrote:

 4. "Under DIP 1016, a call with any T[] will silently "succeed"  by
 converting the slice to void[]"  <--  Do you mean "... with any T[]  
 rvalue ..."? What would be the aim of that call? Can you suggest a 
 particularly sinister construction?


 
 I _think_ what is meant is:
 
 void[] allocate(size_t size);
 bool reallocate(ref void[] b, size_t s);
 void deallocate(ref void[]);
 
 T[] arr = allocate(42);
 arr.reallocate(8192); // reallocates a temporary as T[] is cast to void
 arr.deallocate(); // double free


Affirmative. (Just wrote about the same in another post). Thanks very much.

Andrei
Jan 29 2019
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/29/19 3:35 AM, Manu wrote:

 1. All of this is more useful criticism than the official and final
 criticism affixed to the rejection, which when revised to remove the
 incorrect criticisms, is basically left with the text "The Language
 Maintainers found other issues with the proposal, most of which may
 have been remedied through simple revision"


No. This is a nonnegotiable matter:

void bump(ref int x) { ++x; }
short y = 42;
bump(y);
assert(y == 43); // fails

The code above should not compile. The DIP allows it to compile. We 
didn't see how the DIP can be lightly edited to fix this problem, so we 
recommended a complete rethinking and rewrite.


 2. All of this criticism could have been given at any point in the
 past half a year or so prior to submission, and that would have been
 appreciated, rather than wasting our time.


We have given this criticism ever since ten years ago when you brought 
the matter up. Literally every time, including in person. Whenever you 
said "why not bind rvalues to ref?" we religiously replied with 
(paraphrased of course) "We are worried about rvalues resulting from 
implicit conversion of lvalues. You'd need to find a solution to that."

It is truly remarkable that DIP 1016 provides not only a solution to the 
problem, but almost neglects to mention it.


 3. "It does not influence our decision and should not be construed as
 an essential aspect of the review"  <--  Then why did it feature as
 one of just 3 core criticism in the rejection text? And supplied as
 one of the 2 reasons that could not "have been remedied through simple
 revision".


If the matter above can be resolve through simple revision that would be 
great. I don't think it can, but would love to be proven wrong!


 4. "Under DIP 1016, a call with any T[] will silently "succeed" by
 converting the slice to void[]"  <--  Do you mean "... with any T[]
 rvalue ..."? What would be the aim of that call? Can you suggest a
 particularly sinister construction?


I am talking about this:

int[] a = cast(int[]) alloc.allocate(100 * int.sizeof);
if (alloc.reallocate(a, 200 * int.sizeof)
{
     assert(a.length == 200);
}

By applying the lowering rules in the DIP (including your pending 
revision), the code is lowered to:

int[] a = cast(int[]) alloc.allocate(100 * int.sizeof);
void[] __temp0 = a;
if (alloc.reallocate(__temp0, 200 * int.sizeof)
{
     assert(a.length == 200);
}

The code is far from sinister. But the assertion will fail.

I want to make sure there is mutual understanding that:

1. DIP 1016 proposes this semantics
2. We do not accept this semantics

Even if there is no agreement with (2), these are facts that I want to 
make sure are understood by everyone involved - you, us, the rest of the 
community.


 5. "and recommended that it be rewritten simply because it would be
 easier and would engender a stronger DIP."  <--  I wrote the DIP I
 wrote... your official feedback affixed to the bottom of the DIP was
 pretty much entirely unhelpful, almost offensively so. I would just
 write the same DIP if I started again. I genuinely hope someone can be
 bothered to do this. After 10 years on this, I think I'm over it.


I am sorry you found the review unfit for your needs. I thought it puts 
the main matter plain and simple: we do not accept code like this:

void bump(ref int x) { ++x; }
short y = 42;
bump(y);
assert(y == 43); // fails

Honest, I don't think you have spent 10 years on this DIP. It is not 
thorough work and fails to address the main problem it was supposed to 
address: The one above.


 6. "This result was frustrating and disheartening on our side, too: a
 stronger DIP ..."  <--  I'm sorry you hated it. You could have
 reviewed it at any point, made suggestions at any point, written it
 yourself, or encouraged someone competent to do it.


We didn't hate it. We made suggestions literally every time you brought 
up the issue over the past decade. For my part, I have no idea what more 
input I could have provided. "Solve the rvalues coming from implicit 
conversions" is all feedback the DIP needs.


 7. Your general tone is superior, as usual.


No need to attempt to make this personal, and claim the moral ground in 
the process. Nobody is after you. We all want the D language to become 
better. The DIP is not good. This is what it is.

All is not lost - the DIP is a good inspiration for a couple of ideas 
that deserve investigation. It seems that allowing _only_ rvalues to be 
bound to ref may work. I'm cautious because there may be cases we didn't 
think of.


Andrei
Jan 29 2019
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/29/19 6:45 AM, Andrei Alexandrescu wrote:

 It is truly remarkable that DIP 1016 provides not only a solution to the 
 problem, but almost neglects to mention it.


Meant "...not only no solution..."
Jan 29 2019
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/29/2019 3:45 AM, Andrei Alexandrescu wrote:

 I am talking about this:
 
 int[] a = cast(int[]) alloc.allocate(100 * int.sizeof);
 if (alloc.reallocate(a, 200 * int.sizeof)
 {
      assert(a.length == 200);
 }


Even simpler:

   void func(ref void* p) {
     free(p);	                  // frees (1)
     p = malloc(100);              // (2)
   }

   int* p = cast(int*)malloc(16);  // (1)
   func(p);                        // p copied to temp for conversion to void*
   free(p);                        // frees (1) again
                                   // (2) is left dangling

It's a memory corruption issue, with no way to detect it.
Jan 29 2019
parent reply Manu <turkeyman gmail.com> writes:
On Tue., 29 Jan. 2019, 10:25 pm Walter Bright via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com wrote:


 On 1/29/2019 3:45 AM, Andrei Alexandrescu wrote:

 I am talking about this:

 int[] a = cast(int[]) alloc.allocate(100 * int.sizeof);
 if (alloc.reallocate(a, 200 * int.sizeof)
 {
      assert(a.length == 200);
 }


 Even simpler:

    void func(ref void* p) {
      free(p);                     // frees (1)
      p = malloc(100);              // (2)
    }

    int* p = cast(int*)malloc(16);  // (1)
    func(p);                        // p copied to temp for conversion to
 void*
    free(p);                        // frees (1) again
                                    // (2) is left dangling

 It's a memory corruption issue, with no way to detect it.


Why are you so stuck on this case? The DIP is about accepting rvalues, not
lvalues...
Calling with 'p', an lvalue, is not subject to this DIP.






 Jan 30 2019



  parent reply Neia Neutuladh <neia ikeran.org>  writes:


On Wed, 30 Jan 2019 09:15:36 -0800, Manu wrote:

 Why are you so stuck on this case? The DIP is about accepting rvalues,
 not lvalues...
 Calling with 'p', an lvalue, is not subject to this DIP.


The result of a CastExpression is an rvalue. An implicit cast is a 
compiler-inserted CastExpression. Therefore all lvalues with a potential 
implicit cast are rvalues.



 Jan 30 2019



  parent reply Manu <turkeyman gmail.com>  writes:


On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Wed, 30 Jan 2019 09:15:36 -0800, Manu wrote:

 Why are you so stuck on this case? The DIP is about accepting rvalues,
 not lvalues...
 Calling with 'p', an lvalue, is not subject to this DIP.


 The result of a CastExpression is an rvalue. An implicit cast is a
 compiler-inserted CastExpression. Therefore all lvalues with a potential
 implicit cast are rvalues.


But there's no existing language rule that attempts to perform an
implicit cast where an lvalue is supplied to a ref arg...?
Why is the cast being attempted? 'p' is an lvalue, and whatever that
does should remain exactly as is (ie, emits a compile error).

We could perhaps allow this for `const` args, but that feels like
separate follow-up work to me, and substantially lesser value. This
DIP doesn't want to change anything about lvalues.



 Jan 30 2019



 next sibling parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 30 January 2019 at 18:29:37 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 On Wed, 30 Jan 2019 09:15:36 -0800, Manu wrote:

 Why are you so stuck on this case? The DIP is about 
 accepting rvalues,
 not lvalues...
 Calling with 'p', an lvalue, is not subject to this DIP.


 The result of a CastExpression is an rvalue. An implicit cast 
 is a compiler-inserted CastExpression. Therefore all lvalues 
 with a potential implicit cast are rvalues.


 But there's no existing language rule that attempts to perform 
 an
 implicit cast where an lvalue is supplied to a ref arg...?
 Why is the cast being attempted?


Because of the rewrite that your proposed in your dip.

void fun(ref int x);

fun(10);

{
   T __temp0 = void;
   fun(__temp0 := 10);
}

lets replace 10 with a short variable named: S

void fun(ref int x);

fun(S)
{
   T __temp0 = void;
   fun(__temp0 := S);
}
fun(__temp0 := S) This is where the cast is being attempted. As 
__temp0 is an                            integer type and S is a 
short type



 Jan 30 2019



  parent reply Manu <turkeyman gmail.com>  writes:


On Wed, Jan 30, 2019 at 12:40 PM 12345swordy via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Wednesday, 30 January 2019 at 18:29:37 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com>
 wrote:

 On Wed, 30 Jan 2019 09:15:36 -0800, Manu wrote:

 Why are you so stuck on this case? The DIP is about
 accepting rvalues,
 not lvalues...
 Calling with 'p', an lvalue, is not subject to this DIP.


 The result of a CastExpression is an rvalue. An implicit cast
 is a compiler-inserted CastExpression. Therefore all lvalues
 with a potential implicit cast are rvalues.


 But there's no existing language rule that attempts to perform
 an
 implicit cast where an lvalue is supplied to a ref arg...?
 Why is the cast being attempted?


 Because of the rewrite that your proposed in your dip.

 void fun(ref int x);

 fun(10);

 {
    T __temp0 = void;
    fun(__temp0 := 10);
 }

 lets replace 10 with a short variable named: S


"a short variable named: S" is an lvalue, so why would the rewrite be
attempted? S must be an rvalue for any rewrite to occur. We're talking
about rvalues here.



 Jan 30 2019



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 1/30/2019 5:55 PM, Manu wrote:

 lets replace 10 with a short variable named: S


 "a short variable named: S" is an lvalue, so why would the rewrite be
 attempted? S must be an rvalue for any rewrite to occur. We're talking
 about rvalues here.


This illustrates why this should be compared with C++. Consider this C++ code:

   const int& foo(const int& x) { return x; }

   const int& test() {
     short s;
     return foo(s);
   }

It compiles with clang++. The code generated for test() is:

         push    RBP
         mov     RBP,RSP
         sub     RSP,010h
         lea     RDI,-8[RBP]
         movsx   EAX,word ptr -2[RBP]
         mov     -8[RBP],EAX
         call    foo
         add     RSP,010h
         pop     RBP
         ret

See what it is doing? It's converting s to an int, putting the int into a 
temporary, then passing a reference to that temporary to foo(). So when you ask 
why would a person think that this would happen with the DIP, if they know C++, 
they would assume similar behavior. This is why the DIP needs to specifically 
say this is not the proposed behavior. It is why a comparison to C++ behavior
is 
essential. It is a lot easier to understand the DIP if people can apply their 
existing understanding, with a few modifications, to the D behavior. It's also 
necessary to compare with C++ to see if the DIP missed something important, and 
to justify any other behavioral differences.

The interesting question is, since C++ supports this behavior, what about the 
truck-sized hole? The answer is in the declaration of foo(const int&). The
const 
is the reason. The referenced value cannot be modified. The realloc example is 
blocked by the compiler.

But the DIP says const ref is not required. Therefore, copying an lvalue to a 
temporary cannot be allowed, therefore implicit conversion of lvalues cannot
happen.

Then we're faced with the question of if implicit conversion of lvalues is not 
allowed, should implicit conversion of rvalues be allowed? I'm not so sure it 
should be. For one thing, a lot of people are confused about lvalues vs
rvalues, 
and would find the difference in behavior puzzling. For another, it can 
complicate overloading rules. I'd say allowing the conversions needs a strong 
rationale.



 Jan 31 2019



  parent  Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/31/19 3:25 AM, Walter Bright wrote:

 But the DIP says const ref is not required. Therefore, copying an lvalue 
 to a temporary cannot be allowed, therefore implicit conversion of 
 lvalues cannot happen.


The biggest reason I see to not worry about const is that we already 
don't for member functions. And the world hasn't ended (yet).


 
 Then we're faced with the question of if implicit conversion of lvalues 
 is not allowed, should implicit conversion of rvalues be allowed? I'm 
 not so sure it should be. For one thing, a lot of people are confused 
 about lvalues vs rvalues, and would find the difference in behavior 
 puzzling. For another, it can complicate overloading rules. I'd say 
 allowing the conversions needs a strong rationale.


We could certainly start out with no conversions allowed (except you 
MUST allow conversions of compile-time data -- e.g. literals and CTFE 
produced values -- that is very key), and then relax the rules later if 
that's important.

-Steve



 Jan 31 2019



prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com>  writes:


On 1/30/19 1:29 PM, Manu wrote:

 On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Wed, 30 Jan 2019 09:15:36 -0800, Manu wrote:

 Why are you so stuck on this case? The DIP is about accepting rvalues,
 not lvalues...
 Calling with 'p', an lvalue, is not subject to this DIP.


 The result of a CastExpression is an rvalue. An implicit cast is a
 compiler-inserted CastExpression. Therefore all lvalues with a potential
 implicit cast are rvalues.


 
 But there's no existing language rule that attempts to perform an
 implicit cast where an lvalue is supplied to a ref arg...?


That's exactly what lowerings are for: to precisely specify what should 
happen when the proposed construct is used. DIP 1016 proposes a lowering 
of the form:

{
   T __temp0 = expr;
   fun(__temp0);
}

In the first step, an implicit conversion of an lvalue may take place.


 Why is the cast being attempted? 'p' is an lvalue, and whatever that
 does should remain exactly as is (ie, emits a compile error).


Not according to DIP 1016. Here is an example pasted from it:

========
This inconvenience extends broadly to every manner of rvalue passed to 
functions, including:
...
fun(my_short); // implicit type conversions (ie, short->int promotion)
========

Presumably my_short is a variable of type short. Is that correct?

Again (this is not a rhetorical or sarcastic question): are you sure DIP 
1016 expresses what you are trying to accomplish?


 We could perhaps allow this for `const` args, but that feels like
 separate follow-up work to me, and substantially lesser value. This
 DIP doesn't want to change anything about lvalues.


What we have here is:

* DIP 1016 proposes a hole in the language one could drive a truck through.

* The problem goes undetected in community review.

* Its own author seems to not have an understanding of what the DIP 
proposes.


Andrei



 Jan 30 2019



  parent reply Manu <turkeyman gmail.com>  writes:


On Wed, Jan 30, 2019 at 1:05 PM Andrei Alexandrescu via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/30/19 1:29 PM, Manu wrote:

 On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Wed, 30 Jan 2019 09:15:36 -0800, Manu wrote:

 Why are you so stuck on this case? The DIP is about accepting rvalues,
 not lvalues...
 Calling with 'p', an lvalue, is not subject to this DIP.


 The result of a CastExpression is an rvalue. An implicit cast is a
 compiler-inserted CastExpression. Therefore all lvalues with a potential
 implicit cast are rvalues.


 But there's no existing language rule that attempts to perform an
 implicit cast where an lvalue is supplied to a ref arg...?


 That's exactly what lowerings are for: to precisely specify what should
 happen when the proposed construct is used. DIP 1016 proposes a lowering
 of the form:

 {
    T __temp0 = expr;
    fun(__temp0);
 }

 In the first step, an implicit conversion of an lvalue may take place.


What? `expr` must be an rvalue... if it's not, then none of this applies.



 Why is the cast being attempted? 'p' is an lvalue, and whatever that
 does should remain exactly as is (ie, emits a compile error).


 Not according to DIP 1016. Here is an example pasted from it:

 ========
 This inconvenience extends broadly to every manner of rvalue passed to
 functions, including:
 ...
 fun(my_short); // implicit type conversions (ie, short->int promotion)
 ========


Oh I see.


 fun(short(10)); // implicit type conversions (ie, short->int promotion)


I did not intend for this DIP to apply to anything other than rvalues.
I can totally see how that's not clear. `my_short` should be an rvalue
of some form, like the rest.
Is that the only such line?


 Presumably my_short is a variable of type short. Is that correct?


It is not. It should be an rvalue like everything else. Perhaps it's
an enum... but I should write `short(10)`, that would be clear.


 Again (this is not a rhetorical or sarcastic question): are you sure DIP
 1016 expresses what you are trying to accomplish?


Fairly certain.


 We could perhaps allow this for `const` args, but that feels like
 separate follow-up work to me, and substantially lesser value. This
 DIP doesn't want to change anything about lvalues.


 What we have here is:

 * DIP 1016 proposes a hole in the language one could drive a truck through.


I still can't see a truck-sized hole.


 * The problem goes undetected in community review.


I don't know how I could have influenced this outcome.


 * Its own author seems to not have an understanding of what the DIP
 proposes.


More classy comments. I can't get enough of the way you belittle people.

I made a 1-word error, where I should have written `short(10)` to be clear.
1-word error feels amendment-worthy, and not a call for "let's start
over from scratch".



 Jan 30 2019



 next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com>  writes:


On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 1:05 PM Andrei Alexandrescu via

 fun(my_short); // implicit type conversions (ie, short->int 
 promotion)
 ========


 Oh I see.


 fun(short(10)); // implicit type conversions (ie, short->int 
 promotion)


 I did not intend for this DIP to apply to anything other than 
 rvalues.
 I can totally see how that's not clear. `my_short` should be an 
 rvalue
 of some form, like the rest.
 Is that the only such line?


I think so.


 Presumably my_short is a variable of type short. Is that 
 correct?


 It is not. It should be an rvalue like everything else. Perhaps 
 it's an enum... but I should write `short(10)`, that would be 
 clear.


It would.


 * DIP 1016 proposes a hole in the language one could drive a 
 truck through.


 I still can't see a truck-sized hole.


 * The problem goes undetected in community review.


 I don't know how I could have influenced this outcome.


 * Its own author seems to not have an understanding of what 
 the DIP proposes.


 More classy comments. I can't get enough of the way you 
 belittle people.

 I made a 1-word error, where I should have written `short(10)` 
 to be clear.
 1-word error feels amendment-worthy, and not a call for "let's 
 start
 over from scratch".


You should just PR it back to review with that fix and a note 
about how it lowers to statements (incl. an example of 
lambdification for if/while/for/switch statements (see 
https://forum.dlang.org/post/qysmnatmjquuhylaqumm forum.dlang.org 
))



 Jan 30 2019



  parent reply Manu <turkeyman gmail.com>  writes:


On Wed, Jan 30, 2019 at 7:05 PM Nicholas Wilson via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 1:05 PM Andrei Alexandrescu via

 fun(my_short); // implicit type conversions (ie, short->int
 promotion)
 ========


 Oh I see.


 fun(short(10)); // implicit type conversions (ie, short->int
 promotion)


 I did not intend for this DIP to apply to anything other than
 rvalues.
 I can totally see how that's not clear. `my_short` should be an
 rvalue
 of some form, like the rest.
 Is that the only such line?


 I think so.


 Presumably my_short is a variable of type short. Is that
 correct?


 It is not. It should be an rvalue like everything else. Perhaps
 it's an enum... but I should write `short(10)`, that would be
 clear.


 It would.


 * DIP 1016 proposes a hole in the language one could drive a
 truck through.


 I still can't see a truck-sized hole.


 * The problem goes undetected in community review.


 I don't know how I could have influenced this outcome.


 * Its own author seems to not have an understanding of what
 the DIP proposes.


 More classy comments. I can't get enough of the way you
 belittle people.

 I made a 1-word error, where I should have written `short(10)`
 to be clear.
 1-word error feels amendment-worthy, and not a call for "let's
 start
 over from scratch".


 You should just PR it back to review


I can't do that, it's been rejected, with mostly incorrect rejection
text affixed to the bottom.


 with that fix and a note
 about how it lowers to statements (incl. an example of
 lambdification for if/while/for/switch statements (see
 https://forum.dlang.org/post/qysmnatmjquuhylaqumm forum.dlang.org
 ))


I'm pretty sure that's not necessary. I haven't understood why this
noise about expressions. This DIP applies to statements.
I can't see how there's any problem with the lowering if the statement
is a control statement?

if (ref_fun(10)) { ... }
==>
{
  int __tmp = 10;
  if (ref_fun(__tmp)) { ... }
}

What's the trouble?



 Jan 30 2019



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 1/30/19 10:12 PM, Manu wrote:

 On Wed, Jan 30, 2019 at 7:05 PM Nicholas Wilson via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 1:05 PM Andrei Alexandrescu via

 fun(my_short); // implicit type conversions (ie, short->int
 promotion)
 ========


 Oh I see.


 fun(short(10)); // implicit type conversions (ie, short->int
 promotion)


 I did not intend for this DIP to apply to anything other than
 rvalues.
 I can totally see how that's not clear. `my_short` should be an
 rvalue
 of some form, like the rest.
 Is that the only such line?


 I think so.


 Presumably my_short is a variable of type short. Is that
 correct?


 It is not. It should be an rvalue like everything else. Perhaps
 it's an enum... but I should write `short(10)`, that would be
 clear.


 It would.


 * DIP 1016 proposes a hole in the language one could drive a
 truck through.


 I still can't see a truck-sized hole.


 * The problem goes undetected in community review.


 I don't know how I could have influenced this outcome.


 * Its own author seems to not have an understanding of what
 the DIP proposes.


 More classy comments. I can't get enough of the way you
 belittle people.

 I made a 1-word error, where I should have written `short(10)`
 to be clear.
 1-word error feels amendment-worthy, and not a call for "let's
 start
 over from scratch".


 You should just PR it back to review


 
 I can't do that, it's been rejected, with mostly incorrect rejection
 text affixed to the bottom.
 

 with that fix and a note
 about how it lowers to statements (incl. an example of
 lambdification for if/while/for/switch statements (see
 https://forum.dlang.org/post/qysmnatmjquuhylaqumm forum.dlang.org
 ))


 
 I'm pretty sure that's not necessary. I haven't understood why this
 noise about expressions. This DIP applies to statements.
 I can't see how there's any problem with the lowering if the statement
 is a control statement?
 
 if (ref_fun(10)) { ... }
 ==>
 {
    int __tmp = 10;
    if (ref_fun(__tmp)) { ... }
 }
 
 What's the trouble?


The trouble is major.

Replace "if" with "while":

while (ref_fun(10)) { ... }
==>
{
   int __tmp = 10;
   while (ref_fun(__tmp)) { ... }
}

That means ref_fun is called with the same lvalue multiple times. In all 
likelihood this is not what you want!

A possible retort is: "Of course, while would not be lowered that way, 
but a slightly different way!" etc. The point is, ALL OF THAT must be in 
the DIP, not assumed obvious or clarified in informal discusson outside 
the DIP.

Again: please be thorough, state your assumptions, cover all cases.



 Jan 30 2019



  parent  Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/31/19 2:26 AM, Andrei Alexandrescu wrote:

 The trouble is major.
 
 Replace "if" with "while":
 
 while (ref_fun(10)) { ... }
 ==>
 {
    int __tmp = 10;
    while (ref_fun(__tmp)) { ... }
 }
 
 That means ref_fun is called with the same lvalue multiple times. In all 
 likelihood this is not what you want!


Yes, the trouble specifically is loops. Because loops execute their 
internal expressions over and over again.

Unfortunately, this means lowering isn't possible. That is, lowering to 
something expressible in the normal language isn't possible.

However, we all know that loops are essentially "lowered" in the AST to 
simple ifs and gotos. We just need to operate at that level.

So for instance the above looks something like this in AST:

loop_continue:
if(ref_fun(10))
{
   ...
}
else
    goto loop_end;
goto loop_continue;
loop_end:

(I know I'm omitting a lot of extra stuff like scope cleanup, that is 
implied here, as I don't know the exact details).

What needs to happen is the temporary (with extra scope)is inserted 
between the loop start and the if statement:

loop_continue:
{
    int __tmp = 10;
    if(ref_fun(__tmp))
    {
       ...
    }
    else
       goto loop_end;
}
goto loop_continue;
loop_end:


 A possible retort is: "Of course, while would not be lowered that way, 
 but a slightly different way!" etc. The point is, ALL OF THAT must be in 
 the DIP, not assumed obvious or clarified in informal discusson outside 
 the DIP.
 
 Again: please be thorough, state your assumptions, cover all cases.


Agree, this needs to handle all possible cases. Really I think loops are 
the only problems, foreach, for, and while/do..while

A possible way forward is inventing a new syntax to allow declarations 
in this space, and then lowering can happen. Something similar to 
if(auto x = ...)

-Steve



 Jan 31 2019



prev sibling next sibling parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 1/30/19 9:10 PM, Manu wrote:

 * Its own author seems to not have an understanding of what the DIP
 proposes.


 
 More classy comments. I can't get enough of the way you belittle people.


You're right. I have deleted this post a few seconds after having sent 
it on account of that remark, but somehow it got resuscitated. Please 
accept my apologies.



 Jan 30 2019



prev sibling  parent reply Olivier FAURE <couteaubleu gmail.com>  writes:


On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

 I still can't see a truck-sized hole.


I don't know if it's truck-sized, but here's another corner case:

     int doubleMyValue(ref int x) {
         x *= 2;
         return x;
     }

     Point pt;
     pt.x = 5;
     pt.y = foobar();

     doubleMyValue(pt.x);
     assert(pt.x == 10);

Question: in the above code, will the assertion pass?

Answer: it depends on Point's implementation. If x is a member 
variable, then yes. If it's a getter, then doubleMyValue will 
take a rvalue and x won't be mutated and the assertion will fail.

I think this is a non-trivial conceptual problem.



 Jan 31 2019



 next sibling parent reply Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/31/19 11:04 AM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

 I still can't see a truck-sized hole.


 
 I don't know if it's truck-sized, but here's another corner case:
 
      int doubleMyValue(ref int x) {
          x *= 2;
          return x;
      }
 
      Point pt;
      pt.x = 5;
      pt.y = foobar();
 
      doubleMyValue(pt.x);
      assert(pt.x == 10);
 
 Question: in the above code, will the assertion pass?
 
 Answer: it depends on Point's implementation. If x is a member variable, 
 then yes. If it's a getter, then doubleMyValue will take a rvalue and x 
 won't be mutated and the assertion will fail.
 
 I think this is a non-trivial conceptual problem.


Yeah, that's already a thing that ref in D doesn't protect against:

struct Point
{
    private int _x, _y;
    ref int x() { return _x; }
    ref int y() { return _y; }
}

struct Rect
{
    private Point _origin, _lengths;
    Point origin() { return _origin; }
    Point lengths() { return _lengths; }
    void origin(Point p) { _origin = p; }
    void lengths(Point p) { _lengths = p; }
}

Rect r;
r.origin = Point(1, 2);
r.lengths = Point(5, 5);
doubleMyValue(r.lengths.x);
assert(r.lengths.x == 10); // fail

-Steve



 Jan 31 2019



 next sibling parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.com>  writes:


On 1/31/19 11:38 AM, Steven Schveighoffer wrote:

 On 1/31/19 11:04 AM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

 I still can't see a truck-sized hole.


 I don't know if it's truck-sized, but here's another corner case:

      int doubleMyValue(ref int x) {
          x *= 2;
          return x;
      }

      Point pt;
      pt.x = 5;
      pt.y = foobar();

      doubleMyValue(pt.x);
      assert(pt.x == 10);

 Question: in the above code, will the assertion pass?

 Answer: it depends on Point's implementation. If x is a member 
 variable, then yes. If it's a getter, then doubleMyValue will take a 
 rvalue and x won't be mutated and the assertion will fail.

 I think this is a non-trivial conceptual problem.


 
 Yeah, that's already a thing that ref in D doesn't protect against:
 
 struct Point
 {
     private int _x, _y;
     ref int x() { return _x; }
     ref int y() { return _y; }
 }
 
 struct Rect
 {
     private Point _origin, _lengths;
     Point origin() { return _origin; }
     Point lengths() { return _lengths; }
     void origin(Point p) { _origin = p; }
     void lengths(Point p) { _lengths = p; }
 }
 
 Rect r;
 r.origin = Point(1, 2);
 r.lengths = Point(5, 5);
 doubleMyValue(r.lengths.x);
 assert(r.lengths.x == 10); // fail
 
 -Steve


Affirmative. This discussion should be part of the revised DIP along 
with an assessment of its gravity.

Goes the same with scope-level variables replaced with homonym functions 
that return rvalues.


Andrei



 Jan 31 2019



prev sibling  parent reply Olivier FAURE <couteaubleu gmail.com>  writes:


On Thursday, 31 January 2019 at 16:38:42 UTC, Steven 
Schveighoffer wrote:

 Yeah, that's already a thing that ref in D doesn't protect 
 against:


It took me a while to understand what the compiler was doing.

This really feels like something that shouldn't compile.



 Jan 31 2019



 next sibling parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Thursday, 31 January 2019 at 21:42:04 UTC, Olivier FAURE wrote:

 [snip]

 It took me a while to understand what the compiler was doing.

 This really feels like something that shouldn't compile.


It doesn't compile with dip1000 without first giving the getter 
functions a return attribute for this.



 Jan 31 2019



  parent reply Olivier FAURE <couteaubleu gmail.com>  writes:


On Thursday, 31 January 2019 at 21:44:53 UTC, jmh530 wrote:

 It doesn't compile with dip1000 without first giving the getter 
 functions a return attribute for this.


But it still compiles with -dip1000 once you give x() and y() 
return attributes, even though what's happening is clearly 
different from what the user wants (and the compiler has enough 
info to know that).



 Jan 31 2019



  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Thursday, 31 January 2019 at 21:50:19 UTC, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 21:44:53 UTC, jmh530 wrote:

 It doesn't compile with dip1000 without first giving the 
 getter functions a return attribute for this.


 But it still compiles with -dip1000 once you give x() and y() 
 return attributes, even though what's happening is clearly 
 different from what the user wants (and the compiler has enough 
 info to know that).


Agreed. I had checked that it didn't work and as I figured out 
how to get it work I got distracted reading the documentation and 
return and scope.



 Jan 31 2019



prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.com>  writes:


On 1/31/19 4:42 PM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 16:38:42 UTC, Steven Schveighoffer wrote:

 Yeah, that's already a thing that ref in D doesn't protect against:


 
 It took me a while to understand what the compiler was doing.
 
 This really feels like something that shouldn't compile.


The proposal could actually disallow rvalues that have lvalue syntax, 
such as "symbol", "symbol[expr]", "symbol.symbol", 
"symbol.symbol[expr]", etc. Ugh. Gets hairy quickly.



 Jan 31 2019



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 1/31/2019 1:46 PM, Andrei Alexandrescu wrote:

 The proposal could actually disallow rvalues that have lvalue syntax, such as 
 "symbol", "symbol[expr]", "symbol.symbol", "symbol.symbol[expr]", etc. Ugh.
Gets 
 hairy quickly.


That's why it's problematic to have a rule that rvalues can be implicitly 
converted, but not lvalues. There's not a hard line between lvalues and
rvalues. 
For example,

   foreach(i; 0..2)
   {
     int[] a = [1, 2];
     assert(a[0] == 1]);
     a[0] = 3;  // will this cause the assert to fail?
   }



 Jan 31 2019



  parent  Rubn <where is.this>  writes:


On Thursday, 31 January 2019 at 22:00:10 UTC, Walter Bright wrote:

 On 1/31/2019 1:46 PM, Andrei Alexandrescu wrote:

 The proposal could actually disallow rvalues that have lvalue 
 syntax, such as "symbol", "symbol[expr]", "symbol.symbol", 
 "symbol.symbol[expr]", etc. Ugh. Gets hairy quickly.


 That's why it's problematic to have a rule that rvalues can be 
 implicitly converted, but not lvalues. There's not a hard line 
 between lvalues and rvalues. For example,

   foreach(i; 0..2)
   {
     int[] a = [1, 2];
     assert(a[0] == 1]);
     a[0] = 3;  // will this cause the assert to fail?
   }


Why would it cause the assert to fail? A new array is constructed 
each loop.



 Jan 31 2019



prev sibling  parent  Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/31/19 4:46 PM, Andrei Alexandrescu wrote:

 On 1/31/19 4:42 PM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 16:38:42 UTC, Steven Schveighoffer wrote:

 Yeah, that's already a thing that ref in D doesn't protect against:


 It took me a while to understand what the compiler was doing.

 This really feels like something that shouldn't compile.


 
 The proposal could actually disallow rvalues that have lvalue syntax, 
 such as "symbol", "symbol[expr]", "symbol.symbol", 
 "symbol.symbol[expr]", etc. Ugh. Gets hairy quickly.


No, because those calls might actually do something with side effects! 
rvalues can contain other references.

The only way to fix this would be to overload member functions on the 
lvalue-ness of `this`. I don't recommend this at all, as I see this as a 
weird but rare problem that doesn't affect most D code.

-Steve



 Jan 31 2019



prev sibling  parent reply Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/31/19 4:42 PM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 16:38:42 UTC, Steven Schveighoffer wrote:

 Yeah, that's already a thing that ref in D doesn't protect against:


 
 It took me a while to understand what the compiler was doing.
 
 This really feels like something that shouldn't compile.


The problem is that `this` is passed by reference EVEN for rvalues. 
Knowing this, you can construct difficult situations, but normally these 
don't appear in the wild.

If you've ever tried to make a simple math wrapper type, you would see 
how this is weird. And it has been like this since the beginning of D2.

You get things like:

a + Foo(1); // error
Foo(1) + a; // OK!

That being said, you can look at the fact that most people don't even 
know about this problem, even seasoned veterans, as a sign that it's 
really not a big problem.

-Steve



 Jan 31 2019



 next sibling parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Thursday, 31 January 2019 at 21:57:21 UTC, Steven 
Schveighoffer wrote:

 [snip]

 That being said, you can look at the fact that most people 
 don't even know about this problem, even seasoned veterans, as 
 a sign that it's really not a big problem.


The way you put it makes it sound like a bug...

I don't know if it helps, but below compiles without error.

struct Foo
{
    private int _x;
    int* x() { return &_x; }
}

struct Bar
{
    private Foo _y;
    Foo* y() { return &_y; }
    void y(Foo foo) { _y = foo; }
}

void main() {
     Foo a = Foo(1);
     assert(*a.x == 1);
     *a.x *= 2;
     assert(*a.x == 2);

     Bar b;
     b.y = Foo(1);
     assert(*b.y.x == 1);
     *b.y.x *= 2;
     assert(*b.y.x == 2);
}



 Jan 31 2019



  parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Thu, Jan 31, 2019 at 10:26:39PM +0000, jmh530 via Digitalmars-d-announce
wrote:

 On Thursday, 31 January 2019 at 21:57:21 UTC, Steven Schveighoffer wrote:


[...]

 That being said, you can look at the fact that most people don't
 even know about this problem, even seasoned veterans, as a sign that
 it's really not a big problem.
 


 
 The way you put it makes it sound like a bug...
 
 I don't know if it helps, but below compiles without error.
 
 struct Foo
 {
    private int _x;
    int* x() { return &_x; }
 }
 
 struct Bar
 {
    private Foo _y;
    Foo* y() { return &_y; }
    void y(Foo foo) { _y = foo; }
 }
 
 void main() {
     Foo a = Foo(1);
     assert(*a.x == 1);
     *a.x *= 2;
     assert(*a.x == 2);
 
     Bar b;
     b.y = Foo(1);
     assert(*b.y.x == 1);
     *b.y.x *= 2;
     assert(*b.y.x == 2);
 }


Why is it a problem that this code compiles without error?


T

-- 
Perhaps the most widespread illusion is that if we were in power we would
behave very differently from those who now hold it---when, in truth, in order
to get power we would have to become very much like them. -- Unknown



 Jan 31 2019



  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Thursday, 31 January 2019 at 22:35:26 UTC, H. S. Teoh wrote:

 On Thu, Jan 31, 2019 at 10:26:39PM +0000, jmh530 via 
 Digitalmars-d-announce wrote:

 On Thursday, 31 January 2019 at 21:57:21 UTC, Steven 
 Schveighoffer wrote:


 [...]

 That being said, you can look at the fact that most people 
 don't even know about this problem, even seasoned veterans, 
 as a sign that it's really not a big problem.
 


 
 The way you put it makes it sound like a bug...
 
 I don't know if it helps, but below compiles without error.
 
 struct Foo
 {
    private int _x;
    int* x() { return &_x; }
 }
 
 struct Bar
 {
    private Foo _y;
    Foo* y() { return &_y; }
    void y(Foo foo) { _y = foo; }
 }
 
 void main() {
     Foo a = Foo(1);
     assert(*a.x == 1);
     *a.x *= 2;
     assert(*a.x == 2);
 
     Bar b;
     b.y = Foo(1);
     assert(*b.y.x == 1);
     *b.y.x *= 2;
     assert(*b.y.x == 2);
 }


 Why is it a problem that this code compiles without error?


 T


Sorry if I didn't really complete my thought.

The code below corresponds to the ref version mentioned above and 
gets the same error originally reported. The only difference is 
that I use Foo instead of Foo* for the getter in Bar. So if you 
instead make that member function a ref function, then it also 
compiles without error (regardless of if you use the 
doubleMyValue function or not).

So they were right that the issue is with the getter. However, 
you're not really protected in any way if you have a ref getter 
at one point in a chain and a non-ref getter somewhere else. 
Making all the getters auto ref also avoids the issue.

struct Foo
{
    private int _x;
    ref int x() { return _x; }
}

struct Bar
{
    private Foo _y;
    Foo y() { return _y; }
    void y(Foo foo) { _y = foo; }
}

void main() {
     Foo a = Foo(1);
     assert(a.x == 1);
     a.x *= 2;
     assert(a.x == 2);

     Bar b;
     b.y = Foo(1);
     assert(b.y.x == 1);
     b.y.x *= 2;
     assert(b.y.x == 2);
}



 Jan 31 2019



prev sibling  parent  Olivier FAURE <couteaubleu gmail.com>  writes:


On Thursday, 31 January 2019 at 21:57:21 UTC, Steven 
Schveighoffer wrote:

 That being said, you can look at the fact that most people 
 don't even know about this problem, even seasoned veterans, as 
 a sign that it's really not a big problem.


Isn't it a recurring theme on this forum that D is really cool 
but also kind of obnoxious because of weird corner cases that 
veterans know, but aren't documented anywhere?



 Feb 01 2019



prev sibling  parent reply Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/31/19 11:04 AM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

 I still can't see a truck-sized hole.


 
 I don't know if it's truck-sized, but here's another corner case:
 
      int doubleMyValue(ref int x) {
          x *= 2;
          return x;
      }
 
      Point pt;
      pt.x = 5;
      pt.y = foobar();
 
      doubleMyValue(pt.x);
      assert(pt.x == 10);
 
 Question: in the above code, will the assertion pass?
 
 Answer: it depends on Point's implementation. If x is a member variable, 
 then yes. If it's a getter, then doubleMyValue will take a rvalue and x 
 won't be mutated and the assertion will fail.
 
 I think this is a non-trivial conceptual problem.


BTW, the DIP discusses how to annotate these rare situations:

int doubleMyValue(ref int x) { ... }
 disable int doubleMyValue(int x);

-Steve



 Jan 31 2019



  parent reply Olivier FAURE <couteaubleu gmail.com>  writes:


On Thursday, 31 January 2019 at 18:31:22 UTC, Steven 
Schveighoffer wrote:

 BTW, the DIP discusses how to annotate these rare situations:

 int doubleMyValue(ref int x) { ... }
  disable int doubleMyValue(int x);

 -Steve


I don't think that's a solution. The problem is in the getter 
method, not in doubleMyValue. If nothing else, since the DIP is 
designed to work on existing functions, it could happen on 
doubleMyValue functions which would be both designed by and used 
by people completely unaware of DIP-1016.



 Jan 31 2019



  parent reply Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/31/19 4:46 PM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 18:31:22 UTC, Steven Schveighoffer wrote:

 BTW, the DIP discusses how to annotate these rare situations:

 int doubleMyValue(ref int x) { ... }
  disable int doubleMyValue(int x);


 
 I don't think that's a solution. The problem is in the getter method, 
 not in doubleMyValue. If nothing else, since the DIP is designed to work 
 on existing functions, it could happen on doubleMyValue functions which 
 would be both designed by and used by people completely unaware of 
 DIP-1016.


How is the problem not in doubleMyValue? It's sole purpose is to update 
an lvalue. It is the perfect candidate to mark with  disable for rvalues.

-Steve



 Jan 31 2019



 next sibling parent reply aliak <something something.com>  writes:


On Thursday, 31 January 2019 at 21:50:32 UTC, Steven 
Schveighoffer wrote:

 On 1/31/19 4:46 PM, Olivier FAURE wrote:

 On Thursday, 31 January 2019 at 18:31:22 UTC, Steven 
 Schveighoffer wrote:

 BTW, the DIP discusses how to annotate these rare situations:

 int doubleMyValue(ref int x) { ... }
  disable int doubleMyValue(int x);


 
 I don't think that's a solution. The problem is in the getter 
 method, not in doubleMyValue. If nothing else, since the DIP 
 is designed to work on existing functions, it could happen on 
 doubleMyValue functions which would be both designed by and 
 used by people completely unaware of DIP-1016.


 How is the problem not in doubleMyValue? It's sole purpose is 
 to update an lvalue. It is the perfect candidate to mark with 
  disable for rvalues.

 -Steve


Shouldn't doubleMyValue(pt.x) be a compiler error if pt.x is a 
getter? For it not to be a compile error pt.x should also have a 
setter, in which case the code needs to be lowered to something 
else:

{
   auto __temp = pt.x;
   doubleMyValue(__temp);
   pt.x = __temp;
}


do as well.

Or something... a DIP to fix properties anyone? :)

Also, this applies to a much wider variety of operations on 
properties that return rvalues and not just on functions calls no?

struct B {
     int x;
}
struct A {
   B _b;
    property B b() { return _b; }
}

void main() {
     A a;
     a.b.x += 1;
     writeln(a.b.x); // 0
}



 Feb 01 2019



 next sibling parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 01.02.19 10:10, aliak wrote:


 
 Shouldn't doubleMyValue(pt.x) be a compiler error if pt.x is a getter? 
 For it not to be a compile error pt.x should also have a setter, in 
 which case the code needs to be lowered to something else:
 
 {
    auto __temp = pt.x;
    doubleMyValue(__temp);
    pt.x = __temp;
 }
 

 well.
 
 Or something... a DIP to fix properties anyone? :)


http://wilzbach.github.io/d-dip/DIP24

I'm not sure your rewrite is good though, because it does not preserve 
aliasing during the function call.



 Feb 01 2019



 next sibling parent  aliak <something something.com>  writes:


On Friday, 1 February 2019 at 11:48:51 UTC, Timon Gehr wrote:

 On 01.02.19 10:10, aliak wrote:


 
 Shouldn't doubleMyValue(pt.x) be a compiler error if pt.x is a 
 getter? For it not to be a compile error pt.x should also have 
 a setter, in which case the code needs to be lowered to 
 something else:
 
 {
    auto __temp = pt.x;
    doubleMyValue(__temp);
    pt.x = __temp;
 }
 
 I believe this is something along the lines of what Swift and 

 
 Or something... a DIP to fix properties anyone? :)


 http://wilzbach.github.io/d-dip/DIP24

 I'm not sure your rewrite is good though, because it does not 
 preserve aliasing during the function call.


You mean if __temp is modified in the doubleMyValue and pt.x 
aliases something else? Or? Wouldn't the assignment back "just 
work"?

And is there a rewrite that would work then?



 Feb 01 2019



prev sibling  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Friday, 1 February 2019 at 11:48:51 UTC, Timon Gehr wrote:

 On 01.02.19 10:10, aliak wrote:


 
 Shouldn't doubleMyValue(pt.x) be a compiler error if pt.x is a 
 getter? For it not to be a compile error pt.x should also have 
 a setter, in which case the code needs to be lowered to 
 something else:
 
 {
    auto __temp = pt.x;
    doubleMyValue(__temp);
    pt.x = __temp;
 }
 
 I believe this is something along the lines of what Swift and 

 
 Or something... a DIP to fix properties anyone? :)


 http://wilzbach.github.io/d-dip/DIP24

 I'm not sure your rewrite is good though, because it does not 
 preserve aliasing during the function call.



ref/out parameter.
(The properties should be redefined as accessor, such that you 
can't take the address of it)
-Alex



 Feb 01 2019



  parent reply Aliak <something something.com>  writes:


On Friday, 1 February 2019 at 14:41:52 UTC, 12345swordy wrote:

 On Friday, 1 February 2019 at 11:48:51 UTC, Timon Gehr wrote:

 On 01.02.19 10:10, aliak wrote:

 [...]


 http://wilzbach.github.io/d-dip/DIP24

 I'm not sure your rewrite is good though, because it does not 
 preserve aliasing during the function call.



 ref/out parameter.
 (The properties should be redefined as accessor, such that you 
 can't take the address of it)
 -Alex


By properties I mean accessors? I don’t mean normal fieldsat 

variables, properties, and accessors?



 Feb 01 2019



  parent  12345swordy <alexanderheistermann gmail.com>  writes:


On Friday, 1 February 2019 at 15:58:50 UTC, Aliak wrote:

 On Friday, 1 February 2019 at 14:41:52 UTC, 12345swordy wrote:

 On Friday, 1 February 2019 at 11:48:51 UTC, Timon Gehr wrote:

 On 01.02.19 10:10, aliak wrote:

 [...]


 http://wilzbach.github.io/d-dip/DIP24

 I'm not sure your rewrite is good though, because it does not 
 preserve aliasing during the function call.



 ref/out parameter.
 (The properties should be redefined as accessor, such that you 
 can't take the address of it)
 -Alex


 By properties I mean accessors? I don’t mean normal fieldsat 

 variables, properties, and accessors?


https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/properties



 Feb 01 2019



prev sibling  parent reply Olivier FAURE <couteaubleu gmail.com>  writes:


On Friday, 1 February 2019 at 09:10:15 UTC, aliak wrote:

 Shouldn't doubleMyValue(pt.x) be a compiler error if pt.x is a 
 getter? For it not to be a compile error pt.x should also have 
 a setter, in which case the code needs to be lowered to 
 something else:


The thing is, D doesn't really differentiate between a getter and 
any other method.

So with DIP-1016, when given

     doubleMyValue(pt.x);

The compiler would assume the programmer means
- Call pt.x()
- Store the result in a temporary
- Pass that temporary as a ref parameter to doubleMyValue

At no point is the compiler aware that the user intends for x to 
be interpreted as a getter.



 Feb 01 2019



 next sibling parent  Paul Backus <snarwin gmail.com>  writes:


On Friday, 1 February 2019 at 23:24:44 UTC, Olivier FAURE wrote:

 At no point is the compiler aware that the user intends for x 
 to be interpreted as a getter.


In theory, at least, that's what  property is for.



 Feb 01 2019



prev sibling  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Friday, 1 February 2019 at 23:24:44 UTC, Olivier FAURE wrote:

 On Friday, 1 February 2019 at 09:10:15 UTC, aliak wrote:

 Shouldn't doubleMyValue(pt.x) be a compiler error if pt.x is a 
 getter? For it not to be a compile error pt.x should also have 
 a setter, in which case the code needs to be lowered to 
 something else:


 The thing is, D doesn't really differentiate between a getter 
 and any other method.

 So with DIP-1016, when given

     doubleMyValue(pt.x);

 The compiler would assume the programmer means
 - Call pt.x()
 - Store the result in a temporary
 - Pass that temporary as a ref parameter to doubleMyValue

 At no point is the compiler aware that the user intends for x 
 to be interpreted as a getter.



property to ref parameter arguments.



 Feb 01 2019



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 2/1/19 8:15 PM, 12345swordy wrote:

 On Friday, 1 February 2019 at 23:24:44 UTC, Olivier FAURE wrote:

 On Friday, 1 February 2019 at 09:10:15 UTC, aliak wrote:

 Shouldn't doubleMyValue(pt.x) be a compiler error if pt.x is a 
 getter? For it not to be a compile error pt.x should also have a 
 setter, in which case the code needs to be lowered to something else:


 The thing is, D doesn't really differentiate between a getter and any 
 other method.

 So with DIP-1016, when given

     doubleMyValue(pt.x);

 The compiler would assume the programmer means
 - Call pt.x()
 - Store the result in a temporary
 - Pass that temporary as a ref parameter to doubleMyValue

 At no point is the compiler aware that the user intends for x to be 
 interpreted as a getter.


 

 ref parameter arguments.


Such would be a good conservative approach for us too. We can relax it 
later if we come with a good idea.



 Feb 03 2019



prev sibling  parent  Olivier FAURE <couteaubleu gmail.com>  writes:


On Thursday, 31 January 2019 at 21:50:32 UTC, Steven 
Schveighoffer wrote:

 How is the problem not in doubleMyValue? It's sole purpose is 
 to update an lvalue. It is the perfect candidate to mark with 
  disable for rvalues.


But right now, updating an rvalue is what ref is supposed to be 
used for.

Besides, the fact remains that accepting DIP 1016 would add a new 
corner case with the potential to create hard-to-detect bugs, 
which feels to me like it should be a dealbreaker. The fact that 
this corner case can be patched using  disable isn't good enough, 
because:
- Existing codebases that use ref won't have the  disable patch, 
which means using them will become (slightly) dangerous because 
of DIP 1016.
- Making libraries that behave predictably should be the default 
path (the "pit of success" philosophy), not require an additional 
construct.

Besides, D's type system should be more than capable of 
consistently telling the user "Be careful, you're modifying a 
temporary when it's probably not what you meant".

---

An alternate proposal that just came to mind: allowing the user 
to pass rvalues to ref arguments with the following syntax:

     y = doubleMyValue(cast(ref)10);

This syntax would avoid creating ambiguous situations where the 
compiler thinks you're passing it a getter's return as a 
temporary when you're trying to pass the attribute that the 
getter maps to. Eg, the following code:

     y = doubleMyValue(pt.x);

would still fail the same way it currently does when Point.x() is 
a getter.



 Feb 01 2019



prev sibling  parent reply Nicholas Wilson <iamthewilsonator hotmail.com>  writes:


On Wednesday, 30 January 2019 at 18:29:37 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 The result of a CastExpression is an rvalue. An implicit cast 
 is a compiler-inserted CastExpression. Therefore all lvalues 
 with a potential implicit cast are rvalues.


 But there's no existing language rule that attempts to perform 
 an implicit cast where an lvalue is supplied to a ref arg...?
 Why is the cast being attempted? 'p' is an lvalue, and whatever 
 that does should remain exactly as is (ie, emits a compile 
 error).

 We could perhaps allow this for `const` args, but that feels 
 like separate follow-up work to me, and substantially lesser 
 value. This DIP doesn't want to change anything about lvalues.


It appears to say it does:

fun(my_short); // implicit type conversions (ie, short->int 
promotion)

You should clarify that ;)



 Jan 30 2019



  parent reply Manu <turkeyman gmail.com>  writes:


On Wed, Jan 30, 2019 at 6:40 PM Nicholas Wilson via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Wednesday, 30 January 2019 at 18:29:37 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com>
 wrote:

 The result of a CastExpression is an rvalue. An implicit cast
 is a compiler-inserted CastExpression. Therefore all lvalues
 with a potential implicit cast are rvalues.


 But there's no existing language rule that attempts to perform
 an implicit cast where an lvalue is supplied to a ref arg...?
 Why is the cast being attempted? 'p' is an lvalue, and whatever
 that does should remain exactly as is (ie, emits a compile
 error).

 We could perhaps allow this for `const` args, but that feels
 like separate follow-up work to me, and substantially lesser
 value. This DIP doesn't want to change anything about lvalues.


 It appears to say it does:

 fun(my_short); // implicit type conversions (ie, short->int
 promotion)

 You should clarify that ;)


Yes, as said above, read `short(10)`. I can understand the confusion
that it may look like a variable when taken out of context; but listed
beneath the heading immediately above which says:
"This inconvenience extends broadly to every manner of **rvalue**
passed to functions"
It didn't occur to me the reader might interpret the clearly stated
list of cases of rvalues passed to functions to include arguments that
are not rvalues.
The name was just chosen to indicate the argument is a short, perhaps
an enum, or any expression that is a short... I could have used
`short(10)`, but apparently I didn't think of it at the time.

Is this the basis for the claims of "a hole you could drive a truck
through"? Again, a request for clarification, and a
couldn't-possibly-be-more-trivial revision may resolve this.



 Jan 30 2019



 next sibling parent reply Steven Schveighoffer <schveiguy gmail.com>  writes:


On 1/30/19 10:05 PM, Manu wrote:

 On Wed, Jan 30, 2019 at 6:40 PM Nicholas Wilson via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 You should clarify that ;)


 
 Yes, as said above, read `short(10)`. I can understand the confusion
 that it may look like a variable when taken out of context; but listed
 beneath the heading immediately above which says:
 "This inconvenience extends broadly to every manner of **rvalue**
 passed to functions"
 It didn't occur to me the reader might interpret the clearly stated
 list of cases of rvalues passed to functions to include arguments that
 are not rvalues.
 The name was just chosen to indicate the argument is a short, perhaps
 an enum, or any expression that is a short... I could have used
 `short(10)`, but apparently I didn't think of it at the time.
 
 Is this the basis for the claims of "a hole you could drive a truck
 through"? Again, a request for clarification, and a
 couldn't-possibly-be-more-trivial revision may resolve this.
 


I think changing it to `short(10)` helps the argument that you didn't 
intend it to mean conversions from lvalues, but I'd recommend still 
spelling out that they are forbidden.

Leaving the reader to infer intent is not as good as clarifying intent 
directly. The whole rvalue vs. lvalue thing is confusing to me, because 
I assumed an lvalue converted to a different type changes it to an 
rvalue. I think of it like an implicit function that returns that new value.

-Steve



 Jan 30 2019



  parent  Manu <turkeyman gmail.com>  writes:


On Wed, Jan 30, 2019 at 7:35 PM Steven Schveighoffer via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 1/30/19 10:05 PM, Manu wrote:

 On Wed, Jan 30, 2019 at 6:40 PM Nicholas Wilson via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 You should clarify that ;)


 Yes, as said above, read `short(10)`. I can understand the confusion
 that it may look like a variable when taken out of context; but listed
 beneath the heading immediately above which says:
 "This inconvenience extends broadly to every manner of **rvalue**
 passed to functions"
 It didn't occur to me the reader might interpret the clearly stated
 list of cases of rvalues passed to functions to include arguments that
 are not rvalues.
 The name was just chosen to indicate the argument is a short, perhaps
 an enum, or any expression that is a short... I could have used
 `short(10)`, but apparently I didn't think of it at the time.

 Is this the basis for the claims of "a hole you could drive a truck
 through"? Again, a request for clarification, and a
 couldn't-possibly-be-more-trivial revision may resolve this.


 I think changing it to `short(10)` helps the argument that you didn't
 intend it to mean conversions from lvalues, but I'd recommend still
 spelling out that they are forbidden.


I mean, the heading of the DIP is "ref T accepts r-values", the whole
abstract talks about nothing but rvalues, the header of the confusing
block couldn't say 'rvalues' more clearly... I didn't consider that it
was possible to confuse this as anything other than an rvalue DIP...
but yes, I can certainly spell it out.


 Leaving the reader to infer intent is not as good as clarifying intent
 directly. The whole rvalue vs. lvalue thing is confusing to me, because
 I assumed an lvalue converted to a different type changes it to an
 rvalue. I think of it like an implicit function that returns that new value.


Obviously all of this is true, but I didn't think of it that way;
didn't realise there was a point of confusion, and nobody during the
community reviews appeared to raise confusion either.
I'll obviously revise it, except that it's rejected and moved to the
rejected folder.

For reference, the key point that justifies its mention in the first
place is a little further down:
"It is important that T be defined as the parameter type, and not auto
(ie, the argument type), because it will allow implicit conversions to
occur naturally, with identical behavior as when the parameter is not
ref."
It was important to consider mis-matching types (implicit
conversions), because there is detail in the rules that allows them to
work properly and make the call uniform with the same function if it
passed by-val.



 Jan 30 2019



prev sibling  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 1/30/19 10:05 PM, Manu wrote:

 On Wed, Jan 30, 2019 at 6:40 PM Nicholas Wilson via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Wednesday, 30 January 2019 at 18:29:37 UTC, Manu wrote:

 On Wed, Jan 30, 2019 at 9:20 AM Neia Neutuladh via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com>
 wrote:

 The result of a CastExpression is an rvalue. An implicit cast
 is a compiler-inserted CastExpression. Therefore all lvalues
 with a potential implicit cast are rvalues.


 But there's no existing language rule that attempts to perform
 an implicit cast where an lvalue is supplied to a ref arg...?
 Why is the cast being attempted? 'p' is an lvalue, and whatever
 that does should remain exactly as is (ie, emits a compile
 error).

 We could perhaps allow this for `const` args, but that feels
 like separate follow-up work to me, and substantially lesser
 value. This DIP doesn't want to change anything about lvalues.


 It appears to say it does:

 fun(my_short); // implicit type conversions (ie, short->int
 promotion)

 You should clarify that ;)


 
 Yes, as said above, read `short(10)`. I can understand the confusion
 that it may look like a variable when taken out of context; but listed
 beneath the heading immediately above which says:
 "This inconvenience extends broadly to every manner of **rvalue**
 passed to functions"
 It didn't occur to me the reader might interpret the clearly stated
 list of cases of rvalues passed to functions to include arguments that
 are not rvalues.
 The name was just chosen to indicate the argument is a short, perhaps
 an enum, or any expression that is a short... I could have used
 `short(10)`, but apparently I didn't think of it at the time.
 
 Is this the basis for the claims of "a hole you could drive a truck
 through"?


Affirmative.

With the restriction that the expression passed into the function must 
be an rvalue to start with, by Walter's and my understanding, the 
proposed semantics would work and be helpful.


 Again, a request for clarification, and a
 couldn't-possibly-be-more-trivial revision may resolve this.


Negative.

It must be clear that the reason of this misunderstanding is squarely 
due to the DIP itself. It has multiple problems of informality and vague 
language that have worked together to cause said misunderstanding. (It 
is great it's just that, a misunderstanding; I have been worried people 
would believe such an awful semantics was considered just fine. That 
explains but does not justify my use of unkind language.)

The DIP must convince the reader, and in a way the reader does not "owe" 
the DIP. For good reason, they call the research theme chosen by a 
doctoral candidate a "charge"; the root of "dissertation" is Latin for 
"debate"; and the final doctoral examination is plainly called a 
"defense". The whole thing is structured like a criminal investigation 
:o). Of course we don't want to be as harsh as academics could get, but 
we don't want to transform DIP acceptance into a farmers market 
bargaining process.

So the code with my_short was open to interpretation. Cool. In a 
thorough submission, however, there would have been many places that 
clear that up:

* Use of a distinct notation (non-code non-text font for metalanguage, 
i.e. general expressions);

* Description of the typechecking process, with examples of code that 
passes and code that fails;

* A clarification that lowering proceeds not against all expressions, 
but only against rvalues;

* Several places in text in which it is explained that rvalues resulted 
from implicit conversions are not eligible;

* etc. etc. etc.

So if we rejected the DIP, we didn't do so on account of one word that 
can be so easily changed; we did so on account of a DIP that as a whole 
failed to clarify what it purports to do (and equally importantly, to 
not do).

The purpose of us all is to move things forward, and in that spirit 
allow me to put forward a short list of matters that a revised proposal 
should do, at a minimum:

* Walter has posted about a few issues with various parts of the 
proposal. Those should be addressed.

* The "Reference" section does good to mention the issues, but the 
litany of forum discussions has no value besides "there have been 
repeated discussion in community forums of the topic", and refer to a 
list in an bibliography. Placing them in the "Reference" section 
suggests the reader that they need to read the forum debates in order to 
understand the DIP, which isn't and shouldn't be the case.

* An "Existing Work" section discussing C++ (and possibly Rust) is a 
must. Studious neglect of what other languages do and what problems they 
have does not serve us well. I think Walter could help with that.

* The "Rationale" section currently focuses only on issues caused by the 
current rule. It should have three parts:

- Open with a brief description of the current rule and why it is that 
way. Here we have the advantage that confusing conversions are disallowed.
- Then continue with "However, the binding rule also causes a variety of 
undue limitations whereby valid and useful code is rejected". Insert 
examples.
- Then continue with "If the rule got relaxed so as to only allow 
certain bindings but not all, we could allow the sensible cases 
illustrated above, yet still disallow the problematic cases."

Boom! The reader is already loving it.

* Subsection "Why are we here?" is not necessary. Its first part is 
folded in the "Rationale", and its second part is actually clearly an 
argument that is part of the rationale. IMHO the "pipeline" argument is 
not necessary, or can be moved in an "Examples of use" section toward 
the end.

* "Description" needs a complete rewrite. Here typing rules, lowering, 
exception safety, should be explained in ultimate detail using 
metalanguage (not just a few simplistic examples). It should be 
mechanically clear how an arbitrary expression containing function calls 
can be lowered into preexisting D code. Again I very much recommend 
lowering expressions to expressions instead of statements to statements, 
but anything clear and thorough will do.

* Also the description should clarify what happens in certain odd cases, 
such as alias this, class inheritance, properties that return lvalues 
vs. rvalues using the same syntax, and probably more. An important 
aspect of the proposal is "rvalues look different than lvalues, so the 
user won't get confused".

* "Temporary destruction" is unnecessary.

* "Function calls as arguments" us unnecessary if "Description" 
describes the feature completely.

* "Interaction with return ref" is probably unnecessary as a subsection 
but it would be nice to have as an explanatory example after the general 
lowering has been introduced.

* "Overload resolution" - see Walter's comments.

* Eliminate imprecise language "It has been noted that..." (by whom? 
when? reference?) If no reference, fine - spend a sentence to create 
that note!

* "Why not XXX?" should be massaged into a "Workarounds" section that 
follows the "Rationale" section, or is a subsection of it.

* "Key use cases" is good, and more examples should be only better.

Hope this helps! The outlook is definitely better after this 
misunderstanding has been cleared.


Andrei



 Jan 30 2019



prev sibling  parent  Olivier FAURE <couteaubleu gmail.com>  writes:


On Monday, 28 January 2019 at 17:23:51 UTC, Andrei Alexandrescu 
wrote:

 * Regarding the argument "why not make this an iterative 
 process where concerns are raised and incrementally addressed?" 
 We modeled the DIP process after similar processes - conference 
 papers, journal papers, proposals in other languages. There is 
 a proposal by one or more responsibles, perfected by a 
 community review, and submitted for review. This encourages 
 building a strong proposal - as strong as can be - prior to 
 submission. Washing that down to a negotiation between the 
 proposers and the reviewers leads to a "worst acceptable 
 proposal" state of affairs in which proposers are incentivized 
 to submit the least-effort proposal, reactively change it as 
 issues are raised by reviewers.


Fair enough.



 Jan 29 2019



prev sibling next sibling parent reply Don <don gmail.com>  writes:


I'm on the reviewers side here.

To be honest I never liked this DIP and maybe I'll sound dumb but 
I think this is a case where this could bring more problem than 
anything.

The way I see this would be more like a syntax sugar to create 
temporary variable for ref parameters and that's it.

But what I fail to see is why can't the programmer solve this 
themselves instead of relying on a new feature that would cause 
more harm?

With overload some could do:

void f(int i){
     f(i);
}

void f(ref int i){
     ++i;
     writeln(i);
}

void main(){
     int i = 0;
     f(10);
     f(i);
}

prints:
11
1

The "f" function will work with ref or literal (rvalues).

But this will be controlled by the programmer the way they want 
it.

Donald.



 Jan 29 2019



  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 30 January 2019 at 00:25:17 UTC, Don wrote:

 But what I fail to see is why can't the programmer solve this 
 themselves instead of relying on a new feature that would cause 
 more harm?



 Donald.


...Did you even read the arguments in the dip? This has been 
discuss quite a lot in the forums, it even gives you links to 
them.



 Jan 29 2019



  parent reply Don <don gmail.com>  writes:


On Wednesday, 30 January 2019 at 03:01:36 UTC, 12345swordy wrote:

 On Wednesday, 30 January 2019 at 00:25:17 UTC, Don wrote:

 But what I fail to see is why can't the programmer solve this 
 themselves instead of relying on a new feature that would 
 cause more harm?



 Donald.


 ...Did you even read the arguments in the dip? This has been 
 discuss quite a lot in the forums, it even gives you links to 
 them.


Well, I read the DIP and the whole forum discussion back in the 
day, and again I think this will create more harm than benefits 
the way it was proposed.

And starting from the beginning of this DIP - Rationale example:

    "void fun(int x);

     fun(10); // <-- this is how users expect to call a typical 
function
     But when ref is present:

     void fun(ref int x);

     fun(10); // <-- compile error; not an lvalue!!
     Necessitating the workaround:

     int temp = 10;
     fun(temp);

     This inconvenience extends broadly to every manner of rvalue 
passed to
     functions, including:"

So the solution in the way I understood is pretty much a syntax 
sugar, creating temporary variable with destruction.

But the concept is weird, because originally your function 
signature has a "ref parameter" and we're just creating a 
workaround expanding it to handle rvalues.

I would prefer to handle it myself with overloading instead of 
being presented with new language feature creating different 
scenarios for something that's not the case right now.

Otherwise D will be pretty much like C++ and in this case why 
bother with it?

Donald.



 Jan 29 2019



  parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 30 January 2019 at 04:34:46 UTC, Don wrote:

 On Wednesday, 30 January 2019 at 03:01:36 UTC, 12345swordy 
 wrote:

 On Wednesday, 30 January 2019 at 00:25:17 UTC, Don wrote:

 But what I fail to see is why can't the programmer solve this 
 themselves instead of relying on a new feature that would 
 cause more harm?



 Donald.


 ...Did you even read the arguments in the dip? This has been 
 discuss quite a lot in the forums, it even gives you links to 
 them.


 Well, I read the DIP and the whole forum discussion back in the 
 day, and again I think this will create more harm than benefits 
 the way it was proposed.
 Donald.


I do not accept gut feeling as a valid objection here. The 
current workarounds is shown to be painful as shown in the dip 
and in the discussions that it currently link. That *the* 
motivation here.
I am familiar with the author here, he is very involved with the 
C++<->D compatibility side of things. He knows the pain from 
first hand experience.

-Alex



 Jan 30 2019



  parent reply Don <donald gmail.com>  writes:


On Wednesday, 30 January 2019 at 13:58:38 UTC, 12345swordy wrote:

 I do not accept gut feeling as a valid objection here. The 
 current workarounds is shown to be painful as shown in the dip 
 and in the discussions that it currently link. That *the* 
 motivation here.


Like I said previously I am on the reviews side and that's it.

By the way I don't like your tone when you say: "I do not accept 
gut feeling as a valid objection here".

I don't think you would like if I say that your opinion is biased 
because you know the author either, so don't go that way, because 
it's not only me against this DIP.


 I am familiar with the author here, he is very involved with 
 the C++<->D compatibility side of things. He knows the pain 
 from first hand experience.


Alright we're talking about a change that have been on hold for 
almost 10 years, if it was simple it would already been done.

In this thread we saw some other concerns been emerged.

Finally I only know the author by his postings in this forum, and 
I don't have anything personally against him.

Donald.



 Jan 30 2019



  parent  12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 30 January 2019 at 16:47:48 UTC, Don wrote:

 On Wednesday, 30 January 2019 at 13:58:38 UTC, 12345swordy 
 wrote:

 I do not accept gut feeling as a valid objection here. The 
 current workarounds is shown to be painful as shown in the dip 
 and in the discussions that it currently link. That *the* 
 motivation here.


 Like I said previously I am on the reviews side and that's it.


In terms of what exactly?
Walter had stated they do not rejected the dip in principle.
You apparently *do* rejected it in principle, from judging your 
posts here.


 By the way I don't like your tone when you say: "I do not 
 accept gut feeling as a valid objection here".


If you stated that you think it a bad/good idea without 
explaining why you think it. That is what I call "gut feeling"


 Alright we're talking about a change that have been on hold for 
 almost 10 years, if it was simple it would already been done.


The current dip system didn't exist 10 years prior. I wouldn't 
say that things are already done due to them being simple, as 
there are quite number of "simple" features that wasn't 
implemented already (Looking at you tuples).

-Alex



 Jan 30 2019



prev sibling  parent reply bitwise <bitwise.pvt gmail.com>  writes:


On Thursday, 24 January 2019 at 07:18:58 UTC, Mike Parker wrote:


 Walter and Andrei have declined to accept DIP 1016, "ref T 
 accepts r-values", on the grounds that it has two fundamental 
 flaws that would open holes in the language.




 fun(10)
 ==>
 {
   T __temp0 = void;
   fun(__temp0 := 10);
 }



 the rewrite is from an expression to a statement, rendering it 
 invalid.



 if the first constructor throws an exception, all remaining 
 values will be destroyed in the void state as they never have 
 the chance to become initialized.



 They say that with the current semantics, this function only 
 operates on long values as it should. With the proposed 
 semantics, the call will accept all shared integral types.


I think this solves all of the above:

fun(10);
==>
(int __temp0){ (return)? fun( __temp0 ); }(10);

-The expression/statement issue is solved by the closure
-The initialization issue created by the ":=" approach is not 
present here
-For this rewrite, 'T' is the type of the rvalue argument, not 
the type of the function parameter. This prevents undesired 
implicit conversions.



 Feb 04 2019



  parent reply Dominikus Dittes Scherkl <dominikus scherkl.de>  writes:


On Monday, 4 February 2019 at 17:09:25 UTC, bitwise wrote:


 I think this solves all of the above:

 fun(10);
 ==>
 (int __temp0){ (return)? fun( __temp0 ); }(10);

 -The expression/statement issue is solved by the closure
 -The initialization issue created by the ":=" approach is not 
 present here
 -For this rewrite, 'T' is the type of the rvalue argument, not 
 the type of the function parameter. This prevents undesired 
 implicit conversions.


I don't understand this.
What does "(return)?" mean? Is this valid D syntax? What do I 
miss?



 Feb 04 2019



  parent reply bitwise <bitwise.pvt gmail.com>  writes:


On Monday, 4 February 2019 at 18:32:56 UTC, Dominikus Dittes 
Scherkl wrote:

 I don't understand this.
 What does "(return)?" mean? Is this valid D syntax? What do I 
 miss?


I meant for that to be interpreted like a Regular expression, 
denoting conditional presence of the return statement. I'm not 
sure what the proper notation would be to express such a thing.



 Feb 04 2019



  parent reply Paul Backus <snarwin gmail.com>  writes:


On Monday, 4 February 2019 at 18:35:37 UTC, bitwise wrote:

 On Monday, 4 February 2019 at 18:32:56 UTC, Dominikus Dittes 
 Scherkl wrote:

 I don't understand this.
 What does "(return)?" mean? Is this valid D syntax? What do I 
 miss?


 I meant for that to be interpreted like a Regular expression, 
 denoting conditional presence of the return statement. I'm not 
 sure what the proper notation would be to express such a thing.


It's actually fine to leave the `return` there 
unconditionally--you're allowed to return an expression of type 
`void` from a function.

Example: https://run.dlang.io/is/tnSGN4



 Feb 04 2019



  parent reply bitwise <bitwise.pvt gmail.com>  writes:


On Monday, 4 February 2019 at 20:08:39 UTC, Paul Backus wrote:

 On Monday, 4 February 2019 at 18:35:37 UTC, bitwise wrote:

 On Monday, 4 February 2019 at 18:32:56 UTC, Dominikus Dittes 
 Scherkl wrote:

 I don't understand this.
 What does "(return)?" mean? Is this valid D syntax? What do I 
 miss?


 I meant for that to be interpreted like a Regular expression, 
 denoting conditional presence of the return statement. I'm not 
 sure what the proper notation would be to express such a thing.


 It's actually fine to leave the `return` there 
 unconditionally--you're allowed to return an expression of type 
 `void` from a function.

 Example: https://run.dlang.io/is/tnSGN4


Even better ;)

No one else seems particularly excited about the rewrite though - 
what am I missing?

I suppose a lambda cost significantly more, but I don't think the 
lambda should make it through the optimizer for this case though.



 Feb 08 2019



  parent reply Nicholas Wilson <iamthewilsonator hotmail.com>  writes:


On Friday, 8 February 2019 at 16:00:58 UTC, bitwise wrote:

 On Monday, 4 February 2019 at 20:08:39 UTC, Paul Backus wrote:

 On Monday, 4 February 2019 at 18:35:37 UTC, bitwise wrote:

 [...]


 It's actually fine to leave the `return` there 
 unconditionally--you're allowed to return an expression of 
 type `void` from a function.

 Example: https://run.dlang.io/is/tnSGN4


 Even better ;)

 No one else seems particularly excited about the rewrite though 
 - what am I missing?

 I suppose a lambda cost significantly more, but I don't think 
 the lambda should make it through the optimizer for this case 
 though.


Immediately called lamdas are always inlined.



 Feb 08 2019



  parent reply Dennis <dkorpel gmail.com>  writes:


On Friday, 8 February 2019 at 23:02:34 UTC, Nicholas Wilson wrote:

 Immediately called lamdas are always inlined.


```
extern(C) void main() {
     int a = (() => 1)();
}
```

dmd -inline -O -release -betterC

asm:
```
main:
		push	RBP
		mov	RBP,RSP
		call	qword ptr pure nothrow  nogc  safe int 
onlineapp.main().__lambda1() GOTPCREL[RIP]
		xor	EAX,EAX
		pop	RBP
		ret
```

https://run.dlang.io/is/lZW9B6

Still a lambda call :/



 Feb 08 2019



 next sibling parent  "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Fri, Feb 08, 2019 at 11:34:47PM +0000, Dennis via Digitalmars-d-announce
wrote:

 On Friday, 8 February 2019 at 23:02:34 UTC, Nicholas Wilson wrote:

 Immediately called lamdas are always inlined.


 
 ```
 extern(C) void main() {
     int a = (() => 1)();
 }
 ```
 
 dmd -inline -O -release -betterC
 
 asm:
 ```
 main:
 		push	RBP
 		mov	RBP,RSP
 		call	qword ptr pure nothrow  nogc  safe int
 onlineapp.main().__lambda1() GOTPCREL[RIP]
 		xor	EAX,EAX
 		pop	RBP
 		ret
 ```
 
 https://run.dlang.io/is/lZW9B6
 
 Still a lambda call :/


Does LDC/GDC inline it?

I no longer trust dmd for codegen quality. :-/


T

-- 
Customer support: the art of getting your clients to pay for your own
incompetence.



 Feb 08 2019



prev sibling  parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Fri, Feb 08, 2019 at 03:42:51PM -0800, H. S. Teoh via Digitalmars-d-announce
wrote:

 On Fri, Feb 08, 2019 at 11:34:47PM +0000, Dennis via Digitalmars-d-announce
wrote:

 On Friday, 8 February 2019 at 23:02:34 UTC, Nicholas Wilson wrote:

 Immediately called lamdas are always inlined.


 
 ```
 extern(C) void main() {
     int a = (() => 1)();
 }
 ```




[...]

 Does LDC/GDC inline it?
 
 I no longer trust dmd for codegen quality. :-/


[...]

Just checked: LDC does inline it.  In fact, LDC compiles the whole thing
out and just has `ret` for main(). :-D  Forcing LDC not to elide the
whole thing by inserting a writeln(a) call reveals that the lambda is
indeed inlined.

Yep, the moral of the story is, if codegen quality is important to you,
use ldc (and presumably gdc too) rather than dmd.


T

-- 
Freedom of speech: the whole world has no right *not* to hear my spouting off!



 Feb 08 2019



  parent reply Dennis <dkorpel gmail.com>  writes:


On Friday, 8 February 2019 at 23:58:49 UTC, H. S. Teoh wrote:

 Yep, the moral of the story is, if codegen quality is important 
 to you, use ldc (and presumably gdc too) rather than dmd.


That's definitely true, but that leaves the question whether 
lowering rvalue references to lambdas is acceptable. There's the 
'dmd for fast builds, gdc/ldc for fast code' motto, but if your 
debug builds of your game make it run at 15 fps it becomes 
unusable. I don't want the gap between dmd and compilers with 
modern back-ends to widen.



 Feb 08 2019



 next sibling parent  "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Sat, Feb 09, 2019 at 12:04:20AM +0000, Dennis via Digitalmars-d-announce
wrote:

 On Friday, 8 February 2019 at 23:58:49 UTC, H. S. Teoh wrote:

 Yep, the moral of the story is, if codegen quality is important to
 you, use ldc (and presumably gdc too) rather than dmd.


 
 That's definitely true, but that leaves the question whether lowering
 rvalue references to lambdas is acceptable. There's the 'dmd for fast
 builds, gdc/ldc for fast code' motto, but if your debug builds of your
 game make it run at 15 fps it becomes unusable. I don't want the gap
 between dmd and compilers with modern back-ends to widen.


TBH, I've been finding that ldc compilation times aren't all that bad
compared to dmd.  It's definitely slightly slower, but it's not anywhere
near the gap between, say, dmd and g++.

Recently I've been quite tempted to replace dmd with ldc as my main D
compiler, esp. now that ldc releases are essentially on par with dmd
releases in terms of release schedule of a particular language version.
The slowdown in compilation times isn't enough to offset the benefits,
as long as you're not compiling with, say, -O3 which *would* make the
ldc optimizer run slower (but with the huge benefit of significantly
better codegen -- I've seen performance improvements of up to ~200% with
ldc -O3 vs. dmd -O -inline).

And template-heavy code is slow across all D compilers anyway, so the
relatively small compilation time difference between dmd and ldc doesn't
really matter that much anymore once you have a sufficiently large
codebase with heavy template use.


T

-- 
What doesn't kill me makes me stranger.



 Feb 08 2019



prev sibling  parent reply bitwise <bitwise.pvt gmail.com>  writes:


On Saturday, 9 February 2019 at 00:04:20 UTC, Dennis wrote:

 On Friday, 8 February 2019 at 23:58:49 UTC, H. S. Teoh wrote:

 Yep, the moral of the story is, if codegen quality is 
 important to you, use ldc (and presumably gdc too) rather than 
 dmd.


 That's definitely true, but that leaves the question whether 
 lowering rvalue references to lambdas is acceptable. There's 
 the 'dmd for fast builds, gdc/ldc for fast code' motto, but if 
 your debug builds of your game make it run at 15 fps it becomes 
 unusable. I don't want the gap between dmd and compilers with 
 modern back-ends to widen.


Since the user doesn't explicitly place the lambda in their code, 
wouldn't it be justifiable for the compiler to take it back out 
again at a later step in compilation, even in debug mode?



 Feb 08 2019



  parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Sat, Feb 09, 2019 at 01:08:55AM +0000, bitwise via Digitalmars-d-announce
wrote:

 On Saturday, 9 February 2019 at 00:04:20 UTC, Dennis wrote:

 On Friday, 8 February 2019 at 23:58:49 UTC, H. S. Teoh wrote:

 Yep, the moral of the story is, if codegen quality is important to
 you, use ldc (and presumably gdc too) rather than dmd.


 
 That's definitely true, but that leaves the question whether
 lowering rvalue references to lambdas is acceptable. There's the
 'dmd for fast builds, gdc/ldc for fast code' motto, but if your
 debug builds of your game make it run at 15 fps it becomes unusable.
 I don't want the gap between dmd and compilers with modern back-ends
 to widen.


 
 Since the user doesn't explicitly place the lambda in their code,
 wouldn't it be justifiable for the compiler to take it back out again
 at a later step in compilation, even in debug mode?


Using lowering to lambdas as a way of defining semantics is not the same
thing as actually using lambdas to implement a feature in the compiler!

While it can be convenient to do the latter as a first stab, I'd expect
that the optimizer could make use of special knowledge available in the
compiler to implement this more efficiently. Since the compiler will
always use a fixed pattern for the lowering, the backend could detect
this pattern and optimize accordingly.  Or the compiler implementation
could lower it directly to something more efficient in the first place.


T

-- 
If you look at a thing nine hundred and ninety-nine times, you are perfectly
safe; if you look at it the thousandth time, you are in frightful danger of
seeing it for the first time. -- G. K. Chesterton



 Feb 08 2019



  parent  Daniel N <no public.email>  writes:


On Saturday, 9 February 2019 at 01:31:05 UTC, H. S. Teoh wrote:

 Using lowering to lambdas as a way of defining semantics is not 
 the same thing as actually using lambdas to implement a feature 
 in the compiler!

 While it can be convenient to do the latter as a first stab, 
 I'd expect that the optimizer could make use of special 
 knowledge available in the compiler to implement this more 
 efficiently. Since the compiler will always use a fixed pattern 
 for the lowering, the backend could detect this pattern and 
 optimize accordingly.  Or the compiler implementation could 
 lower it directly to something more efficient in the first 
 place.


 T


The lambda even correctly handles " disable this(this);", I like 
it!

struct One {  disable this(this); }
void fun(ref One one) { }

One gun() { return One.init; }

void main()
{
   One one; one.fun();
   (One __temp0){ return fun( __temp0 ); }(gun()); // OK
}



 Feb 10 2019