www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Binding rvalues to ref parameters redux

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Work has been underway on redoing DIP 1016. I haven't made a pull 
request yet as it's a bit early. Looking for high-level observations:

https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

Thanks in advance for any feedback.
Mar 26
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
I'd factor out ``` struct Point { long x, y, z; ... } Point fun(); ``` from the rationale. No need to repeat it.
Mar 26
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
A couple more points. In the combinatorial explosion example, give the root case an implementation. double distance(ref const Point p1, ref const Point p2); // used by all other overloads -> // used by all other overloads double distance(ref const Point p1, ref const Point p2) { ... }
 More discussion on rationale and motivating examples can be 
 found in the "Rationale" section of DIP 1016.
Thats fine for a draft, for the real thing this will need to include those examples. In particular the main points not included are: * Its a PITA to get proper composition with higher order function templates. * Disruption of pipelines Under the "Binding Rules" please state the types of the symbols used up front (Where do Tk and Uk come from? what are they? Types? Parameters?)
Mar 26
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
 void function(int a, int b, int c) fun();
 int g();
 int h(int);
 int i();
 int j();
 fun(g(), h(j()), i()); // evaluates g() then j() then h() then 
 i()
                        // after which control is transferred to 
 the callee
should be fun()(g(), h(j()), i()).
Mar 26
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
Destructors of temporaries should be a subsection of its own. It needs code examples, because while
The order of destruction for temporaries inserted for a given 
function call is the inverse order of construction.
is fine and easy to understand, the rest is not. It would be far better to describe this through lambda lowering. The document mentions
 Related, returning a ref parameter by ref enables safe and 
 efficient "pipelining" of function calls.
But gives it no further discussion, c.f. the section of return ref in DIP 1016. The document does not deal with overload resolution at all.
Mar 26
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/27/19 12:33 AM, Nicholas Wilson wrote:
 It would be far better to describe this through lambda lowering.
One important realization working on this was that insertion of constructors and destructors cannot be realized via lambda lowering because lifetimes don't follow some natural scoping. For example, the temporaries created on the left-hand side of && survive through the end of the full expression, whereas those created on the right-hand side do not. The "?:" operator is a hot mess including additional hidden state variables, conditional destruction thus breaking the language rules etc. I'll make sure the document mentions why lowering is not the appropriate mechanism to describe insertion of constructors and destructors. One other thing is that these temporary construction and destruction rules preexisted. In fact probably the nicest thing about the DIP is that no new rules are introduced - temporaries are created and destroyed exactly as if there was no "ref" for the parameter. That way we could migrate some of the DIP into the spec, thus simplifying the DIP. I've already done that with the definitions of "full expression", "lvalue", and "rvalue".
Mar 27
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 27 March 2019 at 11:13:02 UTC, Andrei Alexandrescu 
wrote:
 One important realization working on this was that insertion of 
 constructors and destructors cannot be realized via lambda 
 lowering because lifetimes don't follow some natural scoping. 
 For example, the temporaries created on the left-hand side of 
 && survive through the end of the full expression, whereas 
 those created on the right-hand side do not. The "?:" operator 
 is a hot mess including additional hidden state variables, 
 conditional destruction thus breaking the language rules etc. 
 I'll make sure the document mentions why lowering is not the 
 appropriate mechanism to describe insertion of constructors and 
 destructors.
Well the way it is currently worded is _very_ confusing, it could do with examples of whatever transformation is applied.
 One other thing is that these temporary construction and 
 destruction rules preexisted. In fact probably the nicest thing 
 about the DIP is that no new rules are introduced - temporaries 
 are created and destroyed exactly as if there was no "ref" for 
 the parameter.
It should state that.
 That way we could migrate some of the DIP into the spec, thus 
 simplifying the DIP. I've already done that with the 
 definitions of "full expression", "lvalue", and "rvalue".
Mar 27
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/27/19 9:59 AM, Nicholas Wilson wrote:
 On Wednesday, 27 March 2019 at 11:13:02 UTC, Andrei Alexandrescu wrote:
 One important realization working on this was that insertion of 
 constructors and destructors cannot be realized via lambda lowering 
 because lifetimes don't follow some natural scoping. For example, the 
 temporaries created on the left-hand side of && survive through the 
 end of the full expression, whereas those created on the right-hand 
 side do not. The "?:" operator is a hot mess including additional 
 hidden state variables, conditional destruction thus breaking the 
 language rules etc. I'll make sure the document mentions why lowering 
 is not the appropriate mechanism to describe insertion of constructors 
 and destructors.
Well the way it is currently worded is _very_ confusing, it could do with examples of whatever transformation is applied.
Thanks, yah, a few more examples will definitely help.
 One other thing is that these temporary construction and destruction 
 rules preexisted. In fact probably the nicest thing about the DIP is 
 that no new rules are introduced - temporaries are created and 
 destroyed exactly as if there was no "ref" for the parameter.
It should state that.
The current wording is kinda buried in a paragraph: "In expressions containing rvalues bound to ref parameters, the order of evaluation of expressions and the lifetime of temporaries thus generated remain the same as if the ref parameters would be value parameters of the same type." I'll work on expanding on it and make it more prominent. Thanks!
Mar 27
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 27.03.19 02:38, Andrei Alexandrescu wrote:
 Work has been underway on redoing DIP 1016. I haven't made a pull 
 request yet as it's a bit early. Looking for high-level observations:
 
 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a
 
 Thanks in advance for any feedback.
Assuming that we don't add a new function parameter storage class, I think this DIP gets the semantics exactly right. However, I would explain the following two things: - Currently, the main use case of disallowed rvalue-ref calls is to check whether a (templated) function may change your argument using `ref` by trying to call it with an rvalue inside `__traits(compiles, ...)`. Maybe the DIP could state that even though this use case breaks, after this change, the functionality is in fact still available, but instead of an rvalue, you use an overload set containing a property getter and a property setter. (This is in fact a backwards-compatible way to do it.) - It may not be immediately obvious that the DIP does not break overloading of ref and non-ref. [1] The reason why overloading does not break is that a non-ref function may be called with a property getter that also has a setter, while the ref overload may not, so `ref` would keep being more specialized. [2] However, I think adding a new storage class that documents that `ref` is being used for efficiency rather than for (shallow) modification, and restricting the rvalue rewrite to such parameters may make D code more readable, and it will make introspection more effective, because trying to call a function with an rvalue will query intent to modify. Additionally, it would avoid code breakage. But of course, this means we would have two kinds of `ref` arguments, one that accepts lvalues and one that accepts both lvalues and rvalues, and the obvious next step would be to add `ref` arguments that only accept rvalues. :o) [1] In my own frontend, if implemented without also changing the overloading logic, `ref` would no longer be more specialized than non-ref. [2] In my frontend, the most restricted kind of expression that matches a non-ref argument (used to try to call the other function to determine specialization) is currently an rvalue of the respective type, after this change it would need to be an overload set of property getter and setter.
Mar 27
prev sibling next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
From the response to DIP 1016:
 Here, the DIP Author clearly expresses two reasons why a 
 programmer may choose to declare a function to accept `ref` 
 arguments. The Language Maintainers see this as the core of the 
 proposal and would expect the distinction between the two cases 
 to be maintained throughout. [However, this proposal does not 
 maintain the distinction and instead conflates the two][1] 
 cases. The Language Maintainers insist that any proposal 
 allowing `ref` parameters to accept rvalue references must 
 clearly define how functions which make use of `ref` for side 
 effects will not accept rvalues.
I don't see anything in the new DIP that addresses the above issue. [1] It seems like a waste to have a keyword that's just an alias for two others (in = const scope), so why not put 'in' to better use, like to qualify a ref parameter as accepting rvalues, or to outright replace 'ref' for rvalue-ref accepting parameters? Bit
Mar 28
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 28 March 2019 at 14:36:59 UTC, bitwise wrote:
 On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei 
 Alexandrescu wrote:
 Here, the DIP Author clearly expresses two reasons why a 
 programmer may choose to declare a function to accept `ref` 
 arguments. The Language Maintainers see this as the core of 
 the proposal and would expect the distinction between the two 
 cases to be maintained throughout. [However, this proposal 
 does not maintain the distinction and instead conflates the 
 two][1] cases. The Language Maintainers insist that any 
 proposal allowing `ref` parameters to accept rvalue references 
 must clearly define how functions which make use of `ref` for 
 side effects will not accept rvalues.
I don't see anything in the new DIP that addresses the above issue. [1]
Thats because it was never an issue with the dip, the conflation was entirely deliberate, It is the review if the dip that was wrong. This issue is resolved by the fact that temporaries created are not usable after the expression they appear in, and things that look like lvalues (e.g. property functions) are disallowed as candidates for rvalue -> ref conversion.
 It seems like a waste to have a keyword that's just an alias 
 for two others (in = const scope), so why not put 'in' to 
 better use, like to qualify a ref parameter as accepting 
 rvalues, or to outright replace 'ref' for rvalue-ref accepting 
 parameters?
Code breakage.
Mar 28
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/28/19 12:07 PM, Nicholas Wilson wrote:
 things that look like lvalues (e.g.  property functions) are disallowed 
 as candidates for rvalue -> ref conversion
Can you please point to the DIP text that proposes such disallowance? All I see is: fun(x.prop); // properties given as an example of call that should succeed yet fails. The dip is at: https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md
Mar 28
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 28 March 2019 at 16:28:47 UTC, Andrei Alexandrescu 
wrote:
 On 3/28/19 12:07 PM, Nicholas Wilson wrote:
 things that look like lvalues (e.g.  property functions) are 
 disallowed as candidates for rvalue -> ref conversion
Can you please point to the DIP text that proposes such disallowance? All I see is: fun(x.prop); // properties given as an example of call that should succeed yet fails. The dip is at: https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1016.md
That was covered in the nonassignability requirement was it not? (it is confusing with multiple documents)
Mar 28
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 March 2019 at 16:07:17 UTC, Nicholas Wilson wrote:
 It seems like a waste to have a keyword that's just an alias 
 for two others (in = const scope), so why not put 'in' to 
 better use, like to qualify a ref parameter as accepting 
 rvalues, or to outright replace 'ref' for rvalue-ref accepting 
 parameters?
Code breakage.
In this case, I think it's worth it. Dlang's documentation has warned against using 'in' for YEARS: https://dlang.org/spec/function.html#param-storage Any code using 'in' right now deserves to break. (but actually, that may not be necessary) I think 'in' could actually be used without changing it's meaning at all. Technically, it is exactly what's needed:
 struct S { }

 // void func(const scope ref S s)
 void func(in ref S s) {
    // ....
}
If 's' is an rvalue, then the justification for each qualifier would be: const - Modifying 's' has no effect. Allowing it is misleading and should be an error. scope - Temporary arguments should not be allowed to escape, for memory safety. ref - Large objects like 4x4 matrices should be passed by ref for efficiency. So again, 'in' seems like the correct choice for qualifying 'ref' parameters as taking rvalues. As far as using const, I don't really think it's that bad. If I had to have my rvalues qualified with const, that would be fine. To be honest, I don't think I've written any C/C++ code that casts const away in years, and would consider it unjustifiable. Dealing with legacy code is the only reason I can think of that someone may legitimately need to cast const away, which is fine though, because if D's const is used with legacy code (Extern C or C++ code), you can cast it all you want (to the same extent you would in your C code) because you know that what's underneath can't possibly be immutable, as C/C++ have no such concept. So finally, I would suggest that rvalues only bind to 'in ref'.
Apr 01
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01.04.19 18:43, bitwise wrote:
 ... > As far as using const, I don't really think it's that bad.
 
Yes, it really is that bad.
 If I had to have my rvalues qualified with const, that would be fine. To 
 be honest, I don't think I've written any C/C++ code that casts const 
 away in years, ...
There is no D const in C++.
Apr 01
next sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Monday, 1 April 2019 at 20:17:33 UTC, Timon Gehr wrote:
 On 01.04.19 18:43, bitwise wrote:

 If I had to have my rvalues qualified with const, that would 
 be fine. To be honest, I don't think I've written any C/C++ 
 code that casts const away in years, ...
There is no D const in C++.
I meant that if you had a C function that returned a const object, you could use D's const in the extern(C) D function declaration. If you did that, you could safely cast const away from the returned object knowing that it wasn't some immutable data in D hiding underneath. Aside from this case, I would consider it unnecessary, and in most cases unacceptable, to cast const away.
 ... > As far as using const, I don't really think it's that 
 bad.
 
Yes, it really is that bad.
Can you provide an example? Thanks!
Apr 02
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 2 April 2019 at 17:56:47 UTC, bitwise wrote:
 On Monday, 1 April 2019 at 20:17:33 UTC, Timon Gehr wrote:
 Yes, it really is that bad.
Can you provide an example?
I'm not sure if that sounds like a dumb question, but I don't think I've ever seen a senior member of the D community agree that const should be changed, while responding to arguments made in these forums. Are changes to const actually on the horizon?
Apr 02
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 1 April 2019 at 16:43:15 UTC, bitwise wrote:
 On Thursday, 28 March 2019 at 16:07:17 UTC, Nicholas Wilson 
 wrote:
 It seems like a waste to have a keyword that's just an alias 
 for two others (in = const scope), so why not put 'in' to 
 better use, like to qualify a ref parameter as accepting 
 rvalues, or to outright replace 'ref' for rvalue-ref 
 accepting parameters?
Code breakage.
In this case, I think it's worth it. Dlang's documentation has warned against using 'in' for YEARS: https://dlang.org/spec/function.html#param-storage
Try 10 months: https://github.com/dlang/dlang.org/commit/93411bed24382a08212648c273183d0725cf5dfor and 15 months for https://github.com/dlang/dlang.org/commit/71ad1b38d5b5d0a25e383c1dce27e90ed6698f71
 Any code using 'in' right now deserves to break. (but actually, 
 that may not be necessary)
That change is still within our 2 year deprecation period and I think it was not a good move.
 So finally, I would suggest that rvalues only bind to 'in ref'.
This comes down to an opt in vs opt out, 'in ref' (leaving aside the issues with const) mean that bindings and code must be updated to allow use, whereas rvalues not binding to 'out ref' allows immediate use. I'm for opt out.
Apr 02
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 2 April 2019 at 23:40:42 UTC, Nicholas Wilson wrote:
 On Monday, 1 April 2019 at 16:43:15 UTC, bitwise wrote:
 On Thursday, 28 March 2019 at 16:07:17 UTC, Nicholas Wilson 
 wrote:
 It seems like a waste to have a keyword that's just an alias 
 for two others (in = const scope), so why not put 'in' to 
 better use, like to qualify a ref parameter as accepting 
 rvalues, or to outright replace 'ref' for rvalue-ref 
 accepting parameters?
Code breakage.
In this case, I think it's worth it. Dlang's documentation has warned against using 'in' for YEARS: https://dlang.org/spec/function.html#param-storage
Try 10 months: https://github.com/dlang/dlang.org/commit/93411bed24382a08212648c273183d0725cf5d and 15 months for https://github.com/dlang/dlang.org/commit/71ad1b38d5b5d0a25e383c1dce27e90ed6698f71
Hey, thanks for looking this up. The situation with 'in' was also explained to me in a forum conversation. Maybe I'm confusing that with seeing the information in the docs. Based on the second link, it couldn't possibly have been be more than 15 months ago that I got this information, but to be honest, I think 15 months still more than enough to support my claim that 'in' has been sitting around collecting dust for a long time and should be put to good use. Even before the change 15 months ago, it was an alias for const, which is arguably useless, and certainly unnecessary. I think this is all beside the point though, since my argument has evolved to where I believe 'in' can be used as is, as long as the implementation of 'scope' is completed as advertised.
 Any code using 'in' right now deserves to break. (but 
 actually, that may not be necessary)
That change is still within our 2 year deprecation period and I think it was not a good move.
I don't understand - are you saying it was already deprecated? I don't see it here: https://dlang.org/deprecate.html Even if deprecating it was just a discussion - are there already plans for 'in'?
 So finally, I would suggest that rvalues only bind to 'in ref'.
This comes down to an opt in vs opt out, 'in ref' (leaving aside the issues with const) mean that bindings and code must be updated to allow use, whereas rvalues not binding to 'out ref' allows immediate use. I'm for opt out.
Was 'out ref' a typo? In any case, I can think of three ways for a function to accept an rvalue: 1) pass by value 2) use an auto ref template 3) use 'ref' parameters and pass in a temporary local variable created from the rvalue #1 would require updates either way #2 already accepts rvalues, so no changes needed #3 this would still require updates to remove the temporaries and pass the values directly. It would actually be worse because you would have to patch all the invocation sites instead of just the classes/structs that accept the rvalues Did I miss any? I suppose that any new code that calls functions with 'ref' parameters could start passing rvalues, but to take advantage of this, the function would have had to be written like #3. Trying to implement this so things "just work" afterward doesn't seem like a worthwhile constraint to me. I have a 4x4 matrix class, and all things considered, I chose #1.
Apr 03
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 3 April 2019 at 16:57:10 UTC, bitwise wrote:
 On Tuesday, 2 April 2019 at 23:40:42 UTC, Nicholas Wilson wrote:
 On Monday, 1 April 2019 at 16:43:15 UTC, bitwise wrote:
 On Thursday, 28 March 2019 at 16:07:17 UTC, Nicholas Wilson 
 wrote:
 Any code using 'in' right now deserves to break. (but 
 actually, that may not be necessary)
That change is still within our 2 year deprecation period and I think it was not a good move.
I don't understand - are you saying it was already deprecated?
No, I'm saying I don't think it was a good move to change it for const scope to const. I certainly don't remember any community discussion about it at the time.
 Even if deprecating it was just a discussion - are there 
 already plans for 'in'?
In theory it is becoming synonymous with 'const'.
 So finally, I would suggest that rvalues only bind to 'in 
 ref'.
This comes down to an opt in vs opt out, 'in ref' (leaving aside the issues with const) mean that bindings and code must be updated to allow use, whereas rvalues not binding to 'out ref' allows immediate use. I'm for opt out.
Was 'out ref' a typo?
No, sorry I forgot to define it. Out ref would mean 'one point of this function is to mutate that argument as a side effect so it can't be an rvalue.'
Apr 03
parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 4 April 2019 at 00:16:11 UTC, Nicholas Wilson wrote:
 On Wednesday, 3 April 2019 at 16:57:10 UTC, bitwise wrote:
 On Tuesday, 2 April 2019 at 23:40:42 UTC, Nicholas Wilson 
 wrote:
 On Monday, 1 April 2019 at 16:43:15 UTC, bitwise wrote:
 On Thursday, 28 March 2019 at 16:07:17 UTC, Nicholas Wilson 
 wrote:
 Any code using 'in' right now deserves to break. (but 
 actually, that may not be necessary)
That change is still within our 2 year deprecation period and I think it was not a good move.
I don't understand - are you saying it was already deprecated?
No, I'm saying I don't think it was a good move to change it for const scope to const. I certainly don't remember any community discussion about it at the time.
+1. I *want* `in` to be `scope const`. Any code that breaks as a result of DIP1000 that uses `in` deserves to break anyway.
Apr 04
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 4 April 2019 at 13:43:32 UTC, Atila Neves wrote:
 +1. I *want* `in` to be `scope const`. Any code that breaks as 
 a result of DIP1000 that uses `in` deserves to break anyway.
Do you have any preference on whether or not 'in' is required for an rvalue to bind to ref?
Apr 04
parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 4 April 2019 at 19:29:54 UTC, bitwise wrote:
 On Thursday, 4 April 2019 at 13:43:32 UTC, Atila Neves wrote:
 +1. I *want* `in` to be `scope const`. Any code that breaks as 
 a result of DIP1000 that uses `in` deserves to break anyway.
Do you have any preference on whether or not 'in' is required for an rvalue to bind to ref?
No. Especially not before seeing the new DIP for that.
Apr 05
parent bitwise <bitwise.pvt gmail.com> writes:
On Friday, 5 April 2019 at 12:19:34 UTC, Atila Neves wrote:
 On Thursday, 4 April 2019 at 19:29:54 UTC, bitwise wrote:
 On Thursday, 4 April 2019 at 13:43:32 UTC, Atila Neves wrote:
 +1. I *want* `in` to be `scope const`. Any code that breaks 
 as a result of DIP1000 that uses `in` deserves to break 
 anyway.
Do you have any preference on whether or not 'in' is required for an rvalue to bind to ref?
No. Especially not before seeing the new DIP for that.
Could you please explain? At this point, I was suggesting that 'in' be used without changes, so no DIP should be required for that. As far as the new rvalue DIP, I would only add `in` as an additional requirement for rvalues binding to `ref`. The only issue I can see with this is `in` indirectly adding `scope` to a `ref` parameter, which after DIP25, doesn't really make sense. DMD v2.085.0 does allow adding `scope` to a `ref` param with -dip25, which seems odd. Do the duplicate storage classes collapse by design (like C++'s reference collapsing), or will 'scope ref' eventually fail to compile or change meaning?
Apr 06
prev sibling parent Nick Treleaven <nick geany.org> writes:
On Tuesday, 2 April 2019 at 23:40:42 UTC, Nicholas Wilson wrote:
 On Monday, 1 April 2019 at 16:43:15 UTC, bitwise wrote:
 So finally, I would suggest that rvalues only bind to 'in ref'.
This comes down to an opt in vs opt out, 'in ref' (leaving aside the issues with const) mean that bindings and code must be updated to allow use, whereas rvalues not binding to 'out ref' allows immediate use. I'm for opt out.
BTW `out` can't be used like `ref`, there's no way to pass data in to the function. From the docs: out parameter is initialized upon function entry with the default value for its type
Apr 03
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/28/19 10:36 AM, bitwise wrote:
 On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu wrote:
 Work has been underway on redoing DIP 1016. I haven't made a pull 
 request yet as it's a bit early. Looking for high-level observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
From the response to DIP 1016:
 Here, the DIP Author clearly expresses two reasons why a programmer 
 may choose to declare a function to accept `ref` arguments. The 
 Language Maintainers see this as the core of the proposal and would 
 expect the distinction between the two cases to be maintained 
 throughout. [However, this proposal does not maintain the distinction 
 and instead conflates the two][1] cases. The Language Maintainers 
 insist that any proposal allowing `ref` parameters to accept rvalue 
 references must clearly define how functions which make use of `ref` 
 for side effects will not accept rvalues.
I don't see anything in the new DIP that addresses the above issue. [1]
The nonassignability requirement addresses it.
Mar 28
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 March 2019 at 16:14:04 UTC, Andrei Alexandrescu 
wrote:
 On 3/28/19 10:36 AM, bitwise wrote:
 On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei 
 Alexandrescu wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
From the response to DIP 1016:
 Here, the DIP Author clearly expresses two reasons why a 
 programmer may choose to declare a function to accept `ref` 
 arguments. The Language Maintainers see this as the core of 
 the proposal and would expect the distinction between the two 
 cases to be maintained throughout. [However, this proposal 
 does not maintain the distinction and instead conflates the 
 two][1] cases. The Language Maintainers insist that any 
 proposal allowing `ref` parameters to accept rvalue 
 references must clearly define how functions which make use 
 of `ref` for side effects will not accept rvalues.
I don't see anything in the new DIP that addresses the above issue. [1]
The nonassignability requirement addresses it.
I agree that the nonassignability requirement fills a major pothole, but it doesn't address the issue of expressing your intent as the writer of a function with ref parameters. The caller may have trouble determining if a function will modify the argument passed to it. The issue could be mitigated by function/parameter naming convention, comments, documentation, or providing source code to the caller, but all of these solutions have the potential to be spotty or absent, so I believe something should be done to make the writer's intent clear to the caller.
Mar 28
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/28/19 3:59 PM, bitwise wrote:
 I agree that the nonassignability requirement fills a major pothole, but 
 it doesn't address the issue of expressing your intent as the writer of 
 a function with ref parameters. The caller may have trouble determining 
 if a function will modify the argument passed to it. The issue could be 
 mitigated by function/parameter naming convention, comments, 
 documentation, or providing source code to the caller, but all of these 
 solutions have the potential to be spotty or absent, so I believe 
 something should be done to make the writer's intent clear to the caller.
A salient point. Also made in the DIP as "In such cases, w.price is not assignable and calls such as applyDiscount(w.price) or w.price.applyDiscount will succeed but not perform any meaningful update. A maintainer expecting such calls to fail may be surprised. We consider this is an inevitable price to pay for the gained flexibility." I reckon that there's implied that some sort of rvalue attribute would be a better solution, a la void fun( rvalue ref T). I'm afraid this would be widely protested, even more widely than doing nothing or going with the nonassignable requirement. You are of course encouraged to say what you think is right and raise the matter to other folks as well. The nonassignable requirement is a major improvement, but won't catch all potential misuses. I don't know how to do better.
Mar 28
next sibling parent reply Rubn <where is.this> writes:
On Thursday, 28 March 2019 at 20:11:26 UTC, Andrei Alexandrescu 
wrote:
 On 3/28/19 3:59 PM, bitwise wrote:
 I agree that the nonassignability requirement fills a major 
 pothole, but it doesn't address the issue of expressing your 
 intent as the writer of a function with ref parameters. The 
 caller may have trouble determining if a function will modify 
 the argument passed to it. The issue could be mitigated by 
 function/parameter naming convention, comments, documentation, 
 or providing source code to the caller, but all of these 
 solutions have the potential to be spotty or absent, so I 
 believe something should be done to make the writer's intent 
 clear to the caller.
A salient point. Also made in the DIP as "In such cases, w.price is not assignable and calls such as applyDiscount(w.price) or w.price.applyDiscount will succeed but not perform any meaningful update. A maintainer expecting such calls to fail may be surprised. We consider this is an inevitable price to pay for the gained flexibility." I reckon that there's implied that some sort of rvalue attribute would be a better solution, a la void fun( rvalue ref T). I'm afraid this would be widely protested, even more widely than doing nothing or going with the nonassignable requirement. You are of course encouraged to say what you think is right and raise the matter to other folks as well. The nonassignable requirement is a major improvement, but won't catch all potential misuses. I don't know how to do better.
I liked the idea someone else gave in regards to "out". I think it would be too difficult to repurpose out as it is now, but perhaps even just "out ref" would do. void foo(out ref int value); int v; foo(out v); It would be have the same way as C#, but again how many different type of attributes are there going to be. I honestly think there are too many, and the current way in/out are implement, they should just be deprecated. They don't add that much value.
Mar 28
parent bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 28 March 2019 at 20:17:25 UTC, Rubn wrote:
 I liked the idea someone else gave in regards to "out".
I recommended in another post that 'in' be used. Is that what you mean? 'out' would be the wrong keyword for this.
Apr 01
prev sibling next sibling parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Thursday, 28 March 2019 at 20:11:26 UTC, Andrei Alexandrescu 
wrote:
 The nonassignable requirement is a major improvement, but won't 
 catch all potential misuses. I don't know how to do better.
You could be more strict and consider that any call of the form applyDiscount(x.price) is invalid if `price` is a method, whether or not that method is a property. The correct way to pass price's return would then be: applyDiscount(x.price()) This syntax is a little surprising to readers because D users don't expect empty parentheses for function calls, but I'd count that as a plus: this situation is a little ambiguous, so code that makes the user wonder about what it does instead of taking it for granted is good. Additionally, you can probably skip the nonassignable requirement for const references.
Mar 28
parent Alexibu <alexibu sunopti.com> writes:
On Thursday, 28 March 2019 at 23:40:45 UTC, Olivier FAURE wrote:
 On Thursday, 28 March 2019 at 20:11:26 UTC, Andrei Alexandrescu 
 wrote:
 The nonassignable requirement is a major improvement, but 
 won't catch all potential misuses. I don't know how to do 
 better.
You could be more strict and consider that any call of the form applyDiscount(x.price)
The property disaster : Struct X { Currency price; } Evolves Into Struct X { Currency price() const {...} Void price(currency x){...} } Applydiscount(x.price); Compiles to nop. Is this revalue ref work an opportunity to have the compiler to try and call the setter with the result ? Apologies phone post.
Mar 28
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
`On Thu, Mar 28, 2019 at 1:15 PM Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 3/28/19 3:59 PM, bitwise wrote:
 I agree that the nonassignability requirement fills a major pothole, but
 it doesn't address the issue of expressing your intent as the writer of
 a function with ref parameters. The caller may have trouble determining
 if a function will modify the argument passed to it. The issue could be
 mitigated by function/parameter naming convention, comments,
 documentation, or providing source code to the caller, but all of these
 solutions have the potential to be spotty or absent, so I believe
 something should be done to make the writer's intent clear to the caller.
A salient point. Also made in the DIP as "In such cases, w.price is not assignable and calls such as applyDiscount(w.price) or w.price.applyDiscount will succeed but not perform any meaningful update. A maintainer expecting such calls to fail may be surprised. We consider this is an inevitable price to pay for the gained flexibility." I reckon that there's implied that some sort of rvalue attribute would be a better solution, a la void fun( rvalue ref T). I'm afraid this would be widely protested,
See, I would read `void fun( rvalue ref T)` as a *completely* different thing. If that were in the language, I would expect that to be an rvalue-reference, and as such may absolutely only accept rvalues, and not temporaries. I also think there's room on the language for both things, but I always thought rvalue-references were a contentions non-starter, because D has implicit move semantics, and it never seemed to me like anyone wanted to explore explicit move semantics. This talk about move constructors seems to have opened that conversation.
Mar 31
prev sibling parent Manu <turkeyman gmail.com> writes:
On Sun, Mar 31, 2019 at 2:28 PM Manu <turkeyman gmail.com> wrote:
 `On Thu, Mar 28, 2019 at 1:15 PM Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 3/28/19 3:59 PM, bitwise wrote:
 I agree that the nonassignability requirement fills a major pothole, but
 it doesn't address the issue of expressing your intent as the writer of
 a function with ref parameters. The caller may have trouble determining
 if a function will modify the argument passed to it. The issue could be
 mitigated by function/parameter naming convention, comments,
 documentation, or providing source code to the caller, but all of these
 solutions have the potential to be spotty or absent, so I believe
 something should be done to make the writer's intent clear to the caller.
A salient point. Also made in the DIP as "In such cases, w.price is not assignable and calls such as applyDiscount(w.price) or w.price.applyDiscount will succeed but not perform any meaningful update. A maintainer expecting such calls to fail may be surprised. We consider this is an inevitable price to pay for the gained flexibility." I reckon that there's implied that some sort of rvalue attribute would be a better solution, a la void fun( rvalue ref T). I'm afraid this would be widely protested,
See, I would read `void fun( rvalue ref T)` as a *completely* different thing. If that were in the language, I would expect that to be an rvalue-reference, and as such may absolutely only accept rvalues, and not temporaries.
~~, and not temporaries.~~ - sorry, tail from a re-written sentence. That should read "and not lvalue references".
Mar 31
prev sibling next sibling parent reply kinke <noone nowhere.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 Work has been underway on redoing DIP 1016.
Thanks for the reboot! I just quickly skimmed through, and it seems to be what I hoped for, a more formal DIP1016 with nice binding clarifications. Things I noted: * You use types T_k for the arguments in the binding rules, but introduce them as E_k in the 2nd sentence. * Wrt. binding rule 3, I had to think about the difference of assignability vs. remaining lvalues after the previous rules. If I understand correctly, you're proposing to let a non-assignable `const uint` lvalue bind to a `ref long` param, as opposed to a mutable `uint` lvalue argument. I'd find that a bit counter-intuitive (or more complex than necessary) and would rather disallow all remaining lvalues (+ the remaining assignables), forcing the user to add a clarifying cast regardless of the lvalue mutability.
Mar 28
parent kinke <noone nowhere.com> writes:
On Thursday, 28 March 2019 at 23:46:34 UTC, kinke wrote:
 I'd find that a bit counter-intuitive (or more complex than 
 necessary)
Just to elaborate - that would allow to quickly summarize the rules as in: - If the argument is an lvalue, it binds if the types are an exact or qualified match. - If it's an rvalue, it binds if it's implicitly convertible to the param type *and* the argument expression is not assignable (writable property, opIndex expression with opIndexAssign writability, ...).
Mar 28
prev sibling next sibling parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
- During discussions around DIP-1016, it was pointed out that the following code currently compiles, even though it really shouldn't: struct Point { private int _x, _y; ref int x() { return _x; } ref int y() { return _y; } } struct Rect { private Point _origin, _size; Point origin() { return _origin; } Point size() { return _size; } void origin(Point p) { _origin = p; } void size(Point p) { _size = p; } } Rect r; r.origin = Point(1, 2); r.size = Point(5, 5); doubleMyValue(r.size.x); assert(r.lengths.x == 10); // fail How would your proposal affect the above code? I would assume the code would fail to compile (at least I hope it does), in which case you need to include it in Breaking Changes. Either way, the proposal should probably include a description of how the `this` reference of struct methods currently interacts with rvalues, and how it will be affected by the proposal. - Other corner cases that were raised during discussion of DIP-1016: alias this, return ref, auto ref, and refs in foreach. The proposal needs to address those. - return ref in particular is interesting. How do you handle the following code? int getGraphSize(ref Node); Node makeNode(int x); Node makeNode(return ref Node child1, return ref Node child2); // Probably should compile getGraphSize(makeNode(makeNode(1), makeNode(2))); // Probably should not auto persistentNode = makeNode(makeNode(1), makeNode(2)); // Invalid write persitentNode.child1.x = 123;
Mar 29
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/29/19 7:32 AM, Olivier FAURE wrote:
 On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu wrote:
 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
- During discussions around DIP-1016, it was pointed out that the following code currently compiles, even though it really shouldn't:     struct Point     {         private int _x, _y;         ref int x() { return _x; }         ref int y() { return _y; }     }     struct Rect     {         private Point _origin, _size;         Point origin() { return _origin; }         Point size() { return _size; }         void origin(Point p) { _origin = p; }         void size(Point p) { _size = p; }     }     Rect r;     r.origin = Point(1, 2);     r.size = Point(5, 5);     doubleMyValue(r.size.x);     assert(r.lengths.x == 10); // fail How would your proposal affect the above code?
That code would still compile because the draft DIP does not require a DotExpression a.b.c to have lvalues at all levels (i.e. all of a, b, and c); as long as c is a ref, a and b don't matter. This is hardly a problem with the DIP though because this compiles and runs with the same uselessness: r.size.x *= 2; I could change the DIP to require lvalues throughout. Also, we should probably change the language to disallow other constructs as well. Essentially I think a.b.c should be an rvalue (even if it's ostensibly an lvalue) if any of a, b, or c is an rvalue.
 Either way, the proposal should probably include a description of how 
 the `this` reference of struct methods currently interacts with rvalues, 
 and how it will be affected by the proposal.
I have moved that part straight into the specification of lvalues and rvalues. "this" in struct and union types is an lvalue. In classes it's an rvalue.
 - Other corner cases that were raised during discussion of DIP-1016: 
 alias this, return ref, auto ref, and refs in foreach. The proposal 
 needs to address those.
 
 - return ref in particular is interesting. How do you handle the 
 following code?
 
      int getGraphSize(ref Node);
      Node makeNode(int x);
      Node makeNode(return ref Node child1, return ref Node child2);
 
      // Probably should compile
      getGraphSize(makeNode(makeNode(1), makeNode(2)));
 
      // Probably should not
      auto persistentNode = makeNode(makeNode(1), makeNode(2));
      // Invalid write
      persitentNode.child1.x = 123;
Did you mean makeNode to return by reference? Good list, thanks! Andrei
Mar 29
next sibling parent Olivier FAURE <couteaubleu gmail.com> writes:
On Friday, 29 March 2019 at 12:16:40 UTC, Andrei Alexandrescu 
wrote:
 I could change the DIP to require lvalues throughout. Also, we 
 should probably change the language to disallow other 
 constructs as well. Essentially I think a.b.c should be an 
 rvalue (even if it's ostensibly an lvalue) if any of a, b, or c 
 is an rvalue.
Yes please.
 I have moved that part straight into the specification of 
 lvalues and rvalues. "this" in struct and union types is an 
 lvalue. In classes it's an rvalue.
That's not what I meant. What I meant is getFoo(...).applyDiscount(...) should behave the same way whether applyDiscount is a method taking a Foo as its 'this' parameter, or a global function taking a Foo as its first parameter.
      int getGraphSize(ref Node);
      Node makeNode(int x);
      Node makeNode(return ref Node child1, return ref Node 
 child2);
 
      // Probably should compile
      getGraphSize(makeNode(makeNode(1), makeNode(2)));
 
      // Probably should not
      auto persistentNode = makeNode(makeNode(1), makeNode(2));
      // Invalid write
      persitentNode.child1.x = 123;
Did you mean makeNode to return by reference?
No, I meant something like that: struct Node { Node* child1; Node* child2; int x; } Node makeNode(return ref Node child1, return ref Node child2) safe { Node newNode = { &child1, &child2 }; return newNode; } This currently compiles with -dip1000. Generally speaking, I think any function called with a rvalue as a 'return ref' argument should be considered as returning a rvalue. The "a.b.c is a rvalue if a, b, or c is a rvalue" rule could be considered a special case of that principle.
Mar 29
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/29/19 8:16 AM, Andrei Alexandrescu wrote:
 On 3/29/19 7:32 AM, Olivier FAURE wrote:
 On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu wrote:
 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
- During discussions around DIP-1016, it was pointed out that the following code currently compiles, even though it really shouldn't:      struct Point      {          private int _x, _y;          ref int x() { return _x; }          ref int y() { return _y; }      }      struct Rect      {          private Point _origin, _size;          Point origin() { return _origin; }          Point size() { return _size; }          void origin(Point p) { _origin = p; }          void size(Point p) { _size = p; }      }      Rect r;      r.origin = Point(1, 2);      r.size = Point(5, 5);      doubleMyValue(r.size.x);      assert(r.lengths.x == 10); // fail How would your proposal affect the above code?
That code would still compile because the draft DIP does not require a DotExpression a.b.c to have lvalues at all levels (i.e. all of a, b, and c); as long as c is a ref, a and b don't matter. This is hardly a problem with the DIP though because this compiles and runs with the same uselessness: r.size.x *= 2; I could change the DIP to require lvalues throughout. Also, we should probably change the language to disallow other constructs as well. Essentially I think a.b.c should be an rvalue (even if it's ostensibly an lvalue) if any of a, b, or c is an rvalue.
No, please don't. Rvalues can return lvalues just fine: struct S { int* x; ref int getX() { return *x; } } Note that the point being made was not to point out a problem, but to point out that rvalue references have existed for years (ever since struct member functions were added), and have not caused catastrophic failure. -Steve
Mar 29
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu 
wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
It seems very significant to me that Dlang leadership is officially considering this idea. It would be great to have this kind language evolution decision documented somewhere. I suppose what I'm suggesting is some sort of language evolution F.A.Q., roadmap, or maybe a document similar to the current vision document, but one that outlines the entire future of D as it's seen by leadership, instead half a year at a time. It would be nice to have a companion document to this one [1] that outlined the state of discussions on things like whether __traits will ever work on private symbols, the status of scope, ref counted classes, etc.. [1] https://dlang.org/articles/faq.html I think I could probably find those on the forums, but the amount of time it would take is kinda scary. Thanks!
Apr 11
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/11/19 1:00 PM, bitwise wrote:
 On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei Alexandrescu wrote:
 Work has been underway on redoing DIP 1016. I haven't made a pull 
 request yet as it's a bit early. Looking for high-level observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
It seems very significant to me that Dlang leadership is officially considering this idea.
I just moved a bunch of spec wording from the DIP straight into the spec. This should make that DIP and any related ones a lot easier: https://github.com/dlang/dlang.org/pull/2625
Apr 11
parent bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 11 April 2019 at 21:31:03 UTC, Andrei Alexandrescu 
wrote:
 On 4/11/19 1:00 PM, bitwise wrote:
 On Wednesday, 27 March 2019 at 01:38:40 UTC, Andrei 
 Alexandrescu wrote:
 Work has been underway on redoing DIP 1016. I haven't made a 
 pull request yet as it's a bit early. Looking for high-level 
 observations:

 https://gist.github.com/andralex/e5405a5d773f07f73196c05f8339435a

 Thanks in advance for any feedback.
It seems very significant to me that Dlang leadership is officially considering this idea.
I just moved a bunch of spec wording from the DIP straight into the spec. This should make that DIP and any related ones a lot easier: https://github.com/dlang/dlang.org/pull/2625
Perfect, thanks! Bit
Apr 12