www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - rvalue references

reply Walter Bright <newshound2 digitalmars.com> writes:
Previous discussions:

http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com

http://d.puremagic.com/issues/show_bug.cgi?id=9238
Apr 22 2013
next sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:

 Previous discussions:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com

 http://d.puremagic.com/issues/show_bug.cgi?id=9238


I still like the Idea of DIP 36 (http://wiki.dlang.org/DIP36) 
with scope ref / in ref. On this link we have also collected 
several other suggested syntaxes.
But maybe we should discuss first, if auto ref could work for 
non-templates.
This would mean that
1. auto ref generate code bloat for non-templates (doubled the 
function 2^n times)
or
2. auto ref works like scope ref would do (create a temporary 
lvalue and pass this to the function). But that would mean that 
auto ref work differently for templates and non-templates (maybe 
a problem?). If we choose this we should also discuss about the 
lifetime of the temporary:
----
foo(auto ref A a) { }

foo(A());
// should then converted to
{
     A __temp = A();
     foo(__temp); // after this, __temp is destroyed
}
----
or
3. the compiler choose if an argument should be passed by ref, by 
value or should be moved.
For example:
----
A a = A(42);
foo(a); // by ref
foo(A(23)); // move
----
Apr 23 2013
next sibling parent reply "Diggory" <diggsey googlemail.com> writes:
I'd still like someone to explain how exactly "scope ref" would 
differ from "ref" if DIP25/DIP35 were implemented.

If the only difference is that "scope ref" can accept rvalues 
then why would you ever use normal "ref"? There are no extra 
restrictions needed on "scope ref" over and above normal "ref" 
under the assumption of DIP25/DIP35.
Apr 23 2013
next sibling parent reply "Dicebot" <m.strashun gmail.com> writes:
On Tuesday, 23 April 2013 at 07:18:41 UTC, Diggory wrote:

 I'd still like someone to explain how exactly "scope ref" would 
 differ from "ref" if DIP25/DIP35 were implemented.

 If the only difference is that "scope ref" can accept rvalues 
 then why would you ever use normal "ref"? There are no extra 
 restrictions needed on "scope ref" over and above normal "ref" 
 under the assumption of DIP25/DIP35.


No it is not the only difference. "scope ref" (as proposed in 
DIP35) is more restrictive in usage - can't take address of it, 
can't return it, can't implicitly cast it to normal ref. It is 
"scope" primarily and "rvalue ref solution" only secondarily.
Apr 23 2013
next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Tuesday, 23 April 2013 at 07:45:41 UTC, Dicebot wrote:

 ...


Ah, beg my pardon, I though it is yet another misunderstanding 
about DIP36.
Apr 23 2013
prev sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
 No it is not the only difference. "scope ref" (as proposed in 
 DIP35) is more restrictive in usage - can't take address of it, 
 can't return it, can't implicitly cast it to normal ref. It is 
 "scope" primarily and "rvalue ref solution" only secondarily.


My initial reason was to solve the rvalue ref issue. So it is 
completly inverse for me. :-)
Apr 23 2013
parent reply "Dicebot" <m.strashun gmail.com> writes:
On Tuesday, 23 April 2013 at 10:07:57 UTC, Namespace wrote:

 No it is not the only difference. "scope ref" (as proposed in 
 DIP35) is more restrictive in usage - can't take address of 
 it, can't return it, can't implicitly cast it to normal ref. 
 It is "scope" primarily and "rvalue ref solution" only 
 secondarily.


 My initial reason was to solve the rvalue ref issue. So it is 
 completly inverse for me. :-)


And I initially ignored all the threads you have started until 
noticed proposal that looked fundamental enough in its nature. :) 
Features created to support one specific use case usually are not 
worth it. DIP36 as far as I see it is much more about scope than 
about rvalues.
Apr 23 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/23/13 6:39 AM, Dicebot wrote:

 On Tuesday, 23 April 2013 at 10:07:57 UTC, Namespace wrote:

 No it is not the only difference. "scope ref" (as proposed in DIP35)
 is more restrictive in usage - can't take address of it, can't return
 it, can't implicitly cast it to normal ref. It is "scope" primarily
 and "rvalue ref solution" only secondarily.


 My initial reason was to solve the rvalue ref issue. So it is
 completly inverse for me. :-)


 And I initially ignored all the threads you have started until noticed
 proposal that looked fundamental enough in its nature. :) Features
 created to support one specific use case usually are not worth it. DIP36
 as far as I see it is much more about scope than about rvalues.


If it were about scope it would be very careful with lifetime of 
temporaries.

Andrei
Apr 23 2013
parent reply "Dicebot" <m.strashun gmail.com> writes:
On Tuesday, 23 April 2013 at 14:28:54 UTC, Andrei Alexandrescu 
wrote:

 If it were about scope it would be very careful with lifetime 
 of temporaries.

 Andrei


Please explain. In its current form DIP36 only cares that passed 
temporary exists while called function is executed. I can't 
imagine any sane lifetime rules that would result in violating 
this.
Apr 23 2013
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 23 April 2013 at 14:38:12 UTC, Dicebot wrote:

 On Tuesday, 23 April 2013 at 14:28:54 UTC, Andrei Alexandrescu 
 wrote:

 If it were about scope it would be very careful with lifetime 
 of temporaries.

 Andrei


 Please explain. In its current form DIP36 only cares that 
 passed temporary exists while called function is executed. I 
 can't imagine any sane lifetime rules that would result in 
 violating this.


This isn't enough for a DIP. You have to explicitly define the 
lifetime, as it is required to know what is safe and what isn't, 
to get deterministic destruction, or whatever.
Apr 23 2013
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 24 April 2013 01:01, deadalnix <deadalnix gmail.com> wrote:


 On Tuesday, 23 April 2013 at 14:38:12 UTC, Dicebot wrote:


 On Tuesday, 23 April 2013 at 14:28:54 UTC, Andrei Alexandrescu wrote:


 If it were about scope it would be very careful with lifetime of
 temporaries.

 Andrei


 Please explain. In its current form DIP36 only cares that passed
 temporary exists while called function is executed. I can't imagine any
 sane lifetime rules that would result in violating this.


 This isn't enough for a DIP. You have to explicitly define the lifetime,
 as it is required to know what is safe and what isn't, to get deterministic
 destruction, or whatever.


"The r-value being passed is assigned to a stack allocated temporary, which
has a lifetime that is identical to any other local variable, ie, the
lifetime of the function in which it appears."
There, I defined it.
Apr 23 2013
next sibling parent "Diggory" <diggsey googlemail.com> writes:
 No it is not the only difference. "scope ref" (as proposed in 
 DIP35) is more restrictive in usage - can't take address of it, 
 can't return it, can't implicitly cast it to normal ref. It is 
 "scope" primarily and "rvalue ref solution" only secondarily.


With DIP25 these restrictions apply to "ref" as well (with the 
exception that you CAN return a "ref", but it turns out it would 
also be safe to return a "scope ref" in those circumstances).


 DIP25 imposes a number of code-breaking restrictions even in 
  system code, if DIP36 was in place, one could consider 
 imposing the DIP25 restrictions only in SafeD.


Yes, the "addressof" part of DIP25 would cause problems, but the 
part to do with safety of "ref" will not. IMO we shouldn't start 
trying to add rvalue references until we've solved the safety of 
normal references. We should therefore split DIP25 into two parts:

- The part that guarantees safety of "ref" can be implemented 
right away (just make taking the address of a reference unsafe 
instead of adding "addressof"). There is simply no avoiding 
implementing something like this if you want safe references.

- The part to do with fixing syntactic ambiguities of &, adding 
"addressof" and making "ref" more restrictive can be a separate 
DIP. This is the one that would potentially break code.

If we do this first, the issue of rvalue references becomes a 
non-issue. Even without explicitly defining what the lifetime of 
an rvalue reference is it becomes safe to pass them as "ref":

- In DIP25 it is the caller who enforces the safety of "ref", the 
callee just has to follow the same simple rules.

- The caller will always know the lifetime of anything it passes 
in, so it is simple to ensure that it doesn't last beyond that.


 Furthermore if one day the compiler would be sufficiently smart 
 to infer scope automatically, there still would be an important 
 difference between 'ref' and 'scope ref'.


That's not what I'm saying, even with no inference on behalf of 
the compiler there is still no difference between "scope ref" and 
"ref".
Apr 23 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/23/2013 8:33 AM, Manu wrote:

 "The r-value being passed is assigned to a stack allocated temporary, which has
 a lifetime that is identical to any other local variable, ie, the lifetime of
 the function in which it appears."
 There, I defined it.


Locals have a lifetime that is terminated by the closing } of the scope they 
appear in. There can be many such scopes in a function.

There's also the issue of:

   a || b || c

If b creates a temporary, it's life ends at the end of the expression or 
statement - it's complicated.
Apr 23 2013
parent reply Manu <turkeyman gmail.com> writes:
On 24 April 2013 04:44, Walter Bright <newshound2 digitalmars.com> wrote:


 On 4/23/2013 8:33 AM, Manu wrote:


 "The r-value being passed is assigned to a stack allocated temporary,
 which has
 a lifetime that is identical to any other local variable, ie, the
 lifetime of
 the function in which it appears."
 There, I defined it.


 Locals have a lifetime that is terminated by the closing } of the scope
 they appear in. There can be many such scopes in a function.

 There's also the issue of:

   a || b || c

 If b creates a temporary, it's life ends at the end of the expression or
 statement - it's complicated.


Is it actually complicated?
Enclosing scope seems fine too. Can you suggest a case where it could
escalate to an outer scope via a scope-ref argument?
So let's say then, that lifetime should be identical to a local declared in
the same location. No change of any rules is required, it will work as
expected.
Apr 23 2013
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 24 April 2013 at 03:33:07 UTC, Manu wrote:

 On 24 April 2013 04:44, Walter Bright 
 <newshound2 digitalmars.com> wrote:


 On 4/23/2013 8:33 AM, Manu wrote:


 "The r-value being passed is assigned to a stack allocated 
 temporary,
 which has
 a lifetime that is identical to any other local variable, ie, 
 the
 lifetime of
 the function in which it appears."
 There, I defined it.


 Locals have a lifetime that is terminated by the closing } of 
 the scope
 they appear in. There can be many such scopes in a function.

 There's also the issue of:

   a || b || c

 If b creates a temporary, it's life ends at the end of the 
 expression or
 statement - it's complicated.


 Is it actually complicated?
 Enclosing scope seems fine too. Can you suggest a case where it 
 could
 escalate to an outer scope via a scope-ref argument?
 So let's say then, that lifetime should be identical to a local 
 declared in
 the same location. No change of any rules is required, it will 
 work as
 expected.


The issue is that in this case for instance, the temporary is 
conditionally created. Then cannot bind to the enclosing scope.

As temporaries can now be created all over the place, it is 
mandatory to define them in a much more specific way than it is 
done right now.
Apr 23 2013
parent reply Manu <turkeyman gmail.com> writes:
On 24 April 2013 14:07, deadalnix <deadalnix gmail.com> wrote:


 On Wednesday, 24 April 2013 at 03:33:07 UTC, Manu wrote:


 On 24 April 2013 04:44, Walter Bright <newshound2 digitalmars.com> wrote:

  On 4/23/2013 8:33 AM, Manu wrote:

  "The r-value being passed is assigned to a stack allocated temporary,

 which has
 a lifetime that is identical to any other local variable, ie, the
 lifetime of
 the function in which it appears."
 There, I defined it.


 Locals have a lifetime that is terminated by the closing } of the scope
 they appear in. There can be many such scopes in a function.

 There's also the issue of:

   a || b || c

 If b creates a temporary, it's life ends at the end of the expression or
 statement - it's complicated.


 Is it actually complicated?
 Enclosing scope seems fine too. Can you suggest a case where it could
 escalate to an outer scope via a scope-ref argument?
 So let's say then, that lifetime should be identical to a local declared
 in
 the same location. No change of any rules is required, it will work as
 expected.


 The issue is that in this case for instance, the temporary is
 conditionally created. Then cannot bind to the enclosing scope.


Conditionally created, in the event you pass an rvalue or not? What about
binding?
I don't know what you're saying.

As temporaries can now be created all over the place, it is mandatory to

 define them in a much more specific way than it is done right now.



...I don't follow. There's absolutely nothing special about an implicitly
created temp vs an explicit one.
Apr 23 2013
parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 24 April 2013 at 04:41:47 UTC, Manu wrote:

 The issue is that in this case for instance, the temporary is
 conditionally created. Then cannot bind to the enclosing scope.


 Conditionally created, in the event you pass an rvalue or not? 
 What about
 binding?
 I don't know what you're saying.

 As temporaries can now be created all over the place, it is 
 mandatory to

 define them in a much more specific way than it is done right 
 now.



 ...I don't follow. There's absolutely nothing special about an 
 implicitly
 created temp vs an explicit one.


I think you misunderstand the problem. It is usually a bad idea 
to claim no problem exists when you don't get objections 
formulated.

Walter gave a very simple example that defeat you proposal (which 
say that temporary live until end of enclosing scope).

In a || b || c, b is conditionally executed. If it require a 
temporary, then the temporary can't have a lifetime as you 
described as it is conditionally declared (which is impossible 
with explicit declarations).

Problems along the same lines occurs when multiple temporaries 
are necessary. For instance, in what order they are created, what 
happen if the creation of one does throw ?
Apr 23 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/23/2013 8:32 PM, Manu wrote:

 Is it actually complicated?


It's a lot more complicated than simply destructing them at the end of a 
function. You have to define what happens for conditionally executed
expressions 
|| && ?:, what happens at the end of a statement, what happens at a }, what 
happens to for statement initializers, etc., for each statement type.



 Enclosing scope seems fine too. Can you suggest a case where it could escalate
 to an outer scope via a scope-ref argument?
 So let's say then, that lifetime should be identical to a local declared in the
 same location. No change of any rules is required, it will work as expected.


You cannot declare locals in conditionally compiled expressions. So it isn't as 
simple as that.
Apr 23 2013
parent reply Manu <turkeyman gmail.com> writes:
On 24 April 2013 16:52, Walter Bright <newshound2 digitalmars.com> wrote:


 On 4/23/2013 8:32 PM, Manu wrote:


 Is it actually complicated?


 It's a lot more complicated than simply destructing them at the end of a
 function. You have to define what happens for conditionally executed
 expressions || && ?:, what happens at the end of a statement, what happens
 at a }, what happens to for statement initializers, etc., for each
 statement type.



  Enclosing scope seems fine too. Can you suggest a case where it could

 escalate
 to an outer scope via a scope-ref argument?
 So let's say then, that lifetime should be identical to a local declared
 in the
 same location. No change of any rules is required, it will work as
 expected.


 You cannot declare locals in conditionally compiled expressions. So it
 isn't as simple as that.


Okay, I understand the problem now. I'm sure it's possible to address these
issues, but I see the complexity in that case.
Is it worth doing so? Will it be taken seriously, or will another reason to
dismiss it appear?
Apr 24 2013
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/24/2013 1:23 AM, Manu wrote:

 Okay, I understand the problem now. I'm sure it's possible to address these
 issues, but I see the complexity in that case.
 Is it worth doing so? Will it be taken seriously, or will another reason to
 dismiss it appear?


Yes, it's worth doing in whatever the final proposal will be.
Apr 24 2013
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 24 April 2013 at 08:23:57 UTC, Manu wrote:

 Okay, I understand the problem now. I'm sure it's possible to 
 address these
 issues, but I see the complexity in that case.
 Is it worth doing so? Will it be taken seriously, or will 
 another reason to
 dismiss it appear?


I can't talk for Walter, but let me tell what I'd say if I was 
him.

The issue is worth considering. However, clearly in a 2 step 
process :
  - Ensuring lifetime safety.
  - In a second step, ensuring rvalue ref binding when it is safe 
(this include many consideration about temporary lifetime that 
aren't trivial as you can see).



as fundamental, and incurs a fair amount of complexity, so I'd 
delay it until some core recurring problems are sorted out.
Apr 24 2013
parent reply "Diggory" <diggsey googlemail.com> writes:
 r-values can NEVER be accepted by a ref argument, they don't 
 'exist', we're
 talking about implicitly generated temporaries (which also 
 addresses the
 issues with receiving _explicit_ temp's).


They may be temporary but why should that mean they don't exist. 
(wow that's disturbingly close to a harry potter quote)

Currently temporaries exist as long as they are required to 
exist, and their lifetime never exceeds one statement.


 If I were to start arguing upon the basis of your proposal, I 
 would then
 start arguing that auto-ref in this context is pointless, and 
 automatic
 creation of a temp to hold any r-value should be the 
 universal/default
 behaviour.


I believe with the exception of literals (which are probably 
implemented as global immutable variables) temporaries are 
already whenever an rvalue is used...


 Why does the temporary need to exist any longer than the 
 current
 statement? (the current lifetime of temporaries are the 
 statement or
 expression). Surely any longer is just wasting stack space.


 
 scope ref a(scope ref int x) { return x; }
 void b(scope ref int x);
 
 b(a(10));
 
 The temp produced from '10' needs to last longer than a(), 
 because it can
 be returned to b(). Ie, the temp needs the life of any normal 
 local
 declared within that scope.
 
 How do you describe 'wasting' stack space? Does D attempt to 
 recycle stack
 space from previous short-lived locals within a single function?


I said "Why does the temporary need to exist any longer than the 
current statement?". In your example the temporary does not exist 
longer than the current statement, (the current statement in this 
example being "b(a(10))" since that is the context in which the 
temporary would be created)

I would hope that D reuses the stack space it uses for 
temporaries, certainly any decent C++ compiler would...


 The important distinction is that with scope ref, a temps life 
 can be
 prolonged and cascaded via return values.


That's simply not true, even if you allow passing rvalues as 
normal "ref", as long as DIP25A is implemented. The longest a 
temp ever needs to exist is until the end of the statement, with 
or without rvalue references. This means there is no issue with 
conditionals.

Anyway, it seems in general that everyone thinks DIP25A should be 
implemented, or am I mistaken?
Apr 24 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/24/13 6:27 AM, Diggory wrote:

 Anyway, it seems in general that everyone thinks DIP25A should be
 implemented, or am I mistaken?


I'd like to work a bit more on it before a formal review.

Andrei
Apr 24 2013
parent reply "Tove" <tove fransson.se> writes:
On Wednesday, 24 April 2013 at 12:38:19 UTC, Andrei Alexandrescu 
wrote:

 On 4/24/13 6:27 AM, Diggory wrote:

 Anyway, it seems in general that everyone thinks DIP25A should 
 be
 implemented, or am I mistaken?


 I'd like to work a bit more on it before a formal review.

 Andrei


If you find the time one day, please revisit the "Taking address" 
section.

I'm convinced that the goal of DIP25 could be fully realized even 
with some of the restrictions relaxed/lifted, with less 
code-breakage as result.

In particular:
Allowing '&' for non-ref parameters and "Stack-allocated locals" 
in  system.

It encourages bad programming style where heap is preferred over 
stack, just to silence the compiler. Yes '&' is dangerous but 
it's a separate issue, why conflate  a "Sealed references" DIP 
with restrictions on normal "non ref" C-style systems programming?

Thanks for reading this far...
Apr 24 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/24/13 3:08 PM, Tove wrote:

 On Wednesday, 24 April 2013 at 12:38:19 UTC, Andrei Alexandrescu wrote:

 On 4/24/13 6:27 AM, Diggory wrote:

 Anyway, it seems in general that everyone thinks DIP25A should be
 implemented, or am I mistaken?


 I'd like to work a bit more on it before a formal review.

 Andrei


 If you find the time one day, please revisit the "Taking address" section.

 I'm convinced that the goal of DIP25 could be fully realized even with
 some of the restrictions relaxed/lifted, with less code-breakage as result.

 In particular:
 Allowing '&' for non-ref parameters and "Stack-allocated locals" in
  system.

 It encourages bad programming style where heap is preferred over stack,
 just to silence the compiler. Yes '&' is dangerous but it's a separate
 issue, why conflate a "Sealed references" DIP with restrictions on
 normal "non ref" C-style systems programming?

 Thanks for reading this far...


I agree. At any rate, DIP25 is on the aggressive side right now. I think 
we should first implement the noncontroversial parts and defer 
discussion on the breakages.

Andrei
Apr 24 2013
parent "Namespace" <rswhite4 googlemail.com> writes:
What I ask me for some time:
Would it not be possible that a method such as
----
void foo (auto ref A a) {}
----
In such a call:
----
foo (A (42));
----
move the argument, and in a case like this
----
A a = 42;
/ / ...
foo (a);
----
gets the argument by ref? And that without to double the function 
2^n times?
Purely out of interest. Because that would actually (as far as I 
can see) the optimal solution.
Apr 24 2013
prev sibling parent reply kenji hara <k.hara.pg gmail.com> writes:
2013/4/24 Manu <turkeyman gmail.com>


 "The r-value being passed is assigned to a stack allocated temporary,
 which has a lifetime that is identical to any other local variable, ie, the
 lifetime of the function in which it appears."
 There, I defined it.


Good definition. If add more,
"getting address of "scope" parameter would be disallowed, at least in
 safe code, because it would be regarded as the escape of stack allocated
temporary."

Kenji Hara
Apr 23 2013
parent reply "Diggory" <diggsey googlemail.com> writes:
On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:

 2013/4/24 Manu <turkeyman gmail.com>


 "The r-value being passed is assigned to a stack allocated 
 temporary,
 which has a lifetime that is identical to any other local 
 variable, ie, the
 lifetime of the function in which it appears."
 There, I defined it.


 Good definition. If add more,
 "getting address of "scope" parameter would be disallowed, at 
 least in
  safe code, because it would be regarded as the escape of stack 
 allocated
 temporary."

 Kenji Hara


Why does the temporary need to exist any longer than the current 
statement? (the current lifetime of temporaries are the statement 
or expression). Surely any longer is just wasting stack space.
Apr 23 2013
next sibling parent kenji hara <k.hara.pg gmail.com> writes:
Temporaries which created on stack to pass an address to `scope ref`
parameter would be destroyed after the "current statement/expression".

T rvalue();

void foo(scope ref T) { ... }
foo(rvalue());   // here is "current statement/expression"
// lowered to: { auto __tmp = rvalue(); foo(__tmp);  __tmp.~this(); };

Note that the lifetime of taken rvalue is mostly same as non-ref parameter
case.
void bar(T arg) { ... arg.~this(); }
bar(rvalue());
// the rvalue argument is moved into the function 'bar', and destroyed
inside function.

Kenji Hara


2013/4/24 Diggory <diggsey googlemail.com>


 On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:


 2013/4/24 Manu <turkeyman gmail.com>

  "The r-value being passed is assigned to a stack allocated temporary,

 which has a lifetime that is identical to any other local variable, ie,
 the
 lifetime of the function in which it appears."
 There, I defined it.


 Good definition. If add more,
 "getting address of "scope" parameter would be disallowed, at least in
  safe code, because it would be regarded as the escape of stack allocated
 temporary."

 Kenji Hara


 Why does the temporary need to exist any longer than the current
 statement? (the current lifetime of temporaries are the statement or
 expression). Surely any longer is just wasting stack space.
Apr 23 2013
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
scope ref a(scope ref int x) { return x; }
void b(scope ref int x);


On 24 April 2013 11:02, Diggory <diggsey googlemail.com> wrote:


 On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:


 2013/4/24 Manu <turkeyman gmail.com>

  "The r-value being passed is assigned to a stack allocated temporary,

 which has a lifetime that is identical to any other local variable, ie,
 the
 lifetime of the function in which it appears."
 There, I defined it.


 Good definition. If add more,
 "getting address of "scope" parameter would be disallowed, at least in
  safe code, because it would be regarded as the escape of stack allocated
 temporary."

 Kenji Hara


 Why does the temporary need to exist any longer than the current
 statement? (the current lifetime of temporaries are the statement or
 expression). Surely any longer is just wasting stack space.
Apr 23 2013
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
 On 24 April 2013 11:02, Diggory <diggsey googlemail.com> wrote:


 On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:


 2013/4/24 Manu <turkeyman gmail.com>

  "The r-value being passed is assigned to a stack allocated temporary,

 which has a lifetime that is identical to any other local variable, ie,
 the
 lifetime of the function in which it appears."
 There, I defined it.


 Good definition. If add more,
 "getting address of "scope" parameter would be disallowed, at least in
  safe code, because it would be regarded as the escape of stack allocated
 temporary."

 Kenji Hara


 Why does the temporary need to exist any longer than the current
 statement? (the current lifetime of temporaries are the statement or
 expression). Surely any longer is just wasting stack space.




scope ref a(scope ref int x) { return x; }
void b(scope ref int x);

b(a(10));

The temp produced from '10' needs to last longer than a(), because it can
be returned to b(). Ie, the temp needs the life of any normal local
declared within that scope.

How do you describe 'wasting' stack space? Does D attempt to recycle stack
space from previous short-lived locals within a single function?
Apr 23 2013
next sibling parent "Zach the Mystic" <reachzach gggggmail.com> writes:
On Wednesday, 24 April 2013 at 03:41:17 UTC, Manu wrote:

 scope ref a(scope ref int x) { return x; }


One thing to bear in mind about this usage is that 'scope' could 
apply to the implicit 'this' reference, in addition to describing 
the return type, which might make it unfeasible. I don't know how 
often you'd want to tag the 'this' with 'scope' but not the 
return, and the other way around.

struct S {
   scope ref a(scope ref int x) { return x; }
}
Apr 23 2013
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 24 Apr 2013 13:41:05 +1000
schrieb Manu <turkeyman gmail.com>:


 On 24 April 2013 11:02, Diggory <diggsey googlemail.com> wrote:


 On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:


 2013/4/24 Manu <turkeyman gmail.com>

  "The r-value being passed is assigned to a stack allocated temporary,

 which has a lifetime that is identical to any other local variable, ie,
 the
 lifetime of the function in which it appears."
 There, I defined it.


 Good definition. If add more,
 "getting address of "scope" parameter would be disallowed, at least in
  safe code, because it would be regarded as the escape of stack allocated
 temporary."

 Kenji Hara


 Why does the temporary need to exist any longer than the current
 statement? (the current lifetime of temporaries are the statement or
 expression). Surely any longer is just wasting stack space.




 scope ref a(scope ref int x) { return x; }
 void b(scope ref int x);
 
 b(a(10));
 
 The temp produced from '10' needs to last longer than a(), because it can
 be returned to b(). Ie, the temp needs the life of any normal local
 declared within that scope.
 
 Does D attempt to recycle stack
 space from previous short-lived locals within a single function?


That's a backend optimization, not something the
language needs to be built for specifically.

-- 
Marco
Feb 25 2014
parent "w0rp" <devw0rp gmail.com> writes:
I feel I should mention a detail I observed. I am working on a 
library which generates D code from C++ code with wrappers. I 
found an interesting situation when I realised that I need to do 
something like this.

void setPosition(ref const(QPoint) point) {
      ...
      // Static array.
      auto stack = smokeStack(&point);

      cls.classFn(..., stack.ptr);
      ...
}

The above resulting in passing a pointer to a D struct which gets 
a reinterpret_cast on the C++ end and copied from that after 
that. Because I wish to also allow temporaries for this for 
obvious reasons, I must change my function signature to a 
template.

void setPosition()(auto ref const(QPointF) point);

So whatever happens with rvalue references, I would like to see 
the ability to take the address of an rvalue reference in a 
completely unsafe manner. (Or is const& an lvalue? I have 
forgetten my Scott Meyers knowledge.) I would make this an error 
in  safe functions, but allowable for  system and  trusted.
Feb 26 2014
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 24 April 2013 11:22, kenji hara <k.hara.pg gmail.com> wrote:


 Temporaries which created on stack to pass an address to `scope ref`
 parameter would be destroyed after the "current statement/expression".

 T rvalue();

 void foo(scope ref T) { ... }
 foo(rvalue());   // here is "current statement/expression"
 // lowered to: { auto __tmp = rvalue(); foo(__tmp);  __tmp.~this(); };

 Note that the lifetime of taken rvalue is mostly same as non-ref parameter
 case.
 void bar(T arg) { ... arg.~this(); }
 bar(rvalue());
 // the rvalue argument is moved into the function 'bar', and destroyed
 inside function.


The important distinction is that with scope ref, a temps life can be
prolonged and cascaded via return values.
This is very important behaviour, I couldn't write an efficient&convenient
matrix library for instance without this, i would have to choose one or the
other.



 Kenji Hara


 2013/4/24 Diggory <diggsey googlemail.com>


 On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:


 2013/4/24 Manu <turkeyman gmail.com>

  "The r-value being passed is assigned to a stack allocated temporary,

 which has a lifetime that is identical to any other local variable, ie,
 the
 lifetime of the function in which it appears."
 There, I defined it.


 Good definition. If add more,
 "getting address of "scope" parameter would be disallowed, at least in
  safe code, because it would be regarded as the escape of stack allocated
 temporary."

 Kenji Hara


 Why does the temporary need to exist any longer than the current
 statement? (the current lifetime of temporaries are the statement or
 expression). Surely any longer is just wasting stack space.
Apr 23 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/23/2013 8:44 PM, Manu wrote:

 The important distinction is that with scope ref, a temps life can be prolonged
 and cascaded via return values.
 This is very important behaviour, I couldn't write an efficient&convenient
 matrix library for instance without this, i would have to choose one or the
other.


Dealing with this issue, one way or another, is critical for any rvalue ref
design.
Apr 23 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/23/2013 6:02 PM, Diggory wrote:

 Why does the temporary need to exist any longer than the current statement?


And it doesn't in C++.
Apr 23 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/23/13 10:38 AM, Dicebot wrote:

 On Tuesday, 23 April 2013 at 14:28:54 UTC, Andrei Alexandrescu wrote:

 If it were about scope it would be very careful with lifetime of
 temporaries.

 Andrei


 Please explain. In its current form DIP36 only cares that passed
 temporary exists while called function is executed. I can't imagine any
 sane lifetime rules that would result in violating this.


One matter to look into is returning scope ref from functions (which is 
useful).

Andrei
Apr 23 2013
prev sibling parent reply "Tove" <tove fransson.se> writes:
On Tuesday, 23 April 2013 at 07:18:41 UTC, Diggory wrote:

 I'd still like someone to explain how exactly "scope ref" would 
 differ from "ref" if DIP25/DIP35 were implemented.

 If the only difference is that "scope ref" can accept rvalues 
 then why would you ever use normal "ref"? There are no extra 
 restrictions needed on "scope ref" over and above normal "ref" 
 under the assumption of DIP25/DIP35.


DIP25 imposes a number of code-breaking restrictions even in 
 system code, if DIP36 was in place, one could consider imposing 
the DIP25 restrictions only in SafeD.

Furthermore if one day the compiler would be sufficiently smart 
to infer scope automatically, there still would be an important 
difference between 'ref' and 'scope ref'.

ref
rvalue ref would only work if the compiler succeeds in inferring 
scope, it could take a conservative approach to make sure it 
always err:s in the harmless direction... i.e. any '&' or any asm 
block is an automatic failure.

scope ref
Works unless the compiler can prove it wrong(also usable from 
SafeD if marked with  trusted).
Apr 23 2013
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 23 April 2013 at 08:41:16 UTC, Tove wrote:

 DIP25 imposes a number of code-breaking restrictions even in 
  system code, if DIP36 was in place, one could consider 
 imposing the DIP25 restrictions only in SafeD.

 Furthermore if one day the compiler would be sufficiently smart 
 to infer scope automatically, there still would be an important 
 difference between 'ref' and 'scope ref'.


That is the important issue to solve. Many solution can 
jeopardize DIP36, which is why it must be delayed.

Usually conflating issue in adhoc solution ends up in crap that 
must be sorted out later.
Apr 23 2013
parent "Tove" <tove fransson.se> writes:
On Tuesday, 23 April 2013 at 09:06:52 UTC, deadalnix wrote:

 On Tuesday, 23 April 2013 at 08:41:16 UTC, Tove wrote:

 DIP25 imposes a number of code-breaking restrictions even in 
  system code, if DIP36 was in place, one could consider 
 imposing the DIP25 restrictions only in SafeD.

 Furthermore if one day the compiler would be sufficiently 
 smart to infer scope automatically, there still would be an 
 important difference between 'ref' and 'scope ref'.


 That is the important issue to solve. Many solution can 
 jeopardize DIP36, which is why it must be delayed.

 Usually conflating issue in adhoc solution ends up in crap that 
 must be sorted out later.


I see it as a future proof feature, not an issue. You want it to 
be a difference, so you can override the default compiler 
behavior.
Apr 23 2013
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/23/2013 09:07 AM, Namespace wrote:

 On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:

 Previous discussions:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com


 http://d.puremagic.com/issues/show_bug.cgi?id=9238


 I still like the Idea of DIP 36 (http://wiki.dlang.org/DIP36) with scope
 ref / in ref. ...


That's fine. I don't.
Apr 23 2013
next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
On Tuesday, 23 April 2013 at 11:32:57 UTC, Timon Gehr wrote:

 On 04/23/2013 09:07 AM, Namespace wrote:

 On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:

 Previous discussions:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com


 http://d.puremagic.com/issues/show_bug.cgi?id=9238


 I still like the Idea of DIP 36 (http://wiki.dlang.org/DIP36) 
 with scope
 ref / in ref. ...


 That's fine. I don't.


That's fine too. :)
Apr 23 2013
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 23 April 2013 21:32, Timon Gehr <timon.gehr gmx.ch> wrote:


 On 04/23/2013 09:07 AM, Namespace wrote:


 On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:


 Previous discussions:

 http://forum.dlang.org/thread/**4F84D6DD.5090405 digitalmars.**
 com#post-4F84D6DD.5090405:**40digitalmars.com<http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com>


 http://d.puremagic.com/issues/**show_bug.cgi?id=9238<http://d.puremagic.com/issues/show_bug.cgi?id=9238>


 I still like the Idea of DIP 36 (http://wiki.dlang.org/DIP36) with scope
 ref / in ref. ...


 That's fine. I don't.


Can you suggest a superior/more sensible approach?
I can't see the fault in DIP36's reasoning. It just makes sense. Why is
everyone so against it? I'm yet to understand a reason...
Apr 23 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/23/13 10:05 AM, Manu wrote:

 I can't see the fault in DIP36's reasoning. It just makes sense. Why is
 everyone so against it? I'm yet to understand a reason...


1. It defines a new language feature instead of improving the existing 
ones. At this point in the development of the language, our preference 
should be putting the existing features in good order.

2. The proposal is sketchy and does not give many details, such as the 
lifetime of temporaries bound to scope ref objects.

3. The relationship with auto ref is insufficiently described, e.g. 
there should be clarification on why auto ref cannot be improved to 
fulfill the desired role.

4. Above all this is a new language feature and again we want to resort 
to adding new feature only if it is clear that the existing features are 
insufficient and cannot be made sufficient. In particular we are much 
more inclined to impart real, demonstrable safety to "ref" and to make 
"auto ref" work as a reference that can bind to rvalues as well as lvalues.


Andrei
Apr 23 2013
parent reply Manu <turkeyman gmail.com> writes:
On 24 April 2013 00:30, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org>wrote:


 On 4/23/13 10:05 AM, Manu wrote:


 I can't see the fault in DIP36's reasoning. It just makes sense. Why is
 everyone so against it? I'm yet to understand a reason...


 1. It defines a new language feature instead of improving the existing
 ones. At this point in the development of the language, our preference
 should be putting the existing features in good order.


I see it in exactly the opposite way.
This does put an existing feature in good order, ie, scope, which is
defined but barely implemented, and might as well have been invented for
this purpose as far as I can tell from what little information is available
about it.

2. The proposal is sketchy and does not give many details, such as the

 lifetime of temporaries bound to scope ref objects.


Is that the only detail missing? An r-value passed this way produces a
temp, which is a stack variable. It's life is identical to any other stack
variable, ie, it lives for the life of the function where it appears.

3. The relationship with auto ref is insufficiently described, e.g. there

 should be clarification on why auto ref cannot be improved to fulfill the
 desired role.


auto-ref is a template concept (necessary because ref is not part of the
type like in c++). It doesn't make sense to me at all being forced into
this concept. I see people saying it over and over, but I just can't see it.
How does 'auto' apply conceptually? It's a template concept by definition.
It feels like auto-ref is deliberately contriving a concept to fit a
problem, not DIP36.

4. Above all this is a new language feature and again we want to resort to

 adding new feature only if it is clear that the existing features are
 insufficient and cannot be made sufficient.



Again, I can't see how this is a new feature at all. It's precisely what
you say; making an existing feature (scope) that already exists and is
defined actually work, and elegantly solve the problem.
auto-ref on the other hand IS a new feature (in this context), and it also
makes no sense if you ask me. It's a template concept which is not
applicable here.

In particular we are much more inclined to impart real, demonstrable safety

 to "ref"



ref is unsafe by definition. I don't believe this is possible without some
further justification.
DIP36 however creates a situation where it's known that passing a temp is
actually safe.

and to make "auto ref" work as a reference that can bind to rvalues as well

 as lvalues.



What does it mean to make a reference bind to r-values aswell as l-values?
Lots of people keep saying this too, but it doesn't really make sense to me
either.
No reference can bind to r-values, r-values can not be addressed. It's
really a temp copy of said r-value that we're dealing with, which is an
l-value, ie, a local with a lifetime that's unsuitable for passing by
non-scope-ref.
scope-ref would promise that it won't escape the callee, and thus is safe
to pass a temp.

ref is fundamentally broken in D right now. DIP36 creates a situation where
it could be fixed. I would personally take DIP36 one step further, and ban
all local's from being passed to non-scope ref.
Yes, a breaking change, but you could argue that any code that passes a
stack variable to any ref arg is already broken. But this can be addressed
in a future DIP.


...perhaps I'm missing something fundamental in DIP36, or about 'auto ref'?
I can't understand why there seem to be 2 polarised parties on this issue,
which appear to see the problem completely differently, and can't visualise
the counter perspective at all.
Apr 23 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/23/13 11:27 AM, Manu wrote:

 On 24 April 2013 00:30, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org <mailto:SeeWebsiteForEmail erdani.org>>
 wrote:

     On 4/23/13 10:05 AM, Manu wrote:

         I can't see the fault in DIP36's reasoning. It just makes sense.
         Why is
         everyone so against it? I'm yet to understand a reason...


     1. It defines a new language feature instead of improving the
     existing ones. At this point in the development of the language, our
     preference should be putting the existing features in good order.


 I see it in exactly the opposite way.
 This does put an existing feature in good order, ie, scope, which is
 defined but barely implemented, and might as well have been invented for
 this purpose as far as I can tell from what little information is
 available about it.


"scope" is a keyword, not a language feature. In case you are referring 
to scope variables, the feature "scope ref" has little to do with it.


     2. The proposal is sketchy and does not give many details, such as
     the lifetime of temporaries bound to scope ref objects.


 Is that the only detail missing?


Many details are missing. This is not a simple problem.


 An r-value passed this way produces a
 temp, which is a stack variable. It's life is identical to any other
 stack variable, ie, it lives for the life of the function where it appears.


That's a possibility, but it's a departure from current semantics and is 
not mentioned in the DIP.


     3. The relationship with auto ref is insufficiently described, e.g.
     there should be clarification on why auto ref cannot be improved to
     fulfill the desired role.


 auto-ref is a template concept (necessary because ref is not part of the
 type like in c++). It doesn't make sense to me at all being forced into
 this concept. I see people saying it over and over, but I just can't see it.
 How does 'auto' apply conceptually? It's a template concept by
 definition. It feels like auto-ref is deliberately contriving a concept
 to fit a problem, not DIP36.


Well feeling a certain way is fine but it doesn't help rigorous definitions.

The "scope ref" feature is adding no value over what we have and need to 
get in good order. People would come and ask, "how can a function accept 
both rvalues and lvalues by reference?" and then experts will answer 
with, "well is that a template or a non-template? because the means are 
entirely different". "Why are they different?" And the experts will 
answer just like you: "By definition" - thus closing the circle.


     4. Above all this is a new language feature and again we want to
     resort to adding new feature only if it is clear that the existing
     features are insufficient and cannot be made sufficient.


 Again, I can't see how this is a new feature at all. It's precisely what
 you say; making an existing feature (scope) that already exists and is
 defined actually work, and elegantly solve the problem.


This is a new feature. There is no "existing feature scope".


 auto-ref on the other hand IS a new feature (in this context), and it
 also makes no sense if you ask me. It's a template concept which is not
 applicable here.


It is a feature that has been implemented and works, just not in all cases.


     In particular we are much more inclined to impart real, demonstrable
     safety to "ref"


 ref is unsafe by definition.


We want to aim at making ref safe, thus making it useful as restricted 
pass-down pointers. For full possibilities, one should use pointers.


 I don't believe this is possible without
 some further justification.


The justification is that unsafe uses of ref are few and uninteresting 
(they can be replaced with pointers). It would be very powerful to be 
able to guarantee that safe code can use ref.


 DIP36 however creates a situation where it's known that passing a temp
 is actually safe.

     and to make "auto ref" work as a reference that can bind to rvalues
     as well as lvalues.


 What does it mean to make a reference bind to r-values aswell as
 l-values? Lots of people keep saying this too, but it doesn't really
 make sense to me either.


I don't understand the question as the answer is in it.


 No reference can bind to r-values, r-values can not be addressed.


But auto ref and scope ref do bind to r-values.


 It's
 really a temp copy of said r-value that we're dealing with, which is an
 l-value, ie, a local with a lifetime that's unsuitable for passing by
 non-scope-ref.
 scope-ref would promise that it won't escape the callee, and thus is
 safe to pass a temp.


Our aim is to have ref make that promise.


 ref is fundamentally broken in D right now. DIP36 creates a situation
 where it could be fixed.


A new feature is not a fix.


 I would personally take DIP36 one step further,
 and ban all local's from being passed to non-scope ref.
 Yes, a breaking change, but you could argue that any code that passes a
 stack variable to any ref arg is already broken. But this can be
 addressed in a future DIP.


 ...perhaps I'm missing something fundamental in DIP36, or about 'auto ref'?
 I can't understand why there seem to be 2 polarised parties on this
 issue, which appear to see the problem completely differently, and can't
 visualise the counter perspective at all.


DIP36 should be closed. We must focus on making ref safe and on making 
auto ref work with non-templates.


Andrei
Apr 23 2013
next sibling parent reply "Dicebot" <m.strashun gmail.com> writes:
On Tuesday, 23 April 2013 at 17:15:14 UTC, Andrei Alexandrescu 
wrote:

 DIP36 should be closed. We must focus on making ref safe and on 
 making auto ref work with non-templates.


It is marked as such already, since the very moment you have 
stated your opinion.
Apr 23 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/23/13 1:53 PM, Dicebot wrote:

 On Tuesday, 23 April 2013 at 17:15:14 UTC, Andrei Alexandrescu wrote:

 DIP36 should be closed. We must focus on making ref safe and on making
 auto ref work with non-templates.


 It is marked as such already, since the very moment you have stated your
 opinion.


I'd feel uncomfortable wielding this much power. I don't think this DIP 
was ready for a formal review (hence there should be no rejecting it 
yet), but there has been good discussion and experimental implementation 
as well. I suggest we keep it in draft format. If our attempts at 
working on ref/auto ref fail, we should revisit this.

Andrei
Apr 23 2013
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 24 April 2013 03:15, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org>wrote:


 On 4/23/13 11:27 AM, Manu wrote:


 On 24 April 2013 00:30, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org
<mailto:SeeWebsiteForEmail **erdani.org<SeeWebsiteForEmail erdani.org>




 wrote:

     On 4/23/13 10:05 AM, Manu wrote:

         I can't see the fault in DIP36's reasoning. It just makes sense.
         Why is
         everyone so against it? I'm yet to understand a reason...


     1. It defines a new language feature instead of improving the
     existing ones. At this point in the development of the language, our
     preference should be putting the existing features in good order.


 I see it in exactly the opposite way.
 This does put an existing feature in good order, ie, scope, which is
 defined but barely implemented, and might as well have been invented for
 this purpose as far as I can tell from what little information is
 available about it.


 "scope" is a keyword, not a language feature. In case you are referring to
 scope variables, the feature "scope ref" has little to do with it.


How so? 'scope' simply promises that a variable may not escape its scope,
no?
I think it's important to recognise it as 'scope' + 'ref', the 2 don't have
any special meaning when put together, just the logical compound, which
allows for a safe situation for temporaries that wasn't previously
available.

     2. The proposal is sketchy and does not give many details, such as

     the lifetime of temporaries bound to scope ref objects.


 Is that the only detail missing?


 Many details are missing. This is not a simple problem.


So what are some others?

 An r-value passed this way produces a

 temp, which is a stack variable. It's life is identical to any other

 stack variable, ie, it lives for the life of the function where it
 appears.


 That's a possibility, but it's a departure from current semantics and is
 not mentioned in the DIP.


I think it's presumed in the DIP, and it's certainly how Kenji implemented
it.
What 'current' semantic is it a departure from? The one where passing a
literal produces a compile error? Certainly, that's the point.

 auto-ref on the other hand IS a new feature (in this context), and it

 also makes no sense if you ask me. It's a template concept which is not
 applicable here.


 It is a feature that has been implemented and works, just not in all cases.


This isn't a 'case'. It's a separate issue.
Safely passing a temp to a ref function arg, and whether a template
argument is automatically determined to be ref or not are barely related
problems.
I still can't see how auto-ref has any business in this context.

     In particular we are much more inclined to impart real, demonstrable

     safety to "ref"


 ref is unsafe by definition.


 We want to aim at making ref safe, thus making it useful as restricted
 pass-down pointers. For full possibilities, one should use pointers.



Okay, I'm good with that too, but how is that intended to work?
If the intent is to make ref escaping disallowed by default, that is a
major breaking change...
Can we start talking about virtual-by-default again while we're at it?

 I don't believe this is possible without

 some further justification.


 The justification is that unsafe uses of ref are few and uninteresting
 (they can be replaced with pointers). It would be very powerful to be able
 to guarantee that safe code can use ref.


Again, this sounds like a major breaking change.
Why is scope-ref inferior? It's more informative, and offers more
flexibility (ie, the option of ref with or without scope)

 DIP36 however creates a situation where it's known that passing a temp

 is actually safe.

     and to make "auto ref" work as a reference that can bind to rvalues
     as well as lvalues.


 What does it mean to make a reference bind to r-values aswell as
 l-values? Lots of people keep saying this too, but it doesn't really
 make sense to me either.


 I don't understand the question as the answer is in it.


  No reference can bind to r-values, r-values can not be addressed.


 But auto ref and scope ref do bind to r-values.


  It's

 really a temp copy of said r-value that we're dealing with, which is an
 l-value, ie, a local with a lifetime that's unsuitable for passing by
 non-scope-ref.
 scope-ref would promise that it won't escape the callee, and thus is
 safe to pass a temp.


 Our aim is to have ref make that promise.


  ref is fundamentally broken in D right now. DIP36 creates a situation

 where it could be fixed.


 A new feature is not a fix.



If scope is a new feature, then the keyword shouldn't compile and pretend
that it does stuff.
It's an incomplete/unimplemented feature, not a new one.
People are aware of it, they can write code that presumes it's present and
working. It compiles successfully.

 I would personally take DIP36 one step further,

 and ban all local's from being passed to non-scope ref.
 Yes, a breaking change, but you could argue that any code that passes a
 stack variable to any ref arg is already broken. But this can be
 addressed in a future DIP.


 ...perhaps I'm missing something fundamental in DIP36, or about 'auto
 ref'?
 I can't understand why there seem to be 2 polarised parties on this
 issue, which appear to see the problem completely differently, and can't
 visualise the counter perspective at all.


 DIP36 should be closed. We must focus on making ref safe and on making
 auto ref work with non-templates.



I'm fine with that, but it sounds like a massive breaking change.
However upon the presumption of this new goal, I don't see the relevance of
auto-ref anymore? Why continue to bring it up?
If ref is safe, nothing else is needed.



 Apr 23 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/23/13 2:00 PM, Manu wrote:

 On 24 April 2013 03:15, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org <mailto:SeeWebsiteForEmail erdani.org>>
 wrote:
     "scope" is a keyword, not a language feature. In case you are
     referring to scope variables, the feature "scope ref" has little to
     do with it.


 How so? 'scope' simply promises that a variable may not escape its
 scope, no?
 I think it's important to recognise it as 'scope' + 'ref', the 2 don't
 have any special meaning when put together, just the logical compound,
 which allows for a safe situation for temporaries that wasn't previously
 available.


I understand. This could surely be used as an appeal to intuition for 
the added feature, but it's in no way a justification that it's not a 
new feature.


     Many details are missing. This is not a simple problem.


 So what are some others?


Returning a reference is an important topic.


         An r-value passed this way produces a
         temp, which is a stack variable. It's life is identical to any other

         stack variable, ie, it lives for the life of the function where
         it appears.


     That's a possibility, but it's a departure from current semantics
     and is not mentioned in the DIP.


 I think it's presumed in the DIP, and it's certainly how Kenji
 implemented it.
 What 'current' semantic is it a departure from? The one where passing a
 literal produces a compile error? Certainly, that's the point.


Currently, rvalues exist until they have been consumed by a call. By DIP 
36, some rvalues exist through the end of the function.


         auto-ref on the other hand IS a new feature (in this context),
         and it
         also makes no sense if you ask me. It's a template concept which
         is not
         applicable here.


     It is a feature that has been implemented and works, just not in all
     cases.


 This isn't a 'case'. It's a separate issue.
 Safely passing a temp to a ref function arg, and whether a template
 argument is automatically determined to be ref or not are barely related
 problems.
 I still can't see how auto-ref has any business in this context.


They are related inasmuch they solve the same problem (define a function 
that accepts both lvalues and rvalues). They are distinct because 
currently in a template you could at least in theory figure out whether 
the function has been called with an lvalue on rvalue. The code below 
does not currently work but could be made to work:

void fun(T)(auto ref T t)
{
     static if (is(t == ref)) {}
}

If we decide this feature is unnecessary (as I suspect is the case), we 
should change the implementation of auto ref to only use one body for 
both ref and non-ref versions.


              In particular we are much more inclined to impart real,
         demonstrable
              safety to "ref"


         ref is unsafe by definition.


     We want to aim at making ref safe, thus making it useful as
     restricted pass-down pointers. For full possibilities, one should
     use pointers.


 Okay, I'm good with that too, but how is that intended to work?
 If the intent is to make ref escaping disallowed by default, that is a
 major breaking change...


Walter and I are inclined to take the hit because we believe the upside 
is worth it.


 Can we start talking about virtual-by-default again while we're at it?


There are no plans to change that.


         I don't believe this is possible without
         some further justification.


     The justification is that unsafe uses of ref are few and
     uninteresting (they can be replaced with pointers). It would be very
     powerful to be able to guarantee that safe code can use ref.


 Again, this sounds like a major breaking change.
 Why is scope-ref inferior? It's more informative, and offers more
 flexibility (ie, the option of ref with or without scope)


Whether scope ref is inferior to the ref/auto ref combo is a judgment 
call. On the face of it, any new feature has to prove its utility so it 
starts from a somewhat disadvantaged position.


         DIP36 however creates a situation where it's known that passing
         a temp
         is actually safe.

              and to make "auto ref" work as a reference that can bind to
         rvalues
              as well as lvalues.


         What does it mean to make a reference bind to r-values aswell as
         l-values? Lots of people keep saying this too, but it doesn't really
         make sense to me either.


     I don't understand the question as the answer is in it.


         No reference can bind to r-values, r-values can not be addressed.


This is a matter of language definition. Rvalues can be bound to 
references today, and the bound references can be addressed.

struct S { void fun() { writeln(&this); } }
unittest { S().fun(); }


     But auto ref and scope ref do bind to r-values.


         It's
         really a temp copy of said r-value that we're dealing with,
         which is an
         l-value, ie, a local with a lifetime that's unsuitable for
         passing by
         non-scope-ref.
         scope-ref would promise that it won't escape the callee, and thus is
         safe to pass a temp.


     Our aim is to have ref make that promise.


         ref is fundamentally broken in D right now. DIP36 creates a
         situation
         where it could be fixed.


     A new feature is not a fix.


 If scope is a new feature, then the keyword shouldn't compile and
 pretend that it does stuff.


You are confusing a feature with a keyword. A given keyword may support 
many features, e.g. static, final etc.


 It's an incomplete/unimplemented feature, not a new one.
 People are aware of it, they can write code that presumes it's present
 and working. It compiles successfully.

         I would personally take DIP36 one step further,
         and ban all local's from being passed to non-scope ref.
         Yes, a breaking change, but you could argue that any code that
         passes a
         stack variable to any ref arg is already broken. But this can be
         addressed in a future DIP.


         ...perhaps I'm missing something fundamental in DIP36, or about
         'auto ref'?
         I can't understand why there seem to be 2 polarised parties on this
         issue, which appear to see the problem completely differently,
         and can't
         visualise the counter perspective at all.


     DIP36 should be closed. We must focus on making ref safe and on
     making auto ref work with non-templates.


 I'm fine with that, but it sounds like a massive breaking change.
 However upon the presumption of this new goal, I don't see the relevance
 of auto-ref anymore? Why continue to bring it up?
 If ref is safe, nothing else is needed.


auto ref is needed to accept rvalues.

Andrei



 Apr 23 2013



 next sibling parent reply "Namespace" <rswhite4 googlemail.com>  writes:


 auto ref is needed to accept rvalues.


I'm curious as to when that is implemented. Or if I purely watch 
here again in a year and the issue still exists. :)



 Apr 23 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/23/13 2:45 PM, Namespace wrote:

 auto ref is needed to accept rvalues.


 I'm curious as to when that is implemented. Or if I purely watch here
 again in a year and the issue still exists. :)


It is implemented, but currently only for templates.

Andrei



 Apr 23 2013



 next sibling parent reply "Namespace" <rswhite4 googlemail.com>  writes:


On Tuesday, 23 April 2013 at 18:58:50 UTC, Andrei Alexandrescu 
wrote:

 On 4/23/13 2:45 PM, Namespace wrote:

 auto ref is needed to accept rvalues.


 I'm curious as to when that is implemented. Or if I purely 
 watch here
 again in a year and the issue still exists. :)


 It is implemented, but currently only for templates.

 Andrei


I know that. But I meant when it will be "fixed". In other words: 
when it will be implemented for both, templates and non-templates.



 Apr 23 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/23/13 3:01 PM, Namespace wrote:

 On Tuesday, 23 April 2013 at 18:58:50 UTC, Andrei Alexandrescu wrote:

 On 4/23/13 2:45 PM, Namespace wrote:

 auto ref is needed to accept rvalues.


 I'm curious as to when that is implemented. Or if I purely watch here
 again in a year and the issue still exists. :)


 It is implemented, but currently only for templates.

 Andrei


 I know that. But I meant when it will be "fixed". In other words: when
 it will be implemented for both, templates and non-templates.


Oh, I see. Well, we better get working on 
http://d.puremagic.com/issues/show_bug.cgi?id=9238 and 
http://wiki.dlang.org/DIP25.

Andrei



 Apr 23 2013



  parent  "Diggory" <diggsey googlemail.com>  writes:


Based on DIP25/35 but slightly simplified, these rules would make 
"ref" safe, would minimise code breakage, and only marginally 
restricts the usage of "ref". The notion of "lifetime" is purely 
a tool for simplifying the explanation of the rules.

Lifetimes
0 = temporary/rvalue
1 = local variable/non-ref parameter/scope-ref parameter
2 = ref parameter
3 = static/heap allocated value

A value with any lifetime may be passed as any parameter into a 
function.

The & operator is safe only for lifetime >= 3, in other cases it 
is unsafe or system

Only values with lifetime >= 2 can be returned by ref

The lifetime of a return value is determined by the caller as 
follows:
     - Take the lifetimes of all values passed in using "ref" 
without the "scope" qualifier
     - The lifetime of the return value is the minimum lifetime of 
these values

OPTIONAL:
- A return value marked as "out" is guaranteed to have a lifetime 

= 3 (equivalent to all ref parameters being marked "scope")


- Marking a method as "scope" is equivalent to marking the "this" 
parameter as such



 Apr 23 2013



prev sibling  parent reply "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On Tuesday, April 23, 2013 14:58:50 Andrei Alexandrescu wrote:

 On 4/23/13 2:45 PM, Namespace wrote:

 auto ref is needed to accept rvalues.


 
 I'm curious as to when that is implemented. Or if I purely watch here
 again in a year and the issue still exists. :)


 
 It is implemented, but currently only for templates.


I think that the obvious thing to do for auto ref and non-templates is to just 
make it so that underneath the hood it's ref, and if you pass an rvalue to it, 
a variable is created on the stack (presumably for the lifetime of the 
statement that the function call is in) and passed to the function (so then 
it's an lvalue as ref requires).

However, the problem with simply making auto ref do this for non-templated 
functions is that then it functions fundamentally differently for templated and 
non-templated functions. And we can't change auto ref to work that way for 
templated functions, because auto ref's current behavior has proven useful for 
forwarding the refness of an argument (which auto won't do on its own), which 
improves D's forwarding capabilities. But given the combinatorial explosion of 
template instantiations if you have many auto ref variables for a templated 
function, most uses of auto ref with templated functions really should be 
using what I just described for non-templated functions. But to allow both 
behaviors, we pretty much need a new attribute. And if we're using a new 
attribute, we either need to make it do what auto ref currently does for 
templates (and therefore change auto ref to do the new thing and break some 
amount of code out there), or we need to make the new attribute do the new 
thing and never implement auto ref for non-templated functions.

So, probably the best route at this point is to come up with a new attribute 
which is replace with ref in the function definition and transparently creates 
variables for rvalues so that they can be passed to the function as lvalues, 
and then that attribute works with both templated and non-templated functions. 
But it _is_ kind of annoying to not be able to use auto ref for that given 
that that's basically what it was intended for in the first place.

Maybe I'm missing something here, but that's how the situation seems to me.

- Jonathan M Davis



 Apr 23 2013



 next sibling parent reply "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Wednesday, 24 April 2013 at 02:56:42 UTC, Jonathan M Davis 
wrote:

 So, probably the best route at this point is to come up with a 
 new attribute
 which is replace with ref in the function definition and


When you say "replace with ref", does that mean 'ref' won't 
appear in the common case?

ref T fc(scope int a) {}

The problem would be that 'scope' would behave differently, 
implying 'ref' with a value type, from with a reference type, in 
which it shouldn't imply 'ref'. I don't know if it makes sense to 
automatically promote the meaning of 'scope' to include 'ref' for 
value types while leaving it alone for reference types. This was 
objected to by others, but if it were provably harmless to allow, 
it would be an appearance improvement.



 Apr 23 2013



 next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On Wednesday, April 24, 2013 08:54:09 Zach the Mystic wrote:

 On Wednesday, 24 April 2013 at 02:56:42 UTC, Jonathan M Davis
 
 wrote:

 So, probably the best route at this point is to come up with a
 new attribute
 which is replace with ref in the function definition and


 
 When you say "replace with ref", does that mean 'ref' won't
 appear in the common case?
 
 ref T fc(scope int a) {}
 
 The problem would be that 'scope' would behave differently,
 implying 'ref' with a value type, from with a reference type, in
 which it shouldn't imply 'ref'. I don't know if it makes sense to
 automatically promote the meaning of 'scope' to include 'ref' for
 value types while leaving it alone for reference types. This was
 objected to by others, but if it were provably harmless to allow,
 it would be an appearance improvement.


You misunderstand me (possibly because I didn't proofread what I wrote). What 
I mean is that the way that auto ref should work with non-templated functions 
is that

auto foo(auto ref T param) {...}

becomes

auto foo(ref T param) {...}

underneath the hood. Then when you pass an rvalue to it - e.g. foo(T(5)) - 
that gets translated to something like

T __temp = T(5);
foo(__temp);

Then auto ref works with both lvalues and rvalues with only one function 
definition, and ref is unchanged in how it works (it still only accepts 
lvalues).

The problem is that that's not how auto ref works with templated functions, 
and we need the way that it currently works with templated functions (as it 
improves D's forwarding capabilities), but we also really should have the 
ability to do auto ref with templated functions like I just described for non-
templated ones, since the templated approach results in a combinatorial 
explosion of template instantiations as you add more auto ref parameters. And 
obviously auto ref can't mean both for, so if we're going to have both, one of 
the two ends up needing a new attribute. So, either

1. auto ref on templates continues as-is (and is never implemented for non-
templated functions), and  newattr does what I just described for both 
templated and non-templated functions, or

2. auto ref is changed to mean what I just described, and  newattr is used to 
do the forwarding trick that auto ref is often currently used for with 
templated functions.

Nowhere in here am I suggesting that ref itself be treated differently, and 
nothing I'm describing really has anything to do with scope.

- Jonathan M Davis



 Apr 25 2013



  parent reply "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Thursday, 25 April 2013 at 22:36:39 UTC, Jonathan M Davis 
wrote:

 Nowhere in here am I suggesting that ref itself be treated 
 differently, and
 nothing I'm describing really has anything to do with scope.

 - Jonathan M Davis


Okay, we're on the same page then. I was using 'scope' only as 
the ' newattr'. And I agree with your logic, and I understand the 
conflict with the existing templated 'auto ref's.

I also thought of the trick mentioned by Timon, where the 
compiler infers the need or lack thereof for the value/reference 
type in templates based on what things are done within the 
function body. But as you said, figuring out the rules for making 
the compiler smart enough to choose the right one seems like a 
hard task. If it forwards to another 'auto ref' template 
parameter, it needs to descend into that template, determine 
*its* type of parameter, and then assign the type of its own 
parameter accordingly.

It would be great if that proved to be a workable solution, not 
unlike the desirability of the compiler automatically determining 
if a class member function should be virtual or not. I don't know 
all of the cases which must be considered. Certainly if the 
parameter is assigned to, it must be 'ref', since the calling 
function will need the change. If the parameter is returned as 
'auto ref', I guess then it must certainly become 'auto ref' as 
it is currently implemented for templates. If it is passed to a 
'ref', then I would say it must be 'ref' itself. If it is passed 
to 'auto ref', it confuses me... nor do I know what other cases 
must be considered.



 Apr 25 2013



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 04/26/2013 01:58 AM, Zach the Mystic wrote:

 ...

 I also thought of the trick mentioned by Timon, where the compiler
 infers the need or lack thereof for the value/reference type in
 templates based on what things are done within the function body. But as
 you said, figuring out the rules for making the compiler smart enough to
 choose the right one seems like a hard task.


I think it is easy.


 If it forwards to another
 'auto ref' template parameter, it needs to descend into that template,
 determine *its* type of parameter, and then assign the type of its own
 parameter accordingly.
 ...


No, when forwarding to another template, non-lazy parameters will always 
have an address, ref or not ref. lazy auto ref parameters should be 
restricted to the template case.

The cases where it makes a difference are auto ref returns, 
__traits(isRef, ...), and lazy auto ref parameters. (If there are more, 
they will be detected by grepping the compiler source for occurrences of 
the 'ref' "storage class" representation, presumably an enum value.)

The compiler does not have to be particularly smart. IMO this is the way 
to generalize auto ref to non-template functions.



 Apr 26 2013



  parent reply "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Friday, 26 April 2013 at 10:33:17 UTC, Timon Gehr wrote:

 On 04/26/2013 01:58 AM, Zach the Mystic wrote:

 But as
 you said, figuring out the rules for making the compiler smart 
 enough to
 choose the right one seems like a hard task.


 I think it is easy.


 If it forwards to another
 'auto ref' template parameter, it needs to descend into that 
 template,
 determine *its* type of parameter, and then assign the type of 
 its own
 parameter accordingly.
 ...


 No, when forwarding to another template, non-lazy parameters 
 will always have an address, ref or not ref.


This suggestion simplifies the problem greatly. Is the suggestion 
made precisely because it simplifies the problem, or do you also 
consider it better with regard to performance and safety?


 The cases where it makes a difference are auto ref returns, 
 __traits(isRef, ...), and lazy auto ref parameters. (If there 
 are more, they will be detected by grepping the compiler source 
 for occurrences of the 'ref' "storage class" representation, 
 presumably an enum value.)

 The compiler does not have to be particularly smart. IMO this 
 is the way to generalize auto ref to non-template functions.


I think it's useful to realize that if the end result of an 
action is the same, it's not necessary to know exactly how the 
compiler solved the problem. If both creating a temporary, and 
passing the argument to its own template-instantiated function 
produce the same semantics, then to some extent it doesn't matter 
how the compiler did it. That leaves the question of performance. 
If it were demonstrated that neither mechanism outperforms the 
other, then it *really* doesn't matter which it chooses.



 Apr 26 2013



  parent reply "Diggory" <diggsey googlemail.com>  writes:


 You're just asking for bugs if you allow ref to accept rvalues. 
 We've had problems like this before when some literals were 
 treated as lvalues. The behavior of a function which takes its 
 argument by ref and the behavior of one which takes its 
 argument by auto ref are fundamentally different.


The only purpose of rvalue references is to allow the callee to 
mutate the value... Otherwise you would just use a const ref.

In C++ you can do this:
void foo(const int& p) {
}
int bar() { return 1; }

foo(1); // literals pass just fine by const reference
foo(bar()) // and r-values...

The only reason for r-value references is to implement move 
semantics (which means completely destroying the original value, 
and I think that counts as "mutating" it)

And the only reason r-value references have a different syntax is 
so that you can override your function and provide two 
implementations: one which accepts rvalue references and is 
destructive, and the other which accept const references and 
makes a copy.

The reason normal references couldn't be used in C++ is that 
normal non-const lvalues would cause the destructive overload to 
be called by default rather than the non-destructive one because 
they convert to "T&" more easily than "const T&", and the desired 
behaviour is that for lvalues "move" semantics must be explicit 
to prevent surprises.

An alternative I am in favour of is just to give the conversion 
to "const T&" a higher priority than the conversion to "T&" for 
lvalues, so that overloads that take both will normally choose 
the non-destructive one. To explicitly use the destructive form 
one can just explicitly cast to "T&" or call a helper function 
"T& move(T&)" which just makes the syntax nicer.

In D this would obviously be "ref const(T)" and "ref T" instead 
but the same ideas apply.



 Apr 26 2013



 next sibling parent  "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Friday, 26 April 2013 at 21:21:50 UTC, Diggory wrote:

 You're just asking for bugs if you allow ref to accept 
 rvalues. We've had problems like this before when some 
 literals were treated as lvalues. The behavior of a function 
 which takes its argument by ref and the behavior of one which 
 takes its argument by auto ref are fundamentally different.


 The only purpose of rvalue references is to allow the callee to 
 mutate the value... Otherwise you would just use a const ref.


The forum I'm using suggests that your post was written in 
response to me, although it is not my writing!



 Apr 26 2013



prev sibling  parent  "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On Friday, April 26, 2013 23:21:49 Diggory wrote:

 You're just asking for bugs if you allow ref to accept rvalues.
 We've had problems like this before when some literals were
 treated as lvalues. The behavior of a function which takes its
 argument by ref and the behavior of one which takes its
 argument by auto ref are fundamentally different.


 
 The only purpose of rvalue references is to allow the callee to
 mutate the value... Otherwise you would just use a const ref.


Except that const ref doesn't accept rvalues, and D's const is _far_ more 
restrictive than C++'s const, so requiring that a parameter be const in order 
to not care whether it's given an rvalue or lvalue but still pass it
efficiently 
would be a major negative. auto ref doesn't require or imply const, and it 
shouldn't. Granted, it should be logically const, but that can't be guaranteed 
by the type system without guaranteeing physical constness, which is often too 
strong a requirement. So, it would be a big mistake to make it so that 
constness had anything to do with our solution for passing arguments
efficiently 
without caring about refness.

- Jonathan M Davis



 Apr 26 2013



prev sibling  parent reply Manu <turkeyman gmail.com>  writes:


On 26 April 2013 08:36, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Wednesday, April 24, 2013 08:54:09 Zach the Mystic wrote:

 On Wednesday, 24 April 2013 at 02:56:42 UTC, Jonathan M Davis

 wrote:

 So, probably the best route at this point is to come up with a
 new attribute
 which is replace with ref in the function definition and


 When you say "replace with ref", does that mean 'ref' won't
 appear in the common case?

 ref T fc(scope int a) {}

 The problem would be that 'scope' would behave differently,
 implying 'ref' with a value type, from with a reference type, in
 which it shouldn't imply 'ref'. I don't know if it makes sense to
 automatically promote the meaning of 'scope' to include 'ref' for
 value types while leaving it alone for reference types. This was
 objected to by others, but if it were provably harmless to allow,
 it would be an appearance improvement.


 You misunderstand me (possibly because I didn't proofread what I wrote).
 What
 I mean is that the way that auto ref should work with non-templated
 functions
 is that

 auto foo(auto ref T param) {...}

 becomes

 auto foo(ref T param) {...}

 underneath the hood. Then when you pass an rvalue to it - e.g. foo(T(5)) -
 that gets translated to something like

 T __temp = T(5);
 foo(__temp);

 Then auto ref works with both lvalues and rvalues with only one function
 definition, and ref is unchanged in how it works (it still only accepts
 lvalues).


Why bother with 'auto'? Why not just make this default behaviour?



 Apr 25 2013



  parent  "deadalnix" <deadalnix gmail.com>  writes:


On Friday, 26 April 2013 at 04:15:24 UTC, Manu wrote:

 Why bother with 'auto'? Why not just make this default 
 behaviour?


That is the kind of question that we can answer when safety 
problem are solved. This is why I would love to see both problem 
separated : they are different.

Still need to specify precisely the lifetime of temporaries, 
which is a difficult (but solvable) problem.



 Apr 25 2013



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 04/24/2013 04:56 AM, Jonathan M Davis wrote:

...
 However, the problem with simply making auto ref do this for non-templated
 functions is that then it functions fundamentally differently for templated and
 non-templated functions. ...


http://forum.dlang.org/thread/ylebrhjnrrcajnvtthtt forum.dlang.org?page=12#post-kl8m2e:24i7g:241:40digitalmars.com



 Apr 24 2013



  parent  "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On Wednesday, April 24, 2013 16:48:59 Timon Gehr wrote:

 On 04/24/2013 04:56 AM, Jonathan M Davis wrote:

...

 However, the problem with simply making auto ref do this for non-templated
 functions is that then it functions fundamentally differently for
 templated and non-templated functions. ...


 
 http://forum.dlang.org/thread/ylebrhjnrrcajnvtthtt forum.dlang.org?page=12#p
 ost-kl8m2e:24i7g:241:40digitalmars.com


So, you're basically suggesting that auto ref on templated functions continues 
to function as it has been, except that when the compiler is able to determine 
that it could use the same approach as we'd use for non-templated functions 
without affecting the semantics of the function that that approach is used 
instead? That may be a good approach, though it does require the compiler to 
be smarter, which probably won't go well (at least initially). It _would_ 
allow us to avoid needing a new attribute though.

- Jonathan M Davis



 Apr 25 2013



prev sibling  parent  Manu <turkeyman gmail.com>  writes:


On 24 April 2013 04:29, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org>wrote:


 On 4/23/13 2:00 PM, Manu wrote:


 On 24 April 2013 03:15, Andrei Alexandrescu


      Many details are missing. This is not a simple problem.

 So what are some others?


 Returning a reference is an important topic.


I think that's easily solved. We've already touched on this.
It just extends this same rule to the return value, only 'scope ref' may
return a 'scope ref'.

         An r-value passed this way produces a

         temp, which is a stack variable. It's life is identical to any
 other

         stack variable, ie, it lives for the life of the function where
         it appears.


     That's a possibility, but it's a departure from current semantics
     and is not mentioned in the DIP.


 I think it's presumed in the DIP, and it's certainly how Kenji
 implemented it.
 What 'current' semantic is it a departure from? The one where passing a
 literal produces a compile error? Certainly, that's the point.


 Currently, rvalues exist until they have been consumed by a call. By DIP
 36, some rvalues exist through the end of the function.


I don't think this is quite right. r-values never 'exist' at all, and
they're not 'consumed'. They only come into existence when being copied to
the callee's argument list, this makes them local to, and gives them the
life of the callee.
ref args must receive an existing object however, so in the ref case, a
local temp must be created in the caller, which logically has the life of
any other local.

         auto-ref on the other hand IS a new feature (in this context),

         and it
         also makes no sense if you ask me. It's a template concept which
         is not
         applicable here.


     It is a feature that has been implemented and works, just not in all
     cases.


 This isn't a 'case'. It's a separate issue.
 Safely passing a temp to a ref function arg, and whether a template
 argument is automatically determined to be ref or not are barely related
 problems.
 I still can't see how auto-ref has any business in this context.


 They are related inasmuch they solve the same problem (define a function
 that accepts both lvalues and rvalues). They are distinct because currently
 in a template you could at least in theory figure out whether the function
 has been called with an lvalue on rvalue. The code below does not currently
 work but could be made to work:

 void fun(T)(auto ref T t)
 {
     static if (is(t == ref)) {}
 }

 If we decide this feature is unnecessary (as I suspect is the case), we
 should change the implementation of auto ref to only use one body for both
 ref and non-ref versions.


I think you're conflating 2 problems to make them appear related.
auto-ref is well defined as is when used with templates, if you change
that, it'll break the purpose it was designed for.
And again, it's not about accepting 'both lvalues and rvalues', it's got
nothing to do with r-values.
r-values can NEVER be accepted by a ref argument, they don't 'exist', we're
talking about implicitly generated temporaries (which also addresses the
issues with receiving _explicit_ temp's).

              In particular we are much more inclined to impart real,

         demonstrable
              safety to "ref"


         ref is unsafe by definition.


     We want to aim at making ref safe, thus making it useful as
     restricted pass-down pointers. For full possibilities, one should
     use pointers.


 Okay, I'm good with that too, but how is that intended to work?
 If the intent is to make ref escaping disallowed by default, that is a
 major breaking change...


 Walter and I are inclined to take the hit because we believe the upside is
 worth it.


  Can we start talking about virtual-by-default again while we're at it?


 There are no plans to change that.


Well we need to start discussing it then.
The magnitude of the breakage is less than making ref safe, and if that's
on the table, then this surely deserves some attention.

         I don't believe this is possible without

         some further justification.


     The justification is that unsafe uses of ref are few and
     uninteresting (they can be replaced with pointers). It would be very
     powerful to be able to guarantee that safe code can use ref.


 Again, this sounds like a major breaking change.
 Why is scope-ref inferior? It's more informative, and offers more
 flexibility (ie, the option of ref with or without scope)


 Whether scope ref is inferior to the ref/auto ref combo is a judgment
 call. On the face of it, any new feature has to prove its utility so it
 starts from a somewhat disadvantaged position.



Your proposal is an equally new feature, and surely stands on even ground.
I still see that auto-ref is completely irrelevant if you make ref safe.
I'm not sure why auto-ref keeps coming up under your proposal. It's fine
applied to templates, and it's no longer required when applied to functions
in the event ref can safely receive a temp.

I'm worried that your proposal significantly changes the meaning of
multiple existing features (both ref, and auto-ref), at the cost of major
breaking changes, while the DIP36 approach requires no changes to existing
features, and makes perfect sense  logically. It also keeps control in the
programmers hands, which I would never consider a problem.

         DIP36 however creates a situation where it's known that passing

         a temp
         is actually safe.

              and to make "auto ref" work as a reference that can bind to
         rvalues
              as well as lvalues.


         What does it mean to make a reference bind to r-values aswell as
         l-values? Lots of people keep saying this too, but it doesn't
 really
         make sense to me either.


     I don't understand the question as the answer is in it.


         No reference can bind to r-values, r-values can not be addressed.


 This is a matter of language definition. Rvalues can be bound to
 references today, and the bound references can be addressed.

 struct S { void fun() { writeln(&this); } }
 unittest { S().fun(); }


S() must just be an implicit local (just like every other instance of
passing an rvalue-by-ref should be).
If this code is possible, then a serious non-uniformity exists where this
is not applied to all r-values, and that should be fixed on this basis
alone.

     But auto ref and scope ref do bind to r-values.

         It's
         really a temp copy of said r-value that we're dealing with,
         which is an
         l-value, ie, a local with a lifetime that's unsuitable for
         passing by
         non-scope-ref.
         scope-ref would promise that it won't escape the callee, and thus
 is
         safe to pass a temp.


     Our aim is to have ref make that promise.


         ref is fundamentally broken in D right now. DIP36 creates a
         situation
         where it could be fixed.


     A new feature is not a fix.


 If scope is a new feature, then the keyword shouldn't compile and
 pretend that it does stuff.


 You are confusing a feature with a keyword. A given keyword may support
 many features, e.g. static, final etc.


Okay, so what does this add to scope that it doesn't already allege to
promise?
You're saying that "scope doesn't work now, therefore it's not a feature,
only a plan, with reserved syntax"?

 It's an incomplete/unimplemented feature, not a new one.

 People are aware of it, they can write code that presumes it's present
 and working. It compiles successfully.

         I would personally take DIP36 one step further,
         and ban all local's from being passed to non-scope ref.
         Yes, a breaking change, but you could argue that any code that
         passes a
         stack variable to any ref arg is already broken. But this can be
         addressed in a future DIP.


         ...perhaps I'm missing something fundamental in DIP36, or about
         'auto ref'?
         I can't understand why there seem to be 2 polarised parties on
 this
         issue, which appear to see the problem completely differently,
         and can't
         visualise the counter perspective at all.


     DIP36 should be closed. We must focus on making ref safe and on
     making auto ref work with non-templates.


 I'm fine with that, but it sounds like a massive breaking change.
 However upon the presumption of this new goal, I don't see the relevance
 of auto-ref anymore? Why continue to bring it up?
 If ref is safe, nothing else is needed.


 auto ref is needed to accept rvalues.



If I were to start arguing upon the basis of your proposal, I would then
start arguing that auto-ref in this context is pointless, and automatic
creation of a temp to hold any r-value should be the universal/default
behaviour.
You assert that requiring explicit use of 'scope' is a burden. I assert
that requiring a completely meaningless instance of 'auto' is a greater
burden, when it could easily be made the default behaviour.
If you're gonna go about making ref safe, then abandon thoughts of auto-ref
on non-templates; they can now receive a temporary safely without any
justification.



 Apr 23 2013



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 04/23/2013 07:15 PM, Andrei Alexandrescu wrote:

 ...

 The justification is that unsafe uses of ref are few and uninteresting
 (they can be replaced with pointers).


I don't get this. Uses cannot be "unsafe". They are either correct or wrong.


 It would be very powerful to be
 able to guarantee that safe code can use ref.
...


It is not required to make ref guaranteed safe in  system code to 
guarantee this.



 Apr 23 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/23/13 4:24 PM, Timon Gehr wrote:

 On 04/23/2013 07:15 PM, Andrei Alexandrescu wrote:

 ...

 The justification is that unsafe uses of ref are few and uninteresting
 (they can be replaced with pointers).


 I don't get this. Uses cannot be "unsafe". They are either correct or
 wrong.


Unfortunately the safe/unsafe and correct/wrong sets don't overlap 
perfectly.


 It would be very powerful to be
 able to guarantee that safe code can use ref.
 ...


 It is not required to make ref guaranteed safe in  system code to
 guarantee this.


Yah, agreed. It would be nice to have ref offer guarantees to all code 
while also being powerful, but if we can't make that  safe is always a 
possibility.


Andrei



 Apr 23 2013



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 04/23/2013 10:44 PM, Andrei Alexandrescu wrote:

 On 4/23/13 4:24 PM, Timon Gehr wrote:

 On 04/23/2013 07:15 PM, Andrei Alexandrescu wrote:

 ...

 The justification is that unsafe uses of ref are few and uninteresting
 (they can be replaced with pointers).


 I don't get this. Uses cannot be "unsafe". They are either correct or
 wrong.


 Unfortunately the safe/unsafe and correct/wrong sets don't overlap
 perfectly.
...


I now see what you mean.

(It is important to keep in mind that only the latter sets are an 
intrinsic property of the use. The first ones depend on the implemented 
solution. Hence, in order to be able to provide the above justification, 
an appropriate efficient safe/unsafe classification scheme must be 
devised first.)



 Apr 23 2013



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/23/13 3:07 AM, Namespace wrote:

 On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:

 Previous discussions:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com


 http://d.puremagic.com/issues/show_bug.cgi?id=9238


 I still like the Idea of DIP 36 (http://wiki.dlang.org/DIP36) with scope
 ref / in ref. On this link we have also collected several other
 suggested syntaxes.
 But maybe we should discuss first, if auto ref could work for
 non-templates.
 This would mean that
 1. auto ref generate code bloat for non-templates (doubled the function
 2^n times)
 or
 2. auto ref works like scope ref would do (create a temporary lvalue and
 pass this to the function). But that would mean that auto ref work
 differently for templates and non-templates (maybe a problem?). If we
 choose this we should also discuss about the lifetime of the temporary:
 ----
 foo(auto ref A a) { }

 foo(A());
 // should then converted to
 {
 A __temp = A();
 foo(__temp); // after this, __temp is destroyed
 }
 ----
 or
 3. the compiler choose if an argument should be passed by ref, by value
 or should be moved.
 For example:
 ----
 A a = A(42);
 foo(a); // by ref
 foo(A(23)); // move
 ----


That's on the mark. We need to make auto ref work with non-templates as 
well as templates, safely. The implementation mechanics may be 
different. Lifetime of temporaries is an important detail. I discuss 
such in http://d.puremagic.com/issues/show_bug.cgi?id=9238 but I never 
had the time to finish it.

Andrei



 Apr 23 2013



  parent reply "Namespace" <rswhite4 googlemail.com>  writes:


 That's on the mark. We need to make auto ref work with 
 non-templates as well as templates, safely. The implementation 
 mechanics may be different. Lifetime of temporaries is an 
 important detail. I discuss such in 
 http://d.puremagic.com/issues/show_bug.cgi?id=9238 but I never 
 had the time to finish it.

 Andrei



 3. The relationship with auto ref is insufficiently described, 
 e.g. there should be clarification on why auto ref cannot be 
 improved to fulfill the desired role.


I and probably many other have thought that auto ref cannot work 
for non-templates because that would mean, that auto ref would 
work differently for templates and non-templates. That was the 
main reason to search for an alternative.
But have you any ideas how auto ref should do this for 
non-templates?
E.g. with creating temporary lvalues, doubled the function etc.? 
That would make it easier to support you.



 Apr 23 2013



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/23/13 11:31 AM, Namespace wrote:

 That's on the mark. We need to make auto ref work with non-templates
 as well as templates, safely. The implementation mechanics may be
 different. Lifetime of temporaries is an important detail. I discuss
 such in http://d.puremagic.com/issues/show_bug.cgi?id=9238 but I never
 had the time to finish it.

 Andrei



 3. The relationship with auto ref is insufficiently described, e.g.
 there should be clarification on why auto ref cannot be improved to
 fulfill the desired role.


 I and probably many other have thought that auto ref cannot work for
 non-templates because that would mean, that auto ref would work
 differently for templates and non-templates. That was the main reason to
 search for an alternative.


I recall Walter has been the first to put this forward, but has been 
convinced otherwise. We can make auto ref work for non-templates, even 
if the implementation is different. The main issues are returning ref 
from functions and lifetime of temporaries, as discussed in 
http://d.puremagic.com/issues/show_bug.cgi?id=9238.


 But have you any ideas how auto ref should do this for non-templates?
 E.g. with creating temporary lvalues, doubled the function etc.? That
 would make it easier to support you.


There would be one implementation of the function. Binding rules, 
typechecking the function, and lifetime of temporaries may be changed.


Andrei



 Apr 23 2013



prev sibling next sibling parent reply "anonymous" <anonymous example.com>  writes:


On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:

 Previous discussions:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com

 http://d.puremagic.com/issues/show_bug.cgi?id=9238


As far as I understand the "usability disaster" of disallowing 
rvalue references is that I have to define a non-ref overload 
forwarding to my function(ref arg).
That ref can't be for functionality, because I'm expecting 
references to rvalues/temporaries. => ref is merely for 
performance, and the parameter can be const.

Now, this may be too naive, but what about allowing the compiler 
to pass regular (not declared ref) const arguments per reference 
as it sees fit? It would be just another optimization.

I'd declare function(const arg) and that would be it.



 Apr 23 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/23/13 4:28 AM, anonymous wrote:

 On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:

 Previous discussions:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com


 http://d.puremagic.com/issues/show_bug.cgi?id=9238


 As far as I understand the "usability disaster" of disallowing rvalue
 references is that I have to define a non-ref overload forwarding to my
 function(ref arg).
 That ref can't be for functionality, because I'm expecting references to
 rvalues/temporaries. => ref is merely for performance, and the parameter
 can be const.

 Now, this may be too naive, but what about allowing the compiler to pass
 regular (not declared ref) const arguments per reference as it sees fit?
 It would be just another optimization.

 I'd declare function(const arg) and that would be it.


Aliasing would kill that. We considered it.

Andrei



 Apr 23 2013



  parent  "anonymous" <anonymous example.com>  writes:


On Tuesday, 23 April 2013 at 14:25:36 UTC, Andrei Alexandrescu 
wrote:

 Now, this may be too naive, but what about allowing the 
 compiler to pass
 regular (not declared ref) const arguments per reference as it 
 sees fit?
 It would be just another optimization.

 I'd declare function(const arg) and that would be it.


 Aliasing would kill that. We considered it.

 Andrei


I assume you're talking about this:
http://forum.dlang.org/post/4F8C93A4.10007 erdani.com

I guess, "difficult to figure" means that the notorious 
sufficiently smart compiler is unrealistic. Too bad, because I 
don't like the idea of adding verbosity (auto ref) for the common 
case. pure and  safe already have that flaw, and I'd rather see 
no more semi-compulsory keywords.



 Apr 23 2013



prev sibling next sibling parent reply "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:

 Previous discussions:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com

 http://d.puremagic.com/issues/show_bug.cgi?id=9238


This is of course an important issue. I don't know if it can be 
divorced from the larger ref safety issue, the reason being that 
you don't want to allow rvalue references which aren't safe. And 
the ref safety issue in itself has some subtleties.

With this in mind, my analysis so far distinguishes three 
(unimplemented) features which it might be desirable for any 
given ref parameter to have. Let's say a ref parameter which 
accepts an rvalue be tagged with ' rval' for the purpose of this 
analysis. And one which may not return itself as a reference as 
' noreturn'. And one which does not assign by reference to any 
heap, static, or other ref parameter as ' noassign'.

I believe an important goal is to see how much mileage can be 
gotten out of existing syntaxes. The two existing syntaxes in 
question are 'ref' and 'auto ref'. I will leave 'auto ref' aside 
for now.

DIP25: Sealed References is divided into two parts, the first 
part dealing with 'ref' parameters, and the second with the '&' 
operator. In the first part ('DIP25A'), 'ref' is defined in such 
a way as to make it completely safe. This makes perfect sense as 
a beginning, and although it imposes significant restrictions, I 
don't have a great way to know just how much existing code would 
be affected by them. My ideal way to procede would be to 
implement DIP25A and to field test it on existing code with 
willing participants, with an eye toward finding the most common 
and general use cases where DIP25A was indeed too restrictive for 
everyday use. In my DIP35, I suggested the 'copy' function, as 
opposed to Andrei's 'identity' function, which I thought might be 
sufficiently common to warrant an addition to DIP25A, but I'm 
shooting somewhat in the dark without real world data.

The parameter attribute which would alleviate the problem with 
the 'copy' function is ' noreturn' (or some variant), which is 
pretty straightforward as to its functionality.

ref T func(ref T x,  noreturn ref T y) {
   return x; // pass
   return y; // error
   static T* t = &y; // pass!?
}

As you can see, ' noret' is pretty limited in that it doesn't 
actually prevent assigning to a global reference, but it is good 
for the purposes of this analysis. ' noassign' is the opposite.

ref T func(ref T x,  noassign ref T y, ref T* z) {
   return x; // pass
   return y; // pass
   z = &x; // pass
   z = &y; // error
   static T* t = &x; // pass
   t = &y; // error
}

My current experience leads me to look at the above code and 
think that ' noassign' should not be a separate attribute, but 
rather the default for *all*  safe 'ref' parameters. Assigning 
the address of a 'ref', which in itself does not indicate whether 
it is stored on the stack or on the heap, seems inherently 
dangerous, and I can't think of any easy way to verify that it's 
safe. The existing way to allow it would be to mark functions 
which do this ' trusted' or ' system', but there has also been 
suggested the inverse of ' noassign', ' global', which verifies 
at the call site that a non-stack reference will be passed.

void func(ref T a,  global ref T b) {
   static T* t = &a; // error with ' noassign' default
   t = &b; // pass
}

void fun(ref T a,  global ref T b) {
   T c;
   static T d;
   func(c, d); // pass
   func(d, c); // error, 'c' local

   func(a, b); // pass
   func(b, a); // error, 'a' assumed local
}

' global' would only be useful if it proved necessary to 
distinguish the kind of assignments illustrated above from 
' trusted' code, but it would require use case data.

' rval' accepts rvalue references. Safety, in this case, means 
assuming the reference comes from the stack. Clearly, 
' noassign', but what else? If the reference is returned, and 
some form of ' noreturn' is implemented, the calling function can 
know perfectly well just how local the return by reference is. So 
for me the dilemma is whether to merge ' rval' with ' noreturn' 
or not.

ref int func( rval ref int x,  noreturn ref int y,  rval 
 noreturn ref int z) {
   return x; // pass
   return y; // error
   return z; // error
}

ref T fun() {
   T t;
   return func(1, t, 2); //pass, safety enforceable
   return func(1, 2, t); // error, 2 not lvalue
   return func(t, 1, 2); // error, t local, so func() is local
}

One possible definition of 'scope ref' is equivalent to ' rval 
 noreturn  noassign'. The advantage of this definition is that 
there is no need for yet another parameter attribute, 
' noreturn', because it can be folded into 'scope ref'.

Another definition is simply ' rval  noassign'. The advantage 
here is that 'scope ref' parameters can still allow the 
'identity' function and call chains.

My goal with this analysis has been to express the extent of my 
current thought process. I understand the powerful dilemma 
between the desirability of concise and usable syntax and the 
desirability of a powerful and flexible system which covers all 
of the common use cases for the features involved. The best 
analysis uses actual data from the field.



 Apr 23 2013



  parent  "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Tuesday, 23 April 2013 at 19:45:39 UTC, Zach the Mystic wrote:

 ref T func(ref T x,  noreturn ref T y) {
   return x; // pass
   return y; // error
   static T* t = &y; // pass!?
 }


Sorry about the static definition. It should be: 'static T* t; t 
= &y;' instead, and for all subsequent definitions.



 Apr 23 2013



prev sibling next sibling parent  "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On Friday, April 26, 2013 14:15:07 Manu wrote:

 I mean is that the way that auto ref should work with non-templated
 functions
 is that
 
 auto foo(auto ref T param) {...}
 
 becomes
 
 auto foo(ref T param) {...}
 
 underneath the hood. Then when you pass an rvalue to it - e.g. foo(T(5)) -
 that gets translated to something like
 
 T __temp = T(5);
 foo(__temp);
 
 Then auto ref works with both lvalues and rvalues with only one function
 definition, and ref is unchanged in how it works (it still only accepts
 lvalues).


 
 Why bother with 'auto'? Why not just make this default behaviour?


For the same reason that T& doesn't take rvalues in C++ while const T& does. 
There's a big difference between wanting an argument to be passed as
efficiently 
as possible and specifically wanting to alter the argument being passed in. 
Plain ref is for cases where you explicitly want to mutate the argument. 
You're just asking for bugs if you allow ref to accept rvalues. We've had 
problems like this before when some literals were treated as lvalues. The 
behavior of a function which takes its argument by ref and the behavior of one 
which takes its argument by auto ref are fundamentally different.

- Jonathan M Davis



 Apr 26 2013



prev sibling next sibling parent  Manu <turkeyman gmail.com>  writes:


On 27 April 2013 05:31, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Friday, April 26, 2013 14:15:07 Manu wrote:

 I mean is that the way that auto ref should work with non-templated
 functions
 is that

 auto foo(auto ref T param) {...}

 becomes

 auto foo(ref T param) {...}

 underneath the hood. Then when you pass an rvalue to it - e.g.




 foo(T(5)) -

 that gets translated to something like

 T __temp = T(5);
 foo(__temp);

 Then auto ref works with both lvalues and rvalues with only one




 function

 definition, and ref is unchanged in how it works (it still only accepts
 lvalues).


 Why bother with 'auto'? Why not just make this default behaviour?


 For the same reason that T& doesn't take rvalues in C++ while const T&
 does.
 There's a big difference between wanting an argument to be passed as
 efficiently
 as possible and specifically wanting to alter the argument being passed in.
 Plain ref is for cases where you explicitly want to mutate the argument.
 You're just asking for bugs if you allow ref to accept rvalues. We've had
 problems like this before when some literals were treated as lvalues. The
 behavior of a function which takes its argument by ref and the behavior of
 one
 which takes its argument by auto ref are fundamentally different.


So you're saying it should be const ref instead of auto ref...
I agree.



 Apr 26 2013



prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On Saturday, April 27, 2013 10:21:32 Manu wrote:

 Why bother with 'auto'? Why not just make this default behaviour?


 
 For the same reason that T& doesn't take rvalues in C++ while const T&
 does.
 There's a big difference between wanting an argument to be passed as
 efficiently
 as possible and specifically wanting to alter the argument being passed
 in.
 Plain ref is for cases where you explicitly want to mutate the argument.
 You're just asking for bugs if you allow ref to accept rvalues. We've had
 problems like this before when some literals were treated as lvalues. The
 behavior of a function which takes its argument by ref and the behavior of
 one
 which takes its argument by auto ref are fundamentally different.


 
 So you're saying it should be const ref instead of auto ref...
 I agree.


Not at all. const is so much more restrictive in D that it really doesn't make 
sense to have const be required in order to be able to pass an argument to a 
function efficiently without caring whether it's an rvalue or an lvalue. auto 
ref permits const but doesn't require it - which is what we need - whereas 
const ref is always const.

I'm arguing that we need an attribute which differs from naked ref which 
indicates that the function doesn't care whether it's given an lvalue or an 
rvalue - it just wants it to be passed as efficiently as possible. auto ref is 
supposed to serve this purpose. Naked ref on the other hand is specifically for 
when the function needs to alter the argument and not a copy of the argument. 
const ref is ultimately kind of useless IMHO.

The only real hangup with that at this point is that templated functions 
should be able to use the auto ref solution that non-templated functions 
should use (invisibly creating a variable when an rvalue is passed so that an 
lvalue can be passed), but we also need the current auto ref functionality 
that we have for templated functions. So, either we need a new attribute, or 
we need to do what Timon suggested and make the compiler smart enough to figure 
out when it can get away with using the non-templated auto ref solution with a 
templated function without changing the function's semantics.

- Jonathan M Davis



 Apr 26 2013



  parent reply "Namespace" <rswhite4 googlemail.com>  writes:


You often say that 'auto ref' can not work for non-templates.
Why not really?
Because the resulting code bloat is not bearable, or because it 
is technically not possible?

I ask because actually this would be the best performing solution.
----
void foo(auto ref const A a) { }
//is converted to:
void foo(ref const A a) { }
//and
void foo(const A a) {
     return foo(a); //call ref version
}
----

And it work like:
----
A a = A(42);
foo(a); // call foo(ref const A a)
foo(A(23)); // call foo(const A a)
----
So no postblit is called and there is no temporary on the caller 
side.


Also I would like to see an answer of my overlooked question 
(just out of curiosity): 
http://forum.dlang.org/thread/kl4v8r$tkc$1 digitalmars.com?page=7#post-kyicmdsriwnxqiuzkaho:40forum.dlang.org



 Apr 27 2013



  parent reply Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Saturday, April 27, 2013 13:11:54 Namespace wrote:

 You often say that 'auto ref' can not work for non-templates.


Please quote at least part of the message that you're replying to. The 
threading of the various clients is often wrong, and I have no way of knowing 
who you're talking to here (and some people don't use threading at all). Based 
on the content, I'm guessing that you're talknig to me, but I really don't 
know.


 Why not really?
 Because the resulting code bloat is not bearable, or because it
 is technically not possible?


Because the templated solution involves creating new functions, whereas that's 
not going to work for non-templated functions, particularly when you consider 
.di files and prototypes. It also gets particularly nasty when function 
overriding gets added into the mix. Walter himself has said several times in 
the past that the solution used for auto ref and templates can't work with 
non-templated functions.


 Also I would like to see an answer of my overlooked question
 (just out of curiosity):
 http://forum.dlang.org/thread/kl4v8r$tkc$1 digitalmars.com?page=7#post-kyicm
 dsriwnxqiuzkaho:40forum.dlang.org


So, you're asking why the compiler can't just do a move in the rvalue case and 
pass by ref in the lvalue case for auto ref? The semantics are completely 
different. You need _one_ function, and you can't do that with one function. 
That would be like asking the function to be both

auto foo(int* i) {...}

and

auto foo(int i) {...}

only we're talking about ref rather than pointers (but ref is really a pointer 
underneat the hood). A function can only have one signature unless it's a 
template, and the only way that templates can have more is because they 
generate multiple functions (each with only one signature).

Remember that we're dealing with the C linker here, so we're pretty much 
restricted to what you can ultimately do in C as far as function signatures 
go. We have to do name mangling to do function overloading or add additional 
information to function signatures (such as pure or nothrow), and the only 
information you ultimately have is the function signature, so anything 
requiring messing with how the function works either must be done inside the 
function (where it knows nothing about the caller), or you need multiple 
functions, which generally means templates. To get around all of that, we'd 
need to create our own linker which didn't work like C's linker does, which 
would create its own set of problems.

- Jonathan M Davis



 Apr 27 2013



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/27/13 8:17 PM, Jonathan M Davis wrote:

 Walter himself has said several times in
 the past that the solution used for auto ref and templates can't work with
 non-templated functions.


I think he's partly wrong about that. Yes, the same mechanics won't 
work, but the same outcome can be made to work.

Andrei



 Apr 27 2013



  parent  Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Saturday, April 27, 2013 21:17:07 Andrei Alexandrescu wrote:

 On 4/27/13 8:17 PM, Jonathan M Davis wrote:

 Walter himself has said several times in
 the past that the solution used for auto ref and templates can't work with
 non-templated functions.


 
 I think he's partly wrong about that. Yes, the same mechanics won't
 work, but the same outcome can be made to work.


Yes. I agree. It's how auto ref is implemented for templated functions which 
won't work, not the idea. As I've described here and elsewhere, it could be 
done by simply making it so that auto ref translated to ref underneath the 
hood with the difference that when an rvalue was passed to it, a variable would 
be created on the stack which the rvalue would be assigned to so that there 
would be an lvalue to pass to the function by ref. As long as the lifetime of 
the variable matches that of the statement that the function, then it should 
be fine.

But the question here was why we couldn't just use the same solution for auto 
ref as templated functions with non-templated functions, and I think that it's 
fairly clear that that won't work as that relies on being able to generate 
additional functions. It also creates a combinatorial explosion of functions, 
so heavily using auto ref for templated functions could create a lot of 
unnecessary code bloat, which is why it would be so desirable to be able to 
use the non-templated functions solution for auto ref with template functions 
(either by introducing a new attribute - either for what auto ref does for 
templated functions now or for the new auto ref implementation - or by doing 
what Timon suggested and treating the new auto ref implementation as an 
optimization for templated functions, but that does require the compiler to 
have some smarts).

- Jonathan M Davis



 Apr 27 2013



prev sibling  parent  "Namespace" <rswhite4 googlemail.com>  writes:


On Sunday, 28 April 2013 at 00:17:53 UTC, Jonathan M Davis wrote:

 On Saturday, April 27, 2013 13:11:54 Namespace wrote:

 You often say that 'auto ref' can not work for non-templates.


 Please quote at least part of the message that you're replying 
 to. The
 threading of the various clients is often wrong, and I have no 
 way of knowing
 who you're talking to here (and some people don't use threading 
 at all). Based
 on the content, I'm guessing that you're talknig to me, but I 
 really don't
 know.


 Why not really?
 Because the resulting code bloat is not bearable, or because it
 is technically not possible?


 Because the templated solution involves creating new functions, 
 whereas that's
 not going to work for non-templated functions, particularly 
 when you consider
 .di files and prototypes. It also gets particularly nasty when 
 function
 overriding gets added into the mix. Walter himself has said 
 several times in
 the past that the solution used for auto ref and templates 
 can't work with
 non-templated functions.


 Also I would like to see an answer of my overlooked question
 (just out of curiosity):
 http://forum.dlang.org/thread/kl4v8r$tkc$1 digitalmars.com?page=7#post-kyicm
 dsriwnxqiuzkaho:40forum.dlang.org


 So, you're asking why the compiler can't just do a move in the 
 rvalue case and
 pass by ref in the lvalue case for auto ref? The semantics are 
 completely
 different. You need _one_ function, and you can't do that with 
 one function.
 That would be like asking the function to be both

 auto foo(int* i) {...}

 and

 auto foo(int i) {...}

 only we're talking about ref rather than pointers (but ref is 
 really a pointer
 underneat the hood). A function can only have one signature 
 unless it's a
 template, and the only way that templates can have more is 
 because they
 generate multiple functions (each with only one signature).

 Remember that we're dealing with the C linker here, so we're 
 pretty much
 restricted to what you can ultimately do in C as far as 
 function signatures
 go. We have to do name mangling to do function overloading or 
 add additional
 information to function signatures (such as pure or nothrow), 
 and the only
 information you ultimately have is the function signature, so 
 anything
 requiring messing with how the function works either must be 
 done inside the
 function (where it knows nothing about the caller), or you need 
 multiple
 functions, which generally means templates. To get around all 
 of that, we'd
 need to create our own linker which didn't work like C's linker 
 does, which
 would create its own set of problems.

 - Jonathan M Davis


Sorry but you're right I meant you. Thanks for explanation.



 Apr 28 2013



prev sibling  parent reply Manu <turkeyman gmail.com>  writes:


On 27 April 2013 12:26, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Saturday, April 27, 2013 10:21:32 Manu wrote:

 Why bother with 'auto'? Why not just make this default behaviour?


 For the same reason that T& doesn't take rvalues in C++ while const T&
 does.
 There's a big difference between wanting an argument to be passed as
 efficiently
 as possible and specifically wanting to alter the argument being passed
 in.
 Plain ref is for cases where you explicitly want to mutate the




 argument.

 You're just asking for bugs if you allow ref to accept rvalues. We've




 had

 problems like this before when some literals were treated as lvalues.




 The

 behavior of a function which takes its argument by ref and the




 behavior of

 one
 which takes its argument by auto ref are fundamentally different.


 So you're saying it should be const ref instead of auto ref...
 I agree.


 Not at all. const is so much more restrictive in D that it really doesn't
 make
 sense to have const be required in order to be able to pass an argument to
 a
 function efficiently without caring whether it's an rvalue or an lvalue.
 auto
 ref permits const but doesn't require it - which is what we need - whereas
 const ref is always const.

 I'm arguing that we need an attribute which differs from naked ref which
 indicates that the function doesn't care whether it's given an lvalue or an
 rvalue - it just wants it to be passed as efficiently as possible. auto
 ref is
 supposed to serve this purpose. Naked ref on the other hand is
 specifically for
 when the function needs to alter the argument and not a copy of the
 argument.
 const ref is ultimately kind of useless IMHO.

 The only real hangup with that at this point is that templated functions
 should be able to use the auto ref solution that non-templated functions
 should use (invisibly creating a variable when an rvalue is passed so that
 an
 lvalue can be passed), but we also need the current auto ref functionality
 that we have for templated functions. So, either we need a new attribute,
 or
 we need to do what Timon suggested and make the compiler smart enough to
 figure
 out when it can get away with using the non-templated auto ref solution
 with a
 templated function without changing the function's semantics.


I don't see myself ever getting on board with this auto-ref idea. I just
think it's crazy. It makes no sense to me, they are completely unrelated
concepts. It will only lead to confusion.

I'm back at scope-ref. Kenji is right as far as I'm concerned. Not to
mention, he actually did the work.
It makes perfect sense, and I can't see any reason why auto-ref should be
used instead of something that actually makes intuitive sense, and people
will eventually want to use anyway...



 Apr 26 2013



  parent reply "Diggory" <diggsey googlemail.com>  writes:


 I don't see myself ever getting on board with this auto-ref 
 idea. I just
 think it's crazy. It makes no sense to me, they are completely 
 unrelated
 concepts. It will only lead to confusion.

 I'm back at scope-ref. Kenji is right as far as I'm concerned. 
 Not to
 mention, he actually did the work.
 It makes perfect sense, and I can't see any reason why auto-ref 
 should be
 used instead of something that actually makes intuitive sense, 
 and people
 will eventually want to use anyway...


I'm against "scope ref" as the one thing "scope" means is that 
the parameter cannot be returned, and preventing rvalue 
references being returned breaks the one thing they are useful 
for.

The primary use of rvalue references in C++ is so that these 
three cases work:
vector<T> vec;
T t;
vec.push_back(T()) // Move semantics (temporary is destroyed in 
the process)
vec.push_back(t) // Copy semantics ('t' is unmodified)
vec.push_back(move(t)) // Move semantics ('t' is destroyed in the 
process)

This allows best efficiency (move is at least as fast as copy, so 
when the original is no longer needed a move should be performed)

The way it works is that push_back() has two overloads:
push_back(T&& v) {
     // Move 'v' into the vector, afterwards 'v' will have had its 
guts ripped out so to speak...
}
push_back(const T& v) {
     // Copy 'v' into the vector, 'v' is unmodified
}

- r-values such as 'T()' will default to the first overload.
- l-values such as 't' will default to the second overload.
- 'move(t)' returns an r-value causing the first overload to be 
called.

As you can see, using "scope ref" will break this third case 
because 'move()' won't be able to return an r-value, and if only 
the first two cases are going to be possible, this can be done 
without any special notion of r-value references anyway.

I'm starting to think there does need to be new syntax if all 
three of the above cases are to be covered - it would work just 
making "ref" accept rvalues but then "move" would be the default 
for lvalues, and that would be confusing - but I don't think 
using "scope" solves any problems, as the semantics required for 
rvalues are orthogonal to what "scope" actually means.

As good as it is to avoid adding new keywords, I think this might 
be a case where it's warranted...



 Apr 26 2013



 next sibling parent reply Manu <turkeyman gmail.com>  writes:


On 27 April 2013 13:29, Diggory <diggsey googlemail.com> wrote:


 I don't see myself ever getting on board with this auto-ref idea. I just

 think it's crazy. It makes no sense to me, they are completely unrelated
 concepts. It will only lead to confusion.

 I'm back at scope-ref. Kenji is right as far as I'm concerned. Not to
 mention, he actually did the work.
 It makes perfect sense, and I can't see any reason why auto-ref should be
 used instead of something that actually makes intuitive sense, and people
 will eventually want to use anyway...


 I'm against "scope ref" as the one thing "scope" means is that the
 parameter cannot be returned, and preventing rvalue references being
 returned breaks the one thing they are useful for.


It would have to return 'scope ref' also. This returns ownership to the
calling statement, which is fine, because it is where the temporary
originated in the first place.


The primary use of rvalue references in C++ is so that these three cases

 work:
 vector<T> vec;
 T t;
 vec.push_back(T()) // Move semantics (temporary is destroyed in the
 process)
 vec.push_back(t) // Copy semantics ('t' is unmodified)
 vec.push_back(move(t)) // Move semantics ('t' is destroyed in the process)

 This allows best efficiency (move is at least as fast as copy, so when the
 original is no longer needed a move should be performed)

 The way it works is that push_back() has two overloads:
 push_back(T&& v) {
     // Move 'v' into the vector, afterwards 'v' will have had its guts
 ripped out so to speak...
 }
 push_back(const T& v) {
     // Copy 'v' into the vector, 'v' is unmodified
 }

 - r-values such as 'T()' will default to the first overload.
 - l-values such as 't' will default to the second overload.
 - 'move(t)' returns an r-value causing the first overload to be called.

 As you can see, using "scope ref" will break this third case because
 'move()' won't be able to return an r-value, and if only the first two
 cases are going to be possible, this can be done without any special notion
 of r-value references anyway.

 I'm starting to think there does need to be new syntax if all three of the
 above cases are to be covered - it would work just making "ref" accept
 rvalues but then "move" would be the default for lvalues, and that would be
 confusing - but I don't think using "scope" solves any problems, as the
 semantics required for rvalues are orthogonal to what "scope" actually
 means.

 As good as it is to avoid adding new keywords, I think this might be a
 case where it's warranted...


scope ref T func(scope ref T t) { return t; }

I think this solves the problem.



 Apr 26 2013



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/27/13 2:37 AM, Manu wrote:

 scope ref T func(scope ref T t) { return t; }

 I think this solves the problem.


Consider:

scope ref T func(scope ref T t) { return t; }
scope ref T func2() { T t; return func(t); }

Would be great if you went through all of the existing work on this. 
Well after you're done preparing your DConf talks :o).


Andrei



 Apr 27 2013



 next sibling parent  "deadalnix" <deadalnix gmail.com>  writes:


On Saturday, 27 April 2013 at 11:11:05 UTC, Andrei Alexandrescu 
wrote:

 On 4/27/13 2:37 AM, Manu wrote:

 scope ref T func(scope ref T t) { return t; }

 I think this solves the problem.


 Consider:

 scope ref T func(scope ref T t) { return t; }
 scope ref T func2() { T t; return func(t); }

 Would be great if you went through all of the existing work on 
 this. Well after you're done preparing your DConf talks :o).


 Andrei


Given the current way thing bind, scope bind to the function (ie 
the implicit parameter here, probably) and not the return type. 
That is annoying as hell, and getting into the way all the time.



 Apr 27 2013



prev sibling  parent  Manu <turkeyman gmail.com>  writes:


On 27 April 2013 21:11, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org>wrote:


 On 4/27/13 2:37 AM, Manu wrote:


 scope ref T func(scope ref T t) { return t; }

 I think this solves the problem.


 Consider:

 scope ref T func(scope ref T t) { return t; }
 scope ref T func2() { T t; return func(t); }


How does auto-ref address this problem?

I think the solution (and I think you proposed this yourself), is that it
must assume the lifetime of the returned ref to be the same as the argument
with the shortest life, since it can't know which argument was returned,
and it would need to be conservative.
In this case, it knows the lifetime of the argument 't', and that it's a
local. Since it is the shortest life argument, it can't return it from
func2(), because it knows the returned ref could be(/is, in this case) the
local supplied.

In the event the function was inlined, maybe there is opportunity to lift
this restriction, since it can know which argument was returned.

Would be great if you went through all of the existing work on this. Well

 after you're done preparing your DConf talks :o).



I've read the bug where you made your proposals and associated threads. I
think you actually proposed this same solution iirc?
I'm only adding to it that it makes sense for 'scope' to be present,
otherwise you eliminate other useful cases where you may want to return an
arbitrary ref that's not an argument at all.
The rule detailed above should only apply to scope-ref, since it's the only
case that could possibly deal with short-lived temporaries anyway.
If scope ref were implemented in this way, it could receive temp's of
r-values, and safely receive locals. I would then disallow passing local
variables to non-scope-ref args, as this remains fundamentally unsafe, as
it is now.
vanilla 'ref' becomes useful for receiving and returning unrestricted
references, but we gain the confidence that it can't deal with short-lived
stack variables, which solves the problem we have now.

The only issue I've seen raised that I'm not sure of a good solution for is
the one walter raised of conditionally executed statements. But I don't
think that's addressed by any of the designs discussed.
I haven't thought about that yet, but I'm sure a solution exists. I suspect
it will have something to do with splitting the statement containing
conditions into non-conditional sub-statements, and treating lifetimes
normally within the sub-statements...

On a side note, my second talk is almost done. Huzzah! I have slides...
it's a bit rough, but it's good enough for jazz. And it's a long flight...



 Apr 27 2013



prev sibling next sibling parent  "Diggory" <diggsey googlemail.com>  writes:


 scope ref T func(scope ref T t) { return t; }

 I think this solves the problem.


It doesn't for several reasons:
- You're completely changing the meaning of "scope" for one 
specific case
- This will break DIP25A which only works if scope values cannot 
be returned
- "scope" binds to the function rather than the return type, so 
in this case is equivalent to making the hidden "this" pointer a 
"scope" parameter.



 Apr 27 2013



prev sibling  parent reply "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Saturday, 27 April 2013 at 06:37:38 UTC, Manu wrote:

 scope ref T func(scope ref T t) { return t; }

 I think this solves the problem.


struct S {
   scope ref S func() { return this; }
}

Does 'scope' apply to the return value or to the hidden 'this' 
parameter? Or both? I think it makes a difference...



 Apr 27 2013



  parent reply "Diggory" <diggsey googlemail.com>  writes:


 I think 'scope' is yet to be fully defined. Returning a 
 reference doesn't necessarily mean escaping unsafely. The 
 primary goal is safety. From the perspective of the calling 
 function, something assigned 'scope' remains in scope even if 
 it's returned.


The keyword 'scope' is very well defined at the moment - it means 
among other things that the value cannot be returned. DIP25A 
depends on this being the case, changing it will break any hope 
of sensible semantics for safe references. I agree that rvalue 
references can be returned safely - that's my main point - so 
using "scope" to identify them makes no sense.

On Saturday, 27 April 2013 at 16:13:21 UTC, Zach the Mystic wrote:

 On Saturday, 27 April 2013 at 06:37:38 UTC, Manu wrote:

 scope ref T func(scope ref T t) { return t; }

 I think this solves the problem.


 struct S {
   scope ref S func() { return this; }
 }

 Does 'scope' apply to the return value or to the hidden 'this' 
 parameter? Or both? I think it makes a difference...


Unless the meaning of "scope" is dramatically changed it doesn't 
make sense for it to be applied to a return value, since scope 
values cannot be returned. Therefore it unambiguously refers to 
the hidden 'this' pointer.



 Apr 27 2013



  parent reply "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Saturday, 27 April 2013 at 23:38:53 UTC, Diggory wrote:

 The keyword 'scope' is very well defined at the moment - it 
 means among other things that the value cannot be returned.


It's only defined very briefly in the "functions" section of the  
documentation. And it's only implemented for 'scope' delegates, 
because they get a clear benefit. Making 'ref' safe, despite how 
important it is, is not yet encoded in stone. DIP25A has been 
admitted by its author to be an aggressive solution. But there 
are several possible approaches to dealing with that, IMO. The 
most immediate is to simply force the author to mark any function 
which violates the rules ' trusted'. In other words, sweep all 
unsafe cases under the umbrella of the existing  safe-D 
framework. Perhaps the only reason to consider anything else is 
that the number of functions which will have to be marked 
' trusted' will be too high.

I think it's important to decide whether ' trusted' is adequate 
to address 'ref' safety before getting to the next step. I even 
think that a full implementation of DIP25A, as is, would help to 
determine for people just how many functions they will be forced 
to mark ' trusted'. Only if ' trusted' really ends up being too 
blunt should attributes be considered, IMO.

That being said, 'scope' might be the right tool for the job. Or 
'out' return values. I'm not under the impression that 'scope' 
*must* be used to solve the problem. For example, there are even 
more than one way to escape a parameter, and they're not all 
equal.

ref T func(ref T a, ref T* b) {
   return a; // Escape by return
   static T* t;
   t = &a; // Escape by global assign
   b = &a; // Escape by parameter assign
}

'scope' is only one attribute, yet there are three different 
types of escape here. That's why I don't want to jump in 
completely on 'scope' banning all three.



 Apr 27 2013



  parent reply "Diggory" <diggsey googlemail.com>  writes:


On Sunday, 28 April 2013 at 05:19:50 UTC, Zach the Mystic wrote:

 On Saturday, 27 April 2013 at 23:38:53 UTC, Diggory wrote:

 The keyword 'scope' is very well defined at the moment - it 
 means among other things that the value cannot be returned.


 It's only defined very briefly in the "functions" section of 
 the  documentation. And it's only implemented for 'scope' 
 delegates, because they get a clear benefit. Making 'ref' safe, 
 despite how important it is, is not yet encoded in stone. 
 DIP25A has been admitted by its author to be an aggressive 
 solution. But there are several possible approaches to dealing 
 with that, IMO. The most immediate is to simply force the 
 author to mark any function which violates the rules 
 ' trusted'. In other words, sweep all unsafe cases under the 
 umbrella of the existing  safe-D framework. Perhaps the only 
 reason to consider anything else is that the number of 
 functions which will have to be marked ' trusted' will be too 
 high.

 I think it's important to decide whether ' trusted' is adequate 
 to address 'ref' safety before getting to the next step. I even 
 think that a full implementation of DIP25A, as is, would help 
 to determine for people just how many functions they will be 
 forced to mark ' trusted'. Only if ' trusted' really ends up 
 being too blunt should attributes be considered, IMO.

 That being said, 'scope' might be the right tool for the job. 
 Or 'out' return values. I'm not under the impression that 
 'scope' *must* be used to solve the problem. For example, there 
 are even more than one way to escape a parameter, and they're 
 not all equal.

 ref T func(ref T a, ref T* b) {
   return a; // Escape by return
   static T* t;
   t = &a; // Escape by global assign
   b = &a; // Escape by parameter assign
 }

 'scope' is only one attribute, yet there are three different 
 types of escape here. That's why I don't want to jump in 
 completely on 'scope' banning all three.


OK, but I think it should be made clear that using scope for 
rvalue references would change the meaning of an attribute that 
in the near future will be very useful indeed as-is, and we will 
probably end up wanting to make a new attribute " noreturn" or 
something if scope now means something else (TBH I would not 
really be against using " noreturn" as it's somewhat clearer than 
"scope" but swapping one attribute for another seems like 
something people would be against?)

It's quite clear to me that no mathematical model for ref safety 
will be able to cover every possible case without seriously 
degrading performance. Therefore the best we can do is come up 
with a model which covers as many of the common situations as 
possible and rely on " trusted" to handle the (hopefully small) 
number of cases which cannot be handled by the model.

So far the rules in DIP25A+DIP35 seem to get the closest to this 
ideal model, and there's no reason we can't extend the rules even 
further to cover even more cases in the future.



 Apr 28 2013



  parent  "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Sunday, 28 April 2013 at 16:56:26 UTC, Diggory wrote:

 It's quite clear to me that no mathematical model for ref 
 safety will be able to cover every possible case without 
 seriously degrading performance. Therefore the best we can do 
 is come up with a model which covers as many of the common 
 situations as possible and rely on " trusted" to handle the 
 (hopefully small) number of cases which cannot be handled by 
 the model.


This is my thinking also.

One rather bad thing about the ' trusted' method is that it's not 
the called function itself which is unsafe, but only escaping the 
*returned ref*, and so ' trusted' would most accurately have to 
attach to the *calling* function, which I think will become 
untenable rather quickly.

ref T func(ref T a); // Safe in and of itself

 trusted ref T fun2() {
   T b, c;
   b = func(c); // safe
   return func(c); // not safe!!
}

Since func() has no additional attributes and is invisible, 
there's no way at all for the compiler to know what it returns. 
Yet the actual unsafe action only occurs in fun2(), which makes 
it easy to see how unwieldy the system could get, having to mark 
the calling function just because it does something which *could* 
be unsafe.

Looking at the above case makes it seem like at least one 
additional attribute, located in the function signature, must be 
accessible to the compiler in to order to reduce the number of 
functions which must otherwise rather gratuitously be marked 
' trusted'. Even with a single new attribute, 'out', which 
attaches to the function (and does not specify precisely to which 
parameters it refers) I suspect a huge number of frustrating 
cases would be taken care of. 'out', of course, simply assigns 
' noreturn' to all its parameters, even if ' noreturn' itself is 
not allowed!



 Apr 28 2013



prev sibling  parent  "Zach the Mystic" <reachzach gggggmail.com>  writes:


On Saturday, 27 April 2013 at 03:29:10 UTC, Diggory wrote:

 I'm starting to think there does need to be new syntax if all 
 three of the above cases are to be covered - it would work just 
 making "ref" accept rvalues but then "move" would be the 
 default for lvalues, and that would be confusing - but I don't 
 think using "scope" solves any problems, as the semantics 
 required for rvalues are orthogonal to what "scope" actually 
 means.


I think 'scope' is yet to be fully defined. Returning a reference 
doesn't necessarily mean escaping unsafely. The primary goal is 
safety. From the perspective of the calling function, something 
assigned 'scope' remains in scope even if it's returned.

scope ref T gin(scope ref T a) { return a; }
void fun() {
   T t;
   gin(t); // It's still in scope
}

The return is in the outer scope still, so 'scope' is not totally 
out of place. But it does seem like it's pushing it, admittedly.



 Apr 27 2013