www.digitalmars.com         C & C++   DMDScript  

digitalmars.dip.development - First Draft: ref For Variable Declarations

reply Walter Bright <newshound2 digitalmars.com> writes:
https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md
Apr 12
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
- "Enable local variables do be declared as ref."

s/do/to/

- Are they automatically scope as well?

- So this doesn't support borrowing from a data structure?

Why not?

Closing what I call DIP1000's last big hole, would allow this.

```d
DS ds = ...;
Wrapper wrapper = ds[...];
ref item = wrapper.get;

wrapper.destroy;
item.writeln; // error
```

This would be awesome to have.

- What is the point of this if the previous point isn't it?

An alias would be more appropriate for the example given.
Apr 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/12/2024 2:16 PM, Richard (Rikki) Andrew Cattermole wrote:

 - "Enable local variables do be declared as ref."
 
 s/do/to/
 
 - Are they automatically scope as well?


Scope would apply to what the ref points to, not what the ref is.



 - So this doesn't support borrowing from a data structure?
 
 Why not?


Inability to control the lifetime.


 Closing what I call DIP1000's last big hole, would allow this.
 
 ```d
 DS ds = ...;
 Wrapper wrapper = ds[...];
 ref item = wrapper.get;
 
 wrapper.destroy;
 item.writeln; // error
 ```
 
 This would be awesome to have.


Since you can pass wrapper.get to a ref parameter to a function, this already
works.



 - What is the point of this if the previous point isn't it?


??


 An alias would be more appropriate for the example given.


??
Apr 12
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 13/04/2024 9:42 AM, Walter Bright wrote:

 On 4/12/2024 2:16 PM, Richard (Rikki) Andrew Cattermole wrote:

 - "Enable local variables do be declared as ref."

 s/do/to/

 - Are they automatically scope as well?


 
 Scope would apply to what the ref points to, not what the ref is.


So this is introducing a non-transitive scope.


 - So this doesn't support borrowing from a data structure?

 Why not?


 
 Inability to control the lifetime.


This sounds like a consequence due to not having scope on the variable.

Hmm, non-transitive scope, now that I'm thinking about this, it seems 
like the limitation is in the lack of DFA, not in the type system. 
Another thing for type state analysis DFA I suppose.


 Closing what I call DIP1000's last big hole, would allow this.

 ```d
 DS ds = ...;
 Wrapper wrapper = ds[...];
 ref item = wrapper.get;

 wrapper.destroy;
 item.writeln; // error
 ```

 This would be awesome to have.


 
 Since you can pass wrapper.get to a ref parameter to a function, this 
 already works.


Ugh what?

wrapper died, item still alive.

You shouldn't be able to access item after that.

This would likely be the primary use case to using ref on a variable 
declaration and people will try it as soon as they learn they can put 
ref on a variable.


 - What is the point of this if the previous point isn't it?


 
 ??
 

 An alias would be more appropriate for the example given.




The argument made isn't compelling for it to only act as an alias to 
other variable declarations.

Borrowing memory from other variables will be attempted, and will be 
attempted often and that will cause frustration as an obvious feature 
isn't supported.
Apr 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/12/2024 3:13 PM, Richard (Rikki) Andrew Cattermole wrote:

 Scope would apply to what the ref points to, not what the ref is.


 So this is introducing a non-transitive scope.


That's always been true of scope.



 This sounds like a consequence due to not having scope on the variable.


That's correct.


 Hmm, non-transitive scope, now that I'm thinking about this, it seems like the 
 limitation is in the lack of DFA, not in the type system. Another thing for
type 
 state analysis DFA I suppose.


I don't think it's possible to do that for data structures.


 Ugh what?
 
 wrapper died, item still alive.
 
 You shouldn't be able to access item after that.
 
 This would likely be the primary use case to using ref on a variable
declaration 
 and people will try it as soon as they learn they can put ref on a variable.


Having the data structure control its contents is the usual method to deal with 
that.



 Borrowing memory from other variables will be attempted, and will be attempted 
 often and that will cause frustration as an obvious feature isn't supported.


This is not a proposal to add DFA.
Apr 13
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 14/04/2024 4:15 AM, Walter Bright wrote:

 On 4/12/2024 3:13 PM, Richard (Rikki) Andrew Cattermole wrote:

 Borrowing memory from other variables will be attempted, and will be 
 attempted often and that will cause frustration as an obvious feature 
 isn't supported.


 
 This is not a proposal to add DFA.


Indeed.

What I'm seeing is that we are limiting ourselves in designs so that 
things that people will expect to work, will not work. Simply because 
the compiler doesn't have a DFA capable of verifying program security.

For instance here is Atila saying how returning by ref is good because a 
variable cannot be declared as ref.

https://forum.dlang.org/post/ebytqnxztjsbtxtcnzke forum.dlang.org

Except with this DIP this is no longer the case.

And one example from 2021 that shows someone wanting it.

https://forum.dlang.org/post/mtefqmzpefkuehliodfd forum.dlang.org

Someone from C++ wanting to do exactly as I am suggesting 2013.

https://forum.dlang.org/post/mailman.700.1370235898.13711.digitalmars-d-learn puremagic.com

I know there have been more discussions about this over the years even 
if I didn't find them.
Apr 13
prev sibling next sibling parent reply Lance Bachmeier <no spam.net> writes:
 Decades of successful use have clearly demonstrated the utility 
 of this form of restricted pointer.


It would be useful to expand on this. It obviously hasn't been 
used for decades in D programs if you're proposing it as a new 
language feature. I don't remember ever using it in any language.
Apr 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/12/2024 2:34 PM, Lance Bachmeier wrote:

 Decades of successful use have clearly demonstrated the utility of this form 
 of restricted pointer.


 
 It would be useful to expand on this. It obviously hasn't been used for
decades 
 in D programs if you're proposing it as a new language feature. I don't
remember 
 ever using it in any language.


It's been used as ref function parameters and foreach ref parameters, as 
mentioned in the DIP.
Apr 12
parent reply bachmeier <no spam.net> writes:
On Friday, 12 April 2024 at 21:39:59 UTC, Walter Bright wrote:

 On 4/12/2024 2:34 PM, Lance Bachmeier wrote:

 Decades of successful use have clearly demonstrated the 
 utility of this form of restricted pointer.


 
 It would be useful to expand on this. It obviously hasn't been 
 used for decades in D programs if you're proposing it as a new 
 language feature. I don't remember ever using it in any 
 language.


 It's been used as ref function parameters and foreach ref 
 parameters, as mentioned in the DIP.


Then you're missing the very important qualification that they've 
been used successfully in other contexts. That doesn't provide 
evidence relevant to this proposal.
Apr 13
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/13/2024 6:06 AM, bachmeier wrote:

 On Friday, 12 April 2024 at 21:39:59 UTC, Walter Bright wrote:

 On 4/12/2024 2:34 PM, Lance Bachmeier wrote:

 Decades of successful use have clearly demonstrated the utility of this form 
 of restricted pointer.


 It would be useful to expand on this. It obviously hasn't been used for 
 decades in D programs if you're proposing it as a new language feature. I 
 don't remember ever using it in any language.


 It's been used as ref function parameters and foreach ref parameters, as 
 mentioned in the DIP.


 
 Then you're missing the very important qualification that they've been used 
 successfully in other contexts. That doesn't provide evidence relevant to this 
 proposal.


? The DIP says they've been used in other contexts. I don't see how this is not 
relevant.
Apr 13
prev sibling next sibling parent Daniel N <no public.email> writes:
On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


Finally, this is really needed to avoid unnecessary copys!


https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/declarations#reference-variables
Apr 13
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


This hits the same problem my old 
[DIP1022](https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1022.md) was
addressing. What should happen if `ref` variable is initialised with a RValue?
In my opinion it should be an error, but:
  - for `foreach` variables it is not
  - What if it's inside a template and is meant to be `ref` if 
possible but not otherwise?
The answer is the same as my DIP proposed: allowing `auto ref` 
for the variables.

Is `ref` allowed for static / global / `shared` storage class 
values?

All in all this is something that would sometimes be nice, and 
doesn't immediately make me come up with any reason it wouldn't 
work. However it allows to declare two variables that share 
memory, which gives me a feeling there might be some loophole 
that I missed. Hopefully Timon will have a look. If my worry 
turns out to be unfounded, I tend to be in favour of this.
Apr 13
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/13/2024 3:24 AM, Dukc wrote:

 On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


 
 This hits the same problem my old 
 [DIP1022](https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1022.md) was 
 addressing. What should happen if `ref` variable is initialised with a RValue?


An error. The DIP says no rvalue references.



 In my opinion it should be an error, but:
   - for `foreach` variables it is not


I think this is incorrect. Perhaps you meant static foreach?


   - What if it's inside a template and is meant to be `ref` if possible but
not 
 otherwise?
 The answer is the same as my DIP proposed: allowing `auto ref` for the
variables.


Auto ref would be a separate thing. Let's keep this focused on this DIP.



 Is `ref` allowed for static / global / `shared` storage class values?


At the moment, no. I'm not sure there's a compelling purpose for it. Many times 
I've encountered a good use for local ref variables, but never for statics.



 All in all this is something that would sometimes be nice, and doesn't 
 immediately make me come up with any reason it wouldn't work. However it
allows 
 to declare two variables that share memory, which gives me a feeling there
might 
 be some loophole that I missed.


That's what  live addresses.



 Hopefully Timon will have a look. If my worry 
 turns out to be unfounded, I tend to be in favour of this.


Thank you.
Apr 13
parent reply Dukc <ajieskola gmail.com> writes:
On Saturday, 13 April 2024 at 16:12:29 UTC, Walter Bright wrote:

 On 4/13/2024 3:24 AM, Dukc wrote:

 What should happen if `ref` variable is initialised with a 
 RValue?


 An error. The DIP says no rvalue references.


Good so far...


 In my opinion it should be an error, but:
   - for `foreach` variables it is not


 I think this is incorrect. Perhaps you meant static foreach?


No I didn't. This compiles and does nothing:

```D
 safe void main()
{
     foreach(ref el; 0 .. 5) el = 42;
}
```

...which means that if `ref int = 42;` fails to compile, as it 
should, it's inconsistent with the present behaviour!



 Is `ref` allowed for static / global / `shared` storage class 
 values?


 At the moment, no. I'm not sure there's a compelling purpose 
 for it. Many times I've encountered a good use for local ref 
 variables, but never for statics.


No problem - but the dip should explicitly say it's for local 
variables then, not for variables in general.


 However it allows to declare two variables that share memory, 
 which gives me a feeling there might be some loophole that I 
 missed.


 That's what  live addresses.


To be clear I'm happy with variables sharing memory, if it 
doesn't break type safety in any way. I just have a suspicion 
that if it does so, it might be hard to notice at least for me.
Apr 13
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/13/2024 11:22 AM, Dukc wrote:

 This compiles and does nothing:
 
 ```D
  safe void main()
 {
      foreach(ref el; 0 .. 5) el = 42;
 }
 ```


https://issues.dlang.org/show_bug.cgi?id=24499



 No problem - but the dip should explicitly say it's for local variables then, 
 not for variables in general.


The DIP does say:

"There doesn't appear to be a downside of using ref for such declarations, so
by 
extension ordinary local variables should also benefit from being declared as
ref."

Though it could be made more explicit.
Apr 13
prev sibling next sibling parent reply Witold Baryluk <witold.baryluk gmail.com> writes:
On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


I am not exactly sure what is this solving, or how it improves 
code. And DIP draft does not exactly explain that either.

I am afraid D code will then become C++-like, and be riddled with 
all these refs.

For the last 20 years I was actively avoiding using references in 
C++ code, (with only exception of `const&` being ok, only because 
this avoids `->` in C++, and implicitly in general tells me that 
pointer ownership is not being transferred to the called 
function).  And I have seen many companies with large code bases 
doing the same. One of the main reasons for me to avoid it, is 
that it is obstructing call site by making things implicit. One 
could say similar things about `lazy`, but I think that is fine, 
as it is used in way more narrow context. Another reason to avoid 
are reassignment semantics, and complex interactions with 
construction, destruction, which are non trivial.

 From all C++ devs that I do know in professional setting (most of 
them 10-20 years writing C++ code every day), in fact not a 
single one of them, know exact semantics of C++ references. And 
anybody who said they do for sure, in fact didn't.

In C++, I didn't write a single ref variable in my life (by 
choice), other than while debugging others code. And not a single 
(non-const) ref parameter. And it never was limiting me or caused 
readability problems (quite the contrary actually).

(Usage of `&` in lambda capture is fine, as there is no other 
way, and this is useful. But in D, things are currently inferred 
automatically by compiler. Plus in lambda capture semantic is 
pretty clear and very rarely abused in a way that makes code hard 
to follow).

I would say that ref variables is the worst C++ feature from all 
its "features".

In C++, they are kind of needed sometimes, i.e. for example when 
implementing mutable containers, to implement things like `x[foo] 
+= 1`, but it can be abused (i.e. if `x[foo]` is assigned to 
`auto &z` somewhere first, then one mutates `x` in a way, that 
`z` becomes invalid reference, and only then you access `z`. But 
in D, we have nicer and more power ways to deal with things like 
`x[foo] += 1` using `op*` family of function. So we do not need 
these kinds of references most of the time. (I do know some 
people do use non-const ref return values for functions in D. But 
it is not exactly needed most of the time, or can be implemented 
using wrapper structs with some kind of forwarding if really 
really required).

As of safety. I would say references, as they are in C++, make 
C++ less safe. Exactly because of extra complications, 
implicitness, and other issues.

In D, readability is even less of an issue for pointers, as `->` 
is not required, and one will use a `.`, which is nice, and one 
will sporadically see maybe something like  `(*x) =`, or similar 
(rarely, as this is usually used for out parameters for returning 
multiple values, but that can be easily done with `out` or some 
multi-value return / tuple return maybe). And using `*x = ` is in 
fact pretty nice to remind you (and code reviewer, or future you) 
what is happening exactly.


I am not totally against (it is a choice to use or not use this 
future), but if it proliferates into libraries (or phobos!), I 
would be rather disappointed for me personally.

And yes, I do like and use `const ref` in D sometimes (i.e. for 
fixed arrays, or some structs), and sporadically `ref` in 
`foreach` which is handy. Still I not needed, I don't. I.e. I 
trust compiler to not copy some structs in array if I do not 
modify them for example.

And for classes everything is by "ref" (the object itself, not 
the variable), so that mostly avoid copying issues, that C++ 
needs to deal with.

More concretely. Example you show

```c++
ref int dark(ref int x, int i) {
     ref j = i;     // j now points to i
     j = 3;         // now i is 3 as well

...
```

and I am already lost basically. Lets say one has very slightly 
modified same code:


```c++
struct A {
   ~this() {
      ....
   }
}

ref int dark(ref A x, A i) {
     ref j = i;     // j now points to i
     j = A(3);         // now i is 3 as well

...
```

And now, I need to think very hard to know what is actually 
happening. Or even miss the fact that my assignment probably 
caused call to `A`s destructor. And god forgive there were other 
references (by pointers) to object in `j` previously.

I do not like implicit mechanisms that are not easy to track 
without extra context.

And I do fail to see why one would complicating language and 
implementation for very small gain, where in very rare cases 
where one there would be benefit, doing it explicitly using 
pointers is in fact better and cleaner by being explicit. If 
compiler could fully prove correctness, than maybe I could see 
some benefit, but good luck with that.

You can also see languages like Go, where there are pointers, and 
no references. And they are perfectly happy with that, with 
gazillion of libraries showing it is not really an issue.

Regards.
Apr 13
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I appreciate your thoughts.

I agree that ref parameters make it slightly less clear what is happening at
the 
call site. But we don't really put documentation at the call site, we put the 
documentation at the callee. A ref parameter self-documents restrictions on how 
the pointer is used. `out` goes even further, it says that a value is not being 
passed in, only out. Pointers come with other behaviors - they can be 
incremented, and they can be free'd.

In my years of experience, I see a lot of operating system APIs that use a 
pointer. It's often unclear what is going to happen with that pointer. Most of 
the time, they stick to ref semantics, but not always. Like a char* being 
actually a string in C, rather than a reference to a char.

Just recently, I submitted a PR to replace some pointers in the DMD source code 
with refs, like this one:

https://github.com/dlang/dmd/pull/16379/files#diff-d58f278dd62016ccc4d0723aab00357f640541f027b68df715511d8e36c8115dR1773

Note how much simpler the line becomes, though it does exactly the same thing.

I've come to appreciate this difference.

And the more that arrays and refs can replace raw pointer use, the more memory 
safe the code is. Ref also makes it easy to switch a parameter from value <=> 
ref, for trying out which is faster.

It's worthwhile to extend the ability to use refs instead of pointers.

BTW, although Go has pointers, it does not allow pointer arithmetic. That makes 
them much like D's ref types.
Apr 14
parent reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Sunday, 14 April 2024 at 23:28:07 UTC, Walter Bright wrote:

 I appreciate your thoughts.

 I agree that `ref` parameters make it slightly less clear what 
 is happening at the call site. But we don't really put 
 documentation at the call site, we put the documentation at the 
 callee. A `ref` parameter self-documents restrictions on how 
 the pointer is used. `out` goes even further, it says that a 
 value is not being passed in, only out. Pointers come with 
 other behaviors - they can be incremented, and they can be 
 free'd.

 In my years of experience, I see a lot of operating system APIs 
 that use a pointer. It's often unclear what is going to happen 
 with that pointer. Most of the time, they stick to `ref` 
 semantics, but not always. Like a `char*` being actually a 
 string in C, rather than a reference to a `char`.


Zig tries to be explicit about how an argument is passed: It only 
allows pass-by-value. If you want to pass by reference, pass the 
address by value.

While this looks like the same mess as in C, Zig has a whole zoo 
of pointer types that indicate the various intentions behind a 
pointer. Zig distinguishes a single-element pointer and a pointer 
into an array; only the latter allows pointer arithmetic. Zig 
also distinguishes pointers into arrays that are 
sentinel-terminated.
A C `char*` that’s intended to be a string would be typed as 
pointer into 0-terminated array of `char`. A C `int*` that’s 
supposed to be `ref` or `out` would be a single-element pointer 
to `int`. Zig also has `null` indicators, i.e. it distinguishes 
pointers and slices that can be null and those that can’t be.

It’s a complete 180° from what D’s philosophy is. D style is 
passing a `ref` or a slice when you can. This means when you see 
a pointer, it means something more complicated is going on. One 
thing that I’d never do because it would confuse my future self: 
Passing `*null` via ref. It’s legal in D (not in C++), but I have 
never seen that being meaningfully used, i.e. I have yet to see 
an API that takes some `ref` parameter and the docs say that 
passing `*null` does X and Y.

I can see the value in both philosophies. While they seem 
incompatible, they’re not. D can take a concept or two from Zig.


before the argument in the function call. This ship has sailed 

```cs
int i = 0, j = 1;
ref int r = ref i;
r = 2; // as if i = 2
r = ref j; // r points to j now
r = 3; // as if j = 3
```
I’m not saying D should allow reassigning `ref`, just pointing 

regard.
Apr 26
parent Walter Bright <newshound2 digitalmars.com> writes:
Sure, they're all different.





the function call

That makes refactoring much more work. By putting ref at the function 
declaration, you can switch it on and off without having to change anything
else.

C++ can have refs to null:

```
int *p = NULL;
int& r = *p;
```

You're not supposed to do that, but compilers don't detect it.

As for Zig, in D a ref is a single element pointer, a [] is a pointer to an
array.

A special pointer to a 0 terminated array does not confer any useful semantics
I 
can think of - one could do it in D by just making the pointer a field in a 
struct and overload the operators.

Essentially, we're in good shape here.
May 02
prev sibling parent zjh <fqbqrr 163.com> writes:
On Sunday, 14 April 2024 at 00:04:13 UTC, Witold Baryluk wrote:

 I would say that ref variables is the worst C++ feature from 
 all its "features".



Unless necessary, `C++` actually does not advocate pointers. The 
various facilities in `C++` allow you to avoid pointers. Pointers 
are too `low-level`.
`References` in `C++` is very common.
I think it's best to find a `simpler way` to express reference in 
D, or use it like C++'s `&`, otherwise`ref` would be too ugly.
Apr 14
prev sibling next sibling parent Johan <j j.nl> writes:
On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


It would help if you include an example where such reference 
variable makes the code better / enables writing nicer code. One 
example usecase that I can come up with quickly:

```
struct A {...}

ref A foo();

void bar() {
     A* a = foo(); // not allowed
     A* a = &foo(); // workaround
     a.modifySomething(100);

     A b = foo(); // makes a copy
     b.modifySomething(100); // does not modify the thing returned 
by foo.

     auto c = foo(); // makes a copy

     ref d = foo(); // avoids copy
     d.modifySomething(100); // operates on the thing returned by 
foo
                             // instead of on a copy
}
```
In this case, the reference variable avoids having to use a 
pointer. Not a huge gain, but it helps a little (no need to worry 
about null or about changing the pointer itself).


Another comment: please be exhaustive in examples, in order to 
define clearly and *explicitly* the semantics. Are these allowed 
and if yes what do they do?
```
ref int i = ...
ref j = i;
j = 1; // ?
```

```
void foo(ref int i) {
     ref j = i;
     j = 1; // ?
}
```

```
void foo(ref int i);

ref int j = ...
foo(j); // ?
```

```
int* foo();

ref int i = *foo(); // ?
```

```
ref int i = ...
auto j = i; // typeof(j)?
auto p = &i; // typeof(p)?
```

```
// This is allowed in C++, with different syntax of course.
struct S {
     ref int j;

     S(ref int i) {
         j = i;
     }
     // Default constructor is deleted, because j must be 
initialized.
}

int a;
S aa = S(a);
```

```
(ref int)[3] array_of_refs; // (const int*)[3] arr; is currently 
not allowed, so this case probably neither.
```
Apr 14
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


```
Returning a ref variable from a function that returns a ref is 
not allowed. I.e. a ref cannot be assigned to a ref with a scope 
that exceeds the scope of the source.
```

This seems like an unnecessary restriction. From experience, 
unnecessary restrictions always seem to come back to bite us. 
Yes, it can be added later, but this seems like a no brainer.

1. A ref variable cannot be rebound. Which means that at 
declaration time, you know what it points to and will always 
point to. The lifetime of the new ref variable can assume the 
lifetime of the thing it is assigned to.
2. This would not require flow analysis, since the inputs are 
immutable and known at the time semantic is run for the 
declaration.
3. DIP 1000 already allows such lifetime forwarding:

```d
void foo(ref int x)
{
     // ref int y = x;
     // return y;
     // same as:
     ref int bar(ref int x2) {
         return x2;
     }
     return bar(x);
}
```

Other than that, this looks pretty good!

-Steve
Apr 15
parent reply Dennis <dkorpel gmail.com> writes:
On Monday, 15 April 2024 at 16:24:51 UTC, Steven Schveighoffer 
wrote:

 This seems like an unnecessary restriction.


It's necessary to prevent the creation of dangling stack 
pointers, since a `ref` can be bound to a stack variable.


 3. DIP 1000 already allows such lifetime forwarding:

 ```d
 void foo(ref int x)
 {
     // ref int y = x;
     // return y;
     // same as:
     ref int bar(ref int x2) {
         return x2;
     }
     return bar(x);
 }
 ```


When you fix foo's return type to be `ref int`, you get:


 Error: returning `bar(x)` escapes a reference to parameter `x`
Apr 15
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Monday, 15 April 2024 at 17:20:59 UTC, Dennis wrote:

 On Monday, 15 April 2024 at 16:24:51 UTC, Steven Schveighoffer 
 wrote:

 This seems like an unnecessary restriction.


 It's necessary to prevent the creation of dangling stack 
 pointers, since a `ref` can be bound to a stack variable.


Yes, so you don't allow returning the ref in that case? It is 
trivial to know what a `ref` is bound to at compile time.


 When you fix foo's return type to be `ref int`, you get:


 Error: returning `bar(x)` escapes a reference to parameter `x`




Yes, I forgot to update the message after I fixed the return type 
in my test, but it compiled for me. I forgot to add ` safe` as 
well, I always forget that when testing dip1000.

But also, returning `x` is disallowed in this case.

You also have to add `return` to the parameter, and then it 
compiles.

For reference, the edited code which compiles is:

```d
 safe ref int foo(return ref int x)
{
     // ref int y = x;
     // return y;
     // same as:
     ref int bar(ref int x2) {
         return x2;
     }
     return bar(x);
}
```

-Steve
Apr 15
next sibling parent Daniel N <no public.email> writes:
On Monday, 15 April 2024 at 19:21:55 UTC, Steven Schveighoffer 
wrote:

 You also have to add `return` to the parameter, and then it 
 compiles.

 For reference, the edited code which compiles is:

 ```d
  safe ref int foo(return ref int x)
 {
     // ref int y = x;
     // return y;
     // same as:
     ref int bar(ref int x2) {
         return x2;
     }
     return bar(x);
 }
 ```

 -Steve


why not

```d
ref int bar(return ref int x2)
```

is it inferred for nested functions?
Apr 15
prev sibling parent Dukc <ajieskola gmail.com> writes:
On Monday, 15 April 2024 at 19:21:55 UTC, Steven Schveighoffer 
wrote:

 For reference, the edited code which compiles is:

 ```d
  safe ref int foo(return ref int x)
 {
     // ref int y = x;
     // return y;
     // same as:
     ref int bar(ref int x2) {
         return x2;
     }
     return bar(x);
 }
 ```

 -Steve


This also answers your original question. You can't do

```D
 safe ref int fun
     (return ref int a, return ref int b, return ref int c)
{   // ...
     ref int x = b;
     // ...
     return x;
}
```

...because it uses the same lifetime checks `scope` uses, and you 
can't do

```D
 safe int* fun
     (return ref int a, return ref int b, return ref int c)
{   // ...
     scope int* x = &b;
     // ...
     return x;
}
```

either. In the second example there is a good reason for the 
limitation, as `x` could later be assigned to point to a local 
instead.

It could work though. Either `ref`s and immutable/const scope 
pointers that don't refer to non-return `scope` could be made 
returnable, or we could allow `return ref`/`return scope`/`ref 
return scope`/`return ref scope` for local variables too, the 
rule being that `return` references can be assigned `return` 
references with longer lifetime, but not non-`return` local 
references.
Apr 16
prev sibling next sibling parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


Ref variables would be useful for refactoring. E.g. to disallow 
foreach over an array with a non-const `ref` index - related to:
https://issues.dlang.org/show_bug.cgi?id=24499

```d
foreach (ref i, ref e; array) {
     // code that modifies `i`, causing skipped/repeated elements 
of foreach
```
Rewrite to:
```d
for (size_t i; i < array.length; i++) {
     ref e = array[i];
     // code that modifies `i`, which is now less surprising
```

Currently you'd have to write `ref e() => array[i];`, which is 
less intuitive and is not the same if `array` is reassigned 
between calls to `e`.
Apr 17
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 17/04/2024 9:20 PM, Nick Treleaven wrote:

 On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


 
 Ref variables would be useful for refactoring. E.g. to disallow foreach 
 over an array with a non-const `ref` index - related to:
 https://issues.dlang.org/show_bug.cgi?id=24499
 
 ```d
 foreach (ref i, ref e; array) {
      // code that modifies `i`, causing skipped/repeated elements of 
 foreach
 ```
 Rewrite to:
 ```d
 for (size_t i; i < array.length; i++) {
      ref e = array[i];
      // code that modifies `i`, which is now less surprising
 ```
 
 Currently you'd have to write `ref e() => array[i];`, which is less 
 intuitive and is not the same if `array` is reassigned between calls to 
 `e`.


Currently the DIP only shows aliasing of variables, expressions like the 
above do not appear to be supported.
Apr 17
parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 17 April 2024 at 14:07:04 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 Currently the DIP only shows aliasing of variables, expressions 
 like the above do not appear to be supported.


If that was the case this DIP would be completely useless, as you 
could already do everything it enabled with `alias`.

I'm sure the intended restriction is the same as with `ref` 
function parameters: the expression must be an lvalue.
Apr 17
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 17 April 2024 at 14:29:17 UTC, Dukc wrote:

 If that was the case this DIP would be completely useless, as 
 you could already do everything it enabled with `alias`.


I currently sometimes use pointers to alias members.

```D
// Inside struct LightOptimizer
void moveLightGlobal()
{
     auto light = &this.scene.light;
     light.pos += vec3(...);
}

void onMouseclick(ref GameState gs)
{
     auto ui = &gs.ui;
     ui.startDrag();
     ...
}
```

I'd prefer using `ref` since that's safer and enables opIndex 
overloads, so this DIP seems like a nice addition to me. I tried 
replacing those cases with alias but it errors:
```
Error: accessing non-static variable `scene` requires an instance 
of `LightOptimizer`
```
Apr 17
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 17 April 2024 at 16:51:41 UTC, Dennis wrote:

 On Wednesday, 17 April 2024 at 14:29:17 UTC, Dukc wrote:

 If that was the case this DIP would be completely useless, as 
 you could already do everything it enabled with `alias`.


 I currently sometimes use pointers to alias members.


[...]

 I'd prefer using `ref` since that's safer and enables opIndex 
 overloads, so this DIP seems like a nice addition to me. I 
 tried replacing those cases with alias but it errors:


This isn't particularly hard to do with library code:

```d
struct Ref(T)
{
     T* ptr;

     ref inout(T) deref() inout
     {
         return *ptr;
     }
     alias deref this;
}

Ref!T byRef(T)(return ref T lvalue)
{
     return Ref!T(&lvalue);
}

// Example usage:

struct Light
{
     double pos;
}

struct Scene
{
     Light light;
}

struct LightOptimizer
{
     Scene scene;

     void moveLightGlobal()
     {
         auto light = this.scene.light.byRef;
         light.pos += 1.0;
     }
}
```
Apr 18
prev sibling parent Dukc <ajieskola gmail.com> writes:
On Wednesday, 17 April 2024 at 16:51:41 UTC, Dennis wrote:

 On Wednesday, 17 April 2024 at 14:29:17 UTC, Dukc wrote:

 If that was the case this DIP would be completely useless, as 
 you could already do everything it enabled with `alias`.


 I currently sometimes use pointers to alias members.

 ```D
 // Inside struct LightOptimizer
 void moveLightGlobal()
 {
     auto light = &this.scene.light;
     light.pos += vec3(...);
 }

 void onMouseclick(ref GameState gs)
 {
     auto ui = &gs.ui;
     ui.startDrag();
     ...
 }
 ```

 I'd prefer using `ref` since that's safer and enables opIndex 
 overloads, so this DIP seems like a nice addition to me.


Yes I agree. I was just responding to the notion that the DIP 
would only allow referring to variables, not expressions. To me 
it doesn't seem to say that, and I'm sure the intended meaning is 
that expressions are fine as long as they are lvalues.


 I tried replacing those cases with alias but it errors:
 ```
 Error: accessing non-static variable `scene` requires an 
 instance of `LightOptimizer`
 ```


Well this can be arguably seen as a "variable" as opposed to 
"expression", in which case I was wrong - the DIP would be of 
some (but much lesser) use even if it really allowed only 
references to variables. But no one is advocating for that 
restriction so it doesn't matter. At maximum the DIP writing is 
ambiguous on this matter.
Apr 19
prev sibling parent reply Nick Treleaven <nick geany.org> writes:
On Wednesday, 17 April 2024 at 14:29:17 UTC, Dukc wrote:

 On Wednesday, 17 April 2024 at 14:07:04 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:

 Currently the DIP only shows aliasing of variables, 
 expressions like the above do not appear to be supported.


 If that was the case this DIP would be completely useless, as 
 you could already do everything it enabled with `alias`.

 I'm sure the intended restriction is the same as with `ref` 
 function parameters: the expression must be an lvalue.


The behaviour could perhaps be defined as follows - if this would 
compile:
```d
(ref p) { CODE }(lvalue);
```
Then, with some restrictions on `CODE`, you can write:
```d
ref p = lvalue;
CODE
```
And the effects will be equivalent. `CODE` cannot contain 
`return` statements or `__traits(parameters)`, 
`__traits(parent)`, `is(T PS == __parameters)` expressions. Have 
I missed anything else?
Apr 19
parent Dukc <ajieskola gmail.com> writes:
On Friday, 19 April 2024 at 07:23:22 UTC, Nick Treleaven wrote:

 The behaviour could perhaps be defined as follows - if this 
 would compile:
 ```d
 (ref p) { CODE }(lvalue);
 ```
 Then, with some restrictions on `CODE`, you can write:
 ```d
 ref p = lvalue;
 CODE
 ```
 And the effects will be equivalent. `CODE` cannot contain 
 `return` statements or `__traits(parameters)`, 
 `__traits(parent)`, `is(T PS == __parameters)` expressions. 
 Have I missed anything else?


This formulation works IMO but it's better to just write the 
expression used to initialise the `ref` variable must be an 
lvalue, lvalue meaning a value with a memory address. The spec 
already uses "lvalue" and "rvalue" and includes them in the 
glossary so it's not a problem.
Apr 19
prev sibling next sibling parent reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Friday, 12 April 2024 at 20:43:50 UTC, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


Why only `ref`?

`ref` on local variables is much more like `ref` on a parameter 
than a returned object. For parameters, `ref` is one of three: 
The others are `in` and `out`. Why not also include `in`? It 
seems unrelated, but it’s only slightly more general.

An `in` parameter is `const scope`, can bind rvalues, and is 
`ref` unless the type is small, then it’s a copy. That’s nothing 
a local variable couldn’t be and the “copy-if-small” semantic 
isn’t even easily mimicked.

As for `out` I had some ideas, but those are indeed unrelated 
with the DIP. (An `out` local variable would have to be 
uninitialized and have to be *initialized* (not assigned) before 
it is read. The most compelling use case is passing such a 
variable to a function `out` parameter.)
Apr 26
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/26/2024 2:52 AM, Quirin Schroll wrote:

 Why only `ref`?


We can add more later if compelling use cases come up. Let's start with just 
`ref`. You're welcome to make more enhancement proposals.
May 01
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/12/2024 1:43 PM, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


Implementation:

https://github.com/dlang/dmd/pull/16428
Apr 30
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 4/12/24 22:43, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


I am not sure how, but I had missed this thread so far. I am very much 
in favor, I have wished for this many times over the years. I think the 
only risk is implementation bugs in safety checks. The DIP seems a bit 
incomplete in this department, e.g., it does not say what happens when 
you take the address of a `ref` local. Maybe it can point to some prior 
art, but just stating that it cannot be returned seems neither 
sufficient nor necessary.

(Of course, it would be even better if fields could also be `ref`, but 
then you get into initialization safety. This is a general soundness 
problem in the current language though.)
May 01
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/1/2024 2:10 PM, Timon Gehr wrote:

 I am not sure how, but I had missed this thread so far. I am very much in
favor, 
 I have wished for this many times over the years. I think the only risk is 
 implementation bugs in safety checks. The DIP seems a bit incomplete in this 
 department, e.g., it does not say what happens when you take the address of a 
 `ref` local. Maybe it can point to some prior art, but just stating that it 
 cannot be returned seems neither sufficient nor necessary.


The same thing happens when you take the address of a ref parameter - you get 
the address of what the ref refers to.

I did my best to make it match exactly what the existing semantics of `ref`
are. 
If I made a mistake, we can fix it when we find it.



 (Of course, it would be even better if fields could also be `ref`, but then
you 
 get into initialization safety. This is a general soundness problem in the 
 current language though.)


`ref` is not rebindable, so a `ref` field means that support is needed in the 
constructor or default initializer, which needs some investigation on how to do 
it right.
May 01
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 5/2/24 08:51, Walter Bright wrote:

 (Of course, it would be even better if fields could also be `ref`, but 
 then you get into initialization safety. This is a general soundness 
 problem in the current language though.)


 
 `ref` is not rebindable, so a `ref` field means that support is needed 
 in the constructor or default initializer, which needs some 
 investigation on how to do it right.


Yes, exactly. My point was we need to think about that anyway. There are 
other things that ought not be rebindable:

```d
 safe:
class S{
     immutable int x;
     this(int y){
         foo();
         x=y;
         foo();
     }
     static int[immutable(int)*] t;
     void foo(){
         if(&x in t) assert(t[&x]==x); // error, immutable data modified
         t[&x]=x;
     }
}

void main(){
     auto s=new S(2);
}
```

Of course, with `ref` there is an additional challenge: it cannot be 
default-initialized.
May 05
prev sibling parent reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Wednesday, 1 May 2024 at 21:10:54 UTC, Timon Gehr wrote:

 On 4/12/24 22:43, Walter Bright wrote:

 https://github.com/WalterBright/documents/blob/984374ca885e1cb10c2667cf872aebc13b4c1663/varRef.md


 I think the only risk is implementation bugs in safety checks. 
 The DIP seems a bit incomplete in this department, e.g., it 
 does not say what happens when you take the address of a `ref` 
 local. Maybe it can point to some prior art, but just stating 
 that it cannot be returned seems neither sufficient nor 
 necessary.


Can’t `ref T reference` simply be defined to have the same 
semantics as `T* _reference` and `reference` is a shorthand for 
`*_reference`?


 (Of course, it would be even better if fields could also be 
 `ref`, but then you get into initialization safety. This is a 
 general soundness problem in the current language though.)


Actually, you don’t because `null` references are valid in D.
May 21
parent Mike Parker <aldacron gmail.com> writes:
Apologies, all. I should have closed this thread once the thread 
for the second draft was started. The DIP has now been submitted 
and the review process is closed. Please see the final post in 
the second draft thread for more information:

https://forum.dlang.org/post/hpflxpmlgsnpehtqxxhe forum.dlang.org
May 21
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I forgot to mention: ref's are not rebindable:

```
void test(int i, int j)
{
     ref int r = i; // r cannot be rebound to j
}
```
This has an interesting consequence. The lifetime of the ref will always be
less 
than the lifetime of what it was bound to.

```
ref int r = i;
{
     int j;
     r = j; // nope, cannot bind r to j
}
```

```
int i;
{
     ref int r = i; // r has shorter lifetime than i
}
```

which makes for more safety.
May 01
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 02/05/2024 6:58 PM, Walter Bright wrote:

 I forgot to mention: ref's are not rebindable:


And this is why we should expand upon alias instead.

```d
int adder(int a, int b) {
	alias Perform = a + b;
	return Perform;
}
```

This is far more expected as a feature (I have tried to do simple 
aliasing of variables with it in the past).

It basically offers lazy expression inlining.

Which could be quite cool.
May 02
parent Nick Treleaven <nick geany.org> writes:
On Thursday, 2 May 2024 at 07:06:34 UTC, Richard (Rikki) Andrew 
Cattermole wrote:

 ```d
 int adder(int a, int b) {
 	alias Perform = a + b;
 	return Perform;
 }
 ```

 This is far more expected as a feature (I have tried to do 
 simple aliasing of variables with it in the past).

 It basically offers lazy expression inlining.


```d
ref perform() => a + b;
return perform;
```
May 02
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 5/2/24 08:58, Walter Bright wrote:

 I forgot to mention: ref's are not rebindable:
 
 ```
 void test(int i, int j)
 {
      ref int r = i; // r cannot be rebound to j
 }
 ```
 This has an interesting consequence. The lifetime of the ref will always 
 be less than the lifetime of what it was bound to.
 


Well, not always, as not every lifetime in a program follows lexical 
scoping.

```d
Array!int a;
a ~= 1;
ref x = a[0];
a.length = 0;
// (`x` still alive here, but `a[0]` is not)
```

(But this is not ` safe` code.)
May 05