www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - aliasing expressions and identifiers

reply deed <none none.none> writes:
As I've been finding myself constantly desiring to alias 
expressions and identifiers for many years, I'd like to ask again 
whether it would be possible to extend either the alias-statement 
or the with-statement. It would be great to relieve the constant 
tension between descriptiveness, (often outside your control), 
and brevity in a straightforward manner.

There is something missing when your fields and parameters start 
out with proper names but end up as cryptic abbreviations when 
you start using them. Sometimes you want to look at an identifier 
and understand its purpose, other times you want to look at a 
bunch of expressions and statements and understand their pattern. 
I think an expressive language should support both needs without 
tension. It would also isolate names and reduce editing when 
names change.

Some thoughts about extending the with-statement were brought up 
here earlier:
http://forum.dlang.org/post/txpifmwpmmhsvcpbcijw forum.dlang.org
I don't care much whether it would be with, alias or possibly 
something clever already existing, but a solution should be easy 
to use, easy to read and shouldn't introduce any possible 
overhead at runtime.

1) Are there any technical blockers for extending the alias- or 
with-statement?
2) Does it carry its own weight?
May 23 2016
parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Monday, 23 May 2016 at 14:05:43 UTC, deed wrote:

 Some thoughts about extending the with-statement were brought 
 up here earlier:
 http://forum.dlang.org/post/txpifmwpmmhsvcpbcijw forum.dlang.org
 I don't care much whether it would be with, alias or possibly 
 something clever already existing, but a solution should be 
 easy to use, easy to read and shouldn't introduce any possible 
 overhead at runtime.


 From the linked thread:


        ref M()   { return matrix.rawArr; }
        ref Ex1() { return e1.someProperties.someModulusX; }


If we had local refs, we could use this instead:

ref m = matrix.rawArr;

The difference is m is already computed, it is not recomputed 
each time m is read, unlike M(). I think the reason D doesn't 
support local refs is because it would make it harder to design 
 safe, particularly with the planned  rc ref-counting. Because 
M() above is only a return reference, it can't live longer than 
the data it references. My ref m could persist longer than 
matrix.rawArr using (naive) reference counting.

BTW local refs are possible in  system code:
http://forum.dlang.org/post/lmuokynffgljzvrpvkvd forum.dlang.org
May 23 2016
next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Monday, 23 May 2016 at 15:18:51 UTC, Nick Treleaven wrote:

 On Monday, 23 May 2016 at 14:05:43 UTC, deed wrote:

 Some thoughts about extending the with-statement were brought 
 up here earlier:
 http://forum.dlang.org/post/txpifmwpmmhsvcpbcijw forum.dlang.org
 I don't care much whether it would be with, alias or possibly 
 something clever already existing, but a solution should be 
 easy to use, easy to read and shouldn't introduce any possible 
 overhead at runtime.


 From the linked thread:


        ref M()   { return matrix.rawArr; }
        ref Ex1() { return e1.someProperties.someModulusX; }


 If we had local refs, we could use this instead:

 ref m = matrix.rawArr;


Note that this wouldn't work with rvalues, which `with` supports.


 The difference is m is already computed, it is not recomputed 
 each time m is read, unlike M(). I think the reason D doesn't 
 support local refs is because it would make it harder to design 
  safe, particularly with the planned  rc ref-counting. Because 
 M() above is only a return reference, it can't live longer than 
 the data it references. My ref m could persist longer than 
 matrix.rawArr using (naive) reference counting.


At some point during the `scope` discussion, Walter wanted to 
allow local `ref`s, so I guess he's not opposed to the idea. As 
far as I understand, the compiler already supports them 
internally, as they can result from lowering certain constructs, 
there's just no syntax for them. They wouldn't pose a problem for 
lifetime tracking, because they can never be assigned to, only 
initialized once.
May 23 2016
parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Monday, 23 May 2016 at 17:03:32 UTC, Marc Schütz wrote:

 On Monday, 23 May 2016 at 15:18:51 UTC, Nick Treleaven wrote:

 If we had local refs, we could use this instead:

 ref m = matrix.rawArr;


 Note that this wouldn't work with rvalues, which `with` 
 supports.


OK. I suppose supporting local refs is a good reason not to allow 
rvalues to be passed as const ref arguments, if the function also 
returns by ref.


 I think the reason D doesn't support local refs is because it 
 would make it harder to design  safe, particularly with the 
 planned  rc ref-counting. Because M() above is only a return 
 reference, it can't live longer than the data it references. 
 My ref m could persist longer than matrix.rawArr using (naive) 
 reference counting.


 At some point during the `scope` discussion, Walter wanted to 
 allow local `ref`s, so I guess he's not opposed to the idea. As


Great :-)


 far as I understand, the compiler already supports them 
 internally, as they can result from lowering certain 
 constructs, there's just no syntax for them. They wouldn't pose 
 a problem for lifetime tracking, because they can never be 
 assigned to, only initialized once.


What about:

 safe unittest
{
	RCArray!int arr;
	ref r = arr[0];
	arr.destroy; // refcount drops to zero, arr.impl memory freed
	r++; // writes to unallocated memory
}
May 24 2016
next sibling parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On 24/05/2016 14:48, Nick Treleaven wrote:

 On Monday, 23 May 2016 at 17:03:32 UTC, Marc Schütz wrote:

 On Monday, 23 May 2016 at 15:18:51 UTC, Nick Treleaven wrote:

 I think the reason D doesn't support local refs is because it would
 make it harder to design  safe, particularly with the planned  rc
 ref-counting.


 They wouldn't pose a problem for lifetime tracking,
 because they can never be assigned to, only initialized once.


 What about:

  safe unittest
 {
      RCArray!int arr;


+      arr.length = 1;

      ref r = arr[0];
      arr.destroy; // refcount drops to zero, arr.impl memory freed
      r++; // writes to unallocated memory
 }


Here I think local refs must be prevented from initialization by return 
ref (-dip25). The  rc DIP and RCArray would use return ref. It would be 
OK to initialize a local ref with a ref function result that is not 
return ref. Naturally, this would be  safe:

auto slice = [7];
ref r = slice[0];
slice.destroy;
r++; // slice memory still allocated

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
May 25 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Wednesday, 25 May 2016 at 19:47:06 UTC, Nick Treleaven wrote:

 On 24/05/2016 14:48, Nick Treleaven wrote:

 What about:

  safe unittest
 {
      RCArray!int arr;


 +      arr.length = 1;

      ref r = arr[0];
      arr.destroy; // refcount drops to zero, arr.impl memory 
 freed
      r++; // writes to unallocated memory
 }


 Here I think local refs must be prevented from initialization 
 by return ref (-dip25). The  rc DIP and RCArray would use 
 return ref. It would be OK to initialize a local ref with a ref 
 function result that is not return ref. Naturally, this would 
 be  safe:

 auto slice = [7];
 ref r = slice[0];
 slice.destroy;
 r++; // slice memory still allocated


To elaborate: neither `scope` nor reference counting can ever 
protect you against explicit premature destruction of a 
still-referenced owner. But there is a slightly different 
problematic scenario:

RCArray!int arr = [7];
ref r = arr[0];
arr = [9];        // this releases the old array
r++;              // use after free

But this issue exists even without locale `ref`s:

void foo() {
     RCArray!int arr = [7];
     bar(arr, arr[0]);
}

void bar(ref RCArray!int arr, ref int r) {
     arr = [9];    // this releases the old array
     r++;          // use after free
}

It can be solved in one of two ways: Either by making the owner 
(`arr`) non-mutable during the existence of the references, 
thereby forbidding the call to `bar()` (I would prefer this one, 
as it's cleaner and can be used for many more things, e.g. the 
byLine problem), or by making the owner live longer by inserting 
the appropriate AddRef/Release pairs whenever such a situation 
arises.
May 26 2016
parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Thursday, 26 May 2016 at 08:29:41 UTC, Marc Schütz wrote:

 On Wednesday, 25 May 2016 at 19:47:06 UTC, Nick Treleaven wrote:

 On 24/05/2016 14:48, Nick Treleaven wrote:

  safe unittest
 {
      RCArray!int arr;


 +      arr.length = 1;

      ref r = arr[0];
      arr.destroy; // refcount drops to zero, arr.impl memory 
 freed
      r++; // writes to unallocated memory
 }


 Here I think local refs must be prevented from initialization 
 by return ref (-dip25). The  rc DIP and RCArray would use 
 return ref. It would be OK to initialize a local ref with a 
 ref function result that is not return ref.
 ...


 To elaborate: neither `scope` nor reference counting can ever 
 protect you against explicit premature destruction of a 
 still-referenced owner. But there is a slightly different 
 problematic scenario:

 RCArray!int arr = [7];
 ref r = arr[0];
 arr = [9];        // this releases the old array
 r++;              // use after free


This is the same situation. There's nothing special about 
destroy, it just assigns arr = arr.init. destroy is  safe so in 
fact it must work with safe RC. You seem to have ignored my 
suggestion (which maybe wasn't clear enough) to statically 
prevent the above from compiling using:

RCArray(T) {
     ...
     ref opIndex(size_t) return;

Local refs cannot be assigned from a function returning ref if 
that function has any parameters marked with the return 
attribute. If there is no attribute, local refs + function 
returning ref is OK.


 But this issue exists even without locale `ref`s:

 void foo() {
     RCArray!int arr = [7];
     bar(arr, arr[0]);
 }

 void bar(ref RCArray!int arr, ref int r) {
     arr = [9];    // this releases the old array
     r++;          // use after free
 }


This is the reason for the  rc DIP.


 It can be solved in one of two ways: Either by making the owner 
 (`arr`) non-mutable during the existence of the references, 
 thereby forbidding the call to `bar()` (I would prefer this 
 one, as it's cleaner and can be used for many more things, e.g. 
 the byLine problem)


I don't see directly how this affects byLine.front, that does not 
return a reference.
May 27 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Friday, 27 May 2016 at 10:04:14 UTC, Nick Treleaven wrote:

 On Thursday, 26 May 2016 at 08:29:41 UTC, Marc Schütz wrote:

 To elaborate: neither `scope` nor reference counting can ever 
 protect you against explicit premature destruction of a 
 still-referenced owner. But there is a slightly different 
 problematic scenario:

 RCArray!int arr = [7];
 ref r = arr[0];
 arr = [9];        // this releases the old array
 r++;              // use after free


 This is the same situation. There's nothing special about 
 destroy, it just assigns arr = arr.init. destroy is  safe so in 
 fact it must work with safe RC.


If this is what destroy does, then yes, that's ok. But it is a 
misleading name then, IMO.


 You seem to have ignored my suggestion (which maybe wasn't 
 clear enough) to statically prevent the above from compiling 
 using:

 RCArray(T) {
     ...
     ref opIndex(size_t) return;

 Local refs cannot be assigned from a function returning ref if 
 that function has any parameters marked with the return 
 attribute. If there is no attribute, local refs + function 
 returning ref is OK.


Huh? `return` means that the returned reference is owned by the 
RCArray struct and must therefore not outlive it. If the RCArray 
is a local variable (or parameter), the local ref is always 
declared after it (because it must be initialized immediately), 
and will have a shorter scope than the RCArray. Therefore, such 
an assignment is always accepted.


 But this issue exists even without locale `ref`s:

 void foo() {
     RCArray!int arr = [7];
     bar(arr, arr[0]);
 }

 void bar(ref RCArray!int arr, ref int r) {
     arr = [9];    // this releases the old array
     r++;          // use after free
 }


 This is the reason for the  rc DIP.


 It can be solved in one of two ways: Either by making the 
 owner (`arr`) non-mutable during the existence of the 
 references, thereby forbidding the call to `bar()` (I would 
 prefer this one, as it's cleaner and can be used for many more 
 things, e.g. the byLine problem)


 I don't see directly how this affects byLine.front, that does 
 not return a reference.


It returns a reference in the wider sense, namely a slice to a 
private buffer that gets overwritten by each call to `byLine`. 
Currently, DIP25 only applies to `ref`s in the narrow sense, but 
I'm assuming it will be generalized to include pointer, slices, 
class references, AAs and hidden context pointers.

Making the ByLine range constant as long as there's a reference 
to its buffer would prevent surprises like this:

     auto lines = stdin.byLine.array;
     // => all elements of `lines` are the same, should have used 
`byLineCopy`

By the way, there's a theoretical third possibility: 
Force-invalidating the references after the owner has 
(potentially) been mutated. But this would require very advanced 
data flow tracking and is probably impossible to implement in the 
general case.
May 27 2016
parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Friday, 27 May 2016 at 11:50:42 UTC, Marc Schütz wrote:

 On Friday, 27 May 2016 at 10:04:14 UTC, Nick Treleaven wrote:

 On Thursday, 26 May 2016 at 08:29:41 UTC, Marc Schütz wrote:

 RCArray!int arr = [7];
 ref r = arr[0];
 arr = [9];        // this releases the old array
 r++;              // use after free






...

 statically prevent the above from compiling using:

 RCArray(T) {
     ...
     ref opIndex(size_t) return;

 Local refs cannot be assigned from a function returning ref if 
 that function has any parameters marked with the return 
 attribute. If there is no attribute, local refs + function 
 returning ref is OK.


 Huh? `return` means that the returned reference is owned by the 
 RCArray struct and must therefore not outlive it. If the 
 RCArray is a local variable (or parameter), the local ref is 
 always declared after it (because it must be initialized 
 immediately), and will have a shorter scope than the RCArray. 
 Therefore, such an assignment is always accepted.


What about if the RCArray (of ref count 1) is assigned to a 
different one after the local ref is initialised? That is what 
we're discussing -it's your example above(!)


 It can be solved in one of two ways: Either by making the 
 owner (`arr`) non-mutable during the existence of the 
 references, thereby forbidding the call to `bar()` (I would 
 prefer this one, as it's cleaner and can be used for many 
 more things, e.g. the byLine problem)


 I don't see directly how this affects byLine.front, that does 
 not return a reference.


 It returns a reference in the wider sense, namely a slice to a 
 private buffer that gets overwritten by each call to `byLine`. 
 Currently, DIP25 only applies to `ref`s in the narrow sense, 
 but I'm assuming it will be generalized to include pointer, 
 slices, class references, AAs and hidden context pointers.

 Making the ByLine range constant as long as there's a reference 
 to its buffer would prevent surprises like this:

     auto lines = stdin.byLine.array;
     // => all elements of `lines` are the same, should have 
 used `byLineCopy`


So your solution would statically prevent popFront because front 
has escaped. I think we should just prevent front from escaping.
May 29 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Sunday, 29 May 2016 at 14:27:51 UTC, Nick Treleaven wrote:

 What about if the RCArray (of ref count 1) is assigned to a 
 different one after the local ref is initialised? That is what 
 we're discussing -it's your example above(!)


Exactly, and then one of the two suggested approaches will have 
to be used to prevent the use-after-free. But that's something 
that needs to happen on assignment, not when the reference is 
created.


 It can be solved in one of two ways: Either by making the 
 owner (`arr`) non-mutable during the existence of the 
 references, thereby forbidding the call to `bar()` (I would 
 prefer this one, as it's cleaner and can be used for many 
 more things, e.g. the byLine problem)


 I don't see directly how this affects byLine.front, that does 
 not return a reference.


 It returns a reference in the wider sense, namely a slice to a 
 private buffer that gets overwritten by each call to `byLine`. 
 Currently, DIP25 only applies to `ref`s in the narrow sense, 
 but I'm assuming it will be generalized to include pointer, 
 slices, class references, AAs and hidden context pointers.

 Making the ByLine range constant as long as there's a 
 reference to its buffer would prevent surprises like this:

     auto lines = stdin.byLine.array;
     // => all elements of `lines` are the same, should have 
 used `byLineCopy`


 So your solution would statically prevent popFront because 
 front has escaped.


Yes.


 I think we should just prevent front from escaping.


It doesn't necessarily need to escape for the problem to occur. 
Well, it does in this example, but it can be trivially rewritten:

     auto tmp = stdin.byLine;
     auto lines = tmp.array;

Here, `lines` contains references to the buffer owned by `tmp`, 
but doesn't escape (assuming `array` takes its argument by 
`scope` or however the final solution will look like).
May 30 2016
parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Monday, 30 May 2016 at 10:55:57 UTC, Marc Schütz wrote:

 On Sunday, 29 May 2016 at 14:27:51 UTC, Nick Treleaven wrote:

 What about if the RCArray (of ref count 1) is assigned to a 
 different one after the local ref is initialised? That is what 
 we're discussing -it's your example above(!)


 Exactly, and then one of the two suggested approaches will have 
 to be used to prevent the use-after-free. But that's something 
 that needs to happen on assignment, not when the reference is 
 created.


Well my solution does work, disallowing the problematic local 
refs. But it wouldn't be great for generic code or consistency 
with the  rc DIP. For the latter, we can add a temporary RC 
object to keep the referenced memory alive, but we only need to 
do this when the local ref is initialized from a function both 
(1) returning ref and (2) with a parameter marked return.


 I think we should just prevent front from escaping.


 It doesn't necessarily need to escape for the problem to occur. 
 Well, it does in this example, but it can be trivially 
 rewritten:

     auto tmp = stdin.byLine;
     auto lines = tmp.array;

 Here, `lines` contains references to the buffer owned by `tmp`, 
 but doesn't escape (assuming `array` takes its argument by 
 `scope` or however the final solution will look like).


tmp and stdin.byLine are of type ByLine, whose front could be 
scope/return to prevent escaping. Above array() does escape 
ByLine.front so can't mark its argument with scope - the compiler 
would error.
May 30 2016
parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Monday, 30 May 2016 at 14:18:38 UTC, Nick Treleaven wrote:

 On Monday, 30 May 2016 at 10:55:57 UTC, Marc Schütz wrote:

     auto tmp = stdin.byLine;
     auto lines = tmp.array;

 Here, `lines` contains references to the buffer owned by 
 `tmp`, but doesn't escape (assuming `array` takes its argument 
 by `scope` or however the final solution will look like).


 tmp and stdin.byLine are of type ByLine, whose front could be 
 scope/return to prevent escaping. Above array() does escape 
 ByLine.front so can't mark its argument with scope - the 
 compiler would error.


Sorry, I meant using array(scope R range) would not affect the 
compiler error when array attempts to escape range.front into its 
slice.
May 30 2016
prev sibling parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 24 May 2016 at 13:48:41 UTC, Nick Treleaven wrote:

 What about:

  safe unittest
 {
 	RCArray!int arr;
 	ref r = arr[0];
 	arr.destroy; // refcount drops to zero, arr.impl memory freed
 	r++; // writes to unallocated memory
 }


You're calling `destroy` explicitly, what else would you expect 
to happen?
May 26 2016
prev sibling parent reply deed <none none.none> writes:
On Monday, 23 May 2016 at 15:18:51 UTC, Nick Treleaven wrote:

 If we had local refs, we could use this instead:

 ref m = matrix.rawArr;

 The difference is m is already computed, it is not recomputed 
 each time m is read, unlike M(). I think the reason D doesn't 
 support local refs is because it would make it harder to design 
  safe, particularly with the planned  rc ref-counting. Because 
 M() above is only a return reference, it can't live longer than 
 the data it references. My ref m could persist longer than 
 matrix.rawArr using (naive) reference counting.

 BTW local refs are possible in  system code:
 http://forum.dlang.org/post/lmuokynffgljzvrpvkvd forum.dlang.org


Just to be clear: I'm not looking for anything touching the 
semantics, just a simple replacement mechanism/macro expansion. 
Recomputation or not,  safe or  system, should be no different 
from the expanded code. So within a scope, after aliasing 'm', 
'm' should be replaced by 'matrix.rawArr'. Everything stays the 
same as is, even error messages.

{
   // Some scope

   alias i = instance.targetIdx;
   alias m = matrix.rawArr;

   m[i] = m[j] + m[k];
   // detected and conceptually replaced in one of the earlier 
compiler passes by
   // matrix.rawArr[instance.targetIdx] = matrix.rawArr[j] + 
matrix.rawArr[k]
}
May 23 2016
parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Monday, 23 May 2016 at 17:43:49 UTC, deed wrote:

 On Monday, 23 May 2016 at 15:18:51 UTC, Nick Treleaven wrote:
 Recomputation or not,  safe or  system, should be no different 
 from the expanded code. So within a scope, after aliasing 'm', 
 'm' should be replaced by 'matrix.rawArr'. Everything stays the 
 same as is, even error messages.


I think it's more useful to support local refs than alias 
expressions. In fact apart from the rvalue support, I would 
rather use a local ref than the existing with statement (assuming 
the ref was memory safe). The only exception is when using `with 
(EnumType)`.

Also, compile-time expressions use enum, not alias. It could be 
changed, but I think it's useful when reading meta-programming 
code to clearly see the difference.

Currently aliased symbols are expanded by the compiler in error 
messages, but that's probably not ideal.
May 24 2016