www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Safe Regions with Deterministic Destruction

reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
This is an interesting read:

https://cyclone.thelanguage.org/wiki/Region%20Common%20Uses/

The typical use case I see is a tree `T` that `emplace`s all its 
nodes in a region `R` where `R`'s lifetime and, in turn, `T`s 
nodes are all deterministically bound to `T`s lifetime.

Can such a tree be written today with the help of DIP-1000 and 
` live`?

That provides  safe `scope`d access to its nodes in -dip1000.

And would a potential solution differ in implementation depending 
on whether the node is a value (POD or `struct`) or a reference 
type (`class` or pointer)?
Oct 26 2020
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 26 October 2020 at 14:27:49 UTC, Per Nordlöw wrote:

 The typical use case I see is a tree `T` that `emplace`s all 
 its nodes in a region `R` where `R`'s lifetime and, in turn, 
 `T`s nodes are all deterministically bound to `T`s lifetime.


Here's the essence of what I had in mind:

struct Tree(Node) if (is(Node == class))
{
 safe:
     Node root() return scope pure nothrow  nogc { return _root; }
     this(uint dummy)  trusted
     {
         import core.lifetime : emplace;
         _root = emplace!Node(_store);
     }
     private Node _root;
     private void[__traits(classInstanceSize, Node)] _store; // 
replaced with a region
}

class C { this() {} int x; }

 safe pure unittest
{
     C f() {       Tree!C t; return t.root; } // should error?
     C g() { scope Tree!C t; return t.root; } // errors
}

What's the reasoning behind disallowing g() but not f()? Lack of 
`scope`-inference in the compiler? Calling f() will result in a 
memory error aswell.
Oct 26 2020
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 26 October 2020 at 19:47:57 UTC, Per Nordlöw wrote:

 What's the reasoning behind disallowing g() but not f()? Lack 
 of `scope`-inference in the compiler? Calling f() will result 
 in a memory error aswell.


Reduced:

class Node {}

struct Tree
{
     Node root;
}

 safe unittest
{
     Node f() { auto  t = Tree(); return t.root; } // shouldn't 
this error aswell?
     Node g() { scope t = Tree(); return t.root; } // errors
}
Oct 26 2020
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 26 October 2020 at 21:10:21 UTC, Per Nordlöw wrote:

 On Monday, 26 October 2020 at 19:47:57 UTC, Per Nordlöw wrote:
     Node f() { auto  t = Tree(); return t.root; } // shouldn't 
 this error aswell?
     Node g() { scope t = Tree(); return t.root; } // errors


Node is assumed to be gc allocated, but scope is transitive?
It would be easier to reason about if D had a gcpointer type.
Oct 26 2020
parent reply ag0aep6g <anonymous example.com> writes:
On 26.10.20 22:32, Ola Fosheim Grøstad wrote:

 On Monday, 26 October 2020 at 21:10:21 UTC, Per Nordlöw wrote:

 On Monday, 26 October 2020 at 19:47:57 UTC, Per Nordlöw wrote:
     Node f() { auto  t = Tree(); return t.root; } // shouldn't this 
 error aswell?
     Node g() { scope t = Tree(); return t.root; } // errors


 
 Node is assumed to be gc allocated, but scope is transitive?


`scope` isn't transitive. It just applies to the pointers in the 
variable (i.e. `t.root`).

It doesn't apply to the location of the variable, because the compiler 
already knows that references to locals must not escape.
Oct 26 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 27 October 2020 at 06:16:11 UTC, ag0aep6g wrote:

 `scope` isn't transitive. It just applies to the pointers in 
 the variable (i.e. `t.root`).


But technically the t.root pointer address is a value, so 
apparently scope does not apply only to t, but also to the member 
root?

So if Node was a linked list, then t.root is limited by escape 
analysis, but not t.root.next?

I don't think t.root should be limited by scope.
Oct 27 2020
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 27 October 2020 at 07:07:49 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 27 October 2020 at 06:16:11 UTC, ag0aep6g wrote:

 `scope` isn't transitive. It just applies to the pointers in 
 the variable (i.e. `t.root`).


 But technically the t.root pointer address is a value, so 
 apparently scope does not apply only to t, but also to the 
 member root?


Ok, I guess this is to allow scope limiting of custom smart 
pointers...

Maybe it would be better to have some way of telling the compiler 
that something is a smart pointer.
Oct 27 2020
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On 27.10.20 08:07, Ola Fosheim Grøstad wrote:

 But technically the t.root pointer address is a value, so apparently 
 scope does not apply only to t, but also to the member root?


I'm not sure what you mean by "is a value". `t.root` is a class 
reference, which (to `scope`) is a pointer with a fancy name. It being 
an indirection is what makes it interesting. `scope` doesn't do anything 
for non-indirections.

But yes, `scope` applies to `t.root`, because it's the first level of 
indirection in `t`. If it was `t.x.y.z.root`, with `x`, `y`, and `z` 
being more structs, `scope` would still apply to `root`. But if `x`, 
`y`, or `z` were some kind of indirection, `scope` would apply to that 
and not to `root`.

`scope` doesn't work on types. It works on indirections. `scope` sees 
through types until it finds an indirection. And then it stops. If you 
have two or more levels of indirections, `scope` still only affects the 
first.

An example in code:

----
struct A
{
     int* pa;
     B b;
}
struct B
{
     int* pb;
     C* c;
}
struct C
{
     int* pc;
     int x;
}
int* f(scope A a)  safe
{
     return a.pa; /* error; can't return a `scope` pointer */
     return a.b.pb; /* error; still covered by `scope` */
     return a.b.c.pc; /* ok; `scope` is not transitive */
}
----

If you replace `scope` with `const`, all three lines fail, because 
`const` is transitive.

(Note: `ref` is handled differently from other forms of indirection.)


 So if Node was a linked list, then t.root is limited by escape analysis, 
 but not t.root.next?


Exactly.
Oct 27 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 27 October 2020 at 15:33:08 UTC, ag0aep6g wrote:

 On 27.10.20 08:07, Ola Fosheim Grøstad wrote:

 But technically the t.root pointer address is a value, so 
 apparently scope does not apply only to t, but also to the 
 member root?


 I'm not sure what you mean by "is a value". `t.root` is a class 
 reference, which (to `scope`) is a pointer with a fancy name. 
 It being an indirection is what makes it interesting. `scope` 
 doesn't do anything for non-indirections.


Yes, what I meant is that it is a value on the stack. But there 
is a disconnect with classes. "oddball" passes as a class, even 
though it would not pass as a struct:

class A { int x; }
class B { A a; }

A oddball() {
     scope B b = new B;
     b.a = new A;
     return b.a;
}


But WHY does this pass?

class A { int x; }
struct B { A a; }

A misbehave()  safe {
     scope A tmp = new A;
     scope B b;
     b.a = tmp;
     return b.a;
}


 But yes, `scope` applies to `t.root`, because it's the first 
 level of indirection in `t`. If it was `t.x.y.z.root`, with 
 `x`, `y`, and `z` being more structs, `scope` would still apply 
 to `root`. But if `x`, `y`, or `z` were some kind of 
 indirection, `scope` would apply to that and not to `root`.


I understand. So for structs it works the same way as 
scope-parameters?


 then it stops. If you have two or more levels of indirections, 
 `scope` still only affects the first.


So you basically should just apply scope to pointers and smart 
pointers.


 So if Node was a linked list, then t.root is limited by escape 
 analysis, but not t.root.next?


 Exactly.


I wonder how a transitive scope would play out?
Oct 27 2020
parent reply ag0aep6g <anonymous example.com> writes:
On Tuesday, 27 October 2020 at 16:56:40 UTC, Ola Fosheim Grøstad 
wrote:

 Yes, what I meant is that it is a value on the stack. But there 
 is a disconnect with classes. "oddball" passes as a class, even 
 though it would not pass as a struct:

 class A { int x; }
 class B { A a; }

 A oddball() {
     scope B b = new B;
     b.a = new A;
     return b.a;
 }


`oddball` is neither a class nor a struct; it's a function.

If you change both `A` and `B` to structs and update the code to 
`A*` and `B*` where needed, then it works the same as with 
classes. If you change them to structs but don't use pointers, 
then it works differently because you're removing indirections.


 But WHY does this pass?

 class A { int x; }
 struct B { A a; }

 A misbehave()  safe {
     scope A tmp = new A;
     scope B b;
     b.a = tmp;
     return b.a;
 }


It doesn't? The last line gives "Error: scope variable `b` may 
not be returned". Remember to compile with `-preview=dip1000`.

[...]

 I understand. So for structs it works the same way as 
 scope-parameters?


I'm not sure if I understand the question, but as far as I 
understand `scope` locals and `scope` parameters work the same 
for all types. (Maybe there are subtle differences in life time. 
Maybe there are bugs. But basically they're the same thing.)
Oct 27 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 27 October 2020 at 18:24:47 UTC, ag0aep6g wrote:

 On Tuesday, 27 October 2020 at 16:56:40 UTC, Ola Fosheim 
 Grøstad wrote:

 Yes, what I meant is that it is a value on the stack. But 
 there is a disconnect with classes. "oddball" passes as a 
 class, even though it would not pass as a struct:

 class A { int x; }
 class B { A a; }

 A oddball() {
     scope B b = new B;
     b.a = new A;
     return b.a;
 }


 `oddball` is neither a class nor a struct; it's a function.


Yes, obviously, but it it passes as a class.


 If you change both `A` and `B` to structs and update the code 
 to `A*` and `B*` where needed, then it works the same as with 
 classes.


Well, but "scope B b = new B;" will stack allocate, right? So it 
is allocated on the stack like a struct. I suspect a user would 
expect b.a to be restricted as it would have been as a struct.

But yes, I get the "one indirection" thing you mentioned. The 
question is, is that useful enough? Is that something users will 
understand?



 It doesn't? The last line gives "Error: scope variable `b` may 
 not be returned". Remember to compile with `-preview=dip1000`.


You are right, the compile switch was missing.


 I'm not sure if I understand the question, but as far as I 
 understand `scope` locals and `scope` parameters work the same 
 for all types. (Maybe there are subtle differences in life


Except that "scope B b = new B" stack allocates? Or does that 
happen for parameters too? Does this cause stack allocation of B?

   f(scope B b){…}

   f(new B);
Oct 27 2020
parent reply ag0aep6g <anonymous example.com> writes:
On Tuesday, 27 October 2020 at 18:35:40 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 27 October 2020 at 18:24:47 UTC, ag0aep6g wrote:

 On Tuesday, 27 October 2020 at 16:56:40 UTC, Ola Fosheim 
 Grøstad wrote:




[...]

 class A { int x; }
 class B { A a; }

 A oddball() {
     scope B b = new B;
     b.a = new A;
     return b.a;
 }


 `oddball` is neither a class nor a struct; it's a function.


 Yes, obviously, but it it passes as a class.


I didn't understand what you mean by "passing as a class". I 
guess you mean that it returns a class.

[...]

 Well, but "scope B b = new B;" will stack allocate, right? So 
 it is allocated on the stack like a struct. I suspect a user 
 would expect b.a to be restricted as it would have been as a 
 struct.


Ok, I see your point now. I wasn't thinking about `scope` as an 
indicator for stack allocation at all. As far as I'm aware that's 
just a possible optimization, not a guarantee. And it's somewhat 
independent from DIP 1000's `scope`.

If it becomes more than an optimization, then yeah it might make 
sense to not view `scope` class variables as indirections in that 
case. Or maybe it wouldn't be sound. I really have no idea.


 But yes, I get the "one indirection" thing you mentioned. The 
 question is, is that useful enough? Is that something users 
 will understand?


It's fairly obvious to me that `scope` is not easy to understand. 
The buggy implementation and insufficient documentation don't 
help. At the same time, I am myself not interested in finding a 
better solution to the problem.

[...]

 I'm not sure if I understand the question, but as far as I 
 understand `scope` locals and `scope` parameters work the same 
 for all types. (Maybe there are subtle differences in life


 Except that "scope B b = new B" stack allocates? Or does that 
 happen for parameters too? Does this cause stack allocation of 
 B?

   f(scope B b){…}

   f(new B);


I don't think so. You're right that stack allocation is a 
significant difference, but only if it is (or becomes) more than 
an optimization.
Oct 27 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 27 October 2020 at 19:13:28 UTC, ag0aep6g wrote:

 I didn't understand what you mean by "passing as a class". I 
 guess you mean that it returns a class.


Oh, bad wording on my part. I meant "dressed up as a class".


 Ok, I see your point now. I wasn't thinking about `scope` as an 
 indicator for stack allocation at all. As far as I'm aware 
 that's just a possible optimization, not a guarantee.


Ah! I thought it was a feature. :-)


 It's fairly obvious to me that `scope` is not easy to 
 understand. The buggy implementation and insufficient 
 documentation don't help. At the same time, I am myself not 
 interested in finding a better solution to the problem.


I think it is interesting to think about what the consequences 
would be if it was transitive. Basically: anything reached 
through this variable is scoped.
Oct 27 2020
prev sibling parent ag0aep6g <anonymous example.com> writes:
On 26.10.20 22:10, Per Nordlöw wrote:

 Reduced:
 
 class Node {}
 
 struct Tree
 {
      Node root;
 }
 
  safe unittest
 {
      Node f() { auto  t = Tree(); return t.root; } // shouldn't this 
 error aswell?
      Node g() { scope t = Tree(); return t.root; } // errors
 }


You've reduced that a bit too much. There's nothing unsafe left in the 
code. `g` only errors, because you're using `scope` to tell the compiler 
that `t.root` is dangerous (when it's not actually).
Oct 26 2020