www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - DIP1000

reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
How am I supposed to write this:

```d
import std;
 safe:

struct node {
     node* next;
}

auto connect(scope node* a, scope node* b)
{
     a.next = b;
}

void main()
{
     node x;
     node y;
     connect(&x,&y);
}

```


 Error: scope variable `b` assigned to non-scope `(*a).next`
Jun 23 2022
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On Thursday, 23 June 2022 at 16:08:01 UTC, Ola Fosheim Grøstad 
wrote:

 How am I supposed to write this:

 ```d
 import std;
  safe:

 struct node {
     node* next;
 }

 auto connect(scope node* a, scope node* b)
 {
     a.next = b;
 }

 void main()
 {
     node x;
     node y;
     connect(&x,&y);
 }

 ```


 Error: scope variable `b` assigned to non-scope `(*a).next`




DMD accepts this:

```d
 safe:

struct node {
     node* next;
}

void connect(ref scope node a, return scope node* b)
{
     a.next = b;
}

void main()
{
     node y;
     scope node x;
     connect(x, &y);
}
```

But that only works for this very special case. It falls apart 
when you try to add a third node. As far as I understand, `scope` 
cannot handle linked lists. A `scope` pointer cannot point to 
another `scope` pointer.

So as to how you're supposed to do it: with  system.
Jun 23 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 23 June 2022 at 19:38:12 UTC, ag0aep6g wrote:

 ```d
 void connect(ref scope node a, return scope node* b)
 ```


Thanks, so the `return scope` means «allow escape», not 
necessarily return?


 But that only works for this very special case. It falls apart 
 when you try to add a third node. As far as I understand, 
 `scope` cannot handle linked lists. A `scope` pointer cannot 
 point to another `scope` pointer.


One can do two levels, but not three. Got it. Works for some 
basic data-structures.
Jun 23 2022
parent reply ag0aep6g <anonymous example.com> writes:
On Thursday, 23 June 2022 at 20:27:44 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 23 June 2022 at 19:38:12 UTC, ag0aep6g wrote:

 ```d
 void connect(ref scope node a, return scope node* b)
 ```


 Thanks, so the `return scope` means «allow escape», not 
 necessarily return?


It means "may be returned or copied to the first parameter" 
(https://dlang.org/spec/function.html#param-storage). You cannot 
escape via other parameters. It's a weird rule for sure.
Jun 23 2022
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 23 June 2022 at 21:05:57 UTC, ag0aep6g wrote:

 It means "may be returned or copied to the first parameter" 
 (https://dlang.org/spec/function.html#param-storage). You 
 cannot escape via other parameters. It's a weird rule for sure.


Too complicated for what it does… Maybe  trusted isn't so bad 
after all.
Jun 23 2022
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 23 June 2022 at 21:05:57 UTC, ag0aep6g wrote:

 It's a weird rule for sure.


Another slightly annoying thing is that it cares about 
destruction order when there are no destructors.

If there are no destructors the lifetime ought to be considered 
the same for variables in the same scope.
Jun 23 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 23 June 2022 at 21:34:27 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 23 June 2022 at 21:05:57 UTC, ag0aep6g wrote:

 It's a weird rule for sure.


 Another slightly annoying thing is that it cares about 
 destruction order when there are no destructors.

 If there are no destructors the lifetime ought to be considered 
 the same for variables in the same scope.


Having different lifetime rules for different types is worse UX 
than having the same lifetime rules for all types.

Imagine writing a generic function which passes all of your unit 
tests, and then fails when you try to use it in real code, 
because you forgot to test it with a type that has a destructor.
Jun 23 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 24 June 2022 at 03:03:52 UTC, Paul Backus wrote:

 On Thursday, 23 June 2022 at 21:34:27 UTC, Ola Fosheim Grøstad 
 wrote:

 On Thursday, 23 June 2022 at 21:05:57 UTC, ag0aep6g wrote:

 It's a weird rule for sure.


 Another slightly annoying thing is that it cares about 
 destruction order when there are no destructors.

 If there are no destructors the lifetime ought to be 
 considered the same for variables in the same scope.


 Having different lifetime rules for different types is worse UX 
 than having the same lifetime rules for all types.

 Imagine writing a generic function which passes all of your 
 unit tests, and then fails when you try to use it in real code, 
 because you forgot to test it with a type that has a destructor.


No, the lifetime is the same if there is no destructor. Being 
counter intuitive is poor usability.

If you want to help library authors you issue a warning for 
generic code only.
Jun 23 2022
parent reply Dukc <ajieskola gmail.com> writes:
On Friday, 24 June 2022 at 05:11:13 UTC, Ola Fosheim Grøstad 
wrote:

 No, the lifetime is the same if there is no destructor. Being 
 counter intuitive is poor usability.


It depends on whether you expect the rules to be smart or simple. 
Smart is not necessarily better, as the Unix philosophy tells 
you. I'm sure you have experience about programs that are 
unpredictable and thus frustating to use because they try to be 
too smart.
Jun 24 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 24 June 2022 at 09:08:25 UTC, Dukc wrote:

 On Friday, 24 June 2022 at 05:11:13 UTC, Ola Fosheim Grøstad 
 wrote:

 No, the lifetime is the same if there is no destructor. Being 
 counter intuitive is poor usability.


 It depends on whether you expect the rules to be smart or 
 simple. Smart is not necessarily better, as the Unix philosophy 
 tells you. I'm sure you have experience about programs that are 
 unpredictable and thus frustating to use because they try to be 
 too smart.


If this feature is meant to be used by application developers and 
not only library authors then it has to match their intuitive 
mental model of life times. I would expect all simple value types 
to have the same lifetime as the scope.

The other option is to somehow instill a mental model in all 
users that simple types like ints also having default destructors.

If it only is for library authors, then it is ok to deviate from 
"intuition".
Jun 24 2022
prev sibling parent reply Loara <loara noreply.com> writes:
On Thursday, 23 June 2022 at 16:08:01 UTC, Ola Fosheim Grøstad 
wrote:

 How am I supposed to write this:

 ```d
 import std;
  safe:

 struct node {
     node* next;
 }

 auto connect(scope node* a, scope node* b)
 {
     a.next = b;
 }

 void main()
 {
     node x;
     node y;
     connect(&x,&y);
 }

 ```


 Error: scope variable `b` assigned to non-scope `(*a).next`




Why you should use `scope` here? A `scope` pointer variable may 
refer to a stack allocated object that may be destroyed once the 
function returns.

Since a linked list should not contain pointers to stack 
allocated data you should avoid entirely the `scope` attribute 
and use instead `const`.
Jun 24 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 24 June 2022 at 17:53:07 UTC, Loara wrote:

 Why you should use `scope` here?


I probably shouldn't. That is why I asked in the «learn» forum…


 A `scope` pointer variable may refer to a stack allocated 
 object that may be destroyed once the function returns.


The objects are in the calling function, not in the connect() 
function. So they are not destroyed when the connect() function 
returns.


 Since a linked list should not contain pointers to stack 
 allocated data you should avoid entirely the `scope` attribute 
 and use instead `const`.


It was only an example. There is nothing wrong with connecting 
objects on the stack.
Jun 24 2022
parent reply Loara <loara noreply.com> writes:
On Friday, 24 June 2022 at 18:31:14 UTC, Ola Fosheim Grøstad 
wrote:

 The objects are in the calling function, not in the connect() 
 function. So they are not destroyed when the connect() function 
 returns.


When `connect()` returns may happen that `b` is destroyed but `a` 
not, so `a.next` contains a dangling pointer that will bring 
potential segmentation faults that could be detected only with 
tools like Valgrind, just consider

```d
node * calling(return scope node * a)  safe{
   scope node *b = new node(); //b is stack allocated
   connect(a, b);
   return a;
   //b destroyed but a not
}
```

The `scope` attribute tries to avoid these events preventing you 
from doing something potentially dangerous with stack allocated 
objects, and assigning a `scope` pointer to a not-scoped variable 
(`a.next` is not `scope` since this attribute is not transitive) 
is clearly dangerous since `connect` doesn't know which between 
`a` and `b` terminates first.
Jun 28 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 28 June 2022 at 21:40:44 UTC, Loara wrote:

 When `connect()` returns may happen that `b` is destroyed but 
 `a` not, so `a.next` contains a dangling pointer that


Not when connect returns, but the scope that connect was called 
from. Still, this can be deduced, you just have to give the 
scopes an ordering.



 not-scoped variable (`a.next` is not `scope` since this 
 attribute is not transitive)


Well, that is a flaw, if the object is stack allocated then the 
fields are too.


 is clearly dangerous since `connect` doesn't know which between 
 `a` and `b` terminates first.


The compiler could easily deduce it. It is not difficult to see 
what the life time constraint must be.
Jun 28 2022
next sibling parent reply bauss <jj_1337 live.dk> writes:
On Tuesday, 28 June 2022 at 21:58:48 UTC, Ola Fosheim Grøstad 
wrote:

 not-scoped variable (`a.next` is not `scope` since this 
 attribute is not transitive)


 Well, that is a flaw, if the object is stack allocated then the 
 fields are too.


Not necessarily, especially if the fields aren't value types. You 
can have stack allocated "objects" with pointers to heap 
allocated memory (heap allocated "objects".)

You can't, or rather you shouldn't have stack allocated fields 
within heap allocated "objects" however; as that will almost be 
guaranteed to lead to problems. I believe it's possible, but one 
should always refrain from it, but the same isn't true the for 
stack allocated "objects" with heap allocated fields.

Ex. from your example then even if the "node struct" you pass was 
allocated on the stack, then the memory the "next" pointer points 
to might not be allocated same place.

Unless I'm misunderstanding what you're trying to say.
Jun 28 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 29 June 2022 at 05:51:26 UTC, bauss wrote:

 Not necessarily, especially if the fields aren't value types. 
 You can have stack allocated "objects" with pointers to heap 
 allocated memory (heap allocated "objects".)


Those are not fields, those are separate objects… The compiler 
knows which is a field (part of the object).


 You can't, or rather you shouldn't have stack allocated fields 
 within heap allocated "objects" however; as that will almost be 
 guaranteed to lead to problems.


That is perfectly ok if you use RAII and manage life times.


 Ex. from your example then even if the "node struct" you pass 
 was allocated on the stack, then the memory the "next" pointer 
 points to might not be allocated same place.

 Unless I'm misunderstanding what you're trying to say.


You did :). If you look at the post I made in general about 
DIP1000 and flow typing you see that I annotate scope with a 
number to indicate life time ordering.

If you have `connect(int* a,int* b){a.next = b}` then the 
compiler can deduce that the signature with formal parameters 
should be `connect(scope!N(int*) a, scope_or_earlier!N(int*) b)`. 
The compiler then checks that the actual parameters at the call 
site are subtypes (same type or proper subtype).
Jun 29 2022
prev sibling parent reply Loara <loara noreply.com> writes:
On Tuesday, 28 June 2022 at 21:58:48 UTC, Ola Fosheim Grøstad 
wrote:

 Not when connect returns, but the scope that connect was called 
 from. Still, this can be deduced, you just have to give the 
 scopes an ordering.


The deduction can happen even if you don't use `scope` attribute. 
When you use `scope` attribute you're saying to compiler "You 
have to allocate this object on the stack, don't try to use heap 
allocation". If you want to let compiler to decide what is the 
best approach then don't use `scope`.


 Well, that is a flaw, if the object is stack allocated then the 
 fields are too.


No because:

`scope` variable === the variable is a pointer/reference that 
points to stack allocated data

So `scope int v;` is equal to `int v;` since `v` is not a 
pointer, whereas `scope int *p` is different from `int *v;` since 
the latter can't point to stack allocated integers. This is the 
difference.

Since stack allocated objects are destroyed in the reverse order 
allowing a recursive `scope` attribute is a bit dangerous as you 
can see in the following example:

```d

struct A{
   int *i;

   ~this(){
     writeln(*i);
   }
}
...
  {
   A a;
   int i = 2;
   ...
   scope int *j = &i;
   scope A *b = &a;
   (*b).i = j;
  } // i is destroyed before a
```


 The compiler could easily deduce it. It is not difficult to see 
 what the life time constraint must be.


Again if you want to let the compiler to deduce then don't use 
`scope`.
Jun 30 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 30 June 2022 at 19:56:38 UTC, Loara wrote:

 The deduction can happen even if you don't use `scope` 
 attribute.


I don't understand what you mean, it could, but it doesn't. And 
if it did then you would not need `scope`…


 When you use `scope` attribute you're saying to compiler "You 
 have to allocate this object on the stack, don't try to use 
 heap allocation".


These are function pointer parameters, how could it trigger 
allocation on the heap?


 If you want to let compiler to decide what is the best approach 
 then don't use `scope`.


But that doesn't work.


 So `scope int v;` is equal to `int v;` since `v` is not a 
 pointer, whereas `scope int *p` is different from `int *v;` 
 since the latter can't point to stack allocated integers. This 
 is the difference.


No, the latter can most certainly point to any integer. It is 
just that scope/scope ref/return ref is to be checked in  safe. 
Unfortunately it is way too limiting. Even standard flow typing 
appears to be as strong or stronger.


 Since stack allocated objects are destroyed in the reverse 
 order allowing a recursive `scope` attribute is a bit dangerous 
 as you can see in the following example:


If there are destructors then you can think of each stack 
allocated variable as introducing a invisible scope, but the 
compiler can keep track of this easily.

So the compiler knows the ordering. So if my function imposes and 
order on the lifetimes of the parameters, then the compiler 
should be able to check that the ordering constraint is satisfied.


 Again if you want to let the compiler to deduce then don't use 
 `scope`.


But then it won't compile at all in  safe!
Jun 30 2022
parent reply Loara <loara noreply.com> writes:
On Thursday, 30 June 2022 at 21:30:39 UTC, Ola Fosheim Grøstad 
wrote:

 I don't understand what you mean, it could, but it doesn't. And 
 if it did then you would not need `scope`…


If the compiler doesn't optimize your code is a compiler issue. 
When you use the `scope` attribute you're [forcing the compiler 
to do a stack 
allocation](https://dlang.org/spec/attribute.html#scope-class-var), so the
compiler forbids you from doing something stupid.


 These are function pointer parameters, how could it trigger 
 allocation on the heap?


When you declare `scope` function parameters you're saying that 
it may point both to heap allocated and stack allocated data, and 
the compiler choose the more restrictive option when it compiles 
your option, since at that time it doesn't know if your function 
arguments are heap allocated or stack allocated.


 But that doesn't work.


But you first said "The compiler should deduce if a `scope` 
pointer points to heap allocated data or not" and when someone 
tells you this should happen only for not `scope` pointers you 
say "But the compiler doesn't do that".


 No, the latter can most certainly point to any integer. It is 
 just that scope/scope ref/return ref is to be checked in  safe. 
 Unfortunately it is way too limiting. Even standard flow typing 
 appears to be as strong or stronger.


No-one forbids you from writing an unsafe function that does 
unsafe operations, the ` system` and ` trusted` attributes are 
available yet. Simply you should control very well your code in 
order to avoid memory leaks and other issues since memory 
management without garbage collectors can't be safe without 
introducing a lot of restrictions.

If you decide to do so I recommend you this [very useful 
tool](https://valgrind.org/).


 If there are destructors then you can think of each stack 
 allocated variable as introducing a invisible scope, but the 
 compiler can keep track of this easily.

 So the compiler knows the ordering. So if my function imposes 
 and order on the lifetimes of the parameters, then the compiler 
 should be able to check that the ordering constraint is 
 satisfied.


If you want the compiler to optimize your code you should remove 
any additional restriction, and declaring a pointer `scope` is an 
additional restriction not a relaxation. If you don't like it 
then you can always build a new compiler that do this if you need 
it so much, this is how open source software works.


 But then it won't compile at all in  safe!


Find me an example of safe code that needs it and can't work 
without `scope` variables.

Anyway I strongly recommend you to read these 
[documentation](https://dlang.org/spec/attribute.html#scope) 
[pages](https://dlang.org/spec/function.html#scope-parameters) in 
order to understand how `scope` works.

If you don't like `scope` then don't use it.
Jul 02 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 2 July 2022 at 09:42:17 UTC, Loara wrote:

 But you first said "The compiler should deduce if a `scope` 
 pointer points to heap allocated data or not" and when someone 
 tells you this should happen only for not `scope` pointers you 
 say "But the compiler doesn't do that".


This discussion isn't going anywhere… :-)

(Please don't use quotation marks unless you actually quote.).
Jul 02 2022