www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Cannot take slice of scope static array in safe code

reply Dennis <dkorpel gmail.com> writes:
Compiling the following with -dip1000 gives an error.

```
void main()  safe {
     string[1] a0;
     scope int[1] a1;
     scope string[1] a2;

     scope string[] b0 = a0[]; // Fine
     scope int[] b1 = a1[]; // Fine
     scope string[] b2 = a2[]; // Error: cannot take address of 
scope local a2
}
```

Can anyone explain why? I don't see how b2 violates the scope 
constraint of a2.
It might be a compiler bug, but since the int[] case works, I 
could also just be missing something about how `scope` works on 
`string[1]`.
Feb 02 2020
parent reply ag0aep6g <anonymous example.com> writes:
On 02.02.20 14:40, Dennis wrote:

 Compiling the following with -dip1000 gives an error.
 
 ```
 void main()  safe {
      string[1] a0;
      scope int[1] a1;
      scope string[1] a2;
 
      scope string[] b0 = a0[]; // Fine
      scope int[] b1 = a1[]; // Fine
      scope string[] b2 = a2[]; // Error: cannot take address of scope 
 local a2
 }
 ```
 
 Can anyone explain why? I don't see how b2 violates the scope constraint 
 of a2.


To make this easier, let's:

1) type out `string` as `immutable(char)[]`,
2) throw that `immutable` away, because it doesn't matter,
3) replace the arrays with pointers.

Then we're looking at this:

     void main()  safe
     {
         int* a0;
         scope int** b0 = &a0; /* accepted */

         scope int* a2;
         scope int** b2 = &a2; /* rejected */
     }

Now it's important to realize that `scope` only applies to the top-level 
of the type. That means, when you dereference b0 or b2, you get a plain 
`int*` without any `scope` on it.

For b0 that's fine, because a0 isn't `scope` (neither explicit nor 
inferred).

But for b2 it would be a problem, because a2 would lose its `scope`. 
That can't be allowed.


 It might be a compiler bug, but since the int[] case works


With regards to DIP 1000, `string` is very different from `int`. 
`string` has an indirection. `int` doesn't. The analogous type to 
`string[]` is `int[][]`.
Feb 02 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/2/20 10:20 AM, ag0aep6g wrote:

 On 02.02.20 14:40, Dennis wrote:

 Compiling the following with -dip1000 gives an error.

 ```
 void main()  safe {
      string[1] a0;
      scope int[1] a1;
      scope string[1] a2;

      scope string[] b0 = a0[]; // Fine
      scope int[] b1 = a1[]; // Fine
      scope string[] b2 = a2[]; // Error: cannot take address of scope 
 local a2
 }
 ```

 Can anyone explain why? I don't see how b2 violates the scope 
 constraint of a2.


 
 To make this easier, let's:
 
 1) type out `string` as `immutable(char)[]`,
 2) throw that `immutable` away, because it doesn't matter,
 3) replace the arrays with pointers.
 
 Then we're looking at this:
 
      void main()  safe
      {
          int* a0;
          scope int** b0 = &a0; /* accepted */
 
          scope int* a2;
          scope int** b2 = &a2; /* rejected */
      }
 
 Now it's important to realize that `scope` only applies to the top-level 
 of the type. That means, when you dereference b0 or b2, you get a plain 
 `int*` without any `scope` on it.


I think this is wrong. a0 is not a scope array of scope strings, it's a 
scope array of strings. string isn't scope, it points to immutable data 
that's on the heap or in the static segment (generally).

This is the biggest problem of dip1000 -- there's just no granularity in 
scope, it applies to EVERYTHING that is referenced from it.

scope should have been a type constructor.


 
 For b0 that's fine, because a0 isn't `scope` (neither explicit nor 
 inferred).
 
 But for b2 it would be a problem, because a2 would lose its `scope`. 
 That can't be allowed.
 

 It might be a compiler bug, but since the int[] case works


 
 With regards to DIP 1000, `string` is very different from `int`. 
 `string` has an indirection. `int` doesn't. The analogous type to 
 `string[]` is `int[][]`.


How does one declare "I have this array which is scope, because it's 
storage is on the stack. But it points at things that are in the GC 
heap, so it's cool to reference those without scope."

And if that is a simple matter, what about an array stored on the stack 
of arrays stored on the stack, of references to heap data? How do I make 
sure the heap pointers are still heap pointers, and not have to cast the 
compiler into submission?

I've run into problems like this before, they aren't solvable with 
dip1000. And string arrays are usually where it stops working because of 
the double indirection.

-Steve
Feb 02 2020
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Sunday, 2 February 2020 at 18:18:28 UTC, Steven Schveighoffer 
wrote:

 scope should have been a type constructor.


I feel the same way, I find const/immutable much easier to reason 
about than scope in its current state.
Do you think scope as a storage class is fundamentally broken, or 
is it still sound, just not very granular?
Feb 02 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/2/20 1:32 PM, Dennis wrote:

 On Sunday, 2 February 2020 at 18:18:28 UTC, Steven Schveighoffer wrote:

 scope should have been a type constructor.


 
 I feel the same way, I find const/immutable much easier to reason about 
 than scope in its current state.
 Do you think scope as a storage class is fundamentally broken, or is it 
 still sound, just not very granular?


I think it should be like const or immutable -- both storage class and 
type constructor.

-Steve
Feb 02 2020
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On 02.02.20 19:18, Steven Schveighoffer wrote:

 On 2/2/20 10:20 AM, ag0aep6g wrote:


[...]

      void main()  safe
      {
          int* a0;
          scope int** b0 = &a0; /* accepted */

          scope int* a2;
          scope int** b2 = &a2; /* rejected */
      }

 Now it's important to realize that `scope` only applies to the 
 top-level of the type. That means, when you dereference b0 or b2, you 
 get a plain `int*` without any `scope` on it.


 
 I think this is wrong. a0 is not a scope array of scope strings, it's a 
 scope array of strings. string isn't scope, it points to immutable data 
 that's on the heap or in the static segment (generally).


I don't follow. In my code, a0's type is `int*`. In the original code 
it's `string[1]`. Neither of them are `scope`.

[...]

 How does one declare "I have this array which is scope, because it's 
 storage is on the stack. But it points at things that are in the GC 
 heap, so it's cool to reference those without scope."


As far as I understand, you're describing what `scope` does:

     int[][] g1;
     int[] g2;
     void main()  safe
     {
         int[][3] arr = [[1, 2], [3, 4], [5, 6]];
         int[][] slice = arr[]; /* Inferred `scope`. */
         g1 = slice; /* Nope. `slice` is `scope`. */
         g2 = slice[0]; /* Ok. `slice[0]` is not `scope`. */
     }


 And if that is a simple matter, what about an array stored on the stack 
 of arrays stored on the stack, of references to heap data? How do I make 
 sure the heap pointers are still heap pointers, and not have to cast the 
 compiler into submission?


I'm not sure if I got it right, but like this?

     int*[][] g1;
     int*[] g2;
     int* g3;

     void main()  safe
     {
         /* An array stored on the stack, of references to heap data: */
         int*[3] a1 = [new int, new int, new int];
         /* Another such array: */
         int*[3] a2 = [new int, new int, new int];
         /* An array of those arrays, stored on the stack: */
         int*[][2] b = [a1[], a2[]];

         g1 = b[]; /* Nope. */
         g2 = b[0]; /* Nope. */
         g3 = b[0][0]; /* Ok. */
     }


 I've run into problems like this before, they aren't solvable with 
 dip1000. And string arrays are usually where it stops working because of 
 the double indirection.


My experience with DIP 1000 is more that it has many holes where leaks 
don't get detected. That has made it difficult for me to form an 
understanding of what is supposed to work and what isn't. I'd like to 
believe that I got it by now, but I'm not really sure.
Feb 02 2020
parent reply ag0aep6g <anonymous example.com> writes:
On 02.02.20 19:49, ag0aep6g wrote:

 On 02.02.20 19:18, Steven Schveighoffer wrote:
 I'm not sure if I got it right, but like this?
 
      int*[][] g1;
      int*[] g2;
      int* g3;
 
      void main()  safe
      {
          /* An array stored on the stack, of references to heap data: */
          int*[3] a1 = [new int, new int, new int];
          /* Another such array: */
          int*[3] a2 = [new int, new int, new int];
          /* An array of those arrays, stored on the stack: */
          int*[][2] b = [a1[], a2[]];
 
          g1 = b[]; /* Nope. */
          g2 = b[0]; /* Nope. */
          g3 = b[0][0]; /* Ok. */
      }


I guess I kinda missed the point there, and what you want is this:

     scope int*[][] slice = b[];
     /* ... use `slice` in non-leaky ways ... */

Which DIP 1000 doesn't allow, because it wouldn't be able to ensure that 
the references to a1 and a2 don't leak. Yeah, DIP 1000 falls somewhat short.
Feb 02 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/2/20 1:57 PM, ag0aep6g wrote:

 On 02.02.20 19:49, ag0aep6g wrote:

 On 02.02.20 19:18, Steven Schveighoffer wrote:
 I'm not sure if I got it right, but like this?

      int*[][] g1;
      int*[] g2;
      int* g3;

      void main()  safe
      {
          /* An array stored on the stack, of references to heap data:
*/
          int*[3] a1 = [new int, new int, new int];
          /* Another such array: */
          int*[3] a2 = [new int, new int, new int];
          /* An array of those arrays, stored on the stack: */
          int*[][2] b = [a1[], a2[]];

          g1 = b[]; /* Nope. */
          g2 = b[0]; /* Nope. */
          g3 = b[0][0]; /* Ok. */
      }


 
 I guess I kinda missed the point there, and what you want is this:
 
      scope int*[][] slice = b[];
      /* ... use `slice` in non-leaky ways ... */
 
 Which DIP 1000 doesn't allow, because it wouldn't be able to ensure that 
 the references to a1 and a2 don't leak. Yeah, DIP 1000 falls somewhat 
 short.


The simple problem I've had is (I think), I want an array of strings 
that is scope as a parameter (i.e. I won't share any of the addresses of 
those elements outside the function), but the strings themselves are 
strings that are usable anywhere.

I can't figure out where I've seen this, or how to reproduce it (writing 
a simple test seems to work). But I've had it happen to me.

That's the problem with dip1000. It works in most cases, and then you 
get this one tricky case, and you can't figure out how to get around it. 
Or you have to do everything in one expression so the compiler can 
deduce the actual scope-ness of things you aren't allowed to declare. It 
leaves a general feeling of something missed in the design.

-Steve
Feb 02 2020
prev sibling parent Dennis <dkorpel gmail.com> writes:
Thanks for your response.

On Sunday, 2 February 2020 at 15:20:39 UTC, ag0aep6g wrote:

 Now it's important to realize that `scope` only applies to the 
 top-level of the type.


This is where my confusion was.
I knew scope wasn't transitive, so I thought that `scope 
string[1]` meant the static array could not escape the function, 
but the elements of it (the strings) could, since arr[0] would 
yield a `string` and not a `scope string`. I thought that it 
wouldn't have mattered if the element type had indirections 
(string) or not (int).

Now I realize scope applies to the elements, since indexing a 
static array doesn't actually remove a layer of indirection 
unlike a dynamic array.
Feb 02 2020