www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Understanding DIP 1000 semantics -- Where's the bug?

reply Mike Franklin <slavo5150 yahoo.com> writes:
 safe:

// Exhibit A
//--------------------------
int getValue1(int* i)
{
     return *i;
}

int* foo1()
{
     int value;
     int[] y;
     y ~= getValue1(&value); //Error: reference to local variable 
`value` assigned to non-scope parameter `i` calling 
`onlineapp.getValue1`
     return &y[0];
}

// Exhibit B
//--------------------------
int getValue2(ref int i)
{
     return i;
}

int* foo2()
{
     int value;
     int[] y;
     y ~= getValue2(value); // No error
     return &y[0];
}

// Exhibit C (Same as Exhibit A, but with `scope` attribute on 
`i`)
//--------------------------
int getValue3(scope int* i)
{
     return *i;
}

int* foo3()
{
     int value;
     int[] y;
     y ~= getValue3(&value); // No error
     return &y[0];
}

Compile with `-preview=dip1000`.  See it live at 
https://run.dlang.io/is/0zODCr

So, there's an inconsistency.

Possibility 1:
Exhibit A should not emit an error

Possibility 2:
Exhibit B should require the `scope` attribute on `i` or emit an 
error if `foo2`.

Which is it?  Where's the bug?  Shouldn't the compiler treat 
`int* i` and `ref i` consistently?

Thanks for the help.
Mike
Sep 21 2019
next sibling parent reply Suleyman <sahmi.soulaimane gmail.com> writes:
On Sunday, 22 September 2019 at 02:29:49 UTC, Mike Franklin wrote:

 Possibility 1:
 Exhibit A should not emit an error


This looks like the right answer. I don't see any local reference 
escaping in the entire program.
Sep 21 2019
next sibling parent Max Haughton <maxhaton gmail.com> writes:
On Sunday, 22 September 2019 at 02:52:17 UTC, Suleyman wrote:

 On Sunday, 22 September 2019 at 02:29:49 UTC, Mike Franklin 
 wrote:

 Possibility 1:
 Exhibit A should not emit an error


 This looks like the right answer. I don't see any local 
 reference escaping in the entire program.


This would involve inferring scope on non templates, which I 
believe would require modifying the specification.
Sep 21 2019
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 22 September 2019 at 02:52:17 UTC, Suleyman wrote:

 On Sunday, 22 September 2019 at 02:29:49 UTC, Mike Franklin 
 wrote:

 Possibility 1:
 Exhibit A should not emit an error


 This looks like the right answer. I don't see any local 
 reference escaping in the entire program.


When calling a function the compiler only looks at the function 
signature.

You can never pass a scope int* to a function accepting int*, 
regardless of whether the body of the function escapes it.
Sep 22 2019
prev sibling next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 22 September 2019 at 02:29:49 UTC, Mike Franklin wrote:

 So, there's an inconsistency.


I don't think there is.

I think that taking a reference of a local variable should result 
in a scope T*.

In that case `&value` in `y ~= getValue1(&value);` should result 
in a scope int*.
Sep 22 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 22 September 2019 at 07:01:42 UTC, Sebastiaan Koppe 
wrote:

 I think that taking a reference of a local variable should 
 result in a scope T*.

 In that case `&value` in `y ~= getValue1(&value);` should 
 result in a scope int*.


If that's the case then shouldn't Exhibit B (copied below for 
convenience) require getValue2` to be `int getValue2(scope ref 
int i)`?  If not, what's the justification?


// Exhibit B
//--------------------------
int getValue2(ref int i)
{
     return i;
}

int* foo2()
{
     int value;
     int[] y;
     y ~= getValue2(value); // No error
     return &y[0];
}
Sep 22 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 22 September 2019 at 07:54:12 UTC, Mike Franklin wrote:

 On Sunday, 22 September 2019 at 07:01:42 UTC, Sebastiaan Koppe 
 wrote:

 I think that taking a reference of a local variable should 
 result in a scope T*.

 In that case `&value` in `y ~= getValue1(&value);` should 
 result in a scope int*.


 If that's the case then shouldn't Exhibit B (copied below for 
 convenience) require getValue2` to be `int getValue2(scope ref 
 int i)`?  If not, what's the justification?


scope only works on pointers.

---
 safe:

// Exhibit B
//--------------------------
struct P(T) {
     T t;
}

alias Value1 = P!(int);
alias Value2 = P!(int*);

Value1 getValue1(ref Value1 v) { return v; }
Value2 getValue2(ref Value2 v) { return v; }

void foo2()
{
     scope Value1 value1;
     scope Value2 value2;
     getValue1(value1); // No error
     getValue2(value2); // Error: scope variable value2 assigned 
to non-scope parameter v calling onlineapp.getValue2
}
---
Sep 22 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 22 September 2019 at 10:09:14 UTC, Sebastiaan Koppe 
wrote:


 If that's the case then shouldn't Exhibit B (copied below for 
 convenience) require getValue2` to be `int getValue2(scope ref 
 int i)`?  If not, what's the justification?


 scope only works on pointers.

 ---
  safe:

 // Exhibit B
 //--------------------------
 struct P(T) {
     T t;
 }

 alias Value1 = P!(int);
 alias Value2 = P!(int*);

 Value1 getValue1(ref Value1 v) { return v; }
 Value2 getValue2(ref Value2 v) { return v; }

 void foo2()
 {
     scope Value1 value1;
     scope Value2 value2;
     getValue1(value1); // No error
     getValue2(value2); // Error: scope variable value2 assigned 
 to non-scope parameter v calling onlineapp.getValue2
 }
 ---


I wouldn't say that `scope` only works for pointers, but rather 
`scope` only works for references (i.e. pointers and `ref` 
types).  The issue is that `ref int i` behaves differently than 
`int* i`.  I understand that scope doesn't make any sense for 
something like `int i` because it's not a "reference type" (for 
lack of a better term).
Sep 22 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 22 September 2019 at 10:24:39 UTC, Mike Franklin wrote:

 I wouldn't say that `scope` only works for pointers, but rather 
 `scope` only works for references (i.e. pointers and `ref` 
 types).  The issue is that `ref int i` behaves differently than 
 `int* i`.  I understand that scope doesn't make any sense for 
 something like `int i` because it's not a "reference type" (for 
 lack of a better term).


I know that an argument passed to a `ref int i` parameter is 
implemented with a pointer. And you would expect dip1000 to apply 
there as well, but the underlying type is still `int`, and scope 
only works on pointers.

Don't get me wrong, I rather see this work with value types as 
well, but this is how it currently works.
Sep 22 2019
prev sibling next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 22.09.19 04:29, Mike Franklin wrote:

  safe:
 
 // Exhibit A
 //--------------------------
 int getValue1(int* i)
 {
      return *i;
 }
 
 int* foo1()
 {
      int value;
      int[] y;
      y ~= getValue1(&value); //Error: reference to local variable 
 `value` assigned to non-scope parameter `i` calling `onlineapp.getValue1`
      return &y[0];
 }
 
 // Exhibit B
 //--------------------------
 int getValue2(ref int i)
 {
      return i;
 }
 
 int* foo2()
 {
      int value;
      int[] y;
      y ~= getValue2(value); // No error
      return &y[0];
 }


[...]

 So, there's an inconsistency.
 
 Possibility 1:
 Exhibit A should not emit an error
 
 Possibility 2:
 Exhibit B should require the `scope` attribute on `i` or emit an error 
 if `foo2`.
 
 Which is it?  Where's the bug?  Shouldn't the compiler treat `int* i` 
 and `ref i` consistently?


As far as I understand, converting a `ref` to a pointer is still allowed 
with DIP 1000. So they should be handled the same.

Then it's possibility 2: `foo2` should error. Your code can easily be 
expanded to demonstrate memory corruption:

----
 safe:

immutable(int)* ip;

// Exhibit B
//--------------------------
int getValue2(ref immutable int i)
{
     immutable int* my_ip = &i;
     ip = my_ip;
     return i;
}

int* foo2()
{
     immutable int value = 42;
     int[] y;
     y ~= getValue2(value); // No error
     return &y[0];
}

void main()
{
     auto p = foo2();
     import std.stdio;
     writeln(*ip); /* Prints "42". */
     writeln(*ip); /* Prints garbage. An immutable value has changed. */
}
----
Sep 22 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 22 September 2019 at 08:49:14 UTC, ag0aep6g wrote:


 Then it's possibility 2: `foo2` should error. Your code can 
 easily be expanded to demonstrate memory corruption:

 ----
  safe:

 immutable(int)* ip;

 // Exhibit B
 //--------------------------
 int getValue2(ref immutable int i)
 {
     immutable int* my_ip = &i;
     ip = my_ip;
     return i;
 }

 int* foo2()
 {
     immutable int value = 42;
     int[] y;
     y ~= getValue2(value); // No error
     return &y[0];
 }

 void main()
 {
     auto p = foo2();
     import std.stdio;
     writeln(*ip); /* Prints "42". */
     writeln(*ip); /* Prints garbage. An immutable value has 
 changed. */
 }
 ----


Interestingly, taking the address of a `ref` in ` safe` code is 
prevented, but only when compiling *without* -preview=dip1000.  
See https://run.dlang.io/is/GArFzk

That's seems like a bug to me.
Sep 22 2019
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2019-09-22 19:26, Mike Franklin wrote:

 On Sunday, 22 September 2019 at 08:49:14 UTC, ag0aep6g wrote:
 

 Then it's possibility 2: `foo2` should error. Your code can easily be 
 expanded to demonstrate memory corruption:

 ----
  safe:

 immutable(int)* ip;

 // Exhibit B
 //--------------------------
 int getValue2(ref immutable int i)
 {
     immutable int* my_ip = &i;
     ip = my_ip;
     return i;
 }

 int* foo2()
 {
     immutable int value = 42;
     int[] y;
     y ~= getValue2(value); // No error
     return &y[0];
 }

 void main()
 {
     auto p = foo2();
     import std.stdio;
     writeln(*ip); /* Prints "42". */
     writeln(*ip); /* Prints garbage. An immutable value has changed. */
 }
 ----


 
 Interestingly, taking the address of a `ref` in ` safe` code is 
 prevented, but only when compiling *without* -preview=dip1000. See 
 https://run.dlang.io/is/GArFzk
 
 That's seems like a bug to me.


With DIP1000 enabled the compiler can figure out that it is safe to take 
the address of `i` and assign it to `my_ip`. It seems to fail to 
recognize that assigning to `ip` is unsafe. Perhaps that requires data 
flow analysis.

If you assign directly to `ip` it fails to compile.

-- 
/Jacob Carlborg
Sep 22 2019
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On Sunday, 22 September 2019 at 17:26:20 UTC, Mike Franklin wrote:

 Interestingly, taking the address of a `ref` in ` safe` code is 
 prevented, but only when compiling *without* -preview=dip1000.  
 See https://run.dlang.io/is/GArFzk

 That's seems like a bug to me.


That's just the normal pre-DIP-1000 error you get when taking the 
address of any local (`ref` or not). Allowing `&some_local` is 
pretty much the point of DIP 1000, so it's expected that the old 
error doesn't appear anymore.

There is a bug for sure, though. If `ref` implies `scope`, there 
should be a new error ("cannot take address of scope parameter"). 
If `scope` is not implied, it's ok that the function compiles, 
but then it shouldn't be possible to call it on a local.
Sep 22 2019
parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Sunday, 22 September 2019 at 17:54:48 UTC, ag0aep6g wrote:

 There is a bug for sure, though. If `ref` implies `scope`, 
 there should be a new error ("cannot take address of scope 
 parameter"). If `scope` is not implied, it's ok that the 
 function compiles, but then it shouldn't be possible to call it 
 on a local.


It's allowed as long as you don't escape... wait, no, it's always 
allowed. Whoops. The following code compiles with -dip1000.

      safe:

     int* foo(ref int x)
     {
         int* a = &x;
         return a;
     }

     void main() {
         int* p;
         {
             int x;
             p = foo(x);
         }
         *p = 1;			// Memory corruption
     }

That's a bug.
Sep 23 2019
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 23 September 2019 at 08:46:18 UTC, Olivier FAURE wrote:

 Whoops. The following code compiles with -dip1000.

      safe:

     int* foo(ref int x)
     {
         int* a = &x;
         return a;
     }

     void main() {
         int* p;
         {
             int x;
             p = foo(x);
         }
         *p = 1;			// Memory corruption
     }

 That's a bug.


Well, dip1000 doesn't do data-flow analyses. Which means the 
compiler doesn't see that `x` escapes through `a`.
Sep 23 2019
parent reply Meta <jared771 gmail.com> writes:
On Monday, 23 September 2019 at 18:39:03 UTC, Sebastiaan Koppe 
wrote:

 On Monday, 23 September 2019 at 08:46:18 UTC, Olivier FAURE 
 wrote:

 Whoops. The following code compiles with -dip1000.

      safe:

     int* foo(ref int x)
     {
         int* a = &x;
         return a;
     }

     void main() {
         int* p;
         {
             int x;
             p = foo(x);
         }
         *p = 1;			// Memory corruption
     }

 That's a bug.


 Well, dip1000 doesn't do data-flow analyses. Which means the 
 compiler doesn't see that `x` escapes through `a`.


AFAICT, according to dip1000 this code should not allow the 
result of `foo(x)` to be assigned to p, as it has a longer 
lifetime. Likewise, it should not allow foo to return &x without 
the parameter being annotated with `return`. This looks like a 
bug in the implementation.
Sep 23 2019
next sibling parent Meta <jared771 gmail.com> writes:
On Monday, 23 September 2019 at 18:46:40 UTC, Meta wrote:

 AFAICT, according to dip1000 this code should not allow the 
 result of `foo(x)` to be assigned to p, as it has a longer 
 lifetime. Likewise, it should not allow foo to return &x 
 without the parameter being annotated with `return`. This looks 
 like a bug in the implementation.


Specifically, this breaks rule 1 from DIP1000:

A scope variable can only be initialized and assigned from values 
that have lifetimes longer than the variable's lifetime. (As a 
consequence a scope variable can only be assigned to scope 
variables that have shorter lifetime.)

This rule is broken by `p = foo(x)`, since p has a longer 
lifetime than x. Marking the `ref int x` parameter of foo 
`scope`, `return scope` or `scope return` did not help, but 
marking it as `return` *did* cause the compiler to correctly 
reject this code. Also changing foo to:

int* foo(ref int x)
{
     return &x;
}

OR

int* foo(int* x)
{
     return x;
}
...
p = foo(&x);

Causes the compiler to correctly reject this code, so the problem 
seems to specifically be with returning the address of a ref 
parameter stored in a local variable.
Sep 23 2019
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 23 September 2019 at 18:46:40 UTC, Meta wrote:

 On Monday, 23 September 2019 at 18:39:03 UTC, Sebastiaan Koppe

 Well, dip1000 doesn't do data-flow analyses. Which means the 
 compiler doesn't see that `x` escapes through `a`.


 AFAICT, according to dip1000 this code should not allow the 
 result of `foo(x)` to be assigned to p, as it has a longer 
 lifetime. Likewise, it should not allow foo to return &x 
 without the parameter being annotated with `return`.


It does all those things when you replace the body of foo with 
just `return &x;`.
Sep 23 2019
parent reply ag0aep6g <anonymous example.com> writes:
On Monday, 23 September 2019 at 19:11:46 UTC, Sebastiaan Koppe 
wrote:

 On Monday, 23 September 2019 at 18:46:40 UTC, Meta wrote:

 On Monday, 23 September 2019 at 18:39:03 UTC, Sebastiaan Koppe

 Well, dip1000 doesn't do data-flow analyses. Which means the 
 compiler doesn't see that `x` escapes through `a`.


 AFAICT, according to dip1000 this code should not allow the 
 result of `foo(x)` to be assigned to p, as it has a longer 
 lifetime. Likewise, it should not allow foo to return &x 
 without the parameter being annotated with `return`.


 It does all those things when you replace the body of foo with 
 just `return &x;`.


Not really. When you do that, DMD rejects the `return` statement, 
not the assignment `p = foo(x);`.

It works as expected when you mark the parameter with `return`. 
This has nothing to do with missing flow analysis. It's just a 
bug.

Maybe there is some clever idea regarding `ref` parameters and 
the implementation is just buggy, or maybe the bug is that 
they're not treated just like pointers.

I'd suggest treating them just like pointers. Seems to me that's 
the most straight-forward approach.

Meaning:

----
 safe:

int* fp(int* p) { return p; } /* This works. */
int* fr(ref int r)  { return &r; } /* So this should work too. */

void main()
{
     int x;
     fp(&x); /* This doesn't work. */
     fr(x); /* So this shouldn't work either. */
}
----
Sep 23 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 23 September 2019 at 20:50:30 UTC, ag0aep6g wrote:

 On Monday, 23 September 2019 at 19:11:46 UTC, Sebastiaan Koppe 
 wrote:

 On Monday, 23 September 2019 at 18:46:40 UTC, Meta wrote:

 On Monday, 23 September 2019 at 18:39:03 UTC, Sebastiaan Koppe

 Well, dip1000 doesn't do data-flow analyses. Which means the 
 compiler doesn't see that `x` escapes through `a`.


 AFAICT, according to dip1000 this code should not allow the 
 result of `foo(x)` to be assigned to p, as it has a longer 
 lifetime. Likewise, it should not allow foo to return &x 
 without the parameter being annotated with `return`.


 It does all those things when you replace the body of foo with 
 just `return &x;`.


 Not really. When you do that, DMD rejects the `return` 
 statement, not the assignment `p = foo(x);`.


But that is what you want right? You want it to error out.

My reasoning is as follows, because dataflow analyses is missing, 
you need to simplify the body of foo to `return &x`. Then the 
compilers errors out saying you are escaping a reference and 
maybe you need to add the return annotation. After you fix that 
the compiler errors out saying "Error: address of variable x 
assigned to p with longer lifetime". Which is exactly what we 
want.

Ergo, missing dataflow analyses.


 It works as expected when you mark the parameter with `return`. 
 This has nothing to do with missing flow analysis. It's just a 
 bug.


I agree that it will be better to get the 2 errors the first 
time, but is that a bug?


 ----
  safe:

 int* fp(int* p) { return p; } /* This works. */
 int* fr(ref int r)  { return &r; } /* So this should work too. 
 */

 void main()
 {
     int x;
     fp(&x); /* This doesn't work. */
     fr(x); /* So this shouldn't work either. */
 }
 ----


1) fp isn't `scope int*` so the function body compiles without 
errors;
2) you cannot return a reference to a local, so the function `fr` 
errors out correctly;
3) you cannot pass a reference to a local to a non-scope 
argument, so fp(&x) errors out correctly;
4) it is perfectly valid to pass a value to a ref parameter;

Still, I agree that it will be better to get the 2 errors the 
first time.
Sep 24 2019
parent reply ag0aep6g <anonymous example.com> writes:
On 24.09.19 10:05, Sebastiaan Koppe wrote:

 On Monday, 23 September 2019 at 20:50:30 UTC, ag0aep6g wrote:


[...]

 Not really. When you do that, DMD rejects the `return` statement, not 
 the assignment `p = foo(x);`.


 
 But that is what you want right? You want it to error out.


Not at that point. Unless `ref` implies restrictions that pointers don't 
have, `foo` should compile.


 My reasoning is as follows, because dataflow analyses is missing, you 
 need to simplify the body of foo to `return &x`. Then the compilers 
 errors out saying you are escaping a reference and maybe you need to add 
 the return annotation. After you fix that the compiler errors out saying 
 "Error: address of variable x assigned to p with longer lifetime". Which 
 is exactly what we want.
 
 Ergo, missing dataflow analyses.


Aside: Your reasoning is circular. You start and end with "dataflow 
analysis is missing".

When using a pointer instead of `ref`, the code is rejected as expected 
even with the more complex body:

----
 safe:

int* foo(int* x)
{
     int* a = x;
     return x;
}

void main() {
     int* p;
     {
         int x;
         p = foo(&x); /* error here */
     }
}
----

So it works just fine with pointers. But maybe `ref`s are intentionally 
treated differently from pointers. And maybe that means that they need 
more advanced dataflow analysis which isn't currently implemented. If 
that's so, it's a hole in -dip1000.

So I'm saying: Just treat `ref`s like pointers and the trouble 
disappears. I suspect that it's actually intended to work like that, and 
we're just looking at a good old bug.


 It works as expected when you mark the parameter with `return`. This 
 has nothing to do with missing flow analysis. It's just a bug.


 
 I agree that it will be better to get the 2 errors the first time, but 
 is that a bug?


Memory corruption in  safe code is always a bug.

[...]

 1) fp isn't `scope int*` so the function body compiles without errors;


Yup.


 2) you cannot return a reference to a local, so the function `fr` errors 
 out correctly;


A `ref` parameter is significantly different from a true local. In the 
context of DIP 1000, it's more straight-forward to treat it like a pointer.


 3) you cannot pass a reference to a local to a non-scope argument, so 
 fp(&x) errors out correctly;


Yup.


 4) it is perfectly valid to pass a value to a ref parameter;


Not when we treat `ref` like the pointer that it really is.

Pretending that `ref` is anything but a pointer wearing a fancy hat just 
leads to headaches like the one we're experiencing right now.
Sep 24 2019
parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Tuesday, 24 September 2019 at 13:35:04 UTC, ag0aep6g wrote:

 When using a pointer instead of `ref`, the code is rejected as 
 expected even with the more complex body:

 ----
  safe:

 int* foo(int* x)
 {
     int* a = x;
     return x;
 }

 void main() {
     int* p;
     {
         int x;
         p = foo(&x); /* error here */
     }
 }
 ----


That's a bad comparison. Refs aren't equivalent to unqualified 
pointers, they're equivalent to a scope pointers. By the way, 
with the following code:

      safe:

     int* foo(scope int* x)
     {
         int* a = x;
         return a;              // Compile error: scope variable 
*a* may not be returned
     }

     void main() {
         int* p;
         {
             int x;
             p = foo(&x);
         }
         *p = 1;			// Memory corruption
     }

the compiler correctly identifies that the problem isn't 
"foo(&x)", but "return a", and gives an appropriate error message.

So the problem isn't a lack of flow analysis or wrong ref 
semantics, it's that ref isn't implemented the same way as scope.

(Anyone feel like submitting a bug report? I don't have an 
account)
Sep 24 2019
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 24.09.19 19:01, Olivier FAURE wrote:

 Refs aren't equivalent to unqualified pointers, 
 they're equivalent to a scope pointers.


Whether `ref` does or should imply `scope` is topic of the discussion.


 By the way, with the following 
 code:
 
       safe:
 
      int* foo(scope int* x)
      {
          int* a = x;
          return a;              // Compile error: scope
variable *a* may 
 not be returned
      }
 
      void main() {
          int* p;
          {
              int x;
              p = foo(&x);
          }
          *p = 1;            // Memory corruption
      }
 
 the compiler correctly identifies that the problem isn't "foo(&x)", but 
 "return a", and gives an appropriate error message.
 
 So the problem isn't a lack of flow analysis or wrong ref semantics, 
 it's that ref isn't implemented the same way as scope.


I think we're almost on the same page now. There are two ways to attack 
the issue:

1) Make `ref int` more like `scope int*`. I.e., disallow `return a;`.
2) Make `ref int` more like (unqualified) `int*`. I.e., disallow `foo(&x);`.
Sep 24 2019
parent ag0aep6g <anonymous example.com> writes:
On 24.09.19 19:49, ag0aep6g wrote:

 On 24.09.19 19:01, Olivier FAURE wrote:


[...]

 I think we're almost on the same page now.


And now I notice that you're not Sebastiaan Koppe, Olivier. I thought 
your post was by him. Sorry for the additional confusion.
Sep 24 2019
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 24 September 2019 at 17:01:56 UTC, Olivier FAURE 
wrote:

 That's a bad comparison. Refs aren't equivalent to unqualified 
 pointers, they're equivalent to a scope pointers.


Where does it say that? Would be good though.


 By the way, with the following code:

      safe:

     int* foo(scope int* x)
     {
         int* a = x;
         return a;              // Compile error: scope variable 
 *a* may not be returned
     }

     void main() {
         int* p;
         {
             int x;
             p = foo(&x);
         }
         *p = 1;			// Memory corruption
     }

 the compiler correctly identifies that the problem isn't 
 "foo(&x)", but "return a", and gives an appropriate error 
 message.


That is because scope is inferred on `int* a`.
Sep 24 2019
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/23/2019 1:46 AM, Olivier FAURE wrote:

 The following code compiles with -dip1000.
 
       safe:
 
      int* foo(ref int x)
      {
          int* a = &x;
          return a;
      }


That's a bug. Not sure how I missed that. `a` should be inferred as `scope`.
Sep 25 2019
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/25/2019 10:36 PM, Walter Bright wrote:

 That's a bug. Not sure how I missed that. `a` should be inferred as `scope`.


https://issues.dlang.org/show_bug.cgi?id=20245
Sep 25 2019
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Thursday, 26 September 2019 at 05:36:39 UTC, Walter Bright 
wrote:

 On 9/23/2019 1:46 AM, Olivier FAURE wrote:

 The following code compiles with -dip1000.
 
       safe:
 
      int* foo(ref int x)
      {
          int* a = &x;
          return a;
      }


 That's a bug. Not sure how I missed that. `a` should be 
 inferred as `scope`.


That is interesting. Thanks for the clear up.
Sep 26 2019
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2019-09-22 04:29, Mike Franklin wrote:

  safe:
 
 // Exhibit A
 //--------------------------
 int getValue1(int* i)
 {
      return *i;
 }
 
 int* foo1()
 {
      int value;
      int[] y;
      y ~= getValue1(&value); //Error: reference to local variable 
 `value` assigned to non-scope parameter `i` calling `onlineapp.getValue1`
      return &y[0];
 }
 
 // Exhibit B
 //--------------------------
 int getValue2(ref int i)
 {
      return i;
 }
 
 int* foo2()
 {
      int value;
      int[] y;
      y ~= getValue2(value); // No error
      return &y[0];
 }
 
 // Exhibit C (Same as Exhibit A, but with `scope` attribute on `i`)
 //--------------------------
 int getValue3(scope int* i)
 {
      return *i;
 }
 
 int* foo3()
 {
      int value;
      int[] y;
      y ~= getValue3(&value); // No error
      return &y[0];
 }
 
 Compile with `-preview=dip1000`.  See it live at 
 https://run.dlang.io/is/0zODCr
 
 So, there's an inconsistency.
 
 Possibility 1:
 Exhibit A should not emit an error
 
 Possibility 2:
 Exhibit B should require the `scope` attribute on `i` or emit an error 
 if `foo2`.
 
 Which is it?  Where's the bug?  Shouldn't the compiler treat `int* i` 
 and `ref i` consistently?


I think this is working as intended.

In Exhibit A, `i` is a pointer and may escape the function `getValue1`. 
Therefore it's not allowed to pass the address of a local variable to it.

In Exhibit B, `i` is a reference and cannot escape the function `getValue2`.

In Exhibit C, the `scope` annotation of `i` makes sure that `i` cannot 
escape the function `getValue3`. Therefore it's ok to pass the address 
of a local variable to it.

-- 
/Jacob Carlborg
Sep 22 2019
next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 22 September 2019 at 11:03:29 UTC, Jacob Carlborg 
wrote:


 In Exhibit B, `i` is a reference and cannot escape the function 
 `getValue2`.


Ok, that is an interesting observation.  If that's the case, then 
I don't see any reason to ever annotate a `ref` parameter with 
`scope`.  In other words, for `ref` parameters `scope` is 
inferred (or implied).  Do you agree with that?
Sep 22 2019
parent reply Jacob Carlborg <doob me.com> writes:
On 2019-09-22 13:22, Mike Franklin wrote:


 Ok, that is an interesting observation.  If that's the case, then I 
 don't see any reason to ever annotate a `ref` parameter with `scope`.  
 In other words, for `ref` parameters `scope` is inferred (or implied).  
 Do you agree with that?


Yes.

-- 
/Jacob Carlborg
Sep 22 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 22 September 2019 at 16:32:02 UTC, Jacob Carlborg 
wrote:

 On 2019-09-22 13:22, Mike Franklin wrote:


 Ok, that is an interesting observation.  If that's the case, 
 then I don't see any reason to ever annotate a `ref` parameter 
 with `scope`.  In other words, for `ref` parameters `scope` is 
 inferred (or implied).  Do you agree with that?


 Yes.


According to DIP1000 
(https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1000.m
#identity-function) `scope ref` is a thing, so I guess `scope` can't be
inferred or implied for `ref`.

I don't know.  I think Walter is the only one that can clear this 
up.
Sep 22 2019
next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 22 September 2019 at 17:01:59 UTC, Mike Franklin wrote:

 On Sunday, 22 September 2019 at 16:32:02 UTC, Jacob Carlborg 
 wrote:

 On 2019-09-22 13:22, Mike Franklin wrote:


 Ok, that is an interesting observation.  If that's the case, 
 then I don't see any reason to ever annotate a `ref` 
 parameter with `scope`.  In other words, for `ref` parameters 
 `scope` is inferred (or implied).  Do you agree with that?


 Yes.


 According to DIP1000 
 (https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1000.m
#identity-function) `scope ref` is a thing, so I guess `scope` can't be
inferred or implied for `ref`.

 I don't know.  I think Walter is the only one that can clear 
 this up.


Yes, `scope ref` is definitely a thing.  Compare the following.

// Exhibit C
 safe:

struct A
{
      int* ptr;
}

int* gptr;

void foo(ref A a)
{
      gptr = a.ptr; // error, can't leak borrowed a.ptr into 
global context
}

https://run.dlang.io/is/ZU780c


// Exhibit D
 safe:

struct A
{
      int* ptr;
}

int* gptr;

void foo(scope ref A a)
{
      gptr = a.ptr; // error, can't leak borrowed a.ptr into 
global context
}

https://run.dlang.io/is/ZU780c

This was taken from the discussion at 
https://forum.dlang.org/post/twgsrfcolypurgylhizh forum.dlang.org
Sep 22 2019
parent Olivier FAURE <couteaubleu gmail.com> writes:
On Sunday, 22 September 2019 at 17:17:40 UTC, Mike Franklin wrote:

 Yes, `scope ref` is definitely a thing.  Compare the following.

 [...]

 This was taken from the discussion at 
 https://forum.dlang.org/post/twgsrfcolypurgylhizh forum.dlang.org


Huh.

I knew that was a thing, but I assumed the semantics were 
emergent. I didn't realize they might be intentional.
Sep 23 2019
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/22/2019 10:01 AM, Mike Franklin wrote:

 I don't know.  I think Walter is the only one that can clear this up.


   int* foo(scope ref int* p, ref int* q)
   {
     return p; // error
     return q; // ok
   }

I.e. with `ref int*` there are two pointers here - the ref pointer and the * 
pointer. The scope in `scope ref int*` refers to the * pointer, not the ref
pointer.
Sep 25 2019
prev sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 22 September 2019 at 11:03:29 UTC, Jacob Carlborg 
wrote:


 In Exhibit B, `i` is a reference and cannot escape the function 
 `getValue2`.


Actually, ag0aep6g's demonstration at 
https://forum.dlang.org/post/qm7cib$2n07$1 digitalmars.com proves 
that statement false.  Unless it is a bug in the compiler that 
one can convert a `ref` to a pointer in ` safe` code.
Sep 22 2019
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 22 September 2019 at 11:42:44 UTC, Mike Franklin wrote:

 On Sunday, 22 September 2019 at 11:03:29 UTC, Jacob Carlborg 
 wrote:


 In Exhibit B, `i` is a reference and cannot escape the 
 function `getValue2`.


 Actually, ag0aep6g's demonstration at 
 https://forum.dlang.org/post/qm7cib$2n07$1 digitalmars.com 
 proves that statement false.  Unless it is a bug in the 
 compiler that one can convert a `ref` to a pointer in ` safe` 
 code.


Yes. The compiler only looks at the signature when it is checking 
the call site of said function.
Sep 22 2019
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2019-09-22 13:42, Mike Franklin wrote:


 Actually, ag0aep6g's demonstration at 
 https://forum.dlang.org/post/qm7cib$2n07$1 digitalmars.com proves that 
 statement false.  Unless it is a bug in the compiler that one can 
 convert a `ref` to a pointer in ` safe` code.


That might be a bug in the compiler. Adding `scope` to the reference 
doesn't help.

But with DIP1000, there's some documentation missing and I think only 
Walter knows how it should behave.

-- 
/Jacob Carlborg
Sep 22 2019
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2019 7:29 PM, Mike Franklin wrote:

 Which is it?  Where's the bug?  Shouldn't the compiler treat `int* i` and
`ref 
 i` consistently?


`ref` semantics are set up so a ref cannot escape the function it is passed to, 
hence there's no error. Pointers have no such restrictions, and so require an 
extra annotation `scope` to acquire the restriction.
Sep 25 2019