www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.internals - DIP1000 discussion and testing

reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: DIP1000 discussion and testing

--wOClvSkNK3LhkNpiKGvc9cBvr8w7r5rHG
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

So, there is a pull request (https://github.com/dlang/dmd/pull/5972)
which is intended to implement DIP1000
(https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md).

Problem is that Walter wants to prioritize moving on with it (which is
reasonable) but the PR is too complicated to be reviewed by most other
developers.

Trying to solve it, I have been experimenting lately with various
acceptance tests trying to come up with useful design snippets enabled
by that PR. Rationale is that if actual semantics are clear and examples
are all validated as included test cases, review of the code itself
becomes less important.

I'd like to have more open feedback about it because in its current
shape PR5972 doesn't seem useful enough to me to be merged at all - and
much less powerful than DIP1000 seemed to promise.

I'll start separate sub-threads with more details.


--wOClvSkNK3LhkNpiKGvc9cBvr8w7r5rHG--
Oct 16 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: DIP1000 discussion and testing: RC pointer snippet
References: <nu00a6$t5i$1 digitalmars.com>
In-Reply-To: <nu00a6$t5i$1 digitalmars.com>

--vraXSChUTnhqU5bkpie8rqsDHcDoUNcxK
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/16/2016 04:45 PM, Dicebot wrote:

 I'll start separate sub-threads with more details.=20


This snippet is a variation of RCSlice example from the DIP:

------------------------------------------------------------
 safe unittest
{
     safe static struct RCPointer (T)
    {
        private T*      payload;
        private size_t* count;

        this (T initializer)
        {
            this.payload  =3D new T;
            *this.payload =3D initializer;
            this.count    =3D new size_t;
            *count =3D 1;
        }

        this (this)
        {
            if (this.count)
                ++(*this.count);
        }

         trusted ~this ()
        {
            if (this.count && !--(*this.count))
            {
                delete payload;
                delete count;
            }
        }

	// must bind lifetime of return value to lifetime of `this`
        T* get () return scope
        {
            return this.payload;
        }

        alias get this;
    }

    auto ptr =3D RCPointer!int(42);
    auto leak2 =3D ptr.get(); // must not compile unless spec says 'auto'=

                            // can deduce scope
    int* leak1 =3D ptr.get(); // must not compile

    void foo (RCPointer!int)
    {
        assert(*(ptr.count) =3D=3D 2);
    }

    foo(ptr);

    void bar (ref int)
    {
        assert(*(ptr.count) =3D=3D 1);
    }

    bar(*ptr);
}

------------------------------------------------------------

DIP1000 says that `return scope` applied to a method means applying it
to aggregate `this`. It also says that struct is viewed as a composition
of all its fields when it comes to lifetime.

Natural consequence is that it should be possible to define `get` method
like in this example to ensure that pointer won't outlive struct
instance it came from.

This snippet currently compiles, failing to reject both "leak" lines.


--vraXSChUTnhqU5bkpie8rqsDHcDoUNcxK--
Oct 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2016 6:55 AM, Dicebot wrote:

 This snippet currently compiles, failing to reject both "leak" lines.


It compiles because 'scope' is inferred for both leak1 and leak2.
Oct 16 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing: RC pointer snippet
References: <nu00a6$t5i$1 digitalmars.com> <nu00tf$u0a$1 digitalmars.com>
 <nu0lrm$1tm1$1 digitalmars.com>
In-Reply-To: <nu0lrm$1tm1$1 digitalmars.com>

--U458ROC3lnDWdCSgJxEfITovUVTM2FHdP
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/16/2016 10:53 PM, Walter Bright wrote:

 On 10/16/2016 6:55 AM, Dicebot wrote:

 This snippet currently compiles, failing to reject both "leak" lines.


=20
 It compiles because 'scope' is inferred for both leak1 and leak2.


Where does it come from? I don't see any mention of such deduction in
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md and it doesn't
look like a good idea at all.



--U458ROC3lnDWdCSgJxEfITovUVTM2FHdP--
Oct 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2016 1:03 PM, Dicebot wrote:

 Where does it come from?


It comes from trying to compile code with the new safety checks, and having it 
fail to compile and require adding annotations all over the place. The idea is 
to infer such annotations in the obvious places. 'return' was already 
successfully inferred in many places, this just extends an existing practice.



 I don't see any mention of such deduction in
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


It's one of those things that becomes necessary once trying to implement it.



 and it doesn't look like a good idea at all.


Inferring 'scope' and 'return' as much as possible makes  safe much more 
palatable. All I can say is try to break it!
Oct 16 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing: RC pointer snippet
References: <nu00a6$t5i$1 digitalmars.com> <nu00tf$u0a$1 digitalmars.com>
 <nu0lrm$1tm1$1 digitalmars.com> <nu0me7$1umq$1 digitalmars.com>
 <nu0n44$1vmf$1 digitalmars.com>
In-Reply-To: <nu0n44$1vmf$1 digitalmars.com>

--xVnORgmRRjBGIbpf4awD8kdCOumKN6k5p
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/16/2016 11:15 PM, Walter Bright wrote:

 On 10/16/2016 1:03 PM, Dicebot wrote:

 Where does it come from?


=20
 It comes from trying to compile code with the new safety checks, and
 having it fail to compile and require adding annotations all over the
 place. The idea is to infer such annotations in the obvious places.
 'return' was already successfully inferred in many places, this just
 extends an existing practice.
=20
=20

 I don't see any mention of such deduction in
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


=20
 It's one of those things that becomes necessary once trying to implemen=


t

 it.


Walter, there is no way we can merge your PR if it implements things not
mentioned in DIP. If you don't want to spend time adjusting it yourself,
please, at least notify me about what has changed. That will save both
of us a lot of time.


 and it doesn't look like a good idea at all.


=20
 Inferring 'scope' and 'return' as much as possible makes  safe much mor=


e

 palatable. All I can say is try to break it!


It also makes reasoning about what your code does much harder, adding
unacceptable amount of magic to it. I'd much prefer to rely on `auto`
and fix rest of cases myself through deprecations than go with current
"helpful" behavior.

Such semantics are also unprecedented in D as far as I can remember.
Before storage classes would never be inferred for variables with
explicit type. That is highly confusing.

Right now I think this is a bad decision and should be reconsidered.
Would like to now what other developers thing though.


--xVnORgmRRjBGIbpf4awD8kdCOumKN6k5p--
Oct 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2016 9:32 AM, Dicebot wrote:

 On 10/16/2016 11:15 PM, Walter Bright wrote:

 On 10/16/2016 1:03 PM, Dicebot wrote:

 Where does it come from?


 It comes from trying to compile code with the new safety checks, and
 having it fail to compile and require adding annotations all over the
 place. The idea is to infer such annotations in the obvious places.
 'return' was already successfully inferred in many places, this just
 extends an existing practice.



 I don't see any mention of such deduction in
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


 It's one of those things that becomes necessary once trying to implement
 it.


 Walter, there is no way we can merge your PR if it implements things not
 mentioned in DIP. If you don't want to spend time adjusting it yourself,
 please, at least notify me about what has changed. That will save both
 of us a lot of time.


Ok. But it's pretty simple. A local variable that is initialized with a scoped 
value is tagged with scope.



 Inferring 'scope' and 'return' as much as possible makes  safe much more
 palatable. All I can say is try to break it!


 It also makes reasoning about what your code does much harder, adding
 unacceptable amount of magic to it. I'd much prefer to rely on `auto`
 and fix rest of cases myself through deprecations than go with current
 "helpful" behavior.

 Such semantics are also unprecedented in D as far as I can remember.
 Before storage classes would never be inferred for variables with
 explicit type. That is highly confusing.


It's already done by DIP25, and D has been very successful in inferring  safe, 
 trusted,  system, pure,  nogc, and nothrow. I'd like to go even further and 
infer const, but that's a more complex task.



 Right now I think this is a bad decision and should be reconsidered.
 Would like to now what other developers thing though.


It's long been my belief that if lifetime annotations had to be explicit, few 
would be interested in using such a "bondage and discipline" language. As 
further evidence of that, Rust goes to great lengths to infer lifetime 
attributes. Annotations don't excite people.
Oct 20 2016
parent reply meppl <mephisto nordhoff-online.de> writes:
On Thursday, 20 October 2016 at 21:43:59 UTC, Walter Bright wrote:

 It's long been my belief that if lifetime annotations had to be 
 explicit, few would be interested in using such a "bondage and 
 discipline" language. As further evidence of that, Rust goes to 
 great lengths to infer lifetime attributes. Annotations don't 
 excite people.


explicit lifetime annotations...


isnt it possible to upgrade this scope-annotation system 
afterwards?


for example:  hidden scope
after dip1000 is implemented, a new storage_class gets introduced 
- lets say „jailed“.
If „jailed“ appears in code, the compiler scans the AST for the 
route the pointer takes within one module or within one package
and adds the needed hidden_scope annotations automatically.

it would look like that:
void func( T* t)  safe;  // second version of this function 
exists:
                          // void func( hidden_scope T* t)  safe;
void bar( jailed T* t)  safe {
    func(t); // ok, compiler added hidden_scope for void func( T* 
t) itself
}
how it looks like in DIP1000:
void func(scope T* t)  safe;
void bar(scope T* t)  safe {
    func(t); // ok
}

if an opaque function (of different module/package) appears, that 
opaque function must have the scope/jailed annotation again.
Oct 20 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Friday, 21 October 2016 at 01:46:20 UTC, meppl wrote:

 void bar( jailed T* t)  safe {
    func(t); // ok, compiler added hidden_scope for void func( 
 T* t) itself
 }


As said in DIP1000 and implemented in the PR, the compiler infers 
scope for lambda and template function parameters. Not sure what 
you're proposing.
For sure we want to extend any attribute inferrence to functions, 
but how to make that work with separate compilation is still 
undecided. Talking more about inferrence would be an OT 
discussion though.
Oct 21 2016
next sibling parent meppl <mephisto nordhoff-online.de> writes:
On Friday, 21 October 2016 at 12:04:33 UTC, Martin Nowak wrote:

 On Friday, 21 October 2016 at 01:46:20 UTC, meppl wrote:

 void bar( jailed T* t)  safe {
    func(t); // ok, compiler added hidden_scope for void func( 
 T* t) itself
 }


 As said in DIP1000 and implemented in the PR, the compiler 
 infers scope for lambda and template function parameters. Not 
 sure what you're proposing.
 For sure we want to extend any attribute inferrence to 
 functions, but how to make that work with separate compilation 
 is still undecided. Talking more about inferrence would be an 
 OT discussion though.


I just assumed that inference for DIP1000 (for non-template 
private functions) is _possible_ to implement afterwards with 
backward compatibility. I want to know, if it is possible, 
because DIP1000 tells people to scatter another keyword („scope“) 
all over the place in the code.
Oct 21 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 5:04 AM, Martin Nowak wrote:

 For sure we want to extend any attribute inferrence to functions, but how to
 make that work with separate compilation is still undecided. Talking more about
 inferrence would be an OT discussion though.


I know how to make it (inference on regular functions) work, but it is a 
separate topic, and would only further confuse things here.
Oct 22 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: DIP1000 discussion and testing: borrowing a range
References: <nu00a6$t5i$1 digitalmars.com>
In-Reply-To: <nu00a6$t5i$1 digitalmars.com>

--LiNlUgC1X3TLk2kb3sSe8FRpCr6wGUpqb
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/16/2016 04:45 PM, Dicebot wrote:

 I'll start separate sub-threads with more details.


This is probably most ambitious snippet I have come up with so far. It
seems to be possible within current DIP1000 definition but doesn't work
for other reasons I wanted to discuss:

------------------------------------------------------------
 safe unittest
{
     safe static struct Owner
    {
        int data;

        // must bind lifetime of return pointer/ref to one of `this`
	// Error: function has 'return' but does not return any
 	// indirections
        auto borrow () return scope
        {
            struct Range
            {
                int* data;
                enum empty =3D false;
                auto front () { return *this.data; }
                void popFront() { (*this.data)++; }
            }

            return Range(&this.data); // Error: cannot take address of
                                      // parameter this in  safe
function borrow
        }
    }

    Owner instance;
    // must work, lifetime of 'instance' exceeds one of x :
    scope x =3D instance.borrow();
    // must not work, infinite lifetime
    auto y  =3D instance.borrow();
}
------------------------------------------------------------

First error simply to be simply a bug, because it contradicts "struct is
a composition of its fields" statement from DIP1000.

Second error is an existing limitation of  safe code (can't take pointer
from ref) which seems extremely limiting now that tools to enforce scope
lifetime come into play.


--LiNlUgC1X3TLk2kb3sSe8FRpCr6wGUpqb--
Oct 16 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2016 7:02 AM, Dicebot wrote:

     Owner instance;
     // must work, lifetime of 'instance' exceeds one of x :
     scope x = instance.borrow();
     // must not work, infinite lifetime
     auto y  = instance.borrow();
 }



 Second error is an existing limitation of  safe code (can't take pointer
 from ref) which seems extremely limiting now that tools to enforce scope
 lifetime come into play.


It works because 'scope' is inferred as a storage class when an expression that 
needs scope is used to initialize a local. y gets a subset of the life of 
instance. Doing such inference makes using -transition=safe far more palatable.

BTW, reducing the examples to their minimums helps a lot.
Oct 16 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing: borrowing a range
References: <nu00a6$t5i$1 digitalmars.com> <nu01af$upk$1 digitalmars.com>
 <nu0khd$1rja$1 digitalmars.com>
In-Reply-To: <nu0khd$1rja$1 digitalmars.com>

--nrGwsjB1wN20noea94PIuQLJGotE1S75o
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/16/2016 10:30 PM, Walter Bright wrote:

 On 10/16/2016 7:02 AM, Dicebot wrote:

     Owner instance;
     // must work, lifetime of 'instance' exceeds one of x :
     scope x =3D instance.borrow();
     // must not work, infinite lifetime
     auto y  =3D instance.borrow();
 }





 It works because 'scope' is inferred as a storage class when an
 expression that needs scope is used to initialize a local. y gets a
 subset of the life of instance. Doing such inference makes using
 -transition=3Dsafe far more palatable.


Great, thanks for explanation, this will need to be mentioned clearly
with example in DIP document (I'll make a PR once other uncertainties
are cleared too).

However, it also keeps compiling if I change it to use explicit
non-scope type:

// move Range definition outside and mark `borrow`  trusted
Range y =3D instance.borrow(); // compiles, but must require `scope Range=
 y`


 BTW, reducing the examples to their minimums helps a lot.


This helps in debugging but doesn't help in defining good acceptance
tests. I am looking for the latter and that means "minimal snippet that
looks like useful code", not "minimal reduced test case" - and this is
exactly what is needed as part of scope PR test cases to evaluate it.

Now, I would be glad to provide minimal test cases for ease of debugging
too, but that is not possible until I can reason which behavior is bug
and which one is intentional. Getting your comments about why certain
things don't work helps to understand.


--nrGwsjB1wN20noea94PIuQLJGotE1S75o--
Oct 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2016 1:00 PM, Dicebot wrote:

 Range y = instance.borrow(); // compiles, but must require `scope Range y`


'scope' is inferred for 'y'.



 Now, I would be glad to provide minimal test cases for ease of debugging
 too, but that is not possible until I can reason which behavior is bug
 and which one is intentional. Getting your comments about why certain
 things don't work helps to understand.


When saying "this should compile" or "this should not compile" a minimized 
example is best when reasoning about it. Once we have understanding on what 
should and should not compile, a more meta example exploiting this would then
be 
useful.
Oct 16 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing: borrowing a range
References: <nu00a6$t5i$1 digitalmars.com> <nu01af$upk$1 digitalmars.com>
 <nu0khd$1rja$1 digitalmars.com>
In-Reply-To: <nu0khd$1rja$1 digitalmars.com>

--btH5bvnHD6apbRFnqrGaQlnvXnQBm6Slu
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/16/2016 10:30 PM, Walter Bright wrote:

 It works because 'scope' is inferred as a storage class when an
 expression that needs scope is used to initialize a local. y gets a
 subset of the life of instance. Doing such inference makes using
 -transition=3Dsafe far more palatable.
=20
 BTW, reducing the examples to their minimums helps a lot.


This compiles too:

```
int* x;

 safe unittest
{
     safe static struct S
    {
        int data;

        int* borrow () return scope  trusted {
            return &this.data;
        }
    }

    S instance;
    x =3D instance.borrow();
}
```

Now, _this_ must be wrong, right?


--btH5bvnHD6apbRFnqrGaQlnvXnQBm6Slu--
Oct 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2016 9:44 AM, Dicebot wrote:

 This compiles too:

 ```
 int* x;

  safe unittest
 {
      safe static struct S
     {
         int data;

         int* borrow () return scope  trusted {
             return &this.data;
         }
     }

     S instance;
     x = instance.borrow();
 }
 ```

 Now, _this_ must be wrong, right?



It compiles because you added ' trusted' to the bad boy function, disabling the 
check preventing taking the address of a 'ref'. The compiler has no reason to 
believe that the return value of instance.borrow() has any relationship to 
'instance' - it  trusted the programmer to not do that!

The practical result is that a container that is passed by ref can only control 
its uses by returning by ref, not by *.
Oct 20 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing: borrowing a range
References: <nu00a6$t5i$1 digitalmars.com> <nu01af$upk$1 digitalmars.com>
 <nu0khd$1rja$1 digitalmars.com> <nuas8n$30f2$1 digitalmars.com>
 <nuc827$2dj2$1 digitalmars.com>
In-Reply-To: <nuc827$2dj2$1 digitalmars.com>

--JOpCCw1jLSMXRFu7hSrOOwshdfK4Gji8V
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/21/2016 08:11 AM, Walter Bright wrote:

 On 10/20/2016 9:44 AM, Dicebot wrote:

 This compiles too:

 ```
 int* x;

  safe unittest
 {
      safe static struct S
     {
         int data;

         int* borrow () return scope  trusted {
             return &this.data;
         }
     }

     S instance;
     x =3D instance.borrow();
 }
 ```

 Now, _this_ must be wrong, right?


=20
=20
 It compiles because you added ' trusted' to the bad boy function,
 disabling the check preventing taking the address of a 'ref'. The
 compiler has no reason to believe that the return value of
 instance.borrow() has any relationship to 'instance' - it  trusted the
 programmer to not do that!


Wait what? This is in direct contradiction with what DIP1000 states and
what I have been trying to write example of all this time. By existing
specification annotating method with `return scope` means that lifetime
of return value is same as that method `this`.

Are you saying that existing system forgets that `instance` is `this`
the very moment `borrow` methods finishes? If yes, I am afraid it is a
rather useless feature, much worse than one may think reading DIP1000.


 The practical result is that a container that is passed by ref can only=



 control its uses by returning by ref, not by *.


1) Why? You still haven't answered how scope pointer is different from
ref in  safe code.
2) Does that imply that lifetime algebra as described in DIP1000 only
applies to function internal arguments and variables? As it is not
possible to neither have `ref` variables, nor take address of `ref`
return value, lifetime of return value will always be limited to single
expression.


--JOpCCw1jLSMXRFu7hSrOOwshdfK4Gji8V--
Oct 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2016 10:46 PM, Dicebot wrote:

 Wait what?


I've said it multiple times. 'ref' cannot be converted to '*' and be safe.
Hence 
the compiler disallows it.

This is exactly why we need to start with trivial examples.



 This is in direct contradiction with what DIP1000 states and
 what I have been trying to write example of all this time.


I know what you've been trying to write, and each time I point out the same
thing.



 By existing
 specification annotating method with `return scope` means that lifetime
 of return value is same as that method `this`.


If this is a class this then return scope is appropriate, because this is a 
pointer. But when this is a ref, scope then applies to the struct's fields.



 Are you saying that existing system forgets that `instance` is `this`
 the very moment `borrow` methods finishes? If yes, I am afraid it is a
 rather useless feature, much worse than one may think reading DIP1000.


By making borrow  trusted in order to defeat the checking, you defeated the 
connection between 'instance' the argument and the return value from the
function.

Do not convert 'ref' to '*'. It won't work.



 The practical result is that a container that is passed by ref can only
 control its uses by returning by ref, not by *.


 1) Why? You still haven't answered how scope pointer is different from
 ref in  safe code.


I've been trying to answer. There are two things at work for variable 'v':

1. v's address
2. v's value

'ref' deals with the first, 'scope' with the second. It is very, very, very 
important to think about whether you are dealing with v's address or v's value, 
otherwise you will get hopelessly lost.

Next, recall that 'this' for a struct is an address. 'this' for a class is a
value.


 2) Does that imply that lifetime algebra as described in DIP1000 only
 applies to function internal arguments and variables? As it is not
 possible to neither have `ref` variables, nor take address of `ref`
 return value, lifetime of return value will always be limited to single
 expression.


That's what scope variables are for - storing addresses in.

I implore you to look at the test cases with the PR. They are all trivial and 
easy to follow.
Oct 22 2016
parent Martin Nowak <code dawg.eu> writes:
On Saturday, 22 October 2016 at 07:38:18 UTC, Walter Bright wrote:

 If this is a class this then return scope is appropriate, 
 because this is a pointer. But when this is a ref, scope then 
 applies to the struct's fields.


Mmh, and if wanted to return a reference to the struct itself, 
we'd need to resort to return ref? Would it be possible to 
conflate return scope and return ref, at least for that use case?


 Are you saying that existing system forgets that `instance` is 
 `this`
 the very moment `borrow` methods finishes? If yes, I am afraid 
 it is a
 rather useless feature, much worse than one may think reading 
 DIP1000.


 By making borrow  trusted in order to defeat the checking, you 
 defeated the connection between 'instance' the argument and the 
 return value from the function.

 Do not convert 'ref' to '*'. It won't work.


As said further down in this thread, it seems that returning 
casted refs (or structs containing pointers) that alias internal 
memory, but are bound/scoped to the lifetime of the comtainer, is 
a major use-case. Why can't we use this explicit scope annotation 
on a trusted method to make this work?


 The practical result is that a container that is passed by 
 ref can only
 control its uses by returning by ref, not by *.


 1) Why? You still haven't answered how scope pointer is 
 different from
 ref in  safe code.


 I've been trying to answer. There are two things at work for 
 variable 'v':

 1. v's address
 2. v's value

 'ref' deals with the first, 'scope' with the second. It is 
 very, very, very important to think about whether you are 
 dealing with v's address or v's value, otherwise you will get 
 hopelessly lost.


Could you elaborate a bit on that?


 Next, recall that 'this' for a struct is an address. 'this' for 
 a class is a value.


The other way round, right?


 That's what scope variables are for - storing addresses in.

 I implore you to look at the test cases with the PR. They are 
 all trivial and easy to follow.


I hope you agree that we must check the design against our actual 
high level goals. While the breaking down the rules is a good 
approach for a formal view, being able to puzzle together the 
pieces to useful idioms is essential.
Oct 26 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2016 7:02 AM, Dicebot wrote:

 	// [1] Error: function has 'return' but does not return any
  	// indirections
         auto borrow () return scope
         {
             struct Range
             {
                 int* data;
             }

             return Range(&this.data); // [2] Error: cannot take address of
                                       // parameter this in  safe
 function borrow
         }

 First error [1] simply to be simply a bug, because it contradicts "struct is
 a composition of its fields" statement from DIP1000.


I'd ignore that error because it is a cascaded error.

(It is helpful to label errors so it's easier to see which you're referring to. 
I added such annotations to the quoted part.)

As for error [2], 'ref' is meant to be a restricted pointer. Converting a 
restricted pointer to a regular pointer, which is what &this does, undermines 
the whole point of having 'ref', and so is disallowed in  safe code.

Consider:

    safe int* foo(return ref int r) {
     return &r; // Error: cannot take address of parameter r in  safe function
foo
   }

Making it  trusted works. But even if  trusted, escape checking is still done, 
even in  system code:

    trusted int* foo(return ref int r) {
     return &r; // no error, 'cuz it's trusted
   }

   int* bar() {
     int r;
     return foo(r); // Error: escaping reference to local variable r
   }
Oct 16 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing: borrowing a range
References: <nu00a6$t5i$1 digitalmars.com> <nu01af$upk$1 digitalmars.com>
 <nu0lop$1thf$1 digitalmars.com>
In-Reply-To: <nu0lop$1thf$1 digitalmars.com>

--ET4EUDLrjMcIvwXHrpAKeXu9CQsi48MI2
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/16/2016 10:51 PM, Walter Bright wrote:

 As for error [2], 'ref' is meant to be a restricted pointer. Converting=



 a restricted pointer to a regular pointer, which is what &this does,
 undermines the whole point of having 'ref', and so is disallowed in
  safe code.
=20
 Consider:
=20
    safe int* foo(return ref int r) {
     return &r; // Error: cannot take address of parameter r in  safe
 function foo
   }


This must not compile indeed. But this is perfectly fine:

 safe scope int* foo (return scope ref int r) {
    return &r; // taking address of r is fine because returned pointer
can't outlive the reference
}

As far as I can see right now, from the safety PoV scoped pointer is
strictly equivalent to ref.


--ET4EUDLrjMcIvwXHrpAKeXu9CQsi48MI2--
Oct 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2016 9:37 AM, Dicebot wrote:

 But this is perfectly fine:

  safe scope int* foo (return scope ref int r) {
     return &r; // taking address of r is fine because returned pointer
 can't outlive the reference
 }


Nope:

   Error: function foo functions cannot be scope

If foo was a member function, the 'scope' would be allowed and would apply to 
the 'this' reference.
Oct 20 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Friday, 21 October 2016 at 06:09:36 UTC, Walter Bright wrote:

  safe scope int* foo (return scope ref int r) {
     return &r; // taking address of r is fine because returned 
 pointer
 can't outlive the reference
 }


 Nope:

   Error: function foo functions cannot be scope


Scope was supposed to apply to the return value, comare 
https://github.com/dlang/DIPs/blob/731255ed7dc37d596fec0552013f0e12c68f7bea/DIPs/DIP1000.md#scope-function-returns.

Let's maybe work with a better equivalent example.

 safe scope Klass get(return scope ref Unique!Klass);
Oct 21 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 10:38 AM, Martin Nowak wrote:

 Scope was supposed to apply to the return value, comare
 https://github.com/dlang/DIPs/blob/731255ed7dc37d596fec0552013f0e12c68f7bea/DIPs/DIP1000.md#scope-function-returns.


 Let's maybe work with a better equivalent example.

  safe scope Klass get(return scope ref Unique!Klass);


Please use simple examples. 'scope' has nothing to do with templates, and
adding 
in templates just distracts and confuses.
Oct 22 2016
prev sibling next sibling parent Kagamin <spam here.lot> writes:
What about mandatory  system attribute? 
https://github.com/dlang/dmd/pull/5972#issuecomment-242227591
Oct 19 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 16 October 2016 at 13:45:42 UTC, Dicebot wrote:

 So, there is a pull request 
 (https://github.com/dlang/dmd/pull/5972)
 which is intended to implement DIP1000
 (https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md).


Can we first get a high level overview about existing discussions 
and the goals/problems of DIP1000.
Not sure this forum is the right place for detailed DIP/PR 
reviews.

The DIP is named Scoped Pointers, but seems to address any 
escaping references (including classes).
Will it fully enable  safe reference counted memory management?
What pieces of the puzzle are still missing?


I read that RC `opAssign` and `destroy` would remain unsafe b/c 
destroying the owner might leave a dangling ref parameter in 
`fun(ref RCS rc, ref int ri)`.
It seems that unsafe assignment/destruction of RCs would be very 
limiting.
Do we at least have an idea how to tackle this problem? In the 
RCClass idea an additional addRef/decRef call was proposed.
Also see 
https://github.com/dlang/DIPs/pull/35#issuecomment-252345548.
Oct 19 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/19/2016 10:48 AM, Martin Nowak wrote:

 Can we first get a high level overview about existing discussions and the 


goals/problems of DIP1000.

Simply put, it provides the means to guarantee that a pointer supplied as a 
function argument does not escape the scope of that function. Special 
consideration is given to functions that return their arguments.



 The DIP is named Scoped Pointers, but seems to address any escaping references
 (including classes).


It includes classes, delegates, and dynamic arrays.



 Will it fully enable  safe reference counted memory management?


No, because an RC design also requires the preincrement as noted below.



 What pieces of the puzzle are still missing?


The preincrement thing, but that is not DIP1000.






All.



 I read that RC `opAssign` and `destroy` would remain unsafe b/c destroying the
 owner might leave a dangling ref parameter in `fun(ref RCS rc, ref int ri)`.
 It seems that unsafe assignment/destruction of RCs would be very limiting.
 Do we at least have an idea how to tackle this problem? In the RCClass idea an
 additional addRef/decRef call was proposed.
 Also see https://github.com/dlang/DIPs/pull/35#issuecomment-252345548.


The solution to that has been proposed and forgotten a couple of times. It is
to 
have the compiler insert code to preemptively increment the reference count, 
then reassigning the RC object will not invalidate references to its internals. 
This is beyond the scope of DIP1000, though. DIP1000 is necessary for memory 
safety even without reference counting.
Oct 20 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Thursday, 20 October 2016 at 09:00:02 UTC, Walter Bright wrote:

 I read that RC `opAssign` and `destroy` would remain unsafe 
 b/c destroying the
 owner might leave a dangling ref parameter in `fun(ref RCS rc, 
 ref int ri)`.
 It seems that unsafe assignment/destruction of RCs would be 
 very limiting.
 Do we at least have an idea how to tackle this problem? In the 
 RCClass idea an
 additional addRef/decRef call was proposed.
 Also see 
 https://github.com/dlang/DIPs/pull/35#issuecomment-252345548.


 The solution to that has been proposed and forgotten a couple 
 of times. It is to have the compiler insert code to 
 preemptively increment the reference count, then reassigning 
 the RC object will not invalidate references to its internals. 
 This is beyond the scope of DIP1000, though. DIP1000 is 
 necessary for memory safety even without reference counting.


Thought a bit more about this.
The essence of the problem is a simple pointer aliasing problem, 
namely calling
     void dangling(ref A a, ref B b)
is unsafe if A is a "container" that could own a B.
This does not only apply to RC but also things like arrays with 
deterministic MM, unions/variants (changing type of b's pointee), 
or implementations of Nullable/Optional using heap allocated 
memory.

Fortunately any pointer aliasing through untyped memory/pointers 
in A (ubyte[], void*) can only result in a dangling b pointer, if 
doing unsafe operations on a (or incorrectly marked  trusted 
ones).
For pointer aliasing through typed fields of A, the compiler can 
detect a possible aliasing and mark the function call as unsafe.
That would allow operations on a that could free Bs to be 
 trustable.

This aliasing problem seems only slightly related to scope/escape 
analysis.
A container using scope in a  safe manner, should only escape 
scoped references.
So the statement of the problem is actually
   void dangling(ref A a, scope ref B b).

Copying a before the call couldn't solve all those problems, e.g. 
Unique isn't copyable, a union/variant could be in a reference 
type field of A.

But it seems that we can detect all such unsafe function calls 
and the problem only scratches on scope b/c we want to know 
whether the lifetime of b might be limited attached to a. So 
seems indeed fair enough to leave this aside for now.

What would be the plan for RC.opAssign? Making it  trusted after 
DIP1000 and implementing the aliasing detection later,  trusted 
opAssign but conservatively  unsafe any call with multiple 
references (at least one of which being scoped), or  system 
opAssign and changing it to  trusted later?

It seems that the aliasing detect wouldn't be too hard to 
implememt (even implementable as druntime template similar to 
std.traits.hasAliasing). Am I right that conservatively detecting 
the aliasing would allow a slightly limited subset of  safe RC 
usage?
If so going w/  trusted RC.opAssign and the aliasing detection 
seems like a good milestone for 2.073.
Oct 21 2016
next sibling parent Martin Nowak <code dawg.eu> writes:
On Friday, 21 October 2016 at 10:11:28 UTC, Martin Nowak wrote:

 It seems that the aliasing detect wouldn't be too hard to 
 implememt (even implementable as druntime template similar to 
 std.traits.hasAliasing). Am I right that conservatively 
 detecting the aliasing would allow a slightly limited subset of 
  safe RC usage?
 If so going w/  trusted RC.opAssign and the aliasing detection 
 seems like a good milestone for 2.073.


Nice side effect of 
http://forum.dlang.org/post/fzawzedvokbvhmdspxsh forum.dlang.org, 
it's now clear that a good (not too conservative) aliasing 
detection is also needed for the lifetime of function return 
values. So we can reuse that for detecting the dangling ref 
problem.
Oct 21 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 3:11 AM, Martin Nowak wrote:

 What would be the plan for RC.opAssign?


The plan is rather blunt. Andrei is going to design an RC object, and then I am 
going to figure out how to make it work (because it will need a bit of compiler 
help, specifically, dealing with the pre-increment case).

You are correct in that RC is only tangentially related to scope.
Oct 22 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 16 October 2016 at 13:45:42 UTC, Dicebot wrote:

 Problem is that Walter wants to prioritize moving on with it 
 (which is reasonable) but the PR is too complicated to be 
 reviewed by most other developers.


Just through with my first read of DIP1000.
First impressions interesting, incoherent, might or might not 
solve our problems.
Will give it some more time in the next few days.
But IMHO we should ensure that we have sound specs (backed by 
tests) that solve the __right__ use-cases before discussing 
implementations. As part of the implementation we'll have to 
transform this into a more easily understandable spec anyhow, so 
we might as well do that in the beginning.
Oct 19 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/19/2016 3:12 PM, Martin Nowak wrote:

 On Sunday, 16 October 2016 at 13:45:42 UTC, Dicebot wrote:

 Problem is that Walter wants to prioritize moving on with it (which is
 reasonable) but the PR is too complicated to be reviewed by most other
 developers.


 Just through with my first read of DIP1000.
 First impressions interesting, incoherent, might or might not solve our
problems.
 Will give it some more time in the next few days.
 But IMHO we should ensure that we have sound specs (backed by tests) that solve
 the __right__ use-cases before discussing implementations. As part of the
 implementation we'll have to transform this into a more easily understandable
 spec anyhow, so we might as well do that in the beginning.


I suggest looking over the test cases that come with the PR. They are 
straightforward. Whether they are the right use cases or not is debatable, but 
they each illustrate an existing safety hole in D today that the PR plugs.
Oct 20 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
In-Reply-To: <nua19j$1fvb$1 digitalmars.com>

--40X9ck5NCmx7OFRF0Wkj3q1McsEq3Ddam
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/20/2016 12:03 PM, Walter Bright wrote:

 I suggest looking over the test cases that come with the PR. They are
 straightforward. Whether they are the right use cases or not is
 debatable, but they each illustrate an existing safety hole in D today
 that the PR plugs.


If I was to judge by available test cases only, I would have decided
DIP1000 is waste of time and brings nothing useful to the table. They
are terrible in showing implementation goals being low level regression
testing artifacts.

Really, do you think anyone will ever be impressed seeing code like
`int* foo1(return scope int* p) { return p; }`? DIP1000 as defined can
allow much more interesting things and tests should show that too, not
just synthetic artifacts.


--40X9ck5NCmx7OFRF0Wkj3q1McsEq3Ddam--
Oct 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2016 12:27 PM, Dicebot wrote:

 If I was to judge by available test cases only, I would have decided
 DIP1000 is waste of time and brings nothing useful to the table. They
 are terrible in showing implementation goals being low level regression
 testing artifacts.

 Really, do you think anyone will ever be impressed seeing code like
 `int* foo1(return scope int* p) { return p; }`? DIP1000 as defined can
 allow much more interesting things and tests should show that too, not
 just synthetic artifacts.


They are not meant to impress people, nor are they regression tests. They 
illustrate closing of existing safety holes.

   int* bar()
   {
     int p;
     return foo1(&p);
   }

is a currently undetected bug. DIP1000 detects it. DIP1000 also closes safety 
bugs that you posted to Bugzilla (listed in the PR for it).

D must have these fixes, one way or another.

Yes, the test cases are very minimal. But understanding them is necessary in 
order to understand more complex cases - I don't think we've gotten anywhere by 
using complex cases to understand the mechanics of it.
Oct 20 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
In-Reply-To: <nubddb$ulr$1 digitalmars.com>

--DcGiQ6EdFx2bc2mf2kVpEjlvmI86q51u7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/21/2016 12:36 AM, Walter Bright wrote:

 On 10/20/2016 12:27 PM, Dicebot wrote:

 If I was to judge by available test cases only, I would have decided
 DIP1000 is waste of time and brings nothing useful to the table. They
 are terrible in showing implementation goals being low level regressio=




n

 testing artifacts.

 Really, do you think anyone will ever be impressed seeing code like
 `int* foo1(return scope int* p) { return p; }`? DIP1000 as defined can=





 allow much more interesting things and tests should show that too, not=





 just synthetic artifacts.


=20
 They are not meant to impress people, nor are they regression tests.
 They illustrate closing of existing safety holes.
=20
   int* bar()
   {
     int p;
     return foo1(&p);
   }
=20
 is a currently undetected bug. DIP1000 detects it. DIP1000 also closes
 safety bugs that you posted to Bugzilla (listed in the PR for it).


Yes, but that is not enough for such a major change. If we want to
finally implement `scope`, it has to be _awesome_, plain good is not
good enough.


 Yes, the test cases are very minimal. But understanding them is
 necessary in order to understand more complex cases - I don't think
 we've gotten anywhere by using complex cases to understand the mechanic=


s

 of it.


Can you give a clear answer regarding borrowing snippets I am posting?
Like one of:

- Such idioms are not supposed to work with DIP1000 at all (why?) and
you only want to focus on fixing existing  safe holes
- It is supposed to work, but I am doing it wrong (please show how to
write it in that case)
- It is supposed to work, but bugs need fixing
- It is supposed to work, but some parts will be implemented later
(which ones?)

So far you have only given answers regarding technical trivialities
without taking actual example to the heart.


--DcGiQ6EdFx2bc2mf2kVpEjlvmI86q51u7--
Oct 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2016 3:52 PM, Dicebot wrote:

 On 10/21/2016 12:36 AM, Walter Bright wrote:

 On 10/20/2016 12:27 PM, Dicebot wrote:

 If I was to judge by available test cases only, I would have decided
 DIP1000 is waste of time and brings nothing useful to the table. They
 are terrible in showing implementation goals being low level regression
 testing artifacts.

 Really, do you think anyone will ever be impressed seeing code like
 `int* foo1(return scope int* p) { return p; }`? DIP1000 as defined can
 allow much more interesting things and tests should show that too, not
 just synthetic artifacts.


 They are not meant to impress people, nor are they regression tests.
 They illustrate closing of existing safety holes.

   int* bar()
   {
     int p;
     return foo1(&p);
   }

 is a currently undetected bug. DIP1000 detects it. DIP1000 also closes
 safety bugs that you posted to Bugzilla (listed in the PR for it).


 Yes, but that is not enough for such a major change. If we want to
 finally implement `scope`, it has to be _awesome_, plain good is not
 good enough.


Actually, I consider it a minor change, and I *want* the fix for this bug to be 
as minor as possible. Furthermore, Andrei, I, Bartosz, and others have
struggled 
with this particular problem for about 10 years. This is the best (and
simplest, 
by far) solution we've come up with. If you can think of a better one, I'm 
interested.



 Yes, the test cases are very minimal. But understanding them is
 necessary in order to understand more complex cases - I don't think
 we've gotten anywhere by using complex cases to understand the mechanics
 of it.


 Can you give a clear answer regarding borrowing snippets I am posting?
 Like one of:

 - Such idioms are not supposed to work with DIP1000 at all (why?) and
 you only want to focus on fixing existing  safe holes
 - It is supposed to work, but I am doing it wrong (please show how to
 write it in that case)
 - It is supposed to work, but bugs need fixing
 - It is supposed to work, but some parts will be implemented later
 (which ones?)

 So far you have only given answers regarding technical trivialities


That is true, but we must reach mutual understanding on how those trivialities 
work before we can do more. So far, we have not.



 without taking actual example to the heart.


1. D has no notion of "borrowing" and describing DIP1000 in those terms is
going 
to mislead. There is no "borrow checker".

2. This design has nothing in common with Rust's design. Any notions brought 
from Rust will mislead.

3. The design utterly relies on a 'ref' value not being returned as a '*'. Any 
attempts to circumvent this will lose the checking. The rationale for this is a 
dramatic simplification of the design.

4. The effects of 'ref' and 'scope' are not transitive.

5. 'scope' is ignored if applied to a type with no indirections (this is 
necessary to support generic code).

6. What does 'return ref scope' mean? If the function returns by ref, then the 
'return' applies to the 'ref'. Otherwise, it applies to the 'scope'.

7. 'scope' applies to the value of a type, 'ref' applies to its address. Your 
example failed to apply 'scope' to the 'int*' return, because the struct had no 
pointer types in it.

8. A container passed by 'ref' can return a 'ref' to the container's value or 
indirect contents, or by '*' to indirect contents, but it cannot return the 
address of the container by '*'. (This is a restatement of item 3.)
Oct 20 2016
next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Friday, 21 October 2016 at 06:04:41 UTC, Walter Bright wrote:

 3. The design utterly relies on a 'ref' value not being 
 returned as a '*'. Any attempts to circumvent this will lose 
 the checking. The rationale for this is a dramatic 
 simplification of the design.


DIP1000 mentions that disallowing &var in  safe code is overly 
restrictive. This got me fairly confused, please help to clarify 
https://github.com/dlang/DIPs/blob/731255ed7dc37d596fec0552013f0e12c68f7bea/DIPs/DIP1000.md
escaping-via-return or my understanding of it.
Oct 21 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 3:21 AM, Martin Nowak wrote:

 On Friday, 21 October 2016 at 06:04:41 UTC, Walter Bright wrote:

 3. The design utterly relies on a 'ref' value not being returned as a '*'. Any
 attempts to circumvent this will lose the checking. The rationale for this is
 a dramatic simplification of the design.


 DIP1000 mentions that disallowing &var in  safe code is overly restrictive.
This
 got me fairly confused, please help to clarify
 https://github.com/dlang/DIPs/blob/731255ed7dc37d596fec0552013f0e12c68f7bea/DIPs/DIP1000.md#escaping-via-return
 or my understanding of it.


In order to simplify the text, I used &t as an example, even though  safe code 
doesn't allow taking the address of a local. But, this is allowed:

     int[10] sa;
     int[] da = sa[];

and is the semantic equivalent of taking the address of a local. But I didn't 
want to confuse things by introducing arrays, I wanted to be reductionist as 
much as possible.

But the examples in the PR tests use the arrays.
Oct 22 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Friday, 21 October 2016 at 06:04:41 UTC, Walter Bright wrote:

 So far you have only given answers regarding technical 
 trivialities


 That is true, but we must reach mutual understanding on how 
 those trivialities work before we can do more. So far, we have 
 not.


I think we all agree on which overall goal we're trying to 
achieve,  safe deterministic memory management.
So the primitives of DIP1000 need to prove that they properly 
cover the reference escaping part of that goal.

1. Default initializers have infinite lifetime.
    lifetime(e.init) = ∞

    Non default initializers are treated as default init followed 
by an assignment.

2. The sticky lifetime axiom.

    Assignment `v = e` changes lifetime(v) to lifetime(e) if v is 
a pointer or reference.

3. The safe reference axiom.

    Can't assign an expression e to a variable v if that
    would change lifetime(v) so that lifetime(v) < reachability(v).

4. The conservative lifetime axiom.

    lifetime(func(args)) = min(∞, argmin_{e ∈ A}(lifetime(e)))
    with A = {a ∈ args | canAlias(returnType(func), typeof(a))}

    The lifetime of a function call expression, is the minimum of 
infinity and the
    lifetime of all arguments that could be referenced by the 
return type.
    (Shouldn't be type here b/c scope is a storage class and not 
part of the type)

5. The conservative escape axiom.

    The reachability of non-scope ref or pointer paramters is 
assumed to be ∞.

    Hence it's not allowed to assign expressions with finite 
lifetime to non-scope parameters, though the compiler might infer 
scope for parameters.

If this works out we guarantee lifetime(v) ≥ reachability(v) in 
 safe code.

The rest should be deducible using the rules from 
https://github.com/dlang/DIPs/blob/731255ed7dc37d596fec0552013f0e12c68f7bea/DIPs/DIP1000.md#aggregates.

The problem that remains could be stated as lifetime tunneling.

Given `v2 = func(v1)` lifetime(v2) = lifetime(v1) according to 5.
Changing lifetime(v1) by `v1 = e` can change lifetime(v2) to < 
reachability(v2).
Oct 21 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Friday, 21 October 2016 at 11:39:27 UTC, Martin Nowak wrote:

 The problem that remains could be stated as lifetime tunneling.

 Given `v2 = func(v1)` lifetime(v2) = lifetime(v1) according to 
 5.
 Changing lifetime(v1) by `v1 = e` can change lifetime(v2) to < 
 reachability(v2).


This doesn't manifest easily because there are no named reference 
variables and converting references to pointers isn't allowed in 
 safe code.
But the void dangling(ref A a, scope ref B b) is one way to run 
into this.
Can we at least enforce that any value/reference returned by 
`func(v1)` cannot be bound to a v2 in the calling scope while 
still referencing v1?
Oct 21 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 4:54 AM, Martin Nowak wrote:

 On Friday, 21 October 2016 at 11:39:27 UTC, Martin Nowak wrote:

 The problem that remains could be stated as lifetime tunneling.

 Given `v2 = func(v1)` lifetime(v2) = lifetime(v1) according to 5.
 Changing lifetime(v1) by `v1 = e` can change lifetime(v2) to <
reachability(v2).


 This doesn't manifest easily because there are no named reference variables and
 converting references to pointers isn't allowed in  safe code.
 But the void dangling(ref A a, scope ref B b) is one way to run into this.
 Can we at least enforce that any value/reference returned by `func(v1)` cannot
 be bound to a v2 in the calling scope while still referencing v1?


I don't really understand the question. Can you recast it in terms of a short 
example?
Oct 22 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
In-Reply-To: <nucb5n$2hv7$1 digitalmars.com>

--Hjurrn9jjE25PFBstrpIBQMciRmKQfCOD
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/21/2016 09:04 AM, Walter Bright wrote:

 On 10/20/2016 3:52 PM, Dicebot wrote:

 So far you have only given answers regarding technical trivialities


=20
 That is true, but we must reach mutual understanding on how those
 trivialities work before we can do more. So far, we have not.


Well, it doesn't work that way. So far all you explanations made zero
sense until I have only finally understood what your intentions are
thanks to next answer ..


 1. D has no notion of "borrowing" and describing DIP1000 in those terms=



 is going to mislead. There is no "borrow checker".
=20
 2. This design has nothing in common with Rust's design. Any notions
 brought from Rust will mislead.
=20
 3. The design utterly relies on a 'ref' value not being returned as a
 '*'. Any attempts to circumvent this will lose the checking. The
 rationale for this is a dramatic simplification of the design.
=20
 4. The effects of 'ref' and 'scope' are not transitive.
=20
 5. 'scope' is ignored if applied to a type with no indirections (this i=


s

 necessary to support generic code).
=20
 6. What does 'return ref scope' mean? If the function returns by ref,
 then the 'return' applies to the 'ref'. Otherwise, it applies to the
 'scope'.
=20
 7. 'scope' applies to the value of a type, 'ref' applies to its address=


=2E

 Your example failed to apply 'scope' to the 'int*' return, because the
 struct had no pointer types in it.
=20
 8. A container passed by 'ref' can return a 'ref' to the container's
 value or indirect contents, or by '*' to indirect contents, but it
 cannot return the address of the container by '*'. (This is a
 restatement of item 3.)


=2E. and means two things for me:

1) DIP1000 has to be changes a lot because it doesn't list most of those
limitation, leading to natural conclusion that borrowing can be emulated
as a user code concept. Most importantly, the statement "The design
utterly relies on a 'ref' value not being returned as a '*'. Any
attempts to circumvent this will lose the checking" is not mentioned at a=
ll.

2) The whole thing is very disappointing and won't revive dead body of
 safe as I hoped it will. It won't allow to fix `Unique`, won't enable
new RNG design Joe has been going, won't allow easy skipping of
reference count inc/dec - the list goes on.

Leaving aside issue 2, I'd like to give a try at updating DIP1000 this
weekend to be more in match with what you have just explained.


--Hjurrn9jjE25PFBstrpIBQMciRmKQfCOD--
Oct 21 2016
next sibling parent reply meppl <mephisto nordhoff-online.de> writes:
On Friday, 21 October 2016 at 15:38:39 UTC, Dicebot wrote:

 2) The whole thing is very disappointing ...
 ... won't allow easy skipping of reference count inc/dec - ...


Isn't that - in the short term - impossible anyway? If a struct 
leaves the scope the reference counter has to be increased. But 
how does compiled code of a library know, if the foreign struct 
is an RC-object - without doing any runtime checks? So, any 
function has to exist twice, one time for RC and one time for 
no-RC?
I mean, it's probably a big topic that doesn't have to be solved 
in that DIP, too.
Oct 21 2016
parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
 <nudcq0$1bio$1 digitalmars.com> <jmlynifzlzgzghwmtogn forum.dlang.org>
In-Reply-To: <jmlynifzlzgzghwmtogn forum.dlang.org>

--oCpUjQ2k5163goje2KJIkGxpJ1m935FID
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/21/2016 09:43 PM, meppl wrote:

 On Friday, 21 October 2016 at 15:38:39 UTC, Dicebot wrote:

 2) The whole thing is very disappointing ...
 ... won't allow easy skipping of reference count inc/dec - ...


=20
 Isn't that - in the short term - impossible anyway? If a struct leaves
 the scope the reference counter has to be increased. But how does
 compiled code of a library know, if the foreign struct is an RC-object =


-

 without doing any runtime checks? So, any function has to exist twice,
 one time for RC and one time for no-RC?
 I mean, it's probably a big topic that doesn't have to be solved in tha=


t

 DIP, too.


It would be possible with even most simple borrowing semantics, see
http://forum.dlang.org/post/nu00tf$u0a$1 digitalmars.com - if one can
bind lifetime of a scope pointer to lifetime of stack variable, it
becomes OK to implicitly convert reference counted struct to scope
pointer for passing to other functions. So you only define function
taking scope pointer if it doesn't store it (and it works with both RC
and other data sourceS) or taking RC pointer explicitly if does indeed
store it.

But with the system Walter is proposing it is indeed impossible.


--oCpUjQ2k5163goje2KJIkGxpJ1m935FID--
Oct 22 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 8:38 AM, Dicebot wrote:

 1) DIP1000 has to be changes a lot because it doesn't list most of those
 limitation, leading to natural conclusion that borrowing can be emulated
 as a user code concept. Most importantly, the statement "The design
 utterly relies on a 'ref' value not being returned as a '*'. Any
 attempts to circumvent this will lose the checking" is not mentioned at all.


I believe they are implicit (taking the & of a 'ref' variable has never been 
allowed in  safe code), but they are worth reiterating. It also is apparent now 
that it should explicitly say it has nothing in common with Rust's scheme.



 2) The whole thing is very disappointing and won't revive dead body of
  safe as I hoped it will.


What it will do is close the numerous  safe holes that have been identified.



 It won't allow to fix `Unique`, won't enable
 new RNG design Joe has been going, won't allow easy skipping of
 reference count inc/dec - the list goes on.


You're right, scope won't help at all in inc/dec skipping. But all is not lost. 
I have written up a short article on how the compiler can do inc/dec skipping, 
but it'll have to wait until we have an actual RC design.

The idea is to have a memory model that always inserts inc/dec, then allow the 
compiler to remove them as long as it can prove it behaves "as if" the inc/dec 
happened.



 Leaving aside issue 2, I'd like to give a try at updating DIP1000 this
 weekend to be more in match with what you have just explained.


Thank you! Your help is invaluable.
Oct 22 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
 <nudcq0$1bio$1 digitalmars.com> <nuf5qv$18ca$1 digitalmars.com>
In-Reply-To: <nuf5qv$18ca$1 digitalmars.com>

--b81vMXWWtGD5UxNjJeQtQQmPtuS1XQCDM
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/22/2016 10:52 AM, Walter Bright wrote:

 It won't allow to fix `Unique`, won't enable
 new RNG design Joe has been going, won't allow easy skipping of
 reference count inc/dec - the list goes on.


=20
 You're right, scope won't help at all in inc/dec skipping. But all is
 not lost. I have written up a short article on how the compiler can do
 inc/dec skipping, but it'll have to wait until we have an actual RC des=


ign.

Beauty of borrowing is that it allows to to inc/dec skipping with no
special RC support from compiler and works the very same for stack
allocated data. What you are going at is simply replacing GC with
(compiler enhanced) RC and that is not good at all.

Would you at least be willing to consider an alternative proposal before
`scope` branch is merged for release? I have a feeling that with
relatively few changes to DIP1000 rules we can both have the cookie and
eat it (i.e. get a way to express non-transitive borrowing), more in
spirit of original proposal by Marc.


--b81vMXWWtGD5UxNjJeQtQQmPtuS1XQCDM--
Oct 22 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/22/2016 3:50 PM, Dicebot wrote:

 Beauty of borrowing is that it allows to to inc/dec skipping with no
 special RC support from compiler and works the very same for stack
 allocated data. What you are going at is simply replacing GC with
 (compiler enhanced) RC and that is not good at all.


I don't see why supporting something beyond RC and GC is necessary.


 Would you at least be willing to consider an alternative proposal before
 `scope` branch is merged for release? I have a feeling that with
 relatively few changes to DIP1000 rules we can both have the cookie and
 eat it (i.e. get a way to express non-transitive borrowing), more in
 spirit of original proposal by Marc.


It'd have to be a big improvement over DIP1000, without sacrificing easy use. I 
find Rust's system to be seriously restrictive, off-putting, and hear many 
reports of users fighting its borrow checker. (I have not written any Rust code 
myself.)

At some point we need to move forward. We are already years late. Another month 
has slipped by.

I am also not convinced that you cannot do what you want to do with DIP1000. I 
do not understand why it is necessary to return the address of a value passed
by 
ref, when one can return by ref.
Oct 23 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
 <nudcq0$1bio$1 digitalmars.com> <nuf5qv$18ca$1 digitalmars.com>
 <nugqg3$ld8$1 digitalmars.com> <nujrme$1pu9$1 digitalmars.com>
In-Reply-To: <nujrme$1pu9$1 digitalmars.com>

--25nCmsKSB6u2n7qhxX7mjXjbrSg0BMd6d
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/24/2016 05:29 AM, Walter Bright wrote:

 On 10/22/2016 3:50 PM, Dicebot wrote:

 Beauty of borrowing is that it allows to to inc/dec skipping with no
 special RC support from compiler and works the very same for stack
 allocated data. What you are going at is simply replacing GC with
 (compiler enhanced) RC and that is not good at all.


=20
 I don't see why supporting something beyond RC and GC is necessary.


Both GC and RC are forms of automatic memory management with unavoidable
associated overhead. Both have their uses but are unacceptable in most
of the system code. Which means that would would either need to
duplicate all Phobos/druntime functions to work with both RC and plain
pointers or it will become even less suitable for low level code than it
is now.


 Would you at least be willing to consider an alternative proposal befo=




re

 `scope` branch is merged for release? I have a feeling that with
 relatively few changes to DIP1000 rules we can both have the cookie an=




d

 eat it (i.e. get a way to express non-transitive borrowing), more in
 spirit of original proposal by Marc.


=20
 It'd have to be a big improvement over DIP1000, without sacrificing eas=


y

 use. I find Rust's system to be seriously restrictive, off-putting, and=



 hear many reports of users fighting its borrow checker. (I have not
 written any Rust code myself.)


The problem is DIP1000 is a lie. I started to rewrite it to match your
implemented semantics but that would simply result in throwing much of
its content away. Some of crucial things DIP1000 promises to deliver but
are not possible with design as in https://github.com/dlang/dmd/pull/5972=
 :

-  system code like std.internal.scopebuffer can be made  safe
- reference counting systems need not adjust the count when passing
references that do not escape

That means that any counter-proposal that actually allows to do those
things would already be considerably better.

Now, I do agree that we don't want to go with amount of complexity Rust
has. Something that makes basic idioms possible at 10% of complexity
price would be a much better trade-off. Thus I'd propose to keep most of
the stuff in DIP1000 as it is (including non-transitivity), but
enhance/tweak lifetime rules, for example:

- allow taking address of a `ref` and define lifetime of such expression
to be same of `ref` lifetime
- `return scope` of a struct methods binds returns value lifetime to one
of the struct instance itself, not composition of its fields

It is of similar complexity as existing proposal but is good enough to
allow design patterns I have been trying to do.


 At some point we need to move forward. We are already years late.
 Another month has slipped by.


Current implementation does not bring us closer to feature rich
compiler-verified safety in any way. What is even worse, going with it
will close the option of using same syntax for something better and we
will be stuck with it forever.

I have a feeling that you completely misunderstand why Rust system has
all the hype. Would it help if I explained some of compiler-verified
safety patterns there? I have put some time into studying Rust last year.=



 I am also not convinced that you cannot do what you want to do with
 DIP1000. I do not understand why it is necessary to return the address
 of a value passed by ref, when one can return by ref.


You are author of the proposal. It is your obligation to convince
everyone that it is possible to do what you claim possible, not mine. If
you think it can be done, show me the code.


--25nCmsKSB6u2n7qhxX7mjXjbrSg0BMd6d--
Oct 23 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/23/2016 11:49 PM, Dicebot wrote:

 On 10/24/2016 05:29 AM, Walter Bright wrote:

 On 10/22/2016 3:50 PM, Dicebot wrote:

 Beauty of borrowing is that it allows to to inc/dec skipping with no
 special RC support from compiler and works the very same for stack
 allocated data. What you are going at is simply replacing GC with
 (compiler enhanced) RC and that is not good at all.


 I don't see why supporting something beyond RC and GC is necessary.


 Both GC and RC are forms of automatic memory management with unavoidable
 associated overhead. Both have their uses but are unacceptable in most
 of the system code. Which means that would would either need to
 duplicate all Phobos/druntime functions to work with both RC and plain
 pointers or it will become even less suitable for low level code than it
 is now.


I'm afraid that is a bit vague. Can you give a more concrete example?



 The problem is DIP1000 is a lie. I started to rewrite it to match your
 implemented semantics but that would simply result in throwing much of
 its content away. Some of crucial things DIP1000 promises to deliver but
 are not possible with design as in https://github.com/dlang/dmd/pull/5972 :

 -  system code like std.internal.scopebuffer can be made  safe
 - reference counting systems need not adjust the count when passing
 references that do not escape

 That means that any counter-proposal that actually allows to do those
 things would already be considerably better.

 Now, I do agree that we don't want to go with amount of complexity Rust
 has. Something that makes basic idioms possible at 10% of complexity
 price would be a much better trade-off. Thus I'd propose to keep most of
 the stuff in DIP1000 as it is (including non-transitivity), but
 enhance/tweak lifetime rules, for example:

 - allow taking address of a `ref` and define lifetime of such expression
 to be same of `ref` lifetime


Again, why? Why can't the function return by 'ref'?



 - `return scope` of a struct methods binds returns value lifetime to one
 of the struct instance itself, not composition of its fields


'ref' return already does that.



 It is of similar complexity as existing proposal but is good enough to
 allow design patterns I have been trying to do.


Your design patterns all hinge on requiring conversion of a ref to a *. Why is 
this necessary, as opposed to using return by 'ref' ?



 At some point we need to move forward. We are already years late.
 Another month has slipped by.


 Current implementation does not bring us closer to feature rich
 compiler-verified safety in any way.


The numerous safety bugs it corrects, some reported by you, argues otherwise.



 What is even worse, going with it
 will close the option of using same syntax for something better and we
 will be stuck with it forever.

 I have a feeling that you completely misunderstand why Rust system has
 all the hype. Would it help if I explained some of compiler-verified
 safety patterns there? I have put some time into studying Rust last year.


Your understanding of Rust is likely better than mine. But I do know (and Rust 
documentation says this) that one has to rethink design patterns in order to 
work with the borrow checker, it cannot just be slapped onto any existing 
pattern, even if that pattern is correct. Trying to implement a Rust pattern in 
D is likely to be incorrect as well.



 I am also not convinced that you cannot do what you want to do with
 DIP1000. I do not understand why it is necessary to return the address
 of a value passed by ref, when one can return by ref.


 You are author of the proposal. It is your obligation to convince
 everyone that it is possible to do what you claim possible, not mine. If
 you think it can be done, show me the code.


I don't know what you have in mind. I already showed you how it can help with 
unique pointers. The only concrete issue you have with it is that ref cannot be 
returned by *. I ask again why is this important as opposed to returning by ref.
Oct 24 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
 <nudcq0$1bio$1 digitalmars.com> <nuf5qv$18ca$1 digitalmars.com>
 <nugqg3$ld8$1 digitalmars.com> <nujrme$1pu9$1 digitalmars.com>
 <nukaue$2d5l$1 digitalmars.com> <nulojm$1li3$1 digitalmars.com>
In-Reply-To: <nulojm$1li3$1 digitalmars.com>

--sbBsSsMcu5C65WDITokEqjPBmejA1VGpf
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/24/2016 09:49 PM, Walter Bright wrote:

 ...


I think it is obvious that there is fundamental misunderstanding on both
sides here about goals and need of the other one, is is better to make a
step back and resolve that before continuing same discussion in circles.

Yesterday we had a very nice live discussion with Martin about current
scope semantics and my concerns about it. He has mentioned a planned
call with you later on same topic - let's wait for that and return to
discussing details after if something will yet remain unclear.


--sbBsSsMcu5C65WDITokEqjPBmejA1VGpf--
Oct 26 2016
parent Martin Nowak <code dawg.eu> writes:
On Wednesday, 26 October 2016 at 09:59:07 UTC, Dicebot wrote:

 Yesterday we had a very nice live discussion with Martin about 
 current scope semantics and my concerns about it. He has 
 mentioned a planned call with you later on same topic - let's 
 wait for that and return to discussing details after if 
 something will yet remain unclear.


Yes, let's do that on tuesday if that suits all of you, will send 
around an invite. I think we're all in the topic now and finally 
close to mutual understanding ;).
Oct 29 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I think I thought of a solution. Currently, the design ignores 'scope' if it is 
applied to a variable with no indirections. But we can make it apply, in that 
the 'scope' of a function's return value is tied to the 'scope' of the
argument, 
even if that argument has no indirections.

This should make Dicebot's design work.
Oct 27 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
 <nudcq0$1bio$1 digitalmars.com> <nuf5qv$18ca$1 digitalmars.com>
 <nugqg3$ld8$1 digitalmars.com> <nujrme$1pu9$1 digitalmars.com>
 <nukaue$2d5l$1 digitalmars.com> <nulojm$1li3$1 digitalmars.com>
 <nusi80$4h0$1 digitalmars.com>
In-Reply-To: <nusi80$4h0$1 digitalmars.com>

--AXhBLeigNXa5Omm19t2D6vhe9olpbrFt1
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/27/2016 12:43 PM, Walter Bright wrote:

 I think I thought of a solution. Currently, the design ignores 'scope'
 if it is applied to a variable with no indirections. But we can make it=



 apply, in that the 'scope' of a function's return value is tied to the
 'scope' of the argument, even if that argument has no indirections.
=20
 This should make Dicebot's design work.


This is the step in the right direction but doesn't fix more general
issue. While RNG engine example doesn't have any indirections, that
won't be true in all cases where one needs to return a scoped range.

Most obvious problem is an owning container type:

struct Tree (T)
{
    private Node!T* head;

    ~this ( ) { delete this.head; }

    TreeRange range ( ); // returned range must not outlive
                         // this struct instance
}

There really needs to be a way to bind returned value lifetime to `this`
explicitly. No special cases, no magic, just one of straightforward use
cases.


--AXhBLeigNXa5Omm19t2D6vhe9olpbrFt1--
Oct 29 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/29/2016 4:30 AM, Dicebot wrote:

 Most obvious problem is an owning container type:

 struct Tree (T)
 {
     private Node!T* head;

     ~this ( ) { delete this.head; }

     TreeRange range ( ); // returned range must not outlive
                          // this struct instance
 }

 There really needs to be a way to bind returned value lifetime to `this`
 explicitly. No special cases, no magic, just one of straightforward use
 cases.


This is covered by my proposed modification:

struct Tree
{
     TreeRange range ( ) scope;
}
Oct 29 2016
parent reply Dicebot <public dicebot.lv> writes:
On Sunday, 30 October 2016 at 02:17:36 UTC, Walter Bright wrote:

 This is covered by my proposed modification:

 struct Tree
 {
     TreeRange range ( ) scope;
 }


Maybe I misunderstand what you propose in that case? You have 
said "if it is applied to a variable with no indirections", but 
`struct Tree` in my example does contain indirection in form of 
`head` pointer (as struct is composition of fields).
Oct 29 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/29/2016 10:33 PM, Dicebot wrote:

 On Sunday, 30 October 2016 at 02:17:36 UTC, Walter Bright wrote:

 This is covered by my proposed modification:

 struct Tree
 {
     TreeRange range ( ) scope;
 }


 Maybe I misunderstand what you propose in that case? You have said "if it is
 applied to a variable with no indirections", but `struct Tree` in my example
 does contain indirection in form of `head` pointer (as struct is composition of
 fields).


Since the caller cannot see inside the function, it must rely on the interface. 
If the returned pointer is or is not derived from the value of the  scope 
parameter is immaterial, the caller will act as if it is - meaning that the 
scope of the returned value will be restricted to being the same scope of the 
argument.

It's just tracking the scope of expressions:

     return x;

     return x + 3;

     return i ? x : y;

so they work through function calls:

     return foo(x);

     return foo(x + 3);

     return foo(x, y);

The 'return scope' and 'return ref' make that work so that the compiler only
has 
to examine the function's interface, not its implementation.
Oct 30 2016
parent reply Dicebot <public dicebot.lv> writes:
On Monday, 31 October 2016 at 06:43:48 UTC, Walter Bright wrote:

 If the returned pointer is or is not derived from the value of 
 the  scope parameter is immaterial, the caller will act as if 
 it is - meaning that the scope of the returned value will be 
 restricted to being the same scope of the argument.


Is it how you want to change the implementation or how you intend 
it to work now? Because currently it acts differently and no 
lifetime restriction ever happens in absence of explicit `return 
scope` annotation:

int* foo ( scope int* );
int* global;

void main () {
     scope int* ptr;
     global = foo(ptr); // compiles
}

-----------------------------

Let's get back to the old snippet as you haven't answered my 
question:

struct Tree
{
     Node* head;

     TreeRange range ( ) scope;
}

Per my understanding, it is roughly equivalent to this:

struct Tree { Node* head; }
TreeRange range ( scope ref Tree this );

With current rules `scope` here is completely ignored because 
`this` is already a reference and thus is not allowed to escape. 
And `TreeRange` lifetime has no relation with `this` lifetime, 
because such relation is only defined by `return scope`/`return 
ref`.

Now in http://forum.dlang.org/post/nusi80$4h0$1 digitalmars.com 
you propose to make so that such `scope` annotation does indeed 
transfer lifetime: ".. the design ignores 'scope' if it is 
applied to a variable with no indirections. But we can make it 
apply". But that does not seem applicable to discussed example as 
struct Tree does contain indirections.

Another issue is that would be completely out of line from 
existing difference between `scope` and `return scope`.
Oct 31 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/31/2016 1:08 AM, Dicebot wrote:

 On Monday, 31 October 2016 at 06:43:48 UTC, Walter Bright wrote:

 If the returned pointer is or is not derived from the value of the  scope
 parameter is immaterial, the caller will act as if it is - meaning that the
 scope of the returned value will be restricted to being the same scope of the
 argument.


 Is it how you want to change the implementation or how you intend it to work
 now? Because currently it acts differently and no lifetime restriction ever
 happens in absence of explicit `return scope` annotation:

 int* foo ( scope int* );
 int* global;

 void main () {
     scope int* ptr;
     global = foo(ptr); // compiles
 }


Amending it as follows:

    safe int* foo ( return scope int* );
   int* global;

    safe void main () {
     scope int* ptr;
     global = foo(ptr); // compiles
   }

Produces:

   ..\dmd bug -transition=safe
   DMD v2.069 DEBUG
   bug.d(6): Error: scope variable ptr assigned to global with longer lifetime

which works as intended and is the current behavior.

'scope' means the value does not escape the function.

'return scope' means the value does not escape the function, but may be
returned.

It's analogous to 'ref' and 'return ref' parameters. It works the same way, 
except it pertains to the value of the argument rather than its address.

I'm a bit baffled why I am unable to explain this.




 -----------------------------

 Let's get back to the old snippet as you haven't answered my question:

 struct Tree
 {
     Node* head;

     TreeRange range ( ) scope;
 }

 Per my understanding, it is roughly equivalent to this:

 struct Tree { Node* head; }
 TreeRange range ( scope ref Tree this );


Yes. Note that you can ALWAYS rewrite member functions as normal functions with 
the 'this' parameter as a regular parameter. There is nothing magically 
different about them, they MUST MUST MUST behave the same way. Otherwise, 
nothing will make sense.



 With current rules `scope` here is completely ignored because `this` is already
 a reference and thus is not allowed to escape. And `TreeRange` lifetime has no
 relation with `this` lifetime, because such relation is only defined by `return
 scope`/`return ref`.


Again, 'ref' refers to the ADDRESS of the argument. 'scope' refers to the VALUE 
of the argument.

1. 'return ref' says that the function returns that parameter by 'ref' (or is 
treated as if it did).

2. 'return scope' says that the function returns that parameter by value (or is 
treated as if it did).

3. 'return ref scope' is treated as 'return ref' and 'scope' if the function 
returns by ref.

4. 'return ref scope' is treated as 'ref' and 'return scope' if the function 
does not return by ref.

Therefore,

     TreeRange range(scope ref Tree this);

is case 4, and the return value of 'range' will have the scope of the value of 
the argument corresponding to 'this', which corresponds to the scope of the 
argument.


 Now in http://forum.dlang.org/post/nusi80$4h0$1 digitalmars.com you propose to
 make so that such `scope` annotation does indeed transfer lifetime: ".. the
 design ignores 'scope' if it is applied to a variable with no indirections. But
 we can make it apply". But that does not seem applicable to discussed example
as
 struct Tree does contain indirections.


My proposed change is to make it apply even if it does not contain indirections.


 Another issue is that would be completely out of line from existing difference
 between `scope` and `return scope`.


I don't think that is at all the case.
Oct 31 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
 <nudcq0$1bio$1 digitalmars.com> <nuf5qv$18ca$1 digitalmars.com>
 <nugqg3$ld8$1 digitalmars.com> <nujrme$1pu9$1 digitalmars.com>
 <nukaue$2d5l$1 digitalmars.com> <nulojm$1li3$1 digitalmars.com>
 <nusi80$4h0$1 digitalmars.com> <nv2197$2n1r$1 digitalmars.com>
 <nv3l7t$208p$1 digitalmars.com> <aihxaamczpyxgrqelxoz forum.dlang.org>
 <nv6p72$1an0$1 digitalmars.com> <ibyhjqcdhhbwzggamoqo forum.dlang.org>
 <nv74ao$1s38$1 digitalmars.com>
In-Reply-To: <nv74ao$1s38$1 digitalmars.com>

--OxX0R3PGWCHXebaSJUrkK6hfrs7abtJvJ
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/31/2016 11:53 AM, Walter Bright wrote:

 I'm a bit baffled why I am unable to explain this.


Walter it is very uncomfortable to see that are trying to explain me my
own words (".. no lifetime restriction ever happens in absence of
explicit `return scope` annotation.. ") as if I don't understand that.
It sometimes feels like you spot some random keywords and completely
ignore actual content of the question.

You are unable to explain simply because you don't explain what was
asked. I know the difference between `scope` and `return scope`, and I
ask questions exactly because I know it and last proposal doesn't make
sense with it.

See below:


 4. 'return ref scope' is treated as 'ref' and 'return scope' if the
 function does not return by ref.
=20
 Therefore,
=20
     TreeRange range(scope ref Tree this);
=20
 is case 4, and the return value of 'range' will have the scope of the
 value of the argument corresponding to 'this', which corresponds to the=



 scope of the argument.


This makes no sense because declaration is not `TreeRange range(return
ref scope Tree this)`, it is `TreeRange range(ref scope Tree this)`
(absence of `return` is critical!)

Does that mean that you have actually meant you first example to look
like this:

struct Tree
{
    TreeRange range () return ref scope
}

?


--OxX0R3PGWCHXebaSJUrkK6hfrs7abtJvJ--
Oct 31 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/31/2016 3:13 AM, Dicebot wrote:

 4. 'return ref scope' is treated as 'ref' and 'return scope' if the
 function does not return by ref.

 Therefore,

     TreeRange range(scope ref Tree this);

 is case 4, and the return value of 'range' will have the scope of the
 value of the argument corresponding to 'this', which corresponds to the
 scope of the argument.


 This makes no sense because declaration is not `TreeRange range(return
 ref scope Tree this)`, it is `TreeRange range(ref scope Tree this)`
 (absence of `return` is critical!)

 Does that mean that you have actually meant you first example to look
 like this:

 struct Tree
 {
     TreeRange range () return ref scope
 }

 ?


Yeah, sorry, the 'return' is necessary as you said.
Oct 31 2016
parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <nubhr3$15k6$1 digitalmars.com> <nucb5n$2hv7$1 digitalmars.com>
 <nudcq0$1bio$1 digitalmars.com> <nuf5qv$18ca$1 digitalmars.com>
 <nugqg3$ld8$1 digitalmars.com> <nujrme$1pu9$1 digitalmars.com>
 <nukaue$2d5l$1 digitalmars.com> <nulojm$1li3$1 digitalmars.com>
 <nusi80$4h0$1 digitalmars.com> <nv2197$2n1r$1 digitalmars.com>
 <nv3l7t$208p$1 digitalmars.com> <aihxaamczpyxgrqelxoz forum.dlang.org>
 <nv6p72$1an0$1 digitalmars.com> <ibyhjqcdhhbwzggamoqo forum.dlang.org>
 <nv74ao$1s38$1 digitalmars.com> <nv75gs$1u5u$1 digitalmars.com>
 <nv8uv0$1kft$1 digitalmars.com>
In-Reply-To: <nv8uv0$1kft$1 digitalmars.com>

--LlJcOU3XAs8gn6QL9IXqv9RH0jOWIbBl7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 11/01/2016 04:34 AM, Walter Bright wrote:

 Yeah, sorry, the 'return' is necessary as you said.


Great, this brings my understanding back to state of harmony again :)


--LlJcOU3XAs8gn6QL9IXqv9RH0jOWIbBl7--
Nov 01 2016
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Monday, 24 October 2016 at 19:49:10 UTC, Walter Bright wrote:

 On 10/23/2016 11:49 PM, Dicebot wrote:

 On 10/24/2016 05:29 AM, Walter Bright wrote:

 On 10/22/2016 3:50 PM, Dicebot wrote:

 Beauty of borrowing is that it allows to to inc/dec skipping 
 with no
 special RC support from compiler and works the very same for 
 stack
 allocated data. What you are going at is simply replacing GC 
 with
 (compiler enhanced) RC and that is not good at all.


 I don't see why supporting something beyond RC and GC is 
 necessary.


 Both GC and RC are forms of automatic memory management with 
 unavoidable
 associated overhead. Both have their uses but are unacceptable 
 in most
 of the system code. Which means that would would either need to
 duplicate all Phobos/druntime functions to work with both RC 
 and plain
 pointers or it will become even less suitable for low level 
 code than it
 is now.


 I'm afraid that is a bit vague. Can you give a more concrete 
 example?


Also important are Unique ownership (handing out scoped 
references to internal (untyped) memory) and ranges that cannot 
outlive their containers.


 - allow taking address of a `ref` and define lifetime of such 
 expression
 to be same of `ref` lifetime


 Again, why? Why can't the function return by 'ref'?


While returning by ref is good enough to prevent escaping, and we 
do use it in RefCounted for structs, it doesn't allow you to bind 
that returned value anywhere.
So you could only chain method calls 
s.get.call.getNested.doSome.foo.bar;.

While that might be fine as a first step, it doesn't seem to need 
scope/return scope, and is fairly limiting in the mid-term.


 - `return scope` of a struct methods binds returns value 
 lifetime to one
 of the struct instance itself, not composition of its fields


 'ref' return already does that.


Would it be possible to conflate the two different annotatons 
with similar but slightly different semantic? Couldn't return 
scope on the method mean, that a reference to the whole struct 
can be returned.
Oct 29 2016
prev sibling parent reply Stefam Koch <uplink.coder googlemail.com> writes:
On Monday, 24 October 2016 at 02:29:35 UTC, Walter Bright wrote:


 At some point we need to move forward. We are already years 
 late. Another month has slipped by.


That argument can go two ways.
Since we are already years late another month won't matter that 
much.
D's main advantage is hindsight and careful consideration of 
complexity versus power!
We should stick to those values and trying to beat someone in a 
marketing hype.
Oct 24 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 1:13 AM, Stefam Koch wrote:

 That argument can go two ways.
 Since we are already years late another month won't matter that much.


That doesn't work as an argument. Being later makes things worse, not 
irrelevant. Memory safety is constantly growing in importance. It is a key 
advantage D can have over C/C++ - one they will have great difficulty matching.



 D's main advantage is hindsight and careful consideration of complexity versus
 power!
 We should stick to those values and trying to beat someone in a marketing hype.


It's not a question of beating. It is a question of becoming irrelevant. We
lose 
design wins constantly because of this issue.
Oct 24 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 8:38 AM, Dicebot wrote:

 It won't allow to fix `Unique`


What it does mean is given a unique pointer u,

    int foo(scope Unique u);

it is guaranteed that u's value does not escape from foo(). So,

    Unique u = malloc();
    foo(u);
    free(u); // guaranteed to not leave a dangling pointer
Oct 22 2016
parent Martin Nowak <code dawg.eu> writes:
On Saturday, 22 October 2016 at 08:10:51 UTC, Walter Bright wrote:

 On 10/21/2016 8:38 AM, Dicebot wrote:
    int foo(scope Unique u);

 it is guaranteed that u's value does not escape from foo(). So,

    Unique u = malloc();
    foo(u);
    free(u); // guaranteed to not leave a dangling pointer


That's unsafe code calling safe code, but the more relevant case 
is safe code interfacing with trusted code.
Oct 29 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Thursday, 20 October 2016 at 21:36:44 UTC, Walter Bright wrote:

 On 10/20/2016 12:27 PM, Dicebot wrote:
   int* bar()
   {
     int p;
     return foo1(&p);
   }


This is a problem I had reading DIP1000. At first it decomposes 
language constructs into primitives then sets up rules for the 
primitives.
Such a mathematical approach is nice, but it isn't exercised 
throughout the DIP.

For example the most interesting function semantics just refer 
"See section dedicated to discussing methods.", but the Function 
section 
(https://github.com/dlang/DIPs/blob/6f9f50ee579bd0ccc11e5dfeef035cbbb095e0d9/DIPs/D
P1000.md#functions) doesn't contain formal lifetime rules for functions.
Weirdly the Pointer section 
(https://github.com/dlang/DIPs/blob/6f9f50ee579bd0ccc11e5dfeef035cbbb095e0d9/DIPs/
IP1000.md#pointers) seems to contain lifetime rules for function return values.
At this point the DIP regresses a bit into showing examples with 
short notes whether they are errors or correct.
It's somewhat difficult to extract a mental model from that.

The DIP seems to define the lifetime of function return values as 
the longest lifetime that can be conservatively assumed given the 
function arguments.

You mentioned an article and a talk, do you have any cleaner/more 
detailed explanation for functions  Walter?
Oct 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2016 7:57 PM, Martin Nowak wrote:

 For example the most interesting function semantics just refer "See section
 dedicated to discussing methods.", but the Function section
 (https://github.com/dlang/DIPs/blob/6f9f50ee579bd0ccc11e5dfeef035cbbb095e0d9/DIPs/DIP1000.md#functions)
 doesn't contain formal lifetime rules for functions.
 Weirdly the Pointer section
 (https://github.com/dlang/DIPs/blob/6f9f50ee579bd0ccc11e5dfeef035cbbb095e0d9/DIPs/DIP1000.md#pointers)
 seems to contain lifetime rules for function return values.
 At this point the DIP regresses a bit into showing examples with short notes
 whether they are errors or correct.
 It's somewhat difficult to extract a mental model from that.


What you're looking at is I'm not very good at articulating the mental model I 
have for it. I don't know how to write 'formal' rules for these things.



 The DIP seems to define the lifetime of function return values as the longest
 lifetime that can be conservatively assumed given the function arguments.


That's right.



 You mentioned an article and a talk, do you have any cleaner/more detailed
 explanation for functions  Walter?


I haven't written either, yet, though I need to get busy with it Real Soon Now. 
It'll be informal, obviously, but I plan to use trivial examples to make it as 
clear as possible.

I hate slides with more than about 5 lines of code in them - I lose track of
the 
presentation when I'm trying to understand the code, and I presume my audience 
will, too.
Oct 20 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Friday, 21 October 2016 at 05:41:31 UTC, Walter Bright wrote:

 What you're looking at is I'm not very good at articulating the 
 mental model I have for it. I don't know how to write 'formal' 
 rules for these things.


If we're going to go through the DIP process, you're going to 
have to try. There's this big language enhancement in the 
pipeline and I still have only a tenuous understanding of what 
it's meant to do and an even more tenuous understanding of how 
it's meant to be an improvement. Your mental model needs to be 
stated in concise, unambiguous terms before the rest of us can 
catch up and be able to consent to having the language we all use 
modified in such a way.

The DIP process is annoying and obtuse as hell, but it does exist 
for a reason. Your proposals should be held to the same standards 
as the rest of us, because changes to the language affect all of 
us.
Oct 21 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2016 1:30 AM, pineapple wrote:

 There's this big language enhancement in the pipeline


But it's not big.

It's the same as 'return ref', which is already there, except it's 'return 
scope'. The examples on how it works all boil down to a couple lines of code.
Oct 22 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Saturday, 22 October 2016 at 07:19:11 UTC, Walter Bright wrote:

 On 10/21/2016 1:30 AM, pineapple wrote:

 There's this big language enhancement in the pipeline


 But it's not big.

 It's the same as 'return ref', which is already there, except 
 it's 'return scope'. The examples on how it works all boil down 
 to a couple lines of code.


Most people seem to disagree with that opinion.

I was looking at the DIP, and checked out the P.R. branch to test 
it, and I cannot see how it could work if `scope` is not 
transitive.
It is just pushing the problem one level down.

For example:

```
alias FunDG = void delegate ()  safe;

struct Escaper
{
     FunDG DG;
}

FunDG escapeDg1 (scope FunDG d)  safe
{
     Escaper e;
     e.DG = d;
     return e.DG;
}
```

This compiles, and escapes a `scope delegate`.
Used `./src/dmd -transition=safe -dip25 -run test.d` to compile, 
on commit:
`48b7815 - (HEAD, walter/return-scope) add lifetime checks (9 
days ago) <Walter Bright>`. Did I miss something ?


Also, as others already mentioned, the DIP is quite confusing.

I read the whole thread and AFAICS people got confused because of 
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#escaping-via-return
which mentions that "taking the address of a local [...] is restrictive." and
continue with "The `scope` storage class was introduced which annotates the
pointer.", which doesn't quite relate to the previous sentence if scope is not
meant to remove this restriction.
The examples also use a mix of ` safe` and non-annotated 
function, which doesn't help understanding since it's explicitly 
stated that those checks are only done in  safe code (why?).
Finally, I agree with Dicebot that having scope infered when the 
type is specified is extremely confusing, and makes writing test 
cases, and code in general, much more complicated.
Oct 22 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/22/2016 8:52 AM, Mathias Lang wrote:

 I was looking at the DIP, and checked out the P.R. branch to test it, and I
 cannot see how it could work if `scope` is not transitive.
 It is just pushing the problem one level down.

 For example:

 ```
 alias FunDG = void delegate ()  safe;

 struct Escaper
 {
     FunDG DG;
 }

 FunDG escapeDg1 (scope FunDG d)  safe
 {
     Escaper e;
     e.DG = d;
     return e.DG;
 }
 ```

 This compiles, and escapes a `scope delegate`.


1. It is not an example of transitivity. e and d are at the same level, being a 
field does not make it a level down.

2. You're right that the above should error, I'll look into it.



 Finally, I agree with Dicebot that having scope infered when the type is
 specified is extremely confusing,


What is confusing about it?

    scope int* p = ...;
    int* q = p; // q becomes 'scope' as well

The compiler already does tons of inference for types and attributes. People 
should be well used to it by now :-)



 and makes writing test cases, and code in
 general, much more complicated.


Actually, it makes things much simpler allowing the compiler to take care of
it. 
The whole scope thing would be fairly unusable if one had to manually add all 
the annotations.
Oct 22 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/22/2016 7:34 PM, Walter Bright wrote:

 2. You're right that the above should error, I'll look into it.


Added the fix to the PR.
Oct 22 2016
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Sunday, 23 October 2016 at 06:42:41 UTC, Walter Bright wrote:

 On 10/22/2016 7:34 PM, Walter Bright wrote:

 2. You're right that the above should error, I'll look into it.


 Added the fix to the PR.


Somehow my previous (mailed yesterday afternoon) message didn't 
go through, so I'm going to paste it here again:

Just looked at it.

The diff shows:
+fail_compilation/retscope.d(198): Error: scope variable e may 
not be returned
+void* escapeDg1(scope void* d)  safe
+{
+    Escaper e;
+    e.DG = d;
+    return e.DG; // L198
+}

So it means that the storage class of 'e' changes during semantic 
analysis of the function. I don't think that's something we want 
to ever do.
Crafting a test case that exploit this was trivial:
void* escape3 (scope void* p)  safe
{
     Escaper e;
     scope dg = () { return e.e; };
     e.e = p;
     return dg();
}

Not to mention, if we ever relax the restriction on taking the 
address of a local variable, as the DIP seems to hint as a goal, 
you can just do:
Escaper e;
Escaper* pe = &e;
And you'll have the same issue.
Oct 24 2016
next sibling parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org> <nua19j$1fvb$1 digitalmars.com>
 <nub5qg$hks$1 digitalmars.com> <nubddb$ulr$1 digitalmars.com>
 <rxhugjriispyocejnscj forum.dlang.org> <nuc9q8$2g0o$1 digitalmars.com>
 <wrlfmfvelrduhlqugaus forum.dlang.org> <nuf3tc$14t5$1 digitalmars.com>
 <avaeijcpibvujqdosnge forum.dlang.org> <nuh7ij$1512$1 digitalmars.com>
 <nuhm4s$1uva$1 digitalmars.com> <fngwxfoeudirjwkfuxlw forum.dlang.org>
In-Reply-To: <fngwxfoeudirjwkfuxlw forum.dlang.org>

--6Puj17VNp24qpiH5a0I0fGLqI1DWQsKFM
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/24/2016 12:13 PM, Mathias Lang wrote:

 Crafting a test case that exploit this was trivial:
 void* escape3 (scope void* p)  safe
 {
     Escaper e;
     scope dg =3D () { return e.e; };
     e.e =3D p;
     return dg();
 }


I think it is quite possible to keep non-transitive semantics to enable
designing correct  trusted wrappers, the problem comes from trying to
magically work as valid  safe code.

Really, all we need is for developer to be able to say that some
pointer/reference may only be used as long as host aggregate exists.
With that transitivity can be emulated by removing direct access to
reference-type fields and marking matching getters as scope.


--6Puj17VNp24qpiH5a0I0fGLqI1DWQsKFM--
Oct 24 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 2:13 AM, Mathias Lang wrote:

 So it means that the storage class of 'e' changes during semantic analysis of
 the function. I don't think that's something we want to ever do.


Why? D already infers lots of things.


 Crafting a test case that exploit this was trivial:
 void* escape3 (scope void* p)  safe
 {
     Escaper e;
     scope dg = () { return e.e; };
     e.e = p;
     return dg();
 }


I'll investigate that one.
Oct 24 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 12:30 PM, Walter Bright wrote:

 Why? D already infers lots of things.


Like  safe,  trusted,  system,  nogc, pure, nothrow, return.

Consider also that Rust does extensive lifetime attribute inference.
Oct 24 2016
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Monday, 24 October 2016 at 20:57:08 UTC, Walter Bright wrote:

 On 10/24/2016 12:30 PM, Walter Bright wrote:

 Why? D already infers lots of things.


 Like  safe,  trusted,  system,  nogc, pure, nothrow, return.

 Consider also that Rust does extensive lifetime attribute 
 inference.


Don't get me wrong, I love inference. I am afraid my feedback may 
have mixed up two things, so I'm going to reinstate them here:

1) As highlighted in the previous post, we should not change the 
STC / type of a declaration after it is declared. It's a can of 
worms and will break in many ways.

2) I don't know a single place in the language where a single 
"something" (type / attributes / STC) is infered. It's either all 
or nothing (e.g. you cannot have a function infering safety but 
not infering nothrow /  nogc, short of tricking the compiler by 
changing the function's code). Also, I don't know of a single 
place in the language where a type provided explicitly is 
infered. For example, if you do `char[] s = "Hello World";`, 
`const` or `immutable` is not infered, and you get an error, and 
that's a good thing. Changing it would make code way less 
readable and harder to reason about.
Oct 24 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 2:11 PM, Mathias Lang wrote:

 1) As highlighted in the previous post, we should not change the STC / type of
a
 declaration after it is declared. It's a can of worms and will break in many
ways.


The definition of a storage class is a bit fuzzy. 'scope' does not say where
the 
variable is stored - it does not say "put it on the stack", "put it in the code 
segment", "put it on the heap". It's more of an attribute, just like nothrow, 
pure, etc.



 2) I don't know a single place in the language where a single "something" (type
 / attributes / STC) is infered. It's either all or nothing (e.g. you cannot
have
 a function infering safety but not infering nothrow /  nogc, short of tricking
 the compiler by changing the function's code). Also, I don't know of a single
 place in the language where a type provided explicitly is infered. For example,
 if you do `char[] s = "Hello World";`, `const` or `immutable` is not infered,
 and you get an error, and that's a good thing. Changing it would make code way
 less readable and harder to reason about.


'scope' is not a type, although it does affect function type signatures the
same 
as  safe, pure, nothrow and  nogc do, as well as affecting 
covariance/contravariance in an equivalent manner.
Oct 24 2016
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Monday, 24 October 2016 at 21:30:50 UTC, Walter Bright wrote:

 On 10/24/2016 2:11 PM, Mathias Lang wrote:

 1) As highlighted in the previous post, we should not change 
 the STC / type of a
 declaration after it is declared. It's a can of worms and will 
 break in many ways.


 The definition of a storage class is a bit fuzzy. 'scope' does 
 not say where the variable is stored - it does not say "put it 
 on the stack", "put it in the code segment", "put it on the 
 heap". It's more of an attribute, just like nothrow, pure, etc.


`scope o = new Object;`
If you prefer to compare it to `nothrow` or `pure`, that's also 
fine by me.
I still don't know of any place where an attribute can be added 
to a pre-existing variable declaration.



 'scope' is not a type, although it does affect function type 
 signatures the same as  safe, pure, nothrow and  nogc do, as 
 well as affecting covariance/contravariance in an equivalent 
 manner.


The DIP is very clear that `scope` is not a type qualifier.
The parallel with `const` inference was drawn as a reference to 
https://forum.dlang.org/post/nubdqu$v6u$1 digitalmars.com (and 
your later post).
I see both inference as getting in the way of code readability if 
not requested by the user explicitly.


On Monday, 24 October 2016 at 21:40:48 UTC, Walter Bright wrote:

 
 Please wait until I investigate that one.


Fair enough, then I'll leave the point of adding attributes for 
later.



 On Monday, 24 October 2016 at 19:32:45 UTC, Walter Bright 
 wrote:

 On 10/24/2016 9:49 AM, Mathias Lang wrote:


 Users will have to learn `scope` semantic to write ` safe` 
 code.
 If one doesn't want to learn `scope` semantics, one just 
 doesn't use it, `scope`
 is opt-in after all.
 I fail to see any benefit for someone that doesn't want to 
 learn `scope`,


 Writing  system code can be done without paying attention to 
 that stuff, and there is value in that.  safe code comes into 
 play when writing larger scale, longer lived (!) code, and code 
 that is worked on by a team.


Could you explain the value there is in not checking `scope` in 
` system` ?
Also, how does that value offset the user confusion and possible 
bugs caused by people trying to use `scope` in ` system` code 
without realizing it doesn't do any check ?
Quick note: I notice you said ` system`. The DIP said "only 
checked in  safe code". Will  trusted by checked ?

Also, I don't quite see what  safe has to do with code lifetime, 
or collaboration.
Oct 24 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 3:10 PM, Mathias Lang wrote:

 `scope o = new Object;`
 If you prefer to compare it to `nothrow` or `pure`, that's also fine by me.
 I still don't know of any place where an attribute can be added to a
 pre-existing variable declaration.


You're right, that is a bit of a special case, inherited from D1. But storing
it 
on the stack is an optimization, it behaves "as if" it was allocated on the
heap.



 I see both inference as getting in the way of code readability if not requested
 by the user explicitly.


At this point, we'll just have to disagree on that.



 Could you explain the value there is in not checking `scope` in ` system` ?


Quick and dirty code, for one.



 Also, how does that value offset the user confusion and possible bugs caused by
 people trying to use `scope` in ` system` code without realizing it doesn't do
 any check ?


It still checks the interface.



 Quick note: I notice you said ` system`. The DIP said "only checked in  safe
 code". Will  trusted by checked ?


No, because  trusted code is system code.



 Also, I don't quite see what  safe has to do with code lifetime,


Wouldn't you want to take more care when you write code that will be used for 
years, rather than some throwaway piece of code?



  or collaboration.


Because it enables your collaborators to understand your code better.
Oct 24 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
Now that we have https://github.com/dlang/dmd/pull/6253 I came 
back to test it a bit.

First I have a question: What is going to happen with `scope foo 
= new Object` ?
Are we keeping it the way it currently is ? It's something we 
highly rely on at work and it looks like it could work nicely 
with the enhancements, but it wasn't mentioned anywhere.

And from a quick test (Using `-dip25 -transition=safe` and 
5928249 which is the commit of this P.R.), the following code:

```
void main ()  safe
{
     scope o = new Object();
}
```

Results in the following error:
scope2.d(3): Error: delete o is not  safe but is used in  safe 
function main

Which works with the compiler I'm currently using, 2.071.2.

By the way, I reiterate my point that tying `scope` verification 
to ` safe` function is highly misleading and not necessary. I 
originally crafted a couple of test case, and when everything 
worked, I was left puzzled for a minute before realizing I forgot 
` safe`.

Second point, on testing your P.R., I found that the following 
code compiles:

```
alias FunDG = void delegate ()  safe;

void main ()  safe
{
     int x = 42;
     scope FunDG f = () { assert(x == 42); };
     return fun(&f);
}

FunDG fun (scope FunDG* ptr)  safe
{
     return *ptr;
}
```

As well as the following:
```
alias FunDG = void* delegate ()  safe;

void main ()  safe
{
     void* x = fwd();
}

void* fwd ()  safe
{
     int x = 42;
     scope FunDG f = () { return &x; };
     return fun(f);
}

void* fun (scope FunDG ptr)  safe
{
     return ptr();
}
```

I'm not quite sure how one would express taking a pointer to a 
scope value though.
Nov 20 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/20/2016 8:41 AM, Mathias Lang wrote:

 First I have a question: What is going to happen with `scope foo = new Object`
?
 Are we keeping it the way it currently is ?


Yes.



 void main ()  safe
 {
     scope o = new Object();
 }
 ```

 Results in the following error:
 scope2.d(3): Error: delete o is not  safe but is used in  safe function main

 Which works with the compiler I'm currently using, 2.071.2.


I'll look into that.



 By the way, I reiterate my point that tying `scope` verification to ` safe`
 function is highly misleading and not necessary.


I understand your point. The trouble is, it will break too much code.



 I found that the following code compiles:

 ```
 alias FunDG = void delegate ()  safe;

 void main ()  safe
 {
     int x = 42;
     scope FunDG f = () { assert(x == 42); };
     return fun(&f);
 }

 FunDG fun (scope FunDG* ptr)  safe
 {
     return *ptr;
 }
 ```


I get:

bug9.d(7): Error: cannot return non-void from void function



 As well as the following:
 ```
 alias FunDG = void* delegate ()  safe;

 void main ()  safe
 {
     void* x = fwd();
 }

 void* fwd ()  safe
 {
     int x = 42;
     scope FunDG f = () { return &x; };
     return fun(f);
 }

 void* fun (scope FunDG ptr)  safe
 {
     return ptr();
 }


I'll have to investigate that one.
Nov 22 2016
next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 23 November 2016 at 03:25:20 UTC, Walter Bright 
wrote:

 I understand your point. The trouble is, it will break too much 
 code.


Maybe I am missing something here: I didn't consider it as 
breaking more code than it's ` safe` counterpart.


 I found that the following code compiles:
 [...]


 I get:

 bug9.d(7): Error: cannot return non-void from void function


My bad, I tried to reduce the snippet a bit more and broke it :)
Here's the corrected version:

```
void main ()  safe
{
     int x;
     void* escape = bar(&x);
}

void* bar (scope void* x)  safe
{
     return fun(&x);
}

void* fun (scope void** ptr)  safe
{
     return *ptr;
}
```

Using the top of your branch, that is:
b8df860 allow 'return' attribute on nested functions (68 minutes 
ago) <Walter Bright>
Nov 23 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/23/2016 3:05 AM, Mathias Lang wrote:

 [...]


It doesn't actually escape on your version, as main() doesn't return a void*. 
Corrected example:

--------------------
void* abc ()  safe
{
     int x;
     void* escape = bar(&x);
     return escape;  // <== correction
}

void* bar (scope void* x)  safe
{
     return fun(&x);  // where the problem lies
}

void* fun (scope void** ptr)  safe
{
     return *ptr;
}
---------------------

Probably the best solution is to not allow taking the address of a scope local. 
Good catch!
Nov 23 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/22/2016 7:25 PM, Walter Bright wrote:

 As well as the following:
 ```
 alias FunDG = void* delegate ()  safe;

 void main ()  safe
 {
     void* x = fwd();
 }

 void* fwd ()  safe
 {
     int x = 42;
     scope FunDG f = () { return &x; };
     return fun(f);
 }

 void* fun (scope FunDG ptr)  safe
 {
     return ptr();
 }


 I'll have to investigate that one.


I amended:

https://github.com/dlang/dmd/pull/6253

to fix it.
Nov 23 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/20/2016 8:41 AM, Mathias Lang wrote:

 And from a quick test (Using `-dip25 -transition=safe` and 5928249 which is the
 commit of this P.R.), the following code:

 ```
 void main ()  safe
 {
     scope o = new Object();
 }
 ```

 Results in the following error:
 scope2.d(3): Error: delete o is not  safe but is used in  safe function main

 Which works with the compiler I'm currently using, 2.071.2.


This was done here back in June:

   https://github.com/dlang/dmd/commit/e64ae1d3e5aa078a036242864a68499617c9b278

and was added to fix:

   https://issues.dlang.org/show_bug.cgi?id=16195

The trouble is, operator delete is not safe.

What do you suggest?
Nov 22 2016
next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 23 November 2016 at 03:35:16 UTC, Walter Bright 
wrote:

 On 11/20/2016 8:41 AM, Mathias Lang wrote:

 And from a quick test (Using `-dip25 -transition=safe` and 
 5928249 which is the
 commit of this P.R.), the following code:

 ```
 void main ()  safe
 {
     scope o = new Object();
 }
 ```

 Results in the following error:
 scope2.d(3): Error: delete o is not  safe but is used in  safe 
 function main

 Which works with the compiler I'm currently using, 2.071.2.


 This was done here back in June:

   
 https://github.com/dlang/dmd/commit/e64ae1d3e5aa078a036242864a68499617c9b278

 and was added to fix:

   https://issues.dlang.org/show_bug.cgi?id=16195

 The trouble is, operator delete is not safe.

 What do you suggest?


If I understand correctly, it's not  safe because it might free a 
reference which is still used elsewhere, making it dangling.

If we have the correct `scope` enforcement though, we can allow 
it *only at the scope which `new`ed it*, because we know it's not 
possible than any other place has a reference to it, as the 
reference would have to be `scope` as well.

Does that make sense ?
Nov 23 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/23/2016 3:08 AM, Mathias Lang wrote:

 If I understand correctly, it's not  safe because it might free a reference
 which is still used elsewhere, making it dangling.


Yes.


 If we have the correct `scope` enforcement though, we can allow it *only at the
 scope which `new`ed it*, because we know it's not possible than any other place
 has a reference to it, as the reference would have to be `scope` as well.

 Does that make sense ?


Yes. I'll see if I can make that work tomorrow.
Nov 23 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
https://issues.dlang.org/show_bug.cgi?id=16747
Nov 24 2016
prev sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Monday, 24 October 2016 at 19:30:16 UTC, Walter Bright wrote:

 On 10/24/2016 2:13 AM, Mathias Lang wrote:

 So it means that the storage class of 'e' changes during 
 semantic analysis of
 the function. I don't think that's something we want to ever 
 do.


 Why? D already infers lots of things.


To answer your question: Because it isn't even inference.
The specs explicitly says "Unlike module level declarations, 
declarations within function scope are processed in order." 
(https://dlang.org/spec/function.html)
Changing the storage class of a previous declaration is then 
explicitly breaking that line of the spec, and breaking a key 
assumption the specs / compiler relies on. I pointed that problem 
using delegate, I could have done so using templates, nested 
functions, or whatever part of the language that relies on said 
assumption.

On Monday, 24 October 2016 at 19:32:45 UTC, Walter Bright wrote:

 On 10/24/2016 9:49 AM, Mathias Lang wrote:

 Why is this necessary / what would be the problem in allowing 
 it in  system ?


 For one thing, it would require all users to learn scope 
 semantics and annotate them correctly. It would be like 
 requiring const correctness.


Users will have to learn `scope` semantic to write ` safe` code.
If one doesn't want to learn `scope` semantics, one just doesn't 
use it, `scope` is opt-in after all.
I fail to see any benefit for someone that doesn't want to learn 
`scope`, and find it extremely confusing for users that want to 
use it but can't make their code ` safe` (because for example, 
they're accepting a delegate which can be  system so they can't 
just wrap code in ` trusted`).
Oct 24 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 2:02 PM, Mathias Lang wrote:

 On Monday, 24 October 2016 at 19:30:16 UTC, Walter Bright wrote:

 On 10/24/2016 2:13 AM, Mathias Lang wrote:

 So it means that the storage class of 'e' changes during semantic analysis of
 the function. I don't think that's something we want to ever do.


 Why? D already infers lots of things.


 To answer your question: Because it isn't even inference.
 The specs explicitly says "Unlike module level declarations, declarations
within
 function scope are processed in order." (https://dlang.org/spec/function.html)
 Changing the storage class of a previous declaration is then explicitly
breaking
 that line of the spec, and breaking a key assumption the specs / compiler
relies
 on.


I don't believe it breaks anything the language relies on.



 I pointed that problem using delegate, I could have done so using templates,
 nested functions, or whatever part of the language that relies on said
assumption.


Please wait until I investigate that one.



 On Monday, 24 October 2016 at 19:32:45 UTC, Walter Bright wrote:

 On 10/24/2016 9:49 AM, Mathias Lang wrote:

 Why is this necessary / what would be the problem in allowing it in  system ?


 For one thing, it would require all users to learn scope semantics and
 annotate them correctly. It would be like requiring const correctness.


 Users will have to learn `scope` semantic to write ` safe` code.
 If one doesn't want to learn `scope` semantics, one just doesn't use it,
`scope`
 is opt-in after all.
 I fail to see any benefit for someone that doesn't want to learn `scope`,


Writing  system code can be done without paying attention to that stuff, and 
there is value in that.  safe code comes into play when writing larger scale, 
longer lived (!) code, and code that is worked on by a team.



 and find it extremely confusing for users that want to use it but can't make
their
 code ` safe` (because for example, they're accepting a delegate which can be
  system so they can't just wrap code in ` trusted`).


That's the reason for inference. (I'd also like to do inference for 'const'. 
Inference of pure, nothrow, etc., has been very successful, and I don't see any 
fundamental issue with taking it further.)
Oct 24 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 2:13 AM, Mathias Lang wrote:

 Crafting a test case that exploit this was trivial:
 void* escape3 (scope void* p)  safe
 {
     Escaper e;
     scope dg = () { return e.e; };
     e.e = p;
     return dg();
 }


A complete example:

--------
struct Escaper { void* e; }

void* escape3 (scope void* p)  safe {
     Escaper e;
     e.e = p;
     scope dg = () { return e.e; }; // Error: scope variable e may not be
returned
     return dg();
}
-------

But changing the order of things:

--------
struct Escaper { void* e; }

void* escape3 (scope void* p)  safe {
     Escaper e;
     scope dg = () { return e.e; }; // no error
     e.e = p;
     return dg();
}
-------

This definitely is a bug, but is fixable. Thanks for posting it!
Oct 24 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Monday, 24 October 2016 at 23:10:46 UTC, Walter Bright wrote:

 This definitely is a bug, but is fixable. Thanks for posting it!


Let's please talk about how this needs fixing beforehand, see my 
post below.
It seems to me this requires 2 semantic passes through function 
statememts :o.
Oct 25 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/25/2016 12:09 AM, Martin Nowak wrote:

 Let's please talk about how this needs fixing beforehand, see my post below.
 It seems to me this requires 2 semantic passes through function statememts :o.


One way of fixing it besides doing multiple passes is noting its use inside dg, 
and setting a flag saying it cannot be made scope, and so the scoped assignment 
becomes an error.

General flow analysis generally requires an arbitrary number of passes to 
converge on a solution (the global optimizer does this), but we can do it in
one 
pass with the conservative approach above, and just accept that it's
conservative.
Oct 25 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Tuesday, 25 October 2016 at 10:05:50 UTC, Walter Bright wrote:

 One way of fixing it besides doing multiple passes is noting 
 its use inside dg, and setting a flag saying it cannot be made 
 scope, and so the scoped assignment becomes an error.


A tristate tainting, unkown/scope/escape, should indeed work here.
Oct 25 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/25/2016 5:59 PM, Martin Nowak wrote:

 On Tuesday, 25 October 2016 at 10:05:50 UTC, Walter Bright wrote:

 One way of fixing it besides doing multiple passes is noting its use inside
 dg, and setting a flag saying it cannot be made scope, and so the scoped
 assignment becomes an error.


 A tristate tainting, unkown/scope/escape, should indeed work here.


I implemented it, but now the PR got thoroughly fouled up after I rebased it on 
master.
Nov 01 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Tuesday, 1 November 2016 at 09:19:55 UTC, Walter Bright wrote:

 I implemented it, but now the PR got thoroughly fouled up after 
 I rebased it on master.


You need help fixing this? In case of failed rebases it's usually 
best to git rebase --abort. If it's too late for that, you cat 
look at .git/logs/refs/heads/branch_name, to find the git commit 
hash before the rebase, then git reset --hard to that, and try 
again.
Nov 05 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/5/2016 3:58 AM, Martin Nowak wrote:

 On Tuesday, 1 November 2016 at 09:19:55 UTC, Walter Bright wrote:

 I implemented it, but now the PR got thoroughly fouled up after I rebased it
 on master.


 You need help fixing this? In case of failed rebases it's usually best to git
 rebase --abort. If it's too late for that, you cat look at
 .git/logs/refs/heads/branch_name, to find the git commit hash before the
rebase,
 then git reset --hard to that, and try again.


I'm concerned that if I dink around with it, I might wind up
scrambling/deleting 
the commit history of it. I'd rather learn this stuff on something less 
important, so I'd prefer to defer it to you.
Nov 06 2016
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Monday, 24 October 2016 at 09:13:58 UTC, Mathias Lang wrote:

 void* escape3 (scope void* p)  safe
 {
     Escaper e;
     scope dg = () { return e.e; };
     e.e = p;
     return dg();
 }


Seems like this would require that the scope tainting is done in 
a separate pass, before going through the statements in lexical 
order. The dg needs to know that e will escape in order to change 
to return scope.
Not sure the compiler is ready for 2-pass function semantics.
While it's clearly powerful, my intuition also goes a bit against 
it.
- Does the compiler architecture allow for the necessary 
semantics.
- What's a use-case that requires this?
- Might the scope inference graph have cycles?
- Can humans understand scope inference, and can we print proper 
error messages?

It might be a bit too ambituous, and turn out to be an unforced 
error creating too many scope holes.
Oct 25 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Saturday, 22 October 2016 at 15:52:17 UTC, Mathias Lang wrote:

 It's the same as 'return ref', which is already there, except 
 it's 'return scope'. The examples on how it works all boil 
 down to a couple lines of code.


 Most people seem to disagree with that opinion.

 I was looking at the DIP, and checked out the P.R. branch to 
 test it, and I cannot see how it could work if `scope` is not 
 transitive.


Let's clarify the term transitive, it's quite possible different 
people have different understandings of that term.

Only drawing on my conclusion from rewriting the RC/Unique 
implementations.
What's needed is a "container" to wrap return values so that they 
cannot be aliased or escaped, a bit similar to how you can't take 
the address of an rvalue.
If you could get a named alias (e.g. scope ref/ptr) into the 
returned scope value, then the lifetimes of the two variables are 
entangled, and it would be trivial to end up with a dangling 
scoped ref/ptr.

scope S s:
scope int* p = s.getInternalRef; // could be a malloced class ref 
instead of a pointer
if (flipCoin)
   s.destroy; // or s = S();

To make this work, we'd need to disallow any mutation of s, while 
references to it are reachable.
I wonder if that's worth the trouble over simply disallowing any 
binding to a named (and thus reachable) variable. Would require 
chaining of return values though, 
s.get.cannot.put.that.hot.potatoe.anywhere :/.
Oct 25 2016
parent Martin Nowak <code dawg.eu> writes:
On Tuesday, 25 October 2016 at 07:51:21 UTC, Martin Nowak wrote:

 To make this work, we'd need to disallow any mutation of s, 
 while references to it are reachable.


Didn't mention this explicitly. Binding the returned scope value 
as named scope ref argument to a chained function call must of 
course be possible, but that still wouldn't allow to bind to a 
variable that outlives the expression.
So it's really somewhat similar to rvalues.
Oct 25 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
 <mtzackphhklkrtotdbbd forum.dlang.org>
In-Reply-To: <mtzackphhklkrtotdbbd forum.dlang.org>

--F2hGmWhEbQt1Ddxi5Lf0t64BP2RxI75gX
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/20/2016 01:12 AM, Martin Nowak wrote:

 But IMHO we should ensure that we have sound specs (backed by tests)
 that solve the __right__ use-cases before discussing implementations. A=


s

 part of the implementation we'll have to transform this into a more
 easily understandable spec anyhow, so we might as well do that in the
 beginning.


This is exactly what I am trying to do in this thread (which is
continuation of e-mail exchange with Walter on same topic). Coming up
with set of useful practical snippets based on proposed implementation
and using them to figure out subtle semantical details.

Once I understand what is supposed to work and what is not, I'll give my
try at complete rewrite of DIP1000 to make it more coherent with intentio=
ns.


--F2hGmWhEbQt1Ddxi5Lf0t64BP2RxI75gX--
Oct 20 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,i,n,t,e,r,n,a,l,s
Subject: Re: DIP1000 discussion and testing
References: <nu00a6$t5i$1 digitalmars.com>
In-Reply-To: <nu00a6$t5i$1 digitalmars.com>

--63kxrBHdAJWHTHoSJlgg4BO9eS1mR6nPu
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Ok, so I am revisiting DIP1000 right now based on recent comments from
Walter. One thing remains unclear to me though, consider this example in
DIP1000 document:

```
 safe struct RefCountedSlice(T) {
	private T[] payload;

	// ...



	scope ref T opIndex(size_t i) {
		return payload[i];
	}
}
```

It looks like with current semantics this statement is very wrong.
Assuming that you have meant `scope return` instead (because plain scope
does nothing for `this` reference) Right now it does not matter how
`opIndex` is annotated if variable holding `RefCountedSlice` itself is
not annotated/deduced as scope:

```
 safe struct S
{
    // RefCountedSlice code reduced to bare minimum

    private int* payload;

    this (int) {
        this.payload =3D new int;
    }

     trusted ~this () {
        delete this.payload;
    }

    int* get () scope return {
        return this.payload;
    }
}

int* global;

 safe unittest
{
    scope s =3D S(42);
    global =3D s.get(); // Error: scope variable s assigned to global wit=
h
longer lifetime
}

 safe unittest
{
    auto s =3D S(42);
    global =3D s.get(); // compiles, oops
}
```

And indeed, with your explanations it seems to work as intended - `scope
return` annotation is simply forwarding lifetime of `S.payload` to
return value, which happens to be infinity by lifetime rules (GC
allocation). But that means that statement saying destructor can be
 trusted is wrong.

Walter, would you mind showing reference counted example that actually
works under your implementation?


--63kxrBHdAJWHTHoSJlgg4BO9eS1mR6nPu--
Oct 22 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/22/2016 4:22 PM, Dicebot wrote:

 Walter, would you mind showing reference counted example that actually
 works under your implementation?


I can't, because I'm waiting for Andrei to make available what he considers a 
correct ref counted design.

But you are correct in that the 'return' rules can be used to transfer the 
lifetime from an argument to the return value, even if the return value is not 
derived from the argument.
Oct 23 2016
next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 23 October 2016 at 07:36:39 UTC, Walter Bright wrote:

 I can't, because I'm waiting for Andrei to make available what 
 he considers a correct ref counted design.

 But you are correct in that the 'return' rules can be used to 
 transfer the lifetime from an argument to the return value, 
 even if the return value is not derived from the argument.


Glad to hear that. Even after 1 hour thinking in circles we 
couldn't figure out where scope could be used without this.
 From my still limited understanding, the lifetime of the return 
value should be bound to the minimum (shortest) lifetime of all 
arguments that could potentially be aliased.
So if the return value is a struct containing an int* and a S*, 
it might alias any argument containing/or being an int or S (and 
maybe untyped memory).
This seems to be essential for any wrapper using malloced memory 
to safely return sth. (e.g. a range) with a pointer into that 
memory.
Oct 25 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/25/2016 5:39 PM, Martin Nowak wrote:

 From my still limited understanding, the lifetime of the return value should be
 bound to the minimum (shortest) lifetime of all arguments that could
potentially
 be aliased.


That's right.


 So if the return value is a struct containing an int* and a S*, it might alias
 any argument containing/or being an int or S (and maybe untyped memory).


If that parameter is marked "return scope", then yes.
Oct 26 2016
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Sunday, 23 October 2016 at 07:36:39 UTC, Walter Bright wrote:

 On 10/22/2016 4:22 PM, Dicebot wrote:

 Walter, would you mind showing reference counted example that 
 actually
 works under your implementation?


 I can't, because I'm waiting for Andrei to make available what 
 he considers a correct ref counted design.


We were working on that about more than a year ago when return 
ref landed. It didn't solve the escape problem for classes, but 
we have fairly solid understanding how a baseline implementation 
for unique, ref-counted, and weak-ref ownership should work.
Since those are primitives, we should be able to use them for the 
design of scope w/o a concrete RC.

https://github.com/dlang/phobos/pull/3259

If necessary, I can finish the rewrite to inform further 
discussion.
Oct 25 2016
prev sibling next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Sunday, 16 October 2016 at 13:45:42 UTC, Dicebot wrote:

 So, there is a pull request 
 (https://github.com/dlang/dmd/pull/5972)
 which is intended to implement DIP1000
 (https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md).



One thing about the DIP that I'd like more info on is:

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#safe

 Errors for scope violations are only reported in  safe code.


Why is this necessary / what would be the problem in allowing it 
in  system ?
Oct 24 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2016 9:49 AM, Mathias Lang wrote:

 Why is this necessary / what would be the problem in allowing it in  system ?


For one thing, it would require all users to learn scope semantics and annotate 
them correctly. It would be like requiring const correctness.
Oct 24 2016
parent Martin Nowak <code dawg.eu> writes:
On Monday, 24 October 2016 at 19:32:45 UTC, Walter Bright wrote:

 For one thing, it would require all users to learn scope 
 semantics and annotate them correctly. It would be like 
 requiring const correctness.


Could we just turn off scope inference in  system code but escape 
check explicit scope usage?
Oct 25 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 16 October 2016 at 13:45:42 UTC, Dicebot wrote:

To summarize the current state, the Skype call with Walter 
cleared most of the misunderstandings, DIP1000 is about 2 simple 
points.

- make scope work (fix all escaping whatsoever)

It's more complex than return ref because pointers, other than 
references, can be bound to variables, hence the lifetime algebra.

Since it was already possible to create pointers in  safe code 
(ubyte[] a
= staticArray[];), and we finally can prevent escaping of 
pointers,
DIP1000 also includes allowing to take the address of sth. in 
 safe
code, similarly to how &ary[$] is now possible (see 
https://github.com/dlang/dmd/pull/6253).

- add return scope to allow returning scoped arguments (or sth. 
derived
from them)

The current state is to conservatively assume the return value 
has the
minimum lifetime of all return scope parameters (no smartness 
about
possible aliasing of return value to parameters), even for 
 trusted
functions. This should enable RC/Unique like patterns or ranges on
containers.

- Adding scope inference for parameters and variables, we already 
have return inference from DIP25.

Inference will make this actually usable, otherwise we'd end up 
with a huge amount of required extra annotations.

====================

Open questions from my side:

- what about unwanted/incorrect return scope inference

Mixed return branches would conservatively assume a scoped return 
value?
Is this a valid use-case? Should at least work with  system code.

int* func()(int* arg, bool allocate)
{
     return allocate ? new int : arg;
}

- what about return ref without scope

Does return ref on parameters makes sense without scope?
Can you take the address of an unscoped return ref result?
Nov 22 2016
next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 22 November 2016 at 14:09:49 UTC, Martin Nowak wrote:

 similarly to how &ary[$] is now possible


Huh? How could this be  safe? Does the resulting pointer somehow 
become "tainted" to avoid dereferencing it?
Nov 23 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/23/2016 7:32 AM, Marc Schütz wrote:

 On Tuesday, 22 November 2016 at 14:09:49 UTC, Martin Nowak wrote:

 similarly to how &ary[$] is now possible


 Huh? How could this be  safe? Does the resulting pointer somehow become
 "tainted" to avoid dereferencing it?


It's only allowed if it is the operand of a comparison operation.
Nov 24 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/22/2016 6:09 AM, Martin Nowak wrote:

 - what about unwanted/incorrect return scope inference

 Mixed return branches would conservatively assume a scoped return value?
 Is this a valid use-case? Should at least work with  system code.

 int* func()(int* arg, bool allocate)
 {
     return allocate ? new int : arg;
 }


?: returns the most restrictive scope of any of its operands. This applies 
equally as well to a function with numerous parameters that are return scope or 
return ref.



 - what about return ref without scope

 Does return ref on parameters makes sense without scope?


Yes, see dip25.


 Can you take the address of an unscoped return ref result?


Taking the address of a ref makes it a scope.
Dec 09 2016
next sibling parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Saturday, 10 December 2016 at 05:00:42 UTC, Walter Bright 
wrote:

 On 11/22/2016 6:09 AM, Martin Nowak wrote:

 - what about unwanted/incorrect return scope inference

 Mixed return branches would conservatively assume a scoped 
 return value?
 Is this a valid use-case? Should at least work with  system 
 code.

 int* func()(int* arg, bool allocate)
 {
     return allocate ? new int : arg;
 }


 ?: returns the most restrictive scope of any of its operands. 
 This applies equally as well to a function with numerous 
 parameters that are return scope or return ref.


Not quite: https://issues.dlang.org/show_bug.cgi?id=16037

Also, another reason why tainting the STC of an aggregate is not 
a good idea:

struct Foo
{
     int* ptr;
}

void main ()  safe
{
     Foo f;
     escape1(f);
     assert(*f.ptr == 42);
}

void escape1 (ref Foo f)  safe
{
     int i = 42;
     f.ptr = &i;
}
Dec 11 2016
next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
Moving from 
https://forum.dlang.org/post/o2llgd$2tuc$1 digitalmars.com

On Monday, 12 December 2016 at 08:01:19 UTC, Walter Bright wrote:

 On 12/11/2016 11:15 PM, Mathias Lang wrote:

 On Monday, 12 December 2016 at 01:05:20 UTC, Walter Bright 
 wrote:

 On 12/10/2016 6:14 AM, Mathias Lang via Dlang-internal wrote:

 Since scope class are not testable, I wouldn't say it's 
 finished.


 Please be more specific.


 - Due to a regression, we cannot have `scope`-allocated 
 classes in ` safe` code
 (https://github.com/dlang/dmd/pull/6279).


 That's both testable, and has been fixed.


?
The P.R. and bug report are still open. When was it fixed ?
Dec 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/12/2016 7:21 AM, Mathias Lang wrote:

 The P.R. and bug report are still open. When was it fixed ?


It's ready to go.
Dec 12 2016
parent Mathias Lang <mathias.lang sociomantic.com> writes:
On Monday, 12 December 2016 at 21:54:38 UTC, Walter Bright wrote:

 On 12/12/2016 7:21 AM, Mathias Lang wrote:

 The P.R. and bug report are still open. When was it fixed ?


 It's ready to go.


Oh, I haven't noticed that you updated it, great!

 Martin: Could you make sure the scope branch is updated to 
include what's in master once everything gets merged back?
Dec 12 2016
prev sibling next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
```
T* foo (T) (T* arg)
{
     return arg;
}

int* escape ()  safe
{
     int b;
     return foo(&b);
}

void main ()  safe
{
     int* ptr = escape();
}
```

Compiles with `./src/dmd -transition=safe -run test.d`   
ac6f655030f814cd352a33b2c9490df16c84459d
Dec 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/12/2016 11:36 AM, Mathias Lang wrote:

 ```
 T* foo (T) (T* arg)
 {
     return arg;
 }

 int* escape ()  safe
 {
     int b;
     return foo(&b);
 }

 void main ()  safe
 {
     int* ptr = escape();
 }
 ```

 Compiles with `./src/dmd -transition=safe -run test.d`  
 ac6f655030f814cd352a33b2c9490df16c84459d


Thanks. Will investigate.
Dec 12 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/12/2016 1:53 PM, Walter Bright wrote:

 Thanks. Will investigate.


I have it corrected now. Dunno which branch to push it to at the moment, will 
check with Martin.
Dec 13 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Monday, 12 December 2016 at 07:17:11 UTC, Mathias Lang wrote:

 Not quite: https://issues.dlang.org/show_bug.cgi?id=16037


Thanks for all the test cases, it's really the most useful 
approach to close all escape holes. How about tracking all scope 
bugs in Bugzilla using "[scope]" in the title and safe as keyword?

If there are still design-concerns/unclarities about 
(non-)transitivity, we could benefit from a good test case 
illustrating the problem.
Dec 15 2016
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Thursday, 15 December 2016 at 20:34:28 UTC, Martin Nowak wrote:

 Thanks for all the test cases, it's really the most useful 
 approach to close all escape holes. How about tracking all 
 scope bugs in Bugzilla using "[scope]" in the title and safe as 
 keyword?


IMO bugzilla is a more public setup than this NG is - people not 
familiar with the design of scope, or it's goal, might 
misinterpret it. I also doubt it would scale to more feature 
branches. But I have no strong feeling about it so if it fits 
your workflow better, I'm fine with that.


 If there are still design-concerns/unclarities about 
 (non-)transitivity, we could benefit from a good test case 
 illustrating the problem.


Lack of transitivity is one thing, inability to express something 
more complex than a top level scope is another.

There is no way to have a scope pointer to a scope pointer to an 
non scope pointer.
When it comes to scope, you actually want the inverse effect we 
have for const transitivity: the transitivity should go "up" 
instead of going "down".

If we borrow C's syntax for `const` for a minute, you could have:
- `scope * scope * int*` => scope ptr to scope ptr to non scope 
int ptr
- `scope * scope * scope int*` => scope ptr to scope ptr to scope 
ptr to int
- `      * scope * int*` => non scope ptr to a scope ptr to a non 
scope int ptr

The first 2 examples are valid as long as each "level" has a 
lifetime <= to the previous one. What we absolutely want to 
disallow is the ability to have a pointer to something with `>` 
lifetime.

With our current grammar, if scope was transitive, you could 
express the second example by:
`scope int***` or `scope(scope(scope(int*)*)*)`, or any variation.
However, there would be no ability to express the first, valid 
case.

Obviously we can put a blanket ban on those cases, but to me they 
sound valid.

Now, a more practical issue, once again related to changing the 
STC of some variable mid-way through a function semantic analysis:

```
struct Foo { int* ptr; }

void main ()  safe
{
     int* a = escape();
}

int* escape ()  safe
{
     int i;
     Foo f;
     scope int** x = &f.ptr;
     f.ptr = &i;

     return bar(x);
}

int* bar ( scope int** ptr )  safe
{
   return *ptr;
}
```

Compiles with `./src/dmd -transition=safe -dip25 test.d` on 
commit fbbfce8f6e0afb716ec96ba3dc2e82b3536cbaac (with the latest 
2 P.R. merged).
Dec 16 2016
next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Friday, 16 December 2016 at 16:43:30 UTC, Mathias Lang wrote:

 Lack of transitivity is one thing, inability to express 
 something more complex than a top level scope is another.


Another, hopefully even better test case for what I'm trying to 
express:

```
void main ()  safe
{
     int* a = escape();
}

int* escape ()  safe
{
     int i;
     int*[3] a = [ &i, null, null ];
     return bar(&a[0]);
}

int* bar (scope int** x)  safe
{
     return foo(*x);
}

int* foo (int* x)  safe { return x; }
```

As long as we have:
- No way to express different `scope` levels (the "backward" 
transitivity)
- A way to take the address of `scope` pointers

We will still have that hole.


 Compiles with `./src/dmd -transition=safe -dip25 test.d` on 
 commit fbbfce8f6e0afb716ec96ba3dc2e82b3536cbaac (with the 
 latest 2 P.R. merged).
Dec 16 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/16/2016 9:08 AM, Mathias Lang wrote:

 As long as we have:
 - A way to take the address of `scope` pointers


Taking the address of a scope pointer is invalid, I'll fix the method you used 
to do it.
Dec 16 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Saturday, 17 December 2016 at 07:29:17 UTC, Walter Bright 
wrote:

 On 12/16/2016 9:08 AM, Mathias Lang wrote:

 As long as we have:
 - A way to take the address of `scope` pointers


 Taking the address of a scope pointer is invalid, I'll fix the 
 method you used to do it.


What is the reasoning for disallowing that?
Is dealing with second order indirections (scope int**) somehow 
different from from other escape checks?
Dec 17 2016
next sibling parent Martin Nowak <code dawg.eu> writes:
On Sunday, 18 December 2016 at 04:03:55 UTC, Martin Nowak wrote:

 Taking the address of a scope pointer is invalid, I'll fix the 
 method you used to do it.


 What is the reasoning for disallowing that?
 Is dealing with second order indirections (scope int**) somehow 
 different from from other escape checks?


Also see https://github.com/dlang/dmd/pull/6328/files#r92933626 
on that question.
Dec 17 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2016 8:03 PM, Martin Nowak wrote:

 On Saturday, 17 December 2016 at 07:29:17 UTC, Walter Bright wrote:

 On 12/16/2016 9:08 AM, Mathias Lang wrote:

 As long as we have:
 - A way to take the address of `scope` pointers


 Taking the address of a scope pointer is invalid, I'll fix the method you used
 to do it.


 What is the reasoning for disallowing that?


scope is not transitive, and there's no way to declare a scope pointer to a 
scope pointer.


 Is dealing with second order indirections (scope int**) somehow different from
 from other escape checks?


Yes. Only the top level is checked. By disallowing taking the address of a
scope 
variable, only the top level needs to be checked.
Dec 17 2016
parent reply Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 12/18/2016 06:59 AM, Walter Bright wrote:

 On 12/17/2016 8:03 PM, Martin Nowak wrote:

 On Saturday, 17 December 2016 at 07:29:17 UTC, Walter Bright wrote:

 On 12/16/2016 9:08 AM, Mathias Lang wrote:

 As long as we have:
 - A way to take the address of `scope` pointers


 Taking the address of a scope pointer is invalid, I'll fix the method
 you used
 to do it.


 What is the reasoning for disallowing that?


 
 scope is not transitive, and there's no way to declare a scope pointer
 to a scope pointer.


That's not really a reason, it's a symptom of the implementation and
design ;).
Let me suggest a reason, transitive scope checking would be more complex
to implement, is it that?


 Is dealing with second order indirections (scope int**) somehow
 different from
 from other escape checks?


 
 Yes. Only the top level is checked. By disallowing taking the address of
 a scope variable, only the top level needs to be checked.


The biggest limitation of return ref (DIP25) was that there was no way
to obtain indirections (e.g. a named pointer variable).
Hence you'd always need to write (and execute) a.b.c.d.e, with no way to
store temporary results (tmp = a.b.c; tmp.d.e).

No we get 1, but only 1, level of indirections (`tmp = &a.b.c;
tmp.d.e`). Two questions that arise from that.

- How certain are we that this is enough? In other words, why is `tmp =
&a.b; tmp2 = &tmp.c; tmp2.d.e` not necessary.

- Let's imagine for a short moment, what it would mean to make scope
transitive at a later point, if we'd ever find enough reasons to do so.
Maximizing future options is always good.

  - Allowing `&scoped` would be simple b/c it doesn't compile atm.
  - Changing `lifetime(*p) = ∞` to `lifetime(*p) = p` later, could be
harder I guess. How restricting would it be to already add it now?
  - The requirement for `ptr = &var` that `lifetime(var) >=
reachability(ptr)` would remain the same.


Hope I fully got the point, I'm not that much into the details of
DIP1000, so please correct me if that's nonsense.
Dec 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 9:28 AM, Martin Nowak wrote:

 Let me suggest a reason, transitive scope checking would be more complex
 to implement, is it that?


It's a heluva lot more complicated for the user. And since it is not necessary, 
why pay such a large bill?



 - How certain are we that this is enough? In other words, why is `tmp =
 &a.b; tmp2 = &tmp.c; tmp2.d.e` not necessary.


It would be necessary in order to build non-trivial data structures on the 
stack. How necessary is that? I've never needed to in decades of programming. I 
don't think there are any instances in Phobos or dmd.

If one really wants to do that, there's always  system code.

Is it worth burdening everything with a feature when the use case for it is 
rare? I don't think so.

For example, consider 'volatile' in C++. The use case for it is rare (very 
rare). But there's an enormous amount of spec verbiage and thought dedicated to 
how to weave it into the type system, overloading rules, conversion rules, 
casting rules, promotion rules, mangling rules, deduction rules, on and on.
It's 
just not worth it.



 - Let's imagine for a short moment, what it would mean to make scope
 transitive at a later point, if we'd ever find enough reasons to do so.
 Maximizing future options is always good.


This kind of annotation system was first proposed 10 years ago. I've been 
thinking about how to make it simpler ever since, and the original has not 
improved with age.

I also have a lot of experience with transitive const, and how disruptive that 
was, and how many years it took to work the problems out (Andrei says they're 
still a problem).
Dec 18 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
BTW, Rust is a system with annotations that can be applied at any level. Yet it 
still needs  system code for many data structures (any with cycles in it).

Rust's system also (so I've read) requires one to re-architect data structures. 
It's not remotely a matter of just adding annotations.

That alone makes it unacceptable for D.

D's 'return scope' system covers routine coding use. More complex things can be 
encapsulated in  trusted code. I believe this is the sweet spot that will make 
it easy to transition to, use and be effective.

---

It's true that I've repeatedly had problems explaining how it works to you, 
Andrei, Dicebot and Mathias. I can only infer that having a much more complex 
system, which scope as a type constructor would be, would be infinitely worse.
I 
find the prospect of debugging a data structure with 'scope' sprinkled around
on 
random edges in it to be terrifying (as well as a combinatorial template 
instantiation explosion problem).
Dec 18 2016
prev sibling parent Mathias Lang <mathias.lang sociomantic.com> writes:
On Sunday, 18 December 2016 at 17:56:26 UTC, Walter Bright wrote:

 On 12/18/2016 9:28 AM, Martin Nowak wrote:

 Let me suggest a reason, transitive scope checking would be 
 more complex
 to implement, is it that?


 It's a heluva lot more complicated for the user. And since it 
 is not necessary, why pay such a large bill?



 - How certain are we that this is enough? In other words, why 
 is `tmp =
 &a.b; tmp2 = &tmp.c; tmp2.d.e` not necessary.


 It would be necessary in order to build non-trivial data 
 structures on the stack. How necessary is that? I've never 
 needed to in decades of programming. I don't think there are 
 any instances in Phobos or dmd.

 If one really wants to do that, there's always  system code.

 Is it worth burdening everything with a feature when the use 
 case for it is rare? I don't think so.

 For example, consider 'volatile' in C++. The use case for it is 
 rare (very rare). But there's an enormous amount of spec 
 verbiage and thought dedicated to how to weave it into the type 
 system, overloading rules, conversion rules, casting rules, 
 promotion rules, mangling rules, deduction rules, on and on. 
 It's just not worth it.



 - Let's imagine for a short moment, what it would mean to make 
 scope
 transitive at a later point, if we'd ever find enough reasons 
 to do so.
 Maximizing future options is always good.


 This kind of annotation system was first proposed 10 years ago. 
 I've been thinking about how to make it simpler ever since, and 
 the original has not improved with age.

 I also have a lot of experience with transitive const, and how 
 disruptive that was, and how many years it took to work the 
 problems out (Andrei says they're still a problem).



Thank you so much for this explanation.
It clears up many of the points on which I had questions.
Once I run out of simple test cases, I'll try to toy around with 
a more advanced RefCounted design.
Dec 22 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/16/2016 9:08 AM, Mathias Lang wrote:

 void main ()  safe
 {
     int* a = escape();
 }

 int* escape ()  safe
 {
     int i;
     int*[3] a = [ &i, null, null ];
     return bar(&a[0]);
 }

 int* bar (scope int** x)  safe
 {
     return foo(*x);
 }

 int* foo (int* x)  safe { return x; }


https://github.com/dlang/dmd/pull/6329
Dec 17 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Saturday, 17 December 2016 at 14:09:59 UTC, Walter Bright 
wrote:

 On 12/16/2016 9:08 AM, Mathias Lang wrote:

 void main ()  safe
 {
     int* a = escape();
 }

 int* escape ()  safe
 {
     int i;
     int*[3] a = [ &i, null, null ];
     return bar(&a[0]);
 }

 int* bar (scope int** x)  safe
 {
     return foo(*x);
 }

 int* foo (int* x)  safe { return x; }


 https://github.com/dlang/dmd/pull/6329


Merged both of your P.R. in a local branch, and tested again:

```
void main ()  safe
{
     int[] a = escape();
}

int[] escape ()  safe
{
     Foo f;
     return f.foo;
}

struct Foo
{
     int[10] v;
     int[] foo () return  safe { return this.v; }
}
```


 Taking the address of a scope pointer is invalid


Thanks for the clarification. I think this will prove pretty 
limiting in the future. For example, a stack allocator will 
return data that is already typed as `scope`, and as a 
consequence, interior pointers will not be expressible. I'll see 
if I can form an example in terms of code.


And since I started toying around with aggregate methods, here's 
another reason why changing the STC of a field member is a very 
bad idea:
```
void main ()  safe
{
     int* a = escape();
}

int* escape ()  safe
{
     int i;
     Foo f;
     f.v = &i;
     return f.foo;
}

struct Foo
{
     int* v;
     int* foo ()  safe { return this.v; }
}
```

Note: The definition of `Foo` might be in another module, and 
just visible via an header (.di) file.
Dec 17 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2016 7:38 AM, Mathias Lang wrote:

 Taking the address of a scope pointer is invalid


 Thanks for the clarification. I think this will prove pretty limiting in the
 future. For example, a stack allocator will return data that is already typed
as
 `scope`, and as a consequence, interior pointers will not be expressible. I'll
 see if I can form an example in terms of code.


It should work fine.



 And since I started toying around with aggregate methods, here's another reason
 why changing the STC of a field member is a very bad idea:


It doesn't change the STC of field members. Only of locals.

---

I'll investigate the reports. Thanks!
Dec 17 2016
parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 18 December 2016 at 02:43:59 UTC, Walter Bright wrote:

Just a general remark, not specific to the content of this 
conversation.

Being a bit more specific in answers easily makes them less 
misunderstandable.


 On 12/17/2016 7:38 AM, Mathias Lang wrote:

 For example, a stack allocator will return data that is 
 already typed as
 `scope`, and as a consequence, interior pointers will not be 
 expressible. I'll
 see if I can form an example in terms of code.


 It should work fine.


What does "it" refer to here, doesn't hurt much to write 
"Internal pointers in scope structs" or so.
Equally "work fine" doesn't state how the "it" is supposed to 
work.


 And since I started toying around with aggregate methods, 
 here's another reason
 why changing the STC of a field member is a very bad idea:


 It doesn't change the STC of field members. Only of locals.


What's it here? Your DIP or the dmd implementation?
Dec 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2016 7:56 PM, Martin Nowak wrote:

 On 12/17/2016 7:38 AM, Mathias Lang wrote:

 For example, a stack allocator will return data that is already typed as
 `scope`, and as a consequence, interior pointers will not be expressible. I'll
 see if I can form an example in terms of code.


 It should work fine.


 What does "it" refer to here, doesn't hurt much to write "Internal pointers in
 scope structs" or so.
 Equally "work fine" doesn't state how the "it" is supposed to work.


A stack allocator is conceptually returning a slice of a static array allocated 
on the stack. This works with scope.

Understanding scope always goes back to understanding how it works with
pointers 
and addresses. Adding layers of abstraction over that can be confusing by 
obfuscating that relationship, but rewrite the abstract as pointers and 
addresses, and the behavior then becomes clear.

The bugs Mathias has posted have (so far) all been the result of an abstraction 
not conforming to how pointers and addresses work, due to a bug in dmd.



 And since I started toying around with aggregate methods, here's another reason
 why changing the STC of a field member is a very bad idea:


 It doesn't change the STC of field members. Only of locals.


 What's it here? Your DIP or the dmd implementation?


The dmd implementation.
Dec 17 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Sunday, 18 December 2016 at 05:56:42 UTC, Walter Bright wrote:

 On 12/17/2016 7:56 PM, Martin Nowak wrote:

 On 12/17/2016 7:38 AM, Mathias Lang wrote:

 For example, a stack allocator will return data that is 
 already typed as
 `scope`, and as a consequence, interior pointers will not be 
 expressible. I'll
 see if I can form an example in terms of code.


 It should work fine.


 What does "it" refer to here, doesn't hurt much to write 
 "Internal pointers in
 scope structs" or so.
 Equally "work fine" doesn't state how the "it" is supposed to 
 work.


 A stack allocator is conceptually returning a slice of a static 
 array allocated on the stack. This works with scope.

 Understanding scope always goes back to understanding how it 
 works with pointers and addresses. Adding layers of abstraction 
 over that can be confusing by obfuscating that relationship, 
 but rewrite the abstract as pointers and addresses, and the 
 behavior then becomes clear.

 The bugs Mathias has posted have (so far) all been the result 
 of an abstraction not conforming to how pointers and addresses 
 work, due to a bug in dmd.


Thanks for expanding on that, as Martin pointed out, it helps 
reviewers a lot.
I don't feel the point about internal pointers to stack-allocated 
data have been addressed, but maybe it'll be better if I just 
craft a test case.


 https://github.com/dlang/dmd/pull/6331


The problem with this solution is that it forces `scope` on 
unrelated method.
E.g.

```
int* escape ()  safe
{
     int i;
     Foo f;
     f.v = &i;
     return f.foo;
}

struct Foo
{
     int* v;
     int* foo ()  safe { return null; }
}
```

Doesn't compile anymore. It might be your intention, but it needs 
to be clearly expressed in the DIP.
Moreover, this code is obviously correct to the reader:

```
struct Foo
{
     int* v;
     int foo ()  safe { return *v; }
}
```

But won't compile. I tried with static arrays just to be sure, 
same outcome.
Dec 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 3:04 AM, Mathias Lang wrote:

 The problem with this solution is that it forces `scope` on unrelated method.
 E.g.

 ```
 int* escape ()  safe
 {
     int i;
     Foo f;
     f.v = &i;
     return f.foo;
 }

 struct Foo
 {
     int* v;
     int* foo ()  safe { return null; }
 }
 ```

 Doesn't compile anymore.


And it shouldn't. f contains a pointer to a local, and so f must be scope. A 
scoped object can only be used on a scoped method, because otherwise the method 
can squirrel away an escaping copy.



 It might be your intention, but it needs to be clearly
 expressed in the DIP.


The rules need to be clearly expressed, but I don't know how to express every 
consequence of the rules.



 Moreover, this code is obviously correct to the reader:

 ```
 struct Foo
 {
     int* v;
     int foo ()  safe { return *v; }
 }
 ```

 But won't compile. I tried with static arrays just to be sure, same outcome.


It compiles when I try it.
Dec 18 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Sunday, 18 December 2016 at 15:24:34 UTC, Walter Bright wrote:

 Moreover, this code is obviously correct to the reader:

 ```
 struct Foo
 {
     int* v;
     int foo ()  safe { return *v; }
 }
 ```

 But won't compile. I tried with static arrays just to be sure, 
 same outcome.


 It compiles when I try it.


I didn't meant as a standalone example, but in the context of the 
previous snippet. Full code:

```
int bar ()  safe
{
     int i = 42;
     Foo f;
     f.v = &i;
     return f.foo(); // Doesn't compile
}

struct Foo
{
     int* v;
     int foo ()  safe { return *v; }
}
```
Dec 18 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 8:23 AM, Mathias Lang wrote:

 I didn't meant as a standalone example, but in the context of the previous
 snippet. Full code:

 ```
 int bar ()  safe
 {
     int i = 42;
     Foo f;
     f.v = &i;
     return f.foo(); // Doesn't compile
 }

 struct Foo
 {
     int* v;
     int foo ()  safe { return *v; }
 }
 ```


That doesn't compile because the Foo could be written as:

static int* s;
struct Foo
{
     int* v;
     int foo ()  safe { s = v; return *v; }
}

The compiler can't know that from the interface to Foo.foo, and so must assume 
the worst. The safety features are about guarantees, not having faith that the 
programmer didn't do something like that.

The purpose in adding the 'scope' annotation to Foo.foo is to tell the compiler 
that there's no hanky-panky going on in the implementation. When the compiler 
does compile the implementation, it can then check it against the 'scope'
semantics.
Dec 18 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2016 7:38 AM, Mathias Lang wrote:

 void main ()  safe
 {
     int[] a = escape();
 }

 int[] escape ()  safe
 {
     Foo f;
     return f.foo;
 }

 struct Foo
 {
     int[10] v;
     int[] foo () return  safe { return this.v; }
 }


I've said it's all about pointers and addresses. So let's remove the 
abstractions, and rewrite it in the equivalent pointers and addresses form:

```
void main ()  safe
{
     int* a = escape();
}

int* escape ()  safe
{
     int f;
     return foo(f);  // Error: escaping reference to local variable f
}

int* foo (return ref int i)  safe { return &i; }
```

and indeed it does fail to compile. So the bug in the compiler is failing to 
recognize the abstract version of the pointers and addresses logic.
Dec 17 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2016 10:29 PM, Walter Bright wrote:

 So the bug in the compiler is failing to
 recognize the abstract version of the pointers and addresses logic.


https://github.com/dlang/dmd/pull/6330
Dec 17 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2016 7:38 AM, Mathias Lang wrote:

 int* escape ()  safe
 {
     int i;
     Foo f;
     f.v = &i;
     return f.foo;
 }

 struct Foo
 {
     int* v;
     int* foo ()  safe { return this.v; }
 }


Rewriting to the equivalent:

```
int* escape()  safe
{
     int i;
     Foo f;
     f.v = &i;
     return foo(g);  // Error: scope variable f assigned to non-scope
                     // parameter g calling bug.foo
}

struct Foo
{
     int* v;
}

int* foo(ref Foo g)  safe { return g.v; }
```

So the trouble here is dmd not recognizing that f.foo() is the same as foo(f).
Dec 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2016 11:47 PM, Walter Bright wrote:

 So the trouble here is dmd not recognizing that f.foo() is the same as foo(f).


https://github.com/dlang/dmd/pull/6331
Dec 18 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Sunday, 18 December 2016 at 10:04:57 UTC, Walter Bright wrote:

 On 12/17/2016 11:47 PM, Walter Bright wrote:

 So the trouble here is dmd not recognizing that f.foo() is the 
 same as foo(f).


 https://github.com/dlang/dmd/pull/6331


With the two last P.R. you submitted merged, the following 
compiles

```
void escape ()  safe
{
     Foo f;
     bar(f);
}

void bar (scope ref Foo f)  safe
{
     int[10] i;
     f.v = i;
}

struct Foo
{
     int[] v;
}
```
Dec 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 3:09 AM, Mathias Lang wrote:

 With the two last P.R. you submitted merged, the following compiles

 ```
 void escape ()  safe
 {
     Foo f;
     bar(f);
 }

 void bar (scope ref Foo f)  safe
 {
     int[10] i;
     f.v = i;
 }

 struct Foo
 {
     int[] v;
 }
 ```


Thanks, I'll look at it tomorrow.
Dec 18 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Sunday, 18 December 2016 at 15:25:55 UTC, Walter Bright wrote:

 On 12/18/2016 3:09 AM, Mathias Lang wrote:

 With the two last P.R. you submitted merged, the following 
 compiles

 ```
 void escape ()  safe
 {
     Foo f;
     bar(f);
 }

 void bar (scope ref Foo f)  safe
 {
     int[10] i;
     f.v = i;
 }

 struct Foo
 {
     int[] v;
 }
 ```


 Thanks, I'll look at it tomorrow.


Just tested again with the HEAD of scope (e46873f66), it compiles 
with just `ref` instead of `scope ref` for bar's parameter.
Which brings me to the question, is `scope ref` supposed to apply 
to the `ref` or to the type (i.e. `Foo` in this case) ?
Dec 27 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/27/2016 2:18 PM, Mathias Lang wrote:

 Which brings me to the question, is `scope ref` supposed to apply to the `ref`
 or to the type (i.e. `Foo` in this case) ?


scope refers to the parameter's value, ref refers to the parameter's address. 
The two are independent of each other.
Dec 27 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Wednesday, 28 December 2016 at 01:25:22 UTC, Walter Bright 
wrote:

 On 12/27/2016 2:18 PM, Mathias Lang wrote:

 Which brings me to the question, is `scope ref` supposed to 
 apply to the `ref`
 or to the type (i.e. `Foo` in this case) ?


 scope refers to the parameter's value, ref refers to the 
 parameter's address. The two are independent of each other.


Thanks (Y)

I just started testing methods, but didn't get far, because the 
following doesn't compile:
```
void getObj ()  safe
{
     scope o = new Bar;
     auto p = o.foo;   // Line 9
}

class Bar
{
     Object foo () return scope  safe { return this; }
}

// scope1.d(9): Error: scope variable o assigned to non-scope 
parameter this calling scope1.Bar.foo
```

However making `foo` a function which accept a class works:
```
Object foo (return scope Object this_)  safe
{
     return this_;
}
```

Also, it looks like delegates to member functions are not marked 
scope where they should. The following compiles, but obviously 
should not since the context pointer of the delegate will be the 
instance `b`, which is on the stack.

```
void main ()  safe
{
     fwd();
     auto x1 = Generator1();
     auto x2 = Generator2();
}

int delegate()  safe Generator1;
int delegate()  safe Generator2;

struct G1
{
     int generator ()  safe { return 42; }
}

class G2
{
     int generator ()  safe { return 42; }
}

void fwd ()  safe  nogc
{
     G1 g1;
     scope g2 = new G2;
     Generator1 = &g1.generator;
     Generator2 = &g2.generator;
}
```
Dec 30 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Thanks, I'll look into these.
Dec 30 2016
prev sibling parent reply Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 12/30/2016 01:12 PM, Mathias Lang wrote:

Can we please start to track all DIP1000 implementation bugs on
Bugzilla. It's becoming quite chaotic to keep an overview of this.
Until someone creates a scope or dip1000 keyword (does  braddr maintain
Bugzilla), we can use [scope] in the title, that's search-able.

https://issues.dlang.org/buglist.cgi?list_id=212784&query_format=advanced&resolution=---&short_desc=%5Bscope%5D&short_desc_type=substring
Dec 31 2016
next sibling parent Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 12/31/2016 05:18 PM, Martin Nowak wrote:

 On 12/30/2016 01:12 PM, Mathias Lang wrote:
 
 Can we please start to track all DIP1000 implementation bugs on
 Bugzilla. It's becoming quite chaotic to keep an overview of this.
 Until someone creates a scope or dip1000 keyword (does  braddr maintain
 Bugzilla), we can use [scope] in the title, that's search-able.
 
 https://issues.dlang.org/buglist.cgi?list_id=212784&query_format=advanced&resolution=---&short_desc=%5Bscope%5D&short_desc_type=substring



Just filed the first one, https://issues.dlang.org/show_bug.cgi?id=17049.
Dec 31 2016
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Saturday, 31 December 2016 at 16:18:11 UTC, Martin Nowak wrote:

 https://issues.dlang.org/buglist.cgi?list_id=212784&query_format=advanced&resolution=---&short_desc=%5Bscope%5D&short_desc_type=substring


Let's use the existing keyword (safe) AND the [scope] tag in the 
title.
Dec 31 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/16/2016 8:43 AM, Mathias Lang wrote:

 ```
 struct Foo { int* ptr; }

 void main ()  safe
 {
     int* a = escape();
 }

 int* escape ()  safe
 {
     int i;
     Foo f;
     scope int** x = &f.ptr;
     f.ptr = &i;

     return bar(x);
 }

 int* bar ( scope int** ptr )  safe
 {
   return *ptr;
 }
 ```

 Compiles with `./src/dmd -transition=safe -dip25 test.d` on commit
 fbbfce8f6e0afb716ec96ba3dc2e82b3536cbaac (with the latest 2 P.R. merged).


https://github.com/dlang/dmd/pull/6328
Dec 17 2016
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Saturday, 10 December 2016 at 05:00:42 UTC, Walter Bright 
wrote:

 - what about return ref without scope

 Does return ref on parameters makes sense without scope?


 Yes, see dip25.


 Can you take the address of an unscoped return ref result?


 Taking the address of a ref makes it a scope.


Thanks, that was likely the piece of information I was missing.
So even with all the scope changes, a return ref parameter 
provides the same guarantees, it can't be escaped in a  safe 
function.
Dec 15 2016
prev sibling parent reply Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 10/16/2016 03:45 PM, Dicebot wrote:

 Trying to solve it, I have been experimenting lately with various
 acceptance tests trying to come up with useful design snippets enabled
 by that PR. Rationale is that if actual semantics are clear and examples
 are all validated as included test cases, review of the code itself
 becomes less important.
 
 I'd like to have more open feedback about it because in its current
 shape PR5972 doesn't seem useful enough to me to be merged at all - and
 much less powerful than DIP1000 seemed to promise.
 
 I'll start separate sub-threads with more details.



Just build a scope preview, http://nightlies.dlang.org/dmd-scope/.

You can just download it or use

    curl -fsS https://dlang.org/install.sh | bash -s dmd-scope

to try it (also works on Travis-CI anything else using the install.sh
script).

-Martin
Dec 19 2016
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Tuesday, 20 December 2016 at 00:15:23 UTC, Martin Nowak wrote:

 On 10/16/2016 03:45 PM, Dicebot wrote:

 Trying to solve it, I have been experimenting lately with 
 various acceptance tests trying to come up with useful design 
 snippets enabled by that PR. Rationale is that if actual 
 semantics are clear and examples are all validated as included 
 test cases, review of the code itself becomes less important.
 
 I'd like to have more open feedback about it because in its 
 current shape PR5972 doesn't seem useful enough to me to be 
 merged at all - and much less powerful than DIP1000 seemed to 
 promise.
 
 I'll start separate sub-threads with more details.



 Just build a scope preview, 
 http://nightlies.dlang.org/dmd-scope/.

 You can just download it or use

     curl -fsS https://dlang.org/install.sh | bash -s dmd-scope

 to try it (also works on Travis-CI anything else using the 
 install.sh
 script).

 -Martin


That's great Martin! What's the delay between a PR merged in 
https://github.com/dlang/dmd/tree/scope and being available for 
testing via the installer? Is it a nightly build of the scope 
branch, as the URL suggests?
Dec 21 2016
parent Martin Nowak <code dawg.eu> writes:
On Wednesday, 21 December 2016 at 13:48:11 UTC, ZombineDev wrote:

 That's great Martin! What's the delay between a PR merged in 
 https://github.com/dlang/dmd/tree/scope and being available for 
 testing via the installer? Is it a nightly build of the scope 
 branch, as the URL suggests?


Preview builds of branches are manually build at the moment 
(whenever there is sth. interesting).
Nightlies are only for master atm.
Dec 23 2016