www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - DIP1000: Scoped Pointers

reply Dicebot <public dicebot.lv> writes:
The first DIP has just landed into the new queue. It is a 
proposal from language authors and thus it bypasses usual 
nitpicking process and proceeds straight to requesting community 
(your!) feedback.

Essentially, it is an attempt to solve reference lifetime problem 
by extending implementation of `scope` keyword.

Proposal text: 
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

Few notes:

- Please submit pull requests to adjust the markdown document if 
you want to propose any improvements (mentioning  WalterBright 
and  andralex for confirmation).
- The proposal refers to a number of other documents and it is 
recommended to become familiar at least briefly with all of them.
- At this point the question I'd personally suggest to be 
evaluated is "does this proposal enable enough useful designs?". 
A good check would be to try taking some of your projects and see 
if having DIP1000 approved and implemented could improve them.
Aug 10 2016
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 10 August 2016 at 20:35:23 UTC, Dicebot wrote:
 The first DIP has just landed into the new queue. It is a 
 proposal from language authors and thus it bypasses usual 
 nitpicking process and proceeds straight to requesting 
 community (your!) feedback.

 [...]
How will the infinite lifetime of ArrayLiteral and ArrayLiteral[constant] interact with LDC's GC to stack promotion pass?
Aug 10 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2016 4:56 PM, Nicholas Wilson wrote:
 How will the infinite lifetime of ArrayLiteral and ArrayLiteral[constant]
 interact with LDC's GC to stack promotion pass?
I don't know about how that works in LDC, but general such a promotion can only be done if the compiler can prove there are no escaping pointers to the data.
Aug 10 2016
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 11/08/2016 8:35 AM, Dicebot wrote:
 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

 Few notes:

 - Please submit pull requests to adjust the markdown document if you
 want to propose any improvements (mentioning  WalterBright and  andralex
 for confirmation).
 - The proposal refers to a number of other documents and it is
 recommended to become familiar at least briefly with all of them.
 - At this point the question I'd personally suggest to be evaluated is
 "does this proposal enable enough useful designs?". A good check would
 be to try taking some of your projects and see if having DIP1000
 approved and implemented could improve them.
Question: I see RefCountedSlice example, does this mean if I alias this say like: struct FooBar; struct Foo { FooBar* v; scope FooBar* get() { return v; } alias this get; } That it will operate correctly in the below case? func(myFoo); void func(scope FooBar*) If this does work, this is a major addition that I've been waiting for, for my managed memory concept! https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/memory/managed.d After this I'll only need proper ref counting in the language ;)
Aug 10 2016
next sibling parent Bill Baxter via Digitalmars-d-announce writes:
This bit seems odd:

T func(T* t) {
  return t; // ok
}

Is there an implicit conversion from T* to T?


On Wed, Aug 10, 2016 at 10:05 PM, rikki cattermole via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 11/08/2016 8:35 AM, Dicebot wrote:

 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

 Few notes:

 - Please submit pull requests to adjust the markdown document if you
 want to propose any improvements (mentioning  WalterBright and  andralex
 for confirmation).
 - The proposal refers to a number of other documents and it is
 recommended to become familiar at least briefly with all of them.
 - At this point the question I'd personally suggest to be evaluated is
 "does this proposal enable enough useful designs?". A good check would
 be to try taking some of your projects and see if having DIP1000
 approved and implemented could improve them.
Question: I see RefCountedSlice example, does this mean if I alias this say like: struct FooBar; struct Foo { FooBar* v; scope FooBar* get() { return v; } alias this get; } That it will operate correctly in the below case? func(myFoo); void func(scope FooBar*) If this does work, this is a major addition that I've been waiting for, for my managed memory concept! https://github.com/rikkimax/al phaPhobos/blob/master/source/std/experimental/memory/managed.d After this I'll only need proper ref counting in the language ;)
Aug 10 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2016 10:05 PM, rikki cattermole wrote:
 Question:
 I see RefCountedSlice example, does this mean if I alias this say like:

 struct FooBar;

 struct Foo {
     FooBar* v;

     scope FooBar* get() { return v; }
     alias this get;
 }

 That it will operate correctly in the below case?

 func(myFoo);
 void func(scope FooBar*)
Yes, it better :-)
Aug 10 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 11/08/2016 5:41 PM, Walter Bright wrote:
 On 8/10/2016 10:05 PM, rikki cattermole wrote:
 Question:
 I see RefCountedSlice example, does this mean if I alias this say like:

 struct FooBar;

 struct Foo {
     FooBar* v;

     scope FooBar* get() { return v; }
     alias this get;
 }

 That it will operate correctly in the below case?

 func(myFoo);
 void func(scope FooBar*)
Yes, it better :-)
Perfect :)
Aug 10 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2016 11:36 PM, rikki cattermole wrote:
 Perfect :)
The nice thing about this scheme is it can do some things that Rust can't (and Rust can do things that this can't). I suppose it will balance out.
Aug 11 2016
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 11 August 2016 at 07:48:18 UTC, Walter Bright wrote:
 On 8/10/2016 11:36 PM, rikki cattermole wrote:
 Perfect :)
The nice thing about this scheme is it can do some things that Rust can't (and Rust can do things that this can't). I suppose it will balance out.
If this is successfully implemented, what will D not be able to do, that Rust can/will?
Aug 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 5:33 AM, Nordlöw wrote:
 If this is successfully implemented, what will D not be able to do, that Rust
 can/will?
Have ownership semantics for pointers in more complex data structures. In D you'll have to do such with ref counted objects. On the other hand, D code can reference mutable globals in safe code, whereas Rust cannot. Assuming, of course, I understood the Rust semantics correctly.
Aug 12 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12.08.2016 21:39, Walter Bright wrote:
 On 8/12/2016 5:33 AM, Nordlöw wrote:
 If this is successfully implemented, what will D not be able to do,
 that Rust
 can/will?
Have ownership semantics for pointers in more complex data structures. In D you'll have to do such with ref counted objects. On the other hand, D code can reference mutable globals in safe code, whereas Rust cannot. Assuming, of course, I understood the Rust semantics correctly.
AFAIU Rust has safe static TLS.
Aug 12 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 1:08 PM, Timon Gehr wrote:
 On 12.08.2016 21:39, Walter Bright wrote:
 On 8/12/2016 5:33 AM, Nordlöw wrote:
 If this is successfully implemented, what will D not be able to do,
 that Rust
 can/will?
Have ownership semantics for pointers in more complex data structures. In D you'll have to do such with ref counted objects. On the other hand, D code can reference mutable globals in safe code, whereas Rust cannot. Assuming, of course, I understood the Rust semantics correctly.
AFAIU Rust has safe static TLS.
Perhaps. Rust changes regularly. Some of the top hits in google are out of date.
Aug 12 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11.08.2016 09:48, Walter Bright wrote:
 On 8/10/2016 11:36 PM, rikki cattermole wrote:
 Perfect :)
The nice thing about this scheme is it can do some things that Rust can't
What are some of those things?
Aug 12 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 12:03 PM, Timon Gehr wrote:
 On 11.08.2016 09:48, Walter Bright wrote:
 On 8/10/2016 11:36 PM, rikki cattermole wrote:
 Perfect :)
The nice thing about this scheme is it can do some things that Rust can't
What are some of those things?
Accessing mutable globals in a function comes to mind.
Aug 12 2016
prev sibling next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
This looks like the best proposal so far. Many useful ideas, and at the 
first glance it looks like a big step forward. I also really like how 
the useful stack allocation behavior for closures and new'ed values is 
kept alive.

What would be nice to add is a behavior specification for 'scope' member 
variables (lifetime considered equal or slightly shorter than parent 
object lifetime). For example the `RefCountedSlice.payload` and `count` 
fields could be annotated with 'scope' to let the compiler actually 
guarantee that they won't be accessible in a way that conflicts with 
their lifetime (i.e. the 'scope' return of 'opIndex' would actually be 
enforced).

That will just leave one hole in conjunction with the  trusted 
destructor, which is (presumably) not easy to fix without much larger 
changes to the type system, as well as to how container types are built. 
It is still vulnerable to artificial shortening of the elements' 
lifetime, e.g. by using opAssign() or destroy():

 safe {
     RefCountedSlice!int s = ...;
     scope int* el;
     el = &s[0];
     s = RefCountedSlice.init;
     *el = 12; // oops
}

A similar issue affects the library implementation of isolated memory 
that I did a while ago:

 safe {
     class C { int* x; }

     // c is guaranteed to be only reachable through this variable
     Isolated!C c = makeIsolated!C();

     // c.x is a  property that returns a specially wrapped reference to
     // the actual C.x field - with this DIP this is similar to a 'scope'
     // return, but acts transitively
     Scoped!(int*) x = c.x;

     // one of the benefits of Isolated!T is that it allows  safe
     // conversion to immutable:
     immutable(C) ci = c.freeze();
     // c gets cleared by freeze() to disallow any further modifications

     // but, oops, x is still there and can be used to modify the now
     // immutable contents of ci.x
     *x = 12;
}

Disallowing the assignment of scope return references to local scope 
references (either by default, or using some form of additional 
inference/annotation) would solve this particular issue, but not the 
issue in general (the assignment/destruction could for example happen in 
a nested function call).
Aug 11 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 6:38 AM, Sönke Ludwig wrote:
 What would be nice to add is a behavior specification for 'scope' member
 variables (lifetime considered equal or slightly shorter than parent object
 lifetime). For example the `RefCountedSlice.payload` and `count` fields could
be
 annotated with 'scope' to let the compiler actually guarantee that they won't
be
 accessible in a way that conflicts with their lifetime (i.e. the 'scope' return
 of 'opIndex' would actually be enforced).
That adds a fair amount of complication I haven't worked through.
Aug 11 2016
parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Thursday, 11 August 2016 at 22:03:02 UTC, Walter Bright wrote:
 On 8/11/2016 6:38 AM, Sönke Ludwig wrote:
 What would be nice to add is a behavior specification for 
 'scope' member
 variables (lifetime considered equal or slightly shorter than 
 parent object
 lifetime). For example the `RefCountedSlice.payload` and 
 `count` fields could be
 annotated with 'scope' to let the compiler actually guarantee 
 that they won't be
 accessible in a way that conflicts with their lifetime (i.e. 
 the 'scope' return
 of 'opIndex' would actually be enforced).
That adds a fair amount of complication I haven't worked through.
It can probably be done by lowering: scope T* payload; Is conceptually: private T* payload_; property scope T* payload() scope { return payload_; }
Aug 12 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,a,n,n,o,u,n,c,e
Subject: Re: DIP1000: Scoped Pointers
References: <pqsiqmkxenrwxoruzaml forum.dlang.org>
 <nohv4n$1o42$1 digitalmars.com>
In-Reply-To: <nohv4n$1o42$1 digitalmars.com>

--PuhSQ8eFTx0cCDuR2Gs09ecvkqGJHBa69
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/11/2016 04:38 PM, S=C3=B6nke Ludwig wrote:
 That will just leave one hole in conjunction with the  trusted
 destructor, which is (presumably) not easy to fix without much larger
 changes to the type system, as well as to how container types are built=
=2E
 It is still vulnerable to artificial shortening of the elements'
 lifetime, e.g. by using opAssign() or destroy():
=20
  safe {
     RefCountedSlice!int s =3D ...;
     scope int* el;
     el =3D &s[0];
     s =3D RefCountedSlice.init;
     *el =3D 12; // oops
 }
I asked Walter about this in more details and right now plan is to address it in a separate DIP that provides more integration between reference counting and compiler. Within DIP1000 terms such destructor must not be marked as safe - essentially, it will only enable safe usage of stack allocated data in its initial form.
 A similar issue affects the library implementation of isolated memory
 that I did a while ago:
=20
  safe {
     class C { int* x; }
=20
     // c is guaranteed to be only reachable through this variable
     Isolated!C c =3D makeIsolated!C();
=20
     // c.x is a  property that returns a specially wrapped reference to=
     // the actual C.x field - with this DIP this is similar to a 'scope=
'
     // return, but acts transitively
     Scoped!(int*) x =3D c.x;
=20
     // one of the benefits of Isolated!T is that it allows  safe
     // conversion to immutable:
     immutable(C) ci =3D c.freeze();
     // c gets cleared by freeze() to disallow any further modifications=
=20
     // but, oops, x is still there and can be used to modify the now
     // immutable contents of ci.x
     *x =3D 12;
 }
=20
 Disallowing the assignment of scope return references to local scope
 references (either by default, or using some form of additional
 inference/annotation) would solve this particular issue, but not the
 issue in general (the assignment/destruction could for example happen i=
n
 a nested function call).
Note that using scope return in its most basic form will exactly prevent the assigning of reference to a variable because it limits lifetime to expression. --PuhSQ8eFTx0cCDuR2Gs09ecvkqGJHBa69--
Aug 18 2016
parent Nick Treleaven <nick geany.org> writes:
On Thursday, 18 August 2016 at 17:05:05 UTC, Dicebot wrote:
 On 08/11/2016 04:38 PM, Sönke Ludwig wrote:
 That will just leave one hole in conjunction with the  trusted 
 destructor, which is (presumably) not easy to fix without much 
 larger changes to the type system, as well as to how container 
 types are built. It is still vulnerable to artificial 
 shortening of the elements' lifetime, e.g. by using opAssign() 
 or destroy():
 
  safe {
     RefCountedSlice!int s = ...;
     scope int* el;
     el = &s[0];
     s = RefCountedSlice.init;
     *el = 12; // oops
 }
I asked Walter about this in more details and right now plan is to address it in a separate DIP that provides more integration between reference counting and compiler. Within DIP1000 terms such destructor must not be marked as safe - essentially, it will only enable safe usage of stack allocated data in its initial form.
I think RefCountedSlice can have a trusted destructor so long as opAssign is system. (I'll likely make a PR to the DIP soon).
Aug 22 2016
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 10 August 2016 at 20:35:23 UTC, Dicebot wrote:
 - At this point the question I'd personally suggest to be 
 evaluated is "does this proposal enable enough useful 
 designs?". A good check would be to try taking some of your 
 projects and see if having DIP1000 approved and implemented 
 could improve them.
There's a use-case that relates to some of our discussions together in another context, about structs or classes that borrow data via ref: struct MyWrapperStruct (T) { private T* data; public this (ref T input) { this.data = &input; } // ... other functionality relies on // this.data being valid throughout // the lifetime of the struct // note, we could probably avoid everyone // having to use raw pointers as above if // we could use scope properties to create // a `Ref` borrowed-pointer type that would // complement the existing `Unique` } I don't see any examples touching on this, but it would be very useful for implementing e.g. InputRange structs which cannot be copied by value, yet which need to be usable with UFCS range chains. Any chance the proposal authors could add some examples of how scope could affect class/struct fields which borrow data by reference (meaning the class/struct instance should not escape the scope of the input data)?
Aug 11 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 11 August 2016 at 19:59:22 UTC, Joseph Rushton 
Wakeling wrote:
 Any chance the proposal authors could add some examples of how 
 scope could affect class/struct fields which borrow data by 
 reference (meaning the class/struct instance should not escape 
 the scope of the input data)?
Some context for my interest here: https://forum.dlang.org/post/jvgguodpfuaicsvplcgp forum.dlang.org TL;DR is, I think that this functionality potentially unlocks a solution for how to implement really effective and safe solutions for range-based random number functionality (both memory-safe and statistically safe).
Aug 11 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 12:59 PM, Joseph Rushton Wakeling wrote:
 There's a use-case that relates to some of our discussions together in another
 context, about structs or classes that borrow data via ref:
The scheme does not implement borrowing. References to internal data should be returned via 'return ref' or 'return scope', where their usage will be limited to the expression they appear in.
Aug 11 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 11 August 2016 at 22:07:57 UTC, Walter Bright wrote:
 The scheme does not implement borrowing. References to internal 
 data should be returned via 'return ref' or 'return scope', 
 where their usage will be limited to the expression they appear 
 in.
I want to make sure we have the same understanding here: the use-case I'm interested in is a data structure that needs to hold a reference to data it does not own -- where, obviously, the lifetime of the data structure cannot outlive the lifetime of the data it references. Surely this scope proposal ought to address that use-case?
Aug 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 4:12 AM, Joseph Rushton Wakeling wrote:
 On Thursday, 11 August 2016 at 22:07:57 UTC, Walter Bright wrote:
 The scheme does not implement borrowing. References to internal data should be
 returned via 'return ref' or 'return scope', where their usage will be limited
 to the expression they appear in.
I want to make sure we have the same understanding here: the use-case I'm interested in is a data structure that needs to hold a reference to data it does not own -- where, obviously, the lifetime of the data structure cannot outlive the lifetime of the data it references. Surely this scope proposal ought to address that use-case?
Using ref counted objects should deal with that nicely.
Aug 12 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Friday, 12 August 2016 at 12:01:41 UTC, Walter Bright wrote:
 On 8/12/2016 4:12 AM, Joseph Rushton Wakeling wrote:
 On Thursday, 11 August 2016 at 22:07:57 UTC, Walter Bright 
 wrote:
 The scheme does not implement borrowing. References to 
 internal data should be
 returned via 'return ref' or 'return scope', where their 
 usage will be limited
 to the expression they appear in.
I want to make sure we have the same understanding here: the use-case I'm interested in is a data structure that needs to hold a reference to data it does not own -- where, obviously, the lifetime of the data structure cannot outlive the lifetime of the data it references. Surely this scope proposal ought to address that use-case?
Using ref counted objects should deal with that nicely.
I'm not sure I follow. I'm looking for the ability to guarantee that a pointer to a stack-allocated entity will not go out of scope; I'd rather not have to choose between GC allocation or RC allocation (which I presume would both be on the heap...?). Or am I missing some of the potential uses of RC?
Aug 12 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Friday, 12 August 2016 at 12:51:26 UTC, Joseph Rushton 
Wakeling wrote:
 I'm not sure I follow.  I'm looking for the ability to 
 guarantee that a pointer to a stack-allocated entity will not 
 go out of scope
... more precisely, that the pointer will not become invalid because the data it points to goes out of scope.
Aug 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 5:54 AM, Joseph Rushton Wakeling wrote:
 On Friday, 12 August 2016 at 12:51:26 UTC, Joseph Rushton Wakeling wrote:
 I'm not sure I follow.  I'm looking for the ability to guarantee that a
 pointer to a stack-allocated entity will not go out of scope
... more precisely, that the pointer will not become invalid because the data it points to goes out of scope.
That's just what this DIP addresses. struct MyWrapperStruct (T) { private T* data; public this (ref T input) { this.data = &input; // error: not allowed to take address of ref variable } } The DIP does not add ownership annotations or semantics.
Aug 12 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Friday, 12 August 2016 at 19:37:47 UTC, Walter Bright wrote:
 That's just what this DIP addresses.

     struct MyWrapperStruct (T)
     {
         private T* data;

         public this (ref T input)
         {
             this.data = &input; // error: not allowed to take 
 address of ref variable
         }
     }

 The DIP does not add ownership annotations or semantics.
Unless I've misunderstood you, that doesn't address my use-case -- it outright bans it! The above code is unsafe only if the lifetime of `data` outlives the lifetime of `input`. Surely the new scope rules should be able to distinguish the cases? If that's already envisioned, how would that work? BTW, the application here is a design that Dicebot and I have been discussing since DConf 2015 to address the concerns related to range implementations of random number generation and random algorithms.
Aug 13 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/13/2016 1:13 AM, Joseph Rushton Wakeling wrote:
 On Friday, 12 August 2016 at 19:37:47 UTC, Walter Bright wrote:
 That's just what this DIP addresses.

     struct MyWrapperStruct (T)
     {
         private T* data;

         public this (ref T input)
         {
             this.data = &input; // error: not allowed to take address of ref
 variable
         }
     }

 The DIP does not add ownership annotations or semantics.
Unless I've misunderstood you, that doesn't address my use-case -- it outright bans it!
Taking the address of a ref variable has not been allowed in safe code for a long time.
 The above code is unsafe only if the lifetime of `data` outlives the lifetime
of
 `input`.  Surely the new scope rules should be able to distinguish the cases?
 If that's already envisioned, how would that work?
That depends on how the instance of MyWrapperStruct is allocated. How did you intend to allocate it?
Aug 13 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 13 August 2016 at 11:09:05 UTC, Walter Bright wrote:
 Taking the address of a ref variable has not been allowed in 
  safe code for a long time.
Which is understandable given things as they are, but which could probably be relaxed given good scope/lifetime analysis by the compiler...?
 The above code is unsafe only if the lifetime of `data` 
 outlives the lifetime of
 `input`.  Surely the new scope rules should be able to 
 distinguish the cases?
 If that's already envisioned, how would that work?
That depends on how the instance of MyWrapperStruct is allocated. How did you intend to allocate it?
As a normal stack variable. Is that problematic? And if not, what would be problematic cases?
Aug 13 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/13/2016 5:02 AM, Joseph Rushton Wakeling wrote:
 On Saturday, 13 August 2016 at 11:09:05 UTC, Walter Bright wrote:
 Taking the address of a ref variable has not been allowed in  safe code for a
 long time.
Which is understandable given things as they are, but which could probably be relaxed given good scope/lifetime analysis by the compiler...?
It's relaxed in system code.
 The above code is unsafe only if the lifetime of `data` outlives the lifetime
of
 `input`.  Surely the new scope rules should be able to distinguish the cases?
 If that's already envisioned, how would that work?
That depends on how the instance of MyWrapperStruct is allocated. How did you intend to allocate it?
As a normal stack variable. Is that problematic?
Shouldn't be.
 And if not, what would be problematic cases?
Allocating it on the heap.
Aug 13 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 13 August 2016 at 19:51:07 UTC, Walter Bright wrote:
 On 8/13/2016 5:02 AM, Joseph Rushton Wakeling wrote:
 On Saturday, 13 August 2016 at 11:09:05 UTC, Walter Bright 
 wrote:
 Taking the address of a ref variable has not been allowed in 
  safe code for a
 long time.
Which is understandable given things as they are, but which could probably be relaxed given good scope/lifetime analysis by the compiler...?
It's relaxed in system code.
Sure, but doesn't the envisioned DIP create the circumstances in which it could also be permitted in safe code where the compiler can guarantee that the pointer's lifetime will not outlive the data referred to?
 The above code is unsafe only if the lifetime of `data` 
 outlives the lifetime of
 `input`.  Surely the new scope rules should be able to 
 distinguish the cases?
 If that's already envisioned, how would that work?
That depends on how the instance of MyWrapperStruct is allocated. How did you intend to allocate it?
As a normal stack variable. Is that problematic?
Shouldn't be.
 And if not, what would be problematic cases?
Allocating it on the heap.
OK. I wonder if it might be a good idea for us to discuss my use-case directly some time. It's quite subtle, and I'm far from sure that I've ironed out all the details or corner-cases in my head, but I think the details of the scope proposal could be very important in determining whether it's workable or not. The TL;DR here is that I'm concerned about having a solution for random number generators and random algorithms that (i) allows them to be stack-allocated, (ii) ensures that their internal state is not accidentally copied by value, and (iii) allows them to work effectively with (input) range semantics.
Aug 13 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/13/2016 1:50 PM, Joseph Rushton Wakeling wrote:
 Sure, but doesn't the envisioned DIP create the circumstances in which it could
 also be permitted in  safe code where the compiler can guarantee that the
 pointer's lifetime will not outlive the data referred to?
The whole point of ref is that it is a special pointer. If you want pointers, use pointers instead. I don't see much value in making ref just an alternative syntax to *.
 OK.  I wonder if it might be a good idea for us to discuss my use-case directly
 some time.  It's quite subtle, and I'm far from sure that I've ironed out all
 the details or corner-cases in my head, but I think the details of the scope
 proposal could be very important in determining whether it's workable or not.

 The TL;DR here is that I'm concerned about having a solution for random number
 generators and random algorithms that (i) allows them to be stack-allocated,
 (ii) ensures that their internal state is not accidentally copied by value, and
 (iii) allows them to work effectively with (input) range semantics.
It would be good if your case worked with the scheme, but it is not a disaster if it does not. The DIP closes severe and obvious safety holes, and we need this regardless. There will always be desirable cases that are safe but are not allowed by a scheme, and we have system for that. I'm sure we'll find them after this DIP has seen some use for a while.
Aug 13 2016
prev sibling parent reply Guillaume Chatelet <chatelet.guillaume gmail.com> writes:
On Saturday, 13 August 2016 at 20:50:53 UTC, Joseph Rushton 
Wakeling wrote:
 The TL;DR here is that I'm concerned about having a solution 
 for random number generators and random algorithms that (i) 
 allows them to be stack-allocated, (ii) ensures that their 
 internal state is not accidentally copied by value, and (iii) 
 allows them to work effectively with (input) range semantics.
Isn't it what a scoped class is supposed to provide? class Rnd {} void foo() { scope rnd = new Rnd; // reference semantic and stack allocated }
Aug 14 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 14 August 2016 at 10:11:25 UTC, Guillaume Chatelet 
wrote:
 Isn't it what a scoped class is supposed to provide?

 class Rnd {}

 void foo() {
   scope rnd  = new Rnd; // reference semantic and stack 
 allocated
 }
Does that actually work in D2? I thought it was a D1-only thing.
Aug 14 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/14/2016 9:56 PM, Joseph Rushton Wakeling wrote:
 Does that actually work in D2?
Yes.
Aug 14 2016
parent reply Mike <none none.com> writes:
On Monday, 15 August 2016 at 04:58:06 UTC, Walter Bright wrote:
 On 8/14/2016 9:56 PM, Joseph Rushton Wakeling wrote:
 Does that actually work in D2?
Yes.
Can you please clarify the current implementation `scope`, and what DIP1000 proposes to change with respect to the current implementation? According to this [http://dlang.org/deprecate.html#scope%20for%20allocating%20classes 20on%20the%20stack] it was deprecated in favor of std.typecons.scoped. But your previous statement say's scoped variables is still a thing. What exactly is being deprecated with regard to `scope`, if anything? Does the deprecated features page need an update? Will DIP1000 render `std.typecons.scoped` obsolete? In other words, does DIP1000 deprecate the deprecation? Is `scope` being repurposed for DIP1000, or simply expanded? In other words does DIP1000 change the current implementation of scope in any way, or just add to it? Thanks, Mike
Aug 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/16/2016 5:31 PM, Mike wrote:
 On Monday, 15 August 2016 at 04:58:06 UTC, Walter Bright wrote:
 On 8/14/2016 9:56 PM, Joseph Rushton Wakeling wrote:
 Does that actually work in D2?
Yes.
Can you please clarify the current implementation `scope`, and what DIP1000 proposes to change with respect to the current implementation?
It just adds to the existing compiler implementation of 'scope'. It has nothing to do with std.typecons.scoped.
Aug 16 2016
parent reply Mike <none none.com> writes:
On Wednesday, 17 August 2016 at 01:42:00 UTC, Walter Bright wrote:

 Can you please clarify the current implementation `scope`, and 
 what DIP1000
 proposes to change with respect to the current implementation?
It just adds to the existing compiler implementation of 'scope'. It has nothing to do with std.typecons.scoped.
Ok, but the deprecations page [1] gives this example... class A { int x; this(int x) { this.x = x; } } void main() { A obj; { scope A a = new A(1); obj = a; } assert(obj.x == 1); // fails, 'a' has been destroyed } ... as a deprecated pattern, and corrected with ... class A { this(int x) { } } void main() { auto a = std.typecons.scoped!A(1); } However, in DIP1000, the examples use the above deprecated pattern extensively. So what's the story? Does the deprecations page [1] need an update? [1] http://dlang.org/deprecate.html#scope%20for%20allocating%20classes%20on%20the%20stack
Aug 16 2016
parent reply Rory McGuire via Digitalmars-d-announce writes:
On 17 Aug 2016 04:00, "Mike via Digitalmars-d-announce" <
digitalmars-d-announce puremagic.com> wrote:
 On Wednesday, 17 August 2016 at 01:42:00 UTC, Walter Bright wrote:

 Can you please clarify the current implementation `scope`, and what
DIP1000
 proposes to change with respect to the current implementation?
It just adds to the existing compiler implementation of 'scope'. It has
nothing to do with std.typecons.scoped.
 Ok, but the deprecations page [1] gives this example...

 class A
 {
     int x;
     this(int x) { this.x = x; }
 }

 void main()
 {
     A obj;
     {
         scope A a = new A(1);
         obj = a;
     }
     assert(obj.x == 1);  // fails, 'a' has been destroyed
 }

 ... as a deprecated pattern, and corrected with ...

 class A
 {
     this(int x) { }
 }
 void main()
 {
     auto a = std.typecons.scoped!A(1);
 }

 However, in DIP1000, the examples use the above deprecated pattern
extensively. So what's the story? Does the deprecations page [1] need an update?
 [1]
http://dlang.org/deprecate.html#scope%20for%20allocating%20classes%20on%20the%20stack Basically DIP1000 makes it so that:
 void main()
 {
     A obj;
     {
         scope A a = new A(1);
         obj = a;
     }
     assert(obj.x == 1);  // fails, 'a' has been destroyed
 }
Will not compile.
Aug 16 2016
parent reply Mike <none none.com> writes:
On Wednesday, 17 August 2016 at 04:28:33 UTC, Rory McGuire wrote:

 Basically DIP1000 makes it so that:
 void main()
 {
     A obj;
     {
         scope A a = new A(1);
         obj = a;
     }
     assert(obj.x == 1);  // fails, 'a' has been destroyed
 }
Will not compile.
Ok, that makes sense. But then the feature illustrated on the deprecations page is wrong. I think what has been deprecated is `scope` as a type modifier, not `scope` as a storage class, but the example on the deprecations page illustrates the `scope` storage class, not the type modifier. I believe what the deprecations page intended to say was that this has been deprecated: scope class A { this(int x) { } } And DIP1000 finally implements the `scope` storage class properly: void main() { A obj; { scope A a = new A(1); obj = a; // compile-time error. Good! } assert(obj.x == 1); // or is it usage that triggers the compile-time error? } Mike
Aug 16 2016
parent reply Mike <none none.com> writes:
On Wednesday, 17 August 2016 at 06:44:41 UTC, Mike wrote:
 On Wednesday, 17 August 2016 at 04:28:33 UTC, Rory McGuire 
 wrote:

 Basically DIP1000 makes it so that:
 void main()
 {
     A obj;
     {
         scope A a = new A(1);
         obj = a;
     }
     assert(obj.x == 1);  // fails, 'a' has been destroyed
 }
Will not compile.
Ok, that makes sense. But then the feature illustrated on the deprecations page is wrong. I think what has been deprecated is `scope` as a type modifier, not `scope` as a storage class, but the example on the deprecations page illustrates the `scope` storage class, not the type modifier. I believe what the deprecations page intended to say was that this has been deprecated: scope class A { this(int x) { } } And DIP1000 finally implements the `scope` storage class properly: void main() { A obj; { scope A a = new A(1); obj = a; // compile-time error. Good! } assert(obj.x == 1); // or is it usage that triggers the compile-time error? } Mike
Or perhaps DIP1000 changes the current behavior of the `scope` storage class. My understanding is that the `scope` storage class currently allocates a class on the stack (though its usage for this purpose is deprecated in favor of std.typecons.scoped). If DIP1000 is implemented, it will change that behavior, so the allocation will instead be on the GC heap, but the compiler will do some flow-control analysis to prevent escaping references. Is that right? Mike
Aug 17 2016
next sibling parent Mike <none none.com> writes:
On Wednesday, 17 August 2016 at 07:04:26 UTC, Mike wrote:

 Or perhaps DIP1000 changes the current behavior of the `scope` 
 storage class.

 My understanding is that the `scope` storage class currently 
 allocates a class on the stack (though its usage for this 
 purpose is deprecated in favor of std.typecons.scoped).  If 
 DIP1000 is implemented, it will change that behavior, so the 
 allocation will instead be on the GC heap, but the compiler 
 will do some flow-control analysis to prevent escaping 
 references.  Is that right?
I may have found my answer in the DIP, but there is some ambiguity: "Currently, scope is ignored except that a new class use to initialize a scope variable allocates the class instance on the stack. Fortunately, this can work with this new proposal, with an optimization that recognizes that if a new class is unique, and assigned to a scope variable, then that instance can be placed on the stack." "can be placed on the stack", or "will be placed on the stack"? And only "if the new class is unique"? I'm assuming unique mean a new instance with a reference count (for lack of a better word) no greater than 1. Mike
Aug 17 2016
prev sibling parent reply Rory McGuire via Digitalmars-d-announce writes:
On Wed, Aug 17, 2016 at 9:04 AM, Mike via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:
 Or perhaps DIP1000 changes the current behavior of the `scope` storage
 class.

 My understanding is that the `scope` storage class currently allocates a
 class on the stack (though its usage for this purpose is deprecated in
 favor of std.typecons.scoped).
Correct. I believe the reason for the deprecation was that it was too easy to use considering how easy it was to get it's usage wrong, and that a library implementation would be just as good, which would also lessen the number of reserved words D has, and make devs more likely to check the documentation for its caveats.
   If DIP1000 is implemented, it will change that behavior, so the
 allocation will instead be on the GC heap, but the compiler will do some
 flow-control analysis to prevent escaping references.  Is that right?

 Mike
Not correct, the class would still be on the stack so we can have reference semantics during assignment etc, but the instance is on the stack so its faster and the function the code is inside can optionally be nogc. DIP1000 will just make the compiler check that a stack instance does not escape its scope (though it doesn't cover all cases). struct Astruct {} // - on stack by default class Aclass {} // - on heap by default void main() { Astruct a = new Astruct; // override, now Astruct is on the heap (because of "new") Aclass c = new Aclass; // on the heap as per-usual scope Aclass c1 = new Aclass; // override, now class is on the stack (most obvious use: to make all references use the same instance) }
Aug 17 2016
parent reply Mike <none none.com> writes:
On Wednesday, 17 August 2016 at 07:17:24 UTC, Rory McGuire wrote:
   If DIP1000 is implemented, it will change that behavior, so 
 the
 allocation will instead be on the GC heap, but the compiler 
 will do some
 flow-control analysis to prevent escaping references.  Is that 
 right?

 Mike
Not correct, the class would still be on the stack so we can have reference semantics during assignment etc, but the instance is on the stack so its faster and the function the code is inside can optionally be nogc. DIP1000 will just make the compiler check that a stack instance does not escape its scope (though it doesn't cover all cases). struct Astruct {} // - on stack by default class Aclass {} // - on heap by default void main() { Astruct a = new Astruct; // override, now Astruct is on the heap (because of "new") Aclass c = new Aclass; // on the heap as per-usual scope Aclass c1 = new Aclass; // override, now class is on the stack (most obvious use: to make all references use the same instance) }
Got it! Thank you! But it still appears that what's illustrated on the deprecations page is not being deprecated. Mike
Aug 17 2016
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 17 August 2016 at 07:53:49 UTC, Mike wrote:
 Got it!  Thank you!  But it still appears that what's 
 illustrated on the deprecations page is not being deprecated.

 Mike
I imagine that will change if/when DIP1000 is accepted.
Aug 17 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,a,n,n,o,u,n,c,e
Subject: Re: DIP1000: Scoped Pointers
References: <pqsiqmkxenrwxoruzaml forum.dlang.org>
 <xcsurftmdibkhiuvjuue forum.dlang.org> <noisvr$3105$1 digitalmars.com>
 <gjdzqprqnqjrxiipcins forum.dlang.org> <nokdr2$21is$1 digitalmars.com>
 <tivdjqwnwhwkdzfbnfpb forum.dlang.org> <edbeenxfroedmlrodhqv forum.dlang.org>
 <nol8i8$50t$1 digitalmars.com> <cokllkjgrwukvqsnfreq forum.dlang.org>
 <nomv4e$2g1p$1 digitalmars.com> <ugqswaqgqjizmcyyklco forum.dlang.org>
 <nontn8$lvt$1 digitalmars.com> <gwtucsynknucglnovpcj forum.dlang.org>
 <cevjepaweamunmtzfizl forum.dlang.org> <fsksbecoifigcmuomivk forum.dlang.org>
 <nori4t$2qvc$1 digitalmars.com> <jbyxlgawvepbpjczzeoc forum.dlang.org>
 <np0fd8$20kc$1 digitalmars.com> <sccaakhfxkuyjuppxzhk forum.dlang.org>
 <mailman.1043.1471408126.3131.digitalmars-d-announce puremagic.com>
 <rohgwxxmegbkuipkjqoc forum.dlang.org> <lilrtjsllessqqwwoulu forum.dlang.org>
 <mailman.1045.1471418257.3131.digitalmars-d-announce puremagic.com>
 <sjcorezoaqduguvqctgt forum.dlang.org>
In-Reply-To: <sjcorezoaqduguvqctgt forum.dlang.org>

--rnTf1lmdHr7L5wxMm5tiAmqJs4tPqGCXF
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/17/2016 10:53 AM, Mike wrote:
 On Wednesday, 17 August 2016 at 07:17:24 UTC, Rory McGuire wrote:
   If DIP1000 is implemented, it will change that behavior, so the
 allocation will instead be on the GC heap, but the compiler will do s=
ome
 flow-control analysis to prevent escaping references.  Is that right?=
 Mike
Not correct, the class would still be on the stack so we can have reference semantics during assignment etc, but the instance is on the stack so its faster and the function the code is inside can optionally=
 be nogc.

 DIP1000 will just make the compiler check that a stack instance does
 not escape its scope (though it doesn't cover all cases).

 struct Astruct {} // - on stack by default
 class Aclass  {} // - on heap by default
 void main() {
     Astruct a =3D new Astruct; // override, now Astruct is on the heap=
 (because of "new")
     Aclass c =3D new Aclass; // on the heap as per-usual
     scope Aclass c1 =3D new Aclass; // override, now class is on the s=
tack
 (most obvious use: to make all references use the same instance)
 }
=20 Got it! Thank you! But it still appears that what's illustrated on th=
e
 deprecations page is not being deprecated.
=20
 Mike
Yes, it will have to be updated - but I didn't want to adjust it before DIP1000 spec is finalized. Rationale that was driving deprecation of scope storage class is becoming obsolete with DIP1000 implemented but not before. --rnTf1lmdHr7L5wxMm5tiAmqJs4tPqGCXF--
Aug 17 2016
prev sibling parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Monday, 15 August 2016 at 04:56:07 UTC, Joseph Rushton 
Wakeling wrote:
 On Sunday, 14 August 2016 at 10:11:25 UTC, Guillaume Chatelet 
 wrote:
 Isn't it what a scoped class is supposed to provide?

 class Rnd {}

 void foo() {
   scope rnd  = new Rnd; // reference semantic and stack 
 allocated
 }
Does that actually work in D2? I thought it was a D1-only thing.
Yes, but for some unknown to me reason it was deprecated (or it will be in the future) in favour of std.typecons.scoped. Actually, it is used in many places throughout DDMD, so I don't think its going away soon.
Aug 15 2016
parent reply lobo <swamplobo gmail.com> writes:
On Monday, 15 August 2016 at 07:10:00 UTC, ZombineDev wrote:
 On Monday, 15 August 2016 at 04:56:07 UTC, Joseph Rushton 
 Wakeling wrote:
 On Sunday, 14 August 2016 at 10:11:25 UTC, Guillaume Chatelet 
 wrote:
 Isn't it what a scoped class is supposed to provide?

 class Rnd {}

 void foo() {
   scope rnd  = new Rnd; // reference semantic and stack 
 allocated
 }
Does that actually work in D2? I thought it was a D1-only thing.
Yes, but for some unknown to me reason it was deprecated (or it will be in the future) in favour of std.typecons.scoped. Actually, it is used in many places throughout DDMD, so I don't think its going away soon.
When was it deprecated? I use it a lot and DMD 2.071.1 gives no warning. bye, lobo
Aug 15 2016
next sibling parent ZombineDev <petar.p.kirov gmail.com> writes:
On Monday, 15 August 2016 at 07:19:00 UTC, lobo wrote:
 On Monday, 15 August 2016 at 07:10:00 UTC, ZombineDev wrote:
 On Monday, 15 August 2016 at 04:56:07 UTC, Joseph Rushton 
 Wakeling wrote:
 On Sunday, 14 August 2016 at 10:11:25 UTC, Guillaume Chatelet 
 wrote:
 Isn't it what a scoped class is supposed to provide?

 class Rnd {}

 void foo() {
   scope rnd  = new Rnd; // reference semantic and stack 
 allocated
 }
Does that actually work in D2? I thought it was a D1-only thing.
Yes, but for some unknown to me reason it was deprecated (or it will be in the future) in favour of std.typecons.scoped. Actually, it is used in many places throughout DDMD, so I don't think its going away soon.
When was it deprecated? I use it a lot and DMD 2.071.1 gives no warning. bye, lobo
It's not deprecated, but it will be, at least according to this page: http://dlang.org/deprecate. (Search for "scope for allocating classes on the stack".)
Aug 15 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
On Monday, 15 August 2016 at 07:19:00 UTC, lobo wrote:
 When was it deprecated? I use it a lot and DMD 2.071.1 gives no 
 warning.
It was planned for removal because it was very un- safe (no escaping checks whatsoever) and as such was not better than library implementation. Quite likely with DIP1000 implemented there will be no point in deprecating it anymore.
Aug 15 2016
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Monday, 15 August 2016 at 10:27:00 UTC, Dicebot wrote:
 On Monday, 15 August 2016 at 07:19:00 UTC, lobo wrote:
 When was it deprecated? I use it a lot and DMD 2.071.1 gives 
 no warning.
It was planned for removal because it was very un- safe (no escaping checks whatsoever) and as such was not better than library implementation.
I suspected as much.
 Quite likely with DIP1000 implemented there will be no point in 
 deprecating it anymore.
+1 I think that would be great!
Aug 15 2016
parent reply Rory McGuire via Digitalmars-d-announce writes:
On Mon, Aug 15, 2016 at 1:57 PM, ZombineDev via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:

 On Monday, 15 August 2016 at 10:27:00 UTC, Dicebot wrote:

 On Monday, 15 August 2016 at 07:19:00 UTC, lobo wrote:

 When was it deprecated? I use it a lot and DMD 2.071.1 gives no warning.
It was planned for removal because it was very un- safe (no escaping checks whatsoever) and as such was not better than library implementation.
import std.stdio; class Rnd { this() { writeln("created"); } ~this() { writeln("destroyed"); } int i; } auto test() { scope rnd = new Rnd; // reference semantic and stack allocated auto rnd2 = rnd; rnd.i = 2; assert(rnd2.i == 2); return rnd2; } void main() { writeln("start test"); auto v = test(); writeln("test exited", v); } Output: start test created destroyed segmentation fault (core dumped) rdmd scoped_ref_class_semantics.d
 I suspected as much.

 Quite likely with DIP1000 implemented there will be no point in
 deprecating it anymore.
+1 I think that would be great!
The above example should not compile after DIP1000? If so that will be great!
Aug 15 2016
parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,a,n,n,o,u,n,c,e
Subject: Re: DIP1000: Scoped Pointers
References: <pqsiqmkxenrwxoruzaml forum.dlang.org>
 <xcsurftmdibkhiuvjuue forum.dlang.org> <noisvr$3105$1 digitalmars.com>
 <gjdzqprqnqjrxiipcins forum.dlang.org> <nokdr2$21is$1 digitalmars.com>
 <tivdjqwnwhwkdzfbnfpb forum.dlang.org> <edbeenxfroedmlrodhqv forum.dlang.org>
 <nol8i8$50t$1 digitalmars.com> <cokllkjgrwukvqsnfreq forum.dlang.org>
 <nomv4e$2g1p$1 digitalmars.com> <ugqswaqgqjizmcyyklco forum.dlang.org>
 <nontn8$lvt$1 digitalmars.com> <gwtucsynknucglnovpcj forum.dlang.org>
 <cevjepaweamunmtzfizl forum.dlang.org> <fsksbecoifigcmuomivk forum.dlang.org>
 <pensqsoytikyszzoawxt forum.dlang.org> <ahxgtfnjcehyfbfochjk forum.dlang.org>
 <bopimyhfspaqnhgwllup forum.dlang.org> <wodurakokgubvktsxkkb forum.dlang.org>
 <mailman.972.1471264904.3131.digitalmars-d-announce puremagic.com>
In-Reply-To: <mailman.972.1471264904.3131.digitalmars-d-announce puremagic.com>

--0WVg7J3aCMQXqsfpMIUVIt8xhcDQBpbnT
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/15/2016 03:41 PM, Rory McGuire via Digitalmars-d-announce wrote:
 scope rnd  =3D new Rnd; // reference semantic and stack allocated
 auto rnd2 =3D rnd;
=20
 rnd.i =3D 2;
 assert(rnd2.i =3D=3D 2);
 return rnd2;
Point is that that would become illegal if DIP1000 is implemented thus giving scope class concept more justification. It would still have safe holes though because proposed `scope` does not work through many indirection levels - but better than existing situation when nothing is checked. --0WVg7J3aCMQXqsfpMIUVIt8xhcDQBpbnT--
Aug 15 2016
parent reply Rory McGuire via Digitalmars-d-announce writes:
On Mon, Aug 15, 2016 at 2:49 PM, Dicebot via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:

 On 08/15/2016 03:41 PM, Rory McGuire via Digitalmars-d-announce wrote:
 scope rnd  = new Rnd; // reference semantic and stack allocated
 auto rnd2 = rnd;

 rnd.i = 2;
 assert(rnd2.i == 2);
 return rnd2;
Point is that that would become illegal if DIP1000 is implemented thus giving scope class concept more justification. It would still have safe holes though because proposed `scope` does not work through many indirection levels - but better than existing situation when nothing is checked.
okay nice, so that code would not compile but code such as: void test() { scope rnd = new Rnd; // reference semantic and stack allocated auto rnd2 = rnd; some_sneaky_function_that_saves_global_state(rnd); } would still not be checked. And would crash inexplicably at the point the global was accessed?
Aug 15 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,a,n,n,o,u,n,c,e
Subject: Re: DIP1000: Scoped Pointers
References: <pqsiqmkxenrwxoruzaml forum.dlang.org>
 <xcsurftmdibkhiuvjuue forum.dlang.org> <noisvr$3105$1 digitalmars.com>
 <gjdzqprqnqjrxiipcins forum.dlang.org> <nokdr2$21is$1 digitalmars.com>
 <tivdjqwnwhwkdzfbnfpb forum.dlang.org> <edbeenxfroedmlrodhqv forum.dlang.org>
 <nol8i8$50t$1 digitalmars.com> <cokllkjgrwukvqsnfreq forum.dlang.org>
 <nomv4e$2g1p$1 digitalmars.com> <ugqswaqgqjizmcyyklco forum.dlang.org>
 <nontn8$lvt$1 digitalmars.com> <gwtucsynknucglnovpcj forum.dlang.org>
 <cevjepaweamunmtzfizl forum.dlang.org> <fsksbecoifigcmuomivk forum.dlang.org>
 <pensqsoytikyszzoawxt forum.dlang.org> <ahxgtfnjcehyfbfochjk forum.dlang.org>
 <bopimyhfspaqnhgwllup forum.dlang.org> <wodurakokgubvktsxkkb forum.dlang.org>
 <mailman.972.1471264904.3131.digitalmars-d-announce puremagic.com>
 <nosdpq$1536$1 digitalmars.com>
 <mailman.978.1471269280.3131.digitalmars-d-announce puremagic.com>
In-Reply-To: <mailman.978.1471269280.3131.digitalmars-d-announce puremagic.com>

--GPgfCtXdapM28JD7hjWVDRAOpqsTMmUp1
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/15/2016 04:54 PM, Rory McGuire via Digitalmars-d-announce wrote:
 okay nice, so that code would not compile but code such as:
 void test() {
 scope rnd  =3D new Rnd; // reference semantic and stack allocated
 auto rnd2 =3D rnd;
         some_sneaky_function_that_saves_global_state(rnd);
 }
 would still not be checked. And would crash inexplicably at the point
 the global was accessed?
some_sneaky_function_that_saves_global_state would have to be declared as `some_sneaky_function_that_saves_global_state(scope Rnd rnd)` to be allowed to use rnd as argument which prevents escaping to globals. What would still be the problem is if `Rnd` contains reference to another class internally (which gets manually destroyed when Rnd is destroyed) and `some_sneaky_function_that_saves_global_state` saves it instead - because by current design `scope` is a storage class and not transitive. --GPgfCtXdapM28JD7hjWVDRAOpqsTMmUp1--
Aug 15 2016
parent Rory McGuire via Digitalmars-d-announce writes:
On Mon, Aug 15, 2016 at 4:05 PM, Dicebot via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:

 On 08/15/2016 04:54 PM, Rory McGuire via Digitalmars-d-announce wrote:
 okay nice, so that code would not compile but code such as:
 void test() {
 scope rnd  = new Rnd; // reference semantic and stack allocated
 auto rnd2 = rnd;
         some_sneaky_function_that_saves_global_state(rnd);
 }
 would still not be checked. And would crash inexplicably at the point
 the global was accessed?
some_sneaky_function_that_saves_global_state would have to be declared as `some_sneaky_function_that_saves_global_state(scope Rnd rnd)` to be allowed to use rnd as argument which prevents escaping to globals. What would still be the problem is if `Rnd` contains reference to another class internally (which gets manually destroyed when Rnd is destroyed) and `some_sneaky_function_that_saves_global_state` saves it instead - because by current design `scope` is a storage class and not transitive.
Thanks! That is an excellent explanation. Is the below a test case for that? import std.stdio; class Rnd { NormalRefSemantics inner; // protecting this is irrelevant in more complex objects? this() { inner = new NormalRefSemantics(); writeln("created"); } ~this() { delete inner;// this is what causes the segfault writeln("destroyed"); } int i; } void test() { scope rnd = new Rnd; // reference semantic and stack allocated auto rnd2 = rnd; rnd.i = 2; assert(rnd2.i == 2); sneaky_escape(rnd); } void main() { writeln("start test"); test(); writeln("test exited", oops); } class NormalRefSemantics { this() { writeln("I'm alive"); } ~this() { writeln("inner destruction"); } } NormalRefSemantics oops; void sneaky_escape(Rnd r) { oops = r.inner; // how can we protect this inner part of the class from escaping? // would we need to mark classes and functions as "scope safe"? (similar to "thread safe") } == This DIP is really interesting, reminds me of back when we were playing around with "emplace". R
Aug 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/15/2016 6:54 AM, Rory McGuire via Digitalmars-d-announce wrote:
 okay nice, so that code would not compile but code such as:
 void test() {
 scope rnd  = new Rnd; // reference semantic and stack allocated
 auto rnd2 = rnd;
         some_sneaky_function_that_saves_global_state(rnd);
 }
 would still not be checked. And would crash inexplicably at the point the
global
 was accessed?
A local variable initialized with a scoped value will have 'scope' inferred for it.
Aug 15 2016
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Monday, 15 August 2016 at 21:25:22 UTC, Walter Bright wrote:
 On 8/15/2016 6:54 AM, Rory McGuire via Digitalmars-d-announce 
 wrote:
 okay nice, so that code would not compile but code such as:
 void test() {
 scope rnd  = new Rnd; // reference semantic and stack allocated
 auto rnd2 = rnd;
         some_sneaky_function_that_saves_global_state(rnd);
 }
 would still not be checked. And would crash inexplicably at 
 the point the global
 was accessed?
A local variable initialized with a scoped value will have 'scope' inferred for it.
What happens in that case ? void test() { scope rnd = new Rnd; // reference semantic and stack allocated Rnd rnd2; rnd2 = rnd; some_sneaky_function_that_saves_global_state(rnd); } or is that not even possible ? (sorry I'm still a noob in D).
Aug 16 2016
parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,a,n,n,o,u,n,c,e
Subject: Re: DIP1000: Scoped Pointers
References: <pqsiqmkxenrwxoruzaml forum.dlang.org>
 <xcsurftmdibkhiuvjuue forum.dlang.org> <noisvr$3105$1 digitalmars.com>
 <gjdzqprqnqjrxiipcins forum.dlang.org> <nokdr2$21is$1 digitalmars.com>
 <tivdjqwnwhwkdzfbnfpb forum.dlang.org> <edbeenxfroedmlrodhqv forum.dlang.org>
 <nol8i8$50t$1 digitalmars.com> <cokllkjgrwukvqsnfreq forum.dlang.org>
 <nomv4e$2g1p$1 digitalmars.com> <ugqswaqgqjizmcyyklco forum.dlang.org>
 <nontn8$lvt$1 digitalmars.com> <gwtucsynknucglnovpcj forum.dlang.org>
 <cevjepaweamunmtzfizl forum.dlang.org> <fsksbecoifigcmuomivk forum.dlang.org>
 <pensqsoytikyszzoawxt forum.dlang.org> <ahxgtfnjcehyfbfochjk forum.dlang.org>
 <bopimyhfspaqnhgwllup forum.dlang.org> <wodurakokgubvktsxkkb forum.dlang.org>
 <mailman.972.1471264904.3131.digitalmars-d-announce puremagic.com>
 <nosdpq$1536$1 digitalmars.com>
 <mailman.978.1471269280.3131.digitalmars-d-announce puremagic.com>
 <notc01$11uh$1 digitalmars.com> <ercaimigosujzqanfcvf forum.dlang.org>
In-Reply-To: <ercaimigosujzqanfcvf forum.dlang.org>

--aSdFHjWXARGcUsDa9kHGJvmpOhRlAP5n7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/16/2016 07:26 PM, Patrick Schluter wrote:
 What happens in that case ?
=20
 void test() {
 scope rnd  =3D new Rnd; // reference semantic and stack allocated
 Rnd rnd2;
  rnd2 =3D rnd;
          some_sneaky_function_that_saves_global_state(rnd);
 }
=20
 or is that not even possible ? (sorry I'm still a noob in D).
If `Rnd` is supposed to be a class, it won't compile because it would mean escaping scope reference to non-scope variable (with DIP1000). If it is a struct, it won't compile because you are trying to assign `Rnd*` (pointer) to `Rnd` (value) :) --aSdFHjWXARGcUsDa9kHGJvmpOhRlAP5n7--
Aug 16 2016
parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 16 August 2016 at 16:34:05 UTC, Dicebot wrote:
 On 08/16/2016 07:26 PM, Patrick Schluter wrote:
 What happens in that case ?
 
 void test() {
 scope rnd  = new Rnd; // reference semantic and stack allocated
 Rnd rnd2;
  rnd2 = rnd;
          some_sneaky_function_that_saves_global_state(rnd);
 }
 
 or is that not even possible ? (sorry I'm still a noob in D).
If `Rnd` is supposed to be a class, it won't compile because it would mean escaping scope reference to non-scope variable (with DIP1000). If it is a struct, it won't compile because you are trying to assign `Rnd*` (pointer) to `Rnd` (value) :)
What about this? struct Rnd { int* state; } void test() { scope rnd = new Rnd(); Rnd rnd2 = *rnd; saveGlobalState(rnd2); }
Aug 16 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
On Tuesday, 16 August 2016 at 18:25:42 UTC, Meta wrote:
 What about this?

 struct Rnd
 {
     int* state;
 }

 void test()
 {
     scope rnd = new Rnd();
     Rnd rnd2 = *rnd;

     saveGlobalState(rnd2);
 }
Same as far as I understand, because "from a lifetime analysis viewpoint, a struct is considered a juxtaposition of its direct members" (https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#aggregates). You need to add one more level of indirection for things to start going complicated.
Aug 16 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
On Tuesday, 16 August 2016 at 18:55:40 UTC, Dicebot wrote:
 On Tuesday, 16 August 2016 at 18:25:42 UTC, Meta wrote:
 What about this?

 struct Rnd
 {
     int* state;
 }

 void test()
 {
     scope rnd = new Rnd();
     Rnd rnd2 = *rnd;

     saveGlobalState(rnd2);
 }
Same as far as I understand, because "from a lifetime analysis viewpoint, a struct is considered a juxtaposition of its direct members" (https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#aggregates). You need to add one more level of indirection for things to start going complicated.
Ah no, sorry, I have missed that you allocate struct on heap. Yes, this is simplified problem case indeed. Intention is that such struct can be made safe by making all pointer fields private and adding scope semantics in getter methods but it is hard to reason about details at this point.
Aug 16 2016
parent Rory McGuire via Digitalmars-d-announce writes:
On 16 Aug 2016 21:01, "Dicebot via Digitalmars-d-announce" <
digitalmars-d-announce puremagic.com> wrote:
 On Tuesday, 16 August 2016 at 18:55:40 UTC, Dicebot wrote:
 On Tuesday, 16 August 2016 at 18:25:42 UTC, Meta wrote:
 What about this?

 struct Rnd
 {
     int* state;
 }

 void test()
 {
     scope rnd = new Rnd();
     Rnd rnd2 = *rnd;

     saveGlobalState(rnd2);
 }
Same as far as I understand, because "from a lifetime analysis
viewpoint, a struct is considered a juxtaposition of its direct members" ( https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#aggregates). You need to add one more level of indirection for things to start going complicated.
 Ah no, sorry, I have missed that you allocate struct on heap. Yes, this
is simplified problem case indeed. Intention is that such struct can be made safe by making all pointer fields private and adding scope semantics in getter methods but it is hard to reason about details at this point. It will be nice to see a set of tests that are expected to pass, a set that are expected to fail, and a set that segfault. In my questions I was trying to make small examples, that could become tests. The examples in the DIP are quite simple actually. The pointer escaping example is what I was missing.
Aug 16 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Tue, 16 Aug 2016 18:55:40 +0000, Dicebot wrote:
 You need to add one more level of indirection for things to start going
 complicated.
Presumably scope is transitive, so things shouldn't get horribly complex.
Aug 16 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d-announce" writes:
On Wed, Aug 17, 2016 at 01:01:05AM +0000, Chris Wright via
Digitalmars-d-announce wrote:
 On Tue, 16 Aug 2016 18:55:40 +0000, Dicebot wrote:
 You need to add one more level of indirection for things to start
 going complicated.
Presumably scope is transitive, so things shouldn't get horribly complex.
I thought the DIP states the scope is *not* transitive? T -- The right half of the brain controls the left half of the body. This means that only left-handed people are in their right mind. -- Manoj Srivastava
Aug 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/16/2016 6:01 PM, H. S. Teoh via Digitalmars-d-announce wrote:
 On Wed, Aug 17, 2016 at 01:01:05AM +0000, Chris Wright via
Digitalmars-d-announce wrote:
 On Tue, 16 Aug 2016 18:55:40 +0000, Dicebot wrote:
 You need to add one more level of indirection for things to start
 going complicated.
Presumably scope is transitive, so things shouldn't get horribly complex.
I thought the DIP states the scope is *not* transitive?
It is not transitive.
Aug 16 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D,.,a,n,n,o,u,n,c,e
Subject: Re: DIP1000: Scoped Pointers
References: <pqsiqmkxenrwxoruzaml forum.dlang.org>
 <gjdzqprqnqjrxiipcins forum.dlang.org> <nokdr2$21is$1 digitalmars.com>
 <tivdjqwnwhwkdzfbnfpb forum.dlang.org> <edbeenxfroedmlrodhqv forum.dlang.org>
 <nol8i8$50t$1 digitalmars.com> <cokllkjgrwukvqsnfreq forum.dlang.org>
 <nomv4e$2g1p$1 digitalmars.com> <ugqswaqgqjizmcyyklco forum.dlang.org>
 <nontn8$lvt$1 digitalmars.com> <gwtucsynknucglnovpcj forum.dlang.org>
 <cevjepaweamunmtzfizl forum.dlang.org> <fsksbecoifigcmuomivk forum.dlang.org>
 <pensqsoytikyszzoawxt forum.dlang.org> <ahxgtfnjcehyfbfochjk forum.dlang.org>
 <bopimyhfspaqnhgwllup forum.dlang.org> <wodurakokgubvktsxkkb forum.dlang.org>
 <mailman.972.1471264904.3131.digitalmars-d-announce puremagic.com>
 <nosdpq$1536$1 digitalmars.com>
 <mailman.978.1471269280.3131.digitalmars-d-announce puremagic.com>
 <notc01$11uh$1 digitalmars.com> <ercaimigosujzqanfcvf forum.dlang.org>
 <novfa0$ob2$1 digitalmars.com> <dohmghkusfocywvwzkvj forum.dlang.org>
 <xtjufxaqcxtvhmzcwpzk forum.dlang.org> <np0d0h$1rs2$1 digitalmars.com>
In-Reply-To: <np0d0h$1rs2$1 digitalmars.com>

--C7kWqe9KlVOXAsuGnv5LLLOSLPjX0xFWU
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/17/2016 04:01 AM, Chris Wright wrote:
 On Tue, 16 Aug 2016 18:55:40 +0000, Dicebot wrote:
 You need to add one more level of indirection for things to start goin=
g
 complicated.
=20 Presumably scope is transitive, so things shouldn't get horribly comple=
x. It is not transitive and it is not a type qualifier. --C7kWqe9KlVOXAsuGnv5LLLOSLPjX0xFWU--
Aug 17 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Wed, 17 Aug 2016 13:53:37 +0300, Dicebot wrote:

 On 08/17/2016 04:01 AM, Chris Wright wrote:
 On Tue, 16 Aug 2016 18:55:40 +0000, Dicebot wrote:
 You need to add one more level of indirection for things to start
 going complicated.
Presumably scope is transitive, so things shouldn't get horribly complex.
It is not transitive and it is not a type qualifier.
Non-scope is transitive, rather -- you can't have a non-scope pointer to a scope item (at least in safe code).
Aug 17 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/16/2016 11:25 AM, Meta wrote:
 What about this?

 struct Rnd
 {
     int* state;
 }

 void test()
 {
     scope rnd = new Rnd();
     Rnd rnd2 = *rnd;

     saveGlobalState(rnd2);
 }
'state' is set to null by 'new Rnd()', and so no pointers escape.
Aug 16 2016
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d-announce" writes:
On Wed, Aug 10, 2016 at 08:35:23PM +0000, Dicebot via Digitalmars-d-announce
wrote:
 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and proceeds
 straight to requesting community (your!) feedback.
 
 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.
 
 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md
[...] I found some unclear parts of the proposal: - Under "examples for each rule", in fun2(), the comment says "OK, b is a regular int*". However, at this point b isn't declared yet, and the next line which declares b, declares it as int, not int*. So what is the comment supposed to say? - Under "A few more examples combining the rules", in abc(), 3rd line, the comment says "Error, rule 5". But there is no rule 5! T -- I've been around long enough to have seen an endless parade of magic new techniques du jour, most of which purport to remove the necessity of thought about your programming problem. In the end they wind up contributing one or two pieces to the collective wisdom, and fade away in the rearview mirror. -- Walter Bright
Aug 11 2016
prev sibling next sibling parent poliklosio <poliklosio happypizza.com> writes:
On Wednesday, 10 August 2016 at 20:35:23 UTC, Dicebot wrote:
 The first DIP has just landed into the new queue. It is a (...)
The wording is unclear in the section "Implicit Conversion of Function Pointers and Delegates". It says "scope can be added to parameters, but not removed." The trouble is that the word "added" can be understood in two opposite ways. When you assign an address to a pointer variable, you can say that: - you are assigning a new runtime value, so you are adding scope of parameter of the right hand side to the otherwise unscoped parameter of the left hand side, e.g. alias T function(T) fp_t; T bar(scope T); fp_t fp = &bar; // Ok - you are converting to a new variable, so you are adding scope of param of the left hand side to an otherwise unscoped param of the right hand side, e.g. alias T function(scope T) fps_t; T foo(T); fps_t fp = &foo; // Error Half-jokingly, I think its good to recognize that there is an ambiguity before multiple authors implement things with different assumptions and have a month-long discussion on github. :) More seriously, this wording may make its way to documentation after the design is implemented. If wording is fixed, I suggest also checking if all examples are correct.
Aug 11 2016
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 10 August 2016 at 20:35:23 UTC, Dicebot wrote:
 The first DIP has just landed into the new queue. It is a 
 proposal from language authors and thus it bypasses usual 
 nitpicking process and proceeds straight to requesting 
 community (your!) feedback.
Thanks for all the work. One remarks. I guess `RefCountedSlice` should infer access permissions of `payload` and `count` to allow RefCountedSlice!(const(T)) right? Is that beyond the scope of the DIP?
Aug 12 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 5:24 AM, Nordlöw wrote:
 On Wednesday, 10 August 2016 at 20:35:23 UTC, Dicebot wrote:
 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and proceeds
 straight to requesting community (your!) feedback.
Thanks for all the work. One remarks. I guess `RefCountedSlice` should infer access permissions of `payload` and `count` to allow RefCountedSlice!(const(T)) right? Is that beyond the scope of the DIP?
In order for ref counting to work, and because D doesn't support borrowing, the compiler will have to be aware of ref counting. This DIP is necessary for ref counting to work, but is not sufficient, because it doesn't cover how the compiler semantics will work with ref counting.
Aug 12 2016
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Wednesday, 10 August 2016 at 20:35:23 UTC, Dicebot wrote:
 - Please submit pull requests to adjust the markdown document 
 if you want to propose any improvements (mentioning 
  WalterBright and  andralex for confirmation).
Not completely through yet, but it looks really promising. Already made a PR (https://github.com/dlang/DIPs/pull/34) to replace the term visibility w/ reachability, b/c visibility is already used as a replacement for access checks (DIP22) and reachability is a well established term for GC.
Aug 20 2016