www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Memory safety depends entirely on GC ?

reply "FrankLike" <1150015857 qq.com> writes:
Now,some people think D is a 'Memory safety depends entirely on 
GC' system Language,what do you think?
Feb 21 2015
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Sat, 21 Feb 2015 10:00:06 +0000, FrankLike wrote:


 Now,some people think D is a 'Memory safety depends entirely on GC'
 system Language,what do you think?


this is wrong.=
Feb 21 2015
prev sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 21 February 2015 at 10:00:07 UTC, FrankLike wrote:

 Now,some people think D is a 'Memory safety depends entirely on 
 GC' system Language,what do you think?


It's kind of right, at the moment, since  nogc D is still quite 
difficult to use, mostly due to exceptions and library artifacts. 
Both of these can be solved though.
Feb 21 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 21 February 2015 at 18:06:57 UTC, Peter Alexander 
wrote:

 On Saturday, 21 February 2015 at 10:00:07 UTC, FrankLike wrote:

 Now,some people think D is a 'Memory safety depends entirely 
 on GC' system Language,what do you think?


 It's kind of right, at the moment, since  nogc D is still quite 
 difficult to use, mostly due to exceptions and library 
 artifacts. Both of these can be solved though.


That wouldn't make  nogc safe in any way.
Feb 21 2015
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 21 February 2015 at 18:42:54 UTC, deadalnix wrote:

 On Saturday, 21 February 2015 at 18:06:57 UTC, Peter Alexander 
 wrote:

 On Saturday, 21 February 2015 at 10:00:07 UTC, FrankLike wrote:

 Now,some people think D is a 'Memory safety depends entirely 
 on GC' system Language,what do you think?


 It's kind of right, at the moment, since  nogc D is still 
 quite difficult to use, mostly due to exceptions and library 
 artifacts. Both of these can be solved though.


 That wouldn't make  nogc safe in any way.


 safe  nogc

:-)

(I rewrote the post a few times. Originally I just wrote "mark 
main  safe  nogc and you're fine", but I think it's a bit 
misleading since  nogc is still difficult to use, so I wrote 
about that instead and forgot to mention  safe at all. Thanks for 
pointing out.)
Feb 21 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 21 February 2015 at 19:38:02 UTC, Peter Alexander 
wrote:

  safe  nogc

 :-)

 (I rewrote the post a few times. Originally I just wrote "mark 
 main  safe  nogc and you're fine", but I think it's a bit 
 misleading since  nogc is still difficult to use, so I wrote 
 about that instead and forgot to mention  safe at all. Thanks 
 for pointing out.)


free is an unsafe operation. Unless you don't allocate at all or 
choose to leak everything, you won't be able to be safe and nogc.

The only way out that I know of is an ownership system.
Feb 21 2015
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 21 February 2015 at 20:13:26 UTC, deadalnix wrote:

 On Saturday, 21 February 2015 at 19:38:02 UTC, Peter Alexander 
 wrote:

  safe  nogc

 :-)

 (I rewrote the post a few times. Originally I just wrote "mark 
 main  safe  nogc and you're fine", but I think it's a bit 
 misleading since  nogc is still difficult to use, so I wrote 
 about that instead and forgot to mention  safe at all. Thanks 
 for pointing out.)


 free is an unsafe operation. Unless you don't allocate at all 
 or choose to leak everything, you won't be able to be safe and 
 nogc.

 The only way out that I know of is an ownership system.


malloc+free can be trusted if wrapped in something like a ref 
counted pointer, no?
Feb 21 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/21/15 2:13 PM, Peter Alexander wrote:

 On Saturday, 21 February 2015 at 20:13:26 UTC, deadalnix wrote:

 On Saturday, 21 February 2015 at 19:38:02 UTC, Peter Alexander wrote:

  safe  nogc

 :-)

 (I rewrote the post a few times. Originally I just wrote "mark main
  safe  nogc and you're fine", but I think it's a bit misleading since
  nogc is still difficult to use, so I wrote about that instead and
 forgot to mention  safe at all. Thanks for pointing out.)


 free is an unsafe operation. Unless you don't allocate at all or
 choose to leak everything, you won't be able to be safe and nogc.

 The only way out that I know of is an ownership system.


 malloc+free can be trusted if wrapped in something like a ref counted
 pointer, no?


There's more to it, e.g. access to the underlying raw pointer must be 
carefully restricted. -- Andrei
Feb 21 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 21 February 2015 at 22:13:09 UTC, Peter Alexander 
wrote:

 malloc+free can be trusted if wrapped in something like a ref 
 counted pointer, no?


Foo bazoom;

class Foo {
     void bar() {
         bazoom = this;
     }
}

void foo() {
     RefCounted!Foo f = ...
     f.bar();

     // bazoom is now a dandling pointer.
}
Feb 21 2015
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:

 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter Alexander 
 wrote:

 malloc+free can be trusted if wrapped in something like a ref 
 counted pointer, no?


 Foo bazoom;

 class Foo {
     void bar() {
         bazoom = this;
     }
 }

 void foo() {
     RefCounted!Foo f = ...
     f.bar();

     // bazoom is now a dandling pointer.
 }


I see, thanks.

Is assigning 'this' from a member function the only problem case?
Feb 22 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 22 February 2015 at 14:41:43 UTC, Peter Alexander 
wrote:

 On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:

 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter Alexander 
 wrote:

 malloc+free can be trusted if wrapped in something like a ref 
 counted pointer, no?


 Foo bazoom;

 class Foo {
    void bar() {
        bazoom = this;
    }
 }

 void foo() {
    RefCounted!Foo f = ...
    f.bar();

    // bazoom is now a dandling pointer.
 }


 I see, thanks.

 Is assigning 'this' from a member function the only problem 
 case?


No. There's also returning the reference from a member function, 
storing it in a passed-in reference (pointer, ref, out or slice), 
and passing it to other functions that in turn leak the 
reference, as well as throwing it. And leaking closures 
containing the reference.

That's all that I can think of now...
Feb 22 2015
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 22 February 2015 at 14:49:37 UTC, Marc Schütz wrote:

 On Sunday, 22 February 2015 at 14:41:43 UTC, Peter Alexander 
 wrote:

 On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:

 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter 
 Alexander wrote:

 malloc+free can be trusted if wrapped in something like a 
 ref counted pointer, no?


 Foo bazoom;

 class Foo {
   void bar() {
       bazoom = this;
   }
 }

 void foo() {
   RefCounted!Foo f = ...
   f.bar();

   // bazoom is now a dandling pointer.
 }


 I see, thanks.

 Is assigning 'this' from a member function the only problem 
 case?


 No. There's also returning the reference from a member 
 function, storing it in a passed-in reference (pointer, ref, 
 out or slice), and passing it to other functions that in turn 
 leak the reference, as well as throwing it. And leaking 
 closures containing the reference.

 That's all that I can think of now...


Sorry, I meant things other than moving this from a member 
function to somewhere else, i.e. the RefCounted shouldn't leak 
the pointer itself in any other way. Just wondering if there are 
ways to avoid the problem by making 'this' unescapable in certain 
situations.
Feb 22 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 22 February 2015 at 14:54:26 UTC, Peter Alexander 
wrote:

 On Sunday, 22 February 2015 at 14:49:37 UTC, Marc Schütz wrote:

 On Sunday, 22 February 2015 at 14:41:43 UTC, Peter Alexander 
 wrote:

 On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:

 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter 
 Alexander wrote:

 malloc+free can be trusted if wrapped in something like a 
 ref counted pointer, no?


 Foo bazoom;

 class Foo {
  void bar() {
      bazoom = this;
  }
 }

 void foo() {
  RefCounted!Foo f = ...
  f.bar();

  // bazoom is now a dandling pointer.
 }


 I see, thanks.

 Is assigning 'this' from a member function the only problem 
 case?


 No. There's also returning the reference from a member 
 function, storing it in a passed-in reference (pointer, ref, 
 out or slice), and passing it to other functions that in turn 
 leak the reference, as well as throwing it. And leaking 
 closures containing the reference.

 That's all that I can think of now...


 Sorry, I meant things other than moving this from a member 
 function to somewhere else, i.e. the RefCounted shouldn't leak 
 the pointer itself in any other way. Just wondering if there 
 are ways to avoid the problem by making 'this' unescapable in 
 certain situations.


Ah, ok. But escaping the reference is unavoidable if you don't 
want to pass the RC object around (and you don't, because it's 
expensive). And the only safe way to escape it is an ownership 
system.
Feb 22 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/15 6:49 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 No. There's also returning the reference from a member function, storing
 it in a passed-in reference (pointer, ref, out or slice), and passing it
 to other functions that in turn leak the reference, as well as throwing
 it. And leaking closures containing the reference.

 That's all that I can think of now...


Consider

class C { ... client code ... }
alias T = RefCounted!C;
... more client code ...

For reference counting to work transparently, access to the symbol "C" 
must be restricted. RefCounted obviously needs access to it. Client code 
should never have access to it, even in the definition of C.

That means:

1. client code must not be able to declare variables of type C or issue 
calls like "new C" etc.

2. The type of "this" in methods of C must be RefCounted!C, not C.

3. Conversions of C to bases of C and interfaces must be forbidden; only 
their RefCounted!Base versions must be allowed.

4. Returning references to direct members of C must be restricted the 
same way they are for structs (see http://wiki.dlang.org/DIP25). A GC 
class object does not have that restriction.

I think reference counting is an important component of a complete 
solution to resource management. D should implement world-class 
reference counting for safe code.

For 1-4 above, although I am a staunch supporter of library-exclusive 
abstractions, I have reached the conclusion there is no way to implement 
RC in safe code for D classes without changes to the language. The more 
we realize that as a community the quicker we can move to effect it.



Andrei
Feb 22 2015
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu 
wrote:

 Consider

 class C { ... client code ... }
 alias T = RefCounted!C;
 ... more client code ...

 For reference counting to work transparently, access to the 
 symbol "C" must be restricted. RefCounted obviously needs 
 access to it. Client code should never have access to it, even 
 in the definition of C.


What ??? That mean writing all library code twice, for client 
that want GC and for these who don't. That is a looser strategy.


 That means:

 1. client code must not be able to declare variables of type C 
 or issue calls like "new C" etc.

 2. The type of "this" in methods of C must be RefCounted!C, not 
 C.

 3. Conversions of C to bases of C and interfaces must be 
 forbidden; only their RefCounted!Base versions must be allowed.

 4. Returning references to direct members of C must be 
 restricted the same way they are for structs (see 
 http://wiki.dlang.org/DIP25). A GC class object does not have 
 that restriction.

 I think reference counting is an important component of a 
 complete solution to resource management. D should implement 
 world-class reference counting for safe code.


Sounds like a world class RC would not force all the code to be 
written twice.


 For 1-4 above, although I am a staunch supporter of 
 library-exclusive abstractions, I have reached the conclusion 
 there is no way to implement RC in safe code for D classes 
 without changes to the language. The more we realize that as a 
 community the quicker we can move to effect it.


I don't think we want to implement RC in the language, but 
implement what would allow to have safe RC as library.
Feb 22 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/15 10:26 AM, deadalnix wrote:

 On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu wrote:

 Consider

 class C { ... client code ... }
 alias T = RefCounted!C;
 ... more client code ...

 For reference counting to work transparently, access to the symbol "C"
 must be restricted. RefCounted obviously needs access to it. Client
 code should never have access to it, even in the definition of C.


 What ??? That mean writing all library code twice, for client that want
 GC and for these who don't.


I'm not 100% convinced but it seems to me RC vs. GC is a class design 
time decision.


 That is a looser strategy.


I'm sure there are tighter ones :o).


Andrei
Feb 22 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 22 February 2015 at 20:48:58 UTC, Andrei Alexandrescu 
wrote:

 What ??? That mean writing all library code twice, for client 
 that want
 GC and for these who don't.


 I'm not 100% convinced but it seems to me RC vs. GC is a class 
 design time decision.


The right strategy for memory management depend on the usage 
you'll do of an object, not of the object itself.

Or, in other terms, the client code know more about the adapted 
memory management. Complex library solution probably need to 
adopt some strategy, but for most library (especially for 
something like phobos), it make sense to offload that choice on 
the user.


 That is a looser strategy.


 I'm sure there are tighter ones :o).


Padam tshhhhhh !
Feb 22 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/15 8:50 PM, deadalnix wrote:

 On Sunday, 22 February 2015 at 20:48:58 UTC, Andrei Alexandrescu wrote:

 What ??? That mean writing all library code twice, for client that want
 GC and for these who don't.


 I'm not 100% convinced but it seems to me RC vs. GC is a class design
 time decision.


 The right strategy for memory management depend on the usage you'll do
 of an object, not of the object itself.


I used to think the same. But then I considered typechecking things like:

class Widget
{
     private char name[1024];
     char[] getName() { return name[]; }
     ...
}

Such code is safe if Widget is a GC class but not if it uses reference 
counting.


Andrei
Feb 23 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 23 February 2015 at 17:48:27 UTC, Andrei Alexandrescu 
wrote:

 I used to think the same. But then I considered typechecking 
 things like:

 class Widget
 {
     private char name[1024];
     char[] getName() { return name[]; }
     ...
 }

 Such code is safe if Widget is a GC class but not if it uses 
 reference counting.


This is not a valid argument against my point, simply expressing 
difficulties of implementation and lack of imagination of what 
this implementation can be.

I'm not sure how to convey the idea anymore. That is an ownership 
issue and solutions exist for it. Several of them have been 
proposed already.

It feels like we are gonna add a bazillion of stupid hacks to 
work around not introducing ownership into D. The return ref 
thing is one of them. What you are suggesting for refcounting is 
another.
Feb 23 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu 
wrote:

 On 2/22/15 6:49 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 No. There's also returning the reference from a member 
 function, storing
 it in a passed-in reference (pointer, ref, out or slice), and 
 passing it
 to other functions that in turn leak the reference, as well as 
 throwing
 it. And leaking closures containing the reference.

 That's all that I can think of now...


 Consider

 class C { ... client code ... }
 alias T = RefCounted!C;
 ... more client code ...

 For reference counting to work transparently, access to the 
 symbol "C" must be restricted. RefCounted obviously needs 
 access to it. Client code should never have access to it, even 
 in the definition of C.

 That means:

 1. client code must not be able to declare variables of type C 
 or issue calls like "new C" etc.


No, this would require the class to be specialized for 
refcounting. But the memory management method needs to be the 
client code's decision; it can decide to manage some instance by 
refcounting, some by Unique!C, and leave others to the GC. The 
class implementer shouldn't need to care about all that.


 2. The type of "this" in methods of C must be RefCounted!C, not 
 C.

 3. Conversions of C to bases of C and interfaces must be 
 forbidden; only their RefCounted!Base versions must be allowed.


These two points have undesirable consequences: All consumers 
such objects need to be aware of the exact type, which includes 
the management strategy (RC, Unique, GC). But this is a violation 
of the principle of separation of concerns: a consumer shouldn't 
need to have information about the management strategy, it should 
work equally with `RefCounted!C`, `Unique!C` and bare (GC) `C`, 
as long as it doesn't take ownership of the resource.


 4. Returning references to direct members of C must be 
 restricted the same way they are for structs (see 
 http://wiki.dlang.org/DIP25). A GC class object does not have 
 that restriction.


This is only a partial solution that doesn't work efficiently 
with anything other then value members. Slices, pointers and 
classes require introducing an additional, useless indirection, 
and that's not the only problem with it.


 I think reference counting is an important component of a 
 complete solution to resource management. D should implement 
 world-class reference counting for safe code.

 For 1-4 above, although I am a staunch supporter of 
 library-exclusive abstractions, I have reached the conclusion 
 there is no way to implement RC in safe code for D classes 
 without changes to the language. The more we realize that as a 
 community the quicker we can move to effect it.


I'm not sure this is what you're implying, but do you want to 
restrict it to classes only? Why not structs and slices, too?

Of course I agree that a language change is necessary, but I'm 
convinced what you suggest above is not the right direction at 
all. (And I see that deadalnix has already replied and basically 
said the same thing.)

In general, this and related proposals tend to limit themselves 
on memory management (as witnessed by the importance that `ref` 
and ` safe` play in them). This is too narrow IMO. A well 
thought-out solution can be equally applicable to the broader 
field of resource management.
Feb 23 2015
next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 23 February 2015 at 14:56:11 UTC, Marc Schütz wrote:

 violation of the principle of separation of concerns: a 
 consumer shouldn't need to have information about the 
 management strategy, it should work equally with 
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take ownership of the resource.


Just be aware that by not making the refcount part of C you 
either:

1. Need one more indirection.

2. Need a more complicated allocator to avoid alignment padding 
when C has 32byte/64 byte alignment requirements.

3. Risk having the refcount landing on a different cacheline, 
causing more cache misses.

If you do excessive refcounting (ARC) and care about performance 
you actually need to let the implementor of C decide where the 
RC_counter is embedded...
Feb 23 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 23 February 2015 at 15:35:52 UTC, Ola Fosheim Grøstad 
wrote:

 On Monday, 23 February 2015 at 14:56:11 UTC, Marc Schütz wrote:

 violation of the principle of separation of concerns: a 
 consumer shouldn't need to have information about the 
 management strategy, it should work equally with 
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take ownership of the resource.


 Just be aware that by not making the refcount part of C you 
 either:

 1. Need one more indirection.

 2. Need a more complicated allocator to avoid alignment padding 
 when C has 32byte/64 byte alignment requirements.

 3. Risk having the refcount landing on a different cacheline, 
 causing more cache misses.


The refcount would be embedded right next to the object by the 
constructor of `RefCounted!C`, which is responsible for 
allocating memory for it, using an appropriate allocator.


 If you do excessive refcounting (ARC) and care about 
 performance you actually need to let the implementor of C 
 decide where the RC_counter is embedded...


Of course there can be an additional (template) parameter to 
`RefCounted`, or it could be a different, user-implemented type 
altogether.

But my point was that all of `Unique!C`, `RefCounted!C` and `C` 
should decay to `scope C`. This way, most consumers don't need to 
be aware of the actual wrapper type (if any) at all.
Feb 23 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 23 February 2015 at 15:35:52 UTC, Ola Fosheim Grøstad 
wrote:

 If you do excessive refcounting (ARC) and care about 
 performance you actually need to let the implementor of C 
 decide where the RC_counter is embedded...


C providing facility to be refcountable do not equate with C's 
user want refcounting. It means that if C's user want it to be 
refcounted, it is gonna be faster.
Feb 23 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 6:56 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu wrote:

 On 2/22/15 6:49 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>"
 wrote:

 No. There's also returning the reference from a member function, storing
 it in a passed-in reference (pointer, ref, out or slice), and passing it
 to other functions that in turn leak the reference, as well as throwing
 it. And leaking closures containing the reference.

 That's all that I can think of now...


 Consider

 class C { ... client code ... }
 alias T = RefCounted!C;
 ... more client code ...

 For reference counting to work transparently, access to the symbol "C"
 must be restricted. RefCounted obviously needs access to it. Client
 code should never have access to it, even in the definition of C.

 That means:

 1. client code must not be able to declare variables of type C or
 issue calls like "new C" etc.


 No, this would require the class to be specialized for refcounting. But
 the memory management method needs to be the client code's decision; it
 can decide to manage some instance by refcounting, some by Unique!C, and
 leave others to the GC. The class implementer shouldn't need to care
 about all that.


That's not feasible. Code that assumes the class object will live 
forever can simply do things that are not allowed to code that must 
assume the object will go away at some determined point. Consider this 
which I just posted:

class Widget
{
     private char name[1024];
     char[] getName() { return name[]; }
     ...
}

The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know whether 
getName() is okay or not.


 2. The type of "this" in methods of C must be RefCounted!C, not C.

 3. Conversions of C to bases of C and interfaces must be forbidden;
 only their RefCounted!Base versions must be allowed.


 These two points have undesirable consequences: All consumers such
 objects need to be aware of the exact type, which includes the
 management strategy (RC, Unique, GC). But this is a violation of the
 principle of separation of concerns: a consumer shouldn't need to have
 information about the management strategy, it should work equally with
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it doesn't take
 ownership of the resource.


Well I don't know of another way.


 4. Returning references to direct members of C must be restricted the
 same way they are for structs (see http://wiki.dlang.org/DIP25). A GC
 class object does not have that restriction.


 This is only a partial solution that doesn't work efficiently with
 anything other then value members. Slices, pointers and classes require
 introducing an additional, useless indirection, and that's not the only
 problem with it.


So what do you propose instead? I am well aware of the disadvantages of 
a solution that _works_. Do you have something better?


 I think reference counting is an important component of a complete
 solution to resource management. D should implement world-class
 reference counting for safe code.

 For 1-4 above, although I am a staunch supporter of library-exclusive
 abstractions, I have reached the conclusion there is no way to
 implement RC in safe code for D classes without changes to the
 language. The more we realize that as a community the quicker we can
 move to effect it.


 I'm not sure this is what you're implying, but do you want to restrict
 it to classes only? Why not structs and slices, too?


Structs and slices can be implemented  safe-ly after the advent of 
DIP25. It's classes that are the troublemaker.


 Of course I agree that a language change is necessary, but I'm convinced
 what you suggest above is not the right direction at all. (And I see
 that deadalnix has already replied and basically said the same thing.)


That does awfully little to help. I'm telling things as they are. It's 
difficult to disagree with e.g. "a refcounted class cannot be implicitly 
convertible to Object and stay refcounted".


 In general, this and related proposals tend to limit themselves on
 memory management (as witnessed by the importance that `ref` and ` safe`
 play in them). This is too narrow IMO. A well thought-out solution can
 be equally applicable to the broader field of resource management.


Looking forward to your insights.


Andrei
Feb 23 2015
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu 
wrote:

 The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT 
 know whether getName() is okay or not.


Aye, I haven't been following this thread closely, but I thought 
of this case a while ago myself and it actually led me to the 
belief that the this pointer needs to be scope unless the class 
itself is designed solely for GC use; escaping anything through 
it must not be allowed for guaranteed memory safety if it is 
manually freed in any form, whether refcounting, RAII, or free.

If this is scope, then the usage site has freedom of deallocation 
method. If not, it must be known at design time of the class 
itself.
Feb 23 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 10:27 AM, Adam D. Ruppe wrote:

 On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu wrote:

 The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know
 whether getName() is okay or not.


 Aye, I haven't been following this thread closely, but I thought of this
 case a while ago myself and it actually led me to the belief that the
 this pointer needs to be scope unless the class itself is designed
 solely for GC use; escaping anything through it must not be allowed for
 guaranteed memory safety if it is manually freed in any form, whether
 refcounting, RAII, or free.

 If this is scope, then the usage site has freedom of deallocation
 method. If not, it must be known at design time of the class itself.


That's exactly right. There should be a DConf talk about that, 
apparently it's a widely misunderstood topic. -- Andrei
Feb 23 2015
prev sibling next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
Just brainstorming:

1. Move RC!T into the runtime because it can get special compiler 
attention.
2. Duplicate the const/mutable/immutable machinery for 
Scope/RC/GC classes ( / types).
3. RC!T is not struct RC { size_t cntr; T payload } but a T*. The 
allocator makes sure that the reference count is at T*-8 (or at 
T*+RCOffset!T or whatever).
4. RC!T and T (the GC managed type) implicitly convert to 
Scope!T. Scope!T makes sure that no reference escapes in an 
unsafe way (using the foundations in dip25).

Because of 3. we can have polymorphic functions that either 
return a GC allocated or an RC allocated class. It's type depends 
of the type of this, which leads to the equivalent of inout, 
let's call it  rcgc.
Feb 23 2015
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu 
wrote:

 That's not feasible. Code that assumes the class object will 
 live forever can simply do things that are not allowed to code 
 that must assume the object will go away at some determined 
 point. Consider this which I just posted:

 class Widget
 {
     private char name[1024];
     char[] getName() { return name[]; }
     ...
 }

 The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT 
 know whether getName() is okay or not.


Let's use that exemple.


 Well I don't know of another way.


I'm not sure if this is dishonest, but proposal has been made in 
this newgroup, by various person,s including myself.

Let's get around the major concept.

name is owned by the widget instance (let's call it w). That 
means its lifetime is the same as w's lifetime.

w's owner will provide its lifetime. If w's owner is the GC, then 
its lifetime is infinite. if w's owner is some refounting system, 
its lifetime is defined by when the refcounting system will 
destroy it.

getName here assumes the lifetime of the object is infinite. 
That's ok. It means you won't be able to call it when it is owned 
by a RC system. If you want to be able to do so, you'll ned have 
a way to specify the lifetime of the returned slice. For instance 
:

class Widget {
     private char name[1024];
     scope(char[]) getName() scope { return name[]; }
     ...
}

In this case, you can call the method when w is owned by a RC 
system, as it is now explicit to the caller that the lifetime of 
what is returned and the compiler can ensure the slice do not 
exceed its lifetime (ie w's lifetime).

This require to be able to express lifetime and preferably 
ownership as well. So far, what I feel from you and Walter is the 
will to not go into that direction and instead created a myriad 
of hacks for various special cases. I do think this is wrong 
headed and generally not the way forward. Languages should 
provide the most generic and powerful tool so that interesting 
schemes can be implemented on top of it.
Feb 23 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 5:23 PM, deadalnix wrote:

 So far, what I feel from you and Walter is the will to not go into that
 direction and instead created a myriad of hacks for various special
 cases. I do think this is wrong headed and generally not the way
 forward. Languages should provide the most generic and powerful tool so
 that interesting schemes can be implemented on top of it.


This is a free world. Walter and I are working on a DIP. Please work on 
yours. I can't promise we'll choose to your liking, but this is the one 
way you can make your point heard and understood. What doesn't work is 
trash talk.

I guarantee I recognize brilliance when I see it. So if you have a 
brilliant idea, it won't be missed. Have at it. One thing I cannot do is 
choose a solution that you prefer over one I prefer - this does remain a 
subjective topic. I can't help it. But please don't consider me an idiot 
because I don't like what you propose.


Thanks,

Andrei
Feb 23 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 5:43 PM, Andrei Alexandrescu wrote:

 But please don't consider me an idiot because I don't like what you
 propose.


Hmmm... actually I take that back :o). You are totally free to consider 
me such (or at least of intellect inferior to yours etc). What I mean is 
please don't consider me somehow socially or morally obligated to choose 
your solution just because you like it so much more than mine.

Andrei
Feb 23 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 01:43:11 UTC, Andrei Alexandrescu 
wrote:

 This is a free world. Walter and I are working on a DIP. Please 
 work on yours. I can't promise we'll choose to your liking, but 
 this is the one way you can make your point heard and 
 understood. What doesn't work is trash talk.

 I guarantee I recognize brilliance when I see it. So if you 
 have a brilliant idea, it won't be missed. Have at it. One 
 thing I cannot do is choose a solution that you prefer over one 
 I prefer - this does remain a subjective topic. I can't help 
 it. But please don't consider me an idiot because I don't like 
 what you propose.


I don't think you are being fair here. Even if not formally 
expressed as a DIP, at least Mark and myself have come up with 
fairly detailed explanations, in topic you participated in, so we 
can't really do as if it didn't existed. Also, I do not think 
this is a subjective matter. Yes there is a part of it that is 
matter of taste and is subjective, but overall there is a big 
chunk of objectively discussable things is there proposal, like 
language complexity and expressiveness added to the language.

But here is mostly what I think is going on. We are discussion 
various issues, including make the GC faster, enable safe RC, 
make  nogc more usable (for instance exception usability), safe 
ref, enforcing type qualifier contraints, and so on...

For each of these issues, solution are proposed. What I (and I 
think Mark would agree) propose would solve them all. Yes this is 
more complex than any of the solution proposed for each of these. 
But this is way simpler, and enable way more than having a 
unique, simpler solution for each of these problems.
Feb 23 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 24 February 2015 at 02:35:41 UTC, deadalnix wrote:

 For each of these issues, solution are proposed. What I (and I 
 think Mark would agree) propose would solve them all. Yes this 
 is more complex than any of the solution proposed for each of 
 these. But this is way simpler, and enable way more than having 
 a unique, simpler solution for each of these problems.


Indeed I agree.

But judging by Andrei's response above, I'm longer so sure he is 
actually aware of what we proposed.
Feb 24 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 6:35 PM, deadalnix wrote:

 On Tuesday, 24 February 2015 at 01:43:11 UTC, Andrei Alexandrescu wrote:

 This is a free world. Walter and I are working on a DIP. Please work
 on yours. I can't promise we'll choose to your liking, but this is the
 one way you can make your point heard and understood. What doesn't
 work is trash talk.

 I guarantee I recognize brilliance when I see it. So if you have a
 brilliant idea, it won't be missed. Have at it. One thing I cannot do
 is choose a solution that you prefer over one I prefer - this does
 remain a subjective topic. I can't help it. But please don't consider
 me an idiot because I don't like what you propose.


 I don't think you are being fair here. Even if not formally expressed as
 a DIP, at least Mark and myself have come up with fairly detailed
 explanations, in topic you participated in, so we can't really do as if
 it didn't existed. Also, I do not think this is a subjective matter. Yes
 there is a part of it that is matter of taste and is subjective, but
 overall there is a big chunk of objectively discussable things is there
 proposal, like language complexity and expressiveness added to the
 language.

 But here is mostly what I think is going on. We are discussion various
 issues, including make the GC faster, enable safe RC, make  nogc more
 usable (for instance exception usability), safe ref, enforcing type
 qualifier contraints, and so on...

 For each of these issues, solution are proposed. What I (and I think
 Mark would agree) propose would solve them all. Yes this is more complex
 than any of the solution proposed for each of these. But this is way
 simpler, and enable way more than having a unique, simpler solution for
 each of these problems.


Just replied to Marc about this. I should have phrased my response as "I 
don't know of a _reasonable_ solution". BTW please don't call me 
dishonest anymore, it's doubly inappropriate seeing as we also work 
together. You wouldn't think there's an actual possibility I go around 
lying to people about such stuff. Thanks.

Andrei
Feb 24 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 16:22:47 UTC, Andrei Alexandrescu 
wrote:

 Just replied to Marc about this. I should have phrased my 
 response as "I don't know of a _reasonable_ solution". BTW 
 please don't call me dishonest anymore, it's doubly 
 inappropriate seeing as we also work together. You wouldn't 
 think there's an actual possibility I go around lying to people 
 about such stuff. Thanks.

 Andrei


You have my apologies.

Still, qualifying something of _reasonable_ or not do not 
constitute a solid argument IMO. That is where my frustration 
come from here. You are arguing as if nothing was proposed when 
various persons made proposals. That do not foster progress. If 
actual criticism of the proposals were made, then we could argue 
about these criticisms, or come up with improvement of the 
proposals, but if you argue as if these proposal do not exists, 
we are condemned to talk past each other and no progress can be 
made.
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 9:47 AM, deadalnix wrote:

 On Tuesday, 24 February 2015 at 16:22:47 UTC, Andrei Alexandrescu wrote:

 Just replied to Marc about this. I should have phrased my response as
 "I don't know of a _reasonable_ solution". BTW please don't call me
 dishonest anymore, it's doubly inappropriate seeing as we also work
 together. You wouldn't think there's an actual possibility I go around
 lying to people about such stuff. Thanks.

 Andrei


 You have my apologies.

 Still, qualifying something of _reasonable_ or not do not constitute a
 solid argument IMO. That is where my frustration come from here. You are
 arguing as if nothing was proposed when various persons made proposals.
 That do not foster progress. If actual criticism of the proposals were
 made, then we could argue about these criticisms, or come up with
 improvement of the proposals, but if you argue as if these proposal do
 not exists, we are condemned to talk past each other and no progress can
 be made.


I also owe you apologies for not acknowledging that work. I find the 
proposal too complicated for what it provides and that's the short and 
long of it. It's easy to make a large and complex language addition to 
support any sensible abstraction. That doesn't make it automatically good.

Andrei
Feb 24 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2015 10:00 AM, Andrei Alexandrescu wrote:

 I also owe you apologies for not acknowledging that work. I find the proposal
 too complicated for what it provides and that's the short and long of it. It's
 easy to make a large and complex language addition to support any sensible
 abstraction. That doesn't make it automatically good.


My criticisms of it centered around:

1. confusion about whether it was a storage class or a type qualifier.

2. I agree with Andrei that any annotation system can be made to work - but
this 
one (as are most annotation systems) also struck me as wordy, tedious, and 
aesthetically unappealing. I just can't see myself throwing it up on a slide
and 
trying to sell it to the audience as cool.

3. In line with (2), I want a system that relies much more on inference. We've 
made good progress with the existing annotations being inferred.

4. I didn't see how one could, for example, have an array of pointers:

     int*[] pointers;

and then fill that array with pointers of varying ownership annotations.

5. The (4) homogeneity requirement would mean that templated types would get
new 
instantiations every time they are used with a different ownership. This could 
lead to massive code bloat.

6. The 'return ref' scheme, which you have expressed distaste for, was one that 
required the fewest instances of the user having to add an annotation. It
turned 
out that upgrading Phobos to this required only a handful of annotations.

7. 'return ref' makes memory safe ref counted types possible, finally, in D, 
without needing to upend the language or legacy code. And as the example I 
posted showed, they are straightforward to write. Only time and experience will 
tell if this will be successful, but it looks promising and I hope you'll be 
willing to give it a chance.
Feb 24 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 24 February 2015 at 20:53:24 UTC, Walter Bright wrote:

 On 2/24/2015 10:00 AM, Andrei Alexandrescu wrote:

 I also owe you apologies for not acknowledging that work. I 
 find the proposal
 too complicated for what it provides and that's the short and 
 long of it. It's
 easy to make a large and complex language addition to support 
 any sensible
 abstraction. That doesn't make it automatically good.


 My criticisms of it centered around:

 1. confusion about whether it was a storage class or a type 
 qualifier.

 2. I agree with Andrei that any annotation system can be made 
 to work - but this one (as are most annotation systems) also 
 struck me as wordy, tedious, and aesthetically unappealing. I 
 just can't see myself throwing it up on a slide and trying to 
 sell it to the audience as cool.

 3. In line with (2), I want a system that relies much more on 
 inference. We've made good progress with the existing 
 annotations being inferred.

 4. I didn't see how one could, for example, have an array of 
 pointers:

     int*[] pointers;

 and then fill that array with pointers of varying ownership 
 annotations.

 5. The (4) homogeneity requirement would mean that templated 
 types would get new instantiations every time they are used 
 with a different ownership. This could lead to massive code 
 bloat.

 6. The 'return ref' scheme, which you have expressed distaste 
 for, was one that required the fewest instances of the user 
 having to add an annotation. It turned out that upgrading 
 Phobos to this required only a handful of annotations.

 7. 'return ref' makes memory safe ref counted types possible, 
 finally, in D, without needing to upend the language or legacy 
 code. And as the example I posted showed, they are 
 straightforward to write. Only time and experience will tell if 
 this will be successful, but it looks promising and I hope 
 you'll be willing to give it a chance.


Thanks for summarizing your reasons. I knew that you were unhappy 
with 1 - 3); I wasn't aware of 4) and 5). I can't get into detail 
now, as it's already late at night here, but I'll try thinking 
about it tomorrow.

FWIW, Zach just wrote in another thread that he will have his own 
proposal ready soon, based on DIP25, with `scope` being a storage 
class. Let's see how this goes.
Feb 24 2015
prev sibling parent "Zach the Mystic" <reachzach gggmail.com> writes:
Here's my best so far:

http://forum.dlang.org/post/offurllmuxjewizxedab forum.dlang.org

On Tuesday, 24 February 2015 at 20:53:24 UTC, Walter Bright wrote:

 My criticisms of it centered around:

 1. confusion about whether it was a storage class or a type 
 qualifier.


My system has neither. Instead, it just bans unsafe reference 
copying in  safe code.


 2. I agree with Andrei that any annotation system can be made 
 to work - but this one (as are most annotation systems) also 
 struck me as wordy, tedious, and aesthetically unappealing. I 
 just can't see myself throwing it up on a slide and trying to 
 sell it to the audience as cool.

 3. In line with (2), I want a system that relies much more on 
 inference. We've made good progress with the existing 
 annotations being inferred.


Well you know I'm on board with this. The one penalty my system 
requires is two more parameter attributes, which I'm hoping can 
be alleviated by inference as much as possible.


 4. I didn't see how one could, for example, have an array of 
 pointers:

     int*[] pointers;

 and then fill that array with pointers of varying ownership 
 annotations.

 5. The (4) homogeneity requirement would mean that templated 
 types would get new instantiations every time they are used 
 with a different ownership. This could lead to massive code 
 bloat.


I deliberately designed my system to avoid all associations with 
type. No code bloat.


 6. The 'return ref' scheme, which you have expressed distaste 
 for, was one that required the fewest instances of the user 
 having to add an annotation. It turned out that upgrading 
 Phobos to this required only a handful of annotations.

 7. 'return ref' makes memory safe ref counted types possible, 
 finally, in D, without needing to upend the language or legacy 
 code. And as the example I posted showed, they are 
 straightforward to write. Only time and experience will tell if 
 this will be successful, but it looks promising and I hope 
 you'll be willing to give it a chance.


I do give it a chance! See my proposal!
Feb 26 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu 
wrote:

 On 2/23/15 6:56 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 These two points have undesirable consequences: All consumers 
 such
 objects need to be aware of the exact type, which includes the
 management strategy (RC, Unique, GC). But this is a violation 
 of the
 principle of separation of concerns: a consumer shouldn't need 
 to have
 information about the management strategy, it should work 
 equally with
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take
 ownership of the resource.


 Well I don't know of another way.


Ok, I wrote my reply assuming that you are aware of the various 
proposals deadalnix, myself and several other people have made in 
the past, some of them quite specific. But now that I think of 
it, I don't remember that you were ever directly referring to it 
in any of your posts. Maybe you just missed it?

As one example, here is what I originally suggested:
http://wiki.dlang.org/User:Schuetzm/scope

It's not completely up to date, during discussions I gained many 
useful new insights to simplify it and make things more 
consistent. It's also part of a bigger picture (deadalnix's ideas 
about ownership play an important role, too), which unfortunately 
isn't easy to recognize, because this page has become quite large 
und unwieldy. I should make a post explaining this.


 In general, this and related proposals tend to limit 
 themselves on
 memory management (as witnessed by the importance that `ref` 
 and ` safe`
 play in them). This is too narrow IMO. A well thought-out 
 solution can
 be equally applicable to the broader field of resource 
 management.


 Looking forward to your insights.


See above. I had the impression that you were aware of the 
proposals, but for some reason were opposed to them. Maybe that's 
not the case?
Feb 24 2015
next sibling parent "Zach the Mystic" <reachzach gggmail.com> writes:
On Tuesday, 24 February 2015 at 12:44:54 UTC, Marc Schütz wrote:

 On Monday, 23 February 2015 at 18:16:38 UTC, Andrei 
 Alexandrescu wrote:

 On 2/23/15 6:56 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 These two points have undesirable consequences: All consumers 
 such
 objects need to be aware of the exact type, which includes the
 management strategy (RC, Unique, GC). But this is a violation 
 of the
 principle of separation of concerns: a consumer shouldn't 
 need to have
 information about the management strategy, it should work 
 equally with
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take
 ownership of the resource.


 Well I don't know of another way.


 Ok, I wrote my reply assuming that you are aware of the various 
 proposals deadalnix, myself and several other people have made 
 in the past, some of them quite specific. But now that I think 
 of it, I don't remember that you were ever directly referring 
 to it in any of your posts. Maybe you just missed it?

 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope

 It's not completely up to date, during discussions I gained 
 many useful new insights to simplify it and make things more 
 consistent. It's also part of a bigger picture (deadalnix's 
 ideas about ownership play an important role, too), which 
 unfortunately isn't easy to recognize, because this page has 
 become quite large und unwieldy. I should make a post 
 explaining this.


I'm working on my own idea now. I make scope transitive, because 
it's both memory safe and simple to implement, but doing so may 
cause some things which are actually safe to be considered unsafe 
(but then you could just use  system blocks or  trusted lambdas 
to correct this). Also, I don't think `scope` needs to be part of 
the type.

I'm about 90 percent sure, 10 percent unsure that my system will 
work. I'll have it soon enough. It needs DIP25 to be expanded to 
all reference types (not just `ref`), requires my own DIP71, 
http://wiki.dlang.org/DIP71 for total safety, and possibly one or 
two more additions for a reliable ownership. The only real cost 
is added complexity to function signatures (a la DIP25), which 
can and should be inferred in most cases, assuming we aren't 
crippled by an ancient and subpar linking mechanism which 
requires all this manual marking of signatures all the time.

Stay tuned, sir!
Feb 24 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope


Walter and I discussed this proposal at length several times.

I think it can be made to work. I also think it's overly complicated for 
what it does and that it's not brilliant.

These are the most dangerous designs: complicated, not brilliant, but 
can be made to work - because they end up getting implemented. We want 
to get away without implementing it.

Currently we want to explore DIP25, which we believe has a very good 
price/performance profile (whilst not being brilliant itself), and its 
impact on designing safe struct types. It is our suspicion that DIP25 is 
all we need for making involved struct types safe. If that's the case, 
that paves the road toward safe class types using reference counting 
with very small language changes.

There will be stragglers, such as taking slices into stack-allocated 
statically-sized arrays. At that point the question will be whether we 
can implement that as a  safe struct, we need a language change, or we 
can just live with it (after all, using stack-allocated arrays in safe 
code is a rather niche use).

So the short answer is: yes, I've studied your proposal and I don't 
think it's good enough. Furthermore I don't think there are small 
changes to it that will make it good enough. This is not personal and 
please don't take offense over that. In contrast, I think DIP25 is good 
(partly because it's deceptively simple) and I want to pursue it and its 
consequences.


Andrei
Feb 24 2015
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:


 using stack-allocated arrays in safe code is a rather niche 
 use).


I use stack-allocated arrays in supposedly safe code. And having 
a future D compiler that makes that kind of code actually safe is 
a good thing.

Bye,
bearophile
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 8:34 AM, bearophile wrote:

 Andrei Alexandrescu:


 using stack-allocated arrays in safe code is a rather niche use).


 I use stack-allocated arrays in supposedly safe code. And having a
 future D compiler that makes that kind of code actually safe is a good
 thing.


It definitely is a good thing. The question is balancing its goodness 
against the costs of allowing it. It's not impossible you may get asked 
to change your code to use a struct instead of a fixed-sized array.

Andrei
Feb 24 2015
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:


 It definitely is a good thing. The question is balancing its 
 goodness against the costs of allowing it. It's not impossible 
 you may get asked to change your code to use a struct instead 
 of a fixed-sized array.


I don't like the look of the annotations of DIP25. I'd like DIP25 
removed from D language and replaced by a more principled 
solution. Sometimes a heavier solution can be simpler to 
understand (and it can be actually safe).

Bye,
bearophile
Feb 24 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 11:12 AM, bearophile wrote:

 Andrei Alexandrescu:


 It definitely is a good thing. The question is balancing its goodness
 against the costs of allowing it. It's not impossible you may get
 asked to change your code to use a struct instead of a fixed-sized array.


 I don't like the look of the annotations of DIP25.


The way things are going you'll likely have to live with that. -- Andrei
Feb 24 2015
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2015 11:12 AM, bearophile wrote:

 I don't like the look of the annotations of DIP25. I'd like DIP25 removed from
D
 language and replaced by a more principled solution. Sometimes a heavier
 solution can be simpler to understand (and it can be actually safe).


So far, DIP25 has been called by various posters unprincipled, hackish, and 
stupid, without supporting rationale.

Note the thread "A Refcounted Array Type" thread, which uses DIP25 to implement 
a memory safe ref counted container. It is simple and requires exactly one 
annotation. It seems pretty straightforward to me, and so far nobody has shown 
any holes in it, or has even commented on the principle of it. (complaints
about 
bounds checking, delete, etc., are off-topic)
Feb 24 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 21:03:38 UTC, Walter Bright wrote:

 On 2/24/2015 11:12 AM, bearophile wrote:

 I don't like the look of the annotations of DIP25. I'd like 
 DIP25 removed from D
 language and replaced by a more principled solution. Sometimes 
 a heavier
 solution can be simpler to understand (and it can be actually 
 safe).


 So far, DIP25 has been called by various posters unprincipled, 
 hackish, and stupid, without supporting rationale.



I do think this comes from looking at the larger picture.

In isolation, DIP25 is not that bad. It solve a class of problems 
properly, which is already something. Where I think most people 
find it hacky, unprincipled, or whatever is when looking at the 
larger picture.

It seems obvious at this point that DIP25 is not the alpha and 
omega of the problem (this very thread is an example of this). 
For instance, DIP25 do not do anything for classes to be safely 
reference counted.

People are - legitimately - afraid that a set of DIP25 like 
solution will be adopted for each of these problems. When looking 
at the total cost of the set of solution, compared to a more 
complex, more principled solution, it seems that this is not the 
way forward.

In that sense, DIP25 seems like a hack to make reference counting 
work rather than the addition of a basic language feature that 
add a new dimension of expressiveness to the language. A bit like 
Rvalue references in C++ have been added to support std::move and 
std::forward instead of being a generally useful basic block to 
build upon.


 Note the thread "A Refcounted Array Type" thread, which uses 
 DIP25 to implement a memory safe ref counted container. It is 
 simple and requires exactly one annotation. It seems pretty 
 straightforward to me, and so far nobody has shown any holes in 
 it, or has even commented on the principle of it. (complaints 
 about bounds checking, delete, etc., are off-topic)


I still have to review it in details, but I'm sure this is good 
or at least fixable in a way that make it good.

Yet, as mentioned, the problem do not come with DIP25 in 
isolation. DIP25 does its job well. It just does a too limited 
job, and it seems that DIP25 and DIP25 like solution for other 
jobs, will ultimately lead to a more complex situation than where 
the 'too complex' solutions would lead us.
Feb 24 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 1:44 PM, deadalnix wrote:

 For instance, DIP25 do not do anything for classes to be safely
 reference counted.


It does. One small language change is necessary. DIP will be 
forthcoming. -- Andrei
Feb 24 2015
prev sibling next sibling parent reply "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= <kuettler gmail.com> writes:
On Tuesday, 24 February 2015 at 16:18:45 UTC, Andrei Alexandrescu 
wrote:

 On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope


 Walter and I discussed this proposal at length several times.


Is there a place where the outcome of such discussions is 
documented?

This feels inappropriate like asking for additional paperwork. 
Still, there are so many heated discussions going on here. It 
might be helpful to know what was considered, what was decided 
and for what reason. Then again, maybe I just did not get the 
memo.
Feb 24 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 9:35 AM, "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= 
<kuettler gmail.com>" wrote:

 On Tuesday, 24 February 2015 at 16:18:45 UTC, Andrei Alexandrescu wrote:

 On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>"
 wrote:

 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope


 Walter and I discussed this proposal at length several times.


 Is there a place where the outcome of such discussions is documented?

 This feels inappropriate like asking for additional paperwork. Still,
 there are so many heated discussions going on here. It might be helpful
 to know what was considered, what was decided and for what reason. Then
 again, maybe I just did not get the memo.


Closest thing to documentation are http://wiki.dlang.org/DIP69 and 
http://wiki.dlang.org/DIP25, both of which came at the same time 
inspired by that work, and in reaction to its complexity. We are looking 
at what parts of DIP69 are simplified/obviated by DIP25. If we can get 
away with no DIP69 at all, so much the better. -- Andrei
Feb 24 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 24 February 2015 at 16:18:45 UTC, Andrei Alexandrescu 
wrote:

 On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope


 Walter and I discussed this proposal at length several times.

 I think it can be made to work. I also think it's overly 
 complicated for what it does and that it's not brilliant.

 These are the most dangerous designs: complicated, not 
 brilliant, but can be made to work - because they end up 
 getting implemented. We want to get away without implementing 
 it.

 Currently we want to explore DIP25, which we believe has a very 
 good price/performance profile (whilst not being brilliant 
 itself), and its impact on designing safe struct types. It is 
 our suspicion that DIP25 is all we need for making involved 
 struct types safe. If that's the case, that paves the road 
 toward safe class types using reference counting with very 
 small language changes.

 There will be stragglers, such as taking slices into 
 stack-allocated statically-sized arrays. At that point the 
 question will be whether we can implement that as a  safe 
 struct, we need a language change, or we can just live with it 
 (after all, using stack-allocated arrays in safe code is a 
 rather niche use).

 So the short answer is: yes, I've studied your proposal and I 
 don't think it's good enough. Furthermore I don't think there 
 are small changes to it that will make it good enough. This is 
 not personal and please don't take offense over that. In 
 contrast, I think DIP25 is good (partly because it's 
 deceptively simple) and I want to pursue it and its 
 consequences.


I'm certainly not offended. I don't think the proposal is 
"brilliant" either, it was meant as a starting point for 
discussions, not as a completely finished proposal aka DIP. Out 
of interest, because that will be crucial for the further 
development: are you completely opposed to any kind of ownership 
system, or is it something specific about this particular 
proposal that you don't like?
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 9:47 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 Out of interest, because that will be crucial for the further
 development: are you completely opposed to any kind of ownership system,
 or is it something specific about this particular proposal that you
 don't like?


There's no opposition on matters of principle/dogma. It's just too 
complicated for what it provides.

Andrei
Feb 24 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 18:01:35 UTC, Andrei Alexandrescu 
wrote:

 On 2/24/15 9:47 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 Out of interest, because that will be crucial for the further
 development: are you completely opposed to any kind of 
 ownership system,
 or is it something specific about this particular proposal 
 that you
 don't like?


 There's no opposition on matters of principle/dogma. It's just 
 too complicated for what it provides.

 Andrei


Compared to what ?
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 11:14 AM, deadalnix wrote:

 On Tuesday, 24 February 2015 at 18:01:35 UTC, Andrei Alexandrescu wrote:

 On 2/24/15 9:47 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>"
 wrote:

 Out of interest, because that will be crucial for the further
 development: are you completely opposed to any kind of ownership system,
 or is it something specific about this particular proposal that you
 don't like?


 There's no opposition on matters of principle/dogma. It's just too
 complicated for what it provides.

 Andrei


 Compared to what ?


There is no comparison involved. The proposal is too complicated for 
what it does. -- Andrei
Feb 24 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 19:20:37 UTC, Andrei Alexandrescu 
wrote:

 Compared to what ?


 There is no comparison involved. The proposal is too 
 complicated for what it does. -- Andrei


There is always a comparison involved. Comparison options 
includes:
  - Not solving some of the issues we have and call it a day.
  - Add simpler solution dedicated to each of these problems.

The problems we are talking about are:
  - safe RC
  - fast GC by using type qualifier guarantee.
  - less reliance on GC by producing less garbage.
  - enforcing type qualifier constraint.
  - enabling more code to be  nogc (especially exception)
  - being able to delegate the memory management policy to the 
user.
  - immutable and shared building.
  - manipulating object of limited lifetime (DIP25 solve this 
partially).
  - `unique` message passing .
  - safe parallelism on graph of object instead of only value 
types.
  - lock on graph of objects (shared containers).

Now it is clear that we can come up with simpler solution for 
some of these problem than whatever all encompassing solution 
would be. It is also clear that is is not possible to get a set 
of simpler solution that is, as a whole, simpler than an all 
encompassing solution. That means that some of the above 
mentioned point must be acknowledged as not going to be fixed.

It is also to be noted that these above mentioned problems are 
not orthogonal. For instance, getting various GC optimizations in 
that take advantage of type qualifiers guarantee require that 
these guarantee are enforced. To get these constraints enforced, 
you need to be able to build immutable and/or shared in a way 
that don't break these constraints, and so on.
Feb 24 2015