www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Next step on reference counting topics

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
There's been a lot of talk lately regarding improving resource 
management for D, and I'd like to figure the next logical step to take. 
It seems clear that we have reached a collective impasse on a few 
fundamentals, and that more just talk about it all is at the point of 
diminishing returns.

One action item that is hopefully useful to people of all viewpoints is 
to double down on library support, and see how far we can get and what 
insights we collect from the experience.

For that I'm proposing we start real work toward a state-of-the-art 
std.refcounted module. It would include adapters for class, array, and 
pointer types, and should inform language improvements for qualifiers 
(i.e. the tail-const problem), copy elision, literals, operators, and such.

Who wants to champion this effort?


Thanks,

Andrei
May 12 2014
next sibling parent Jacob Carlborg <doob me.com> writes:
On 12/05/14 21:00, Andrei Alexandrescu wrote:

 There's been a lot of talk lately regarding improving resource
 management for D, and I'd like to figure the next logical step to take.
 It seems clear that we have reached a collective impasse on a few
 fundamentals, and that more just talk about it all is at the point of
 diminishing returns.

 One action item that is hopefully useful to people of all viewpoints is
 to double down on library support, and see how far we can get and what
 insights we collect from the experience.

 For that I'm proposing we start real work toward a state-of-the-art
 std.refcounted module. It would include adapters for class, array, and
 pointer types, and should inform language improvements for qualifiers
 (i.e. the tail-const problem), copy elision, literals, operators, and such.


Perhaps, as has been already stared, sprinkle Phobos with output ranges 
and/or allocators.

-- 
/Jacob Carlborg
May 12 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 12 May 2014 at 19:00:33 UTC, Andrei Alexandrescu wrote:

 For that I'm proposing we start real work toward a 
 state-of-the-art std.refcounted module. It would include 
 adapters for class, array, and pointer types, and should inform 
 language improvements for qualifiers (i.e. the tail-const 
 problem), copy elision, literals, operators, and such.


We don't have language tools to do it. Main problem with 
implementing reference counted pointers as structs is that you 
are forced to abandon polymorphism:

class A {}
class B : A {}

void foo(RefCounted!A) {}

void main()
{
     RefCounted!B b = new B();
     foo(b); // no way to make it work
}

This severely limits applicability of such library solution, 
especially when it comes to something like exceptions.
May 13 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/13/14, 6:41 AM, Dicebot wrote:

 On Monday, 12 May 2014 at 19:00:33 UTC, Andrei Alexandrescu wrote:

 For that I'm proposing we start real work toward a state-of-the-art
 std.refcounted module. It would include adapters for class, array, and
 pointer types, and should inform language improvements for qualifiers
 (i.e. the tail-const problem), copy elision, literals, operators, and
 such.


 We don't have language tools to do it. Main problem with implementing
 reference counted pointers as structs is that you are forced to abandon
 polymorphism:

 class A {}
 class B : A {}

 void foo(RefCounted!A) {}

 void main()
 {
      RefCounted!B b = new B();
      foo(b); // no way to make it work
 }

 This severely limits applicability of such library solution, especially
 when it comes to something like exceptions.


"alias this" should be of help here. Could you please try it? Thanks! -- 
Andrei
May 13 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/13/2014 05:11 PM, Andrei Alexandrescu wrote:

 On 5/13/14, 6:41 AM, Dicebot wrote:

 On Monday, 12 May 2014 at 19:00:33 UTC, Andrei Alexandrescu wrote:

 For that I'm proposing we start real work toward a state-of-the-art
 std.refcounted module. It would include adapters for class, array, and
 pointer types, and should inform language improvements for qualifiers
 (i.e. the tail-const problem), copy elision, literals, operators, and
 such.


 We don't have language tools to do it. Main problem with implementing
 reference counted pointers as structs is that you are forced to abandon
 polymorphism:

 class A {}
 class B : A {}

 void foo(RefCounted!A) {}

 void main()
 {
      RefCounted!B b = new B();
      foo(b); // no way to make it work
 }

 This severely limits applicability of such library solution, especially
 when it comes to something like exceptions.


 "alias this" should be of help here. Could you please try it? Thanks! --
 Andrei


class A{
     RefCounted!A r(){ ... }
}

class B : A {
     override RefCounted!B r(){ ... } // ...
}

RefCounted!B[] b;
RefCounted!(const(A))[] a=b; // ...

etc.
May 13 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 13 May 2014 at 15:11:48 UTC, Andrei Alexandrescu 
wrote:

 On 5/13/14, 6:41 AM, Dicebot wrote:

 On Monday, 12 May 2014 at 19:00:33 UTC, Andrei Alexandrescu 
 wrote:

 For that I'm proposing we start real work toward a 
 state-of-the-art
 std.refcounted module. It would include adapters for class, 
 array, and
 pointer types, and should inform language improvements for 
 qualifiers
 (i.e. the tail-const problem), copy elision, literals, 
 operators, and
 such.


 We don't have language tools to do it. Main problem with 
 implementing
 reference counted pointers as structs is that you are forced 
 to abandon
 polymorphism:

 class A {}
 class B : A {}

 void foo(RefCounted!A) {}

 void main()
 {
     RefCounted!B b = new B();
     foo(b); // no way to make it work
 }

 This severely limits applicability of such library solution, 
 especially
 when it comes to something like exceptions.


 "alias this" should be of help here. Could you please try it? 
 Thanks! -- Andrei


We don't have multiple alias this to cover all parent classes. 
Also you can't throw struct as an Exception. Also Timon's example.
May 13 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On actual topic of next step - I agree with Jacob, removing all
internal allocations from Phobos is most beneficial short-term
goal.
May 13 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/13/14, 6:43 AM, Dicebot wrote:

 On actual topic of next step - I agree with Jacob, removing all
 internal allocations from Phobos is most beneficial short-term
 goal.


That's a given. This task and improving RC don't compete (except, of 
course, for bearing on the same people). I ask again, who wants to 
champion std.refcounted? Rainer? Manu? Michel? Are you there? -- Andrei
May 13 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 13 May 2014 at 15:14:16 UTC, Andrei Alexandrescu 
wrote:

 On 5/13/14, 6:43 AM, Dicebot wrote:

 On actual topic of next step - I agree with Jacob, removing all
 internal allocations from Phobos is most beneficial short-term
 goal.


 That's a given. This task and improving RC don't compete 
 (except, of course, for bearing on the same people). I ask 
 again, who wants to champion std.refcounted? Rainer? Manu? 
 Michel? Are you there? -- Andrei


You should start with at least some sort of design document for 
what such module should look like. Right now it is not clear to 
me at all what kind of improvements are desired within current 
language limits.
May 15 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/15/14, 6:02 AM, Dicebot wrote:

 On Tuesday, 13 May 2014 at 15:14:16 UTC, Andrei Alexandrescu wrote:

 On 5/13/14, 6:43 AM, Dicebot wrote:

 On actual topic of next step - I agree with Jacob, removing all
 internal allocations from Phobos is most beneficial short-term
 goal.


 That's a given. This task and improving RC don't compete (except, of
 course, for bearing on the same people). I ask again, who wants to
 champion std.refcounted? Rainer? Manu? Michel? Are you there? -- Andrei


 You should start with at least some sort of design document for what
 such module should look like.


The design is part of the championing. But haven't heard a peep - it 
seems everybody wants to go to heaven, but nobody wants to die :o).


 Right now it is not clear to me at all
 what kind of improvements are desired within current language limits.


The clarify it to yourself and others. Start with a library approach to 
making reference counting easy and convenient. Then see how it informs 
possible language improvements.


Andrei
May 15 2014
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 15 May 2014 at 15:46:13 UTC, Andrei Alexandrescu 
wrote:

 On 5/15/14, 6:02 AM, Dicebot wrote:

 On Tuesday, 13 May 2014 at 15:14:16 UTC, Andrei Alexandrescu 
 wrote:

 On 5/13/14, 6:43 AM, Dicebot wrote:

 On actual topic of next step - I agree with Jacob, removing 
 all
 internal allocations from Phobos is most beneficial 
 short-term
 goal.


 That's a given. This task and improving RC don't compete 
 (except, of
 course, for bearing on the same people). I ask again, who 
 wants to
 champion std.refcounted? Rainer? Manu? Michel? Are you there? 
 -- Andrei


 You should start with at least some sort of design document 
 for what
 such module should look like.


 The design is part of the championing. But haven't heard a peep 
 - it seems everybody wants to go to heaven, but nobody wants to 
 die :o).


 Right now it is not clear to me at all
 what kind of improvements are desired within current language 
 limits.


 The clarify it to yourself and others. Start with a library 
 approach to making reference counting easy and convenient. Then 
 see how it informs possible language improvements.


Well I have already pointed out my concerns in this thread. Until 
some solution is found new refcount module doesn't seem worth 
putting the effort into.
May 15 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2014 6:43 AM, Dicebot wrote:

 On actual topic of next step - I agree with Jacob, removing all
 internal allocations from Phobos is most beneficial short-term
 goal.


I'm working on it. Could use some help!
May 13 2014
parent "Ethan" <gooberman gmail.com> writes:
I've been trying to think of a solution to use over here at 
Remedy for making the garbage collector reference count 
allocations instead of the current scan method (even with Rainer 
Schütze's GC it still does a scan, and I'd feel much more 
comfortable not having to schedule a GC collection somewhere in 
our update loop).

I've totally missed the other threads on reference counting, but 
should reference counting be a part of some library that you can 
optionally use or should the garbage collector hide all that from 
you?

My thinking on the subject is that you only need one additional 
function added to the garbage collector that handles pointer 
assignments. The parameters are a ref to the pointer you're 
assigning to, and the new pointer. The GC would then decrement 
the reference count for the memory referenced by the pointer 
you're assigning to, and increment the reference count fr the new 
memory referenced.

This, I imagine, would fit neatly in with slices and classes 
equally. It would also be optional - the current default GC 
wouldn't do anything with the function and . The downside is that 
a pointer assign would invoke a GC call and that it would still 
need to deal with circular dependencies in some fashion (however 
dealing with that at an allocation level rather than a class 
level seems potentially less problematic to me).

If this is beyond the scope of this thread, I'll start up a new 
one to discuss all the potential pitfalls and problems that could 
be encountered. I haven't had any time to actually do research in 
to it or trial it yet.
May 15 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 12 May 2014 at 19:00:33 UTC, Andrei Alexandrescu wrote:

 There's been a lot of talk lately regarding improving resource 
 management for D, and I'd like to figure the next logical step 
 to take. It seems clear that we have reached a collective 
 impasse on a few fundamentals, and that more just talk about it 
 all is at the point of diminishing returns.

 One action item that is hopefully useful to people of all 
 viewpoints is to double down on library support, and see how 
 far we can get and what insights we collect from the experience.

 For that I'm proposing we start real work toward a 
 state-of-the-art std.refcounted module. It would include 
 adapters for class, array, and pointer types, and should inform 
 language improvements for qualifiers (i.e. the tail-const 
 problem), copy elision, literals, operators, and such.

 Who wants to champion this effort?


 Thanks,

 Andrei


Also please std.typecons.refcounted, let's stop that flat
hierarchy madness.
May 13 2014