www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Rust moving away from GC into reference counting

reply "Paulo Pinto" <pjmlp progtools.org> writes:
Even as GC fanboy, I have to admit that reference counting is in 
trend for system languages.

Rust developers are thinking to move GC support to the language 
library while keeping reference counting as the main way to deal 
with memory management.

http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/

Quite in sync with the latest discussions going on.

--
Paulo
Jun 03 2013
next sibling parent reply "Dicebot" <m.strashun gmail.com> writes:
Is it an official position or just a blog post / proposal from 
one of developers?

Anyway, given  the recent findings by Adam, D is not _that_ far 
away here. Add configurable global allocators, finally implement 
"scope", move GC to library and reduce runtime a bit - and result 
may be pretty awesome. Hardest part is still "global allocator" 
and assumptions compiler does about runtime.

However, the fact that Rust developers already think about this 
and in D community most real attention to such stuff only came 
together with Manu gives them some advantage.

On Monday, 3 June 2013 at 09:10:14 UTC, Paulo Pinto wrote:

 Even as GC fanboy, I have to admit that reference counting is 
 in trend for system languages.

 Rust developers are thinking to move GC support to the language 
 library while keeping reference counting as the main way to 
 deal with memory management.

 http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/

 Quite in sync with the latest discussions going on.

 --
 Paulo
Jun 03 2013
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
I think we could do the whole owned pointer concept in D too, by 
making a struct with a disabled copy constructor. I don't know 
much about Rust's specifics though so not sure if that's an exact 
match.

But I'm already running with the idea that all slices are 
"borrowed", so we could say the same thing about pointers, if you 
have one, assume it is borrowed and you shouldn't store it nor 
free it.
Jun 03 2013
parent reply "Dicebot" <m.strashun gmail.com> writes:
On Monday, 3 June 2013 at 14:04:23 UTC, Adam D. Ruppe wrote:

 I think we could do the whole owned pointer concept in D too, 
 by making a struct with a disabled copy constructor. I don't 
 know much about Rust's specifics though so not sure if that's 
 an exact match.


I don't see how struct with a disabled copy constructor is 
relevant to owned pointers. Most similar thing D has is "scope" 
qualifier concept (and that is why I do want it so hard :)) - 
hard guarantee that no pointer to your data will live longer than 
data itself. All normal pointers become managed pointers in that 
sense.
Jun 03 2013
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 3 June 2013 at 14:20:06 UTC, Dicebot wrote:

 I don't see how struct with a disabled copy constructor is 
 relevant to owned pointers.


My thought is then you can control access to the inner pointer 
better that way. You couldn't even pass this struct to a function 
without calling some kind of method, which could return a 
different type to indicate that it is a lent pointer.

It also wouldn't need to be reference counted, since the refcount 
is always going to be 1 because copying it is impossible.


 Most similar thing D has is "scope" qualifier concept (and that 
 is why I do want it so hard :)) - hard guarantee that no 
 pointer to your data will live longer than data itself. All 
 normal pointers become managed pointers in that sense.


Yes, I'm just trying to do what we can with the language today.
Jun 03 2013
next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 3 June 2013 at 14:33:12 UTC, Adam D. Ruppe wrote:

 My thought is then you can control access to the inner pointer 
 better that way. You couldn't even pass this struct to a 
 function without calling some kind of method, which could 
 return a different type to indicate that it is a lent pointer.


Better - sure. But without type system support it will always be 
inferior. One issues that immediately comes to my mind is that it 
is yet another case when you either have right qualifier as a 
default or need to resort to automatic inference (owning 
pointers/references are useless if majority of Phobos does not 
accept borrowed ones). Another one is template bloat, of course.

If we need to resort to workarounds instead of providing good 
solid approach for interested users, it just won't work. This 
will be interesting to hack around as a proof-of-concept 
experiment but won't suit for a production usage, IMHO.
Jun 03 2013
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jun 03, 2013 at 04:33:10PM +0200, Adam D. Ruppe wrote:

 On Monday, 3 June 2013 at 14:20:06 UTC, Dicebot wrote:

I don't see how struct with a disabled copy constructor is
relevant to owned pointers.


 
 My thought is then you can control access to the inner pointer
 better that way. You couldn't even pass this struct to a function
 without calling some kind of method, which could return a different
 type to indicate that it is a lent pointer.


This is an old idea. It's the same as C++'s auto_ptr, and the same as an
old idea I independently came up with many years ago. Basically, you
have two kinds of pointers: a reference pointer and an owner pointer. An
owner pointer can be freely copied into a reference pointer, but never
the other way around. An owner pointer is unique, so it has destructive
copy semantics (passing it into a function invalidates it in the
caller's scope, for example -- if a function doesn't need ownership of
the passed object, it should take a reference argument, to which an
owner pointer implicitly converts). When it goes out of scope, it's
freed. Reference pointers are never freed.

Of course, this scheme doesn't correctly deal with dangling reference
pointers, but in a manual memory management system, you have to deal
with that manually anyway, so it doesn't really matter. Distinguishing
between these two kinds of pointers for the most part eliminates a lot
of pointer-related bugs.

For the most part, this scheme is sufficient to handle a majority of
pointer uses. The remaining cases involve multiple references to objects
with no clear ownership designation or lifetime.  IME, this is
relatively confined to specific areas of usage, for which the GC is a
better memory management scheme anyway.



 It also wouldn't need to be reference counted, since the refcount is
 always going to be 1 because copying it is impossible.


One should always be aware that there's a rough hierarchy of memory
management schemes:

- direct malloc/free: maximum control, most error-prone
- auto_ptr (the above scheme): less error-prone
- reference counting: more convenient
- GC: very little control, least error-prone

The further down you go, the less you have control over the memory
management process, but also the more convenient it is to write code.
The further up you go, the more control you have, but also the harder it
is to write correct code (and the more error-prone it is).

Given D's design of correctness-first, it would seem that GC by default
is the correct choice. But it should definitely also allow moving up the
hierarchy for applications that require finer control over memory
management. Currently D allows this in theory, but in practice, much of
Phobos assumes the GC, which greatly reduces its usefulness in such
cases.



Most similar thing D has is "scope" qualifier concept (and that is
why I do want it so hard :)) - hard guarantee that no pointer to
your data will live longer than data itself. All normal pointers
become managed pointers in that sense.


 
 Yes, I'm just trying to do what we can with the language today.


auto_ptr can be easily implemented with the language today. It's more a
question of Phobos / certain language constructs *using* it.


T

-- 
Some ideas are so stupid that only intellectuals could believe them. -- George
Orwell
Jun 03 2013
parent reply "Rob T" <alanb ucora.com> writes:
On Monday, 3 June 2013 at 16:01:29 UTC, H. S. Teoh wrote:

C++11 deprecated auto_ptr in favor of unique_ptr, but it's 
basically the same concept, and it works very well in cases where 
you prefer to manage your own memory. D should have the same 
thing in the std lib, it's not difficult to implement. The ref 
counted pointers are another similar thing, a bit more difficult 
to implement correctly and there's a circular ref issue with them.

The "hierarchy of memory management schemes" is something we 
should embrace instead of shun in favor of the GC. I dislike 
being forced to use the GC, or having to jump through hoops to 
avoid it, and it's insane to have no control over it even when I 
really do want to make use out of it.

The GC can of course be made a lot better, and at least some 
important manual  control can be given to the programmer, for 
example we previously discussed ideas like specifying a maximum 
time limit for each GC run, and also specifying when the GC gets 
called. Currently we have virtually zero contol over the GC 
(other than enable and disable which is far too trivial) but I 
see no reason at all why this must be the case. Allowing some 
significant control over the GC should be independent of the GC 
implementation, so having a better GC design should in no way 
reduce or remove the requirement for having control.

Also a better GC in no way invalidates the need for other memory 
management schemes because there will always be situations where 
a GC is not an appropriate solution, at least not until someone 
invents the perfect one size fits all GC.

--rt
Jun 03 2013
next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 3 June 2013 at 16:47:35 UTC, Rob T wrote:

 On Monday, 3 June 2013 at 16:01:29 UTC, H. S. Teoh wrote:

 C++11 deprecated auto_ptr in favor of unique_ptr, but it's 
 basically the same concept, and it works very well in cases 
 where you prefer to manage your own memory. D should have the 
 same thing in the std lib, it's not difficult to implement. The 
 ref counted pointers are another similar thing, a bit more 
 difficult to implement correctly and there's a circular ref 
 issue with them.



I can't comment how good it is though.
Jun 03 2013
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Monday, 3 June 2013 at 16:47:35 UTC, Rob T wrote:

 Also a better GC in no way invalidates the need for other 
 memory management schemes because there will always be 
 situations where a GC is not an appropriate solution, at least 
 not until someone invents the perfect one size fits all GC.


phobos has Unique
Jun 03 2013
parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 3 June 2013 at 18:10:50 UTC, Kagamin wrote:

 On Monday, 3 June 2013 at 16:47:35 UTC, Rob T wrote:

 Also a better GC in no way invalidates the need for other 
 memory management schemes because there will always be 
 situations where a GC is not an appropriate solution, at least 
 not until someone invents the perfect one size fits all GC.


 phobos has Unique 




RefCounted notably does not work with classes yet (just because 
no one has taken the time to add support).
Jun 03 2013
prev sibling parent reply Manu <turkeyman gmail.com> writes:
Haha, wow. Indeed, isn't that well timed with respect to recent
discussions! ;)
Post-dconf, I gave it some serious thought and I'm kinda convincing myself
more and more each day that it's the way to go.

On 3 June 2013 19:10, Paulo Pinto <pjmlp progtools.org> wrote:


 Even as GC fanboy, I have to admit that reference counting is in trend for
 system languages.

 Rust developers are thinking to move GC support to the language library
 while keeping reference counting as the main way to deal with memory
 management.

 http://pcwalton.github.io/**blog/2013/06/02/removing-**
 garbage-collection-from-the-**rust-language/<http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/>

 Quite in sync with the latest discussions going on.

 --
 Paulo
Jun 03 2013
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 03.06.2013 18:00, schrieb Manu:

 Haha, wow. Indeed, isn't that well timed with respect to recent
 discussions! ;)
 Post-dconf, I gave it some serious thought and I'm kinda convincing
 myself more and more each day that it's the way to go.


As I mentioned I prefer GC based solutions, but then again I live in the
JVM/.NET world, so I don't have the memory/timing pressure you have to 
deal with.

But when looking at systems programming languages that offer reference 
counting as the main memory management, ATS, Parasail, Objective-C, and 
now Rust, all of them share a common feature:

- Compiler support to remove extra increment/decrement operations.

I guess on D's case this support would also be required.

--
Paulo
Jun 03 2013
next sibling parent Manu <turkeyman gmail.com> writes:
On 4 June 2013 03:41, Paulo Pinto <pjmlp progtools.org> wrote:


 Am 03.06.2013 18:00, schrieb Manu:

  Haha, wow. Indeed, isn't that well timed with respect to recent

 discussions! ;)
 Post-dconf, I gave it some serious thought and I'm kinda convincing
 myself more and more each day that it's the way to go.


 As I mentioned I prefer GC based solutions, but then again I live in the
 JVM/.NET world, so I don't have the memory/timing pressure you have to
 deal with.


I don't expect the GC would go anywhere. It would remain in use in typical
computing environments.
It would just be an option for realtime users/embedded platforms.

But when looking at systems programming languages that offer reference

 counting as the main memory management, ATS, Parasail, Objective-C, and now
 Rust, all of them share a common feature:

 - Compiler support to remove extra increment/decrement operations.

 I guess on D's case this support would also be required.


Yeah, indeed. I have no idea how much work that would be. I'm not holding
my breath that it'll appear any time soon :)
Jun 03 2013
prev sibling parent reply evansl <cppljevans suddenlink.net> writes:
On 06/03/13 12:41, Paulo Pinto wrote:

 Am 03.06.2013 18:00, schrieb Manu:

 Haha, wow. Indeed, isn't that well timed with respect to recent
 discussions! ;)
 Post-dconf, I gave it some serious thought and I'm kinda convincing
 myself more and more each day that it's the way to go.


 
 As I mentioned I prefer GC based solutions, but then again I live in the
 JVM/.NET world, so I don't have the memory/timing pressure you have to
 deal with.
 
 But when looking at systems programming languages that offer reference
 counting as the main memory management, ATS, Parasail, Objective-C, and
 now Rust, all of them share a common feature:
 
 - Compiler support to remove extra increment/decrement operations.
 
 I guess on D's case this support would also be required.
 
 -- 
 Paulo
 


IIUC, a system programming language would use threading at least a fair
amount.  In that case, I'm not sure reference counting is appropriate,
at least according to the following:

.> However, I think you are also talking about RC in multi-threaded
.> (not distributed) systems. Here, the cost of making RC thread-safe
.> (both RC and pointer manipulations must be atomic) can be expensive
.> if done naively. It's worth taking a moment to think about
.> this. Consider a scenario with 3 objects, A, B and C, and two
.> threads, T1 and T2. Suppose that T1 writes a reference to B into a
.> slot in A and T2 writes a reference to C into the same slot. (You
.> can say that such a program is broken but memory managers but cope
.> with broken code). Both the reference count operations and the
.> pointer updates must be synchronised so that the reference counts
.> do reflect the final state of the reference held in A. This is
.> prohibitively expensive if used for general memory management
.> (i.e. rather than just for managing a few special objects).

This is from a private email(which Jones said I could forward to
c.l.c++.m, which, I assume means I can quote here also):

.> Date: Fri, 15 Apr 2005 10:50:50 +0100
.> From: Richard Jones <R.E.Jones kent.ac.uk>

Of course maybe RC has improved since 2005.  I'm no expert :(

-regards,
Larry
Jun 04 2013
parent reply "Jonathan A Dunlap" <jdunlap outlook.com> writes:
Interesting read on the subject of ARC and GC: 
http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

It does seem that ARC would be a preferred strategy for D to 
pursue (even if it's a first pass before resorting to GC sweeps).
Jul 10 2013
parent "Jonathan A Dunlap" <jdunlap outlook.com> writes:
Opps, just saw that this link was already posted here:
http://forum.dlang.org/thread/krhjq8$2r8l$1 digitalmars.com
Jul 10 2013