www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Mem Mgmt: With & Without the GC

reply Zane <zane.sims gmail.com> writes:
Hey all,

My knowledge of GCs is limited, but my faith in them (for most 
applications) has greatly increased with advances (like with 
Golang's recent updates). I am now trying to get a better sense 
for the direction D is going regarding memory management, both in 
relation to the GC and without it. Excuse my ignorance, but can 
someone clarify the following for me?

1) If using the GC, but for whatever reason, I need to free 
something _right now_, is core.GC.free() the proper way to do 
this?

2) Does calling object.destroy() mean that the object is marked 
for future collection? If not, how can I ensure it is properly 
marked.

3) How can I mark a non-object for future collection? For 
example, a dynamic array: int[] a = new int[10];. Do I just set 
any references to null?

4) What if I want to immediately deallocate a dynamically 
allocated non-object (such as the dynamic array) - Can I use 
core.GC.free?

5) Is there a way to do simple heap allocation with 'new' while 
ensuring the GC doesn't deallocate until I want it to?

6) If the GC is off, how is allocation/deallocation handled? Can 
I still use new for example (and how do I dealloc)?

Thank you for any help you can provide.
Aug 21 2016
next sibling parent reply Cauterite <cauterite gmail.com> writes:
On Sunday, 21 August 2016 at 16:14:53 UTC, Zane wrote:

 5) Is there a way to do simple heap allocation with 'new' while 
 ensuring the GC doesn't deallocate until I want it to?


I can answer this at least,
If you don't want the GC to ever collect the object itself, 
here's the best way:
Allocate the object with a non-GC allocator (such as std.c.malloc 
), then use `emplace` to construct the object in that memory (
Aug 21 2016
parent reply Zane <zane.sims gmail.com> writes:
On Sunday, 21 August 2016 at 16:38:09 UTC, Cauterite wrote:

 On Sunday, 21 August 2016 at 16:14:53 UTC, Zane wrote:

 5) Is there a way to do simple heap allocation with 'new' 
 while ensuring the GC doesn't deallocate until I want it to?


 I can answer this at least,
 If you don't want the GC to ever collect the object itself, 
 here's the best way:
 Allocate the object with a non-GC allocator (such as 
 std.c.malloc ), then use `emplace` to construct the object in 



I see - That makes sense, but is there no way to "pause/stop" the 
GC, but still be able to use the 'new' syntax? I like how clear 
the syntax is, I just don't _always_ want the GC. I'm thinking of 
something like:

core.memory.GC.disable();
Obj o = new Obj();
/* ...use o... */
core.memory.GC.free(o); // or whatever

if possible, is there a drawback to this (other than no automatic 
GC) compared to say, malloc?

Regarding the marking, I guess my question is: what must be done 
to ensure something allocated with 'new' will be a candidate for 
auto-collection later (when GC is enabled)?
Aug 21 2016
parent Cauterite <cauterite gmail.com> writes:
On Sunday, 21 August 2016 at 18:31:26 UTC, Zane wrote:

 I see - That makes sense, but is there no way to "pause/stop" 
 the GC, but still be able to use the 'new' syntax?


Oh you can use `new` when the GC is disabled, no problem. All the 
GC's functionality is still available.

But be careful about what I said with `new` not returning the 
base of the allocation. It might not be safe to explicitly 
`free()` memory allocated by `new` if there could be multiple 
objects in the same memory block. I honestly don't know the facts 
about this.

You can always `GC.free()` memory you've allocated yourself with 
`GC.malloc()`, so malloc+emplace is an option. You could define a 
template to give more convenient syntax.

Also I think you can overload the `new` operator. I've never 
tried it.


 Regarding the marking, I guess my question is: what must be 
 done to ensure something allocated with 'new' will be a 
 candidate for auto-collection later (when GC is enabled)?


I don't think it's possible with a conservative garbage 
collector, because anything that looks like a pointer to your 
object can prevent it from being collected.

However, if there are no actual live pointers to it, the chances 
that it will be collected are very high, especially on 64-bit 
systems.

So for now, your best bet is to make sure your object is not 
accessible (set all live pointers to it to null). It will only 
stay in memory if you're very unlucky.

Once we have a precise garbage collector (should be soon) you can 
be sure an object will get collected if it is not accessible from 
any live pointers.

--------------------------------------

By the way, when I say "live pointer", I mean a pointer which is 
accessible (through any number of indirections) from the memory 
roots.

e.g. If you have a linked list on the heap, with each node 
pointing to the next, but no other pointers to any of the nodes 
(e.g. from the stack) those pointers are not live. The list as a 
whole is not accessible.
Aug 21 2016
prev sibling next sibling parent Cauterite <cauterite gmail.com> writes:
On Sunday, 21 August 2016 at 16:14:53 UTC, Zane wrote:

 6) If the GC is off, how is allocation/deallocation handled? 
 Can I still use new for example (and how do I dealloc)?


All the allocation/deallocation functionality is the same as 
normal, except the GC won't start a collection cycle unless it's 
out of memory.

You can still use `free()` to deallocate, and it will free memory 
straight away.
Aug 21 2016
prev sibling next sibling parent Cauterite <cauterite gmail.com> writes:
On Sunday, 21 August 2016 at 16:14:53 UTC, Zane wrote:

 2) Does calling object.destroy() mean that the object is marked 
 for future collection? If not, how can I ensure it is properly 
 marked.


Because the GC is not of the incremental type, it can't perform 
any marking outside of a stop-the-world mark/sweep cycle. 
Instead, what `destroy()` does is finalise an object: that is, 
runs any destructors and puts it in an invalid state — in 
particular, all pointers contained within the object are 
nullified, so it doesn't reference any other objects.

When I say 'object' I mean anything; class instance, structure, 
array, primitive, whatever.
Aug 21 2016
prev sibling next sibling parent Cauterite <cauterite gmail.com> writes:
On Sunday, 21 August 2016 at 16:14:53 UTC, Zane wrote:

 5) Is there a way to do simple heap allocation with 'new' while 
 ensuring the GC doesn't deallocate until I want it to?


While my earlier suggestion of using malloc/emplace is one 
option, another is to use `GC.addRoot(objPtr)`. It ensures the 
object is never deallocated until you call 
`GC.removeRoot(objPtr)`.
Aug 21 2016
prev sibling next sibling parent Cauterite <cauterite gmail.com> writes:
On Sunday, 21 August 2016 at 16:14:53 UTC, Zane wrote:

 1) If using the GC, but for whatever reason, I need to free 
 something _right now_, is core.GC.free() the proper way to do 
 this?


The main problem is that `new` does not necessarily give you a 
pointer to the start of an allocation, and `GC.free()` does not 
work if you give it a pointer to the interior of an allocated 
block.

You could use `GC.addrOf()` to get the base address from an 
interior pointer, but I don't know whether there could be other 
objects/arrays sharing the same memory block.

If you explicitly allocated the memory block yourself with 
`GC.malloc()` then you have full control over what is placed in 
it and can safely `GC.free()` the memory using the base address.

Keep in mind, `GC.free()` does not call finalisers.
Aug 21 2016
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 21 August 2016 at 16:14:53 UTC, Zane wrote:

 Hey all,

 My knowledge of GCs is limited, but my faith in them (for most 
 applications) has greatly increased with advances (like with 
 Golang's recent updates). I am now trying to get a better sense 
 for the direction D is going regarding memory management, both 
 in relation to the GC and without it. Excuse my ignorance, but 
 can someone clarify the following for me?

 1) If using the GC, but for whatever reason, I need to free 
 something _right now_, is core.GC.free() the proper way to do 
 this?


If you need to destroy a class object right now, the proper way 
is to call .destroy on it (or use something that uses it like 
Scoped!T, Unique!T, Refcounted!T...).

If you need to destroy a struct object right now, the proper way 
is to end the scope.

If you need to free memory right now, better not allocate it from 
the GC, but instead with malloc/free. The GC does not guarantee 
to collect freed memory when you want.



 2) Does calling object.destroy() mean that the object is marked 
 for future collection? If not, how can I ensure it is properly 
 marked.


Calling .destoy on an object calls its destructor but does not 
mark it for collection.

Objects are collected when they are not reachable anymore.
At this point, the destructor is called if it wasn't already (and 
then the object is overwritten with .init).

Actually you shouldn't have to worry when an object memory is 
collected.



 3) How can I mark a non-object for future collection? For 
 example, a dynamic array: int[] a = new int[10];. Do I just set 
 any references to null?


Yes, it is marked for future collection when it's not reachable 
anymore.

Then again, the GC might not reclaim memory right now because 
it's imprecise and you could have "false pointers".



 4) What if I want to immediately deallocate a dynamically 
 allocated non-object (such as the dynamic array) - Can I use 
 core.GC.free?


I don't know if this will deallocate with 100% certainty.


 5) Is there a way to do simple heap allocation with 'new' while 
 ensuring the GC doesn't deallocate until I want it to?


Keep a reference somewhere reachable (by default: stack, heap, 
globals: everything is reachable and scanned).



 6) If the GC is off, how is allocation/deallocation handled?


If you have called GC.disable, no GC collect is performed on any 
allocation.
This is only suitable if you don't have a long-running software 
that would risk running out of memory.
This is known to speed-up short processes.


 Can I still use new for example


You can still use new.


 (and how do I dealloc)?


I don't know then if you can deallocate.
Aug 21 2016
next sibling parent Zane <zane.sims gmail.com> writes:
Thanks to both of you. I think that answers everything, except if 
new'd objects can be freed when GC is disabled. I guess I could 
always enable and call collect().
Aug 21 2016
prev sibling parent Mike Parker <aldacron gmail.com> writes:
On Sunday, 21 August 2016 at 19:30:51 UTC, Guillaume Piolat wrote:


 5) Is there a way to do simple heap allocation with 'new' 
 while ensuring the GC doesn't deallocate until I want it to?


 Keep a reference somewhere reachable (by default: stack, heap, 
 globals: everything is reachable and scanned).


Or just tell the GC directly not to collect it via GC.addRoot:

https://dlang.org/phobos/core_memory.html#.GC.addRoot
Aug 21 2016