digitalmars.D.learn - custom memory management
- "Simon =?UTF-8?B?QsO8cmdlciI=?= <simon.buerger rwth-aachen.de> (39/39) Feb 27 2014 I am trying to implement a structure with value semantics which
- Namespace (7/7) Feb 27 2014 A struct is a value type. So it is passed by value and is placed
- "Simon =?UTF-8?B?QsO8cmdlciI=?= <simon.buerger rwth-aachen.de> (12/19) Feb 27 2014 On the stack yes. But not on the heap:
- Steven Schveighoffer (16/20) Feb 27 2014 =
- "Simon =?UTF-8?B?QsO8cmdlciI=?= <simon.buerger rwth-aachen.de> (7/11) Feb 27 2014 I don't think it does, so I actually implemented it myself (not
- Bienlein (3/3) Feb 28 2014 I asked something similar some days ago. Maybe this provides some
- Dicebot (8/11) Feb 28 2014 I think you misiterpret the spec. If object is collected,
- "Simon =?UTF-8?B?QsO8cmdlciI=?= <simon.buerger rwth-aachen.de> (10/22) Feb 28 2014 If you are right that would mean that the current dmd/runtime
- Dicebot (14/23) Feb 28 2014 Ah, structs are a bit tricky. Spec has override for struct
- Namespace (3/20) Feb 28 2014 What can still take a long time. It annoys me very much that with
I am trying to implement a structure with value semantics which
uses an internal buffer. The first approach looks like this:
struct S
{
byte[] buf;
this(int size) { buf = new byte[size]; }
this(this) { buf = buf.dup; }
~this(this) { delete buf; }
}
This works fine as long as such an object is allocated on the
stack (so the destructor is called at the end of the scope).
However when the destructor is called by the gc, the buffer might
already be collected, and freeing it a second time is obviously
invalid.
My second approach was to allocate the buffer outside the
gc-managed heap, like so:
this(int size) {
buf = (cast(byte*)core.stdc.stdlib.malloc(size))[0..size];
}
~this(this) {
core.stdc.stdlib.free(buf);
}
Sadly, this is incorrect as well. Because if such an object is
collected by the gc, but the gc decides not to run the
destructor, the buffer will never be free'd.
If the gc would either always or never call struct-destructors,
one of my two solutions would work. But the current situation is
(in compliance with the language spec), that it is called
_sometimes_, which breaks both solutions.
One way the first approach could work would be for the destructor
to check wether it was called by the gc, and skip the
deallocation in that case. But as far as I know, the gc does not
provide such a method. It would be trivial to implement, but
seems kinda hackish.
I know the suggested way in D is to not deallocate the buffer at
all, but rely on the gc to collect it eventually. But it still
puzzles me that it seems to be impossible to do. Anybody have an
idea how I could make it work?
thanks, simon
Feb 27 2014
A struct is a value type. So it is passed by value and is placed
on the stack.
{
S s;
}
S DTor is called at the end of the scope. So you can rely on RAII
as long as you use structs.
Feb 27 2014
"Simon =?UTF-8?B?QsO8cmdlciI=?= <simon.buerger rwth-aachen.de>
On Thursday, 27 February 2014 at 22:04:50 UTC, Namespace wrote:
A struct is a value type. So it is passed by value and is
placed on the stack.
{
S s;
}
S DTor is called at the end of the scope. So you can rely on
RAII as long as you use structs.
On the stack yes. But not on the heap:
S[] s = new S[17];
s = null;
the GC will collect the memory eventually, but without calling
any destructor.
On the other hand:
class C { S s; }
C c = new c;
c = null;
in this case, when the gc collects the memory, it will call both
destrcutors. The one of C as well as of S.
Feb 27 2014
On Thu, 27 Feb 2014 16:46:15 -0500, Simon B=C3=BCrger = <simon.buerger rwth-aachen.de> wrote:I know the suggested way in D is to not deallocate the buffer at all, ==but rely on the gc to collect it eventually. But it still puzzles me =that it seems to be impossible to do. Anybody have an idea how I could==make it work?Unfortunately, nothing is foolproof. The most "correct" solution is like= ly = to use malloc/free. Yes, if you just new one of these, you will have to = = destroy it. But if you have a destructor that uses GC allocated memory such an objec= t = can NEVER be a member of a heap-allocated class. More and more, I think a thread-local flag of "I'm in the GC collection = = cycle" would be hugely advantageous -- if it doesn't already exist... -Steve
Feb 27 2014
On Thursday, 27 February 2014 at 22:15:41 UTC, Steven Schveighoffer wrote:On Thu, 27 Feb 2014 16:46:15 -0500, Simon Bürger [...] More and more, I think a thread-local flag of "I'm in the GC collection cycle" would be hugely advantageous -- if it doesn't already exist...I don't think it does, so I actually implemented it myself (not thread-local, but same locking as the rest of the gc): github.com/Krox/druntime/commit/38b718f1dcf08ab8dabb6eed10ff1073e215890f . But now that you mention it, a thread-local flag might be better.
Feb 27 2014
I asked something similar some days ago. Maybe this provides some information tat is helpful to you: http://forum.dlang.org/thread/mekdjoyejtfpafpcdvrg forum.dlang.org
Feb 28 2014
On Thursday, 27 February 2014 at 21:46:17 UTC, Simon Bürger wrote:Sadly, this is incorrect as well. Because if such an object is collected by the gc, but the gc decides not to run the destructor, the buffer will never be free'd.I think you misiterpret the spec. If object is collected, destructor is guaranteed to run. But not all objects are guaranteed to be collected. For example, no collection happens at program termination. So it is OK to release resources in destructor that will be reclaimed by OS at program termination anyway. List of such resources is OS-specific but heap memory tends to be there.
Feb 28 2014
On Friday, 28 February 2014 at 10:40:17 UTC, Dicebot wrote:On Thursday, 27 February 2014 at 21:46:17 UTC, Simon Bürger wrote:If you are right that would mean that the current dmd/runtime does not follow the spec. Curious. The current implementation is not aware of strcut-destructors on the heap, i.e. the GC.BlkAttr.FINALIZE flag is not set for structs (or arrays of structs). In the struct-inside-a-class example, the struct destructor is called by the (automatically generated) class-destructor. The gc only knows about class-destrcutor and calls only that one directly.Sadly, this is incorrect as well. Because if such an object is collected by the gc, but the gc decides not to run the destructor, the buffer will never be free'd.I think you misiterpret the spec. If object is collected, destructor is guaranteed to run. But not all objects are guaranteed to be collected. For example, no collection happens at program termination. So it is OK to release resources in destructor that will be reclaimed by OS at program termination anyway. List of such resources is OS-specific but heap memory tends to be there.
Feb 28 2014
On Friday, 28 February 2014 at 12:36:48 UTC, Simon Bürger wrote:If you are right that would mean that the current dmd/runtime does not follow the spec. Curious. The current implementation is not aware of strcut-destructors on the heap, i.e. the GC.BlkAttr.FINALIZE flag is not set for structs (or arrays of structs).Ah, structs are a bit tricky. Spec has override for struct destructors that says explicitly "Destructors are called when an object goes out of scope" so one can argue heap ignorance matches it. But the very next line contradicts it : "Their purpose is to free up resources owned by the struct object". I believe it is a DMD bug though. There is no reason why destructors can't be run here. Most likely it is just defect of current GC implementation that everyone got used to. This bug report discussion confirms it : https://d.puremagic.com/issues/show_bug.cgi?id=2834 Looks like decision was to fix it once precise GC gets added.In the struct-inside-a-class example, the struct destructor is called by the (automatically generated) class-destructor. The gc only knows about class-destructor and calls only that one directly.Yes, that matches my current observations. Did not occure before because of C coding habits :) Lucky me.
Feb 28 2014
On Friday, 28 February 2014 at 12:54:32 UTC, Dicebot wrote:On Friday, 28 February 2014 at 12:36:48 UTC, Simon Bürger wrote:What can still take a long time. It annoys me very much that with arrays I can not rely on that the struct DTors are called.If you are right that would mean that the current dmd/runtime does not follow the spec. Curious. The current implementation is not aware of strcut-destructors on the heap, i.e. the GC.BlkAttr.FINALIZE flag is not set for structs (or arrays of structs).Ah, structs are a bit tricky. Spec has override for struct destructors that says explicitly "Destructors are called when an object goes out of scope" so one can argue heap ignorance matches it. But the very next line contradicts it : "Their purpose is to free up resources owned by the struct object". I believe it is a DMD bug though. There is no reason why destructors can't be run here. Most likely it is just defect of current GC implementation that everyone got used to. This bug report discussion confirms it : https://d.puremagic.com/issues/show_bug.cgi?id=2834 Looks like decision was to fix it once precise GC gets added.
Feb 28 2014
On Friday, 28 February 2014 at 13:06:05 UTC, Namespace wrote:What can still take a long time. It annoys me very much that with arrays I can not rely on that the struct DTors are called.Yep, this bug has immediately got my vote :) It does require someone knowledgable of GC to fix in forseeable future unfortunately.
Feb 28 2014
On Friday, 28 February 2014 at 13:16:40 UTC, Dicebot wrote:On Friday, 28 February 2014 at 13:06:05 UTC, Namespace wrote:I will vote, too. It's somewhat strange: Since it works with delete it should also work with the current GC, or? Someone should figure out why and how delete works this way. :)What can still take a long time. It annoys me very much that with arrays I can not rely on that the struct DTors are called.Yep, this bug has immediately got my vote :) It does require someone knowledgable of GC to fix in forseeable future unfortunately.
Feb 28 2014
On Friday, 28 February 2014 at 13:32:33 UTC, Namespace wrote:I will vote, too. It's somewhat strange: Since it works with delete it should also work with the current GC, or? Someone should figure out why and how delete works this way. :)Well, delete is deprecated so it can do any kind of arcane horrors :) More idiomatic destroy + GC.free pair will work because destroy is a template function.
Feb 28 2014
On Friday, 28 February 2014 at 13:38:59 UTC, Dicebot wrote:On Friday, 28 February 2014 at 13:32:33 UTC, Namespace wrote:No, currently it is not deprecated. It is suggested to be deprecated. :P And destroy doesn't finalize the data. :/ See: http://forum.dlang.org/thread/bug-12256-3 https.d.puremagic.com%2Fissues%2F and http://forum.dlang.org/thread/bug-12274-3 https.d.puremagic.com%2Fissues%2F But that is only a workaround. I don't want to call every time "arr.finalize" because the GC is silly... I meant that someone should analyse the internal delete code and implement something like this for the current GC related to struct arrays (and AA's).I will vote, too. It's somewhat strange: Since it works with delete it should also work with the current GC, or? Someone should figure out why and how delete works this way. :)Well, delete is deprecated so it can do any kind of arcane horrors :) More idiomatic destroy + GC.free pair will work because destroy is a template function.
Feb 28 2014
On Friday, 28 February 2014 at 14:08:11 UTC, Namespace wrote:No, currently it is not deprecated. It is suggested to be deprecated. :P And destroy doesn't finalize the data. :/ See: http://forum.dlang.org/thread/bug-12256-3 https.d.puremagic.com%2Fissues%2F and http://forum.dlang.org/thread/bug-12274-3 https.d.puremagic.com%2Fissues%2F But that is only a workaround. I don't want to call every time "arr.finalize" because the GC is silly..."intended to be deprecated" is a better word. There is not a smallest chance it will stay in the long term, better get used to it. Quick solution would have been to merge "finalize" with destroy itself. As I have mentioned, it is a template and has all necessary information for traversal. Proper solution will be to fix the struct destructor bug as it is the root cause for your array issues too and then patch destroy to only do traversal when pointers are not owned by GC.I meant that someone should analyse the internal delete code and implement something like this for the current GC related to struct arrays (and AA's).I am too scared of what I may find :)
Feb 28 2014
On Friday, 28 February 2014 at 14:47:31 UTC, Dicebot wrote:On Friday, 28 February 2014 at 14:08:11 UTC, Namespace wrote:I'm not sure if that is possible with the current gc. But I hope it!No, currently it is not deprecated. It is suggested to be deprecated. :P And destroy doesn't finalize the data. :/ See: http://forum.dlang.org/thread/bug-12256-3 https.d.puremagic.com%2Fissues%2F and http://forum.dlang.org/thread/bug-12274-3 https.d.puremagic.com%2Fissues%2F But that is only a workaround. I don't want to call every time "arr.finalize" because the GC is silly..."intended to be deprecated" is a better word. There is not a smallest chance it will stay in the long term, better get used to it. Quick solution would have been to merge "finalize" with destroy itself. As I have mentioned, it is a template and has all necessary information for traversal. Proper solution will be to fix the struct destructor bug as it is the root cause for your array issues too and then patch destroy to only do traversal when pointers are not owned by GC.I meant that someone should analyse the internal delete code and implement something like this for the current GC related to struct arrays (and AA's).I am too scared of what I may find :)
Feb 28 2014