www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Suggestion: noInternalPointers property

reply dsimcha <dsimcha yahoo.com> writes:
As I've mentioned previously, D's partially conservative garbage collection is
often a significant limitation for me when working with large amounts of data.
 As a workaround, I'd like to suggest that the GC be given a
noInternalPointers function.  When called with a pointer to the start of a
block, it would tell the GC to only consider pointers to the start of the
block, not internal pointers.  This would prevent false pointers from wreaking
havoc on large data structures.  In exchange, you'd have to be careful that a
reference to the start of the data structure was somewhere on the stack.  Of
course, you could still pass internal around all you wanted as long as you
didn't try to return an internal pointer from the function that allocated the
data structure.

The noInternalPointers property could also be a property of builtin dynamic
arrays.  Appending a large amount of times to a large array breaks the GC in
the current form because, when the array gets reallocated, the old memory
blocks are not necessarily cleaned up due to the false pointer problem.  If an
array has the noInternalPointers property, every time it is reallocated due to
resizing, the pointer to the start of it would be automatically marked with
the noInternalPointers attribute.

Does all this sound feasible/reasonable?
Aug 06 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from dsimcha (dsimcha yahoo.com)'s article

 As I've mentioned previously, D's partially conservative garbage collection is
 often a significant limitation for me when working with large amounts of data.
  As a workaround, I'd like to suggest that the GC be given a
 noInternalPointers function.  When called with a pointer to the start of a
 block, it would tell the GC to only consider pointers to the start of the
 block, not internal pointers.  This would prevent false pointers from wreaking
 havoc on large data structures.  In exchange, you'd have to be careful that a
 reference to the start of the data structure was somewhere on the stack.  Of
 course, you could still pass internal around all you wanted as long as you
 didn't try to return an internal pointer from the function that allocated the
 data structure.
 The noInternalPointers property could also be a property of builtin dynamic
 arrays.  Appending a large amount of times to a large array breaks the GC in
 the current form because, when the array gets reallocated, the old memory
 blocks are not necessarily cleaned up due to the false pointer problem.  If an
 array has the noInternalPointers property, every time it is reallocated due to
 resizing, the pointer to the start of it would be automatically marked with
 the noInternalPointers attribute.
 Does all this sound feasible/reasonable?


It means that slices of an array won't keep the array data alive, but other than
that I think it's reasonable.


Sean
Aug 06 2008
parent reply BCS <ao pathlink.com> writes:
Reply to Sean,


 == Quote from dsimcha (dsimcha yahoo.com)'s article
 

 As I've mentioned previously, D's partially conservative garbage
 collection is
 often a significant limitation for me when working with large amounts
 of data.
 As a workaround, I'd like to suggest that the GC be given a
 noInternalPointers function.  When called with a pointer to the start
 of a
 block, it would tell the GC to only consider pointers to the start of
 the
 block, not internal pointers.  This would prevent false pointers from
 wreaking
 havoc on large data structures.  In exchange, you'd have to be
 careful that a
 reference to the start of the data structure was somewhere on the
 stack.  Of
 course, you could still pass internal around all you wanted as long
 as you
 didn't try to return an internal pointer from the function that
 allocated the
 data structure.
 The noInternalPointers property could also be a property of builtin
 dynamic
 arrays.  Appending a large amount of times to a large array breaks
 the GC in
 the current form because, when the array gets reallocated, the old
 memory
 blocks are not necessarily cleaned up due to the false pointer
 problem.  If an
 array has the noInternalPointers property, every time it is
 reallocated due to
 resizing, the pointer to the start of it would be automatically
 marked with
 the noInternalPointers attribute.
 Does all this sound feasible/reasonable?


 It means that slices of an array won't keep the array data alive, but
 other than that I think it's reasonable.
 
 Sean
 


IIRC there is a contains pointers flag in the GC (and that's all I remember)
Aug 06 2008
parent dsimcha <dsimcha yahoo.com> writes:
Yeah, I was actually just looking at that today.  That flag specifies whether
the
GC-allocated block could contain pointers to *other* GC-allocated blocks, not
whether there could be pointers, i.e. from the stack, pointing internally *into*
the block.
Aug 06 2008