www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Partial arrays reclaimed?

reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
Suppose an array is being used like a FIFO:

-----------------------
T[] slice;

// Add:
slice ~= T();

// Remove:
slice = slice[1..$];
-----------------------

Assuming of course there's no other references to the memory, as this 
gets used, does the any of the memory from the removed elements ever get 
GC'd?

Also, if this is a long-running process, isn't there a potential danger 
in the array just marching through the address space and running out of 
room? (ie either running out of of continuous space, or hitting 0xFFF....)
Jan 27 2017
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 27 January 2017 at 23:22:17 UTC, Nick Sabalausky wrote:

 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, 
 as this gets used, does the any of the memory from the removed 
 elements ever get GC'd?

 Also, if this is a long-running process, isn't there a 
 potential danger in the array just marching through the address 
 space and running out of room? (ie either running out of of 
 continuous space, or hitting 0xFFF....)


It should reclaim the memory from the beginning of the array.
But your could just have written a small tool that does this and 
look at the memory use over time.
Generally a good idea, to perform experiments with things you are 
interested in.

And there is always the risk of running out of memory on machines 
which have no mmu.
Jan 27 2017
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 27 January 2017 at 23:36:58 UTC, Stefan Koch wrote:

 It should reclaim the memory from the beginning of the array.


I doubt the current implementation will though, you should check.
Jan 27 2017
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, January 27, 2017 18:22:17 Nick Sabalausky via Digitalmars-d-learn 
wrote:

 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, as this
 gets used, does the any of the memory from the removed elements ever get
 GC'd?

 Also, if this is a long-running process, isn't there a potential danger
 in the array just marching through the address space and running out of
 room? (ie either running out of of continuous space, or hitting 0xFFF....)


Given that a dynamic array refers to a block of memory, whether that's the
whole block of memory or just a portion of it, and that as you shrink the
array, it's just moving where the pointer refers to in the block, there's no
way that it can release the memory. A reallocation would have to take place
so that the elements would be copied to a new block of memory and the
dynamic array would then refer to the new block. And the only times that a
reallocation is going to take place are when you append to the array or when
you call reserve, and the array doesn't have enough capacity to grow in
place to fulfill that reserve.

So, if all you're doing is shrinking the array, then there's no way that any
of its memory is going to be freed. You'd pretty much have to dup it at some
point and assign the new array back to it if you want to make that possible.

- Jonathan M Davis
Jan 29 2017
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On Friday, 27 January 2017 at 23:22:17 UTC, Nick Sabalausky wrote:

 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, 
 as this gets used, does the any of the memory from the removed 
 elements ever get GC'd?


If the array has no additional capacity, then appending will 
relocate the data. I.e., copy it to a larger allocation. The old 
data can then be collected. Since the old first element is not 
part of the new array, it's doesn't get copied over. So the 
allocation doesn't grow indefinitely.


 Also, if this is a long-running process, isn't there a 
 potential danger in the array just marching through the address 
 space and running out of room? (ie either running out of of 
 continuous space, or hitting 0xFFF....)


If you append and pop the front over and over, the program should 
reuse old locations, cycling through them.
Jan 29 2017
prev sibling parent Ivan Kazmenko <gassa mail.ru> writes:
On Friday, 27 January 2017 at 23:22:17 UTC, Nick Sabalausky wrote:

 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, 
 as this gets used, does the any of the memory from the removed 
 elements ever get GC'd?


As I see it, the line
slice = slice[1..$];
effectively ends slice's in-place appending capabilities.  So 
each append after remove will likely reallocate.  You have to use 
assumeSafeAppend to re-enable appending in place.

Here [1] is an old thread about the caveats of using built-in 
arrays as queues and stacks.  If not in a hurry, the better 
option is perhaps to just write the respective wrapper structs 
which explicitly store indices, instead of using built-in slices 
and assumeSafeAppend all over the place.

Ivan Kazmenko.

[1] 
http://forum.dlang.org/post/yrxspdrpusrrijmfyldc forum.dlang.org
Jan 31 2017