www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - length = 0 clears reserve

reply Jethro <qyzz gr.ff> writes:
arrays have the ability to reserve but when setting the length to 
0, it removes the reserve!! ;/

char[] buf;
buf.reserve = 1000;
buf.length = 0;
assert(buf.capacity == 0);

But I simply want to clear the buffer, not change it's 
reserve/capacity.

I've tried to hack by setting the length to 0 through a pointer, 
but that still clears the capacity!

I want to do this because I want to be able to reuse the array 
without ever reallocating(I'll set the capacity to the max that 
will ever be used, I don't have to worry about conflicts since it 
will always be ran serially).

It is a bug if dmd is setting the capacity to 0 and reallocating 
the array when the length is set to 0. It should either do it 
lazily or not at all(or, say setting the length to -1 does not 
clear the capacity, only sets the length to 0).
Apr 10 2017
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, April 11, 2017 01:42:32 Jethro via Digitalmars-d-learn wrote:

 arrays have the ability to reserve but when setting the length to
 0, it removes the reserve!! ;/

 char[] buf;
 buf.reserve = 1000;
 buf.length = 0;
 assert(buf.capacity == 0);

 But I simply want to clear the buffer, not change it's
 reserve/capacity.

 I've tried to hack by setting the length to 0 through a pointer,
 but that still clears the capacity!

 I want to do this because I want to be able to reuse the array
 without ever reallocating(I'll set the capacity to the max that
 will ever be used, I don't have to worry about conflicts since it
 will always be ran serially).

 It is a bug if dmd is setting the capacity to 0 and reallocating
 the array when the length is set to 0. It should either do it
 lazily or not at all(or, say setting the length to -1 does not
 clear the capacity, only sets the length to 0).


You can't reuse the memory of a dynamic array by simply setting its length
to 0. If that were allowed, it would risk allow dynamic arrays to stomp on
each others memory (since there is no guarantee that there are no other
dynamic arrays referring to the same memory). However, if you know that
there are no other dynamic arrays referrin to the same memory, then you can
call assumeSafeAppend on the dynamic array, and then the runtime will assume
that there are no other dynamic arrays referring to the same memory.

If you haven't already, I would advise reading

http://dlang.org/d-array-article.html

It uses the wrong terminology, but otherwise, it's an excellent, informative
article. The article refers to T[] as a slice, and the GC-allocated block of
memory as a dynamic array. However, the official terminology is that T[] is
a dynamic array which refers to a block of memory which may or may not be
GC-allocated, and there is no special term for a GC-allocated block of
memory. T[] _is_ also a slice of whatever memory it refers to, but it's the
dynamic array, not the block of memory. And in fact, all of the dynamic
array operations work perfectly fine regardless of whether the block of
memory is GC-allocated or not (it's just that if it's not, the capacity is
0, so appending will always result in a reallocation). In any case, while
the terminology in the article isn't quite right, it gives a good overview
of how dynamic arrays work in D, and it explains what's going with the
capacity of the array and why setting the length of a dynamic array to 0
results in the capacity being 0.

- Jonathan M Davis
Apr 10 2017
parent reply Jon Degenhardt <jond noreply.com> writes:
On Tuesday, 11 April 2017 at 01:59:57 UTC, Jonathan M Davis wrote:

 On Tuesday, April 11, 2017 01:42:32 Jethro via 
 Digitalmars-d-learn wrote:

 arrays have the ability to reserve but when setting the length 
 to 0, it removes the reserve!! ;/

 char[] buf;
 buf.reserve = 1000;
 buf.length = 0;
 assert(buf.capacity == 0);

 But I simply want to clear the buffer, not change it's 
 reserve/capacity.

 I've tried to hack by setting the length to 0 through a 
 pointer, but that still clears the capacity!

 I want to do this because I want to be able to reuse the array 
 without ever reallocating(I'll set the capacity to the max 
 that will ever be used, I don't have to worry about conflicts 
 since it will always be ran serially).

 [snip]


 You can't reuse the memory of a dynamic array by simply setting 
 its length to 0. If that were allowed, it would risk allow 
 dynamic arrays to stomp on each others memory (since there is 
 no guarantee that there are no other dynamic arrays referring 
 to the same memory). However, if you know that there are no 
 other dynamic arrays referrin to the same memory, then you can 
 call assumeSafeAppend on the dynamic array, and then the 
 runtime will assume that there are no other dynamic arrays 
 referring to the same memory.

 [snip]


Another technique that works for many cases is to use an Appender 
(std.array). Appender supports reserve and clear, the latter 
setting the length to zero without reallocating. A typical use 
case is an algorithm doing a series of appends, then setting the 
length to zero and starts appending again.

--Jon
Apr 10 2017
parent reply Jethro <qyzz gr.ff> writes:
On Tuesday, 11 April 2017 at 03:00:29 UTC, Jon Degenhardt wrote:

 On Tuesday, 11 April 2017 at 01:59:57 UTC, Jonathan M Davis 
 wrote:

 On Tuesday, April 11, 2017 01:42:32 Jethro via 
 Digitalmars-d-learn wrote:

 [...]


 You can't reuse the memory of a dynamic array by simply 
 setting its length to 0. If that were allowed, it would risk 
 allow dynamic arrays to stomp on each others memory (since 
 there is no guarantee that there are no other dynamic arrays 
 referring to the same memory). However, if you know that there 
 are no other dynamic arrays referrin to the same memory, then 
 you can call assumeSafeAppend on the dynamic array, and then 
 the runtime will assume that there are no other dynamic arrays 
 referring to the same memory.

 [snip]


 Another technique that works for many cases is to use an 
 Appender (std.array). Appender supports reserve and clear, the 
 latter setting the length to zero without reallocating. A 
 typical use case is an algorithm doing a series of appends, 
 then setting the length to zero and starts appending again.

 --Jon


Appender reports clear? Are you sure?

Seems appender is no different than string, maybe worse? string 
as assumeSafeAppend, reserve and clear(although clear necessarily 
reallocates. They should have a function called empty, which 
resets the length to zero but doesn't reallocate.
Apr 11 2017
parent Jon Degenhardt <jond noreply.com> writes:
On Tuesday, 11 April 2017 at 20:00:48 UTC, Jethro wrote:

 On Tuesday, 11 April 2017 at 03:00:29 UTC, Jon Degenhardt wrote:

 On Tuesday, 11 April 2017 at 01:59:57 UTC, Jonathan M Davis 
 wrote:

 On Tuesday, April 11, 2017 01:42:32 Jethro via 
 Digitalmars-d-learn wrote:

 [...]


 You can't reuse the memory of a dynamic array by simply 
 setting its length to 0. If that were allowed, it would risk 
 allow dynamic arrays to stomp on each others memory (since 
 there is no guarantee that there are no other dynamic arrays 
 referring to the same memory). However, if you know that 
 there are no other dynamic arrays referrin to the same 
 memory, then you can call assumeSafeAppend on the dynamic 
 array, and then the runtime will assume that there are no 
 other dynamic arrays referring to the same memory.

 [snip]


 Another technique that works for many cases is to use an 
 Appender (std.array). Appender supports reserve and clear, the 
 latter setting the length to zero without reallocating. A 
 typical use case is an algorithm doing a series of appends, 
 then setting the length to zero and starts appending again.

 --Jon


 Appender reports clear? Are you sure?

 Seems appender is no different than string, maybe worse? string 
 as assumeSafeAppend, reserve and clear(although clear 
 necessarily reallocates. They should have a function called 
 empty, which resets the length to zero but doesn't reallocate.


See the Appender.clear documentation 
(https://dlang.org/phobos/std_array.html#.Appender.clear), the 
key piece being:

     Clears the managed array. This allows the elements of the 
array to be reused for appending.

I've tried using both dynamic array and appender in this way, 
setting the length of the dynamic array to zero vs using 
Appender.clear, in this cycle of fill-the-array by appending, 
operate on the array, clearing, and repeating. Appender is 
dramatically faster. And, if you look at GC reports you find that 
setting a dynamic array to zero creates garbage to collect, while 
Appender.clear does not. (Use the --DRT-gcopt=profile:1 command 
line option to see GC reports, described here: 
https://dlang.org/spec/garbage.html#gc_config).
Apr 11 2017