www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Appending to a dynamic array at specific offset

reply Joseph Bell <josephabell tx.rr.com> writes:
Hi.

I'm working with D arrays, getting my bearings, and was looking at 
treating a ubtye array as a formatted message of sorts.

Consider the first byte is always a message type header.
The second and third bytes are a length indicator.
The fourth and subsequent bytes are the actual data this message conveys.

If I declare a dynamic array:

ubyte[] msg;

I can easily write

msg ~= TYPE; // TYPE is a ubyte value
msg ~= LEN_HI; // upper byte of 16-bit length
msg ~= LEN_LOW; // lower byte of 16-bit length

At this point I can

msg ~= data; // where data is a ubyte[]

But if I want to change the data, I'd like to be able to just append the 
new data at the offset it needs to go at.

msg[3] ~= data; // doesn't work, msg[3] is a ubyte, can't append ubyte[]
&msg[3] ~= data; // Doesn't work, perhaps trying to write C there

This does work
msg.length = 3;
msg ~= data; // Effectively resize the array back to 3 and then append

I don't know how ineffecient the above approach is.

Also the following doesn't work:

msg[3..$] = data; // Array sizes don't match for copy
msg[3..$] ~= data; // Slice is not a modifiable lvalue

So the upshot:  I have a working solution to the problem, but it looks 
unnatural to me.  Am I missing a magic syntactical expression for this?

Thanks much for any replies,

Joe
Budding D Zealot trying to kick C/C++ to the curb
Jan 22 2007
next sibling parent renoX <renosky free.fr> writes:
Joseph Bell Wrote:
[cut]

 This does work
 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append
 
 I don't know how inefficient the above approach is.


Well if the new data is smaller than the old data, then it is an in place
replacement otherwise there is a new allocation and a copy, but this is
unavoidable.

You can check it by printing msg.ptr before and after (I've did the check
before replying)

As for a better notation, sorry but I don't know enough D to answer.
renoX
Jan 23 2007
prev sibling next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Mon, 22 Jan 2007 22:34:51 -0600, Joseph Bell wrote:


 Hi.
 
 I'm working with D arrays, getting my bearings, and was looking at 
 treating a ubtye array as a formatted message of sorts.
 
 Consider the first byte is always a message type header.
 The second and third bytes are a length indicator.
 The fourth and subsequent bytes are the actual data this message conveys.
 
 If I declare a dynamic array:
 
 ubyte[] msg;
 
 I can easily write
 
 msg ~= TYPE; // TYPE is a ubyte value
 msg ~= LEN_HI; // upper byte of 16-bit length
 msg ~= LEN_LOW; // lower byte of 16-bit length
 
 At this point I can
 
 msg ~= data; // where data is a ubyte[]
 
 But if I want to change the data, I'd like to be able to just append the 
 new data at the offset it needs to go at.
 
 msg[3] ~= data; // doesn't work, msg[3] is a ubyte, can't append ubyte[]
 &msg[3] ~= data; // Doesn't work, perhaps trying to write C there


Of course, 'append' does mean 'add to the end of something'. But what you
seem to be saying is that you wish to replace any existing data from offset
3 onwards with new data.



 This does work
 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append
 
 I don't know how ineffecient the above approach is.


Quite efficient and is probably the better way to do it due to its
simplicity.


 Also the following doesn't work:
 
 msg[3..$] = data; // Array sizes don't match for copy
 msg[3..$] ~= data; // Slice is not a modifiable lvalue


You could do this I suppose ...

 msg.length = data.length + 3;
 msg[3..$] = data;

which might be faster if replacing a larger message with a smaller one as
the msg array will not get reallocated.

-- 
Derek Parnell
Jan 23 2007
parent Kevin Bealer <kevinbealer gmail.com> writes:
Derek Parnell wrote:

 On Mon, 22 Jan 2007 22:34:51 -0600, Joseph Bell wrote:


...


I think these two:


 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append




and


  msg.length = data.length + 3;
  msg[3..$] = data;


Are identical in practice, except that the second does not touch the 
length unless it has to change.  But as far as reallocation and other 
criteria, I don't think there is a difference; D knows how big the msg[] 
block is, so it can do the "~=" or "msg.length=..." in the efficient way 
in either case.

I think it also will not always realloc for a larger size, because the I 
think the underlying implementation does "capacity" doubling, something 
like the way C++'s vector is normally written to do.

Kevin
Jan 25 2007
prev sibling parent Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Joseph Bell wrote:

 Hi.
 
 I'm working with D arrays, getting my bearings, and was looking at 
 treating a ubtye array as a formatted message of sorts.
 
 Consider the first byte is always a message type header.
 The second and third bytes are a length indicator.
 The fourth and subsequent bytes are the actual data this message conveys.
 
 If I declare a dynamic array:
 
 ubyte[] msg;
 
 I can easily write
 
 msg ~= TYPE; // TYPE is a ubyte value
 msg ~= LEN_HI; // upper byte of 16-bit length
 msg ~= LEN_LOW; // lower byte of 16-bit length
 
 At this point I can
 
 msg ~= data; // where data is a ubyte[]
 
 But if I want to change the data, I'd like to be able to just append the 
 new data at the offset it needs to go at.
 
 msg[3] ~= data; // doesn't work, msg[3] is a ubyte, can't append ubyte[]
 &msg[3] ~= data; // Doesn't work, perhaps trying to write C there
 
 This does work
 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append
 
 I don't know how ineffecient the above approach is.
 
 Also the following doesn't work:
 
 msg[3..$] = data; // Array sizes don't match for copy
 msg[3..$] ~= data; // Slice is not a modifiable lvalue
 
 So the upshot:  I have a working solution to the problem, but it looks 
 unnatural to me.  Am I missing a magic syntactical expression for this?


(Assuming your new data is the same length as the old, since you're not 
adjusting the length information)

I think you have three basic options. One is the one you've found:
msg.length = 3;
msg ~= data;

This is actually pretty efficient as long as msg isn't a (non-prefix) 
slice of another array. If that's the case, and as long as the new data 
isn't longer than the old data, it just copies over it without reallocating.

Another option is:
msg = msg[0 .. 3] ~ data;

This is short but unfortunately always reallocates.

A third option is:
msg[3 .. $] = data;

This just plain copies the new data over the old. This should be 
slightly more efficient than the first method since that one adjusts the 
length twice, while this one doesn't touch the length at all.

The first method does have the benefit that it also works if the new 
data is of a different length as the old data, while only reallocating 
if necessary.
The second also has the benefit that it works for any data lengths, but 
always reallocates. That can also be a benefit, however, if you want to 
preserve the old array.
Jan 23 2007