www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Appending to a dynamic array at specific offset

reply Joseph Bell <josephabell tx.rr.com> writes:
Hi.

I'm working with D arrays, getting my bearings, and was looking at 
treating a ubtye array as a formatted message of sorts.

Consider the first byte is always a message type header.
The second and third bytes are a length indicator.
The fourth and subsequent bytes are the actual data this message conveys.

If I declare a dynamic array:

ubyte[] msg;

I can easily write

msg ~= TYPE; // TYPE is a ubyte value
msg ~= LEN_HI; // upper byte of 16-bit length
msg ~= LEN_LOW; // lower byte of 16-bit length

At this point I can

msg ~= data; // where data is a ubyte[]

But if I want to change the data, I'd like to be able to just append the 
new data at the offset it needs to go at.

msg[3] ~= data; // doesn't work, msg[3] is a ubyte, can't append ubyte[]
&msg[3] ~= data; // Doesn't work, perhaps trying to write C there

This does work
msg.length = 3;
msg ~= data; // Effectively resize the array back to 3 and then append

I don't know how ineffecient the above approach is.

Also the following doesn't work:

msg[3..$] = data; // Array sizes don't match for copy
msg[3..$] ~= data; // Slice is not a modifiable lvalue

So the upshot:  I have a working solution to the problem, but it looks 
unnatural to me.  Am I missing a magic syntactical expression for this?

Thanks much for any replies,

Joe
Budding D Zealot trying to kick C/C++ to the curb
Jan 22 2007
next sibling parent renoX <renosky free.fr> writes:
Joseph Bell Wrote:
[cut]
 This does work
 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append
 
 I don't know how inefficient the above approach is.
Well if the new data is smaller than the old data, then it is an in place replacement otherwise there is a new allocation and a copy, but this is unavoidable. You can check it by printing msg.ptr before and after (I've did the check before replying) As for a better notation, sorry but I don't know enough D to answer. renoX
Jan 23 2007
prev sibling next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Mon, 22 Jan 2007 22:34:51 -0600, Joseph Bell wrote:

 Hi.
 
 I'm working with D arrays, getting my bearings, and was looking at 
 treating a ubtye array as a formatted message of sorts.
 
 Consider the first byte is always a message type header.
 The second and third bytes are a length indicator.
 The fourth and subsequent bytes are the actual data this message conveys.
 
 If I declare a dynamic array:
 
 ubyte[] msg;
 
 I can easily write
 
 msg ~= TYPE; // TYPE is a ubyte value
 msg ~= LEN_HI; // upper byte of 16-bit length
 msg ~= LEN_LOW; // lower byte of 16-bit length
 
 At this point I can
 
 msg ~= data; // where data is a ubyte[]
 
 But if I want to change the data, I'd like to be able to just append the 
 new data at the offset it needs to go at.
 
 msg[3] ~= data; // doesn't work, msg[3] is a ubyte, can't append ubyte[]
 &msg[3] ~= data; // Doesn't work, perhaps trying to write C there
Of course, 'append' does mean 'add to the end of something'. But what you seem to be saying is that you wish to replace any existing data from offset 3 onwards with new data.
 This does work
 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append
 
 I don't know how ineffecient the above approach is.
Quite efficient and is probably the better way to do it due to its simplicity.
 Also the following doesn't work:
 
 msg[3..$] = data; // Array sizes don't match for copy
 msg[3..$] ~= data; // Slice is not a modifiable lvalue
You could do this I suppose ... msg.length = data.length + 3; msg[3..$] = data; which might be faster if replacing a larger message with a smaller one as the msg array will not get reallocated. -- Derek Parnell
Jan 23 2007
parent Kevin Bealer <kevinbealer gmail.com> writes:
Derek Parnell wrote:
 On Mon, 22 Jan 2007 22:34:51 -0600, Joseph Bell wrote:
... I think these two:
 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append
and
  msg.length = data.length + 3;
  msg[3..$] = data;
Are identical in practice, except that the second does not touch the length unless it has to change. But as far as reallocation and other criteria, I don't think there is a difference; D knows how big the msg[] block is, so it can do the "~=" or "msg.length=..." in the efficient way in either case. I think it also will not always realloc for a larger size, because the I think the underlying implementation does "capacity" doubling, something like the way C++'s vector is normally written to do. Kevin
Jan 25 2007
prev sibling parent Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Joseph Bell wrote:
 Hi.
 
 I'm working with D arrays, getting my bearings, and was looking at 
 treating a ubtye array as a formatted message of sorts.
 
 Consider the first byte is always a message type header.
 The second and third bytes are a length indicator.
 The fourth and subsequent bytes are the actual data this message conveys.
 
 If I declare a dynamic array:
 
 ubyte[] msg;
 
 I can easily write
 
 msg ~= TYPE; // TYPE is a ubyte value
 msg ~= LEN_HI; // upper byte of 16-bit length
 msg ~= LEN_LOW; // lower byte of 16-bit length
 
 At this point I can
 
 msg ~= data; // where data is a ubyte[]
 
 But if I want to change the data, I'd like to be able to just append the 
 new data at the offset it needs to go at.
 
 msg[3] ~= data; // doesn't work, msg[3] is a ubyte, can't append ubyte[]
 &msg[3] ~= data; // Doesn't work, perhaps trying to write C there
 
 This does work
 msg.length = 3;
 msg ~= data; // Effectively resize the array back to 3 and then append
 
 I don't know how ineffecient the above approach is.
 
 Also the following doesn't work:
 
 msg[3..$] = data; // Array sizes don't match for copy
 msg[3..$] ~= data; // Slice is not a modifiable lvalue
 
 So the upshot:  I have a working solution to the problem, but it looks 
 unnatural to me.  Am I missing a magic syntactical expression for this?
(Assuming your new data is the same length as the old, since you're not adjusting the length information) I think you have three basic options. One is the one you've found: msg.length = 3; msg ~= data; This is actually pretty efficient as long as msg isn't a (non-prefix) slice of another array. If that's the case, and as long as the new data isn't longer than the old data, it just copies over it without reallocating. Another option is: msg = msg[0 .. 3] ~ data; This is short but unfortunately always reallocates. A third option is: msg[3 .. $] = data; This just plain copies the new data over the old. This should be slightly more efficient than the first method since that one adjusts the length twice, while this one doesn't touch the length at all. The first method does have the benefit that it also works if the new data is of a different length as the old data, while only reallocating if necessary. The second also has the benefit that it works for any data lengths, but always reallocates. That can also be a benefit, however, if you want to preserve the old array.
Jan 23 2007