digitalmars.D.learn - ubytes to ulong problem
- Charles Hixson (36/36) Dec 21 2013 I was planning to ask if there were a better way to do this, but instead...
- ponce (2/37) Dec 21 2013
- ponce (2/40) Dec 21 2013 Use the exponentiation operator which is spelled: ^^
- Charles Hixson (5/43) Dec 21 2013 Thanks. I was *sure* it was something stupid on my part. It's 'good'
- John Colvin (38/73) Dec 21 2013 As pointed out before, you're using '^' which is xor, instead of
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (22/24) Dec 21 2013 Assuming that the program needs to support only big endian and little
- John Colvin (3/30) Dec 21 2013 Nevermind equally fast, that will be much faster. -10 brain
- Charles Hixson (6/31) Dec 21 2013 Will that work even when the alignment is to odd bytes? Because that's
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (21/59) Dec 21 2013 No, it is not guaranteed to work unless the alignment is right.
- Charles Hixson (24/85) Dec 22 2013 Nice, but the block is longer than 8 bytes, so I should use a "for (i =
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (4/6) Dec 22 2013 Makes sense. That reminded me of the Phobos function that does exactly
- Charles Hixson (4/10) Dec 22 2013 No, thanks. That's precisely what I was looking for.
- John Colvin (4/51) Dec 22 2013 It's just an unaligned load. If your target cpu architecture
I was planning to ask if there were a better way to do this, but instead
I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array, and an
index of 0, it returns a value of 8, even though all the values in the
array are 0.
The error has to be somewhere in the "ret = " statement, but I sure
don't see it.
/** Convert 8 consecutive bytes sliced from a ubyte[] into a ulong
* param block The array from which to slice.
* param n The starting index within the block */
ulong ubytesToUlong(ubyte[] block, int n)
{ ulong ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret = cast(ulong)block[n] * 2^21
+ cast(ulong)block[n+1] * 2^18
+ cast(ulong)block[n+2] * 2^15
+ cast(ulong)block[n+3] * 2^12
+ cast(ulong)block[n+4] * 2^9
+ cast(ulong)block[n+5] * 2^6
+ cast(ulong)block[n+6] * 2^3
+ cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return ret;
}
--
Charles Hixson
Dec 21 2013
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson
wrote:
I was planning to ask if there were a better way to do this,
but instead I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array,
and an index of 0, it returns a value of 8, even though all the
values in the array are 0.
The error has to be somewhere in the "ret = " statement, but I
sure don't see it.
/** Convert 8 consecutive bytes sliced from a ubyte[] into a
ulong
* param block The array from which to slice.
* param n The starting index within the block */
ulong ubytesToUlong(ubyte[] block, int n)
{ ulong ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret = cast(ulong)block[n] * 2^21
+ cast(ulong)block[n+1] * 2^18
+ cast(ulong)block[n+2] * 2^15
+ cast(ulong)block[n+3] * 2^12
+ cast(ulong)block[n+4] * 2^9
+ cast(ulong)block[n+5] * 2^6
+ cast(ulong)block[n+6] * 2^3
+ cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return ret;
}
Dec 21 2013
On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote:On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson wrote:Use the exponentiation operator which is spelled: ^^I was planning to ask if there were a better way to do this, but instead I need to ask what's my mistake? For some reason, if called with an uninitialized ubyte array, and an index of 0, it returns a value of 8, even though all the values in the array are 0. The error has to be somewhere in the "ret = " statement, but I sure don't see it. /** Convert 8 consecutive bytes sliced from a ubyte[] into a ulong * param block The array from which to slice. * param n The starting index within the block */ ulong ubytesToUlong(ubyte[] block, int n) { ulong ret; assert (n >= 0); assert (n + 8 <= block.length); writefln ("n = %s", n); writefln ("block[0] = %s", cast(ulong)block[0]); writefln ("block[1] = %s", cast(ulong)block[1]); writefln ("block[2] = %s", cast(ulong)block[2]); writefln ("block[3] = %s", cast(ulong)block[3]); writefln ("block[4] = %s", cast(ulong)block[4]); writefln ("block[5] = %s", cast(ulong)block[5]); writefln ("block[6] = %s", cast(ulong)block[6]); writefln ("block[7] = %s", cast(ulong)block[7]); ret = cast(ulong)block[n] * 2^21 + cast(ulong)block[n+1] * 2^18 + cast(ulong)block[n+2] * 2^15 + cast(ulong)block[n+3] * 2^12 + cast(ulong)block[n+4] * 2^9 + cast(ulong)block[n+5] * 2^6 + cast(ulong)block[n+6] * 2^3 + cast(ulong)block[n+7] * 2^0; writefln ("ret = %s", ret); return ret; }
Dec 21 2013
Charles Hixson <charleshixsn earthlink.net> On 12/21/2013 02:31 PM, ponce wrote:On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote:Thanks. I was *sure* it was something stupid on my part. It's 'good' to know that I was right about *that*. -- Charles HixsonOn Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson wrote:Use the exponentiation operator which is spelled: ^^I was planning to ask if there were a better way to do this, but instead I need to ask what's my mistake? For some reason, if called with an uninitialized ubyte array, and an index of 0, it returns a value of 8, even though all the values in the array are 0. The error has to be somewhere in the "ret = " statement, but I sure don't see it. /** Convert 8 consecutive bytes sliced from a ubyte[] into a ulong * param block The array from which to slice. * param n The starting index within the block */ ulong ubytesToUlong(ubyte[] block, int n) { ulong ret; assert (n >= 0); assert (n + 8 <= block.length); writefln ("n = %s", n); writefln ("block[0] = %s", cast(ulong)block[0]); writefln ("block[1] = %s", cast(ulong)block[1]); writefln ("block[2] = %s", cast(ulong)block[2]); writefln ("block[3] = %s", cast(ulong)block[3]); writefln ("block[4] = %s", cast(ulong)block[4]); writefln ("block[5] = %s", cast(ulong)block[5]); writefln ("block[6] = %s", cast(ulong)block[6]); writefln ("block[7] = %s", cast(ulong)block[7]); ret = cast(ulong)block[n] * 2^21 + cast(ulong)block[n+1] * 2^18 + cast(ulong)block[n+2] * 2^15 + cast(ulong)block[n+3] * 2^12 + cast(ulong)block[n+4] * 2^9 + cast(ulong)block[n+5] * 2^6 + cast(ulong)block[n+6] * 2^3 + cast(ulong)block[n+7] * 2^0; writefln ("ret = %s", ret); return ret; }
Dec 21 2013
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson
wrote:
I was planning to ask if there were a better way to do this,
but instead I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array,
and an index of 0, it returns a value of 8, even though all the
values in the array are 0.
The error has to be somewhere in the "ret = " statement, but I
sure don't see it.
/** Convert 8 consecutive bytes sliced from a ubyte[] into a
ulong
* param block The array from which to slice.
* param n The starting index within the block */
ulong ubytesToUlong(ubyte[] block, int n)
{ ulong ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret = cast(ulong)block[n] * 2^21
+ cast(ulong)block[n+1] * 2^18
+ cast(ulong)block[n+2] * 2^15
+ cast(ulong)block[n+3] * 2^12
+ cast(ulong)block[n+4] * 2^9
+ cast(ulong)block[n+5] * 2^6
+ cast(ulong)block[n+6] * 2^3
+ cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return ret;
}
As pointed out before, you're using '^' which is xor, instead of
^^. Ideally the compiler will optimise your version to be fast,
but you may find you get better performance by doing the bit
manipulations eplicitly:
/**Convert 8 consecutive bytes sliced from a ubyte[]
* to a ulong using bigendian byte ordering.
* param block The array from which to slice.
* param n The starting index within the block, default 0*/
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
return ((cast(ulong)block[n ]) << 56)
| ((cast(ulong)block[n+1]) << 48)
| ((cast(ulong)block[n+2]) << 40)
| ((cast(ulong)block[n+3]) << 32)
| ((cast(ulong)block[n+4]) << 24)
| ((cast(ulong)block[n+5]) << 16)
| ((cast(ulong)block[n+6]) << 8)
| (cast(ulong)block[n+7]);
}
unittest
{
ubyte[8] a = [0,0,0,0,0,0,0,0];
assert(a[].ubytesToUlong() == 0);
a[7] = 3;
a[6] = 1;
assert(a[].ubytesToUlong() == 259);
}
Dec 21 2013
On 12/21/2013 03:13 PM, John Colvin wrote:Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly:Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali
Dec 21 2013
On Saturday, 21 December 2013 at 23:52:05 UTC, Ali Çehreli wrote:On 12/21/2013 03:13 PM, John Colvin wrote:Nevermind equally fast, that will be much faster. -10 brain points for me tonight...Ideally the compiler will optimise your version to be fast,but you mayfind you get better performance by doing the bitmanipulations eplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali
Dec 21 2013
On 12/21/2013 03:52 PM, Ali Çehreli wrote:On 12/21/2013 03:13 PM, John Colvin wrote:Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes. -- Charles HixsonIdeally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulationseplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali
Dec 21 2013
On 12/21/2013 05:44 PM, Charles Hixson wrote:On 12/21/2013 03:52 PM, Ali Çehreli wrote:No, it is not guaranteed to work unless the alignment is right. How about this, then: :) import std.array; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = block.front; block.popFront(); foreach (ub; block) { value <<= 8; value |= ub; } return value; } AliOn 12/21/2013 03:13 PM, John Colvin wrote:Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes.Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulationseplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali
Dec 21 2013
On 12/21/2013 07:57 PM, Ali Çehreli wrote:On 12/21/2013 05:44 PM, Charles Hixson wrote:Nice, but the block is longer than 8 bytes, so I should use a "for (i = n; i < n + 8; i++)" rather than a foreach, and index off of i. I clearly need to redo the documentation a bit (even though it's form me of a few months from now). It needs to say something like "Convert a 8 byte slice from a ubyte array starting at index n into a ulong." n should always be required to be specified, so I don't want a default value. (0 was used as a test case, because I'd made a really stupid mistake and used "^" for exponentiation, and then couldn't see what was going on, so I was simplifying everything...and I still couldn't see it. Actually the array starts with a ushort, which specifies the number of ulongs to follow before a bunch of bytes that are unintelligible data to the class that's using this function. (OTOH, it seems like something generally useful, so I'll probably put it in a utils.d file, with some other generally useful routines.) OTOH, if I'm going to consider this to be a general utility function, then I really don't want to make assumptions about where things start, etc. Perhaps I should throw an exception (other than assertion error) if the index is bad or the array is to short for the given index. I need to think about that a bit more. The alternative is to use enforce rather than assertions...though as long as I'm the only user assertions suffice. (It's not going to be separately compiled.) -- Charles HixsonOn 12/21/2013 03:52 PM, Ali Çehreli wrote:No, it is not guaranteed to work unless the alignment is right. How about this, then: :) import std.array; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = block.front; block.popFront(); foreach (ub; block) { value <<= 8; value |= ub; } return value; } AliOn 12/21/2013 03:13 PM, John Colvin wrote:Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes.Ideally the compiler will optimise your version to be fast, butyou mayfind you get better performance by doing the bit manipulationseplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali
Dec 22 2013
On 12/22/2013 01:04 AM, Charles Hixson wrote:Nice, but the block is longer than 8 bytes, so I should use a "for (i = n; i < n + 8; i++)" rather than a foreach, and index off of i.Makes sense. That reminded me of the Phobos function that does exactly what you want. Have you considered std.bitmanip.read? Ali
Dec 22 2013
On 12/22/2013 02:22 AM, Ali Çehreli wrote:On 12/22/2013 01:04 AM, Charles Hixson wrote:No, thanks. That's precisely what I was looking for. -- Charles HixsonNice, but the block is longer than 8 bytes, so I should use a "for (i = n; i < n + 8; i++)" rather than a foreach, and index off of i.Makes sense. That reminded me of the Phobos function that does exactly what you want. Have you considered std.bitmanip.read? Ali
Dec 22 2013
On Sunday, 22 December 2013 at 03:57:38 UTC, Ali Çehreli wrote:On 12/21/2013 05:44 PM, Charles Hixson wrote:It's just an unaligned load. If your target cpu architecture can't do unaligned loads then you're either using something very small or very old.On 12/21/2013 03:52 PM, Ali Çehreli wrote:No, it is not guaranteed to work unless the alignment is right.On 12/21/2013 03:13 PM, John Colvin wrote:Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes.Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulationseplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n >= 0); assert (n + 8 <= block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali
Dec 22 2013