digitalmars.D.learn - 24-bit int
- EntangledQuanta (4/4) Sep 01 2017 Is there a way to create a 24-bit int? One that for all practical
- Biotronic (116/120) Sep 01 2017 I haven't looked at endianness beyond it working on my computer.
- EntangledQuanta (2/9) Sep 01 2017 Thanks, I'll check it out and see.
- Nicholas Wilson (3/21) Sep 01 2017 You may also want to put an align(1) on it so that you dont waste
- EntangledQuanta (9/36) Sep 01 2017 The whole point is so that there is no wasted space, so if it
- Mike Parker (11/18) Sep 01 2017 It's not a bug, but a feature. Data structure alignment is
- EntangledQuanta (5/27) Sep 01 2017 You fail to read correctly. A bug in his code. If he is treating
- kinke (8/17) Sep 02 2017 struct int24 {
- Mike Parker (3/10) Sep 02 2017 Yes, that does make sense. It doesn't make sense that I didn't
- Biotronic (9/21) Sep 02 2017 The very first test in my code checks this:
- Patrick Schluter (5/19) Sep 03 2017 will not work on big endian machine.
- Ilya Yaroshenko (14/18) Sep 01 2017 Hi,
- EntangledQuanta (3/21) Sep 01 2017 Thanks. Seems useful.
- Ilya Yaroshenko (8/35) Sep 02 2017 Just added `bytegroup` topology. Released in v0.6.12 (will be
- EntangledQuanta (9/45) Sep 03 2017 Thanks! I might end up using this. Is this basically just a
- Ilya (6/32) Sep 03 2017 The implementation can be found here
- solidstate1991 (6/10) Sep 02 2017 If you need to perform math on integer audio, I strongly suggest
Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.
Sep 01 2017
On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point: struct int24 { ubyte[3] _payload; this(int x) { value = x; } property int value() { int val = *cast(int*)&_payload & 0xFFFFFF; if (val & 0x800000) { val |= 0xFF000000; } return val; } property int value(int x) { _payload = (cast(ubyte*)&x)[0..3]; return value; } auto opUnary(string op)() { static if (op == "++") { value = value + 1; } else static if (op == "--") { value = value - 1; } else static if (op == "+") { return value; } else static if (op == "-") { return -value; } else static if (op == "~") { return ~value; } else { static assert(false, "Unary operator '"~op~"' is not supported by int24."); } } auto opOpAssign(string op)(int x) { static assert(__traits(compiles, {mixin("value = value "~op~" x;");}), "Binary operator '"~op~"' is not supported by int24."); mixin("value = value "~op~" x;"); return this; } alias value this; } unittest { int24[3] a; assert(a.sizeof == 9); // Max value a[1] = 8388607; assert(a[1] == 8388607); // Test for buffer overflow: assert(a[0] == 0); assert(a[2] == 0); // Overflow a[1] = 8388608; assert(a[1] == -8388608); // Test for buffer overflow: assert(a[0] == 0); assert(a[2] == 0); // negative value a[1] = -1; assert(a[1] == -1); // Test for buffer overflow: assert(a[0] == 0); assert(a[2] == 0); // Unary operators a[1] = 0; assert(~a[1] == -1); a[1]--; assert(a[1] == -1); assert(-a[1] == 1); assert(+a[1] == -1); a[1]++; assert(a[1] == 0); // Binary operators a[1] = 0; a[1] = a[1] + 1; assert(a[1] == 1); a[1] += 1; assert(a[1] == 2); a[1] = a[1] - 1; assert(a[1] == 1); a[1] -= 1; assert(a[1] == 0); a[1] = 3; a[1] = a[1] * 2; assert(a[1] == 6); a[1] = a[1] / 2; assert(a[1] == 3); a[1] *= 2; assert(a[1] == 6); a[1] /= 2; assert(a[1] == 3); a[1] = a[1] << 1; assert(a[1] == 6); a[1] <<= 1; assert(a[1] == 12); a[1] = a[1] >> 1; assert(a[1] == 6); a[1] >>= 1; assert(a[1] == 3); a[1] |= 4; assert(a[1] == 7); a[1] &= 5; assert(a[1] == 5); a[1] = a[1] | 2; assert(a[1] == 7); a[1] = a[1] & 3; assert(a[1] == 3); } -- Biotronic
Sep 01 2017
EntangledQuanta <EQ universe.com> On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:Thanks, I'll check it out and see.[...]I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point: [...]
Sep 01 2017
On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:You may also want to put an align(1) on it so that you dont waste 25% of the allocated memory in an array of int24'sIs there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point: struct int24 { ubyte[3] _payload; this(int x) { value = x; } ... } -- Biotronic
Sep 01 2017
On Saturday, 2 September 2017 at 00:43:00 UTC, Nicholas Wilson wrote:On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:The whole point is so that there is no wasted space, so if it requires that then it's not a waste of space but a bug. Audio that is in24 is 3 bytes per sample, not 4. Every 3 bytes are a sample, not every 3 out of 4. Basically a byte[] cast to a int24 array should be 1/3 the size and every 3 bytes are the same as an int24. Thanks for pointing this out if it is necessary.On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:You may also want to put an align(1) on it so that you dont waste 25% of the allocated memory in an array of int24'sIs there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point: struct int24 { ubyte[3] _payload; this(int x) { value = x; } ... } -- Biotronic
Sep 01 2017
On Saturday, 2 September 2017 at 01:19:52 UTC, EntangledQuanta wrote:The whole point is so that there is no wasted space, so if it requires that then it's not a waste of space but a bug. Audio that is in24 is 3 bytes per sample, not 4. Every 3 bytes are a sample, not every 3 out of 4. Basically a byte[] cast to a int24 array should be 1/3 the size and every 3 bytes are the same as an int24. Thanks for pointing this out if it is necessary.It's not a bug, but a feature. Data structure alignment is important for efficient reads, so several languages (D, C, C++, Ada, and more) will automatically pad structs so that they can maintain specific byte alignments. On a 32-bit system, 4-byte boundaries are the default. So a struct with 3 ubytes is going to be padded with an extra byte at the end. Telling the compiler to align on a 1-byte boundary (essentially disabling alignment) will save you space, but will will generally cost you cycles in accessing the data.
Sep 01 2017
On Saturday, 2 September 2017 at 02:37:08 UTC, Mike Parker wrote:On Saturday, 2 September 2017 at 01:19:52 UTC, EntangledQuanta wrote:You fail to read correctly. A bug in his code. If he is treating in24's as int32's and simply ignoring the upper byte then it is not a true int24 and all the work he did would be pointless. I can do that by simply reading an int32 and masking the high bit.The whole point is so that there is no wasted space, so if it requires that then it's not a waste of space but a bug. Audio that is in24 is 3 bytes per sample, not 4. Every 3 bytes are a sample, not every 3 out of 4. Basically a byte[] cast to a int24 array should be 1/3 the size and every 3 bytes are the same as an int24. Thanks for pointing this out if it is necessary.It's not a bug, but a feature. Data structure alignment is important for efficient reads, so several languages (D, C, C++, Ada, and more) will automatically pad structs so that they can maintain specific byte alignments. On a 32-bit system, 4-byte boundaries are the default. So a struct with 3 ubytes is going to be padded with an extra byte at the end. Telling the compiler to align on a 1-byte boundary (essentially disabling alignment) will save you space, but will will generally cost you cycles in accessing the data.
Sep 01 2017
On Saturday, 2 September 2017 at 02:37:08 UTC, Mike Parker wrote:It's not a bug, but a feature. Data structure alignment is important for efficient reads, so several languages (D, C, C++, Ada, and more) will automatically pad structs so that they can maintain specific byte alignments. On a 32-bit system, 4-byte boundaries are the default. So a struct with 3 ubytes is going to be padded with an extra byte at the end. Telling the compiler to align on a 1-byte boundary (essentially disabling alignment) will save you space, but will will generally cost you cycles in accessing the data.struct int24 { ubyte[3] _payload; } static assert(int24.sizeof == 3); static assert(int24.alignof == 1); Making absolute sense. ubytes don't need any specific alignment to be read efficiently.
Sep 02 2017
On Saturday, 2 September 2017 at 07:20:07 UTC, kinke wrote:
struct int24 {
ubyte[3] _payload;
}
static assert(int24.sizeof == 3);
static assert(int24.alignof == 1);
Making absolute sense. ubytes don't need any specific alignment
to be read efficiently.
Yes, that does make sense. It doesn't make sense that I didn't
realize it.
Sep 02 2017
On Saturday, 2 September 2017 at 00:43:00 UTC, Nicholas Wilson wrote:On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:The very first test in my code checks this: int24[3] a; assert(a.sizeof == 9); On the other hand, using Mir's well-tested code instead of something I hacked together in 10 minutes is probably a good idea. -- Biotronicstruct int24 { ubyte[3] _payload; this(int x) { value = x; } ... }You may also want to put an align(1) on it so that you dont waste 25% of the allocated memory in an array of int24's
Sep 02 2017
On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:big endian is indeed problematic.Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point:property int value(int x) { _payload = (cast(ubyte*)&x)[0..3]; return value; }will not work on big endian machine. version(BigEndian) _payload = (cast(ubyte*)&x)[1..4];
Sep 03 2017
On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.Hi, Probably you are looking for bitpack ndslice topology: sizediff_t[] data; // creates a packed signed integer slice with max allowed value equal to `2^^24 - 1`. auto packs = data[].sliced.bitpack!24; packs has the same API as D arrays Package is Mir Algorithm http://code.dlang.org/packages/mir-algorithm Best, Ilya
Sep 01 2017
On Saturday, 2 September 2017 at 02:49:41 UTC, Ilya Yaroshenko wrote:On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:Thanks. Seems useful.Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.Hi, Probably you are looking for bitpack ndslice topology: sizediff_t[] data; // creates a packed signed integer slice with max allowed value equal to `2^^24 - 1`. auto packs = data[].sliced.bitpack!24; packs has the same API as D arrays Package is Mir Algorithm http://code.dlang.org/packages/mir-algorithm Best, Ilya
Sep 01 2017
On Saturday, 2 September 2017 at 03:29:20 UTC, EntangledQuanta wrote:On Saturday, 2 September 2017 at 02:49:41 UTC, Ilya Yaroshenko wrote:Just added `bytegroup` topology. Released in v0.6.12 (will be available in DUB after few minutes.) http://docs.algorithm.dlang.io/latest/mir_ndslice_topology.html#bytegroup It is faster for your task then `bitpack`. Best regards, IlyaOn Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:Thanks. Seems useful.Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.Hi, Probably you are looking for bitpack ndslice topology: sizediff_t[] data; // creates a packed signed integer slice with max allowed value equal to `2^^24 - 1`. auto packs = data[].sliced.bitpack!24; packs has the same API as D arrays Package is Mir Algorithm http://code.dlang.org/packages/mir-algorithm Best, Ilya
Sep 02 2017
On Sunday, 3 September 2017 at 04:01:34 UTC, Ilya Yaroshenko wrote:On Saturday, 2 September 2017 at 03:29:20 UTC, EntangledQuanta wrote:Thanks! I might end up using this. Is this basically just a logical mapping(cast(int)bytes[i*3]) & 0xFFFFFF) type of stuff or is there more of a performance hit? I could do the mapping myself if that is the case as I do not need much of a general solution. I'll probably be using in a just a few lines of code. It just needs to be nearly as fast as direct access.On Saturday, 2 September 2017 at 02:49:41 UTC, Ilya Yaroshenko wrote:Just added `bytegroup` topology. Released in v0.6.12 (will be available in DUB after few minutes.) http://docs.algorithm.dlang.io/latest/mir_ndslice_topology.html#bytegroup It is faster for your task then `bitpack`. Best regards, IlyaOn Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:Thanks. Seems useful.Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.Hi, Probably you are looking for bitpack ndslice topology: sizediff_t[] data; // creates a packed signed integer slice with max allowed value equal to `2^^24 - 1`. auto packs = data[].sliced.bitpack!24; packs has the same API as D arrays Package is Mir Algorithm http://code.dlang.org/packages/mir-algorithm Best, Ilya
Sep 03 2017
On Sunday, 3 September 2017 at 23:30:43 UTC, EntangledQuanta wrote:On Sunday, 3 September 2017 at 04:01:34 UTC, Ilya Yaroshenko wrote:The implementation can be found here https://github.com/libmir/mir-algorithm/blob/master/source/mir/ndslice/iterator.d It uses unions and byte loads. The speed should be almost the same as direct access.On Saturday, 2 September 2017 at 03:29:20 UTC, EntangledQuanta wrote:Thanks! I might end up using this. Is this basically just a logical mapping(cast(int)bytes[i*3]) & 0xFFFFFF) type of stuff or is there more of a performance hit? I could do the mapping myself if that is the case as I do not need much of a general solution. I'll probably be using in a just a few lines of code. It just needs to be nearly as fast as direct access.On Saturday, 2 September 2017 at 02:49:41 UTC, Ilya Yaroshenko wrote:Just added `bytegroup` topology. Released in v0.6.12 (will be available in DUB after few minutes.) http://docs.algorithm.dlang.io/latest/mir_ndslice_topology.html#bytegroup It is faster for your task then `bitpack`. Best regards, Ilya[...]Thanks. Seems useful.
Sep 03 2017
On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.If you need to perform math on integer audio, I strongly suggest to use int32 for this purpose to avoid performance degradation, then convert back to int24 when you're finished. Probably going to add such feature into my PCM library.
Sep 02 2017