digitalmars.D.learn - Reading and converting binary file 2 bits at a time
- Andrew Brown (16/16) Aug 27 2015 Hi,
- Rikki Cattermole (8/23) Aug 27 2015 You do not use the std.file to prefix call read.
- rumbu (9/25) Aug 27 2015 auto bytes = cast(ubyte[])read("binaryfile");
- Andrew Brown (7/39) Aug 27 2015 That's lovely, thank you. One quick question, the length of the
- Olivier Pisano (3/9) Aug 27 2015 Well, since the length of ubyte is 1, the length of the file is
- Gary Willoughby (30/71) Aug 27 2015 You can also avoid the bitshifts to make the code a little more
- Mike James (33/49) Aug 27 2015 How about...
- Gary Willoughby (2/3) Aug 27 2015 A lot nicer. :)
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (23/23) Aug 29 2015 Just cast to `Crumbs[]` directly:
- Mike James (2/25) Aug 29 2015 I like that :-)
- Gary Willoughby (3/32) Aug 29 2015 But it might not be safe:
- anonymous (3/5) Aug 29 2015 That link just takes me to this thread here again.
- Gary Willoughby (3/8) Aug 30 2015 Here's the correct link.
- Andrew Brown (30/30) Aug 31 2015 Thanks very much for all the help, your advice worked a treat.
- Adam D. Ruppe (10/11) Aug 31 2015 Yes, the reason is because the nested struct has a hidden member
Hi, I need to read a binary file, and then process it two bits at a time. But I'm a little stuck on the first step. So far I have: import std.file; import std.stdio; void main(){ auto f = std.file.read("binaryfile"); auto g = cast(bool[]) f; writeln(g); } but all the values of g then are just true, could you tell me what I'm doing wrong? I've also looked at the bitmanip module, I couldn't get it to help, but is that the direction I should be looking? Thanks very much Andrew
Aug 27 2015
On 27/08/15 9:00 PM, Andrew Brown wrote:Hi, I need to read a binary file, and then process it two bits at a time. But I'm a little stuck on the first step. So far I have: import std.file; import std.stdio; void main(){ auto f = std.file.read("binaryfile"); auto g = cast(bool[]) f; writeln(g); } but all the values of g then are just true, could you tell me what I'm doing wrong? I've also looked at the bitmanip module, I couldn't get it to help, but is that the direction I should be looking? Thanks very much AndrewYou do not use the std.file to prefix call read. The bool type is one byte in size. A value of 0 is false, anything else is true. But commonly defined as 1. So if you want to read only one bit you will need to use e.g. bit shifts and bitwise and operator. I believe bitsSet may help you.
Aug 27 2015
On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:Hi, I need to read a binary file, and then process it two bits at a time. But I'm a little stuck on the first step. So far I have: import std.file; import std.stdio; void main(){ auto f = std.file.read("binaryfile"); auto g = cast(bool[]) f; writeln(g); } but all the values of g then are just true, could you tell me what I'm doing wrong? I've also looked at the bitmanip module, I couldn't get it to help, but is that the direction I should be looking? Thanks very much Andrewauto bytes = cast(ubyte[])read("binaryfile"); foreach(b; bytes) { writeln((b & 0xC0) >> 6); //bits 7, 6 writeln((b & 0x30) >> 4); //bits 5, 4 writeln((b & 0x0C) >> 2); //bits 3, 2 writeln((b & 0x03)); //bits 1, 0 }
Aug 27 2015
On Thursday, 27 August 2015 at 09:26:55 UTC, rumbu wrote:On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:That's lovely, thank you. One quick question, the length of the file is not a multiple of the length of ubyte, but the cast still seems to work. Do you know how it converts a truncated final section? Thanks again AndrewHi, I need to read a binary file, and then process it two bits at a time. But I'm a little stuck on the first step. So far I have: import std.file; import std.stdio; void main(){ auto f = std.file.read("binaryfile"); auto g = cast(bool[]) f; writeln(g); } but all the values of g then are just true, could you tell me what I'm doing wrong? I've also looked at the bitmanip module, I couldn't get it to help, but is that the direction I should be looking? Thanks very much Andrewauto bytes = cast(ubyte[])read("binaryfile"); foreach(b; bytes) { writeln((b & 0xC0) >> 6); //bits 7, 6 writeln((b & 0x30) >> 4); //bits 5, 4 writeln((b & 0x0C) >> 2); //bits 3, 2 writeln((b & 0x03)); //bits 1, 0 }
Aug 27 2015
On Thursday, 27 August 2015 at 09:38:52 UTC, Andrew Brown wrote:That's lovely, thank you. One quick question, the length of the file is not a multiple of the length of ubyte, but the cast still seems to work. Do you know how it converts a truncated final section? Thanks again AndrewWell, since the length of ubyte is 1, the length of the file is necessarily a multiple of the length of ubyte.
Aug 27 2015
On Thursday, 27 August 2015 at 09:38:52 UTC, Andrew Brown wrote:On Thursday, 27 August 2015 at 09:26:55 UTC, rumbu wrote:You can also avoid the bitshifts to make the code a little more readable like this: import std.stdio; import std.bitmanip; struct Crumbs { mixin(bitfields!( uint, "one", 2, uint, "two", 2, uint, "three", 2, uint, "four", 2 )); } void main(string[] args) { ubyte[] buffer = [123, 12, 126, 244, 35]; foreach (octet; buffer) { auto crumbs = Crumbs(octet); ubyte* representation = cast(ubyte*)&crumbs; writefln("Crumb: %08b", *representation); writefln("Crumb one: %s", crumbs.one); writefln("Crumb two: %s", crumbs.two); writefln("Crumb three: %s", crumbs.three); writefln("Crumb four: %s", crumbs.four); writeln("---------------------"); } } Now you can read a crumb at a time.On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:That's lovely, thank you. One quick question, the length of the file is not a multiple of the length of ubyte, but the cast still seems to work. Do you know how it converts a truncated final section? Thanks again AndrewHi, I need to read a binary file, and then process it two bits at a time. But I'm a little stuck on the first step. So far I have: import std.file; import std.stdio; void main(){ auto f = std.file.read("binaryfile"); auto g = cast(bool[]) f; writeln(g); } but all the values of g then are just true, could you tell me what I'm doing wrong? I've also looked at the bitmanip module, I couldn't get it to help, but is that the direction I should be looking? Thanks very much Andrewauto bytes = cast(ubyte[])read("binaryfile"); foreach(b; bytes) { writeln((b & 0xC0) >> 6); //bits 7, 6 writeln((b & 0x30) >> 4); //bits 5, 4 writeln((b & 0x0C) >> 2); //bits 3, 2 writeln((b & 0x03)); //bits 1, 0 }
Aug 27 2015
On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:Hi, I need to read a binary file, and then process it two bits at a time. But I'm a little stuck on the first step. So far I have: import std.file; import std.stdio; void main(){ auto f = std.file.read("binaryfile"); auto g = cast(bool[]) f; writeln(g); } but all the values of g then are just true, could you tell me what I'm doing wrong? I've also looked at the bitmanip module, I couldn't get it to help, but is that the direction I should be looking? Thanks very much AndrewHow about... module main; import std.bitmanip; import std.stdio; struct Crumbs { property ref ubyte whole() { return m_whole; } union { private ubyte m_whole; mixin(bitfields!( ubyte, "one", 2, ubyte, "two", 2, ubyte, "three", 2, ubyte, "four", 2 )); } } void main(string[] argv) { ubyte[] buffer = [123, 12, 126, 244, 35]; Crumbs cmb; foreach (octet; buffer) { cmb.whole = octet; writefln("Crumb: %08b", octet); writefln("Crumb one: %s", cmb.one); writefln("Crumb two: %s", cmb.two); writefln("Crumb three: %s", cmb.three); writefln("Crumb four: %s", cmb.four); } } Regards, Mike.
Aug 27 2015
On Thursday, 27 August 2015 at 12:40:07 UTC, Mike James wrote:How about...A lot nicer. :)
Aug 27 2015
Just cast to `Crumbs[]` directly: import std.bitmanip; import std.stdio; import std.file; struct Crumbs { mixin(bitfields!( ubyte, "one", 2, ubyte, "two", 2, ubyte, "three", 2, ubyte, "four", 2 )); } void main(string[] argv) { auto raw = read("binaryfile"); auto buffer = cast(Crumbs[]) raw; foreach (cmb; buffer) { writefln("Crumb one: %s", cmb.one); writefln("Crumb two: %s", cmb.two); writefln("Crumb three: %s", cmb.three); writefln("Crumb four: %s", cmb.four); } }
Aug 29 2015
On Saturday, 29 August 2015 at 20:15:53 UTC, Marc Schütz wrote:Just cast to `Crumbs[]` directly: import std.bitmanip; import std.stdio; import std.file; struct Crumbs { mixin(bitfields!( ubyte, "one", 2, ubyte, "two", 2, ubyte, "three", 2, ubyte, "four", 2 )); } void main(string[] argv) { auto raw = read("binaryfile"); auto buffer = cast(Crumbs[]) raw; foreach (cmb; buffer) { writefln("Crumb one: %s", cmb.one); writefln("Crumb two: %s", cmb.two); writefln("Crumb three: %s", cmb.three); writefln("Crumb four: %s", cmb.four); } }I like that :-)
Aug 29 2015
On Saturday, 29 August 2015 at 21:50:12 UTC, Mike James wrote:On Saturday, 29 August 2015 at 20:15:53 UTC, Marc Schütz wrote:But it might not be safe: http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.orgJust cast to `Crumbs[]` directly: import std.bitmanip; import std.stdio; import std.file; struct Crumbs { mixin(bitfields!( ubyte, "one", 2, ubyte, "two", 2, ubyte, "three", 2, ubyte, "four", 2 )); } void main(string[] argv) { auto raw = read("binaryfile"); auto buffer = cast(Crumbs[]) raw; foreach (cmb; buffer) { writefln("Crumb one: %s", cmb.one); writefln("Crumb two: %s", cmb.two); writefln("Crumb three: %s", cmb.three); writefln("Crumb four: %s", cmb.four); } }I like that :-)
Aug 29 2015
On Saturday, 29 August 2015 at 23:34:47 UTC, Gary Willoughby wrote:But it might not be safe: http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.orgThat link just takes me to this thread here again.
Aug 29 2015
On Sunday, 30 August 2015 at 00:02:16 UTC, anonymous wrote:On Saturday, 29 August 2015 at 23:34:47 UTC, Gary Willoughby wrote:Here's the correct link. http://forum.dlang.org/thread/sugxdshytelayxnsthwl forum.dlang.orgBut it might not be safe: http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.orgThat link just takes me to this thread here again.
Aug 30 2015
Thanks very much for all the help, your advice worked a treat. One final question, originally I was defining the struct inside the main loop and it was using 4 bytes per field rather than 2 bits, e.g.: import std.bitmanip; import std.stdio; struct Crumb1 { mixin(bitfields!( ubyte, "one", 2, ubyte, "two", 2, ubyte, "three", 2, ubyte, "four", 2)); } void main() { struct Crumb2 { mixin(bitfields!( ubyte, "one", 2, ubyte, "two", 2, ubyte, "three", 2, ubyte, "four", 2)); } writeln(Crumb1.sizeof, " ", Crumb2.sizeof); } outputs: 1 16 Is this correect behaviour? Andrew
Aug 31 2015
On Monday, 31 August 2015 at 18:00:54 UTC, Andrew Brown wrote:Is this correect behaviour?Yes, the reason is because the nested struct has a hidden member - a pointer to its stack context. This allows it to access variables from the surrounding local scope. It adds 8 bytes on 64 bit cuz that's pointer size, and the other bytes are padding so the pointer is aligned on a word boundary. If you change it to `static struct` inside the main function (or any other function), you'll find it then has the size of one again, since a static struct does not have access to outer variables, so it leaves that hidden pointer out.
Aug 31 2015