www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - What is the D way to map a binary file to a structure?

reply "cym13" <cpicard openmailbox.org> writes:
Hi,

Let's say I have a simple binary file whose structure is 
well-known. Here is
an example which stores points:

struct Point {
     long x;
     long y;
     long z;
}

struct BinFile {
     uint    magicNumber;  // Some identifier
     ulong   pointsNumber;
     Point[] points;       // Array of pointsNumber points.
}

What is the best way to read some file and fill a structure with 
it? Would
reading the file into a void[] and then casting it to the struct 
work with
things like internal struct padding?
Aug 29 2015
next sibling parent reply drug <drug2004 bk.ru> writes:
29.08.2015 15:56, cym13 пишет:
 Hi,

 Let's say I have a simple binary file whose structure is well-known.
 Here is
 an example which stores points:

 struct Point {
      long x;
      long y;
      long z;
 }

 struct BinFile {
      uint    magicNumber;  // Some identifier
      ulong   pointsNumber;
      Point[] points;       // Array of pointsNumber points.
 }

 What is the best way to read some file and fill a structure with it? Would
 reading the file into a void[] and then casting it to the struct work with
 things like internal struct padding?
Try, for example, MessagePack https://github.com/msgpack/msgpack-d.git
Aug 29 2015
parent reply "cym13" <cpicard openmailbox.org> writes:
On Saturday, 29 August 2015 at 13:56:10 UTC, drug wrote:
 Try, for example, MessagePack 
 https://github.com/msgpack/msgpack-d.git
Thanks, but it isn't answering the question at all. I'm not looking for a serialization method, I'm looking for the best way to read a binary file.
Aug 29 2015
parent reply drug <drug2004 bk.ru> writes:
29.08.2015 17:17, cym13 пишет:
 On Saturday, 29 August 2015 at 13:56:10 UTC, drug wrote:
 Try, for example, MessagePack https://github.com/msgpack/msgpack-d.git
Thanks, but it isn't answering the question at all. I'm not looking for a serialization method, I'm looking for the best way to read a binary file.
It depends on what is the best for you. But using MessagePack you can easily read the file and fill a structure with it.
Aug 29 2015
parent reply "cym13" <cpicard openmailbox.org> writes:
On Saturday, 29 August 2015 at 14:52:51 UTC, drug wrote:
 29.08.2015 17:17, cym13 пишет:
 On Saturday, 29 August 2015 at 13:56:10 UTC, drug wrote:
 Try, for example, MessagePack 
 https://github.com/msgpack/msgpack-d.git
Thanks, but it isn't answering the question at all. I'm not looking for a serialization method, I'm looking for the best way to read a binary file.
It depends on what is the best for you. But using MessagePack you can easily read the file and fill a structure with it.
No, because messagepack is one format of binary file. That isn't going to be of any help for any other binary file format. It is not a way to read binary files. It is a serialization format.
Aug 29 2015
parent drug <drug2004 bk.ru> writes:
29.08.2015 18:05, cym13 пишет:
 On Saturday, 29 August 2015 at 14:52:51 UTC, drug wrote:
 29.08.2015 17:17, cym13 пишет:
 On Saturday, 29 August 2015 at 13:56:10 UTC, drug wrote:
 Try, for example, MessagePack https://github.com/msgpack/msgpack-d.git
Thanks, but it isn't answering the question at all. I'm not looking for a serialization method, I'm looking for the best way to read a binary file.
It depends on what is the best for you. But using MessagePack you can easily read the file and fill a structure with it.
No, because messagepack is one format of binary file. That isn't going to be of any help for any other binary file format. It is not a way to read binary files. It is a serialization format.
I see.
Aug 29 2015
prev sibling next sibling parent reply "Laeeth Isharc" <Laeeth.nospam nospam-laeeth.com> writes:
On Saturday, 29 August 2015 at 12:56:08 UTC, cym13 wrote:
 Hi,

 Let's say I have a simple binary file whose structure is 
 well-known. Here is
 an example which stores points:

 struct Point {
     long x;
     long y;
     long z;
 }

 struct BinFile {
     uint    magicNumber;  // Some identifier
     ulong   pointsNumber;
     Point[] points;       // Array of pointsNumber points.
 }

 What is the best way to read some file and fill a structure 
 with it? Would
 reading the file into a void[] and then casting it to the 
 struct work with
 things like internal struct padding?
Align(1) ?
Aug 29 2015
parent reply "cym13" <cpicard openmailbox.org> writes:
On Saturday, 29 August 2015 at 16:47:23 UTC, Laeeth Isharc wrote:
 Align(1) ?
That should do it, thanks :)
Aug 29 2015
parent "Suliman" <evermind live.ru> writes:
On Saturday, 29 August 2015 at 16:55:44 UTC, cym13 wrote:
 On Saturday, 29 August 2015 at 16:47:23 UTC, Laeeth Isharc 
 wrote:
 Align(1) ?
That should do it, thanks :)
Do not forget to post code example, please, it's interesting to look at your solution...
Aug 29 2015
prev sibling next sibling parent "Atila Neves" <atila.neves gmail.com> writes:
On Saturday, 29 August 2015 at 12:56:08 UTC, cym13 wrote:
 Hi,

 Let's say I have a simple binary file whose structure is 
 well-known. Here is
 an example which stores points:

 struct Point {
     long x;
     long y;
     long z;
 }

 struct BinFile {
     uint    magicNumber;  // Some identifier
     ulong   pointsNumber;
     Point[] points;       // Array of pointsNumber points.
 }

 What is the best way to read some file and fill a structure 
 with it? Would
 reading the file into a void[] and then casting it to the 
 struct work with
 things like internal struct padding?
https://github.com/atilaneves/cerealed Just pass the bytes obtained from reading the file to `Decerealiser`. Atila
Aug 30 2015
prev sibling parent reply "mzfhhhh" <mzfhhhh foxmail.com> writes:
On Saturday, 29 August 2015 at 12:56:08 UTC, cym13 wrote:
 Hi,

 Let's say I have a simple binary file whose structure is 
 well-known. Here is
 an example which stores points:

 struct Point {
     long x;
     long y;
     long z;
 }

 struct BinFile {
     uint    magicNumber;  // Some identifier
     ulong   pointsNumber;
     Point[] points;       // Array of pointsNumber points.
 }

 What is the best way to read some file and fill a structure 
 with it? Would
 reading the file into a void[] and then casting it to the 
 struct work with
 things like internal struct padding?
struct Point { long x; long y; long z; } struct BinFile { uint magicNumber; // Some identifier ulong pointsNumber; Point[] points; // Array of pointsNumber points. this(ubyte [] buf) { auto f = cast(BinFile *)buf.ptr; this = cast(BinFile)*f; this.points = (cast(Point*)&f.points)[0..cast(uint)this.pointsNumber]; } }
Aug 30 2015
parent "cym13" <cpicard openmailbox.org> writes:
On Monday, 31 August 2015 at 01:01:32 UTC, mzfhhhh wrote:
 On Saturday, 29 August 2015 at 12:56:08 UTC, cym13 wrote:
 Hi,

 Let's say I have a simple binary file whose structure is 
 well-known. Here is
 an example which stores points:

 struct Point {
     long x;
     long y;
     long z;
 }

 struct BinFile {
     uint    magicNumber;  // Some identifier
     ulong   pointsNumber;
     Point[] points;       // Array of pointsNumber points.
 }

 What is the best way to read some file and fill a structure 
 with it? Would
 reading the file into a void[] and then casting it to the 
 struct work with
 things like internal struct padding?
struct Point { long x; long y; long z; } struct BinFile { uint magicNumber; // Some identifier ulong pointsNumber; Point[] points; // Array of pointsNumber points. this(ubyte [] buf) { auto f = cast(BinFile *)buf.ptr; this = cast(BinFile)*f; this.points = (cast(Point*)&f.points)[0..cast(uint)this.pointsNumber]; } }
Thank you, this is what I was looking for :) I had to combine it with align(1) though. Complete working example: import std.conv; import std.stdio; align(1) struct Point { long x; long y; long z; } struct BinFile { align(1): uint magicNumber; // Some identifier ulong pointsNumber; Point[] points; // Array of pointsNumber points. this(ubyte [] buf) { auto f = cast(BinFile *)buf.ptr; this = cast(BinFile)*f; this.points = (cast(Point*)&f.points)[0..pointsNumber]; } } void main(string[] args) { /* auto f = File(args[1], "rb"); ubyte[] buffer = []; foreach (chunk ; f.byChunk(4096)) buffer ~= chunk; */ ubyte[] buffer = [ 0xef, 0xbe, 0xad, 0xde, // magicNumber 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // pointsNumber 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // points[0].x 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // points[0].y 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // points[0].z 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // points[1].x 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // points[1].y 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // points[1].z ]; auto bf = BinFile(buffer); writefln("Magic number: %x", bf.magicNumber); writeln("Number of points: ", bf.pointsNumber); writeln("Points:"); foreach (p ; bf.points) writeln(" ", p); } Thanks all!
Aug 31 2015