www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Stream to struct

reply "Miguel Ferreira Simões" <Kobold netcabo.pt> writes:
Look at the following code:

 struct BitmapFileHeader {
   ushort bfType;
   uint   bfSize;
   ushort bfReserved1;
   ushort bfReserved2;
   uint   bfOffBits;
  }

  BitmapFileHeader fileHeader;

  //method one for reading

  stream.seekSet(0);
  stream.read(fileHeader.bfType);
  stream.read(fileHeader.bfSize);
  stream.read(fileHeader.bfReserved1);
  stream.read(fileHeader.bfReserved2);
  stream.read(fileHeader.bfOffBits);

  //method two for reading

  stream.seekSet(0);
  stream.readExact(&fileHeader, fileHeader.sizeof);

Can someone explain me the reason I am getting different results between 
method one and two.

Thanks

-- 
Miguel Ferreira Simões
Mar 09 2005
next sibling parent reply "Martin M. Pedersen" <martin moeller-pedersen.dk> writes:
"Miguel Ferreira Simões" <Kobold netcabo.pt> skrev i en meddelelse 
news:d0mmad$20ta$1 digitaldaemon.com...

 Can someone explain me the reason I am getting different results between 
 method one and two.


It is probably due to alignment, and I suspect that method 1 is the one that 
gives you the expected results. The first member of the struct is 16 bit, 
and the next is 32 bit. Therefore the compiler needs to insert two padding 
bytes in the struct to maintain 32 bit alignment of field two. Even if the 
compiler had (or has?) methods of controlling padding, it easily winds up 
being compiler-specific, which is something you definitely do not want when 
working with a non-compiler-specific format as BMP. So I would consider it 
bad practice to use method 2 even if applicable.

Regards,
Martin
Mar 09 2005
next sibling parent reply "Miguel Ferreira Simões" <Kobold netcabo.pt> writes:
I suppose is due to alignment too.
Structs have a property .alignof can we control alignment?
Mar 09 2005
parent David Medlock <amedlock nospam.org> writes:
Miguel Ferreira Simões wrote:

 I suppose is due to alignment too.
 Structs have a property .alignof can we control alignment?
 
 
 


Its like private/protected/public sections syntax.

align(1):  <members here>

align(2){
   some other members here.
}
Mar 09 2005
prev sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Martin M. Pedersen wrote:


Can someone explain me the reason I am getting different results between 
method one and two.


 
 It is probably due to alignment, and I suspect that method 1 is the one that 
 gives you the expected results. The first member of the struct is 16 bit, 
 and the next is 32 bit. Therefore the compiler needs to insert two padding 
 bytes in the struct to maintain 32 bit alignment of field two. Even if the 
 compiler had (or has?) methods of controlling padding, it easily winds up 
 being compiler-specific, which is something you definitely do not want when 
 working with a non-compiler-specific format as BMP. So I would consider it 
 bad practice to use method 2 even if applicable.


And also, alignment is only half of the problem. Your program
could probably live with the bad performance of having things
aligned to non-optimal boundaries, but everything larger than
bytes (say short and float, for instance) are also *endian*...

Which means that if you read/write directly from memory,
you'll have a harder time to port to a different platform
(BigEndian/LittleEndian). If you read/write each field,
then you can add the required byteswap operations there ?

In portable storage formats, this quickly becomes a problem.
I suggest you fill out the struct one field at a time, and
use a disk buffer for reading (if required) ? This buffer
could use the align(1) struct... But you should have one
format for disk (storage), and one format for memory (usage)
This will also enable you to e.g. compress the storage format.


Speaking of this, where does D stand on Object Serialization ?
http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html

--anders
Mar 09 2005
parent reply pragma <pragma_member pathlink.com> writes:
In article <d0mv1u$29so$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...

In portable storage formats, this quickly becomes a problem.
I suggest you fill out the struct one field at a time, and
use a disk buffer for reading (if required) ? This buffer
could use the align(1) struct... But you should have one
format for disk (storage), and one format for memory (usage)
This will also enable you to e.g. compress the storage format.


Speaking of this, where does D stand on Object Serialization ?
http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html


As far as I understand, the only way to perform serialization of objects is
explicitly.  Its currently a burden placed on the developer.  For example, see
Mango's Pickling interface for a solid example of object serialization in D.

The reason for this is simple: D lacks a solid reflection interface. In order to
accomplish truely transparent serialization, one would need access to the
following features:

- runtime lookup of class members and/or methods.
- runtime creation of an object via classname (akin to java's classloader)

(...which you could implement yourself via a family of mixins or somesuch, but
at that point you're really implementing your own type metadata system)

D's type system has plenty of other missing features, which Walter acknowledged
many moons ago.  Your guess is as good as mine as to why its been sitting on the
back burner.

- EricAnderton at yahoo
Mar 09 2005
parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
pragma wrote:


Speaking of this, where does D stand on Object Serialization ?
http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html


 
 As far as I understand, the only way to perform serialization of objects is
 explicitly.  Its currently a burden placed on the developer.  For example, see
 Mango's Pickling interface for a solid example of object serialization in D.


Ah, thanks. Strange name for that particular interface, don't you think?
http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html


 The reason for this is simple: D lacks a solid reflection interface. In order
to
 accomplish truely transparent serialization, one would need access to the
 following features:
 
 - runtime lookup of class members and/or methods.
 - runtime creation of an object via classname (akin to java's classloader)
 
 (...which you could implement yourself via a family of mixins or somesuch, but
 at that point you're really implementing your own type metadata system)


Wonder if this will ever be possible in D, being fully native and all ?
At least not in http://www.prowiki.org/wiki4d/wiki.cgi?LanguagesVersusD


 D's type system has plenty of other missing features, which Walter acknowledged
 many moons ago.  Your guess is as good as mine as to why its been sitting on
the
 back burner.


Probably has other bugs to fix first...

--anders
Mar 09 2005
parent reply pragma <pragma_member pathlink.com> writes:
In article <d0n47c$2frl$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...

 For example, see Mango's Pickling interface for a solid example of object
serialization in D.


Ah, thanks. Strange name for that particular interface, don't you think?
http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html


I have to admit, the first time I read through Kris' source, I dismissed
'Pickle' for having such an odd name.  However, the concept of actually
'pickling' an object is an apt metaphor for the process.  Also, it originated in
the python language and has cropped up as a boost module. 

However, I cant get rid of the mental image of floppy disks in sealed jars of
vinegar, on shelves in a dark basement.


 - runtime lookup of class members and/or methods.
 - runtime creation of an object via classname (akin to java's classloader)


Wonder if this will ever be possible in D, being fully native and all ?
At least not in http://www.prowiki.org/wiki4d/wiki.cgi?LanguagesVersusD


I'd say its more than possible.  All the needed information exists at compile
time, so all that's left is to drop it in the output somewhere.  I also feel
that the door is open for features like creating types from scratch at runtime. 


 Your guess is as good as mine as to why its been sitting on the
 back burner.


Probably has other bugs to fix first...


Agreed.  If only he accepted code patches, I'd write it *for* him. ;)

- EricAnderton at yahoo
Mar 09 2005
next sibling parent reply Kris <Kris_member pathlink.com> writes:
In article <d0n5rg$2hq9$1 digitaldaemon.com>, pragma says...

In article <d0n47c$2frl$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...

 For example, see Mango's Pickling interface for a solid example of object
serialization in D.


Ah, thanks. Strange name for that particular interface, don't you think?
http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html


I have to admit, the first time I read through Kris' source, I dismissed
'Pickle' for having such an odd name.  However, the concept of actually
'pickling' an object is an apt metaphor for the process.  Also, it originated in
the python language and has cropped up as a boost module. 

However, I cant get rid of the mental image of floppy disks in sealed jars of
vinegar, on shelves in a dark basement.



Truth be told: 'pickle' is a whole lot easier to type that 'serialization' :-)

But I agree; it is hard to forego the thought of a faint waft rising from the
ethernet-packets as they traverse a cluster ...

FWIW ~ the Mango pickling is actually a whole lot faster than "managed"
serialization (as Java has), and just about as convenient for non-trivial use
(efficiency is often a notable factor in serialization). Would still be better
to have full reflection though -- perhaps in 2.0?

- Kris
Mar 09 2005
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Kris wrote:


 FWIW ~ the Mango pickling is actually a whole lot faster than "managed"
 serialization (as Java has), and just about as convenient for non-trivial use
 (efficiency is often a notable factor in serialization). Would still be better
 to have full reflection though -- perhaps in 2.0?


That sounds just great, reminds me to check out whether Mango 1.2 
compiles now ? Just awaiting a new version of GDC, before I do it.

And of course some available spare time to do it, still reading
up on the changes that JDK 1.5 brought to that part of the woods...

--anders
Mar 09 2005
prev sibling parent Charles Hixson <charleshixsn earthlink.net> writes:
pragma wrote:

 In article <d0n47c$2frl$1 digitaldaemon.com>,
 =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
 

For example, see Mango's Pickling interface for a solid example of object
serialization in D.


Ah, thanks. Strange name for that particular interface, don't you think?
http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html


 
 
 I have to admit, the first time I read through Kris' source, I dismissed
 'Pickle' for having such an odd name.  However, the concept of actually
 'pickling' an object is an apt metaphor for the process.  Also, it originated
in
 the python language and has cropped up as a boost module. 
 ..
 
 - EricAnderton at yahoo


Actually, pickling is a whole lot older than Python.  The first 
time I ever ran into is was in LBL during the 1970's or 80's.  (I 
didn't touch it...but then that wasn't my need.  Still, it 
existed and was in use under that name.)

I suspect that the implementation language was Fortran.
Mar 15 2005
prev sibling parent Ben Hinkle <Ben_member pathlink.com> writes:
In article <d0mmad$20ta$1 digitaldaemon.com>, Miguel Ferreira Simões says...

Look at the following code:

 struct BitmapFileHeader {
   ushort bfType;
   uint   bfSize;
   ushort bfReserved1;
   ushort bfReserved2;
   uint   bfOffBits;
  }

  BitmapFileHeader fileHeader;

  //method one for reading

  stream.seekSet(0);
  stream.read(fileHeader.bfType);
  stream.read(fileHeader.bfSize);
  stream.read(fileHeader.bfReserved1);
  stream.read(fileHeader.bfReserved2);
  stream.read(fileHeader.bfOffBits);

  //method two for reading

  stream.seekSet(0);
  stream.readExact(&fileHeader, fileHeader.sizeof);

Can someone explain me the reason I am getting different results between 
method one and two.

Thanks

-- 
Miguel Ferreira Simões 


You should read the same way you write. Meaning if the stream that you are
writing to uses the first method then the stream you are reading should also use
the first method. If the stream you are writing to uses the second then read
using the second. If you are not writing to the stream sending the data then you
need to be careful about alignment, the size and layout of the data - in general
if that is the case I would recommend using the first method.
Mar 09 2005