www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Write struct to file

reply "Chopin" <robert.bue gmail.com> writes:
Hello!

import std.stdio;

struct nagger
{
     string name;
     int age;
     double weight;
     string msg;
}

void main()
{
     auto lal = new nagger();
     lal.name = "AHAHAHAHHA";
     lal.age = 23;
     lal.weight = 108.5;
     lal.msg = "fgfdgfdgfdgfdgfdgfdg";
     writeln(cast(ubyte[])(lal));
}

Gives error: Error: e2ir: cannot cast lal of type nagger* to type 
ubyte[]
I have bad knowledge about this and don't understand why I cant 
do this :(
What I really want is an array of naggers and save them binary to 
a file :)
This was like my step 1, and it failed miserably...

Thanks for help.
Feb 25 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/25/2012 07:03 PM, Chopin wrote:

 Hello!

 import std.stdio;

 struct nagger
 {
 string name;
 int age;
 double weight;
 string msg;
 }

 void main()
 {
 auto lal = new nagger();
 lal.name = "AHAHAHAHHA";
 lal.age = 23;
 lal.weight = 108.5;
 lal.msg = "fgfdgfdgfdgfdgfdgfdg";
 writeln(cast(ubyte[])(lal));
 }

 Gives error: Error: e2ir: cannot cast lal of type nagger* to type ubyte[]
 I have bad knowledge about this and don't understand why I cant do this :(
 What I really want is an array of naggers and save them binary to a file :)
 This was like my step 1, and it failed miserably...

 Thanks for help.


Structs are not castable to arrays directly. You can achieve what you 
want like this (untested, but should work).

(cast(void*)&lal)[0..nagger.sizeof];
Feb 25 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/25/2012 07:15 PM, Timon Gehr wrote:

 On 02/25/2012 07:03 PM, Chopin wrote:

 Hello!

 import std.stdio;

 struct nagger
 {
 string name;
 int age;
 double weight;
 string msg;
 }

 void main()
 {
 auto lal = new nagger();
 lal.name = "AHAHAHAHHA";
 lal.age = 23;
 lal.weight = 108.5;
 lal.msg = "fgfdgfdgfdgfdgfdgfdg";
 writeln(cast(ubyte[])(lal));
 }

 Gives error: Error: e2ir: cannot cast lal of type nagger* to type ubyte[]
 I have bad knowledge about this and don't understand why I cant do
 this :(
 What I really want is an array of naggers and save them binary to a
 file :)
 This was like my step 1, and it failed miserably...

 Thanks for help.


 Structs are not castable to arrays directly. You can achieve what you
 want like this (untested, but should work).

 (cast(void*)&lal)[0..nagger.sizeof];


You also have to change the first line of main to:

auto lol = nagger();

This will allocate the struct on the stack.

If you want to keep auto lal = new nagger(); for some reason, then do 
the cast like this:

(cast(void*)lal)[0..nagger.sizeof]
Feb 25 2012
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Well first I'd recommend not allocating the struct on the heap. Then you can do:

import std.stdio;

struct nagger
{
    string name;
    int age;
    double weight;
    string msg;
}

void main()
{
    nagger lal;
    lal.name   = "name";
    lal.age    = 23;
    lal.weight = 108.5;
    lal.msg    = "msg";

    auto file = File("test.bin", "w");
    auto writeBytes = fwrite(&lal, byte.sizeof, lal.sizeof, file.getFP());
    file.close();

    nagger dup;
    file = File("test.bin", "r");
    auto readBytes = fread(&dup, byte.sizeof, dup.sizeof, file.getFP());

    assert(lal == dup);
}

This doesn't work for heap-allocated structs.
Feb 25 2012
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/25/2012 10:33 AM, Andrej Mitrovic wrote:

 Well first I'd recommend not allocating the struct on the heap. Then you can
do:

 import std.stdio;

 struct nagger
 {
      string name;
      int age;
      double weight;
      string msg;
 }

 void main()
 {
      nagger lal;
      lal.name   = "name";
      lal.age    = 23;
      lal.weight = 108.5;
      lal.msg    = "msg";

      auto file = File("test.bin", "w");
      auto writeBytes = fwrite(&lal, byte.sizeof, lal.sizeof, file.getFP());


But there is no way for fwrite to follow name.ptr to also write the 
characters that are in the string, right?


      file.close();

      nagger dup;
      file = File("test.bin", "r");
      auto readBytes = fread(&dup, byte.sizeof, dup.sizeof, file.getFP());

      assert(lal == dup);


That passes because lal.name.ptr and dup.name.ptr have the same value. 
Maybe that wasn't the intention but the data is not really in the file.


 }

 This doesn't work for heap-allocated structs.


Ali
Feb 25 2012
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/25/12, Ali =C7ehreli <acehreli yahoo.com> wrote:

 That passes because lal.name.ptr and dup.name.ptr have the same value.
 Maybe that wasn't the intention but the data is not really in the file.


I'm not sure where you're getting that from:

import std.stdio;

struct nagger
{
    string name;
    int age;
    double weight;
    string msg;
}

void main(string[] args)
{
    if (args[1] =3D=3D "write")
    {
        nagger lal;
        lal.name   =3D "name";
        lal.age    =3D 23;
        lal.weight =3D 108.5;
        lal.msg    =3D "msg";

        auto file       =3D File("test.bin", "w");
        auto writeBytes =3D fwrite(&lal, byte.sizeof, lal.sizeof, file.getF=
P());
    }
    else
    if (args[1] =3D=3D "read")
    {
        nagger dup;
        auto file =3D File("test.bin", "r");
        auto readBytes =3D fread(&dup, byte.sizeof, dup.sizeof, file.getFP(=
));
        writeln(dup);
    }
}

$ rdmd test.d write
$ rdmd test.d read
nagger("name", 23, 108.5, "msg")
Feb 25 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
To be honest the C fread and fwrite aren't even necessary since you
can do a rawRead and rawWrite instead.
Feb 25 2012
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/25/12, Ali =C7ehreli <acehreli yahoo.com> wrote:

 But there is no way for fwrite to follow name.ptr to also write the
 characters that are in the string, right?


Oh my I just got a big fat zero on the finals. You're absolutely
right, what gets copied is the length and the pointer. The only reason
my last sample worked is because the memory for the strings was
allocated on the stack, which had the same address between runs.
Feb 25 2012
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/25/2012 03:00 PM, Andrej Mitrovic wrote:

 On 2/25/12, Ali Çehreli<acehreli yahoo.com>  wrote:

 But there is no way for fwrite to follow name.ptr to also write the
 characters that are in the string, right?


 Oh my I just got a big fat zero on the finals. You're absolutely
 right, what gets copied is the length and the pointer. The only reason
 my last sample worked is because the memory for the strings was
 allocated on the stack, which had the same address between runs.


Ha ha! :) Sorry, I wasn't clear before. That was exactly what I meant. 
Whenever I see objects written by &object, I hear warning bells. :)

Ali
Feb 25 2012
parent reply "Chopin" <robert.bue gmail.com> writes:
Hello! OP here :)
I'm gonna be more precise why I want this.
I'm writing my own blog, I will write my own HTTP server etc. and 
the saving of the data myself. Everything will be done in D! Why? 
Learn D. I don't want to hear the reinvent the wheel :(, I think 
you learn a lot by doing thing like this.

So, when I POST a blog-entry, I will read it, and write it to a 
file. I don't want write strings in a file, like:

1--||--Title--||--The entry--||--Date

I have little knowledge about system programming, and writing raw 
files. But my dream was like:

SOME_INT_WITH_LENGTH_OF_STRUCT_TO_THE_RIGHT 
HERE_IS_THE_RAW_STRUCT 
SOME_INT_WITH_LENGTH_OF_STRUCT_TO_THE_RIGHT 
HERE_IS_THE_RAW_STRUCT etc.etc.etc.

Then read those structs in array :)

I thought this was kinda easy in C, but I could be very wrong! So 
I thought it must be super easy in D! I don't have the 
knowledge...

ae.utils.json <-- this will be not as "cool" as writing it raw 
binary? :p
Well. Thank you all for information, tips and tricks!!
Great community! Thanks!
Feb 26 2012
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/26/2012 03:22 PM, Chopin wrote:


 I don't want to hear the reinvent the wheel :(, I think you
 learn a lot by doing thing like this.


Agreed.


 So, when I POST a blog-entry, I will read it, and write it to a file. I
 don't want write strings in a file, like:

 1--||--Title--||--The entry--||--Date


I encourage you to stay with human-readable formats unless there is a 
reason not to. You can ditch XML, fine, but consider other formats like 
json. :) But I understand that you want to do this to learn. Fine...


 I have little knowledge about system programming, and writing raw files.
 But my dream was like:

 SOME_INT_WITH_LENGTH_OF_STRUCT_TO_THE_RIGHT HERE_IS_THE_RAW_STRUCT
 SOME_INT_WITH_LENGTH_OF_STRUCT_TO_THE_RIGHT HERE_IS_THE_RAW_STRUCT
 etc.etc.etc.


You can do that only with structs that contain the entire data. As soon 
as there is a reference member, you must dereference that member to grab 
that member's data. For example, when you have strings, slices, 
associative arrays, class variables, pointers, and other user types with 
reference semantics, the data is not within the struct itself.

No matter how long the string is, the following struct is always the 
same size:

struct S
{
     char[] s;
}

On the other hand, the following is fine for what you want to do:

struct S
{
     char[1000] s;
}

But is 1000 always enough? Is it wasteful (every instance of S will be 
very large.)?


 Then read those structs in array :)


Again, that part is easy as long as the struct doesn't have reference 
members.


 I thought this was kinda easy in C, but I could be very wrong!


C has exactly the same issues. As soon as you have a pointer member you 
must /follow/ that member to reach the actual data.


 So I
 thought it must be super easy in D! I don't have the knowledge...


D is awesome compared to C as it enables serializing/deserializing data 
with its generic programming and compile-time reflection features like this:

   http://dlang.org/traits.html#allMembers

I don't know their details but I would imagine that serialization 
libraries must be written taking advantage of allMembers.


 ae.utils.json <-- this will be not as "cool" as writing it raw binary? :p


Must be subjective. I find json output cooler that binary. :p

Ali
Feb 26 2012
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/27/12, Ali =C7ehreli <acehreli yahoo.com> wrote:

 D is awesome compared to C as it enables serializing/deserializing data
 with its generic programming and compile-time reflection features like th=


is:

    http://dlang.org/traits.html#allMembers


ae's json uses .tupleof, and does it in 40(!)* lines of code.

* Well there's a helper function that escapes some strings, but in
essence it's 40 lines of code that do most of the work.

I think I'm going to print this out, frame it, and put it on a wall
because it's just so damn concise:
https://github.com/CyberShadow/ae/blob/master/utils/json.d#L69

(This is Vladimir's code for those not in the know)


  > ae.utils.json <-- this will be not as "cool" as writing it raw binary?=


 :p

 Must be subjective. I find json output cooler that binary. :p


Yeah and again if there are no references he can simply use the
cast(void*)[0..len] trick. The challenge would be to implement this
for reference types.
Feb 26 2012
parent "Chopin" <robert.bue gmail.com> writes:
Ah, I get it now.
Thank you so much!
I think I will go for JSON. It will be mostly string data anyway 
:)!
Feb 27 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/26/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 allocated on the stack


Sorry, I meant the data segment not the stack. That's -1 score for me.
Feb 25 2012
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/25/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 I'm not sure where you're getting that from:


Let that be a classic lesson on what never to do. Here's a
demonstration on how wrong I was:

import std.stdio;

struct Foo
{
    char[] name;
}

void main(string[] args)
{
   if (args[1] == "write")
   {
       Foo foo;
       foo.name = new char[](16);
       foo.name[] = 'a';
       writeln(foo);

       auto file       = File("test.bin", "w");
       auto writeBytes = fwrite(&foo, byte.sizeof, foo.sizeof, file.getFP());
   }
   else
   if (args[1] == "read")
   {
       Foo foo;
       auto file = File("test.bin", "r");
       auto readBytes = fread(&foo, byte.sizeof, foo.sizeof, file.getFP());
       writeln(foo);
   }
}

$ D:\dev\code\d_code>test write
Foo("aaaaaaaaaaaaaaaa")

$ D:\dev\code\d_code>test read
Foo(x"D8 6E 43 00 01 00 00 00 08 00 00 00 90 A0 42 00"c)

:)

To OP: If you want to serialize I recommend ae's json module
(ae.util.json) from https://github.com/CyberShadow/ae . There's also
Orange but it's based on xml and seems to be buggy the last time I've
tried it.
Feb 25 2012
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/25/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 This doesn't work for heap-allocated structs.


Sorry my bad. .sizeof should always be set on Types and not variable
names, because a struct pointer will have sizeof == size_t, whereas a
simple struct variable will have sizeof equal to the struct size. So
it can work just fine with heap-allocated structs:

void main()
{
    auto lal = new nagger();
    lal.name   = "name";
    lal.age    = 23;
    lal.weight = 108.5;
    lal.msg    = "msg";

    auto file = File("test.bin", "w");
    auto writeBytes = fwrite(cast(void*)lal, byte.sizeof,
nagger.sizeof, file.getFP());
    file.close();

    nagger dup;
    file = File("test.bin", "r");
    auto readBytes = fread(&dup, byte.sizeof, nagger.sizeof, file.getFP());

    assert(dup == *lal);
}
Feb 25 2012