www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Save/load data to a file

reply "nobody" <somebody somewhere.com> writes:
I would like to be able to save and load a lot of data to/from a file. (in 
D1)
For example a struct like this:

struct Fruit
{
    int banana;
    double[][] orange;
    bool[] apple;
}

Practically all the examples that I've come across only deal with saving and 
loading text, so I'm having a hard time dealing with saving/loading 
arrays/floats/bools/etc.

What would be a good way to do this?
Nov 16 2008
next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
nobody wrote:

 I would like to be able to save and load a lot of data to/from a file. (in 
 D1)
 For example a struct like this:
 
 struct Fruit
 {
     int banana;
     double[][] orange;
     bool[] apple;
 }
 
 Practically all the examples that I've come across only deal with saving and 
 loading text, so I'm having a hard time dealing with saving/loading 
 arrays/floats/bools/etc.
 
 What would be a good way to do this? 


XML is commonly used, and while I don't particularly like it, I find 
it's still a reasonable choice in many circumstances.

If you're using Tango, you can check out tango.text.xml.Document, which 
will let you construct an XML document, and tango.text.xml.DocPrinter, 
which will let you get the textual representation of such a document.

In phobos, there's std.xml, which should offer equivalent functionality, 
but I haven't used it.
Nov 16 2008
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Christopher Wright:

 XML is commonly used, and while I don't particularly like it, I find 
 it's still a reasonable choice in many circumstances.


I don't know if a D library is available for Json file format, it has some
advantages and some disadvantages over XML ().

Bye,
bearophile
Nov 16 2008
parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Sun, Nov 16, 2008 at 3:03 PM, bearophile <bearophileHUGS lycos.com> wrote:

 Christopher Wright:

 XML is commonly used, and while I don't particularly like it, I find
 it's still a reasonable choice in many circumstances.


 I don't know if a D library is available for Json file format, it has some
advantages and some disadvantages over XML ().


Tango has one, tango.text.json.Json.
Nov 16 2008
prev sibling parent reply "nobody" <somebody somewhere.com> writes:
"Christopher Wright" <dhasenan gmail.com> wrote in message 
news:gfpsdt$2uhd$1 digitalmars.com...

 nobody wrote:

 I would like to be able to save and load a lot of data to/from a file. 
 (in D1)
 For example a struct like this:

 struct Fruit
 {
     int banana;
     double[][] orange;
     bool[] apple;
 }

 Practically all the examples that I've come across only deal with saving 
 and loading text, so I'm having a hard time dealing with saving/loading 
 arrays/floats/bools/etc.

 What would be a good way to do this?


 XML is commonly used, and while I don't particularly like it, I find it's 
 still a reasonable choice in many circumstances.

 If you're using Tango, you can check out tango.text.xml.Document, which 
 will let you construct an XML document, and tango.text.xml.DocPrinter, 
 which will let you get the textual representation of such a document.

 In phobos, there's std.xml, which should offer equivalent functionality, 
 but I haven't used it.


Unfortunately I'm using phobos, and I think std.xml is only for D2.0.

Is there no similar for D1.0?
Nov 16 2008
parent reply Alan Knowles <alan akbkhome.com> writes:
This should work, although I really need to rewrite some of it to use 
streams.
http://www.akbkhome.com/svn/D_Stuff/json.d




nobody wrote:

 "Christopher Wright" <dhasenan gmail.com> wrote in message 
 news:gfpsdt$2uhd$1 digitalmars.com...

 nobody wrote:

 I would like to be able to save and load a lot of data to/from a file. 
 (in D1)
 For example a struct like this:

 struct Fruit
 {
     int banana;
     double[][] orange;
     bool[] apple;
 }

 Practically all the examples that I've come across only deal with saving 
 and loading text, so I'm having a hard time dealing with saving/loading 
 arrays/floats/bools/etc.

 What would be a good way to do this?


 XML is commonly used, and while I don't particularly like it, I find it's 
 still a reasonable choice in many circumstances.

 If you're using Tango, you can check out tango.text.xml.Document, which 
 will let you construct an XML document, and tango.text.xml.DocPrinter, 
 which will let you get the textual representation of such a document.

 In phobos, there's std.xml, which should offer equivalent functionality, 
 but I haven't used it.


 
 Unfortunately I'm using phobos, and I think std.xml is only for D2.0.
 
 Is there no similar for D1.0?
Nov 17 2008
parent reply "nobody" <somebody somewhere.com> writes:
"Alan Knowles" <alan akbkhome.com> wrote in message 
news:4922079E.3080401 akbkhome.com...

 This should work, although I really need to rewrite some of it to use 
 streams.
 http://www.akbkhome.com/svn/D_Stuff/json.d




 nobody wrote:

 "Christopher Wright" <dhasenan gmail.com> wrote in message 
 news:gfpsdt$2uhd$1 digitalmars.com...

 nobody wrote:

 I would like to be able to save and load a lot of data to/from a file. 
 (in D1)
 For example a struct like this:

 struct Fruit
 {
     int banana;
     double[][] orange;
     bool[] apple;
 }

 Practically all the examples that I've come across only deal with 
 saving and loading text, so I'm having a hard time dealing with 
 saving/loading arrays/floats/bools/etc.

 What would be a good way to do this?


 XML is commonly used, and while I don't particularly like it, I find 
 it's still a reasonable choice in many circumstances.

 If you're using Tango, you can check out tango.text.xml.Document, which 
 will let you construct an XML document, and tango.text.xml.DocPrinter, 
 which will let you get the textual representation of such a document.

 In phobos, there's std.xml, which should offer equivalent functionality, 
 but I haven't used it.


 Unfortunately I'm using phobos, and I think std.xml is only for D2.0.

 Is there no similar for D1.0?




Hmm, I'm clearly doing something wrong because I can't get it to properly 
save doubles/reals..

void main() {

  struct A
  {
   char[] sval = "test";
   bool bval = true;
   int ival = 3;
   double dval = 4f/3;
   real rval = 8f/3;
  }

  A a;

  auto x = jsonO();
  //x.add("key1",a.sval);
  //x.add("key2",a.bval);
  x.add("key3",a.ival);
  x.add("key4",jsonV(a.dval));
  x.add("key5",jsonV(a.rval));

  auto encoded = x.encode();

  std.file.write("test.dat",encoded);

  ubyte[] file = cast(ubyte[])std.file.read("test.dat");

  auto decoded = jsonDecode(file);

  //writefln(decoded.getN("key1"));
  //writefln(decoded.getN("key2"));
  writefln(decoded.getN("key3"));
  writefln(decoded.getN("key4"));
  writefln(decoded.getN("key5"));

}

This writes to file: {"key3":3,"key4":1,"key5":2}
So the double & real seem to be converted to int.

What am I doing wrong?
Nov 18 2008
parent Niels <niels.kooiman gmail.com> writes:
  struct A
  {
   char[] sval = "test";
   bool bval = true;
   int ival = 3;
   double dval = 4f/3;
   real rval = 8f/3;
  }

should likely be:

  struct A
  {
   char[] sval = "test";
   bool bval = true;
   int ival = 3;
   double dval = 4f/3f; //  4/3f should work to
   real rval = 8f/3f;
  }
Nov 21 2008
prev sibling next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
16.11.08 в 18:55 nobody в своём письме писал(а):


 I would like to be able to save and load a lot of data to/from a file.  
 (in
 D1)
 For example a struct like this:

 struct Fruit
 {
     int banana;
     double[][] orange;
     bool[] apple;
 }

 Practically all the examples that I've come across only deal with saving  
 and
 loading text, so I'm having a hard time dealing with saving/loading
 arrays/floats/bools/etc.

 What would be a good way to do this?


I know there is a doost.serializer  
(http://dsource.org/projects/doost/wiki/Serializer) but I never used it  
myself so I can't say whether it fill fit you.
I am also prototyping another one at the moment, but I don't know how far  
will it take.
Nov 16 2008
parent reply "Bill Baxter" <wbaxter gmail.com> writes:
T24gTW9uLCBOb3YgMTcsIDIwMDggYXQgNTo1MyBBTSwgRGVuaXMgS29yb3NraW4gPDJrb3JkZW5A
Z21haWwuY29tPiB3cm90ZToKPiAxNi4xMS4wOCDXIDE4OjU1IG5vYm9keSDXINPXz6PNINDJ09jN
xSDQydPBzCjBKToKPgo+PiBJIHdvdWxkIGxpa2UgdG8gYmUgYWJsZSB0byBzYXZlIGFuZCBsb2Fk
IGEgbG90IG9mIGRhdGEgdG8vZnJvbSBhIGZpbGUuIChpbgo+PiBEMSkKPj4gRm9yIGV4YW1wbGUg
YSBzdHJ1Y3QgbGlrZSB0aGlzOgo+Pgo+PiBzdHJ1Y3QgRnJ1aXQKPj4gewo+PiAgICBpbnQgYmFu
YW5hOwo+PiAgICBkb3VibGVbXVtdIG9yYW5nZTsKPj4gICAgYm9vbFtdIGFwcGxlOwo+PiB9Cj4+
Cj4+IFByYWN0aWNhbGx5IGFsbCB0aGUgZXhhbXBsZXMgdGhhdCBJJ3ZlIGNvbWUgYWNyb3NzIG9u
bHkgZGVhbCB3aXRoIHNhdmluZwo+PiBhbmQKPj4gbG9hZGluZyB0ZXh0LCBzbyBJJ20gaGF2aW5n
IGEgaGFyZCB0aW1lIGRlYWxpbmcgd2l0aCBzYXZpbmcvbG9hZGluZwo+PiBhcnJheXMvZmxvYXRz
L2Jvb2xzL2V0Yy4KPj4KPj4gV2hhdCB3b3VsZCBiZSBhIGdvb2Qgd2F5IHRvIGRvIHRoaXM/Cj4+
Cj4+Cj4KPiBJIGtub3cgdGhlcmUgaXMgYSBkb29zdC5zZXJpYWxpemVyCj4gKGh0dHA6Ly9kc291
cmNlLm9yZy9wcm9qZWN0cy9kb29zdC93aWtpL1NlcmlhbGl6ZXIpIGJ1dCBJIG5ldmVyIHVzZWQg
aXQKPiBteXNlbGYgc28gSSBjYW4ndCBzYXkgd2hldGhlciBpdCBmaWxsIGZpdCB5b3UuCj4gSSBh
bSBhbHNvIHByb3RvdHlwaW5nIGFub3RoZXIgb25lIGF0IHRoZSBtb21lbnQsIGJ1dCBJIGRvbid0
IGtub3cgaG93IGZhcgo+IHdpbGwgaXQgdGFrZS4KPgoKdGVhbTB4ZiBoYXMgYSBsaWJyYXJ5IGNh
bGxlZCB4cG9zZQp3aGljaCBkb2VzIHNlcmlhbGl6YXRpb24gdG8gYW5kIGZyb20gYSBiaW5hcnkg
Zm9ybWF0LgoKQ2FuIGJlIGZvdW5kIGhlcmU6Cmh0dHA6Ly90ZWFtMHhmLmNvbTo4MDgwL3hmL2Zp
bGUvMWViNDNmMDY1N2VjL3hwb3NlLwoKSXQncyBwYXJ0IG9mIHhmLCB3aGljaCB5b3UgY2FuIGdl
dCB1c2luZyBoZzoKaGcgY2xvbmUgaHR0cDovL3RlYW0weGYuY29tOjgwODAveGYKCgotLWJiCg==
Nov 16 2008
parent reply "nobody" <somebody somewhere.com> writes:
"Bill Baxter" <wbaxter gmail.com> wrote in message 
news:mailman.5.1226871464.22690.digitalmars-d-learn puremagic.com...

 On Mon, Nov 17, 2008 at 5:53 AM, Denis Koroskin <2korden gmail.com> wrote:

 16.11.08  18:55 nobody  ϣ  ():


 I would like to be able to save and load a lot of data to/from a file. 
 (in
 D1)
 For example a struct like this:

 struct Fruit
 {
    int banana;
    double[][] orange;
    bool[] apple;
 }

 Practically all the examples that I've come across only deal with saving
 and
 loading text, so I'm having a hard time dealing with saving/loading
 arrays/floats/bools/etc.

 What would be a good way to do this?


 I know there is a doost.serializer
 (http://dsource.org/projects/doost/wiki/Serializer) but I never used it
 myself so I can't say whether it fill fit you.
 I am also prototyping another one at the moment, but I don't know how far
 will it take.


 team0xf has a library called xpose
 which does serialization to and from a binary format.

 Can be found here:
 http://team0xf.com:8080/xf/file/1eb43f0657ec/xpose/

 It's part of xf, which you can get using hg:
 hg clone http://team0xf.com:8080/xf


 --bb


Oh, I've also used some other team0xf stuff, and that worked splendidly, so 
I'll check it out.
Thanks.
Nov 17 2008
parent "Bill Baxter" <wbaxter gmail.com> writes:
T24gTW9uLCBOb3YgMTcsIDIwMDggYXQgMTE6MDcgUE0sIG5vYm9keSA8c29tZWJvZHlAc29tZXdo
ZXJlLmNvbT4gd3JvdGU6Cj4KPiAiQmlsbCBCYXh0ZXIiIDx3YmF4dGVyQGdtYWlsLmNvbT4gd3Jv
dGUgaW4gbWVzc2FnZQo+IG5ld3M6bWFpbG1hbi41LjEyMjY4NzE0NjQuMjI2OTAuZGlnaXRhbG1h
cnMtZC1sZWFybkBwdXJlbWFnaWMuY29tLi4uCj4+IE9uIE1vbiwgTm92IDE3LCAyMDA4IGF0IDU6
NTMgQU0sIERlbmlzIEtvcm9za2luIDwya29yZGVuQGdtYWlsLmNvbT4gd3JvdGU6Cj4+PiAxNi4x
MS4wOCDXIDE4OjU1IG5vYm9keSDXINPXz6PNINDJ09jNxSDQydPBzCjBKToKPj4+Cj4+Pj4gSSB3
b3VsZCBsaWtlIHRvIGJlIGFibGUgdG8gc2F2ZSBhbmQgbG9hZCBhIGxvdCBvZiBkYXRhIHRvL2Zy
b20gYSBmaWxlLgo+Pj4+IChpbgo+Pj4+IEQxKQo+Pj4+IEZvciBleGFtcGxlIGEgc3RydWN0IGxp
a2UgdGhpczoKPj4+Pgo+Pj4+IHN0cnVjdCBGcnVpdAo+Pj4+IHsKPj4+PiAgICBpbnQgYmFuYW5h
Owo+Pj4+ICAgIGRvdWJsZVtdW10gb3JhbmdlOwo+Pj4+ICAgIGJvb2xbXSBhcHBsZTsKPj4+PiB9
Cj4+Pj4KPj4+PiBQcmFjdGljYWxseSBhbGwgdGhlIGV4YW1wbGVzIHRoYXQgSSd2ZSBjb21lIGFj
cm9zcyBvbmx5IGRlYWwgd2l0aCBzYXZpbmcKPj4+PiBhbmQKPj4+PiBsb2FkaW5nIHRleHQsIHNv
IEknbSBoYXZpbmcgYSBoYXJkIHRpbWUgZGVhbGluZyB3aXRoIHNhdmluZy9sb2FkaW5nCj4+Pj4g
YXJyYXlzL2Zsb2F0cy9ib29scy9ldGMuCj4+Pj4KPj4+PiBXaGF0IHdvdWxkIGJlIGEgZ29vZCB3
YXkgdG8gZG8gdGhpcz8KPj4+Pgo+Pj4+Cj4+Pgo+Pj4gSSBrbm93IHRoZXJlIGlzIGEgZG9vc3Qu
c2VyaWFsaXplcgo+Pj4gKGh0dHA6Ly9kc291cmNlLm9yZy9wcm9qZWN0cy9kb29zdC93aWtpL1Nl
cmlhbGl6ZXIpIGJ1dCBJIG5ldmVyIHVzZWQgaXQKPj4+IG15c2VsZiBzbyBJIGNhbid0IHNheSB3
aGV0aGVyIGl0IGZpbGwgZml0IHlvdS4KPj4+IEkgYW0gYWxzbyBwcm90b3R5cGluZyBhbm90aGVy
IG9uZSBhdCB0aGUgbW9tZW50LCBidXQgSSBkb24ndCBrbm93IGhvdyBmYXIKPj4+IHdpbGwgaXQg
dGFrZS4KPj4+Cj4+Cj4+IHRlYW0weGYgaGFzIGEgbGlicmFyeSBjYWxsZWQgeHBvc2UKPj4gd2hp
Y2ggZG9lcyBzZXJpYWxpemF0aW9uIHRvIGFuZCBmcm9tIGEgYmluYXJ5IGZvcm1hdC4KPj4KPj4g
Q2FuIGJlIGZvdW5kIGhlcmU6Cj4+IGh0dHA6Ly90ZWFtMHhmLmNvbTo4MDgwL3hmL2ZpbGUvMWVi
NDNmMDY1N2VjL3hwb3NlLwo+Pgo+PiBJdCdzIHBhcnQgb2YgeGYsIHdoaWNoIHlvdSBjYW4gZ2V0
IHVzaW5nIGhnOgo+PiBoZyBjbG9uZSBodHRwOi8vdGVhbTB4Zi5jb206ODA4MC94Zgo+Pgo+Pgo+
PiAtLWJiCj4+Cj4KPiBPaCwgSSd2ZSBhbHNvIHVzZWQgc29tZSBvdGhlciB0ZWFtMHhmIHN0dWZm
LCBhbmQgdGhhdCB3b3JrZWQgc3BsZW5kaWRseSwgc28KPiBJJ2xsIGNoZWNrIGl0IG91dC4KPiBU
aGFua3MuCgpJIHNob3VsZCBtZW50aW9uIHRoYXQgaXQncyBub3QgcXVpdGUgZG9uZSB5ZXQuICBC
dXQgbWFueSBmYWlybHkgdHJpY2t5CnRoaW5ncyBkbyB3b3JrIGFscmVhZHkuICBGb3IgaW5zdGFu
Y2UsIEknbSBwcmV0dHkgc3VyZSB0aGF0CmRlLXNlcmlhbGl6YXRpb24gb2Ygc3ViY2xhc3NlcyB3
YXMgd29ya2luZyBwcm9wZXJseSBsYXN0IEkgdHJpZWQgaXQuClRoYXQgaXMsIGlmIHlvdSB3cml0
ZSBvdXQgYSBEZXJpdmVkLCBhbmQgdGhlbiB0cnkgdG8gcmVhZCBiYWNrIGEgQmFzZSwKdGhlIEJh
c2UgeW91IGdldCBiYWNrIHdpbGwgYWN0dWFsbHkgYmUgYSBwb2ludGVyIHRvIGEgRGVyaXZlZC4g
ICBBbmQKaWYgeW91IHdyaXRlIG91dCB0aGUgc2FtZSBpbnN0YW5jZSBtdWx0aXBsZSB0aW1lcyBp
dCBvbmx5IGdldHMgc2F2ZWQKb25jZSwgYW5kIGdldHMgbG9hZGVkIGJhY2sganVzdCBvbmNlIGFs
c28sIGFuZCB0aGUgb3RoZXIgTi0xIGNvcGllcwphcmUganVzdCBhICBjb3B5IG9mIHRoZSBwb2lu
dGVyIHRvIHRoZSBvbmUgbG9hZGVkIGluc3RhbmNlLiAgIEknbQpwcmV0dHkgc3VyZSBhbGwgdGhh
dCB3YXMgd29ya2luZyBsYXN0IHRpbWUgSSB0cmllZCBpdC4gIFRoZXJlIHdhcwpzb21ldGhpbmcg
aXQgY291bGRuJ3QgZG8gdGhhdCBJIHdhcyB3YW50aW5nLCBidXQgSSBkb24ndCBxdWl0ZSByZWNh
bGwKbm93Li4uCgpPbiB0aGUgZG93biBzaWRlLCBpdCdzIG5vdCBleGFjdGx5IGVhc3kgY29kZSB0
byBleHRlbmQgYW5kL29yIGRlYnVnLgoKLS1iYgo=
Nov 17 2008
prev sibling parent Derek Parnell <derek psych.ward> writes:
On Sun, 16 Nov 2008 16:55:16 +0100, nobody wrote:


 I would like to be able to save and load a lot of data to/from a file. (in 
 D1)
 For example a struct like this:
 
 struct Fruit
 {
     int banana;
     double[][] orange;
     bool[] apple;
 }
 
 Practically all the examples that I've come across only deal with saving and 
 loading text, so I'm having a hard time dealing with saving/loading 
 arrays/floats/bools/etc.
 
 What would be a good way to do this?


It depends ... do you wish to save the data as a text representation or in
binary form?

Text takes up more room but can be more portable to different applications.

Binary is smaller and usually a lot faster.

For text output, you can format it using one of the popular styles, such as
XML, JSON, or CSV, or you can invent something more suitable to your
specific requirements. These generally require you to convert each piece of
data to a string and wrap that with 'start'/'end' markers.

For binary, you need to decide if 'endian'-ness is an issue or not. If you
are happy with the binary layout of the struct data in RAM and you can NOT
going to be using the data in other CPU architectures, then the simplest is
to just copy out the RAM bytes to disk. Of course, you have to add some
'structure' info to deal with the variable-length arrays but that can be as
simple as prefixing the data with the length value.


-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Nov 16 2008