www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - More binary I/O problems

reply Charles Hixson via Digitalmars-d-learn writes:
 On 03/25/2016 11:32 AM, Adam D. Ruppe via Digitalmars-d-learn wrote:

 On Friday, 25 March 2016 at 18:25:28 UTC, Charles Hixson wrote:

 But when I try to cast a Chnk to a ubyte[], I get an error, and


 rawWrite takes a generic array of anything... you should be able to 
 rawWrite((&your_object)[0 .. 1])


 Thanks, that compiled.  It's running a test now...and I'm leaving it 
 for a few hours.


And that worked, but suddenly (after a compiler upgrade, did that 
matter? I'd also changed the program, though in ways that shouldn't have 
affected this.) it stopped working with the message:
let4a.d(138): Error: no [] operator overload for type Chnk

Well, there isn't, Chnk is just a struct.
The offending line was:
f.rawWrite (&(val)[0..1]);
which I already though was unreasonably convoluted.  I just wanted 
something like:
f.write(&val, val.sizeof);
which used to work, but the corresponding read doesn't seem to exist 
anymore, so I can't test it.  It's documented, but when I try to use it:
f.read(&c, c.sizeof);
results in:
let4a.d(172): Error: template std.file.read cannot deduce function from 
argument types !()(File, Chnk*, ulong), candidates are:
/usr/include/dmd/phobos/std/file.d(222):        std.file.read(R)(R name, 
size_t upTo = size_t.max) if (isInputRange!R && 
isSomeChar!(ElementEncodingType!R) && !isConvertibleToString!R)
/usr/include/dmd/phobos/std/file.d(248): std.file.read(R)(auto ref R 
name, size_t upTo = size_t.max) if (isConvertibleToString!R)

And I'm trying to do a binary read, not a string read.  (I also tried it 
with f.read(c, c.sizeof); and got equivalent error messages.

And the struct was (abbreviated as noted):
struct    Chnk
{
   char[20]    wrd;
   ubyte        length;
   ulong        id;

     this    (ulong id,    string wrd)   {  ...   }

     uint    hCode ()    {    return    hash(wrdStr) % fcnt;    }

     string    toString ()    {   ... }

     string    wrdStr()    {  ... }
}    //    struct    Chnk

There are no indirections.  It's just a simple struct that I want to 
read and write to a randomly accessible file.  This used to be easy, but 
now I keep running into problems.

FWIW, it's stored in a ChnkTbl, thus:
const    ushort    fcnt    =    128;
alias    Chnk[string]    ChnkTbl;
ChnkTbl[fcnt]    chnks;

with the idea that each ChnkTbl would be stored in a separate file. But 
it's the binary read/write I keep having problems with.
Apr 08 2016
parent reply Basile B. <b2.temp gmx.com> writes:
On Friday, 8 April 2016 at 20:58:06 UTC, Charles Hixson wrote:

[...]


 And that worked, but suddenly (after a compiler upgrade, did 
 that matter? I'd also changed the program, though in ways that 
 shouldn't have affected this.) it stopped working with the 
 message:
 let4a.d(138): Error: no [] operator overload for type Chnk

 [...]


you can cast too (void*)

struct Chnk
{
     char[20] wrd;
     ubyte  length;
     ulong  id;
     this (ulong id, string wrd) {}
     uint hCode () {return  0;}
     string toString () { return "";}
     string wrdStr() {return "";}
}

void main()
{
     import std.file: read, write;
     Chnk c;
     write("a.txt", (cast(void*)&c)[0..Chnk.sizeof]);
     auto buff = read("a.txt", Chnk.sizeof);
     (cast(void*)&c)[0..Chnk.sizeof] = buff[0..Chnk.sizeof];
}
Apr 08 2016
parent reply Charles Hixson via Digitalmars-d-learn writes:
On 04/08/2016 07:42 PM, Basile B. via Digitalmars-d-learn wrote:

 On Friday, 8 April 2016 at 20:58:06 UTC, Charles Hixson wrote:

 [...]


 And that worked, but suddenly (after a compiler upgrade, did that 
 matter? I'd also changed the program, though in ways that shouldn't 
 have affected this.) it stopped working with the message:
 let4a.d(138): Error: no [] operator overload for type Chnk

 [...]


 you can cast too (void*)

 struct Chnk
 {
     char[20] wrd;
     ubyte  length;
     ulong  id;
     this (ulong id, string wrd) {}
     uint hCode () {return  0;}
     string toString () { return "";}
     string wrdStr() {return "";}
 }

 void main()
 {
     import std.file: read, write;
     Chnk c;
     write("a.txt", (cast(void*)&c)[0..Chnk.sizeof]);
     auto buff = read("a.txt", Chnk.sizeof);
     (cast(void*)&c)[0..Chnk.sizeof] = buff[0..Chnk.sizeof];
 }


Thanks.  That syntax looks like it would work, but gee!  The syntax is 
getting worse and worse.
OTOH, I notice that you're using a file name rather than a File. Perhaps 
that's my problem, but if so how do I specify that I want a binary 
read/write rather than a text one.  (I notice that you named the file 
"a.txt", and I explicitly do not want to do a text conversion.  That 
would result in different numbers taking up different numbers of bytes, 
and so random addressing wouldn't work.)  Also read indicates that it 
tries to read in the entire file, so perhaps it's not intended for this 
purpose, and I should use rawRead/rawWrite which should work with that 
hideous cast & copy to/from ubyte[] step.

I suppose I could code the I/O in C...that way I might need to use 
foreign calling conventions, but the I/O would be straightforwards. And 
since everything is fixed length it should be an easy transition...
I want to use D because the garbage collection is useful and the 
handling of unicode strings/chars is hard to beat.  It would also mean I 
didn't need to use ZMQ when I got around to parallel execution.  But 
this code was working, and now it suddenly isn't.  I know you pointed 
out that I could get around this by copying it to a buffer, and writing 
out the buffer, and then reading in the buffer, and then copying the 
buffer to the object, with lots of casting. And I admit that it's 
doable, but I'm having trouble believing how ugly it is.  Not to mention 
that this is needed because code that used to be working stopped 
working.  Fortunately I was still developing this area, or it might have 
broken totally and I wouldn't have known until I tried to put the pieces 
together.
Apr 08 2016
parent reply Basile B. <b2.temp gmx.com> writes:
On Saturday, 9 April 2016 at 03:15:58 UTC, Charles Hixson wrote:

 On 04/08/2016 07:42 PM, Basile B. via Digitalmars-d-learn wrote:

 On Friday, 8 April 2016 at 20:58:06 UTC, Charles Hixson wrote:

 [...]


 And that worked, but suddenly (after a compiler upgrade, did 
 that matter? I'd also changed the program, though in ways 
 that shouldn't have affected this.) it stopped working with 
 the message:
 let4a.d(138): Error: no [] operator overload for type Chnk

 [...]


 you can cast too (void*)


 Thanks.  That syntax looks like it would work, but gee!  The 
 syntax is getting worse and worse.
 OTOH, I notice that you're using a file name rather than a 
 File. Perhaps that's my problem,


I've missed the first part of the topic, you can probably do it 
also with a file. The point of the answer is just to show the 
cast.


 but if so how do I specify that I want a binary read/write 
 rather than a text one.  (I notice that you named the file 
 "a.txt", and I explicitly do not want to do a text conversion.


It's a binary file, the extension is meaningless.
Apr 08 2016
parent Charles Hixson via Digitalmars-d-learn writes:
On 04/08/2016 09:25 PM, Basile B. via Digitalmars-d-learn wrote:

 On Saturday, 9 April 2016 at 03:15:58 UTC, Charles Hixson wrote:

 On 04/08/2016 07:42 PM, Basile B. via Digitalmars-d-learn wrote:

 On Friday, 8 April 2016 at 20:58:06 UTC, Charles Hixson wrote:

 [...]


 And that worked, but suddenly (after a compiler upgrade, did that 
 matter? I'd also changed the program, though in ways that shouldn't 
 have affected this.) it stopped working with the message:
 let4a.d(138): Error: no [] operator overload for type Chnk

 [...]


 you can cast too (void*)


 Thanks.  That syntax looks like it would work, but gee!  The syntax 
 is getting worse and worse.
 OTOH, I notice that you're using a file name rather than a File. 
 Perhaps that's my problem,


 I've missed the first part of the topic, you can probably do it also 
 with a file. The point of the answer is just to show the cast.


 but if so how do I specify that I want a binary read/write rather 
 than a text one.  (I notice that you named the file "a.txt", and I 
 explicitly do not want to do a text conversion.


 It's a binary file, the extension is meaningless.


Well, the extension is always meaningless except as a mnemonic.  But it 
appears that read/write aren't intended for use with fixed block files, 
so rawRead/rawWrite would be the better choice.
Apr 09 2016