www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - rawRead using a struct with variable leght

reply ade90036 <andrea.rizzini gmail.com> writes:
Hi everyone,

I'm trying out Dland, always been and have been a big fan. So to 
give it a good run i wanted to create is a java class parser, 
based on the spec released here. ( 
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html.)

The class file can be represented in the following "struct" like:

ClassFile {
     u4             magic;
     u2             minor_version;
     u2             major_version;
     u2             constant_pool_count;
     cp_info        constant_pool[constant_pool_count-1];
     u2             access_flags;
     u2             this_class;
     u2             super_class;
     u2             interfaces_count;
     u2             interfaces[interfaces_count];
     u2             fields_count;
     field_info     fields[fields_count];
     u2             methods_count;
     method_info    methods[methods_count];
     u2             attributes_count;
     attribute_info attributes[attributes_count];
}

where:
u4 == ubyte[4] --> integer
u2 == ubyte[2] --> short
u1 == ubyte[1] --> byte

I have the first 4 fields parsing, however i stumble upon an 
example where you can use a rawRead() with a struct, therefore 
representing the entire structure and then reading from rawRead() 
like

````d
ClassFile[1] classFileStruct;
f.rawRead(claddFileStruct);
````

Unfortunately the struct doesn't know at compile time what the 
size of the constant_pool array, or at-least was not able to 
specify it dynamically.

What is the best approach to go about parsing such structure?

Should i have structs to represent the blocks of fixed fields 
from the structure and then parsing each variable (length) 
structure manually?


Something like this?

````
ClassHeader[1] classHeader;
f.rawRead(classHeader);

CpInfo[] cpInfo = new CpInfo[classHeader.constant_pool_count];
f.rawRead(cpInfo);
`````

Really appreciate any knowledgeable suggestions.

Regards

Ade
Jun 05 2017
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 5 June 2017 at 16:04:28 UTC, ade90036 wrote:


 Unfortunately the struct doesn't know at compile time what the 
 size of the constant_pool array, or at-least was not able to 
 specify it dynamically.


  It also won't know ahead of time how many fields, methods or 
attributes you have either.

  First I'd say all the arrays will have to be redefined to use 
[], rather than a fixed size.

  Glancing at the chapter information, you're probably not going 
to have an easy time, and will have to simply have to fill in the 
fields individually in order followed by allocating the arrays 
and probably filling/loading those immediately (although it's 
possible the array contents are done at the end, though it seems 
doubtful).
Jun 05 2017
parent reply ade90036 <andrea.rizzini gmail.com> writes:
On Monday, 5 June 2017 at 16:30:53 UTC, Era Scarecrow wrote:

 On Monday, 5 June 2017 at 16:04:28 UTC, ade90036 wrote:


 Unfortunately the struct doesn't know at compile time what the 
 size of the constant_pool array, or at-least was not able to 
 specify it dynamically.


  It also won't know ahead of time how many fields, methods or 
 attributes you have either.

  First I'd say all the arrays will have to be redefined to use 
 [], rather than a fixed size.

  Glancing at the chapter information, you're probably not going 
 to have an easy time, and will have to simply have to fill in 
 the fields individually in order followed by allocating the 
 arrays and probably filling/loading those immediately (although 
 it's possible the array contents are done at the end, though it 
 seems doubtful).


Thanks for the reply.

I guess i didnt pick such an easy task afterall.

I shall parse each fields individually.

Thanks again..
Jun 07 2017
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Wed, Jun 07, 2017 at 06:24:22PM +0000, ade90036 via Digitalmars-d-learn
wrote:

 On Monday, 5 June 2017 at 16:30:53 UTC, Era Scarecrow wrote:

 On Monday, 5 June 2017 at 16:04:28 UTC, ade90036 wrote:
 

 Unfortunately the struct doesn't know at compile time what the
 size of the constant_pool array, or at-least was not able to
 specify it dynamically.


 
  It also won't know ahead of time how many fields, methods or
  attributes you have either.
 
  First I'd say all the arrays will have to be redefined to use [],
  rather than a fixed size.
 
  Glancing at the chapter information, you're probably not going to
  have an easy time, and will have to simply have to fill in the
  fields individually in order followed by allocating the arrays and
  probably filling/loading those immediately (although it's possible
  the array contents are done at the end, though it seems doubtful).


 
 Thanks for the reply.
 
 I guess i didnt pick such an easy task afterall.
 
 I shall parse each fields individually.


[...]

"Structs" with variable size fields have no direct equivalent in D's
type system, so you'll probably have a hard time mapping this directly.

What you *could* do, though, is to load the data into a ubyte[] buffer,
then create a proxy struct containing arrays where you have
variable-sized fields, with the arrays slicing the ubyte[] buffer
appropriately.  Unfortunately, yes, this means you have to parse the
fields individually in order to construct these slices.


T

-- 
This is a tpyo.
Jun 07 2017
parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Wednesday, 7 June 2017 at 18:31:41 UTC, H. S. Teoh wrote:

 "Structs" with variable size fields have no direct equivalent 
 in D's type system, so you'll probably have a hard time mapping 
 this directly.

 What you *could* do, though, is to load the data into a ubyte[] 
 buffer, then create a proxy struct containing arrays where you 
 have variable-sized fields, with the arrays slicing the ubyte[] 
 buffer appropriately.  Unfortunately, yes, this means you have 
 to parse the fields individually in order to construct these 
 slices.


  I'm reminded a little bit of how I ended up handling the records 
and subrecords for Morrowind files; I ended up creating a range 
type which recognized the different types and returned the 
records, then a second one that cycled through the sub records 
and generated the structs as it went.

  Although those were incredibly simple, it was 2 fields, the name 
of the field and then the length of the whole thing together for 
the record (char, int). For subrecords it was the same, except 
additional int and other string fields, all fixed length, no 
weird dynamic allocation required.

  Unless the arrays are stored/saved after the rest of the data, I 
don't see how you could bulk load the other fields so easily.
Jun 07 2017