www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Reordered class fields?

reply "bearophile" <bearophileHUGS lycos.com> writes:
D classes are free to reorder their fields, so maybe Bar1 should 
reorder its fields as Bar3, to save 4 bytes for each instance on 
32 bit systems:


class Bar1 {
     void Hello() {}
     float f;
     double d;
}
class Bar2 {
     void Hello() {}
     align(4) float f;
     align(4) double d;
}
class Bar3 {
     void Hello() {}
     double d;
     float f;
}
void main() {
     pragma(msg, __traits(classInstanceSize, Bar1)); // 24
     pragma(msg, __traits(classInstanceSize, Bar2)); // 20
     pragma(msg, __traits(classInstanceSize, Bar3)); // 20
}



This benchmark shows that if you allocate the class instances on 
the heap one at a time the total amount of memory used is the 
same for the various Bar (maybe because of the GC), so that 
optimization is useful for emplace() only and similar in-place 
allocations:


class Bar1 {
     void Hello() {}
     float f;
     double d;
}
class Bar2 {
     void Hello() {}
     align(4) float f;
     align(4) double d;
}
class Bar3 {
     void Hello() {}
     double d;
     float f;
}
int main() {
     pragma(msg, __traits(classInstanceSize, Bar1)); // 24
     pragma(msg, __traits(classInstanceSize, Bar2)); // 20
     pragma(msg, __traits(classInstanceSize, Bar3)); // 20
     //--------------
     //auto arr = new Bar1[1_000_000]; // 38.2 MB
     //auto arr = new Bar2[1_000_000]; // 38.2 MB
     auto arr = new Bar3[1_000_000]; // 38.2  MB
     foreach (ref a; arr)
         a = new typeof(arr[0])();
     int count;
     foreach (i; 0 .. 500_000_000) count++;
     return count;
}


So is such class field reordering worth an enhancement request in 
Bugzilla?

Bye,
bearophile
Oct 21 2012
next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2012-36-22 01:10, bearophile <bearophileHUGS lycos.com> wrote:


 This benchmark shows that if you allocate the class instances on the  
 heap one at a time the total amount of memory used is the same for the  
 various Bar (maybe because of the GC), so that optimization is useful  
 for emplace() only and similar in-place allocations


The current GC always allocates a power of two, with a minimum of 16
bytes. You should see an effect if you make a class that will be above
such a threshold without reordering, and below with.



 So is such class field reordering worth an enhancement request in  
 Bugzilla?


Nothing bad can come of it.

-- 
Simen
Oct 22 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Simen Kjaeraas:


 The current GC always allocates a power of two, with a minimum 
 of 16
 bytes. You should see an effect if you make a class that will 
 be above such a threshold without reordering, and below with.


Right.



 Nothing bad can come of it.


OK :-)

----------------------

Jacob Carlborg:


 Are D allowed to reorder the class fields?


This page says:
http://dlang.org/class.html


The D compiler is free to rearrange the order of fields in a 
class to optimally pack them in an implementation-defined 
manner. Consider the fields much like the local variables in a 
function - the compiler assigns some to registers and shuffles 
others around all to get the optimal stack frame layout. This 
frees the code designer to organize the fields in a manner that 
makes the code more readable rather than being forced to 
organize it according to machine optimization rules. Explicit 
control of field layout is provided by struct/union types, not 
classes.<


Bye,
bearophile
Oct 22 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-22 10:48, bearophile wrote:


 This page says:
 http://dlang.org/class.html


 The D compiler is free to rearrange the order of fields in a class to
 optimally pack them in an implementation-defined manner. Consider the
 fields much like the local variables in a function - the compiler
 assigns some to registers and shuffles others around all to get the
 optimal stack frame layout. This frees the code designer to organize
 the fields in a manner that makes the code more readable rather than
 being forced to organize it according to machine optimization rules.
 Explicit control of field layout is provided by struct/union types,
 not classes.<




Ok, I didn't know that.

-- 
/Jacob Carlborg
Oct 22 2012
parent reply "Peter Summerland" <p.summerland gmail.com> writes:
On Monday, 22 October 2012 at 11:06:50 UTC, Jacob Carlborg wrote:

 On 2012-10-22 10:48, bearophile wrote:


 This page says:
 http://dlang.org/class.html


 The D compiler is free to rearrange the order of fields in a 
 class to
 optimally pack them in an implementation-defined manner. 
 Consider the
 fields much like the local variables in a function - the 
 compiler
 assigns some to registers and shuffles others around all to 
 get the
 optimal stack frame layout. This frees the code designer to 
 organize
 the fields in a manner that makes the code more readable 
 rather than
 being forced to organize it according to machine optimization 
 rules.
 Explicit control of field layout is provided by struct/union 
 types,
 not classes.<




 Ok, I didn't know that.


The order of the fields is rearranged for packing. Does that 
affect the tupleof property? The example in 
http://dlang.org/class.html for Class properties tulpleof seems 
to implie that the the fields the returned Expression Tuple are 
arranged in lexical order (i.e., as defined by the programmer in 
the class definition). Is this always true for classes? What 
about structs?
Oct 30 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-31 02:53, Peter Summerland wrote:


 The order of the fields is rearranged for packing. Does that affect the
 tupleof property? The example in http://dlang.org/class.html for Class
 properties tulpleof seems to implie that the the fields the returned
 Expression Tuple are arranged in lexical order (i.e., as defined by the
 programmer in the class definition). Is this always true for classes?
 What about structs?


I don't know. But I would not count on the order of tupleof. I would 
consider that implementation defined.

-- 
/Jacob Carlborg
Oct 31 2012
parent reply "Peter Summerland" <p.summerland gmail.com> writes:
On Wednesday, 31 October 2012 at 07:19:19 UTC, Jacob Carlborg 
wrote:

 On 2012-10-31 02:53, Peter Summerland wrote:


 The order of the fields is rearranged for packing. Does that 
 affect the
 tupleof property? The example in http://dlang.org/class.html 
 for Class
 properties tulpleof seems to implie that the the fields the 
 returned
 Expression Tuple are arranged in lexical order (i.e., as 
 defined by the
 programmer in the class definition). Is this always true for 
 classes?
 What about structs?


 I don't know. But I would not count on the order of tupleof. I 
 would consider that implementation defined.


Thanks for the help.

Should the the following example, taken from the D Language 
Reference, be considered incorrect or at least misleading? It 
clearly depends on lexical ordering of the returned fields:

Class Properties

The .tupleof property returns an ExpressionTuple of all the 
fields in the class, excluding the hidden fields and the fields 
in the base class.

class Foo { int x; long y; }
void test(Foo foo) {
   foo.tupleof[0] = 1; // set foo.x to 1
   foo.tupleof[1] = 2; // set foo.y to 2
   foreach (x; foo.tupleof)
     writef(x);        // prints 12
}


It would be nice if the Language Reference was specific on this 
point.  I am aware that the order of the members returned by 
__traits(allMembers, D) is not defined (per the LR). But that is 
a larger, more complex list.

I am just beginning with D, but I think having the tupleof 
property for classes and structs return their fields in lexical 
order might be useful.

E.g., I have written a "scan" function to load up a struct or 
class from a row returned by a database query. I have, say, 
scan!"x,y,z"(obj) to load data into fields x,y,z of obj. Based on 
the example above and by experimenting, it appears that at least 
for dmd running on Linux, that the fields returned by tupleof are 
indeed in lexical order.  That allowed me to have a "" default 
for the string of field names which indicates that all the fields 
should be loaded. I.e., I have scan!""(obj), which can be written 
scan(obj). Again, it seems to work for simple classes and structs 
with Dmd on Ubuntu.

Don't get me wrong -- I'll be happy without the default version 
if that is the answer. I'm not suggesting any changes.
Oct 31 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-31 09:14, Peter Summerland wrote:


 Thanks for the help.

 Should the the following example, taken from the D Language Reference,
 be considered incorrect or at least misleading? It clearly depends on
 lexical ordering of the returned fields:

 Class Properties

 The .tupleof property returns an ExpressionTuple of all the fields in
 the class, excluding the hidden fields and the fields in the base class.

 class Foo { int x; long y; }
 void test(Foo foo) {
    foo.tupleof[0] = 1; // set foo.x to 1
    foo.tupleof[1] = 2; // set foo.y to 2
    foreach (x; foo.tupleof)
      writef(x);        // prints 12
 }


I would at least consider it bad practice. Even if the compiler 
guarantees a specific order it's quite easy for the developer to change 
the order of the fields in Foo and then have "test" break.

I would suggest one always access the fields by name. Example:

https://github.com/jacob-carlborg/orange/blob/master/orange/serialization/Serializer.d#L1448

https://github.com/jacob-carlborg/orange/blob/master/orange/util/Reflection.d#L260


 It would be nice if the Language Reference was specific on this point.
 I am aware that the order of the members returned by
 __traits(allMembers, D) is not defined (per the LR). But that is a
 larger, more complex list.

 I am just beginning with D, but I think having the tupleof property for
 classes and structs return their fields in lexical order might be useful.


1. It wouldn't hurt if it was clearly specified in the language 
specification.
2. You can always sort the list
3. I wouldn't count on the order regardless, see above


 E.g., I have written a "scan" function to load up a struct or class from
 a row returned by a database query. I have, say, scan!"x,y,z"(obj) to
 load data into fields x,y,z of obj. Based on the example above and by
 experimenting, it appears that at least for dmd running on Linux, that
 the fields returned by tupleof are indeed in lexical order.  That
 allowed me to have a "" default for the string of field names which
 indicates that all the fields should be loaded. I.e., I have
 scan!""(obj), which can be written scan(obj). Again, it seems to work
 for simple classes and structs with Dmd on Ubuntu.


Indeed, DMD seems to return them in lexical order, but again, I wouldn't 
rely on the, see above.


 Don't get me wrong -- I'll be happy without the default version if that
 is the answer. I'm not suggesting any changes.




-- 
/Jacob Carlborg
Oct 31 2012
parent "Peter Summerland" <p.summerland gmail.com> writes:
On Wednesday, 31 October 2012 at 08:23:26 UTC, Jacob Carlborg 
wrote:

 On 2012-10-31 09:14, Peter Summerland wrote:


 Thanks for the help.

 Should the the following example, taken from the D Language 
 Reference,
 be considered incorrect or at least misleading? It clearly 
 depends on
 lexical ordering of the returned fields:

 Class Properties

 The .tupleof property returns an ExpressionTuple of all the 
 fields in
 the class, excluding the hidden fields and the fields in the 
 base class.

 class Foo { int x; long y; }
 void test(Foo foo) {
   foo.tupleof[0] = 1; // set foo.x to 1
   foo.tupleof[1] = 2; // set foo.y to 2
   foreach (x; foo.tupleof)
     writef(x);        // prints 12
 }


 I would at least consider it bad practice. Even if the compiler 
 guarantees a specific order it's quite easy for the developer 
 to change the order of the fields in Foo and then have "test" 
 break.



 I would suggest one always access the fields by name. Example:

 https://github.com/jacob-carlborg/orange/blob/master/orange/serialization/Serializer.d#L1448

 https://github.com/jacob-carlborg/orange/blob/master/orange/util/Reflection.d#L260


Thanks for the links. I can probably use some of the Reflection 
module in my code. I am making a very minimal sqlite3 library, 
which mostly just manages connections and statements to ensure 
they are closed and finalized automatically. I did add bind, step 
and scan methods to the Sqlite3Stmt struct but that's about it. 
Other than that you have to use the native c interface.

More importantly, I think that it is really very instructive to 
read over your code.

Re breaking test. You are right of course. But in my case the POD 
object is meant only to be used to package a row of data. The 
client cannot change it. Here is an example of how I might make a 
function to query a database

// POD structure to package rows returned by query
struct Employee
{
   int id_num;
   string name;
   int rank;
}

// returns an array of Employee objs for employees with given rank
Employee[] getEmployees(Sqlite3 db, int rank)
{
   static string query =
     "select e_num, e_name, e_rank from employee where e_rank = ?";

   auto app = appender!(Employee[]);
   auto stmt = db.prepare(query);
   stmt.bind(rank);
   stmt.scanFields!("id_num,name,rank")(app); // SAFE
   //stmt.scanFields(app);  // WORKS FOR POD STRUCTS LIKE EMPLOYEE
   return app.data;
}

It's not the greatests: You have to change the query if the 
database changes or if you want to change Employee, etc. But at 
least the query and Employee are defined locally in your code.


 It would be nice if the Language Reference was specific on 
 this point.
 I am aware that the order of the members returned by
 __traits(allMembers, D) is not defined (per the LR). But that 
 is a
 larger, more complex list.

 I am just beginning with D, but I think having the tupleof 
 property for
 classes and structs return their fields in lexical order might 
 be useful.


 1. It wouldn't hurt if it was clearly specified in the language 
 specification.
 2. You can always sort the list
 3. I wouldn't count on the order regardless, see above


 E.g., I have written a "scan" function to load up a struct or 
 class from
 a row returned by a database query. I have, say, 
 scan!"x,y,z"(obj) to
 load data into fields x,y,z of obj. Based on the example above 
 and by
 experimenting, it appears that at least for dmd running on 
 Linux, that
 the fields returned by tupleof are indeed in lexical order.  
 That
 allowed me to have a "" default for the string of field names 
 which
 indicates that all the fields should be loaded. I.e., I have
 scan!""(obj), which can be written scan(obj). Again, it seems 
 to work
 for simple classes and structs with Dmd on Ubuntu.


 Indeed, DMD seems to return them in lexical order, but again, I 
 wouldn't rely on the, see above.


 Don't get me wrong -- I'll be happy without the default 
 version if that
 is the answer. I'm not suggesting any changes.
Oct 31 2012
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Peter Summerland:


 The order of the fields is rearranged for packing. Does that 
 affect the tupleof property? The example in 
 http://dlang.org/class.html for Class properties tulpleof seems 
 to implie that the the fields the returned Expression Tuple are 
 arranged in lexical order (i.e., as defined by the programmer 
 in the class definition). Is this always true for classes?


I have added your question here:
http://d.puremagic.com/issues/show_bug.cgi?id=8873



 What about structs?


D structs are almost PODs (if they are nested inside a function 
and they are not defined as static they have one more field at 
the beginning), so tupleof for structs gives the fields in 
definition order.

Bye,
bearophile
Oct 31 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-10-22 01:36, bearophile wrote:

 D classes are free to reorder their fields, so maybe Bar1 should reorder
 its fields as Bar3, to save 4 bytes for each instance on 32 bit systems:


Are D allowed to reorder the class fields? What happens then to binary 
compatibility?

-- 
/Jacob Carlborg
Oct 22 2012