www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Packing of Struct Fields

reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
Why is `T.sizeof` 12 instead of 8 when `U.sizeof` is 8 in the 
following example?

struct S
{
     int i;
     bool b;
}

struct T
{
     S s;
     char c;
}

struct U
{
     int i;
     bool b;
     char c;
}

?
Oct 16 2020
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 16.10.20 22:32, Per Nordlöw wrote:

 Why is `T.sizeof` 12 instead of 8 when `U.sizeof` is 8 in the following 
 example?
 
 struct S
 {
      int i;
      bool b;
 }
 
 struct T
 {
      S s;
      char c;
 }
 
 struct U
 {
      int i;
      bool b;
      char c;
 }
 
 ?


S.sizeof: 4 bytes for the int + 1 byte for the bool + 3 bytes padding so 
that the int is aligned = 8 bytes.

T.sizeof: 8 bytes for the S + 1 byte for the char + 3 bytes padding so 
that the S is aligned = 12 bytes.

U.sizeof: 4 bytes for the int + 1 byte for the bool + 1 byte for the 
char + 2 bytes padding so that the int is aligned = 8 bytes.
Oct 16 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/16/20 4:44 PM, ag0aep6g wrote:

 On 16.10.20 22:32, Per Nordlöw wrote:

 Why is `T.sizeof` 12 instead of 8 when `U.sizeof` is 8 in the 
 following example?

 struct S
 {
      int i;
      bool b;
 }

 struct T
 {
      S s;
      char c;
 }

 struct U
 {
      int i;
      bool b;
      char c;
 }

 ?


 
 S.sizeof: 4 bytes for the int + 1 byte for the bool + 3 bytes padding so 
 that the int is aligned = 8 bytes.
 
 T.sizeof: 8 bytes for the S + 1 byte for the char + 3 bytes padding so 
 that the S is aligned = 12 bytes.
 
 U.sizeof: 4 bytes for the int + 1 byte for the bool + 1 byte for the 
 char + 2 bytes padding so that the int is aligned = 8 bytes.


To further explain this -- the padding is added so things like pointer 
arithmetic on an array work.

For example, if you have a T* t, and you say t += 1, you want it to go 
to the next T, not to a misaligned spot.

You can also override this with align keyword. But I don't recommended 
this unless you know what you are doing. Misaligned reads/writes are 
different on different architectures, but even if they work and don't 
crash your program, they are going to be slower.

https://dlang.org/spec/attribute.html#align

-Steve
Oct 16 2020
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 16 October 2020 at 21:26:12 UTC, Steven Schveighoffer 
wrote:

 To further explain this -- the padding is added so things like 
 pointer arithmetic on an array work.


In my code sample above one can only access the first element 
anyhow so I don't understand why this restriction is imposed here.

struct S
{
     int i;
     bool b;
}

struct T
{
     S s; // reinterpreting this as an array can only access this 
first element anyway
     char c; // so why can't this be aligned directly after `s` 
without any padding?
}
Oct 17 2020
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 17 October 2020 at 12:35:37 UTC, Per Nordlöw wrote:

 On Friday, 16 October 2020 at 21:26:12 UTC, Steven 
 Schveighoffer wrote:

 To further explain this -- the padding is added so things like 
 pointer arithmetic on an array work.


 In my code sample above one can only access the first element 
 anyhow so I don't understand why this restriction is imposed 
 here.

 struct S
 {
     int i;
     bool b;
 }

 struct T
 {
     S s; // reinterpreting this as an array can only access 
 this first element anyway
     char c; // so why can't this be aligned directly after `s` 
 without any padding?
 }


So AFAICT the key question becomes:

Can `align`s be inserted in S or/and T so that T is packed to 8 
bytes but still aligned to 8 bytes? I don't see why this 
shouldn't be the default behaviour...
Oct 17 2020
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 17 October 2020 at 12:44:44 UTC, Per Nordlöw wrote:

 Can `align`s be inserted in S or/and T so that T is packed to 8 
 bytes but still aligned to 8 bytes?


Yes. Put an align on the OUTSIDE of the struct you are nesting, 
then put one INSIDE the struct you want the contents packed.


 I don't see why this shouldn't be the default behaviour...


It is generally slower.
Oct 17 2020
prev sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 17 October 2020 at 12:44:44 UTC, Per Nordlöw wrote:

 Can `align`s be inserted in S or/and T so that T is packed to 8 
 bytes but still aligned to 8 bytes? I don't see why this 
 shouldn't be the default behaviour...


I though this would do the trick but not...

     struct S
     {
         int i;                  // 4 bytes
         short s;                // 2 byte
         bool b;                 // 1 byte
     }
     static assert(S.sizeof == 8);
     static assert(S.alignof == 4);
     align(4) struct T
     {
         align(4) S s;
         align(1) char c;
     }
     static assert(T.alignof == 4);
     // TODO: static assert(T.sizeof == 8);

T.sizeof is still 12 bytes, I want it to be 8.
Oct 17 2020
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On 17.10.20 14:35, Per Nordlöw wrote:

 struct S
 {
      int i;
      bool b;
 }
 
 struct T
 {
      S s; // reinterpreting this as an array can only access this first 
 element anyway
      char c; // so why can't this be aligned directly after `s` without 
 any padding?
 }
 


c does come directly after s. The padding between b and c is part of s. 
If you don't want that padding, you can use `align(1)` to define S 
without padding. But then 75% of the ints in an S[] will be misaligned.
Oct 17 2020
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 17 October 2020 at 12:51:21 UTC, ag0aep6g wrote:

 c does come directly after s. The padding between b and c is 
 part of s. If you don't want that padding, you can use 
 `align(1)` to define S without padding. But then 75% of the 
 ints in an S[] will be misaligned.


I understand that. I don't want the alignment of `S` to change. I 
want the padding after `s` in `T` to be avoided and have `c` 
start at byte-offset 7. I don't see why this padding is needed in 
the case where only a single (1-element array of) `S` is stored 
as a field inside another aggregate.

Ali's code prints:

=== Memory layout of 'T' (.sizeof: 12, .alignof: 4) ===
    0: S s
    8: char c
    9: ... 3-byte PADDING
Oct 17 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 17 October 2020 at 13:00:59 UTC, Per Nordlöw wrote:

 I understand that. I don't want the alignment of `S` to change. 
 I want the padding after `s`


That padding is part of S. It is at the end, after its fields, 
but still part of it.

S's layout doesn't depend on what else is around it.


 in `T` to be avoided and have `c` start at byte-offset 7.


Use a union.

     struct S
     {
         int i;                  // 4 bytes
         short s;                // 2 byte
         bool b;                 // 1 byte
     }
     static assert(S.sizeof == 8);
     static assert(S.alignof == 4);

struct T {
    union {
        S s;
        struct {
            align(1):
            ubyte[7] _ignore_me;
            char c;
        }
    }
}

static assert(T.alignof == 4);
static assert(T.sizeof == 8);
static assert(T.c.offsetof == 7);
Oct 17 2020
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 17 October 2020 at 13:23:38 UTC, Adam D. Ruppe wrote:

 Use a union.


Nice! Thanks!
Oct 17 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/17/20 9:00 AM, Per Nordlöw wrote:

 On Saturday, 17 October 2020 at 12:51:21 UTC, ag0aep6g wrote:

 c does come directly after s. The padding between b and c is part of 
 s. If you don't want that padding, you can use `align(1)` to define S 
 without padding. But then 75% of the ints in an S[] will be misaligned.


 
 I understand that. I don't want the alignment of `S` to change. I want 
 the padding after `s` in `T` to be avoided and have `c` start at 
 byte-offset 7. I don't see why this padding is needed in the case where 
 only a single (1-element array of) `S` is stored as a field inside 
 another aggregate.
 
 Ali's code prints:
 
 === Memory layout of 'T' (.sizeof: 12, .alignof: 4) ===
     0: S s
     8: char c
     9: ... 3-byte PADDING


There might be some good reasons for this.

For one, what happens if you copy the s? Does it copy the c too?

For another, this is how C does it, so I would expect it to at least 
match C for compatibility.

I think it *should* be possible to do this, if it's not already, just 
with pragmas. (i.e. pack T but not S).

-Steve
Oct 17 2020
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 17 October 2020 at 13:42:46 UTC, Steven 
Schveighoffer wrote:

 I think it *should* be possible to do this, if it's not 
 already, just with pragmas. (i.e. pack T but not S).


Agree, a pragma, say `pragma(pack)`, to control this would be 
great to avoid the unsafe union hack.
Oct 17 2020
prev sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 10/16/20 1:32 PM, Per Nordl=C3=B6w wrote:

 Why is `T.sizeof` 12 instead of 8 when `U.sizeof` is 8 in the following=


=20

 example?
=20
 struct S
 {
  =C2=A0=C2=A0=C2=A0 int i;
  =C2=A0=C2=A0=C2=A0 bool b;
 }
=20
 struct T
 {
  =C2=A0=C2=A0=C2=A0 S s;
  =C2=A0=C2=A0=C2=A0 char c;
 }
=20
 struct U
 {
  =C2=A0=C2=A0=C2=A0 int i;
  =C2=A0=C2=A0=C2=A0 bool b;
  =C2=A0=C2=A0=C2=A0 char c;
 }
=20
 ?


I have a function that dumps member layout of structs, which someone may =

find useful:

   http://ddili.org/ders/d.en/memory.html#ix_memory..offsetof

It prints the following for these types:

=3D=3D=3D Memory layout of 'S' (.sizeof: 8, .alignof: 4) =3D=3D=3D
    0: int i
    4: bool b
    5: ... 3-byte PADDING
=3D=3D=3D Memory layout of 'T' (.sizeof: 12, .alignof: 4) =3D=3D=3D
    0: S s
    8: char c
    9: ... 3-byte PADDING
=3D=3D=3D Memory layout of 'U' (.sizeof: 8, .alignof: 4) =3D=3D=3D
    0: int i
    4: bool b
    5: char c
    6: ... 2-byte PADDING

Copied here:

struct S
{
     int i;
     bool b;
}

struct T
{
     S s;
     char c;
}

struct U
{
     int i;
     bool b;
     char c;
}

void printObjectLayout(T)()
         if (is (T =3D=3D struct) || is (T =3D=3D union)) {
     import std.stdio;
     import std.string;

     writefln("=3D=3D=3D Memory layout of '%s'" ~
              " (.sizeof: %s, .alignof: %s) =3D=3D=3D",
              T.stringof, T.sizeof, T.alignof);

     /* Prints a single line of layout information. */
     void printLine(size_t offset, string info) {
         writefln("%4s: %s", offset, info);
     }

     /* Prints padding information if padding is actually
      * observed. */
     void maybePrintPaddingInfo(size_t expectedOffset,
                                size_t actualOffset) {
         if (expectedOffset < actualOffset) {
             /* There is some padding because the actual offset
              * is beyond the expected one. */

             const paddingSize =3D actualOffset - expectedOffset;

             printLine(expectedOffset,
                       format("... %s-byte PADDING",
                              paddingSize));
         }
     }

     /* This is the expected offset of the next member if there
      * were no padding bytes before that member. */
     size_t noPaddingOffset =3D 0;

     /* Note: __traits(allMembers) is a 'string' collection of
      * names of the members of a type. */
     foreach (memberName; __traits(allMembers, T)) {
         mixin (format("alias member =3D %s.%s;",
                       T.stringof, memberName));

         const offset =3D member.offsetof;
         maybePrintPaddingInfo(noPaddingOffset, offset);

         const typeName =3D typeof(member).stringof;
         printLine(offset,
                   format("%s %s", typeName, memberName));

         noPaddingOffset =3D offset + member.sizeof;
     }

     maybePrintPaddingInfo(noPaddingOffset, T.sizeof);
}

void main() {
   printObjectLayout!S();
   printObjectLayout!T();
   printObjectLayout!U();
}

Ali
Oct 16 2020