www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - proper bit fields in the D2 language?

reply nobody <hehe hehe.com> writes:
I remember read D's introduction page that C's bit field is something will be
dropped from D.

But now I see dmd/src/phobos/std/bitmanip.d line 30 ~ 98: bit field functions
are generated at compile time,

private template createAccessors(...) {
  // getter
  // setter
  ...
}

I wonder how efficient is this. If we are going to have bit fields in the
*std* library, why not support it properly in the language. I think the
compiler can generate much better code than these compile-time generated
functions, which looks like a hack to me.

comments?
Apr 21 2009
parent reply Brad Roberts <braddr puremagic.com> writes:
nobody wrote:

 I remember read D's introduction page that C's bit field is something will be
 dropped from D.
 
 But now I see dmd/src/phobos/std/bitmanip.d line 30 ~ 98: bit field functions
 are generated at compile time,
 
 private template createAccessors(...) {
   // getter
   // setter
   ...
 }
 
 I wonder how efficient is this. If we are going to have bit fields in the
 *std* library, why not support it properly in the language. I think the
 compiler can generate much better code than these compile-time generated
 functions, which looks like a hack to me.
 
 comments?


The auto-generated code from the library is the same code the compiler
would end up generating.  You can test that theory by comparing the
produced assembly for a C vs a D implementation.

Later,
Brad
Apr 21 2009
parent reply nobody <hehe hehe.com> writes:
== Quote from Brad Roberts (braddr puremagic.com)'s article

 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.


If the generated code are same, and it's in the std library, which means no
difference in the backend.

Then I'd rather write:

struct A
{
        bool flag1:  1;
        bool flag2:  1;
        // uint "",  6;   // this should be auto-magically generated by the
compiler
}

(the code is more clear, and the compiler can give better message).

than this:

struct A
{
    mixin(bitfields!(
        bool, "flag1",    1,
        bool, "flag2",    1,
        uint, "",         6));
}
Apr 21 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
nobody wrote:

 == Quote from Brad Roberts (braddr puremagic.com)'s article

 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.


 
 If the generated code are same, and it's in the std library, which means no
 difference in the backend.
 
 Then I'd rather write:
 
 struct A
 {
         bool flag1:  1;
         bool flag2:  1;
         // uint "",  6;   // this should be auto-magically generated by the
compiler
 }
 
 (the code is more clear, and the compiler can give better message).
 
 than this:
 
 struct A
 {
     mixin(bitfields!(
         bool, "flag1",    1,
         bool, "flag2",    1,
         uint, "",         6));
 }


Except that bitfields don't appear to be a widely used feature.  If you
can put it in the library with no performance penalty AND simplify the
compiler at the same time, that looks like a good thing to me, and I
believe Walter agrees.

  -- Daniel
Apr 21 2009
parent reply grauzone <none example.net> writes:
Daniel Keep wrote:

 
 nobody wrote:

 == Quote from Brad Roberts (braddr puremagic.com)'s article

 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.


 If the generated code are same, and it's in the std library, which means no
 difference in the backend.

 Then I'd rather write:

 struct A
 {
         bool flag1:  1;
         bool flag2:  1;
         // uint "",  6;   // this should be auto-magically generated by the
compiler
 }

 (the code is more clear, and the compiler can give better message).

 than this:

 struct A
 {
     mixin(bitfields!(
         bool, "flag1",    1,
         bool, "flag2",    1,
         uint, "",         6));
 }


 
 Except that bitfields don't appear to be a widely used feature.  If you
 can put it in the library with no performance penalty AND simplify the
 compiler at the same time, that looks like a good thing to me, and I
 believe Walter agrees.


Yeah, let's move everything from the compiler to the library. Damn, why 
couldn't they have const implemented as library feature?


   -- Daniel
Apr 21 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
grauzone wrote:

 Daniel Keep wrote:

 nobody wrote:

 == Quote from Brad Roberts (braddr puremagic.com)'s article

 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.


 If the generated code are same, and it's in the std library, which
 means no
 difference in the backend.

 Then I'd rather write:

 struct A
 {
         bool flag1:  1;
         bool flag2:  1;
         // uint "",  6;   // this should be auto-magically generated
 by the compiler
 }

 (the code is more clear, and the compiler can give better message).

 than this:

 struct A
 {
     mixin(bitfields!(
         bool, "flag1",    1,
         bool, "flag2",    1,
         uint, "",         6));
 }


 Except that bitfields don't appear to be a widely used feature.  If you
 can put it in the library with no performance penalty AND simplify the
 compiler at the same time, that looks like a good thing to me, and I
 believe Walter agrees.


 
 Yeah, let's move everything from the compiler to the library. Damn, why
 couldn't they have const implemented as library feature?
 

   -- Daniel




Except that bitfields don't appear to be a widely used feature.

  -- Daniel
Apr 21 2009
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Daniel Keep:

 Except that bitfields don't appear to be a widely used feature.


I have found plenty of them in code to translate from C to other languages.

Another possible solution is the same intermediate one that will be used for
the associative arrays: keep only the syntax in the language and implement
their semantics in the std lib.

Bye,
bearophile
Apr 22 2009
prev sibling parent reply Mattias Holm <mattias.holm openorbit.REMOVE.THIS.org> writes:
 Except that bitfields don't appear to be a widely used feature.


You clearly have not written systems code in Ada... :)

C-bitfields are problematic because the bit-ordering is implementation 
defined, GCC have the bits appear in the order of definition on 
big-endian machines and the reverse order on little endian machines. The 
appropriate way is to have them as the big-endian fields in GCC, this is 
also what most Ada-compilers seem to be doing.

They are incredibly useful for any systems and protocol programming, and 
this is why they are nice to have in a language like D.

Bitfields are not used in C because they are not platform neutral, a 
proper definition in D would mean that people would use them, at least 
in the mentioned domains.

/ Mattias
Apr 27 2009
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Mattias Holm:

 You clearly have not written systems code in Ada... :)


Bitfields are quite useful, also take a look at the high level language Erlang,
that has some high-level ways to use them. In the end the current design of D2
bitfields is acceptable. Inecessary some syntax sugar can be added later, as
macro form.

Bye,
bearophile
Apr 27 2009
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Mattias Holm wrote:

 Except that bitfields don't appear to be a widely used feature.


 
 You clearly have not written systems code in Ada... :)
 
 C-bitfields are problematic because the bit-ordering is implementation 
 defined, GCC have the bits appear in the order of definition on 
 big-endian machines and the reverse order on little endian machines. The 
 appropriate way is to have them as the big-endian fields in GCC, this is 
 also what most Ada-compilers seem to be doing.
 
 They are incredibly useful for any systems and protocol programming, and 
 this is why they are nice to have in a language like D.
 
 Bitfields are not used in C because they are not platform neutral, a 
 proper definition in D would mean that people would use them, at least 
 in the mentioned domains.


Bitfields in Phobos are defined portably: always populated from lsb to 
msb, the total size must be 8, 16, 32, or 64, and there is no hidden 
padding (you obtain padding with anonymous fields).

Andrei
Apr 27 2009
parent reply Mattias Holm <mattias.holm openorbit.REMOVE.THIS.org> writes:
 Bitfields in Phobos are defined portably: always populated from lsb to 
 msb, the total size must be 8, 16, 32, or 64, and there is no hidden 
 padding (you obtain padding with anonymous fields).
 
 Andrei


If this is the case, the bitfields are not portable and the behaviour is 
the same as for GCC, except it is reversed.

The order should be:

LSByte to MSByte on little endian machines
MSByte to LSByte on big endian machines


This means that bitfields are defined in human readable order with 
respect to the physical layout on the machine.

The point being:

struct {
   uint a:4;
   uint b:28;
}

should give a structure:
  _ _______
|.|.......|

where each dot represents 4 bits. And this structure should be identical 
if the data is moved between different systems, that is a TCP header 
definition mapping into a byte stream, should be identical on both PPC 
and x86. If the LSB to MSB ordering is used in phobos, then the 
bitfields are as useful as the GCC bitfields, i.e. not at all.


/Mattias
Apr 27 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Mattias Holm wrote:

 Bitfields in Phobos are defined portably: always populated from lsb to 
 msb, the total size must be 8, 16, 32, or 64, and there is no hidden 
 padding (you obtain padding with anonymous fields).

 Andrei


 
 If this is the case, the bitfields are not portable and the behaviour is 
 the same as for GCC, except it is reversed.
 
 The order should be:
 
 LSByte to MSByte on little endian machines
 MSByte to LSByte on big endian machines
 
 
 This means that bitfields are defined in human readable order with 
 respect to the physical layout on the machine.
 
 The point being:
 
 struct {
   uint a:4;
   uint b:28;
 }
 
 should give a structure:
  _ _______
 |.|.......|
 
 where each dot represents 4 bits. And this structure should be identical 
 if the data is moved between different systems, that is a TCP header 
 definition mapping into a byte stream, should be identical on both PPC 
 and x86. If the LSB to MSB ordering is used in phobos, then the 
 bitfields are as useful as the GCC bitfields, i.e. not at all.


Not at all is a tad extreme as I'm using them already. I see your point 
though. If you could submit a bug report, that would be great.


Andrei
Apr 27 2009
prev sibling parent grauzone <none example.net> writes:
nobody wrote:

 == Quote from Brad Roberts (braddr puremagic.com)'s article

 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.


 
 If the generated code are same, and it's in the std library, which means no
 difference in the backend.
 
 Then I'd rather write:
 
 struct A
 {
         bool flag1:  1;
         bool flag2:  1;
         // uint "",  6;   // this should be auto-magically generated by the
compiler
 }
 
 (the code is more clear, and the compiler can give better message).
 
 than this:
 
 struct A
 {
     mixin(bitfields!(
         bool, "flag1",    1,
         bool, "flag2",    1,
         uint, "",         6));
 }


Bitfields are a rather obscure feature. There's no real reason to have 
it in the compiler. But because D is so great, you still can have it in 
the standard library. Sadly, this comes with a bit more ugly syntax. If 
we had macros (which were ditched for the oh-so-great [actually 
questionable] features const and immutable), then it could be at least a 
_bit_ more beautiful. Like defining identifiers as identifiers, and not 
strings.

What I'm really concerned about is that this mixin MAGICALLY inserts 
unknown symbols into the current scope. I was too lazy to look in the 
Phobos code, but it's clear that the mixin must generate "hidden" fields 
to store the actual bit values. (With a bit of luck, the author 
[Andrei?] was careful enough to allow several bitfield mixins in the 
same struct.) I also will blow up if later you add serialization or 
anything in this direction.

A clean way would to to let the bitfields thing generate a real struct type:


alias Bitfields!(bool, "flag1", 1, bool, "flag2", 6) MyBitfields;

No more magical insertion of identifiers into your code. And a 
serialization mechanism just could detect the type Bitfields and write 
its actual members, and not the hidden bit array field (or whatever the 
implementation uses, it just insert its own, unknown-to-you symbols into 
YOUR scope).
Apr 21 2009