www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Inline aggregate types

reply Ethan Watson <gooberman gmail.com> writes:
https://github.com/Remedy-Entertainment/binderoo/blob/master/binderoo_client/d/src/binderoo/bitpacking.d

So I've been not-all-that-pleased with std.bitmanip.bitfields for 
a while. It's nice that it's there, but I'm binding to C++ 
objects where the meaningful default values require those packed 
values to have initial values. It's rather painful to get that 
working with the Phobos implementation.

I wanted to make a bitfield where you would simply give it a 
dummy struct type, complete with UDAs to tell it how many bits to 
take as well as standard default values. For example:

struct SomeBitField
{
    PackSize( 3 ) int iSomeInt = 3;
    PackSize( 1 ) bool bSomeBool = true;
    PackSize( 4 ) int iSomeOtherInt;
}

I also don't want this struct to exist outside of the mixin 
declaration for it. Essentially, I want the code to boil down to:

mixin BitPack!( struct {
    PackSize( 3 ) int iSomeInt = 3;
    PackSize( 1 ) bool bSomeBool = true;
    PackSize( 4 ) int iSomeOtherInt;
} );

Nice. Readable. Maintainable. You don't even need to read the 
documentation to add new members to the bit field, or change 
default values.

The compiler disagrees though. The second it sees that struct 
keyword, it freaks out. Sigh. Alrighty, what if we just remove 
the struct keyword? Nope. It tells me that I'm actually passing a 
lambda in to the BitPack mixin template.

Right. If that's the way you want to play compiler:

mixin template BitPack( Descriptor )
{
   mixin( GenerateBitPackBody!( Descriptor )() );
}

mixin template BitPack( string ElementDescription )
{
   mixin( "mixin BitPack!( typeof( { struct BitPackData { " ~ 
ElementDescription ~ " } return BitPackData.init; }() ) );" );
}

What in Zod's name is that abomination? Well. If it thinks it's a 
lambda that I'm trying to pass it, let's just make it explicit. 
I'll make an actual function literal. Take in the variables I 
want as a string instead of a plain old struct. Use a string 
mixin to generate this lambda. Get a typeof of the return type 
(deep in to Voldemort territory here) and passing that along to 
the main BitPack mixin template. And because I *really* don't 
want that struct to persist, my mixin template that takes the 
string descriptor inlines the entire thing with a string mixin.

So the invokation, while not as readable, looks like:

mixin BitPack!( " PackSize( 3 ) int iSomeInt = 3;  PackSize( 1 ) 
bool bSomeBool = true;  PackSize( 4 ) int iSomeOtherInt;" );

I'm sure I just made someone cry looking at that.

I'm also sure there's plenty of other legit uses for inline 
aggregate types. So while the feature doesn't exist in the 
language yet, at least you can now see that there's quite a legit 
hacky-as-all-fuck workaround for it.
Dec 01 2016
next sibling parent ag0aep6g <anonymous example.com> writes:
On 12/01/2016 09:31 PM, Ethan Watson wrote:

 I'm also sure there's plenty of other legit uses for inline aggregate
 types. So while the feature doesn't exist in the language yet, at least
 you can now see that there's quite a legit hacky-as-all-fuck workaround
 for it.


Interestingly, we do have anonymous classes:

     auto c = new class {  attr int foo; float bar; };

No anonymous structs, though. You can put a struct into the anonymous 
class and extract it immediately, but that's hacky again:

     auto t = typeof(
         new class { static struct S { int foo; float bar; } }
     ).S();
Dec 01 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/01/2016 03:31 PM, Ethan Watson wrote:

 I'm binding to C++ objects where the meaningful default values require
 those packed values to have initial values. It's rather painful to get
 that working with the Phobos implementation.


The simplest way to go here is to accept an initializer for the entire 
ubyte/ushort/uint/ulong underlying the bitfields. -- Andrei
Dec 01 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 1 December 2016 at 20:57:13 UTC, Andrei Alexandrescu 
wrote:

 The simplest way to go here is to accept an initializer for the 
 entire ubyte/ushort/uint/ulong underlying the bitfields. -- 
 Andrei


In terms of efficiency at compile time, indeed. But there's 
multiple paths there. You could assign to the value directly, 
which would require intimate knowledge of how the implementation 
works. Which then means you'd need a helper.

Under the hood for my implementation, in fact, it makes that 
single initializer for the bitfield. I'm currently storing in a 
ubyte array that stores in multiples of 4 bytes, and as a part of 
parsing the defined struct it also creates an initializer for 
that array. I'll have to do a bit more work so that I can also 
specify the underlying storage type or intelligently work it out 
(the example I used here, in fact, would only ever want one byte 
of storage to match C++ built-in bitfields exactly unless I'm 
mistaken).
Dec 01 2016
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Dec 01, 2016 at 08:31:32PM +0000, Ethan Watson via Digitalmars-d wrote:
[...]

 So the invokation, while not as readable, looks like:
 
 mixin BitPack!( " PackSize( 3 ) int iSomeInt = 3;  PackSize( 1 ) bool
 bSomeBool = true;  PackSize( 4 ) int iSomeOtherInt;" );
 
 I'm sure I just made someone cry looking at that.


In cases like these, q{} comes to the rescue:

	mixin BitPack!(q{
		 PackSize( 3 ) int iSomeInt = 3;
		 PackSize( 1 ) bool bSomeBool = true;
		 PackSize( 4 ) int iSomeOtherInt;
	});

See? Not so bad after all. ;-)

(And *that* is why we have a q{} literal in a language that, arguably,
already has far too many ways to write a string literal.)


T

-- 
Customer support: the art of getting your clients to pay for your own
incompetence.
Dec 01 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 1 December 2016 at 20:58:53 UTC, H. S. Teoh wrote:

 In cases like these, q{} comes to the rescue:


I had precisely zero idea this existed.
Dec 01 2016
prev sibling next sibling parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 1 December 2016 at 20:31:32 UTC, Ethan Watson wrote:

 https://github.com/Remedy-Entertainment/binderoo/blob/master/binderoo_client/d/src/binderoo/bitpacking.d


Stefan also reminded me that this would be a perfect spot where 
__symbol CTFE variables would make life much easier (especially 
since my entire implementation of the BitPack templates lean 
heavily on CTFE for code generation).

http://forum.dlang.org/post/kqyqnxrirjhtnpvfbvsw forum.dlang.org
Dec 01 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-01 21:31, Ethan Watson wrote:

 https://github.com/Remedy-Entertainment/binderoo/blob/master/binderoo_client/d/src/binderoo/bitpacking.d


 So I've been not-all-that-pleased with std.bitmanip.bitfields for a
 while. It's nice that it's there, but I'm binding to C++ objects where
 the meaningful default values require those packed values to have
 initial values. It's rather painful to get that working with the Phobos
 implementation.

 I wanted to make a bitfield where you would simply give it a dummy
 struct type, complete with UDAs to tell it how many bits to take as well
 as standard default values. For example:

 struct SomeBitField
 {
    PackSize( 3 ) int iSomeInt = 3;
    PackSize( 1 ) bool bSomeBool = true;
    PackSize( 4 ) int iSomeOtherInt;
 }

 I also don't want this struct to exist outside of the mixin declaration
 for it. Essentially, I want the code to boil down to:

 mixin BitPack!( struct {
    PackSize( 3 ) int iSomeInt = 3;
    PackSize( 1 ) bool bSomeBool = true;
    PackSize( 4 ) int iSomeOtherInt;
 } );

 Nice. Readable. Maintainable. You don't even need to read the
 documentation to add new members to the bit field, or change default
 values.

 The compiler disagrees though. The second it sees that struct keyword,
 it freaks out. Sigh. Alrighty, what if we just remove the struct
 keyword? Nope. It tells me that I'm actually passing a lambda in to the
 BitPack mixin template.


Seems like a job for AST macros :). But wWhat about an anonymous class 
as ag0aep6g mentioned? This compiles and prints "iSomeInt" as expected:

template Foo(T)
{
     pragma(msg, __traits(identifier, T.tupleof[0]));
}

struct PackSize
{
     int size;
}

void main()
{
     mixin Foo!(
         typeof(
             new class {
                  PackSize( 3 ) int iSomeInt = 3;
                  PackSize( 1 ) bool bSomeBool = true;
                  PackSize( 4 ) int iSomeOtherInt;
             }
         )
     );
}

-- 
/Jacob Carlborg
Dec 02 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Friday, 2 December 2016 at 08:33:17 UTC, Jacob Carlborg wrote:

 But wWhat about an anonymous class as ag0aep6g mentioned?


Usability I see is something that is key here. Having to do a 
typeof yourself is one extra step away from an intuitive API.

So I tried making a definition like this:

mixin template BitPack( Descriptor instance, string NameAlias = 
Descriptor.stringof )
{
   mixin( GenerateBitPackBody!( Descriptor, Descriptor.stringof 
)() );
}

...but I can't work out how to make the template evaluator treat 
descriptor as a templated type without explicitly defining that 
type as the first parameter.

Even trying:

mixin template BitPack( alias Descriptor ) if( is( typeof( 
Descriptor ) ) )

or:

mixin template BitPack( alias Descriptor ) if( __traits( 
compiles, &Descriptor ) )

wasn't getting me valid code (in the case of the latter, it 
claims it's not a valid template value argument; in the former, 
it meant I would have needed to add additional constraints to the 
other BitPack template).

I bet there's a trick out there that'll let me do what I want. 
But I won't go looking for it at least.
Dec 02 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-02 10:11, Ethan Watson wrote:

 On Friday, 2 December 2016 at 08:33:17 UTC, Jacob Carlborg wrote:

 But wWhat about an anonymous class as ag0aep6g mentioned?


 Usability I see is something that is key here. Having to do a typeof
 yourself is one extra step away from an intuitive API.

 So I tried making a definition like this:

 mixin template BitPack( Descriptor instance, string NameAlias =
 Descriptor.stringof )
 {
   mixin( GenerateBitPackBody!( Descriptor, Descriptor.stringof )() );
 }

 ...but I can't work out how to make the template evaluator treat
 descriptor as a templated type without explicitly defining that type as
 the first parameter.

 Even trying:

 mixin template BitPack( alias Descriptor ) if( is( typeof( Descriptor ) ) )

 or:

 mixin template BitPack( alias Descriptor ) if( __traits( compiles,
 &Descriptor ) )

 wasn't getting me valid code (in the case of the latter, it claims it's
 not a valid template value argument; in the former, it meant I would
 have needed to add additional constraints to the other BitPack template).

 I bet there's a trick out there that'll let me do what I want. But I
 won't go looking for it at least.


Using an alias without a template constraint works for me. No "typeof" 
is required when using the template:

template Foo(alias T)
{
     pragma(msg, __traits(identifier, T.tupleof[0]));
}

struct PackSize
{
     int size;
}

void main()
{
     mixin Foo!(
         new class {
              PackSize( 3 ) int iSomeInt = 3;
              PackSize( 1 ) bool bSomeBool = true;
              PackSize( 4 ) int iSomeOtherInt;
         }
     );
}

The only difference now from your original example is "new class" 
instead of "struct".

-- 
/Jacob Carlborg
Dec 02 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Friday, 2 December 2016 at 10:16:17 UTC, Jacob Carlborg wrote:

 Using an alias without a template constraint works for me.


I was attempting to support all methods. new class isn't the 
cleanest way of doing things either, so I decided I'd support all 
the things and let the user choose what they're comfortable with.

But I did derp a bit, and realised that I could have just changed 
the string mixin method to not invoke the main template with 
typeof( return value ).

Thus:

mixin template BitPack( alias Descriptor, string NameAlias = 
typeof( Descriptor ).stringof )
{
   mixin( GenerateBitPackBody!( typeof( Descriptor ), NameAlias 
)() );
}

mixin template BitPack( string ElementDescriptor, string 
NameAlias = "BitPackData" )
{
   mixin( "mixin BitPack!( { struct BitPackData { " ~ 
ElementDescriptor ~ " } return BitPackData.init; }(), NameAlias 
);" );
}

Works for both string definitions and inlined new class 
definitions.
Dec 02 2016
parent reply Guillaume Chatelet <chatelet.guillaume gmail.com> writes:
On Friday, 2 December 2016 at 11:11:30 UTC, Ethan Watson wrote:

 On Friday, 2 December 2016 at 10:16:17 UTC, Jacob Carlborg 
 wrote:

 [...]


 I was attempting to support all methods. new class isn't the 
 cleanest way of doing things either, so I decided I'd support 
 all the things and let the user choose what they're comfortable 
 with.

 [...]


Do you plan on contributing this back to phobos? I also came 
across this exact same problem.
Dec 05 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Monday, 5 December 2016 at 11:57:18 UTC, Guillaume Chatelet 
wrote:

 Do you plan on contributing this back to phobos? I also came 
 across this exact same problem.


It'll want to go through a few polish iterations before I even 
think of doing that; and it'll need support for things like 
toString/toHash/etc.

Another critical bit of functionality that std.bitmanip provides 
that I currently don't is a bit array. I also want to support 
statically sized bit arrays, Phobos only provides support for 
dynamically sized.
Dec 05 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-12-05 13:32, Ethan Watson wrote:


 I also want to support statically sized
 bit arrays, Phobos only provides support for dynamically sized.


That would be nice, I had a need for that.

-- 
/Jacob Carlborg
Dec 05 2016