digitalmars.D.announce - Cerealed v0.6.1: even less boilerplate for binary serialization
- Atila Neves (24/24) Aug 03 2015 http://code.dlang.org/packages/cerealed
- Dicebot (19/32) Aug 03 2015 This deserialization will be identical to casting like this,
- Atila Neves (19/55) Aug 03 2015 In this case, yes. In the real-life case I was simplifying, it
- Dicebot (2/20) Aug 03 2015 Thanks. Very nice indeed.
- Dmitry Olshansky (4/39) Aug 03 2015 Plus/minus network byte order.
- Atila Neves (3/31) Aug 03 2015 Oh yeah, and that. Cerealed always does network order.
- Walter Bright (2/3) Aug 03 2015 Please put this as the first comment on the reddit post.
- Atila Neves (3/6) Aug 03 2015 Which one?
- Walter Bright (10/16) Aug 03 2015 The one you started the thread with.
- "Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> (8/32) Aug 03 2015 1. Are there any convenience functions similar to msgpack's
- Atila Neves (14/57) Aug 03 2015 Yes. `cerealise` and `decerealise`. The former is slightly weird
- "Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> (7/12) Aug 03 2015 Nice!
- Atila Neves (7/20) Aug 03 2015 I've thought about that (and even JSON or XML) but I'm not 100%
- TC (18/31) Aug 03 2015 As I tried the changes on the same test as in [0]
- Jacob Carlborg (6/8) Aug 03 2015 That would be Orange [1]. Hopefully I'll get it into Phobos at some poin...
http://code.dlang.org/packages/cerealed What's new? * Performance improvements * New UDAs for networking packets for even less required boilerplate The first new thing is self-explanatory. The second one is explained briefly in this blog post: https://www.reddit.com/r/programming/comments/3flnlt/cerealed_a_d_library_for_declarative_binary/ (also on HN but you know how that goes) The summary is you can now write this: struct UdpPacket { static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } enum headerSize = unalignedSizeof!Header; alias header this; Header header; LengthInBytes("length - headerSize") ubyte[] data; } Code? Who needs code when the compiler can write it for you? Atila
Aug 03 2015
On Monday, 3 August 2015 at 09:21:50 UTC, Atila Neves wrote:The summary is you can now write this: struct UdpPacket { static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } enum headerSize = unalignedSizeof!Header; alias header this; Header header; LengthInBytes("length - headerSize") ubyte[] data; }This deserialization will be identical to casting like this, right? (Not trying to diminish your work, just making sure I get semantics :)) align(1) struct UdpPacket { align(1) static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } Header header; ubyte[0] data; } // ... auto packet = cast(UdpPacket*) raw_data.ptr;
Aug 03 2015
On Monday, 3 August 2015 at 09:27:03 UTC, Dicebot wrote:On Monday, 3 August 2015 at 09:21:50 UTC, Atila Neves wrote:In this case, yes. In the real-life case I was simplifying, it wasn't a ubyte[] array, it was an array of structs with non-trivial serialisation that also depended on a previous deserialised variable. It was more like this: struct Outer { static struct Header { ... } Header header; LengthInBytes("length - headerSize") Inner[] array; } struct Inner { static struct Header { ... } Header header; ArrayLength("length") Unit[] units; //actual length of the array instead of in bytes } struct Unit { ... } So maybe not as useless after all ;) AtilaThe summary is you can now write this: struct UdpPacket { static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } enum headerSize = unalignedSizeof!Header; alias header this; Header header; LengthInBytes("length - headerSize") ubyte[] data; }This deserialization will be identical to casting like this, right? (Not trying to diminish your work, just making sure I get semantics :)) align(1) struct UdpPacket { align(1) static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } Header header; ubyte[0] data; } // ... auto packet = cast(UdpPacket*) raw_data.ptr;
Aug 03 2015
On Monday, 3 August 2015 at 09:40:59 UTC, Atila Neves wrote:In this case, yes. In the real-life case I was simplifying, it wasn't a ubyte[] array, it was an array of structs with non-trivial serialisation that also depended on a previous deserialised variable. It was more like this: struct Outer { static struct Header { ... } Header header; LengthInBytes("length - headerSize") Inner[] array; } struct Inner { static struct Header { ... } Header header; ArrayLength("length") Unit[] units; //actual length of the array instead of in bytes } struct Unit { ... } So maybe not as useless after all ;) AtilaThanks. Very nice indeed.
Aug 03 2015
On 03-Aug-2015 12:27, Dicebot wrote:On Monday, 3 August 2015 at 09:21:50 UTC, Atila Neves wrote:Plus/minus network byte order. -- Dmitry OlshanskyThe summary is you can now write this: struct UdpPacket { static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } enum headerSize = unalignedSizeof!Header; alias header this; Header header; LengthInBytes("length - headerSize") ubyte[] data; }This deserialization will be identical to casting like this, right? (Not trying to diminish your work, just making sure I get semantics :)) align(1) struct UdpPacket { align(1) static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } Header header; ubyte[0] data; } // ... auto packet = cast(UdpPacket*) raw_data.ptr;
Aug 03 2015
On Monday, 3 August 2015 at 10:37:05 UTC, Dmitry Olshansky wrote:On 03-Aug-2015 12:27, Dicebot wrote:Oh yeah, and that. Cerealed always does network order. AtilaOn Monday, 3 August 2015 at 09:21:50 UTC, Atila Neves wrote:Plus/minus network byte order.[...]This deserialization will be identical to casting like this, right? (Not trying to diminish your work, just making sure I get semantics :)) align(1) struct UdpPacket { align(1) static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } Header header; ubyte[0] data; } // ... auto packet = cast(UdpPacket*) raw_data.ptr;
Aug 03 2015
On 8/3/2015 2:21 AM, Atila Neves wrote:[...]Please put this as the first comment on the reddit post.
Aug 03 2015
On Monday, 3 August 2015 at 11:13:50 UTC, Walter Bright wrote:On 8/3/2015 2:21 AM, Atila Neves wrote:Which one? Atila[...]Please put this as the first comment on the reddit post.
Aug 03 2015
On 8/3/2015 4:46 AM, Atila Neves wrote:On Monday, 3 August 2015 at 11:13:50 UTC, Walter Bright wrote:The one you started the thread with. On a more meta note, when people look at a discussion on reddit and see zero comments, they move on. Having a comment there saying briefly what the link is about and why the reader might be interested in it is far more effective at getting readers to spend some time reading your article. It also gives you an opportunity to shape the discussion in a direction you'd like. If some troll makes the first post, it sets a trollish tone and can completely ruin all the effort you put into the article. Don't leave things to chance!On 8/3/2015 2:21 AM, Atila Neves wrote:Which one?[...]Please put this as the first comment on the reddit post.
Aug 03 2015
On Monday, 3 August 2015 at 09:21:50 UTC, Atila Neves wrote:http://code.dlang.org/packages/cerealed What's new? * Performance improvements * New UDAs for networking packets for even less required boilerplate The first new thing is self-explanatory. The second one is explained briefly in this blog post: https://www.reddit.com/r/programming/comments/3flnlt/cerealed_a_d_library_for_declarative_binary/ (also on HN but you know how that goes) The summary is you can now write this: struct UdpPacket { static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } enum headerSize = unalignedSizeof!Header; alias header this; Header header; LengthInBytes("length - headerSize") ubyte[] data; } Code? Who needs code when the compiler can write it for you? Atila1. Are there any convenience functions similar to msgpack's pack() and unpack() providing compactness and elegance as import msgpack; auto x = [1,2]; assert(x.pack.unpack!typeof(x) == x); 2. How does the Cereal performance compare to Msgpack after the recent optimizations?
Aug 03 2015
On Monday, 3 August 2015 at 11:43:15 UTC, Per Nordlöw wrote:On Monday, 3 August 2015 at 09:21:50 UTC, Atila Neves wrote:Yes. `cerealise` and `decerealise`. The former is slightly weird for performance reasons. It takes a lambda that tells it what to do with the resulting bytes. import cerealed; auto x = MyStruct(); x.cerealise!(bytes => writeln(bytes)); ubyte[] bytes = [...]; auto x = bytes.decerealise!MyStruct;http://code.dlang.org/packages/cerealed What's new? * Performance improvements * New UDAs for networking packets for even less required boilerplate The first new thing is self-explanatory. The second one is explained briefly in this blog post: https://www.reddit.com/r/programming/comments/3flnlt/cerealed_a_d_library_for_declarative_binary/ (also on HN but you know how that goes) The summary is you can now write this: struct UdpPacket { static struct Header { ushort srcPort; ushort dstPort; ushort length; ushort checksum; } enum headerSize = unalignedSizeof!Header; alias header this; Header header; LengthInBytes("length - headerSize") ubyte[] data; } Code? Who needs code when the compiler can write it for you? Atila1. Are there any convenience functions similar to msgpack's pack() and unpack() providing compactness and elegance as import msgpack; auto x = [1,2]; assert(x.pack.unpack!typeof(x) == x);2. How does the Cereal performance compare to Msgpack after the recent optimizations?Close with LDC and DMD, faster with GDC: http://forum.dlang.org/post/nkcelouzpjsgmqtvnonq forum.dlang.org I haven't even tried optimising it myself though, I just merged a contribution from someone else. Atila
Aug 03 2015
On Monday, 3 August 2015 at 11:51:24 UTC, Atila Neves wrote:Yes. `cerealise` and `decerealise`. The former is slightly weird for performance reasons. It takes a lambda that tells it what to do with the resulting bytes. Close with LDC and DMD, faster with GDC: http://forum.dlang.org/post/nkcelouzpjsgmqtvnonq forum.dlang.orgNice! Are there any plans to add different backends (for instance msgpack) to Cereal? Then we could have one package to rule them all! I'll try Cereal in favor of msgpack next time! Thanks, Atila!
Aug 03 2015
On Monday, 3 August 2015 at 12:21:05 UTC, Per Nordlöw wrote:On Monday, 3 August 2015 at 11:51:24 UTC, Atila Neves wrote:I've thought about that (and even JSON or XML) but I'm not 100% convinced that it would work with the existing, and more importantly to me, not high on my list of priorities. I'm definitely not opposed to it.Yes. `cerealise` and `decerealise`. The former is slightly weird for performance reasons. It takes a lambda that tells it what to do with the resulting bytes. Close with LDC and DMD, faster with GDC: http://forum.dlang.org/post/nkcelouzpjsgmqtvnonq forum.dlang.orgNice! Are there any plans to add different backends (for instance msgpack) to Cereal? Then we could have one package to rule them all!I'll try Cereal in favor of msgpack next time! Thanks, Atila!Glad to be useful. :) Atila
Aug 03 2015
On Monday, 3 August 2015 at 12:21:05 UTC, Per Nordlöw wrote:On Monday, 3 August 2015 at 11:51:24 UTC, Atila Neves wrote:As I tried the changes on the same test as in [0] It went on my PC like this: DMD debug: Cerealed: 2 secs, 854 ms, 687 μs, and 2 hnsecs MsgPack: 2 secs, 328 ms, 698 μs, and 3 hnsecs DMD release: Cerealed: 1 sec, 619 ms, 570 μs, and 9 hnsecs MsgPack: 1 sec, 554 ms, and 185 μs LDC2 release: Cerealed: 966 ms, 398 μs, and 4 hnsecs MsgPack: 926 ms, 792 μs, and 3 hnsecs GDC release: Cerealed: 1 sec, 97 ms, 892 μs, and 1 hnsec MsgPack: 1 sec, 138 ms, 359 μs, and 2 hnsecs So it's almost the same now. [0] http://forum.dlang.org/post/nkcelouzpjsgmqtvnonq forum.dlang.orgYes. `cerealise` and `decerealise`. The former is slightly weird for performance reasons. It takes a lambda that tells it what to do with the resulting bytes. Close with LDC and DMD, faster with GDC: http://forum.dlang.org/post/nkcelouzpjsgmqtvnonq forum.dlang.orgNice! Are there any plans to add different backends (for instance msgpack) to Cereal? Then we could have one package to rule them all! I'll try Cereal in favor of msgpack next time! Thanks, Atila!
Aug 03 2015
On 03/08/15 14:21, Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> wrote:Are there any plans to add different backends (for instance msgpack) to Cereal? Then we could have one package to rule them all!That would be Orange [1]. Hopefully I'll get it into Phobos at some point. [1] https://github.com/jacob-carlborg/orange -- /Jacob Carlborg
Aug 03 2015