www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - std.serialization

reply "Orvid King" <blah38621 gmail.com> writes:
Well, I wrote the code for this a while back, and although it was 
originally intended as a replacement for just std.json (thus the 
repo name), it does have the framework in place to be a 
generalized serialization framework, and there is the start of 
xml, and bson implementations, so I'm releasing it as 
std.serialization. The JSON implementation is the only one I'd 
consider ready for production use however. The (de)serialization 
framework takes a step back and asks, "Why do we need pull 
parsers?", the answer to which is that allocations are slow, so 
don't allocate. And that's exactly what I do. The serializer does 
absolutely *no* allocations of it's own (except for float->string 
conversion, which I don't understand the algorithms enough to 
implement myself) even going so far as to create an output range 
based version of to!string(int/uint/long/ulong/etc.). And the 
benefits of doing it this way are very clearly reflected in the 
pure speed of the serializer. On my 2ghz i5 Macbook Air, it takes 
50ms to serialize 100k objects with roughly 600k integers 
contained in them when compiled with DMD, this roughly half the 
time it takes to generate the data to serialize. Compile it with 
GDC or LDC and that time is cut in half. I have done the exact 
same thing with deserialization as well, the only allocations 
done are for the output objects, because there is no intermediate 
representation.

So how do I use this greatness? Simple! import std.serialization, 
and apply the  serializable UDA to the class/struct you want to 
serialize, then call toJOSN(yourObject) and 
fromJSON!YourType(yourString) to your heart's content!

Now, there are other serialization libraries out there, such as 
orange, that take the compile-time reflection approach, but the 
amount of code required to implement a single format is just 
massive 2100 lines for the XMLArchive. The entire JSON 
(de)serialization, which *includes* both the lexer and parser is 
only 900 lines.




Wow, that went a bit more towards a salesman-like description 
than I as aiming for, so I'll just end this here and give you the 
link, before this ends up looking like a massive, badly written, 
sales pitch :D

https://github.com/Orvid/JSONSerialization
Feb 13 2014
next sibling parent Rory McGuire <rjmcguire gmail.com> writes:
Nice, hope the code is prettier than your speech. :D


On Fri, Feb 14, 2014 at 12:56 AM, Orvid King <blah38621 gmail.com> wrote:


 Well, I wrote the code for this a while back, and although it was
 originally intended as a replacement for just std.json (thus the repo
 name), it does have the framework in place to be a generalized
 serialization framework, and there is the start of xml, and bson
 implementations, so I'm releasing it as std.serialization. The JSON
 implementation is the only one I'd consider ready for production use
 however. The (de)serialization framework takes a step back and asks, "Why
 do we need pull parsers?", the answer to which is that allocations are
 slow, so don't allocate. And that's exactly what I do. The serializer does
 absolutely *no* allocations of it's own (except for float->string
 conversion, which I don't understand the algorithms enough to implement
 myself) even going so far as to create an output range based version of
 to!string(int/uint/long/ulong/etc.). And the benefits of doing it this
 way are very clearly reflected in the pure speed of the serializer. On my
 2ghz i5 Macbook Air, it takes 50ms to serialize 100k objects with roughly
 600k integers contained in them when compiled with DMD, this roughly half
 the time it takes to generate the data to serialize. Compile it with GDC or
 LDC and that time is cut in half. I have done the exact same thing with
 deserialization as well, the only allocations done are for the output
 objects, because there is no intermediate representation.

 So how do I use this greatness? Simple! import std.serialization, and
 apply the  serializable UDA to the class/struct you want to serialize, then
 call toJOSN(yourObject) and fromJSON!YourType(yourString) to your heart's
 content!

 Now, there are other serialization libraries out there, such as orange,
 that take the compile-time reflection approach, but the amount of code
 required to implement a single format is just massive 2100 lines for the
 XMLArchive. The entire JSON (de)serialization, which *includes* both the
 lexer and parser is only 900 lines.




 Wow, that went a bit more towards a salesman-like description than I as
 aiming for, so I'll just end this here and give you the link, before this
 ends up looking like a massive, badly written, sales pitch :D

 https://github.com/Orvid/JSONSerialization
Feb 14 2014
prev sibling next sibling parent reply "Francesco Cattoglio" <francesco.cattoglio gmail.com> writes:
On Thursday, 13 February 2014 at 22:56:38 UTC, Orvid King wrote:

 so I'm releasing it as std.serialization.


What does that even mean? I'm pretty sure you should NEVER call a 
library "std.something" if it hasn't been approved for inclusion 
into standard library.

Other than that, nice work.
Feb 14 2014
parent "Orvid King" <blah38621 gmail.com> writes:
On Friday, 14 February 2014 at 10:41:54 UTC, Francesco Cattoglio 
wrote:

 On Thursday, 13 February 2014 at 22:56:38 UTC, Orvid King wrote:

 so I'm releasing it as std.serialization.


 What does that even mean? I'm pretty sure you should NEVER call 
 a library "std.something" if it hasn't been approved for 
 inclusion into standard library.

 Other than that, nice work.


Yes, well, I'm bad at coming up with creative names, and, once I 
get around to writing full documentation for it, as well as do a 
bit of other cleanup, I'll submit it for inclusion in Phobos.
Feb 14 2014
prev sibling next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Orvid King"  wrote in message news:ntpjdeutsxqicjywtoxc forum.dlang.org... 


 (except for float->string 
 conversion, which I don't understand the algorithms enough to 
 implement myself) even going so far as to create an output range 
 based version of to!string(int/uint/long/ulong/etc.). 


std.format.formatValue / std.format.formattedWrite ?
Feb 14 2014
parent "Orvid King" <blah38621 gmail.com> writes:
On Friday, 14 February 2014 at 11:22:22 UTC, Daniel Murphy wrote:

 "Orvid King"  wrote in message 
 news:ntpjdeutsxqicjywtoxc forum.dlang.org...


 (except for float->string conversion, which I don't understand 
 the algorithms enough to implement myself) even going so far 
 as to create an output range based version of 
 to!string(int/uint/long/ulong/etc.).


 std.format.formatValue / std.format.formattedWrite ?


Both of them fall back on a form of printf internally.
Feb 14 2014
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Thursday, 13 February 2014 at 22:56:38 UTC, Orvid King wrote:

 Wow, that went a bit more towards a salesman-like description 
 than I as aiming for, so I'll just end this here and give you 
 the link, before this ends up looking like a massive, badly 
 written, sales pitch :D

 https://github.com/Orvid/JSONSerialization


It's much easier for people to get the gist of it if you generate 
documentation. It's kind of off-putting to have to rummage 
through source files with no direction.

In the vein of shameless self-promotion, I recommend using 
bootDoc[1]. :)

[1] https://github.com/JakobOvrum/bootDoc
Feb 14 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-02-13 23:56, Orvid King wrote:

 Well, I wrote the code for this a while back, and although it was
 originally intended as a replacement for just std.json (thus the repo
 name), it does have the framework in place to be a generalized
 serialization framework, and there is the start of xml, and bson
 implementations, so I'm releasing it as std.serialization. The JSON
 implementation is the only one I'd consider ready for production use
 however. The (de)serialization framework takes a step back and asks,
 "Why do we need pull parsers?", the answer to which is that allocations
 are slow, so don't allocate. And that's exactly what I do. The
 serializer does absolutely *no* allocations of it's own (except for
 float->string conversion, which I don't understand the algorithms enough
 to implement myself) even going so far as to create an output range
 based version of to!string(int/uint/long/ulong/etc.). And the benefits
 of doing it this way are very clearly reflected in the pure speed of the
 serializer. On my 2ghz i5 Macbook Air, it takes 50ms to serialize 100k
 objects with roughly 600k integers contained in them when compiled with
 DMD, this roughly half the time it takes to generate the data to
 serialize. Compile it with GDC or LDC and that time is cut in half. I
 have done the exact same thing with deserialization as well, the only
 allocations done are for the output objects, because there is no
 intermediate representation.


What features does it support? How does it handle:

* Arrays
* Slices
* Pointers
* Reference types
* Support for events
* Custom serialization
* Serialization of third party types


 So how do I use this greatness? Simple! import std.serialization, and
 apply the  serializable UDA to the class/struct you want to serialize,
 then call toJOSN(yourObject) and fromJSON!YourType(yourString) to your
 heart's content!


Why require a UDA?


 Now, there are other serialization libraries out there, such as orange,
 that take the compile-time reflection approach, but the amount of code
 required to implement a single format is just massive 2100 lines for the
 XMLArchive. The entire JSON (de)serialization, which *includes* both the
 lexer and parser is only 900 lines.


The reason for that might be:

1. XML is untyped unlike JSON
2. It supports quite a lot of features that most other serialization 
libraries don't support

-- 
/Jacob Carlborg
Feb 14 2014
parent reply "Orvid King" <blah38621 gmail.com> writes:
 What features does it support? How does it handle:

 * Arrays
 * Slices
 * Pointers
 * Reference types
 * Support for events
 * Custom serialization
 * Serialization of third party types


Slices are handled as arrays, because of the fact that they need 
to be handled in such a way that many different types of 
serialization formats can support them, and be inter-operable 
with implementations in languages other than D.

Pointers are not supported, because in my opinion, they should 
_NEVER_ be serialized.

Reference types are serialized as they are encountered, I haven't 
handled the circular reference case yet.

Events are not supported due to the fact it would require 
complete knowledge of the source and target environments of the 
serialization.

Custom serialization is supported by either supporting 
to!YourType(string) or YourType.parse(string) / 
to!string(valueOfYourType) or valueOfYourType.toString(), and are 
handled transparently by the base serialization handler, the 
actual serialization format sees them simply as strings. Each 
serialization format however does have the ability to select any 
type it wants to support being serialized.

And third party types are only supported if they have the 
requisite UDA, or support custom serialization.


 Why require a UDA?


The UDA is required for the exact same reason it's required in 
the .net framework, because it makes sure that the type you are 
trying to serialize is serialization aware, meaning that it's not 
serializing cache fields, and also makes sense to actually be 
serializing the type.
Feb 16 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-02-16 18:52, Orvid King wrote:


 Slices are handled as arrays, because of the fact that they need to be
 handled in such a way that many different types of serialization formats
 can support them, and be inter-operable with implementations in
 languages other than D.

 Pointers are not supported, because in my opinion, they should _NEVER_
 be serialized.


Why not? Think of languages like C and C++, they only support pointers. 
Pointers to basic types are not so interesting but pointers to structs are.


 Reference types are serialized as they are encountered, I haven't
 handled the circular reference case yet.


If the same reference value is encountered multiple times, is it 
serialized once or multiple times?


 Events are not supported due to the fact it would require complete
 knowledge of the source and target environments of the serialization.


What? I'm referring to methods being called before and after 
serialization of a given value.


 Custom serialization is supported by either supporting
 to!YourType(string) or YourType.parse(string) /
 to!string(valueOfYourType) or valueOfYourType.toString(), and are
 handled transparently by the base serialization handler, the actual
 serialization format sees them simply as strings. Each serialization
 format however does have the ability to select any type it wants to
 support being serialized.

 And third party types are only supported if they have the requisite UDA,
 or support custom serialization.


 Why require a UDA?


 The UDA is required for the exact same reason it's required in the .net
 framework, because it makes sure that the type you are trying to
 serialize is serialization aware, meaning that it's not serializing
 cache fields, and also makes sense to actually be serializing the type.


I prefer opt-out rather than opt-in.

Can it serialize through base class references?

-- 
/Jacob Carlborg
Feb 16 2014
next sibling parent reply =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
I believe it's worth the try to get Python-style serialization 
simplicity to D by default. We use Python cpickle at work and its 
fantastically simple and fast. It handles just about everthing 
you need by default as long as you have imported the modules that 
contain the types involved in the serialization.

It would attract more people from dynamical languages like Python 
if D got default-powers in std.serialization that does everything 
that msgpack does plus followings references.

This would make it possible to serialize arbitrary graph-likes 
structures with potential reference cycles. Resolving these 
cycles can be solved by using a map that contain a list of 
references/classs that already have serialized.

I'm however not sure how serialization of base and superclasses 
should be implemented in the most convenient way. Maybe somehow 
has a good suggestion for this.
Feb 16 2014
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/16/14, "Nordlöw" <per.nordlow gmail.com> wrote:

 I'm however not sure how serialization of base and superclasses
 should be implemented in the most convenient way. Maybe somehow
 has a good suggestion for this.


One way to do it:
https://github.com/msgpack/msgpack-d#use-own-deserialization-routine-for-class-and-struct
Feb 16 2014
parent "Atila Neves" <atila.neves gmail.com> writes:
On Sunday, 16 February 2014 at 20:57:28 UTC, Andrej Mitrovic 
wrote:

 On 2/16/14, "Nordlöw" <per.nordlow gmail.com> wrote:

 I'm however not sure how serialization of base and superclasses
 should be implemented in the most convenient way. Maybe somehow
 has a good suggestion for this.


 One way to do it:
 https://github.com/msgpack/msgpack-d#use-own-deserialization-routine-for-class-and-struct


That suffers from needing two functions to serialise/deserialise 
when only one should be needed.

Atila
Feb 19 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-02-16 21:45, "Nordlöw" wrote:

 I believe it's worth the try to get Python-style serialization
 simplicity to D by default. We use Python cpickle at work and its
 fantastically simple and fast. It handles just about everthing you need
 by default as long as you have imported the modules that contain the
 types involved in the serialization.

 It would attract more people from dynamical languages like Python if D
 got default-powers in std.serialization that does everything that
 msgpack does plus followings references.

 This would make it possible to serialize arbitrary graph-likes
 structures with potential reference cycles. Resolving these cycles can
 be solved by using a map that contain a list of references/classs that
 already have serialized.


I already have all this in Orange [1], which I'm in progress of adapting 
to a package for Phobos.


 I'm however not sure how serialization of base and superclasses should
 be implemented in the most convenient way. Maybe somehow has a good
 suggestion for this.


This requires registering the subclass in some way. In my implementation 
one needs to call:

Serializer.register!(Sub);

For full example see [2].

[1] https://github.com/jacob-carlborg/orange
[2] https://github.com/jacob-carlborg/orange/wiki/Base-Class

-- 
/Jacob Carlborg
Feb 16 2014
next sibling parent reply "Orvid King" <blah38621 gmail.com> writes:
 I already have all this in Orange [1], which I'm in progress of adapting  
 to a package for Phobos.


...

 This requires registering the subclass in some way. In my implementation  
 one needs to call:

 Serializer.register!(Sub);

 For full example see [2].

 [1] https://github.com/jacob-carlborg/orange
 [2] https://github.com/jacob-carlborg/orange/wiki/Base-Class


The problem with the way your doing it though is that it requires that the  
library doing the deserialization is fully aware of the semantics used in  
the serialization implementation, rather than just the syntax and  
semantics of the selected serialization format.
Feb 16 2014
parent "Jacob Carlborg" <doob me.com> writes:
On Sunday, 16 February 2014 at 22:33:07 UTC, Orvid King wrote:


 The problem with the way your doing it though is that it 
 requires that the library doing the deserialization is fully 
 aware of the semantics used in the serialization 
 implementation, rather than just the syntax and semantics of 
 the selected serialization format.


I don't see why that is required.

--
/Jacob Carlborg
Feb 16 2014
prev sibling parent reply Rory McGuire <rjmcguire gmail.com> writes:
Do you have a JSON driver for Orange yet? Would be interesting to benchmark
Orange against a purpose designed JSON serialization library.
If your design is mostly compile time the experiment would be very
interesting (I think).




On Mon, Feb 17, 2014 at 12:03 AM, Jacob Carlborg <doob me.com> wrote:


 On 2014-02-16 21:45, "Nordl=C3=B6w" wrote:


 I believe it's worth the try to get Python-style serialization
 simplicity to D by default. We use Python cpickle at work and its
 fantastically simple and fast. It handles just about everthing you need
 by default as long as you have imported the modules that contain the
 types involved in the serialization.

 It would attract more people from dynamical languages like Python if D
 got default-powers in std.serialization that does everything that
 msgpack does plus followings references.

 This would make it possible to serialize arbitrary graph-likes
 structures with potential reference cycles. Resolving these cycles can
 be solved by using a map that contain a list of references/classs that
 already have serialized.


 I already have all this in Orange [1], which I'm in progress of adapting
 to a package for Phobos.


  I'm however not sure how serialization of base and superclasses should

 be implemented in the most convenient way. Maybe somehow has a good
 suggestion for this.


 This requires registering the subclass in some way. In my implementation
 one needs to call:

 Serializer.register!(Sub);

 For full example see [2].

 [1] https://github.com/jacob-carlborg/orange
 [2] https://github.com/jacob-carlborg/orange/wiki/Base-Class

 --
 /Jacob Carlborg
Feb 16 2014
parent "Jacob Carlborg" <doob me.com> writes:
On Monday, 17 February 2014 at 06:14:51 UTC, Rory McGuire wrote:

 Do you have a JSON driver for Orange yet? Would be interesting 
 to benchmark
 Orange against a purpose designed JSON serialization library.
 If your design is mostly compile time the experiment would be 
 very
 interesting (I think).


Unfortunately no. I have worked on adapting a range interface and 
also making the current requirements for implementing an archiver 
(format) more flexible.

--
/Jacob Carlborg
Feb 16 2014
prev sibling next sibling parent reply "Orvid King" <blah38621 gmail.com> writes:
 Why not? Think of languages like C and C++, they only support pointers.  
 Pointers to basic types are not so interesting but pointers to structs  
 are.


Because, by serializing a pointer, you are implying that mechanism that  
will be deserializing the value both exists on the same machine, and lies  
within the same address space, otherwise it will be referencing incorrect  
data.



 If the same reference value is encountered multiple times, is it  
 serialized once or multiple times?


It is currently serialized multiple times. Serializing once would require  
a mechanism to exist on both the serializer and deserializer that  
understands and can interpret those references. As there is not a standard  
mechanism in JSON to support this, I haven't actually gotten around to  
implementing that.


 What? I'm referring to methods being called before and after  
 serialization of a given value.


Woops, that means I simply mis-understood your question. The answer to  
your actual question is somewhat, there is a single code path for dynamic  
types, I've only implemented support in a modified version of  
Destructionator's JSVar library, but it should be possible to add support  
to Variant without any real issue, that does support javascript's 2  
parameter json serialization, there are not however callbacks for the  
start and end of serialization.



 I prefer opt-out rather than opt-in.


Well, if I ever get around to cleaning up the codebase enough to submit it  
for inclusion in Phobos, there can be a nice long discussion about the  
pro's and con's of each, because it's a very simple check to remove.


 Can it serialize through base class references?


It currently retrieves all fields present in the class heirarchy, so if  
you have classes A, B, and C defined as follows:
class A
{
      optional
     int aA = 10;
}
class B : A
{
      serializeAs("Bob")
     int bob;
}
class C : B
{
     int cA;
      nonSerialized
     int cB;
}


Then calling toJSON on an instance of class C, and have modified the value  
of aA, it will produce a result containing fields defined as aA, Bob, and  
cA.

Because cB is marked as  nonSerialized, it is ignored during serialization.

Next, bob is serialized as Bob because it is marked as  serializeAs which  
is intended to account for the difference in naming conventions between  
different languages, in this case the source of the serialized data may  

are programming in D, where, if you're me at least, you use camelCase for  
fields.

Lastly, if we hadn't modified the value of aA, and it was still 10, it  
would not be included in serialization results, because it is marked as  
 optional, and contains the same value it would were it default  
constructed.
Feb 16 2014
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Sunday, 16 February 2014 at 22:41:59 UTC, Orvid King wrote:

 Why not? Think of languages like C and C++, they only support 
 pointers. Pointers to basic types are not so interesting but 
 pointers to structs are.


 Because, by serializing a pointer, you are implying that 
 mechanism that will be deserializing the value both exists on 
 the same machine, and lies within the same address space, 
 otherwise it will be referencing incorrect data.


No, it should just serialize the pointed value and make the same 
difference upon deserialization - if it is a value, write to it, 
otherwise allocate new instance on heap and assign its address.
Feb 16 2014
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/16/14, Dicebot <public dicebot.lv> wrote:

 No, it should just serialize the pointed value and make the same
 difference upon deserialization - if it is a value, write to it,
 otherwise allocate new instance on heap and assign its address.


Speaking of related things like pointers and cyclic references I have
support for this in my fork of msgpack, but the problem is that
msgpack is a defined format, so you can't arbitrarily implement your
own features without breaking the format:

https://github.com/msgpack/msgpack-d/issues/7

It's 11:48 PM here in case I'm completely off topic. :P
Feb 16 2014
parent "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Sunday, 16 February 2014 at 22:48:51 UTC, Andrej Mitrovic 
wrote:

 On 2/16/14, Dicebot <public dicebot.lv> wrote:

 No, it should just serialize the pointed value and make the 
 same
 difference upon deserialization - if it is a value, write to 
 it,
 otherwise allocate new instance on heap and assign its address.


 Speaking of related things like pointers and cyclic references 
 I have
 support for this in my fork of msgpack, but the problem is that
 msgpack is a defined format, so you can't arbitrarily implement 
 your
 own features without breaking the format:

 https://github.com/msgpack/msgpack-d/issues/7

 It's 11:48 PM here in case I'm completely off topic. :P


I just commented.
Maybe, we can support cyclic references by using ext type.

https://github.com/msgpack/msgpack-d/issues/7#issuecomment-35230360

I will try it.
Feb 16 2014
prev sibling parent "Jacob Carlborg" <doob me.com> writes:
On Sunday, 16 February 2014 at 22:41:59 UTC, Orvid King wrote:


 Because, by serializing a pointer, you are implying that 
 mechanism that will be deserializing the value both exists on 
 the same machine, and lies within the same address space, 
 otherwise it will be referencing incorrect data.


No. I'm serializing a pointer by dereferencing and serialize what 
it points to as I normally would. Then it's indicated in the 
serialized format it is a pointer. Then when deserializing I just 
use "new" to get a pointer and just set the values.


 It is currently serialized multiple times. Serializing once 
 would require a mechanism to exist on both the serializer and 
 deserializer that understands and can interpret those 
 references. As there is not a standard mechanism in JSON to 
 support this, I haven't actually gotten around to implementing 
 that.


I see.


 Woops, that means I simply mis-understood your question. The 
 answer to your actual question is somewhat, there is a single 
 code path for dynamic types, I've only implemented support in a 
 modified version of Destructionator's JSVar library, but it 
 should be possible to add support to Variant without any real 
 issue, that does support javascript's 2 parameter json 
 serialization, there are not however callbacks for the start 
 and end of serialization.


Ok.



 It currently retrieves all fields present in the class 
 heirarchy, so if you have classes A, B, and C defined as 
 follows:
 class A
 {
      optional
     int aA = 10;
 }
 class B : A
 {
      serializeAs("Bob")
     int bob;
 }
 class C : B
 {
     int cA;
      nonSerialized
     int cB;
 }


 Then calling toJSON on an instance of class C, and have 
 modified the value of aA, it will produce a result containing 
 fields defined as aA, Bob, and cA.

 Because cB is marked as  nonSerialized, it is ignored during 
 serialization.

 Next, bob is serialized as Bob because it is marked as 
  serializeAs which is intended to account for the difference in 
 naming conventions between different languages, in this case 

 the convention is typically PascalCase. We however are 
 programming in D, where, if you're me at least, you use 
 camelCase for fields.

 Lastly, if we hadn't modified the value of aA, and it was still 
 10, it would not be included in serialization results, because 
 it is marked as  optional, and contains the same value it would 
 were it default constructed.


I mean this case:

A c = new C;
toJSON(c); // the static type info of C is lost here

It seems you have limited yourself to what the JSON format 
supports. I tried to be as flexible as possible. My upcoming 
modifications to Orange, or rather std.serialization am working 
on, will have some small differences to the current API of 
Orange. Although I'm hoping it will be much more flexible then 
current API.

--
/Jacob Carlborg
Feb 16 2014
prev sibling parent reply Rory McGuire <rjmcguire gmail.com> writes:
A base class reference is: from your example an A which actually contains a
C or B.

Honestly I haven't tried this I would have assumed that D still gives you
the real type when using reflection but have you tried it?


On Mon, Feb 17, 2014 at 12:18 AM, Orvid King <blah38621 gmail.com> wrote:


 Why not? Think of languages like C and C++, they only support pointers.

 Pointers to basic types are not so interesting but pointers to structs are.


 Because, by serializing a pointer, you are implying that mechanism that
 will be deserializing the value both exists on the same machine, and lies
 within the same address space, otherwise it will be referencing incorrect
 data.



  If the same reference value is encountered multiple times, is it

 serialized once or multiple times?


 It is currently serialized multiple times. Serializing once would require
 a mechanism to exist on both the serializer and deserializer that
 understands and can interpret those references. As there is not a standard
 mechanism in JSON to support this, I haven't actually gotten around to
 implementing that.


  What? I'm referring to methods being called before and after

 serialization of a given value.


 Woops, that means I simply mis-understood your question. The answer to
 your actual question is somewhat, there is a single code path for dynamic
 types, I've only implemented support in a modified version of
 Destructionator's JSVar library, but it should be possible to add support
 to Variant without any real issue, that does support javascript's 2
 parameter json serialization, there are not however callbacks for the start
 and end of serialization.



  I prefer opt-out rather than opt-in.


 Well, if I ever get around to cleaning up the codebase enough to submit it
 for inclusion in Phobos, there can be a nice long discussion about the
 pro's and con's of each, because it's a very simple check to remove.


  Can it serialize through base class references?


 It currently retrieves all fields present in the class heirarchy, so if
 you have classes A, B, and C defined as follows:
 class A
 {
      optional
     int aA = 10;
 }
 class B : A
 {
      serializeAs("Bob")
     int bob;
 }
 class C : B
 {
     int cA;
      nonSerialized
     int cB;
 }


 Then calling toJSON on an instance of class C, and have modified the value
 of aA, it will produce a result containing fields defined as aA, Bob, and
 cA.

 Because cB is marked as  nonSerialized, it is ignored during serialization.

 Next, bob is serialized as Bob because it is marked as  serializeAs which
 is intended to account for the difference in naming conventions between
 different languages, in this case the source of the serialized data may

 are programming in D, where, if you're me at least, you use camelCase for
 fields.

 Lastly, if we hadn't modified the value of aA, and it was still 10, it
 would not be included in serialization results, because it is marked as
  optional, and contains the same value it would were it default constructed.





 Feb 16 2014



  parent reply "Jacob Carlborg" <doob me.com>  writes:


On Monday, 17 February 2014 at 06:09:35 UTC, Rory McGuire wrote:


 A base class reference is: from your example an A which 
 actually contains a
 C or B.

 Honestly I haven't tried this I would have assumed that D still 
 gives you
 the real type when using reflection but have you tried it?


Unfortunately the only why to get a value of field by reflection 
requires the static type to be known. When serializing through a 
base class reference the static type is lost. Therefore it's 
required to register the subclass, one way or another.

--
/Jacob Carlborg



 Feb 16 2014



 next sibling parent reply Rory McGuire <rjmcguire gmail.com>  writes:


Interesting, do you have to run though all registered classes that are
sub-classes of the base class and cast them checking for null?


On Mon, Feb 17, 2014 at 9:24 AM, Jacob Carlborg <doob me.com> wrote:


 On Monday, 17 February 2014 at 06:09:35 UTC, Rory McGuire wrote:

  A base class reference is: from your example an A which actually contains

 a
 C or B.

 Honestly I haven't tried this I would have assumed that D still gives you
 the real type when using reflection but have you tried it?


 Unfortunately the only why to get a value of field by reflection requires
 the static type to be known. When serializing through a base class
 reference the static type is lost. Therefore it's required to register the
 subclass, one way or another.

 --
 /Jacob Carlborg





 Feb 17 2014



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2014-02-17 13:20, Rory McGuire wrote:

 Interesting, do you have to run though all registered classes that are
 sub-classes of the base class and cast them checking for null?


No, that's not required. When a subclass is registered I'm storing a 
templated delegate which performs the downcast. Have a look at these two 
methods:

https://github.com/jacob-carlborg/orange/blob/master/orange/serialization/Serializer.d#L241-L262

-- 
/Jacob Carlborg



 Feb 17 2014



  parent  Rory McGuire <rjmcguire gmail.com>  writes:


On Mon, Feb 17, 2014 at 10:10 PM, Jacob Carlborg <doob me.com> wrote:

 No, that's not required. When a subclass is registered I'm storing a
 templated delegate which performs the downcast. Have a look at these two
 methods:

 https://github.com/jacob-carlborg/orange/blob/master/orange/serialization/
 Serializer.d#L241-L262

 --
 /Jacob Carlborg


interesting thanks.



 Feb 17 2014



prev sibling  parent  "Atila Neves" <atila.neves gmail.com>  writes:


On Monday, 17 February 2014 at 07:24:11 UTC, Jacob Carlborg wrote:

 On Monday, 17 February 2014 at 06:09:35 UTC, Rory McGuire wrote:


 A base class reference is: from your example an A which 
 actually contains a
 C or B.

 Honestly I haven't tried this I would have assumed that D 
 still gives you
 the real type when using reflection but have you tried it?


 Unfortunately the only why to get a value of field by 
 reflection requires the static type to be known. When 
 serializing through a base class reference the static type is 
 lost. Therefore it's required to register the subclass, one way 
 or another.

 --
 /Jacob Carlborg


Yeah, I still have to implement that for cerealed.

Atila



 Feb 19 2014