www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Orange - Free from D1/Tango

reply Jacob Carlborg <doob me.com> writes:
I just stripped out all D1 and Tango related code from Orange. D1/Tango 
is still supported in the d1 branch. Hopefully this will make it easier 
to integrate into Phobos.

It also now supports UDA's for indicating a field/class/struct shouldn't 
be serialized:

class Foo
{
      nonSerialized int a;
}

 nonSerialized class Bar { }

For those unfamiliar with Orange, it's a serialization library. More 
information available at github:

https://github.com/jacob-carlborg/orange

-- 
/Jacob Carlborg
Feb 17 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange. D1/Tango is
still
 supported in the d1 branch. Hopefully this will make it easier to integrate
into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct shouldn't be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


Hmm, shouldn't it be the other way around - marking the ones to be serialized?
Feb 17 2013
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 18 February 2013 07:18, Walter Bright <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:


 I just stripped out all D1 and Tango related code from Orange. D1/Tango
 is still
 supported in the d1 branch. Hopefully this will make it easier to
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct shouldn't
 be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to be
 serialized?


I think both are useful, but I would expect opt-in rather than opt out as
Walter asserts. For instance:

 serialise class Foo // everything in the class
{
     int a;
      noSerialise int cacheValue; // except this
}
Feb 17 2013
parent reply "js.mdnq" <js_adddot+mdng gmail.com> writes:
On Sunday, 17 February 2013 at 21:31:31 UTC, Manu wrote:

 On 18 February 2013 07:18, Walter Bright 
 <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:


 I just stripped out all D1 and Tango related code from 
 Orange. D1/Tango
 is still
 supported in the d1 branch. Hopefully this will make it 
 easier to
 integrate into
 Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct shouldn't
 be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones 
 to be
 serialized?


 I think both are useful, but I would expect opt-in rather than 
 opt out as
 Walter asserts. For instance:

  serialise class Foo // everything in the class
 {
      int a;
       noSerialise int cacheValue; // except this
 }


yes, I think this is most useful, and:

 noSerialise class Foo // nothing in the class
{
      int a;
       Serialise int cacheValue; // except this
}
Feb 20 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-20 14:08, js.mdnq wrote:


 yes, I think this is most useful, and:

  noSerialise class Foo // nothing in the class
 {
       int a;
        Serialise int cacheValue; // except this
 }


I don't see any point in having both.

-- 
/Jacob Carlborg
Feb 20 2013
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Feb 17, 2013 at 01:18:05PM -0800, Walter Bright wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

I just stripped out all D1 and Tango related code from Orange.
D1/Tango is still supported in the d1 branch. Hopefully this will
make it easier to integrate into Phobos.

It also now supports UDA's for indicating a field/class/struct
shouldn't be serialized:

class Foo
{
      nonSerialized int a;
}

 nonSerialized class Bar { }


 
 Hmm, shouldn't it be the other way around - marking the ones to be
 serialized?


I believe the purpose of the library is to provide convenient saving /
loading of objects (among other things), so you want most of your data
serialized, and non-serialized fields are the exception rather than the
norm.  So it makes sense to serialize by default and mark out fields
that shouldn't be serialized.


T

-- 
"640K ought to be enough" -- Bill G., 1984. "The Internet is not a
primary goal for PC usage" -- Bill G., 1995. "Linux has no impact on
Microsoft's strategy" -- Bill G., 1999.
Feb 17 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2013 1:43 PM, H. S. Teoh wrote:

 I believe the purpose of the library is to provide convenient saving /
 loading of objects (among other things), so you want most of your data
 serialized, and non-serialized fields are the exception rather than the
 norm.  So it makes sense to serialize by default and mark out fields
 that shouldn't be serialized.


I've done some work with serialization, and it was a minority of the data types 
that needed serialization.
Feb 17 2013
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Feb 17, 2013 at 07:22:03PM -0800, Walter Bright wrote:

 On 2/17/2013 1:43 PM, H. S. Teoh wrote:

I believe the purpose of the library is to provide convenient saving /
loading of objects (among other things), so you want most of your data
serialized, and non-serialized fields are the exception rather than the
norm.  So it makes sense to serialize by default and mark out fields
that shouldn't be serialized.


 
 I've done some work with serialization, and it was a minority of the
 data types that needed serialization.


Wait, I thought the whole point was that the user code specifically asks
for the serialization of a particular data structure, so presumeably
most of the objects recursively referenced by that object will tend to
be data objects where the vast majority of fields should be serialized.

I don't think the point was to blindly serialize every object in the
program?


T

-- 
If Java had true garbage collection, most programs would delete
themselves upon execution. -- Robert Sewell
Feb 17 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-17 22:43, H. S. Teoh wrote:


 I believe the purpose of the library is to provide convenient saving /
 loading of objects (among other things), so you want most of your data
 serialized, and non-serialized fields are the exception rather than the
 norm.  So it makes sense to serialize by default and mark out fields
 that shouldn't be serialized.


Exactly, Orange has had this behavior from day one.

-- 
/Jacob Carlborg
Feb 17 2013
prev sibling next sibling parent reply "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 17 Feb 2013 13:18:05 -0800, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange. D1/Tango  
 is still
 supported in the d1 branch. Hopefully this will make it easier to  
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct  
 shouldn't be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to be  
 serialized?


Every serialization library i've used is opt-out.

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
Feb 17 2013
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 17 February 2013 at 21:52:53 UTC, Adam Wilson wrote:

 On Sun, 17 Feb 2013 13:18:05 -0800, Walter Bright 
 <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from 
 Orange. D1/Tango is still
 supported in the d1 branch. Hopefully this will make it 
 easier to integrate into
 Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones 
 to be serialized?


 Every serialization library i've used is opt-out.


That doesn't mean it is the way.

I have to say I don't know if it is better to opt in or out, 
simply that most serialization framewrok I have used as of now 
weren't that great, and I'm sure we can do better with both 
compile time reflection and annotations.
Feb 17 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-17 22:52, Adam Wilson wrote:


 Every serialization library i've used is opt-out.


Java - Requires annotation
Ruby - Doesn't require anything

-- 
/Jacob Carlborg
Feb 17 2013
parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Monday, 18 February 2013 at 07:36:07 UTC, Jacob Carlborg wrote:

 On 2013-02-17 22:52, Adam Wilson wrote:


 Every serialization library i've used is opt-out.


 Java - Requires annotation
 Ruby - Doesn't require anything


.NET languages as well.
Feb 20 2013
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 17 Feb 2013 16:18:05 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange. D1/Tango  
 is still
 supported in the d1 branch. Hopefully this will make it easier to  
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct  
 shouldn't be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to be  
 serialized?


I would think D's type system would be capable enough where a  
serialization library can tell whether a type can be serialized or not.

I also think it would be a tremendous burden to put  serialized on every  
type that could be serialized.


annotate every type I wanted to serialize, and every type that was used  
within that type.

What is the concern with an opt-out approach?  I would think the default  
would be the most common option -- save everything.

-Steve
Feb 17 2013
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 17 February 2013 at 22:13:21 UTC, Steven Schveighoffer 
wrote:

 On Sun, 17 Feb 2013 16:18:05 -0500, Walter Bright 
 <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from 
 Orange. D1/Tango is still
 supported in the d1 branch. Hopefully this will make it 
 easier to integrate into
 Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones 
 to be serialized?


 I would think D's type system would be capable enough where a 
 serialization library can tell whether a type can be serialized 
 or not.


It isn't as it don't convey ownership. And hopefully as getting 
rid of ownership is important for idioms involving immutability.
Feb 17 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 17 Feb 2013 22:08:26 -0500, deadalnix <deadalnix gmail.com> wrote:


 On Sunday, 17 February 2013 at 22:13:21 UTC, Steven Schveighoffer wrote:

 On Sun, 17 Feb 2013 16:18:05 -0500, Walter Bright  
 <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange.  
 D1/Tango is still
 supported in the d1 branch. Hopefully this will make it easier to  
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct  
 shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to be  
 serialized?


 I would think D's type system would be capable enough where a  
 serialization library can tell whether a type can be serialized or not.


 It isn't as it don't convey ownership. And hopefully as getting rid of  
 ownership is important for idioms involving immutability.


Right, but in that case, you may have a common reference serialized  
several times.  Not necessarily the end of the world.

Perhaps what we need is two things:

1. A UDA that indicates "this type is certified correct for serialization"
2. A flag to serializer somehow that indicates "only serialize types that  
are certified correct for serialization, and pure value types"

I would hate to (and have hated to) mark serializable types when it is a  
trivial type, just because it's used inside a serializable type.

-Steve
Feb 17 2013
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 18 February 2013 at 03:22:07 UTC, Steven Schveighoffer 
wrote:

 Right, but in that case, you may have a common reference 
 serialized several times.  Not necessarily the end of the world.


This is a problem as D promote the concept of identity in 
addition to value.
Feb 17 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 04:22, Steven Schveighoffer wrote:


 Right, but in that case, you may have a common reference serialized
 several times.  Not necessarily the end of the world.


No reference type should be serialized more than once. If it is, it's bug.


 Perhaps what we need is two things:

 1. A UDA that indicates "this type is certified correct for serialization"
 2. A flag to serializer somehow that indicates "only serialize types
 that are certified correct for serialization, and pure value types"


Hmm, I'm wondering if that's getting too complex.


 I would hate to (and have hated to) mark serializable types when it is a
 trivial type, just because it's used inside a serializable type.


Me too.

-- 
/Jacob Carlborg
Feb 17 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 18 Feb 2013 02:42:50 -0500, Jacob Carlborg <doob me.com> wrote:


 On 2013-02-18 04:22, Steven Schveighoffer wrote:


 Right, but in that case, you may have a common reference serialized
 several times.  Not necessarily the end of the world.


 No reference type should be serialized more than once. If it is, it's  
 bug.


Then maybe I didn't understand deadalnix's point, or maybe you have solved  
his problem?  I don't know enough about your library.


 Perhaps what we need is two things:

 1. A UDA that indicates "this type is certified correct for  
 serialization"
 2. A flag to serializer somehow that indicates "only serialize types
 that are certified correct for serialization, and pure value types"


 Hmm, I'm wondering if that's getting too complex.


If there isn't a problem, then we don't need a solution.  But if there is  
a problem, we may need something like this.

-Steve
Feb 18 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 15:32, Steven Schveighoffer wrote:


 Then maybe I didn't understand deadalnix's point, or maybe you have
 solved his problem?  I don't know enough about your library.


I didn't really understand deadalnix's point. But I answered your 
comment. My answer to your comment is true but I don't know if that has 
anything to do with what deadalnix said.


 If there isn't a problem, then we don't need a solution.  But if there
 is a problem, we may need something like this.


I don't know, is there a problem?

-- 
/Jacob Carlborg
Feb 18 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 18 Feb 2013 10:09:30 -0500, Jacob Carlborg <doob me.com> wrote:


 On 2013-02-18 15:32, Steven Schveighoffer wrote:


 Then maybe I didn't understand deadalnix's point, or maybe you have
 solved his problem?  I don't know enough about your library.


 I didn't really understand deadalnix's point. But I answered your  
 comment. My answer to your comment is true but I don't know if that has  
 anything to do with what deadalnix said.


Hm... maybe something like this:

class Widget
{
   Display d;
}

If you serialize Widget, then it will try to serialize d, but d is not  
owned by Widget, it's just referenced by Widget.

How does Orange deal with this?  What if whoever wrote Widget never  
intended it to be serialized, but wasn't aware of the serialization  
library, so didn't know to mark it as  nonSerialized

-Steve
Feb 18 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 18:46, Steven Schveighoffer wrote:


 Hm... maybe something like this:

 class Widget
 {
    Display d;
 }

 If you serialize Widget, then it will try to serialize d, but d is not
 owned by Widget, it's just referenced by Widget.

 How does Orange deal with this?


The first time Orange sees "d" it will just serialize it. If there's any 
other reference to "d", somewhere in the data structure that is being 
serialized, it will serialize a reference.

If "d" is not owned by Widget it will serialize it anyway. It doesn't 
care where it comes from.

This code:

http://pastebin.com/n1jwG9vE

Will serialize to this XML:

http://pastebin.com/tHbzVRqZ


 What if whoever wrote Widget never
 intended it to be serialized, but wasn't aware of the serialization
 library, so didn't know to mark it as  nonSerialized


I don't know if there is solution for that if the serialization library 
is using an opt-out approach. The one serializing the data structure is 
responsible.

-- 
/Jacob Carlborg
Feb 18 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
 I would think D's type system would be capable enough where a
 serialization library can tell whether a type can be serialized or not.


Sometimes you don't want to serialize a field, regardless if the 
serialization library can serialize it or not. This will require a 
manual annotation.


 I also think it would be a tremendous burden to put  serialized on every
 type that could be serialized.


 annotate every type I wanted to serialize, and every type that was used
 within that type.

 What is the concern with an opt-out approach?  I would think the default
 would be the most common option -- save everything.


I agree.

-- 
/Jacob Carlborg
Feb 17 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 17 February 2013 at 21:18:12 UTC, Walter Bright wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange. 
 D1/Tango is still
 supported in the d1 branch. Hopefully this will make it easier 
 to integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct 
 shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to 
 be serialized?


Surely not...

I shouldn't have to explicitly mark every type as 
allowed-to-be-serialised, it makes much more sense to opt-out.
Feb 17 2013
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sun, 17 Feb 2013 13:18:05 -0800
Walter Bright <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange.
 D1/Tango is still supported in the d1 branch. Hopefully this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a field/class/struct
 shouldn't be serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


 
 Hmm, shouldn't it be the other way around - marking the ones to be
 serialized?
 


You're already opting-in to serialization anyway when you say
"serialize object foobar".  serializable would just be redundant.
Feb 17 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky  
<SeeWebsiteToContactMe semitwist.com> wrote:


 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange.
 D1/Tango is still supported in the d1 branch. Hopefully this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a field/class/struct
 shouldn't be serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to be
 serialized?


 You're already opting-in to serialization anyway when you say
 "serialize object foobar".  serializable would just be redundant.


I think Walter's point is that the author of foobar may not have opted in.

My response to that is, so?  If someone is trying to serialize your class,  
and you didn't test for that, too bad for that person if it doesn't work.

The reality is, an author may write foobar without intending it to be  
serializable, but it actually is.  In that case, it is still on the user,  
but if it works, great!  The user is taking the risk that later the  
serialization breaks, and could always submit a patch to the author of  
foobar if it somehow becomes not-working in a future version.

To have to say:

struct S
{
   int x;
   int y;
}

is serializable seems like super-redundant info.  The D type system is one  
of the most advanced I've ever seen.  Let's try and use it!

-Steve
Feb 17 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 02:27, Steven Schveighoffer wrote:


 I think Walter's point is that the author of foobar may not have opted in.

 My response to that is, so?  If someone is trying to serialize your
 class, and you didn't test for that, too bad for that person if it
 doesn't work.

 The reality is, an author may write foobar without intending it to be
 serializable, but it actually is.  In that case, it is still on the
 user, but if it works, great!  The user is taking the risk that later
 the serialization breaks, and could always submit a patch to the author
 of foobar if it somehow becomes not-working in a future version.

 To have to say:

 struct S
 {
    int x;
    int y;
 }

 is serializable seems like super-redundant info.  The D type system is
 one of the most advanced I've ever seen.  Let's try and use it!


I agree.

-- 
/Jacob Carlborg
Feb 17 2013
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
18-Feb-2013 05:27, Steven Schveighoffer пишет:

 On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky
 <SeeWebsiteToContactMe semitwist.com> wrote:


 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange.
 D1/Tango is still supported in the d1 branch. Hopefully this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a field/class/struct
 shouldn't be serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to be
 serialized?


 You're already opting-in to serialization anyway when you say
 "serialize object foobar".  serializable would just be redundant.


 I think Walter's point is that the author of foobar may not have opted in.

 My response to that is, so?  If someone is trying to serialize your
 class, and you didn't test for that, too bad for that person if it
 doesn't work.

 The reality is, an author may write foobar without intending it to be
 serializable, but it actually is.  In that case, it is still on the
 user, but if it works, great!  The user is taking the risk that later
 the serialization breaks, and could always submit a patch to the author
 of foobar if it somehow becomes not-working in a future version.

 To have to say:

 struct S
 {
    int x;
    int y;
 }

 is serializable seems like super-redundant info.  The D type system is
 one of the most advanced I've ever seen.  Let's try and use it!


Exactly.

Would be nice to be able to mark the whole structs as non-serializable 
though. And of course, to not have to import serialization library just 
to get the annotations.

 -Steve



-- 
Dmitry Olshansky
Feb 18 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 10:43, Dmitry Olshansky wrote:


 Exactly.

 Would be nice to be able to mark the whole structs as non-serializable
 though. And of course, to not have to import serialization library just
 to get the annotations.


import orange.serialization.Serializable;

 nonSerialized class Bar { }

-- 
/Jacob Carlborg
Feb 18 2013
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
18-Feb-2013 14:28, Jacob Carlborg пишет:

 On 2013-02-18 10:43, Dmitry Olshansky wrote:


 Exactly.

 Would be nice to be able to mark the whole structs as non-serializable
 though. And of course, to not have to import serialization library just
 to get the annotations.


 import orange.serialization.Serializable;

  nonSerialized class Bar { }


Good.

It's just would be really nice to have  nonSerialized and simillar 
somewhere in object.di with prescribed semantics. Then e.g. any 
serialization libraries including some 3rd party ones will play
by the same basic rules as far as user is concerned.

-- 
Dmitry Olshansky
Feb 18 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 11:56, Dmitry Olshansky wrote:


 Good.

 It's just would be really nice to have  nonSerialized and simillar
 somewhere in object.di with prescribed semantics. Then e.g. any
 serialization libraries including some 3rd party ones will play
 by the same basic rules as far as user is concerned.


Yeah, when/if in Phobos that would be a good idea.

-- 
/Jacob Carlborg
Feb 18 2013
parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 18 Feb 2013 12:03:29 +0100
schrieb Jacob Carlborg <doob me.com>:


 On 2013-02-18 11:56, Dmitry Olshansky wrote:
 

 Good.

 It's just would be really nice to have  nonSerialized and simillar
 somewhere in object.di with prescribed semantics. Then e.g. any
 serialization libraries including some 3rd party ones will play
 by the same basic rules as far as user is concerned.


 
 Yeah, when/if in Phobos that would be a good idea.
 


Does this really belong into object.di? It adds more stuff to the
global namespace (and druntime). Couldn't we just place "standard,
library attributes" into std.attributes or something similar?
Feb 18 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 12:51, Johannes Pfau wrote:


 Does this really belong into object.di? It adds more stuff to the
 global namespace (and druntime). Couldn't we just place "standard,
 library attributes" into std.attributes or something similar?


That's true. I don't know what's best.

-- 
/Jacob Carlborg
Feb 18 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
18-Feb-2013 15:51, Johannes Pfau пишет:

 Am Mon, 18 Feb 2013 12:03:29 +0100
 schrieb Jacob Carlborg <doob me.com>:


 On 2013-02-18 11:56, Dmitry Olshansky wrote:


 Good.

 It's just would be really nice to have  nonSerialized and simillar
 somewhere in object.di with prescribed semantics. Then e.g. any
 serialization libraries including some 3rd party ones will play
 by the same basic rules as far as user is concerned.


 Yeah, when/if in Phobos that would be a good idea.


 Does this really belong into object.di? It adds more stuff to the
 global namespace (and druntime). Couldn't we just place "standard,
 library attributes" into std.attributes or something similar?


I think it doesn't matter and shouldn't become a roadblock to its 
inclusion someplace standard. But if it's in object.d or core.attributes 
then druntime too can make use of it. It's a recurring problem that 
druntime has no access to certain useful things like std.traits simply 
because of their placement in fs ;) and I'd hate to see this problem 
repeat again.

I also suspect that it'd be around 10-20 attributes tops.

-- 
Dmitry Olshansky
Feb 18 2013
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 18 February 2013 at 12:28:01 UTC, Dmitry Olshansky 
wrote:

 18-Feb-2013 15:51, Johannes Pfau пишет:

 Am Mon, 18 Feb 2013 12:03:29 +0100
 schrieb Jacob Carlborg <doob me.com>:


 On 2013-02-18 11:56, Dmitry Olshansky wrote:


 Good.

 It's just would be really nice to have  nonSerialized and 
 simillar
 somewhere in object.di with prescribed semantics. Then e.g. 
 any
 serialization libraries including some 3rd party ones will 
 play
 by the same basic rules as far as user is concerned.


 Yeah, when/if in Phobos that would be a good idea.


 Does this really belong into object.di? It adds more stuff to 
 the
 global namespace (and druntime). Couldn't we just place 
 "standard,
 library attributes" into std.attributes or something similar?


 I think it doesn't matter and shouldn't become a roadblock to 
 its inclusion someplace standard. But if it's in object.d or 
 core.attributes then druntime too can make use of it. It's a 
 recurring problem that druntime has no access to certain useful 
 things like std.traits simply because of their placement in fs 
 ;) and I'd hate to see this problem repeat again.

 I also suspect that it'd be around 10-20 attributes tops.


This has no meaning without the lib.

This belong with the lib or in a separated module that the lib 
uses as well.
Feb 18 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 13:27, Dmitry Olshansky wrote:


 I think it doesn't matter and shouldn't become a roadblock to its
 inclusion someplace standard. But if it's in object.d or core.attributes
 then druntime too can make use of it. It's a recurring problem that
 druntime has no access to certain useful things like std.traits simply
 because of their placement in fs ;) and I'd hate to see this problem
 repeat again.


Yeah, druntime you use this attribute if it was added. We don't want to 
serialize threads and IO objects.

-- 
/Jacob Carlborg
Feb 18 2013
prev sibling parent reply "qznc" <qznc go.to> writes:
On Monday, 18 February 2013 at 01:27:17 UTC, Steven Schveighoffer 
wrote:

 On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky 
 <SeeWebsiteToContactMe semitwist.com> wrote:


 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from 
 Orange.
 D1/Tango is still supported in the d1 branch. Hopefully 
 this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct
 shouldn't be serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones 
 to be
 serialized?


 You're already opting-in to serialization anyway when you say
 "serialize object foobar".  serializable would just be 
 redundant.


 I think Walter's point is that the author of foobar may not 
 have opted in.

 My response to that is, so?  If someone is trying to serialize 
 your class, and you didn't test for that, too bad for that 
 person if it doesn't work.

 The reality is, an author may write foobar without intending it 
 to be serializable, but it actually is.  In that case, it is 
 still on the user, but if it works, great!  The user is taking 
 the risk that later the serialization breaks, and could always 
 submit a patch to the author of foobar if it somehow becomes 
 not-working in a future version.

 To have to say:

 struct S
 {
   int x;
   int y;
 }

 is serializable seems like super-redundant info.  The D type 
 system is one of the most advanced I've ever seen.  Let's try 
 and use it!


I agree that for lots of code it is redundant (data structures in 
general). It might even be desirable for library code, whose 
author did not consider Orange. However, it might lead into 
subtle bugs in some cases.

For example, what happens with a mutex? Would Orange serialize it 
with all its state (locked or unlocked)?

What happens with data structures, which are used in parallel. 
For a consistent serialization it might be necessary to take a 
lock first. In this case, the data structure must be able to 
provide its own serialization method.

I have not seen a "skip on serialize, but re-initialize on 
deserialize" annotation, which would be the correct behavior for 
uncontested thread-safe data structures using locks.

The most complex case would probably be something like a GUI 
toolkit. If I serialize a GtkD app on Linux and deserialize it on 
Windows, will it be able to produce a working GUI? It must 
provide a custom deserialize method to do that.

Looking at all those edge cases, an opt-in approach seems not 
that stupid to me. It might be tedious to add all those 
 serializeable annotations, but at least you do not run into 
weird deadlocks.
Feb 18 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 18 Feb 2013 18:43:16 -0500, qznc <qznc go.to> wrote:


 On Monday, 18 February 2013 at 01:27:17 UTC, Steven Schveighoffer wrote:

 On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky  
 <SeeWebsiteToContactMe semitwist.com> wrote:


 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:


 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from > Orange.
 D1/Tango is still supported in the d1 branch. Hopefully > this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a > field/class/struct
 shouldn't be serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to be
 serialized?


 You're already opting-in to serialization anyway when you say
 "serialize object foobar".  serializable would just be redundant.


 I think Walter's point is that the author of foobar may not have opted  
 in.

 My response to that is, so?  If someone is trying to serialize your  
 class, and you didn't test for that, too bad for that person if it  
 doesn't work.

 The reality is, an author may write foobar without intending it to be  
 serializable, but it actually is.  In that case, it is still on the  
 user, but if it works, great!  The user is taking the risk that later  
 the serialization breaks, and could always submit a patch to the author  
 of foobar if it somehow becomes not-working in a future version.

 To have to say:

 struct S
 {
   int x;
   int y;
 }

 is serializable seems like super-redundant info.  The D type system is  
 one of the most advanced I've ever seen.  Let's try and use it!


 I agree that for lots of code it is redundant (data structures in  
 general). It might even be desirable for library code, whose author did  
 not consider Orange. However, it might lead into subtle bugs in some  
 cases.

 For example, what happens with a mutex? Would Orange serialize it with  
 all its state (locked or unlocked)?


Most likely a mutex is a core library feature, and we can mark it as not  
serializeable.  If (when?) phobos gets serialization, I would expect the  
core types to be marked as such.


 What happens with data structures, which are used in parallel. For a  
 consistent serialization it might be necessary to take a lock first. In  
 this case, the data structure must be able to provide its own  
 serialization method.


For D, we have shared which indicates it may be viewed by more than one  
thread.  Shared data could be opt-in for serialization.


 I have not seen a "skip on serialize, but re-initialize on deserialize"  
 annotation, which would be the correct behavior for uncontested  
 thread-safe data structures using locks.

 The most complex case would probably be something like a GUI toolkit. If  
 I serialize a GtkD app on Linux and deserialize it on Windows, will it  
 be able to produce a working GUI? It must provide a custom deserialize  
 method to do that.


I would expect that something that complex has to be specifically written  
to deal with serialization.


 Looking at all those edge cases, an opt-in approach seems not that  
 stupid to me. It might be tedious to add all those  serializeable  
 annotations, but at least you do not run into weird deadlocks.


Just because someone is stupid and tries to serialize a non-serializable  
construct such as a gui toolkit, it's not the serializer's fault.

I don't think opt-in is stupid, it's just a more conservative set of  
rules.  I think the number of constructs that WON'T be serializable will  
be far less than the ones that will be.

And multi-thread access items could be made opt-in.  I'm not sure what  
Orange does now.

-Steve
Feb 18 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-19 04:26, Steven Schveighoffer wrote:


 Most likely a mutex is a core library feature, and we can mark it as not
 serializeable.  If (when?) phobos gets serialization, I would expect the
 core types to be marked as such.


Exactly. IO objects, threads are also included in this category.


 I would expect that something that complex has to be specifically
 written to deal with serialization.


Yes, exactly, see my other reply:

http://forum.dlang.org/thread/kfrfs4$2h6f$1 digitalmars.com?page=4#post-kfvard:242nfi:241:40digitalmars.com


 Just because someone is stupid and tries to serialize a non-serializable
 construct such as a gui toolkit, it's not the serializer's fault.

 I don't think opt-in is stupid, it's just a more conservative set of
 rules.  I think the number of constructs that WON'T be serializable will
 be far less than the ones that will be.


I agree.


 And multi-thread access items could be made opt-in.  I'm not sure what
 Orange does now.


Orange just serialize everything. I haven't actually tried with anything 
marked as "shared". Perhaps I should do that.

-- 
/Jacob Carlborg
Feb 18 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-19 00:43, qznc wrote:


 I agree that for lots of code it is redundant (data structures in
 general). It might even be desirable for library code, whose author did
 not consider Orange. However, it might lead into subtle bugs in some cases.

 For example, what happens with a mutex? Would Orange serialize it with
 all its state (locked or unlocked)?


It would just serialize it. It serialize all internal sate, except for 
void pointers.


 What happens with data structures, which are used in parallel. For a
 consistent serialization it might be necessary to take a lock first. In
 this case, the data structure must be able to provide its own
 serialization method.


Yes, or perhaps the user can use a lock or similar when perform the 
serialization?


 I have not seen a "skip on serialize, but re-initialize on deserialize"
 annotation, which would be the correct behavior for uncontested
 thread-safe data structures using locks.


The library supports custom serialization, both for your own types and 
for third party types.

https://github.com/jacob-carlborg/orange/blob/master/tests/Custom.d
https://github.com/jacob-carlborg/orange/blob/master/tests/NonIntrusive.d


 The most complex case would probably be something like a GUI toolkit. If
 I serialize a GtkD app on Linux and deserialize it on Windows, will it
 be able to produce a working GUI? It must provide a custom deserialize
 method to do that.


Yes, it will most likely need a custom (de)serialization method. If we 
take DWT as an example. It could automatically serialize all internal 
state except for the native widget that is stored. So in this case it 
would probably be best to use custom serialization and only use the 
public API to set the properties. This would make sure that the 
underlying native widget is updated as well.

Theoretically it would be possible to serialize the native widget as 
well, at least on Mac OS X. That's how Xcode/InterfaceBuilder is working.


 Looking at all those edge cases, an opt-in approach seems not that
 stupid to me. It might be tedious to add all those  serializeable
 annotations, but at least you do not run into weird deadlocks.


I don't agree. I think that most data types _can_ be serialized.

-- 
/Jacob Carlborg
Feb 18 2013
prev sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Sunday, 17 February 2013 at 21:18:12 UTC, Walter Bright wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange. 
 D1/Tango is still
 supported in the d1 branch. Hopefully this will make it easier 
 to integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct 
 shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }


 Hmm, shouldn't it be the other way around - marking the ones to 
 be serialized?


The vast majority of objects can be serialized without any 
problems. Those that can't usually aren't referenced in objects 
that are being serialized and you know what they are ahead of 
time. Forcing opt-in serialization would make every single 
library out there at the moment unusable with std.serialize. In a 
Component system for example, you won't know ahead of time what 
needs to be serialized, and a single object not being marked 
Serialized will completely prevent serialization.

That being said, personally I think serialization in Phobos has 
been delayed far too long for something that's so important to 
many programs. I'd be happy with either opt-in or opt-out, 
however I believe that there should be a way to customize how an 
object gets serialized, such as an onSerialize method or 
interface.
Feb 18 2013