www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - pass-by-ref semantics for structs (was Deque impl.)

reply d coder <dlang.coder gmail.com> writes:
On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra <monarchdodra gmail.com> wrote:

 The pull is kind of stuck in limbo, specifically because of the problems
 associated with implementing reference semantics with structs :/



Thanks for the enlightening email.

I am of the considered view that reference semantics with structs in D
is tough (if not impossible) with default constructor and postblit
constructor (used when passing objects). This is because you can not
initialize any object (wrapped in the struct) in the default
constructor and if you are passing the struct as a function parameter,
it is not possible to initialize these internal objects (before
passing them) in the postblit constructor. I faced this in some of my
code. Do not know if you are facing the same scenario in the DList
implementation.

Are there any possible solutions in the pipe?

Regards
- Puneet
Jan 31 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 12:28:00 UTC, d coder wrote:

 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra 
 <monarchdodra gmail.com> wrote:

 The pull is kind of stuck in limbo, specifically because of 
 the problems
 associated with implementing reference semantics with structs 
 :/



 Thanks for the enlightening email.

 I am of the considered view that reference semantics with 
 structs in D
 is tough (if not impossible) with default constructor and 
 postblit
 constructor (used when passing objects). This is because you 
 can not
 initialize any object (wrapped in the struct) in the default
 constructor and if you are passing the struct as a function 
 parameter,
 it is not possible to initialize these internal objects (before
 passing them) in the postblit constructor. I faced this in some 
 of my
 code. Do not know if you are facing the same scenario in the 
 DList
 implementation.

 Are there any possible solutions in the pipe?

 Regards
 - Puneet


This keeps coming up, so I'll wrap up how it goes:

Conclusions:
* The language will not change.
* Use "static S opCall()" to emulate no-arg constructor

The conversation:
- "But opCall is not good enough: It's not a constructor"
- "But it's all you'll get"

If you want some history, specifically, you need to look for 
"no-arg constructor".
Jan 31 2013
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 07:27:08 -0500, d coder <dlang.coder gmail.com> wrote:


 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra <monarchdodra gmail.com>  
 wrote:

 The pull is kind of stuck in limbo, specifically because of the problems
 associated with implementing reference semantics with structs :/



 Thanks for the enlightening email.

 I am of the considered view that reference semantics with structs in D
 is tough (if not impossible) with default constructor and postblit
 constructor (used when passing objects). This is because you can not
 initialize any object (wrapped in the struct) in the default
 constructor and if you are passing the struct as a function parameter,
 it is not possible to initialize these internal objects (before
 passing them) in the postblit constructor. I faced this in some of my
 code. Do not know if you are facing the same scenario in the DList
 implementation.


It's pretty simple.  Containers are reference types, and it makes most  
sense to pass them by reference.

In order to be sane, containers must be implemented as classes.   
Otherwise, you have behavior like this:

void foo(int[int] aa, int x, int y)
{
    aa[x] = y;
}

void main()
{
    int[int] aa;
    foo(aa, 1, 2);
    aa[3] = 4;
    foo(aa, 5, 6);
    assert(!(1 in aa));
    assert(aa[3] == 4);
    assert(aa[5] == 6);
}

In other words, the only way to turn a default-instantiated struct into a  
valid struct is to use it once.  Not the behavior you want.

Classes don't have this problem, because you must instantiate them to use  
them.

-Steve
Jan 31 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:03 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 07:27:08 -0500, d coder <dlang.coder gmail.com> wrote:


 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra
 <monarchdodra gmail.com> wrote:

 The pull is kind of stuck in limbo, specifically because of the problems
 associated with implementing reference semantics with structs :/



 Thanks for the enlightening email.

 I am of the considered view that reference semantics with structs in D
 is tough (if not impossible) with default constructor and postblit
 constructor (used when passing objects). This is because you can not
 initialize any object (wrapped in the struct) in the default
 constructor and if you are passing the struct as a function parameter,
 it is not possible to initialize these internal objects (before
 passing them) in the postblit constructor. I faced this in some of my
 code. Do not know if you are facing the same scenario in the DList
 implementation.


 It's pretty simple. Containers are reference types, and it makes most
 sense to pass them by reference.

 In order to be sane, containers must be implemented as classes.
 Otherwise, you have behavior like this:

 void foo(int[int] aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 int[int] aa;
 foo(aa, 1, 2);
 aa[3] = 4;
 foo(aa, 5, 6);
 assert(!(1 in aa));
 assert(aa[3] == 4);
 assert(aa[5] == 6);
 }

 In other words, the only way to turn a default-instantiated struct into
 a valid struct is to use it once. Not the behavior you want.

 Classes don't have this problem, because you must instantiate them to
 use them.


As far as I can tell classes have the same problem.


Andrei
Jan 31 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 1/31/13 10:03 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 07:27:08 -0500, d coder <dlang.coder gmail.com>  
 wrote:


 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra
 <monarchdodra gmail.com> wrote:

 The pull is kind of stuck in limbo, specifically because of the  
 problems
 associated with implementing reference semantics with structs :/



 Thanks for the enlightening email.

 I am of the considered view that reference semantics with structs in D
 is tough (if not impossible) with default constructor and postblit
 constructor (used when passing objects). This is because you can not
 initialize any object (wrapped in the struct) in the default
 constructor and if you are passing the struct as a function parameter,
 it is not possible to initialize these internal objects (before
 passing them) in the postblit constructor. I faced this in some of my
 code. Do not know if you are facing the same scenario in the DList
 implementation.


 It's pretty simple. Containers are reference types, and it makes most
 sense to pass them by reference.

 In order to be sane, containers must be implemented as classes.
 Otherwise, you have behavior like this:

 void foo(int[int] aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 int[int] aa;
 foo(aa, 1, 2);
 aa[3] = 4;
 foo(aa, 5, 6);
 assert(!(1 in aa));
 assert(aa[3] == 4);
 assert(aa[5] == 6);
 }

 In other words, the only way to turn a default-instantiated struct into
 a valid struct is to use it once. Not the behavior you want.

 Classes don't have this problem, because you must instantiate them to
 use them.


 As far as I can tell classes have the same problem.


Nope.

void foo(someclass aa, int x, int y)
{
    aa[x] = y;
}

void main()
{
    someclass aa;
    foo(aa, 1, 2); // segfault
    ...
}

-Steve
Jan 31 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


We could easily arrange things to segfault just the same with a 
struct-based implementation.

Andrei
Jan 31 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 15:21:02 UTC, Andrei Alexandrescu 
wrote:

 We could easily arrange things to segfault just the same with a 
 struct-based implementation.

 Andrei


Not if you have a "lazy initialization scheme", which is the 
current scheme we are using...

...and will probably continue to use, since we have no standard 
way of forcing initialization :(
Jan 31 2013
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 10:21:04 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a  
 struct-based implementation.


So you want to make a struct that acts just like a class?  I'm not seeing  
the point.

-Steve
Jan 31 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:29 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:21:04 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a
 struct-based implementation.


 So you want to make a struct that acts just like a class? I'm not seeing
 the point.


It has a destructor.

Andrei
Jan 31 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 1/31/13 10:29 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:21:04 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a
 struct-based implementation.


 So you want to make a struct that acts just like a class? I'm not seeing
 the point.


 It has a destructor.


One which is not called if allocated on the heap.

It's possible to make a class reference that is destroyed when going out  
of scope.  Then you have the option, heap destroyed or stack destroyed.

Not possible with structs (at least for now).

-Steve
Jan 31 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 11:00 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 It has a destructor.


 One which is not called if allocated on the heap.


That is correct. My point was that with structs we get to implement 
full-fledged reference counting for containers.

Reference counting makes a lot of sense for containers. They inherently 
own their internals, don't have cycles, and are massive enough to make 
pass by value effectively an anti-pattern (as it is for C++). At the 
same time memory reclamation is of high interest. Reference counting is 
really the sweet spot for containers.


 It's possible to make a class reference that is destroyed when going out
 of scope. Then you have the option, heap destroyed or stack destroyed.

 Not possible with structs (at least for now).


I don't understand this. Is it about the deprecated meaning of scope?


Andrei
Jan 31 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 11:17:14 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 It has a destructor.


 One which is not called if allocated on the heap.


 That is correct. My point was that with structs we get to implement  
 full-fledged reference counting for containers.


But that won't work if the struct is on the heap.  You will leak the  
memory.

Not only that, but if the container uses the GC to allocate nodes, calling  
the destructor during the GC cycle would be a bad thing.


 Reference counting makes a lot of sense for containers. They inherently  
 own their internals, don't have cycles, and are massive enough to make  
 pass by value effectively an anti-pattern (as it is for C++). At the  
 same time memory reclamation is of high interest. Reference counting is  
 really the sweet spot for containers.


Reference counting makes a lot of sense for any reference type.  Including  
classes.

I think your point is that we can do reference counting with structs, and  
we can't with classes.  I think:

a) you can do reference counting with both (at the moment, this requires a  
wrapper struct for the class)
b) structs that depend on destruction to free their memory do not work  
with the current runtime implementation.


 It's possible to make a class reference that is destroyed when going out
 of scope. Then you have the option, heap destroyed or stack destroyed.

 Not possible with structs (at least for now).


 I don't understand this. Is it about the deprecated meaning of scope?


It means, classes have a mechanism for destruction via the heap (GC) or  
via the stack (scope, or whatever currently replaces scope (Scoped? I  
can't remember) ).

Structs by default are easily destroyed from the stack, but lack the  
fundamental cogs to get GC-based destruction.

In other words, you can't put a reference-counted struct on the heap, it  
won't work correctly (the destructor won't get called).  This is not to  
say this can't be fixed, but if we are talking about current  
implementation limitations, classes are superior in how they can be used.

-Steve
Jan 31 2013
parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 31 January 2013 at 18:59:20 UTC, Steven 
Schveighoffer wrote:

 Structs by default are easily destroyed from the stack, but 
 lack the fundamental cogs to get GC-based destruction.

 -Steve


Perhaps _d_newitemT() and buddies can check whether it creates 
structs, store a collection of pointers to structs it allocates 
and at the end of program some druntime routine loops through out 
of the collection to run dtors?
Jan 31 2013
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 14:22:24 -0500, Maxim Fomin <maxim maxim-fomin.ru>  
wrote:


 On Thursday, 31 January 2013 at 18:59:20 UTC, Steven Schveighoffer wrote:

 Structs by default are easily destroyed from the stack, but lack the  
 fundamental cogs to get GC-based destruction.

 -Steve


 Perhaps _d_newitemT() and buddies can check whether it creates structs,  
 store a collection of pointers to structs it allocates and at the end of  
 program some druntime routine loops through out of the collection to run  
 dtors?


What is needed is a precise GC, where the struct typeinfo and destructor  
are stored along with the block.

However, we ALSO need two functions, a destructor (Called from the stack  
deterministically) and a finalizer (called from the GC).

Like I said, this isn't an unsolvable problem, but neither is making  
classes reference counted.

-Steve
Jan 31 2013
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 17:17, Andrei Alexandrescu wrote:

 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 It has a destructor.


 One which is not called if allocated on the heap.


 That is correct. My point was that with structs we get to implement
 full-fledged reference counting for containers.

 Reference counting makes a lot of sense for containers. They inherently
 own their internals, don't have cycles, and are massive enough to make
 pass by value effectively an anti-pattern (as it is for C++). At the
 same time memory reclamation is of high interest. Reference counting is
 really the sweet spot for containers.


I can understand interest in early and deterministic reclamation of 
memory, but I have some issues with reference counting. Maybe you can 
shed some light on these:

- how do you reference count immutable containers? You'll have to cast 
the payload to mutable and assume it is not in read-only memory.

- how do you do reference counting when accessing shared containers in 
multiple threads? Consider clearing the last reference to a shared 
reference counted object while another thread reads this reference. With 
usual atomic operations on the reference count only:

   1. the reading thread reads the payload pointer
   2. the writing thread reads the payload pointer, decrements the 
reference count and frees the array
   3. the reading thread increments the reference count, but accesses 
the deallocated array.

With atomic operations, I can only imagine fixing this with CAS2 
operations reading/changing both pointer and reference count at the same 
time, but even then, it is not obvious. Anyway, this operation is not 
supported by the currently popular processors.

In contrast, GC managed memory is memory- safe in multi-threading 
environments by design.
Jan 31 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:


 On 31.01.2013 17:17, Andrei Alexandrescu wrote:

 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 It has a destructor.


 One which is not called if allocated on the heap.


 That is correct. My point was that with structs we get to implement
 full-fledged reference counting for containers.

 Reference counting makes a lot of sense for containers. They inherently
 own their internals, don't have cycles, and are massive enough to make
 pass by value effectively an anti-pattern (as it is for C++). At the
 same time memory reclamation is of high interest. Reference counting is
 really the sweet spot for containers.


 I can understand interest in early and deterministic reclamation of  
 memory, but I have some issues with reference counting. Maybe you can  
 shed some light on these:

 - how do you reference count immutable containers? You'll have to cast  
 the payload to mutable and assume it is not in read-only memory.


The reference count part is not immutable.

e.g.:

struct refcountstruct
{
    int count;
    immutable(containerimpl) impl;
}


 - how do you do reference counting when accessing shared containers in  
 multiple threads? Consider clearing the last reference to a shared  
 reference counted object while another thread reads this reference. With  
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the  
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses  
 the deallocated array.


I would be concerned if a thread can get ahold of a reference counted  
pointer without having the counter incremented already on its behalf.

-Steve
Jan 31 2013
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 21:48, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:


 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.


 The reference count part is not immutable.

 e.g.:

 struct refcountstruct
 {
     int count;
     immutable(containerimpl) impl;
 }


What about immutable(refcountstruct)? Maybe only implicitely as field of 
an immutable class.


 - how do you do reference counting when accessing shared containers in
 multiple threads? Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.


 I would be concerned if a thread can get ahold of a reference counted
 pointer without having the counter incremented already on its behalf.


You have to read the pointer to increment the counter to get hold of the 
reference.
Jan 31 2013
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 21:55, Rainer Schuetze wrote:

 On 31.01.2013 21:48, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:


 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.


 The reference count part is not immutable.

 e.g.:

 struct refcountstruct
 {
     int count;
     immutable(containerimpl) impl;
 }


 What about immutable(refcountstruct)? Maybe only implicitely as field of
 an immutable class.


That would probably be better immutable(refcountstruct*) as field of the 
immutable container struct.
Jan 31 2013
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 15:55:11 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:


 On 31.01.2013 21:48, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:


 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.


 The reference count part is not immutable.

 e.g.:

 struct refcountstruct
 {
     int count;
     immutable(containerimpl) impl;
 }


 What about immutable(refcountstruct)? Maybe only implicitely as field of  
 an immutable class.


You wouldn't be able to do that.  The reference count needs to be  
tail-immutable, meaning the reference count is not immutable, but the data  
it protects is.

Anticipating your answer, no, there isn't a good way to do this at the  
moment, it's something D definitely lacks.


 - how do you do reference counting when accessing shared containers in
 multiple threads? Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.


 I would be concerned if a thread can get ahold of a reference counted
 pointer without having the counter incremented already on its behalf.


 You have to read the pointer to increment the counter to get hold of the  
 reference.


Right, but where do you get the original pointer from?  Wouldn't that  
reference have incremented the count?

-Steve
Jan 31 2013
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 22:37, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 15:55:11 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:

 - how do you do reference counting when accessing shared containers in
 multiple threads? Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.


 I would be concerned if a thread can get ahold of a reference counted
 pointer without having the counter incremented already on its behalf.


 You have to read the pointer to increment the counter to get hold of
 the reference.


 Right, but where do you get the original pointer from?  Wouldn't that
 reference have incremented the count?


Any reference that is accessible from multiple threads, e.g. a global 
shared variable.

shared(Container) allObjects;

Without any thread having a reference to it, the payload's reference 
count is 1. The reading thread above increments the counter when making 
a local copy, but the writing thread decrements the counter when 
assigning Container.init. If both run concurrently, with the sequence 1 
to 3 above, trouble is inevitable.
Jan 31 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:


 Any reference that is accessible from multiple threads, e.g. a global  
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference  
 count is 1. The reading thread above increments the counter when making  
 a local copy, but the writing thread decrements the counter when  
 assigning Container.init. If both run concurrently, with the sequence 1  
 to 3 above, trouble is inevitable.


Right, but isn't that an issue with having ANY shared variable that isn't  
protected by a lock?

I was thinking you had passed the pointer in via a message or something.

-Steve
Jan 31 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 17:37:39 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze <r.sagitario gmx.de>  
 wrote:


 Any reference that is accessible from multiple threads, e.g. a global  
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference  
 count is 1. The reading thread above increments the counter when making  
 a local copy, but the writing thread decrements the counter when  
 assigning Container.init. If both run concurrently, with the sequence 1  
 to 3 above, trouble is inevitable.


 Right, but isn't that an issue with having ANY shared variable that  
 isn't protected by a lock?

 I was thinking you had passed the pointer in via a message or something.


BTW, couldn't we solve this with a function that increments the retain  
count, and then returns a pointer to the object?  Such a method would have  
to be atomic on shared instances.

-Steve
Jan 31 2013
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 00:14, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 17:37:39 -0500, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:


 On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze
 <r.sagitario gmx.de> wrote:


 Any reference that is accessible from multiple threads, e.g. a global
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference
 count is 1. The reading thread above increments the counter when
 making a local copy, but the writing thread decrements the counter
 when assigning Container.init. If both run concurrently, with the
 sequence 1 to 3 above, trouble is inevitable.


 Right, but isn't that an issue with having ANY shared variable that
 isn't protected by a lock?

 I was thinking you had passed the pointer in via a message or something.


 BTW, couldn't we solve this with a function that increments the retain
 count, and then returns a pointer to the object?  Such a method would
 have to be atomic on shared instances.


The problem is to make it atomic without expensive locks. Lacking the 
CAS2 operation (that does a CAS on two arbitrary memory locations 
simultaneously), my first thought was that it is not possible.

Considering it again, I guess it can be done by not using atomic 
increment/decrement, but splitting up the operations. The main idea is 
to no longer touch the data after the reference count has become 0 once:

// CAS returns *var before operation, only modifies if *var == expected
size_t CAS(size_t* var, size_t expected, size_t newval);

struct PayLoad(T)
{
	T* array;
	size_t size;
	size_t refCount;

	PayLoad!T* ref()
	{
		size_t cnt = refCount;
		while(cnt)
		{
			int old = CAS(&refCount, cnt, cnt+1);
			if(old == cnt)
				return this;
			cnt = refCount;
		}
		// in the rare case the payload has become invalid,
		// return a new empty object to avoid having to
		// check for null (could also be null if the container
		// always checks its payload pointer)
		return new PayLoad!T;
	}
	void deref()
	{
		size_t cnt, old;
		do
		{
			assert(refCount);
			cnt = refCount;
			old = CAS(&refCount, cnt, cnt-1);
		}
		while(old != cnt);
		if(refCount == 0)
			free(array);		
	}
}

struct Container(T)
{
	PayLoad!T* payload;

	this()	// I know this doesn't work, but would be convenient
	{
		payload = new Payload!T;
	}
	this(this)
	{
		payload = payload.ref();
	}
	~this()
	{
		payload.deref();
	}
         void opAssign(typeof(this) rhs)
	{
		if(rhs.payload !is payload)
		{
			payload.deref();
			payload = rhs.payload.ref();
		}
	}
}

PayLoad!T will still have to be garbage collected, though.

BTW: Maybe I am misunderstanding it, but according to 
http://dlang.org/operatoroverloading.html overloading opAssign on the 
same type as in "void opAssign(typeof(this) rhs)" is not allowed. Is 
this correct? You find it in phobos as well.

Another BTW: the links to anchors on the same page are broken as they 
don't include the html file.
Feb 01 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 01 Feb 2013 14:03:51 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:

 The problem is to make it atomic without expensive locks. Lacking the  
 CAS2 operation (that does a CAS on two arbitrary memory locations  
 simultaneously), my first thought was that it is not possible.


I don't think expensive locks are an issue here.  Having an expensive lock  
to copy a pointer from a shared location into a thread-local location is  
worth having an expensive lock, and may even be necessary.

Once you have the thread-local copy of the reference, incrementing and  
decrementing the reference count can be done via CAS.

-Steve
Feb 01 2013
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 21:02, Steven Schveighoffer wrote:

 On Fri, 01 Feb 2013 14:03:51 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:

 The problem is to make it atomic without expensive locks. Lacking the
 CAS2 operation (that does a CAS on two arbitrary memory locations
 simultaneously), my first thought was that it is not possible.


 I don't think expensive locks are an issue here.  Having an expensive
 lock to copy a pointer from a shared location into a thread-local
 location is worth having an expensive lock, and may even be necessary.


It depends on the application. At work we put a lot of effort into 
transferring data lock-free from/to a low latency thread processing 
audio. No blocking is allowed in this thread. (You cannot guarantee this 
with D at the moment, but IMO it only takes a few tweaks to the runtime.)


 Once you have the thread-local copy of the reference, incrementing and
 decrementing the reference count can be done via CAS.


Yes. My proposed code is only slightly more complex than atomic 
increments/decrements, but should also work for shared(Container).
Feb 01 2013
prev sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 23:37, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:


 Any reference that is accessible from multiple threads, e.g. a global
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference
 count is 1. The reading thread above increments the counter when
 making a local copy, but the writing thread decrements the counter
 when assigning Container.init. If both run concurrently, with the
 sequence 1 to 3 above, trouble is inevitable.


 Right, but isn't that an issue with having ANY shared variable that
 isn't protected by a lock?


True, as soon as you start mutating the variable. In case of simple 
reference/pointer without reference count, the change is atomic though 
(depending on the CPU or the implementation of "shared") and the reader 
thread just gets the old or the new pointer, but never an invalid pointer.

My main point actually was that a lot of people seem to think that 
reference counting with atomic incrementing/decrementing the counter 
(like it is used in COM) is thread safe in general. I think they are wrong.


 I was thinking you had passed the pointer in via a message or something.


I agree, message passing should not have this issue. It needs that 
single reference being accessed from different threads.
Jan 31 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
01-Feb-2013 00:32, Rainer Schuetze пишет:

 On 31.01.2013 17:17, Andrei Alexandrescu wrote:

 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 It has a destructor.


 One which is not called if allocated on the heap.


 That is correct. My point was that with structs we get to implement
 full-fledged reference counting for containers.

 Reference counting makes a lot of sense for containers. They inherently
 own their internals, don't have cycles, and are massive enough to make
 pass by value effectively an anti-pattern (as it is for C++). At the
 same time memory reclamation is of high interest. Reference counting is
 really the sweet spot for containers.


 I can understand interest in early and deterministic reclamation of
 memory, but I have some issues with reference counting. Maybe you can
 shed some light on these:

 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.


Containers of immutables can have ref-count easily.
And I'd say fully immutable containers are rare case and can rely on GC.


 - how do you do reference counting when accessing shared containers in
 multiple threads?


Atomic Inc/Dec.


 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference. With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.


Can't happen as reading thread got to have a reference and so does the 
writing thread, then refcount >= 2.


 With atomic operations, I can only imagine fixing this with CAS2
 operations reading/changing both pointer and reference count at the same
 time, but even then, it is not obvious. Anyway, this operation is not
 supported by the currently popular processors.

 In contrast, GC managed memory is memory- safe in multi-threading
 environments by design.



-- 
Dmitry Olshansky
Feb 01 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
Whilst we're talking about containers and refcounted, I've been 
encountering on and off a very strange bug with Array and 
RefCounted.

It is creating stack overflows, and I'm *certain* is creating 
problems when trying to nest connectors. I also think it is the 
"tip" of a more serious bug.

Could anybody with the required skills try to investigate this 
one?

http://d.puremagic.com/issues/show_bug.cgi?id=9438
Feb 01 2013
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 09:06, Dmitry Olshansky wrote:

 01-Feb-2013 00:32, Rainer Schuetze пишет:

 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.


 Containers of immutables can have ref-count easily.
 And I'd say fully immutable containers are rare case and can rely on GC.


Do you want different implementations of the containers depending on the 
mutable/immutable/shared modifier? Would that be possible?


 - how do you do reference counting when accessing shared containers in
 multiple threads?


 Atomic Inc/Dec.


Not good enough as shown in the discussion with Steven.


 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference. With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.


 Can't happen as reading thread got to have a reference and so does the
 writing thread, then refcount >= 2.


Can happen with a single shared reference accessed from two threads.
Feb 01 2013
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 1 February 2013 at 19:10:26 UTC, Rainer Schuetze wrote:

 On 01.02.2013 09:06, Dmitry Olshansky wrote:

 01-Feb-2013 00:32, Rainer Schuetze пишет:

 - how do you reference count immutable containers? You'll 
 have to cast
 the payload to mutable and assume it is not in read-only 
 memory.


 Containers of immutables can have ref-count easily.
 And I'd say fully immutable containers are rare case and can 
 rely on GC.


 Do you want different implementations of the containers 
 depending on the mutable/immutable/shared modifier? Would that 
 be possible?


You can only do that on the qualifier of the *parameters*, not 
the container itself.

This is not possible because an S "is a" const(S). If the 
implementation of an S was different from a const(S), then you'd 
violate that. At best, a const(S) is an S with restricted 
possibilities, bot not *different* possibilities.
Feb 01 2013
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 20:18, monarch_dodra wrote:

 On Friday, 1 February 2013 at 19:10:26 UTC, Rainer Schuetze wrote:

 On 01.02.2013 09:06, Dmitry Olshansky wrote:

 01-Feb-2013 00:32, Rainer Schuetze пишет:

 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.


 Containers of immutables can have ref-count easily.
 And I'd say fully immutable containers are rare case and can rely on GC.


 Do you want different implementations of the containers depending on
 the mutable/immutable/shared modifier? Would that be possible?


 You can only do that on the qualifier of the *parameters*, not the
 container itself.


Ok, but that won't help changing reference count semantics, because the 
type of the elements are irrelevant here. It's about the container itself.


 This is not possible because an S "is a" const(S). If the implementation
 of an S was different from a const(S), then you'd violate that. At best,
 a const(S) is an S with restricted possibilities, bot not *different*
 possibilities.


I guess you would have to convert it to a different type, but it can be 
an efficient operation transferring the payload unmodified. Also, when 
converting to shared, you might even be able to verify that there are no 
other references left (that must be non-sharing).
Feb 01 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
01-Feb-2013 23:10, Rainer Schuetze пишет:

 On 01.02.2013 09:06, Dmitry Olshansky wrote:

 01-Feb-2013 00:32, Rainer Schuetze пишет:

 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.


 Containers of immutables can have ref-count easily.
 And I'd say fully immutable containers are rare case and can rely on GC.


 Do you want different implementations of the containers depending on the
 mutable/immutable/shared modifier? Would that be possible?


In fact I do ;) It's possible base on element type not the whole 
container itself.


 - how do you do reference counting when accessing shared containers in
 multiple threads?


 Atomic Inc/Dec.


 Not good enough as shown in the discussion with Steven.


 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference. With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.


 Can't happen as reading thread got to have a reference and so does the
 writing thread, then refcount >= 2.


 Can happen with a single shared reference accessed from two threads.


You mean shared reference to shared ref-counter object?
Awful and BTW impossible in D as you'd have to cast your way into it :)


-- 
Dmitry Olshansky
Feb 01 2013
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 20:36, Dmitry Olshansky wrote:

 01-Feb-2013 23:10, Rainer Schuetze пишет:

 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.


 Can't happen as reading thread got to have a reference and so does the
 writing thread, then refcount >= 2.


 Can happen with a single shared reference accessed from two threads.


 You mean shared reference to shared ref-counter object?
 Awful and BTW impossible in D as you'd have to cast your way into it :)


Maybe I'm coding too much C++, but isn't this what the "shared" modifier 
is all about?
Feb 01 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 19:21, Andrei Alexandrescu пишет:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a
 struct-based implementation.


Structs are quite borked in this regard e.g. without extra efforts the 
following:

somclass aa = someclass();
foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init

The current workaround I find the most sensible is:
-  disable this();
- make all constructors private
- define opCall and forward it to private constructors. 0-arg versions 
have to pass dummy and/or default values to get struct constructed

- automate this boilerplate until something in language is fixed? :)

The advantage is that the following is illegal:
someclass aa;

and the following works as expected:
auto aa = someclass(); //create empty container damn it!

-- 
Dmitry Olshansky
Jan 31 2013
parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky 
wrote:

 31-Jan-2013 19:21, Andrei Alexandrescu пишет:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a
 struct-based implementation.


 Structs are quite borked in this regard e.g. without extra 
 efforts the following:

 somclass aa = someclass();
 foor(aa, 1, 2); // segfault, surprize someclass() is 
 someclass.init

 The current workaround I find the most sensible is:
 -  disable this();
 - make all constructors private
 - define opCall and forward it to private constructors. 0-arg 
 versions have to pass dummy and/or default values to get struct 
 constructed

 - automate this boilerplate until something in language is 
 fixed? :)

 The advantage is that the following is illegal:
 someclass aa;

 and the following works as expected:
 auto aa = someclass(); //create empty container damn it!


If desired default struct value is constant for all instances of 
that struct, there is another workaround by changing init values:
-------------------
import std.stdio;

class A { int x = 5; }

struct S { int x = 5; A a; }

void main()
{
     S s; // defaults to preallocated a, a is not null
     assert(S.x is 100 && s.a !is null && s.a.x is 100);
}

import runtime;

mixin(declareExternInitZPointer!S);

static this()
{
	A a = new A;
	a.x = 100;
	S s =  S(100, a);
	rtSetDefaultHeapInitializer(s);
	rtSetDefaultStackInitializer(s, mixin(passExternInitZPointer!S));
}
----------------------

Where runtime module is http://dpaste.dzfl.pl/40b59a5d

This does not fully replace default ctor, since all instances of 
S are affected.
Jan 31 2013
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 23:32, Maxim Fomin пишет:

 On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky wrote:

 31-Jan-2013 19:21, Andrei Alexandrescu пишет:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a
 struct-based implementation.


 Structs are quite borked in this regard e.g. without extra efforts the
 following:

 somclass aa = someclass();
 foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init

 The current workaround I find the most sensible is:
 -  disable this();
 - make all constructors private
 - define opCall and forward it to private constructors. 0-arg versions
 have to pass dummy and/or default values to get struct constructed

 - automate this boilerplate until something in language is fixed? :)

 The advantage is that the following is illegal:
 someclass aa;

 and the following works as expected:
 auto aa = someclass(); //create empty container damn it!


 If desired default struct value is constant for all instances of that
 struct, there is another workaround by changing init values:
 -------------------
 import std.stdio;

 class A { int x = 5; }

 struct S { int x = 5; A a; }

 void main()
 {
      S s; // defaults to preallocated a, a is not null
      assert(S.x is 100 && s.a !is null && s.a.x is 100);
 }


This is fine except that it doesn't solve the problem of:
Container a = Container(); //should be empty container, not illegal 
null-container

Basically  dsiable this() is optional, but I like to help user avoid bugs.


 import runtime;

 mixin(declareExternInitZPointer!S);

 static this()
 {
      A a = new A;
      a.x = 100;
      S s =  S(100, a);
      rtSetDefaultHeapInitializer(s);
      rtSetDefaultStackInitializer(s, mixin(passExternInitZPointer!S));
 }
 ----------------------

 Where runtime module is http://dpaste.dzfl.pl/40b59a5d

 This does not fully replace default ctor, since all instances of S are
 affected.



-- 
Dmitry Olshansky
Jan 31 2013
next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 31 January 2013 at 19:34:29 UTC, Dmitry Olshansky 
wrote:

 This is fine except that it doesn't solve the problem of:
 Container a = Container(); //should be empty container, not 
 illegal null-container

 Basically  dsiable this() is optional, but I like to help user 
 avoid bugs.


You have static opCall for such things. Since when this was a 
problem?
Jan 31 2013
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 23:51, Maxim Fomin пишет:

 On Thursday, 31 January 2013 at 19:34:29 UTC, Dmitry Olshansky wrote:

 This is fine except that it doesn't solve the problem of:
 Container a = Container(); //should be empty container, not illegal
 null-container

 Basically  dsiable this() is optional, but I like to help user avoid
 bugs.


 You have static opCall for such things. Since when this was a problem?


Yes, that's why I use it still...

it can't initialize a const struct *and* requires a T.init or T a = 
void; or forward to this(int dummy=42) container.


-- 
Dmitry Olshansky
Jan 31 2013
prev sibling parent reply Robert burner Schadek <realburner gmx.de> writes:
On 01/31/2013 08:34 PM, Dmitry Olshansky wrote:

 31-Jan-2013 23:32, Maxim Fomin пишет:

 On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky wrote:

 31-Jan-2013 19:21, Andrei Alexandrescu пишет:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a
 struct-based implementation.


 Structs are quite borked in this regard e.g. without extra efforts the
 following:

 somclass aa = someclass();
 foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init

 The current workaround I find the most sensible is:
 -  disable this();
 - make all constructors private
 - define opCall and forward it to private constructors. 0-arg versions
 have to pass dummy and/or default values to get struct constructed

 - automate this boilerplate until something in language is fixed? :)

 The advantage is that the following is illegal:
 someclass aa;

 and the following works as expected:
 auto aa = someclass(); //create empty container damn it!


 If desired default struct value is constant for all instances of that
 struct, there is another workaround by changing init values:
 -------------------
 import std.stdio;

 class A { int x = 5; }

 struct S { int x = 5; A a; }

 void main()
 {
      S s; // defaults to preallocated a, a is not null
      assert(S.x is 100 && s.a !is null && s.a.x is 100);
 }


 This is fine except that it doesn't solve the problem of:
 Container a = Container(); //should be empty container, not illegal 
 null-container


Just use a payload struct and check whether that is null before every 
operation.

 Basically  dsiable this() is optional, but I like to help user avoid 
 bugs.


 import runtime;

 mixin(declareExternInitZPointer!S);

 static this()
 {
      A a = new A;
      a.x = 100;
      S s =  S(100, a);
      rtSetDefaultHeapInitializer(s);
      rtSetDefaultStackInitializer(s, mixin(passExternInitZPointer!S));
 }
 ----------------------

 Where runtime module is http://dpaste.dzfl.pl/40b59a5d

 This does not fully replace default ctor, since all instances of S are
 affected.
Jan 31 2013
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 23:54, Robert burner Schadek пишет:

 On 01/31/2013 08:34 PM, Dmitry Olshansky wrote:

 31-Jan-2013 23:32, Maxim Fomin пишет:

 On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky wrote:

 31-Jan-2013 19:21, Andrei Alexandrescu пишет:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:

 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu

 As far as I can tell classes have the same problem.


 Nope.

 void foo(someclass aa, int x, int y)
 {
 aa[x] = y;
 }

 void main()
 {
 someclass aa;
 foo(aa, 1, 2); // segfault
 ...
 }


 We could easily arrange things to segfault just the same with a
 struct-based implementation.


 Structs are quite borked in this regard e.g. without extra efforts the
 following:

 somclass aa = someclass();
 foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init

 The current workaround I find the most sensible is:
 -  disable this();
 - make all constructors private
 - define opCall and forward it to private constructors. 0-arg versions
 have to pass dummy and/or default values to get struct constructed

 - automate this boilerplate until something in language is fixed? :)

 The advantage is that the following is illegal:
 someclass aa;

 and the following works as expected:
 auto aa = someclass(); //create empty container damn it!


 If desired default struct value is constant for all instances of that
 struct, there is another workaround by changing init values:
 -------------------
 import std.stdio;

 class A { int x = 5; }

 struct S { int x = 5; A a; }

 void main()
 {
      S s; // defaults to preallocated a, a is not null
      assert(S.x is 100 && s.a !is null && s.a.x is 100);
 }


 This is fine except that it doesn't solve the problem of:
 Container a = Container(); //should be empty container, not illegal
 null-container


 Just use a payload struct and check whether that is null before every
 operation.


Lazy evaluation, great. Except that is a useless opcode per every 
operation with container + keep in in mind the problem:

void func(cont a)
{
	//if a was empty and we go with lazy-initialization
	// then the outside cont is not updated
	a.insert(...);
}

void main()
{
	...
	cont x;
	...
	//this could be non-trivial code that
	//may or may not touch x in all path
	func(x);
	...
}


-- 
Dmitry Olshansky
Jan 31 2013
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 31 January 2013 at 19:55:01 UTC, Robert burner 
Schadek wrote:

 Just use a payload struct and check whether that is null before 
 every operation.


The following problem is inherent to this approach:

---
struct Array(T) {
   static struct Payload {
     T* data;
     size_t length;
   }
   Payload* p;
   // ...
}

void fill(T)(Array!T a) {
   a.length = 3;
   a[0] = 1;
   a[1] = 2;
   a[2] = 3;
}

void client() {
   Array!int a;
   fill(a);
   // Oops, a is still empty (with null payload).
}
---

Such containers are "almost, but not quite" reference types; 
reason enough to  disable this() in order to improve the user 
experience, in my opinion.

David
Jan 31 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 2:54 PM, Robert burner Schadek wrote:
[snip]

Please avoid overquoting - this is the worst there is: one line inserted 
without whitespace in the middle of a long quote.

Thanks,

Andrei
Jan 31 2013
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei Alexandrescu 
wrote:

 As far as I can tell classes have the same problem.


 Andrei


Regarding classes, would there be any chance of being to create a 
class instance that isn't a child of "object"?

I'm not entirely sure what the implications are, but it seems 
that is the major "no-sell" argument against using classes.

I know *I* would be using them a whole lot more if this was the 
case.
Jan 31 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:27 AM, monarch_dodra wrote:

 On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei Alexandrescu wrote:

 As far as I can tell classes have the same problem.


 Andrei


 Regarding classes, would there be any chance of being to create a class
 instance that isn't a child of "object"?


Using classes entails buying into an entire object model with its own 
pluses and minuses. Adding classes that don't inherit Object would wreck 
havoc all over the place.


 I'm not entirely sure what the implications are, but it seems that is
 the major "no-sell" argument against using classes.

 I know *I* would be using them a whole lot more if this was the case.


What would be the problem?


Andrei
Jan 31 2013
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, January 31, 2013 10:39:46 Andrei Alexandrescu wrote:

 On 1/31/13 10:27 AM, monarch_dodra wrote:

 On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei Alexandrescu wrote:

 As far as I can tell classes have the same problem.
 
 
 Andrei


 
 Regarding classes, would there be any chance of being to create a class
 instance that isn't a child of "object"?


 
 Using classes entails buying into an entire object model with its own
 pluses and minuses. Adding classes that don't inherit Object would wreck
 havoc all over the place.


And what would be the gain, anyway?

I don't understand what the downside is to having a single base object besides 
the issues with toString, toHash, opEquals, and opCmp. And we decided to 
remove all of them from Object (though no actual progress has been made beyond 
the decision), and with those gone, Object will have next to nothing on it 
anyway. But not having Object be the base of all classes would definitely cause 
quite a few problems.

- Jonathan M Davis
Jan 31 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 16:17:09 UTC, Jonathan M Davis 
wrote:

 On Thursday, January 31, 2013 10:39:46 Andrei Alexandrescu 
 wrote:

 On 1/31/13 10:27 AM, monarch_dodra wrote:

 On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei 
 Alexandrescu wrote:

 As far as I can tell classes have the same problem.
 
 
 Andrei


 
 Regarding classes, would there be any chance of being to 
 create a class
 instance that isn't a child of "object"?


 
 Using classes entails buying into an entire object model with 
 its own
 pluses and minuses. Adding classes that don't inherit Object 
 would wreck
 havoc all over the place.


 And what would be the gain, anyway?

 I don't understand what the downside is to having a single base 
 object besides
 the issues with toString, toHash, opEquals, and opCmp. And we 
 decided to
 remove all of them from Object (though no actual progress has 
 been made beyond
 the decision), and with those gone, Object will have next to 
 nothing on it
 anyway. But not having Object be the base of all classes would 
 definitely cause
 quite a few problems.

 - Jonathan M Davis


I guess it still has the vtable and monitor.

In any case, the question was mostly asked to "test the waters" 
in regards to "class usage best practice", and to probe the 
consequences of such a change.

Please disregard the question in further discussion.
Jan 31 2013