www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - funky property error with assoc array

reply "Dan" <dbdavidson yahoo.com> writes:
This fails to compile when accessing as m.pgoo() complaining 
about postblit.
What is wrong with this?

Note: If I alias as array instead of map: alias const(X)[] Map;
it compiles fine.

Thanks
Dan
-----------------------------
struct X { this(this) {} }
alias const(X)[string] Map;
 property int pgoo(ref Map x) { return 3; }

void main() {
   Map m;
   pgoo(m);
   m.pgoo();
}
Nov 14 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 14, 2012 20:48:57 Dan wrote:

 This fails to compile when accessing as m.pgoo() complaining
 about postblit.
 What is wrong with this?
 
 Note: If I alias as array instead of map: alias const(X)[] Map;
 it compiles fine.
 
 Thanks
 Dan
 -----------------------------
 struct X { this(this) {} }
 alias const(X)[string] Map;
  property int pgoo(ref Map x) { return 3; }
 
 void main() {
 Map m;
 pgoo(m);
 m.pgoo();
 }


Postblit doesn't work with const objects and there's not currently any way to 
make it work with const objects. So, any attempt to copy any element in your 
AA won't work. So, if the AA implementation has any code in it which requires 
copying an element (and it probably does), then a type with a postblit in it 
won't work if it's const.

At present const and postblit don't get allow at all, and it sounds like the 
solution that Andrei and Walter have been cooking up will probably involve 
adding copy constructors, but nothing has actually happened yet. postblit 
fundamentally doesn't work with const or immutable though and can't, because 
you can't modify const or immutable objects, and postblit requires you to 
alter the object. So, as far as const and immutable go, postblit is 
fundamentally flawed.

- Jonathan M Davis
Nov 14 2012
parent reply "Dan" <dbdavidson yahoo.com> writes:
On Wednesday, 14 November 2012 at 22:07:10 UTC, Jonathan M Davis 
wrote:

 Postblit doesn't work with const objects and there's not 
 currently any way to
 make it work with const objects. So, any attempt to copy any 
 element in your
 AA won't work. So, if the AA implementation has any code in it 
 which requires
 copying an element (and it probably does), then a type with a 
 postblit in it
 won't work if it's const.


Jonathan, thanks for taking the time on this and all my other 
questions. Coming from C++ and trying to hammer out how to use D 
I've hit this postblit many times and many ways. I agree that 
postblit does not work with const - and my feeling is maybe it 
should not and I hope there is a work around (for structs). For 
instance:
------------
import opmix.mix;
import std.stdio;
struct Big {
   char [] c ;
}
void main () {
   const ( Big ) s = { " test " };
   Big other = s.gdup;
   writeln(other);
}
------------


 At present const and postblit don't get allow at all, and it 
 sounds like the
 solution that Andrei and Walter have been cooking up will 
 probably involve


I think that is fine they don't get along. postblit can not 
guarantee no data is shared after the call, so they should not. 
In the example here I don't access the map at all and the call 
works fine in non-property form.


 adding copy constructors, but nothing has actually happened yet.


I do not see how copy constructors can help. Receiving const(T) 
or const(T) ref to anything and hoping to copy it is the 
challenge and doing it from a copy constructor is no easier than 
a postblit. I think bearophile pointed it out best:

    In practice to copy something const to something that's
    not const you need a deep copy function, because inside
    the array there can be other arrays that are const,
    etc. Transitive const requires transitive copy, it's
    easy :-)

If you have any extra cycles I would love feedback on (dup and 
Global Dup) in:
https://github.com/patefacio/d-help/blob/master/doc/canonical.pdf

Actually feedback on all would be great because it establishes 
how I want to work with structs and being new to D I'd like 
assurances it is safe/correct.

Thanks
Dan


 postblit
 fundamentally doesn't work with const or immutable though and 
 can't, because
 you can't modify const or immutable objects, and postblit 
 requires you to
 alter the object. So, as far as const and immutable go, 
 postblit is
 fundamentally flawed.

 - Jonathan M Davis
Nov 15 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, November 15, 2012 14:21:41 Dan wrote:

 I do not see how copy constructors can help. Receiving const(T)
 or const(T) ref to anything and hoping to copy it is the
 challenge and doing it from a copy constructor is no easier than
 a postblit.


Doing it from a postblit is impossible. Doing it from a copy constructor _is_ 
possible (assuming that they're added to the language - they don't help you 
any until they are). The core problem with postblit constructor is that it 
does a shallow copy _before_ entering the postblit constructor. This seems 
great at first, because then you only have to copy stuff in the postblit itself 
which actually needs a deep copy. But it fails completely with const and 
immutable, because it's illegal to alter anything which is const or immutable 
- even by casting them to mutable temporarily to make the change (doing so is 
undefined behavior). So, you can't change _anything_ in a const postblit 
constructor, meaning that you can't make a deep copy of anything.

A copy constructor, on the other hand, does not do any copying beforehand. It 
all must still be constructed. The copy constructor then constructs the new 
object with deep copies of the data. This works with const and immutable, 
because it's construction rather than mutation. The postblit constructor fails 
because it requires you to use mutation, which doesn't work with const.

So, as nice an idea as it is, the postblit constructor is a failure. And as 
such, I believe that Walter and Andrei intend to replace it with copy 
constructors in the long term (though the postblit constructor will probably 
stay in the language for compatibility for a very, very long time, if not 
permanently). Unfortunately however, nothing has been done about this on the 
implementation side of things (they've only discussed it, not done anything 
about it), so right now, we don't have a solution, since it's not like you can 
go and declare your own copy constructor and have it work, because it'll never 
be called unless you call it explicitly. Rather, the language would use the 
postblit constructor if it's there and complain about it not working with 
const if you try it, even if you have a copy constructor which would do the 
job.

- Jonathan M Davis
Nov 15 2012
parent reply "Dan" <dbdavidson yahoo.com> writes:
On Thursday, 15 November 2012 at 17:24:44 UTC, Jonathan M Davis 
wrote:

 On Thursday, November 15, 2012 14:21:41 Dan wrote:

 I do not see how copy constructors can help. Receiving const(T)
 or const(T) ref to anything and hoping to copy it is the
 challenge and doing it from a copy constructor is no easier 
 than
 a postblit.


 Doing it from a postblit is impossible.


I understand and this makes sense. postblit is not a deep copy, 
it is only what you choose to make it.


 Doing it from a copy constructor _is_
 possible (assuming that they're added to the language - they 
 don't help you
 any until they are).


I question the need for this - at least for structs composed of 
structs, dynamic arrays and associative arrays. Classes are 
another issue.


 The core problem with postblit constructor is that it
 does a shallow copy _before_ entering the postblit constructor. 
 This seems
 great at first, because then you only have to copy stuff in the 
 postblit itself
 which actually needs a deep copy. But it fails completely with 
 const and
 immutable, because it's illegal to alter anything which is 
 const or immutable
 - even by casting them to mutable temporarily to make the 
 change (doing so is
 undefined behavior). So, you can't change _anything_ in a const 
 postblit
 constructor, meaning that you can't make a deep copy of 
 anything.


Understood. I don't think the problem with postblit is what it 
does _before_ I do my thing. The problem for this case is it can 
not guarantee what I do *in* my postblit and therefore it does 
not and can not know I'm leaving with no sharing. I don't see 
this as a failure, though. You want a copy, then you need to make 
a copy. Find a way other than postblit. The suggestion is use 
copy constructors instead. I don't see how this will help - but I 
have not found the proposal (do you have a link?).  I have read 
DIP10 which was an early attempt to get at this stuff - but I 
think it did not go far.


 A copy constructor, on the other hand, does not do any copying 
 beforehand. It
 all must still be constructed. The copy constructor then 
 constructs the new
 object with deep copies of the data. This works with const and 
 immutable,
 because it's construction rather than mutation. The postblit 
 constructor fails
 because it requires you to use mutation, which doesn't work 
 with const.


A copy ctor does not copy anything beforehand. True. Then the 
idea with this new feature is to leave it to you to copy the 
fields from inside your new copy ctor. But you will no more be 
able to copy them from a copy ctor than you were from a postblit 
because you have a const instance as your source and something 
extra is needed for each field. Now if that something extra is 
more copy constructors all down the line - I think that would 
work but is unnecessary. I think that the real need is a global 
dup or deep copy function. It should be possible without touching 
our structs and I think I have something fairly close when it 
comes to structs, associative arrays, and dynamic arrays.


 So, as nice an idea as it is, the postblit constructor is a 
 failure.


I feel it is not a failure. It is doing its job just fine. The 
natural tendency, especially coming from c++ is to assume const 
values assigned into non-const lvalues should be ok - because we 
are used to deep copy semantics just working. The only problem is 
copy construction (and therefore postblit) is not the right tool 
to copy const instances with reference semantics. I think a 
global dup function would be the right tool and it is doable, 
hopefully with a reasonable performance overhead.

The issue is you want to copy a const(T). Suppose I give you a 
function that works like this (by the way this is working code (I 
use 2.061))

import std.stdio;
import opmix.mix;
struct S {
   char[] c;
}

void foo(ref const(S) s) {
   // I really need a copy here
   auto copy = s.gdup;
   assert(s.c.ptr != copy.c.ptr);
   writeln(copy);
}
void main() {
   S s = {['a','b','c']};
   foo(s);
}

Now if you had this gdup, all your copy ctors would look like 
this:
struct S {
   char[] c;
   S(ref const(S) s) {
      swap(this, s.gdup);
   }
}

But if you had that gdup in the first place, there would be no 
need for copy ctors. Besides, if the compiler could *know* that 
my new copy ctor was really doing a deep copy - why make me write 
it in the first place. Just provide the implementation for me. 
Allowing me to do anything in the copy ctor allows me to 
introduce sharing which is forbidden.

Does this make sense?

Thanks
Dan
Nov 15 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, November 15, 2012 19:30:03 Dan wrote:

 On Thursday, 15 November 2012 at 17:24:44 UTC, Jonathan M Davis
 
 wrote:

 On Thursday, November 15, 2012 14:21:41 Dan wrote:

 I do not see how copy constructors can help. Receiving const(T)
 or const(T) ref to anything and hoping to copy it is the
 challenge and doing it from a copy constructor is no easier
 than
 a postblit.


 
 Doing it from a postblit is impossible.


 
 I understand and this makes sense. postblit is not a deep copy,
 it is only what you choose to make it.


The entire reason that postblit exists is to allow deep copies, just like with 
a copy constructor. You can certainly create a type which doesn't do a deep 
copy in its postblit constructor, but the reason that it's in the language is 
to make it so that you can do a deep copy of member variables which normally 
would end up being shallowly copied. It's D's attempt at making a better copy 
constructor. Pure and simple. But because of issues with const, it fails at 
that.


 The problem for this case is it can
 not guarantee what I do *in* my postblit and therefore it does
 not and can not know I'm leaving with no sharing.


The main problem is that once you're inside the postblit construct, the new 
object has already been fully constructed. And since it's illegal to cast away 
const and modify an object, it is therefore impossible to modify the new 
object inside the postblit constructor, making the postblit constructor 
utterly pointless and making it impossible to copy that struct through any 
standard means. You could create your own dup method, but that doesn't help 
one iota with all of the places in the language where you need the language to 
be making copies for you (e.g. when you dup an array of that struct type).

Creating a non-const copy of a const object doesn't work either, but you're 
totally screwed if you can't even make a deep copy of const copy of a const 
object. Copy constructors would solve both problems.


 The suggestion is use
 copy constructors instead. I don't see how this will helpx


A copy constructor will help, because it will make it possible to copy const 
objects which have member variables which are reference types and have a deep 
copy made because you'll be constructing the new object directly rather than 
copying it and then making changes to it. Right now, all you can get is a 
shallow copy, because postblit doesn't work with const.


 but I
 have not found the proposal (do you have a link?). I have read
 DIP10 which was an early attempt to get at this stuff - but I
 think it did not go far.


There is no DIP for it. Andrei and Walter have discussed it in private, and 
all they would say for a while was that they had a solution. They wouldn't 
give what it was. Andrei did finally mention not too long ago that it involved 
introducing copy constructors and phasing out postblit constructors. But 
nothing has happened yet, and it has not yet been discussed in detail in the 
newsgroup, so what exactly is involved beyond the fact that they intend to 
introduce copy constructors to solve the problem, I don't know.


 The only problem is
 copy construction (and therefore postblit) is not the right tool
 to copy const instances with reference semantics.


That's a separate issue from the fact that const postblits don't work. If 
you've given a type reference semantics, then obviously copy constructors and 
postblits aren't going to do deep copies. For that, you'd probably create a 
dup function. But many, many types need to be able to be copied as value types 
even though they have member variables which are reference types - which is 
what copy constructors and postblit constructors are for - and they can't do 
that in D right if they're const, because const and postblit don't mix.

- Jonathan M Davis
Nov 15 2012
parent reply "Dan" <dbdavidson yahoo.com> writes:
Thanks, I think I get it. const and postblit don't mix. Copy 
ctors don't yet exist but are the hoped for solution to inability 
to copy const instances. Copy ctor proposal/solution has not yet 
been written up in the newsgroups or details discussed beyond 
Andrei and Walter, but have been and maybe still are being worked.

I wish I knew more about the copy ctor proposal.

Here is the issue that haunts me: Say it is January 15 and we 
have:

module goo;
struct Goo {
   int x = 42;
}

module foo;
struct Foo {
   char[] c;
   Goo goo;
   this(ref const(Foo) other) {
     c = other.dup;
     goo = other;
   }
}

February comes around and developers of Goo want to introduce a 
new private member *int[int] ageToWeight*. Now foo won't compile, 
because Goo does not also have a copy constructor and went from 
value semantics to reference semantics in the change. This seems 
very fragile - but maybe that is not the proposal. I guess I was 
just hoping copying of const can be taken out of the struct's 
hands entirely. Contrast that situation with the state on 
February with a proper global dup. There is no copy ctor problem, 
no postblit issue. Using something other than the copy ctor and 
postblit to do the copy of the const has greatly reduced effort. 
This code below does work right now the way you would expect. I'm 
not saying it is a panacea - I've only tried it out a bit, but 
you have to admit a global dup like function that could copy 
*any* const you throw at it might put this entire issue to rest 
without the need for any change. Mine could do not that; I'm 
specifically avoiding classes. But I don't know what inherent 
limitation would prevent it. Even if you were concerned with some 
type T that had its own low level memory management, you could 
still have it provide it's own dup with a proper const qualifier, 
which gdup could call in preference to a recursive call to gdup.

You mention that possibly postblits will go away. That is 
discomforting.
In terms of priority, where does this fall?

Thanks
Dan

-----------------------------------------
import std.stdio;
import std.traits;
import opmix.mix;
struct Goo {
   private int[int] ageToWeight;
   int x = 42;
}

struct Foo {
   char[] c;
   Goo goo;
}

void main() {
   Foo foo = { ['a'], Goo([15:123, 30:180, 45:195]) };
   auto copy = foo.gdup;
   // No sharing of c and ageToWeight copied
   assert(copy.c.ptr != foo.c.ptr &&
          copy.goo.ageToWeight == foo.goo.ageToWeight);
   // No sharing of ageToWeight
   copy.goo.ageToWeight[70] = 178;
   assert(copy.goo.ageToWeight != foo.goo.ageToWeight);
   writeln(foo.gdup);
}
-------------------------------------------

Thanks,
Dan
Nov 15 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, November 16, 2012 06:17:55 Dan wrote:

 You mention that possibly postblits will go away. That is
 discomforting.


It would likely stay around for a very long time for backwards compatibility 
and simply be discouraged, but I don't know. Walter is big on backwards 
compatilbility though, so it's not like postblit will suddenly be removed and 
break everyone's code.


 In terms of priority, where does this fall?


No idea. Clearly, it's not super high priority, or it would have been dealt 
with by now. But there have also been plenty of cases where nothing happens 
with something ages and then Walter suddenly fixes it. So who knows when it'll 
happen.

- Jonathan M Davis
Nov 15 2012