www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - logical const without casts!

reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
I just thought of an interesting way to make a logical const object  
without casts.  It requires a little extra storage, but works without  
changes to the current compiler (and requires no casts).

Here's the code, then I'll talk about the implications:

import std.stdio;

class D
{
     D getme() { return this;}
     void foo() {writeln("mutable!");}
}

class C
{
     D delegate() d;
     this()
     {
       auto dinst = new D;
       this.d = &dinst.getme;
     }
     void bar() const { d().foo();}
}

void main()
{
     auto c = new C;
     c.bar();
}

outputs:

mutable!

So how does it work?  It works because delegates and especially the  
delegate data is *not* affected by const.  So even when C is temporarily  
cast to const, the delegate is not affected (i.e. it's context pointer is  
not temporarily cast to const).

Doesn't this poke holes in const?  Of course it does, but no more holes  
than are present via another logical const scheme (i.e. using a globally  
stored AA to retrieve the data).

Why is this better than simply casting away const?

First, because simply casting away const and modifying the data is  
undefined, I don't think the method I created results in undefined  
behavior.  Second, because compiler is still fully enforcing  
const/immutable.  For example, you could not assign d to &dinst.getme if  
dinst is const or immutable, and once inside a const function, I cannot  
change the delegate, I can only call it.  Third, because the D instance  
*doesn't actually live* in the C object.  You can see, I *never* assigned  
a D instance to a member variable of C, I only assigned the delegate.  The  
D instance lives on the heap (and technically can be passed in via the  
constructor if need be).

The only drawbacks here are, we have to needlessly store the delegate  
function pointer (assuming we are always going to use &getme), and the  
delegate cannot be inlined.

I'm actually thinking that very controlled patterns of logical const like  
this could be implemented via mixin, and be sanctioned by the library.   
The way this pattern works, you can dictate as the author of a class  
whether that class can be a logically const part of another object or not,  
simply by choosing to implement getme or not.

What do people think about this?

-Steve
Sep 29 2011
next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).


[snip]

 What do people think about this?


This is what I think about it:

class A {
     int n;
     void delegate( ) dg;
}

pure
A createAnA( int n ) {
     A result = new A;
     result.n = n;
     result.dg = (){ result.n++; };
     return result;
}

void main( ) {
     immutable A tmp = createAnA( 3 );
     assert( tmp.n == 3 );
     tmp.dg();
     assert( tmp.n == 3 );
}

-- 
   Simen
Sep 29 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas  
<simen.kjaras gmail.com> wrote:


 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:


 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).


 [snip]

 What do people think about this?


 This is what I think about it:

 class A {
      int n;
      void delegate( ) dg;
 }

 pure
 A createAnA( int n ) {
      A result = new A;
      result.n = n;
      result.dg = (){ result.n++; };
      return result;
 }

 void main( ) {
      immutable A tmp = createAnA( 3 );
      assert( tmp.n == 3 );
      tmp.dg();
      assert( tmp.n == 3 );
 }


I agree this breaks immutability, and needs to be addressed.  I think  
probably implicit casting of delegates (or items that containe delegates)  
to immutable from strong-pure functions should be disallowed.

But it's not the pattern I described, and uses a relatively new trick  
(implicit immutable casting).  I'll file a bug for this case.

-Steve
Sep 29 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 29 Sep 2011 13:26:11 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas  
 <simen.kjaras gmail.com> wrote:


 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:


 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).


 [snip]

 What do people think about this?


 This is what I think about it:

 class A {
      int n;
      void delegate( ) dg;
 }

 pure
 A createAnA( int n ) {
      A result = new A;
      result.n = n;
      result.dg = (){ result.n++; };
      return result;
 }

 void main( ) {
      immutable A tmp = createAnA( 3 );
      assert( tmp.n == 3 );
      tmp.dg();
      assert( tmp.n == 3 );
 }


 I agree this breaks immutability, and needs to be addressed.  I think  
 probably implicit casting of delegates (or items that containe  
 delegates) to immutable from strong-pure functions should be disallowed.

 But it's not the pattern I described, and uses a relatively new trick  
 (implicit immutable casting).  I'll file a bug for this case.


http://d.puremagic.com/issues/show_bug.cgi?id=6741

-Steve
Sep 29 2011
parent reply travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145767), a
 écrit :

 On Thu, 29 Sep 2011 13:26:11 -0400, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:
 

 On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas  
 <simen.kjaras gmail.com> wrote:


 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:


 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).


 [snip]

 What do people think about this?


 This is what I think about it:


 I agree this breaks immutability, and needs to be addressed.  I think  
 probably implicit casting of delegates (or items that containe  
 delegates) to immutable from strong-pure functions should be disallowed.

 But it's not the pattern I described, and uses a relatively new trick  
 (implicit immutable casting).  I'll file a bug for this case.


 
 http://d.puremagic.com/issues/show_bug.cgi?id=6741



Well, if the langage wants to be consistent, you should prevent implicit 
casting of delegates to const, not just to immutable.

What you revealed is very interesting, but it is clearly a bug. A 
delegate context pointer in a const object should be const, and the 
delegate function should be const with regard to its context pointer if 
you want the function to be called when the delegate is const. 
Constness has to apply to delegates just like it applies to structs. The 
explanation gets messy. An example:


This is what the compiler should say:

 class A {
      int n;
      void delegate( ) dg;
 }

 pure
 A createAnA( int n ) {
      A result = new A;
      result.n = n;
      result.dg = (){ result.n++; };
      return result;
 }

 void main( ) {
      immutable A tmp = createAnA( 3 );
      assert( tmp.n == 3 );
      tmp.dg();






^error: cannot call non const delegate A.dg from an immutable object.

      assert( tmp.n == 3 );
 }






An this is the fix:

 class A {
      int n;






      void delegate( ) const dg; // or const(void delegate()) dg;

 }

 pure
 A createAnA( int n ) {
      A result = new A;
      result.n = n;






      // result.dg = (){ result.n++; }; would result in:
      // error: non const (unamed) delegate cannot be assigned to 
      // const delegate A.dg.
      result.dg = (){ result.n+1; }; // this one is fine.

      return result;
 }

 void main( ) {
      immutable A tmp = createAnA( 3 );
      assert( tmp.n == 3 );






         tmp.dg(); // OK, A.dg is const so it can be called

      assert( tmp.n == 3 );
 }







I guess you could also hide a mutable pointer of an immutable object 
during its construction with the same trick.

-- 
Christophe
Sep 30 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 09:27:31 -0400, Christophe  =

<travert phare.normalesup.org> wrote:


 "Steven Schveighoffer" , dans le message (digitalmars.D:145767), a
  =C3=A9crit :

 On Thu, 29 Sep 2011 13:26:11 -0400, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:


 On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas
 <simen.kjaras gmail.com> wrote:


 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:


 I just thought of an interesting way to make a logical const objec=










t

 without casts.  It requires a little extra storage, but works with=










out

 changes to the current compiler (and requires no casts).


 [snip]

 What do people think about this?


 This is what I think about it:


 I agree this breaks immutability, and needs to be addressed.  I thin=






k

 probably implicit casting of delegates (or items that containe
 delegates) to immutable from strong-pure functions should be  =







 disallowed.

 But it's not the pattern I described, and uses a relatively new tric=






k

 (implicit immutable casting).  I'll file a bug for this case.


 http://d.puremagic.com/issues/show_bug.cgi?id=3D6741



 Well, if the langage wants to be consistent, you should prevent implic=


it

 casting of delegates to const, not just to immutable.

 What you revealed is very interesting, but it is clearly a bug. A
 delegate context pointer in a const object should be const, and the
 delegate function should be const with regard to its context pointer i=


f

 you want the function to be called when the delegate is const.
 Constness has to apply to delegates just like it applies to structs. T=


he

 explanation gets messy.


If const must be a part of the signature, then you have lost the typeles=
s  =

aspect of the context pointer.  If we expose the const (or not const)  =

nature of the pointer, then you lose the interoperability that delegates=
  =

provide.  The beauty of delegates is you don't *have to* care what the  =

type of the context pointer is.  That's defined by the function the  =

delegate represents.

It's why, for example, a function accepting a delegate does not  =

distinguish between a delegate literal and a delegate to a member functi=
on  =

of type Foo or a delegate to a member function of type const(Bar).  You =
 =

must think of the context pointer as a hidden parameter to the delegate,=
  =

as defined when the delegate was created *not* when it is called.  The  =

fact that it's actually stored with the delegate pointer is irrelevant. =
  =

Conceptually, it's not stored anywhere, it's just a parameter.

But in the case of the bug I filed, we are talking about a  =

compiler-sanctioned implicit transformation to immutable.  We *must*  =

guarantee that when we allow an implicit transformation, that there are =
no  =

existing mutable copies of the data.  An explicit transformation should =
be  =

allowed, because then the user has accepted responsibility.

The same is not true for const.  It's *perfectly legal* to have a const =
 =

and mutable reference to an object at the same time.  The only reason  =

transitive const exists and is necessary is to support transitive  =

immtuable.

Const is never guaranteed to prevent changing data on an object:

class C
{
    int x;
    void multiply(C other) const {other.x *=3D x;}
}

Is multiply guaranteed not to change the object?  No:

c.multiply(c);

However, if c is immutable, it *is* guaranteed, because there's no way t=
o  =

pass c as the parameter to multiply.

That is why I think the solution works -- if you allow the compiler to  =

enforce the rules of const and immutable, you can still do unexpected  =

things, but you do not break the guarantees of immutability.  It's  =

definitely a loophople, but I think it's valid.


 I guess you could also hide a mutable pointer of an immutable object
 during its construction with the same trick.


But saving a mutable object as immutable requires a cast.  It means  =

"compiler, I take responsibility for ensuring this no longer has any  =

mutable references".  When the cast is *not* required, it's the compiler=
's  =

responsibility.

-Steve
Sep 30 2011
parent reply travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145821), a
 écrit :

 If const must be a part of the signature, then you have lost the typeless  
 aspect of the context pointer.  If we expose the const (or not const)  
 nature of the pointer, then you lose the interoperability that delegates  
 provide.  The beauty of delegates is you don't *have to* care what the  
 type of the context pointer is.  That's defined by the function the  
 delegate represents.


No, you don't lose the typeless aspect of the context pointer. You lose 
a litle bit of that aspect, but not that much.

In D, const is transitive. That means: everything you touch via a const 
object remains const. non-const delegate context pointer breaks that 
rule.

You have const member functions, and initially, delegates were member 
functions associated with an object, so why could not the delegate be 
const ? A const delegate would just be a delegate that promise it 
doesn't touch to its context. It's not a type-defined delegate.

If I implement a delegate at the library level, when I call the 
function, I have to make a cast to access the context pointer. With cast 
removing implicitely the constness of the context pointer, I get the 
current D implementation of delegates. This is legal, but this is also 
undefined behavior. I believe that langage delegate work that way, that 
they are thus undefined behavior, and this should be corrected.

I agree the langage could stipulate that delegate context pointer are 
exceptions, to allow the valuable trick like you revealed. But, to me, 
that is not the current policy about constness.


Do you see the problem in the following code:

class Foo
{
  this(ref int _i) { i = () { return _i; } }
  ref int delegate() dg;
  ref int i() const { return dg(); }
}

int globalInt;

immutable Foo globalFoo = Foo(globalInt);

int bar(const Foo foo)
{
    return foo.i++;
}

int globalBar()
{
    return bar(globalFoo);
}


Answer:

In multithreaded application, globalFoo, which is immutable, is 
automatically shared. However, globalInt is not. globalBar allows you to 
access to (and modify) globalInt. But globalInt is not protected for 
concurrent access. And globalInt from which thread is accessed ?

Possible solutions:

- do nothing, and let the programmer introducing multi-threading in 
the application deal with this, even if the programmers of Foo, bar and 
buggy did not cared to document the impact of their code for 
multithreaded applications.

- forbid to put/call any delegate into an immutable object: that almost 
means forbiding to put/call a delegate into a const object.

- what I propose: implement the separate kind of const delegates, that 
allows to protect their context pointers, and that you can safely call 
from const/immutable data.


 But in the case of the bug I filed, we are talking about a  
 compiler-sanctioned implicit transformation to immutable.  We *must*  
 guarantee that when we allow an implicit transformation, that there are no  
 existing mutable copies of the data.  An explicit transformation should be  
 allowed, because then the user has accepted responsibility.


With my proposal, you can very easily keep an immutable reference in a 
const delegate. The delegate will just not be callable if its 
function pointer is not const with regard to the context pointer.


In any case, if the langage decides that delegate context pointer should 
escape const protection, great care should be taken to make sure they 
don't escape purity. In my example, there should be a compiler error if 
I tried to declare Foo.i and bar pure (maybe there is already an error, 
I didn't test).

-- 
Christophe
Sep 30 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 12:16:03 -0400, Christophe  =

<travert phare.normalesup.org> wrote:


 "Steven Schveighoffer" , dans le message (digitalmars.D:145821), a
  =C3=A9crit :

 If const must be a part of the signature, then you have lost the  =





 typeless
 aspect of the context pointer.  If we expose the const (or not const)=





 nature of the pointer, then you lose the interoperability that delega=




tes

 provide.  The beauty of delegates is you don't *have to* care what th=




e

 type of the context pointer is.  That's defined by the function the
 delegate represents.


 No, you don't lose the typeless aspect of the context pointer. You los=


e

 a litle bit of that aspect, but not that much.

 In D, const is transitive. That means: everything you touch via a cons=


t

 object remains const. non-const delegate context pointer breaks that
 rule.


But transitive const by itself doesn't get you any guarantees.  It does =
 =

not prevent you from modifying that (mutable) value via another pointer.=


What it does is allow code that works with both transitive immutable and=
  =

mutable without having to copy code.


 You have const member functions, and initially, delegates were member
 functions associated with an object, so why could not the delegate be
 const ? A const delegate would just be a delegate that promise it
 doesn't touch to its context. It's not a type-defined delegate.


A delegate for a const member function today already does not touch its =
 =

context.  There is no need to expose the type.


 If I implement a delegate at the library level, when I call the
 function, I have to make a cast to access the context pointer. With ca=


st

 removing implicitely the constness of the context pointer, I get the
 current D implementation of delegates. This is legal, but this is also=



 undefined behavior. I believe that langage delegate work that way, tha=


t

 they are thus undefined behavior, and this should be corrected.


As long as you assume the context pointer is part of the state of the  =

aggregate, then yes.  But I don't see it that way.  The context pointer =
is  =

part of the state of the delegate, and the delegate reference itself is =
 =

the same as a function pointer -- it's not affected by immutable or cons=
t.


 Do you see the problem in the following code:

 class Foo
 {
   this(ref int _i) { i =3D () { return _i; } }
   ref int delegate() dg;
   ref int i() const { return dg(); }
 }

 int globalInt;

 immutable Foo globalFoo =3D Foo(globalInt);

 int bar(const Foo foo)
 {
     return foo.i++;
 }

 int globalBar()
 {
     return bar(globalFoo);
 }


 Answer:

 In multithreaded application, globalFoo, which is immutable, is
 automatically shared. However, globalInt is not. globalBar allows you =


to

 access to (and modify) globalInt. But globalInt is not protected for
 concurrent access. And globalInt from which thread is accessed ?


Yes, I see the problem.  It comes from immutable being shared implicitly=
.   =

To reiterate, the fact that the data is not immutable or const is *not* =
an  =

issue, globalInt is not immutable.  The issue is purely the sharing aspe=
ct  =

of immutable.


 Possible solutions:

 - do nothing, and let the programmer introducing multi-threading in
 the application deal with this, even if the programmers of Foo, bar an=


d

 buggy did not cared to document the impact of their code for
 multithreaded applications.

 - forbid to put/call any delegate into an immutable object: that almos=


t

 means forbiding to put/call a delegate into a const object.

 - what I propose: implement the separate kind of const delegates, that=



 allows to protect their context pointers, and that you can safely call=



 from const/immutable data.


This also doesn't work:

class Foo
{
    this(ref int _i) { i =3D () { return _i; } }
    ref const(int) delegate() const dg;
    ref const(int) i() const { return dg(); }
}

int globalInt;

immutable Foo globalFoo =3D Foo(globalInt);

void thread1()
{
    writeln(globalFoo.i);
}

int globalBar()
{
    spawn(&thread1);
    globalInt =3D 5; // does thread1 see this change or not?
}

It looks like any time you call a delegate member of an immutable object=
,  =

it could access a context pointer that is not implicitly shared.

Not to mention, what the hell does a const function mean for a delegate =
 =

literal?  The context pointer is the context of the function, not an  =

object.  How does that even work?

What is probably the "right" solution is to disallow implicit immutable =
 =

objects which have a delegate.  This means:

a) you cannot initialize a shared or immutable such object without a  =

cast.  This means the line immutable Foo globalFoo =3D new Foo(globalInt=
) is  =

an error without a cast.
b) Any time you have a const object that contains a delegate, it can be =
 =

assumed that the object is not shared.

And then we avoid dealing with the const delegate issue altogether.


 But in the case of the bug I filed, we are talking about a
 compiler-sanctioned implicit transformation to immutable.  We *must*
 guarantee that when we allow an implicit transformation, that there a=




re  =


 no
 existing mutable copies of the data.  An explicit transformation shou=




ld  =


 be
 allowed, because then the user has accepted responsibility.


 With my proposal, you can very easily keep an immutable reference in a=



 const delegate. The delegate will just not be callable if its
 function pointer is not const with regard to the context pointer.


You can also very easily keep a mutable reference in a const delegate  =

inside an immutable object.  It's not mutable through the delegate, but =
 =

it's also not immutable.


 In any case, if the langage decides that delegate context pointer shou=


ld

 escape const protection, great care should be taken to make sure they
 don't escape purity. In my example, there should be a compiler error i=


f

 I tried to declare Foo.i and bar pure (maybe there is already an error=


,

 I didn't test).


Yes, delegates that are pure must be typed that way (there's actually a =
 =

bug I think where you can't explicitly name a pure delegate type).  But =
a  =

pure attribute applies to the *function* not the context pointer.

-Steve
Sep 30 2011
parent reply travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145850), a

 As long as you assume the context pointer is part of the state of the  
 aggregate, then yes.  But I don't see it that way.  The context pointer is  
 part of the state of the delegate, and the delegate reference itself is  
 the same as a function pointer -- it's not affected by immutable or const.


That is actually where our opinions differ.
I wouldn't blame the langage for choosing you proposal rather than mine. 
I just personnaly think my proposal in more in the spirit of the 
langage, and offer more guarantees when dealing with immutables.


 This also doesn't work:


Indeed. The compiler should not allow the implicit conversion of 
globalFoo to immutable.


 class Foo
 {
     this(ref int _i) { dg = () { return _i; } }
     ref const(int) delegate() const dg;
     ref const(int) i() const { return dg(); }
 }
 
 int globalInt;
 
 immutable Foo globalFoo = Foo(globalInt);


^ error: can't cast ref int _i to immutable ref int in immutable 
Foo.this()._delegate_1.


 void thread1()
 {
     writeln(globalFoo.i);
 }
 
 int globalBar()
 {
     spawn(&thread1);
     globalInt = 5; // does thread1 see this change or not?
 }



 It looks like any time you call a delegate member of an immutable object,  
 it could access a context pointer that is not implicitly shared.


Not if the compiler cares to check the delegate context is indeed 
immutable when the immutable delegate is created. It is not much more 
complicated than if Foo contained an explicit pointer to i instead of a 
delegate


 Not to mention, what the hell does a const function mean for a delegate  
 literal?  The context pointer is the context of the function, not an  
 object.  How does that even work?


Internally, it is a pointer to a structure containing the delegate 
context that is generated by the compiler. If the delegate is const, the 
pointer should be a const pointer. If the delegate is immutable, it 
should be an immutable pointer, and the compiler as to check there is no 
mutable version of that pointer before making an implicit cast.


 What is probably the "right" solution is to disallow implicit immutable  
 objects which have a delegate.  This means:
 
 a) you cannot initialize a shared or immutable such object without a  
 cast.  This means the line immutable Foo globalFoo = new Foo(globalInt) is  
 an error without a cast.


Agreed, but with my proposal, you can have an immutable Foo, if the 
delegate context pointer can be cast to immutable.


 b) Any time you have a const object that contains a delegate, it can be  
 assumed that the object is not shared.


If the object is const, you have no guarantee that the objet is not 
shared, making an exception for object containing a delegate is quite 
wierd.


 And then we avoid dealing with the const delegate issue altogether.


Dealing with this issue could be very useful for multithreaded 
applications.


 With my proposal, you can very easily keep an immutable reference in a
 const delegate. The delegate will just not be callable if its
 function pointer is not const with regard to the context pointer.


 
 You can also very easily keep a mutable reference in a const delegate  
 inside an immutable object.  It's not mutable through the delegate, but  
 it's also not immutable.


No, the compiler should check the delegate context is immutable when 
making the cast to immutable.

I admit you have found a very interesting corner-case for my proposal 
that I had not thought of that case. Thank you for that. I think the 
proposal passed the test (until now).
Sep 30 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 14:51:19 -0400, Christophe  
<travert phare.normalesup.org> wrote:


 "Steven Schveighoffer" , dans le message (digitalmars.D:145850), a



 This also doesn't work:


 Indeed. The compiler should not allow the implicit conversion of
 globalFoo to immutable.


 class Foo
 {
     this(ref int _i) { dg = () { return _i; } }
     ref const(int) delegate() const dg;
     ref const(int) i() const { return dg(); }
 }

 int globalInt;

 immutable Foo globalFoo = Foo(globalInt);


 ^ error: can't cast ref int _i to immutable ref int in immutable
 Foo.this()._delegate_1.


But ref int _i is the parameter to the constructor.  Here is where your  
proposal breaks down.

It's entirely feasible for class Foo to be compiled *separately* from the  
module that contains globalFoo.  And it's also possible that when  
compiling the above line, the language *does not have* access to the  
source code of the constructor.  How does it know that _i gets put into a  
const delegate?

Likewise, when compiling Foo, how does the compiler know that the  
constructor is being used to cast to immutable?  I just don't think there  
is enough information for the compiler to make the correct decision.


 void thread1()
 {
     writeln(globalFoo.i);
 }

 int globalBar()
 {
     spawn(&thread1);
     globalInt = 5; // does thread1 see this change or not?
 }



 It looks like any time you call a delegate member of an immutable  
 object,
 it could access a context pointer that is not implicitly shared.


 Not if the compiler cares to check the delegate context is indeed
 immutable when the immutable delegate is created. It is not much more
 complicated than if Foo contained an explicit pointer to i instead of a
 delegate


What if it has no access to the delegate creation code?  How does it  
disallow it?


 Not to mention, what the hell does a const function mean for a delegate
 literal?  The context pointer is the context of the function, not an
 object.  How does that even work?


 Internally, it is a pointer to a structure containing the delegate
 context that is generated by the compiler. If the delegate is const, the
 pointer should be a const pointer. If the delegate is immutable, it
 should be an immutable pointer, and the compiler as to check there is no
 mutable version of that pointer before making an implicit cast.


Hm.. I don't know if there's a way to create such a delegate.  I'd have to  
check.


 What is probably the "right" solution is to disallow implicit immutable
 objects which have a delegate.  This means:

 a) you cannot initialize a shared or immutable such object without a
 cast.  This means the line immutable Foo globalFoo = new Foo(globalInt)  
 is
 an error without a cast.


 Agreed, but with my proposal, you can have an immutable Foo, if the
 delegate context pointer can be cast to immutable.


You can also have an immutable Foo if you cast it to immutable.  For  
example, this line should compile:

immutable foo = cast(immutable(Foo))new Foo(globalInt);

But then the onus is on you to ensure you are doing the right thing.  So  
in both solutions, it's possible.  While I don't think your solution is  
feasible or solves the problem you set out to solve, if you did find a way  
to make the compiler make more guarantees, it would be better.  I just  
don't know if it's worth the extra syntax and restrictions.


 b) Any time you have a const object that contains a delegate, it can be
 assumed that the object is not shared.


 If the object is const, you have no guarantee that the objet is not
 shared, making an exception for object containing a delegate is quite
 wierd.


Well, since you cannot implicitly create such an immutable object, it  
could be an assumption you can make.  However, I can see why it wouldn't  
be a good idea to make that assumption (especially if you created one via  
casting).

We can probably drop this assumption, because to write code like the  
above, even if explicitly casting, you are asking for race conditions.   
Probably safe to assume that it works just like any other const object.


 And then we avoid dealing with the const delegate issue altogether.


 Dealing with this issue could be very useful for multithreaded
 applications.


 With my proposal, you can very easily keep an immutable reference in a
 const delegate. The delegate will just not be callable if its
 function pointer is not const with regard to the context pointer.


 You can also very easily keep a mutable reference in a const delegate
 inside an immutable object.  It's not mutable through the delegate, but
 it's also not immutable.


 No, the compiler should check the delegate context is immutable when
 making the cast to immutable.


As I said above, it's not always possible to check the context.

-Steve
Sep 30 2011
parent travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145867), a
 écrit :

 But ref int _i is the parameter to the constructor.  Here is where your  
 proposal breaks down.
 
 It's entirely feasible for class Foo to be compiled *separately* from the  
 module that contains globalFoo.  And it's also possible that when  
 compiling the above line, the language *does not have* access to the  
 source code of the constructor.  How does it know that _i gets put into a  
 const delegate?
 
 Likewise, when compiling Foo, how does the compiler know that the  
 constructor is being used to cast to immutable?  I just don't think there  
 is enough information for the compiler to make the correct decision.


You say it is impossible for the compiler to check the constness of the 
arguments passed to the constructor ?
Well, allow me to doubt your statement, because my compiler can (it is 
gdc 4.4.5 using dmd 2.052).

Easy test, with a pointer instead of a delegate:

struct Foo
{
  int* _i;
  ref int i() { return *_i; }
  this(int* m_i)
  { _i = m_i;  }
  this(const int* m_i) const
  { _i = m_i; }
  this(immutable(int)* m_i) immutable // change immutable attribute of
        // the constructor and main won't compile.
  { _i = m_i; }
}

void main()
{
  int ma;
  Foo mfoo = Foo(&ma);
  const int ca;
  const Foo cfoo = Foo(&ca);
  immutable int ia;
  immutable Foo ifoo = Foo(&ia);
}

Now try to remove the const or the immutable attributes in the different 
this(int*) overload. You can play and try many changes, if you find a 
way to put an immutable int inside a mutable Foo, you get a banana for 
having found a great hole in the langage (or at least in the compiler).

Protect can (and has to!) be assured for a simple pointer.
Why would it not be the case with a delegate context pointer ?

Now the full test with delegate:

module a;

struct B
{
  int* _b;

  this(int* m_b) { _b = m_b; }
  this(const(int)* m_b) const { _b = m_b; }
  this(immutable(int)* m_b) immutable { _b = m_b; }

  int* b() { return _b; }
  const(int)* b() const { return _b; }
  immutable(int)* b() immutable { return _b; }
}

struct Foo
{
  int* delegate() _i;
  ref int i() { return *_i(); }
  this(int* m_i)
  {
    B b = B(m_i);
    _i = &b.b;
  }
  this(const int* m_i) const
  {
    const B b = B(m_i);
    _i = &b.b;
  }
  this(immutable int* m_i) immutable
  {
    immutable B b = B(m_i);
    _i = &b.b;
    // try to uncomment the following line:
    //    _i = () { return m_i; };
  }
}


module test;
import a;

void main()
{
  int ma;
  Foo mfoo = Foo(&ma);

  const int ca;
  const Foo cfoo = Foo(&ca);

  immutable int ia;
  immutable Foo ifoo = Foo(&ia);
}


To know why I created the struct B, try to uncomment the 
line I indicated in a.d and compile. What do you discover ?

a.d:34: Error: cannot implicitly convert expression (__dgliteral1) of 
type immutable(int*) delegate() to immutable(int* delegate())

So the type I thought I was proposing, the const or immutable delegate 
that protects its context pointer already exists in my compiler. The 
only problem is that at the moment, you have to use an helper structure 
to create this type of delegate...


 What if it has no access to the delegate creation code?  How does it  
 disallow it?


The compiler cannot cast a newly constructed structure to immutable if 
it cannot access the creation code, unless the creation code is marked 
immutable (and then there is actually no cast to do).


Me thinking I could improve the langage... Well, the conversation was 
interesting and I learned a lot about d today.

-- 
Christophe
Sep 30 2011
prev sibling next sibling parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 29.09.2011, 16:54 Uhr, schrieb Steven Schveighoffer  
<schveiguy yahoo.com>:


 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).

 Here's the code, then I'll talk about the implications:

 import std.stdio;

 class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }

 class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }

 void main()
 {
      auto c = new C;
      c.bar();
 }

 [...]

 What do people think about this?

 -Steve


I think this is a creative use of delegates!

--------

class Vendor {
	void sell() {
		++_sold;
	}

	 property
	int sold() {
		return _sold;
	}

private:
	int _sold = 1;
}

class Car {
	this(Vendor vendor) {
		_vendorDlg = () { return vendor; };
	}

	Vendor getVendor() const {
		return _vendorDlg();
	}

private:
	Vendor delegate() _vendorDlg;
}

void main(string[] args) {
	immutable Car car = new immutable(Car)(new Vendor());
	Vendor vendor = car.getVendor();
	vendor.sell();
}
Sep 30 2011
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).
 
 Here's the code, then I'll talk about the implications:
 
 import std.stdio;
 
 class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
 
 class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
 
 void main()
 {
      auto c = new C;
      c.bar();
 }
 
 outputs:
 
 mutable!
 
 So how does it work?  It works because delegates and especially the  
 delegate data is *not* affected by const.  So even when C is 
 temporarily  cast to const, the delegate is not affected (i.e. it's 
 context pointer is  not temporarily cast to const).
 
 Doesn't this poke holes in const?  Of course it does, but no more holes 
  than are present via another logical const scheme (i.e. using a 
 globally  stored AA to retrieve the data).


This is a hole in the transitive const, because the delegate contains a 
pointer to mutable data. It also is a potential source of of low level 
races since returning that type from a pure function could make it 
immutable, which can then make this mutable data accessible to multiple 
threads with no synchronization or atomics to protect the data's 
integrity.



 I'm actually thinking that very controlled patterns of logical const 
 like  this could be implemented via mixin, and be sanctioned by the 
 library.   The way this pattern works, you can dictate as the author of 
 a class  whether that class can be a logically const part of another 
 object or not,  simply by choosing to implement getme or not.


Whatever the implementation I think this is deeply needed. It is needed 
because people are trying all sorts of things to work around const 
transitivity, many of which are subtly unsafe.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Sep 30 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 13:38:31 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 I just thought of an interesting way to make a logical const object   
 without casts.  It requires a little extra storage, but works without   
 changes to the current compiler (and requires no casts).
  Here's the code, then I'll talk about the implications:
  import std.stdio;
  class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
  class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
  void main()
 {
      auto c = new C;
      c.bar();
 }
  outputs:
  mutable!
  So how does it work?  It works because delegates and especially the   
 delegate data is *not* affected by const.  So even when C is  
 temporarily  cast to const, the delegate is not affected (i.e. it's  
 context pointer is  not temporarily cast to const).
  Doesn't this poke holes in const?  Of course it does, but no more  
 holes  than are present via another logical const scheme (i.e. using a  
 globally  stored AA to retrieve the data).


 This is a hole in the transitive const, because the delegate contains a  
 pointer to mutable data. It also is a potential source of of low level  
 races since returning that type from a pure function could make it  
 immutable, which can then make this mutable data accessible to multiple  
 threads with no synchronization or atomics to protect the data's  
 integrity.


Simen Kjaeraas and Christophe brought up the same points.  I filed a bug  
on it:

http://d.puremagic.com/issues/show_bug.cgi?id=6741

-Steve
Sep 30 2011
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 13:46:13 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 Simen Kjaeraas and Christophe brought up the same points.  I filed a bug  
 on it:

 http://d.puremagic.com/issues/show_bug.cgi?id=6741


Also now:

http://d.puremagic.com/issues/show_bug.cgi?id=6747

-Steve
Sep 30 2011
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-09-30 17:46:13 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Fri, 30 Sep 2011 13:38:31 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 

 On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:
 

 I just thought of an interesting way to make a logical const object   
 without casts.  It requires a little extra storage, but works without   
 changes to the current compiler (and requires no casts).
  Here's the code, then I'll talk about the implications:
  import std.stdio;
  class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
  class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
  void main()
 {
      auto c = new C;
      c.bar();
 }
  outputs:
  mutable!
  So how does it work?  It works because delegates and especially the   
 delegate data is *not* affected by const.  So even when C is  
 temporarily  cast to const, the delegate is not affected (i.e. it's  
 context pointer is  not temporarily cast to const).
  Doesn't this poke holes in const?  Of course it does, but no more  
 holes  than are present via another logical const scheme (i.e. using a  
 globally  stored AA to retrieve the data).


 
 This is a hole in the transitive const, because the delegate contains a 
  pointer to mutable data. It also is a potential source of of low level 
  races since returning that type from a pure function could make it  
 immutable, which can then make this mutable data accessible to multiple 
  threads with no synchronization or atomics to protect the data's  
 integrity.


 
 Simen Kjaeraas and Christophe brought up the same points.  I filed a 
 bug  on it:
 
 http://d.puremagic.com/issues/show_bug.cgi?id=6741


Interesting proposal. You realize if we had a true 'mutable' storage 
class for class members it could obey the same rule as you propose in 
that bug report: it should be illegal to cast a mutable-containing 
object or struct to immutable implicitly. An explicit cast should be 
required.

Also, if we had shared delegates -- delegates which are guarantied to 
only manipulate shared data -- we could allow the class that contains a 
shared delegate to be implicitly converted to immutable.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Sep 30 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 14:00:36 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2011-09-30 17:46:13 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 On Fri, 30 Sep 2011 13:38:31 -0400, Michel Fortin   
 <michel.fortin michelf.com> wrote:


 On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer"   
 <schveiguy yahoo.com> said:


 I just thought of an interesting way to make a logical const object    
 without casts.  It requires a little extra storage, but works  
 without   changes to the current compiler (and requires no casts).
  Here's the code, then I'll talk about the implications:
  import std.stdio;
  class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
  class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
  void main()
 {
      auto c = new C;
      c.bar();
 }
  outputs:
  mutable!
  So how does it work?  It works because delegates and especially  
 the   delegate data is *not* affected by const.  So even when C is   
 temporarily  cast to const, the delegate is not affected (i.e. it's   
 context pointer is  not temporarily cast to const).
  Doesn't this poke holes in const?  Of course it does, but no more   
 holes  than are present via another logical const scheme (i.e. using  
 a  globally  stored AA to retrieve the data).


  This is a hole in the transitive const, because the delegate contains  
 a  pointer to mutable data. It also is a potential source of of low  
 level  races since returning that type from a pure function could make  
 it  immutable, which can then make this mutable data accessible to  
 multiple  threads with no synchronization or atomics to protect the  
 data's  integrity.


  Simen Kjaeraas and Christophe brought up the same points.  I filed a  
 bug  on it:
  http://d.puremagic.com/issues/show_bug.cgi?id=6741


 Interesting proposal. You realize if we had a true 'mutable' storage  
 class for class members it could obey the same rule as you propose in  
 that bug report: it should be illegal to cast a mutable-containing  
 object or struct to immutable implicitly. An explicit cast should be  
 required.


Yes, but the benefit of the proposal is -- it already works in the current  
compiler :)  You have a monstrous hill to climb to convince Walter to  
*add* mutable storage class, but we don't even have to ask for this one,  
it works today.


 Also, if we had shared delegates -- delegates which are guarantied to  
 only manipulate shared data -- we could allow the class that contains a  
 shared delegate to be implicitly converted to immutable.


I'd like to avoid the complication of having any attributes that apply to  
the context pointer.  I like how delegates just fit in anywhere, no matter  
what the context pointer type is (or its constancy).

-Steve
Sep 30 2011
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2011-09-30 18:07:54 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Fri, 30 Sep 2011 14:00:36 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 

 Interesting proposal. You realize if we had a true 'mutable' storage  
 class for class members it could obey the same rule as you propose in  
 that bug report: it should be illegal to cast a mutable-containing  
 object or struct to immutable implicitly. An explicit cast should be  
 required.


 
 Yes, but the benefit of the proposal is -- it already works in the 
 current  compiler :)  You have a monstrous hill to climb to convince 
 Walter to  *add* mutable storage class, but we don't even have to ask 
 for this one,  it works today.


It works today but is unsafe due to the implicit cast. But yeah, lets 
use delegates for now.

After the implicit cast is fixed it might be the right time to point 
out to Walter that this exception for delegate is no worse than having 
a direct mutable keyword. Especially if everyone is using this delegate 
detour to achieve what would be simpler and more efficient with 
'mutable'.



 Also, if we had shared delegates -- delegates which are guarantied to  
 only manipulate shared data -- we could allow the class that contains a 
  shared delegate to be implicitly converted to immutable.


 
 I'd like to avoid the complication of having any attributes that apply 
 to  the context pointer.  I like how delegates just fit in anywhere, no 
 matter  what the context pointer type is (or its constancy).


But that's no longer true. Delegates no longer fit anywhere with the 
fix you just proposed. Only 'shared' delegates would fit anywhere. I 
really means *anywhere* since implicitly casting a shared delegate to a 
non-shared delegates is a non-issue: it does not change the type of the 
variables the context is pointing to.

So basically you'd need to require a shared delegate only where a 
normal delegate cannot do the job (like in a class you want to make 
immutable, or wanting to pass the delegate across threads).

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Sep 30 2011