www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Safe cast away from immutable

reply Iakh <iaktakh gmail.com> writes:
import std.stdio;

struct S
{
     void[] arr;

     auto f() pure  safe
     {
         int[] a = new int[4];
         arr = a;
         return a;
     }
}
void main()  safe
{
     S s;
     immutable a = s.f();
     int[] b = (cast(int[])s.arr);
     writeln(a);
     b[0] = 1;
     writeln(a);
}

http://dpaste.dzfl.pl/13751913d2ff
Feb 08 2016
next sibling parent reply Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 8 February 2016 at 19:07:01 UTC, Iakh wrote:

 import std.stdio;

 struct S
 {
     void[] arr;

     auto f() pure  safe
     {
         int[] a = new int[4];
         arr = a;
         return a;
     }
 }
 void main()  safe
 {
     S s;
     immutable a = s.f();
     int[] b = (cast(int[])s.arr);
     writeln(a);
     b[0] = 1;
     writeln(a);
 }

 http://dpaste.dzfl.pl/13751913d2ff


I'm pretty sure this is not safe. Works, but not safe. You happen 
to be referencing the same memory block you created on the heap, 
but I believe the compiler is free to move the memory around if 
it can find a reason to do so.

Remember, when you cast the compiler is no longer protecting you. 
By modifying the array you're taking on the responsibility to 
know where that memory is located.
Feb 08 2016
parent reply Iakh <iaktakh gmail.com> writes:
On Monday, 8 February 2016 at 19:33:54 UTC, Jesse Phillips wrote:


 I'm pretty sure this is not safe. Works, but not safe. You


So it is bug?
Feb 08 2016
parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 8 February 2016 at 20:07:49 UTC, Iakh wrote:

 On Monday, 8 February 2016 at 19:33:54 UTC, Jesse Phillips 
 wrote:


 I'm pretty sure this is not safe. Works, but not safe. You


 So it is bug?


Yeah, I missed a couple items of your code. You'd marked the 
functions  safe, and also the cast(int[]) was to convert void[] 
not to cast away an immutable.
Feb 08 2016
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 8 February 2016 at 19:07:01 UTC, Iakh wrote:

 import std.stdio;

 struct S
 {
     void[] arr;

     auto f() pure  safe
     {
         int[] a = new int[4];
         arr = a;
         return a;
     }
 }
 void main()  safe
 {
     S s;
     immutable a = s.f();
     int[] b = (cast(int[])s.arr);
     writeln(a);
     b[0] = 1;
     writeln(a);
 }

 http://dpaste.dzfl.pl/13751913d2ff


The bug is that it's even legal to declare a as immutable. pure 
functions that are strongly pure allow for the return value to 
have its mutability altered, because it's guaranteed that what's 
being returned isn't referenced anywhere else in the program. 
However, f is not strongly pure. It's taking the this 
pointer/reference as mutable, not immutable, which allows for the 
memory that's allocated in it to escape - as you've done in this 
example.

So, the compiler logic with regards to pure is buggy. Simply 
having

     struct S
     {
         void[] arr;

         auto f() pure  safe
         {
             int[] a = new int[4];
             arr = a;
             return a;
         }
     }
     void main()  safe
     {
         S s;
         immutable a = s.f();
     }


in a bug report should be sufficient to show the bug, even 
without the rest of what you're doing.

In general, it should be impossible for member functions to be 
considered strongly pure unless they're marked as immutable, 
though the compiler could certainly be improved to determine that 
no escaping of the return value or anything referencing it occurs 
within the function and that thus it can get away with treating 
the return value as if it's the only reference to that data, but 
that would likely be farther into the realm of code flow analysis 
than the compiler typically does.

Regardless, your example definitely shows a bug. Please report 
it. Thanks.

- Jonathan M Davis
Feb 08 2016
parent reply Iakh <iaktakh gmail.com> writes:
On Monday, 8 February 2016 at 20:43:23 UTC, Jonathan M Davis 
wrote:

 in a bug report should be sufficient to show the bug, even 
 without the rest of what you're doing.

 In general, it should be impossible for member functions to be 
 considered strongly pure unless they're marked as immutable, 
 though the compiler could certainly be improved to determine 
 that no escaping of the return value or anything referencing it 
 occurs within the function and that thus it can get away with 
 treating the return value as if it's the only reference to that 
 data, but that would likely be farther into the realm of code 
 flow analysis than the compiler typically does.


It does. A bit. If S.arr is int[] the program fails to compile.
Is all prams being const(but not immutable) not enough for
function to be Pure?


 Regardless, your example definitely shows a bug. Please report 
 it. Thanks.


Done
https://issues.dlang.org/show_bug.cgi?id=15660
Feb 08 2016
next sibling parent anonymous <anonymous example.com> writes:
On 08.02.2016 22:14, Iakh wrote:

 Is all prams being const(but not immutable) not enough for
 function to be Pure?


The hidden `this` parameter must be const, too. And mutable indirections 
in the return type must be considered.

This article explains it in detail:
http://klickverbot.at/blog/2012/05/purity-in-d/
Feb 08 2016
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 8 February 2016 at 21:14:11 UTC, Iakh wrote:

 On Monday, 8 February 2016 at 20:43:23 UTC, Jonathan M Davis 
 wrote:

 in a bug report should be sufficient to show the bug, even 
 without the rest of what you're doing.

 In general, it should be impossible for member functions to be 
 considered strongly pure unless they're marked as immutable, 
 though the compiler could certainly be improved to determine 
 that no escaping of the return value or anything referencing 
 it occurs within the function and that thus it can get away 
 with treating the return value as if it's the only reference 
 to that data, but that would likely be farther into the realm 
 of code flow analysis than the compiler typically does.


 It does. A bit. If S.arr is int[] the program fails to compile.
 Is all prams being const(but not immutable) not enough for
 function to be Pure?


Whether the function can be pure or not has nothing to do with 
const or immutable. All that pure means in and of itself is that 
the function cannot access any module-level or static variables 
whose state can ever change after they're initialized (though 
immutable is fine, and const is fine if there can't be any other 
references to the same data). So, every bit of state that can 
change over the course of the program that a pure function can 
access is through its arguments (which includes the invisible 
this pointer/reference in the case of member functions).

Now, depending on the types of the arguments, the compiler can do 
various optimizations. The most basic would be that if all of the 
arguments are immutable, then if the same values are given, the 
compiler can elide multiple calls - e.g. pureFunc(a) * 
pureFunc(a) could just call pureFunc once and then reuse the 
result rather than calling it twice as would occur normally.

One of the most useful things that the compiler can do if a 
function is pure is that if it can guarantee that the return 
value did not get passed in as an argument or was otherwise 
obtained via an argument, then it knows that the argument was 
created within the function and that therefore the return value 
is the only reference to that data, and so it's safe to alter its 
mutability - e.g. change a mutable array to an immutable one. 
Exactly which conditions under which that can be determined to be 
safe are not exactly straightforward. The simplest is when all of 
the arguments were immutable, but there are others, some of which 
are much more complicated (and under some circumstances, const 
can help with that, whereas in others, it can't - it really 
depends on the types involved). I don't know how sophisticated 
the compiler is in determining that right now, but clearly, what 
it currently has is buggy, because it failed to take the 
invisible this pointer/reference into account in your example and 
thus incorrectly determined that it was not possible for another 
reference to the same data to exist after the function returned.


 Regardless, your example definitely shows a bug. Please report 
 it. Thanks.


 Done
 https://issues.dlang.org/show_bug.cgi?id=15660


Thanks.

- Jonathan M Davis
Feb 08 2016
next sibling parent Iakh <iaktakh gmail.com> writes:
On Monday, 8 February 2016 at 21:48:30 UTC, Jonathan M Davis 
wrote:


 that right now, but clearly, what it currently has is buggy,


Yeah. Looks like it just traverse params's AST and search for
exactly match with ReturnType.

The code with replaced (void, int) with (class A, class B : A)
behaves the same way as original:

import std.stdio;

class A
{
     int i;
}

class B : A
{
}

struct S
{
     A a;

     auto f() pure  safe
     {
         B b = new B;
         a = b;
         return b;
     }
}

void main()  safe
{
     S s;
     immutable a = s.f();
     A b = s.a;
     writeln(a.i);
     b.i = 1;
     writeln(a.i);
}
Feb 08 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 2/8/16 4:48 PM, Jonathan M Davis wrote:


 One of the most useful things that the compiler can do if a function is
 pure is that if it can guarantee that the return value did not get
 passed in as an argument or was otherwise obtained via an argument, then
 it knows that the argument was created within the function and that
 therefore the return value is the only reference to that data, and so
 it's safe to alter its mutability - e.g. change a mutable array to an
 immutable one. Exactly which conditions under which that can be
 determined to be safe are not exactly straightforward. The simplest is
 when all of the arguments were immutable, but there are others, some of
 which are much more complicated (and under some circumstances, const can
 help with that, whereas in others, it can't - it really depends on the
 types involved). I don't know how sophisticated the compiler is in
 determining that right now, but clearly, what it currently has is buggy,
 because it failed to take the invisible this pointer/reference into
 account in your example and thus incorrectly determined that it was not
 possible for another reference to the same data to exist after the
 function returned.


I'm not so sure. I think there is something we never considered, and the 
compiler might be more clever than it should be.

I tested this out:

int[] f(ref void[] m) pure
{
     auto result = new int[5];
     m = result;
     return result;
}

void main()
{
     void[] v;
     immutable x = f(v);
}

Compiles.

I think the rules at the moment are that the compiler allows implicit 
conversion if the return value could not have come from any of the 
parameters. But what it may not consider is if the parameters could be 
return values themselves via a reference!

What I think is happening is that the compiler is saying "Well, there is 
no way that int[] could come from m (without a cast), so I'm going to 
assume it's unique". However, we can see it's not. m actually is 
generated from it!

I'll amend the bug.

-Steve
Feb 09 2016
parent Iakh <iaktakh gmail.com> writes:
On Tuesday, 9 February 2016 at 16:32:07 UTC, Steven Schveighoffer 
wrote:

 I think the rules at the moment are that the compiler allows 
 implicit conversion if the return value could not have come 
 from any of the parameters. But what it may not consider is if 
 the parameters could be return values themselves via a 
 reference!


AFAIK it just traverse members and search for exactly match with 
return
type. Even more there is different checks for "pure" and "unique 
owned
result".

import std.stdio;

int[] f(void[] a)  safe pure
{
     return cast(int[])a;
}

void main()  safe
{
     int[] a = new int[4];

     immutable b = a.f();
     writeln(b);
     a[0] = 1;
     writeln(b);
}

This works until you change void[] to int[].
And error message is about casting to immutable not about purity.

This is terribly because if there is a code:

int[] f(SophisticatedClass a){...}
immutable a = f(new SophisticatedClass);

that was working last 100 years but then somebody adds member
of type int[] into SophisticatedClass.field.field.field
You take sort of this:
Error: cannot implicitly convert expression (f(a)) of type int[] 
to immutable(int[])

It's hard to prove that result is unique. So maybe don't try to 
do?
Feb 09 2016