www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: Against enforce()

reply Kagamin <spam here.lot> writes:
Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it should be  
 able to be pure.  Even delegates which modify TLS data should be able to  
 be pure (weak-pure, but still pure).

Pure functions calling weakly pure functions are also weakly pure and so on. This effectively leaves you without purity.
Mar 18 2011
next sibling parent reply Don <nospam nospam.com> writes:
Kagamin wrote:
 Steven Schveighoffer Wrote:
 
 As long as the delegate does not access shared/global data, it should be  
 able to be pure.  Even delegates which modify TLS data should be able to  
 be pure (weak-pure, but still pure).


TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables.
 Pure functions calling weakly pure functions are also weakly pure and so on. 
This effectively leaves you without purity.

I presume you mean "Pure functions calling weakly pure functions *would also be* weakly pure and so on." ?
Mar 18 2011
next sibling parent Kagamin <spam here.lot> writes:
Don Wrote:

 Pure functions calling weakly pure functions are also weakly pure and so on. 
This effectively leaves you without purity.

I presume you mean "Pure functions calling weakly pure functions *would also be* weakly pure and so on." ?

What's the difference?
Mar 18 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 18 Mar 2011 04:34:54 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it should  
 be  able to be pure.  Even delegates which modify TLS data should be  
 able to  be pure (weak-pure, but still pure).


TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables.

However, it's still not shared. This, for example, is a weak pure function: void foo(int *n) pure { *n = 5;} Because TLS variables are not shared, you should be able to do this: int x; void bar() { foo(&x); } But you are right, there is a huge difference between a local reference to TLS data and directly accessing TLS data -- the latter can be obscured from the compiler, resulting in the compiler thinking the function can be strong pure. So I don't know exactly how to mitigate this, but in my mind, it feels like this should work: int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;} int x; void bar() { foo(x == 4, x = 5); } It seems not too different from the above example where you pass the address of x. But obviously the x = 5 delegate cannot be pure (it modifies TLS data). We may have no recourse to get this to work. It may be a lost cause, and you just can't have lazy variables for pure functions. Cue the request for macros where you can just rewrite the syntax vs. having to use lazy in the first place... -Steve
Mar 18 2011
parent reply Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 04:34:54 -0400, Don <nospam nospam.com> wrote:
 
 Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it 
 should be  able to be pure.  Even delegates which modify TLS data 
 should be able to  be pure (weak-pure, but still pure).


TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables.

However, it's still not shared. This, for example, is a weak pure function: void foo(int *n) pure { *n = 5;} Because TLS variables are not shared, you should be able to do this: int x; void bar() { foo(&x); }

Yes, that compiles fine. But bar() is not pure.
 
 But you are right, there is a huge difference between a local reference 
 to TLS data and directly accessing TLS data -- the latter can be 
 obscured from the compiler, resulting in the compiler thinking the 
 function can be strong pure.
 
 So I don't know exactly how to mitigate this, but in my mind, it feels 
 like this should work:
 
 int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
 
 int x;
 
 void bar()
 {
    foo(x == 4, x = 5);
 }
 
 It seems not too different from the above example where you pass the 
 address of x.  But obviously the x = 5 delegate cannot be pure (it 
 modifies TLS data).
 
 We may have no recourse to get this to work.  It may be a lost cause, 
 and you just can't have lazy variables for pure functions.

It's not a lost cause, it's a two-liner! mtype.c line 5045: if (fparam->storageClass & STClazy) { - error(0, "cannot have lazy parameters to a pure function"); + tf->purity = PUREweak; + break; } This is a bit conservative: it would be possible to allow lazy parameters to be marked as pure, which would allow them to be strongly pure. But that would probably not be worth the extra complexity.
Mar 18 2011
parent reply Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 14:35:27 -0400, Don <nospam nospam.com> wrote:
 
 Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 04:34:54 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it 
 should be  able to be pure.  Even delegates which modify TLS data 
 should be able to  be pure (weak-pure, but still pure).


TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables.

This, for example, is a weak pure function: void foo(int *n) pure { *n = 5;} Because TLS variables are not shared, you should be able to do this: int x; void bar() { foo(&x); }

Yes, that compiles fine. But bar() is not pure.
  But you are right, there is a huge difference between a local 
 reference to TLS data and directly accessing TLS data -- the latter 
 can be obscured from the compiler, resulting in the compiler thinking 
 the function can be strong pure.
  So I don't know exactly how to mitigate this, but in my mind, it 
 feels like this should work:
  int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
  int x;
  void bar()
 {
    foo(x == 4, x = 5);
 }
  It seems not too different from the above example where you pass the 
 address of x.  But obviously the x = 5 delegate cannot be pure (it 
 modifies TLS data).
  We may have no recourse to get this to work.  It may be a lost 
 cause, and you just can't have lazy variables for pure functions.

It's not a lost cause, it's a two-liner! mtype.c line 5045: if (fparam->storageClass & STClazy) { - error(0, "cannot have lazy parameters to a pure function"); + tf->purity = PUREweak; + break; } This is a bit conservative: it would be possible to allow lazy parameters to be marked as pure, which would allow them to be strongly pure. But that would probably not be worth the extra complexity.

I'm not sure this works. Aren't you allowed to pass in a delegate to a lazy parameter?

Yes.
 
 For example:
 
 shared int x;
 
 int foo()
 {
    return x;
 }
 
 int bar(lazy int n) pure
 {
    return n;
 }
 
 void baz()
 {
    bar(&foo);
 }
 
 or alternatively:
 
 void baz()
 {
    bar(x);
 }

This compiles just fine. (Well, you need to use bar(foo) not bar(&foo)). But if you try to mark baz() as pure, here's what you get: test0.d(135): Error: pure function 'baz' cannot call impure function 'foo' or for the second case: test0.d(136): Error: pure function 'baz' cannot access mutable static data 'x' bar is just weakly pure.
 The no-shared-data rule prevents you from passing in a shared int 
 reference to a pure function, but how do you stop a delegate from 
 accessing shared data?

Delegates are either marked as pure, or not. In the case above, foo() is not pure.
Mar 18 2011
parent reply Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 20:06:16 -0400, Don <nospam nospam.com> wrote:
 
 Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 14:35:27 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 04:34:54 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it 
 should be  able to be pure.  Even delegates which modify TLS 
 data should be able to  be pure (weak-pure, but still pure).


TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables.

This, for example, is a weak pure function: void foo(int *n) pure { *n = 5;} Because TLS variables are not shared, you should be able to do this: int x; void bar() { foo(&x); }

Yes, that compiles fine. But bar() is not pure.
  But you are right, there is a huge difference between a local 
 reference to TLS data and directly accessing TLS data -- the latter 
 can be obscured from the compiler, resulting in the compiler 
 thinking the function can be strong pure.
  So I don't know exactly how to mitigate this, but in my mind, it 
 feels like this should work:
  int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
  int x;
  void bar()
 {
    foo(x == 4, x = 5);
 }
  It seems not too different from the above example where you pass 
 the address of x.  But obviously the x = 5 delegate cannot be pure 
 (it modifies TLS data).
  We may have no recourse to get this to work.  It may be a lost 
 cause, and you just can't have lazy variables for pure functions.

It's not a lost cause, it's a two-liner! mtype.c line 5045: if (fparam->storageClass & STClazy) { - error(0, "cannot have lazy parameters to a pure function"); + tf->purity = PUREweak; + break; } This is a bit conservative: it would be possible to allow lazy parameters to be marked as pure, which would allow them to be strongly pure. But that would probably not be worth the extra complexity.

to a lazy parameter?

Yes.
  For example:
  shared int x;
  int foo()
 {
    return x;
 }
  int bar(lazy int n) pure
 {
    return n;
 }
  void baz()
 {
    bar(&foo);
 }
  or alternatively:
  void baz()
 {
    bar(x);
 }

This compiles just fine. (Well, you need to use bar(foo) not bar(&foo)). But if you try to mark baz() as pure, here's what you get: test0.d(135): Error: pure function 'baz' cannot call impure function 'foo'

But does this make sense? A pure function (bar) is reading a shared integer via foo, I thought that was a big no-no?
 or for the second case:

 test0.d(136): Error: pure function 'baz' cannot access mutable static 
 data 'x'

 bar is just weakly pure.

But I wasn't saying baz is pure, I was saying bar is pure (probably should be more diverse in the names). But I'm concerned about a pure function being able to indirectly read/write shared data. Does this make sense? I guess a weak-pure function acts like a normal function when called from a normal-function. Is that why it's ok?

Yes.
 Will there 
 not be an expectation that a pure function will not read/write shared 
 data that will be broken (i.e. why did the compiler allow this, I 
 thought I was safe from this!)?

If you pass the address of global or shared data into a weakly pure function, it won't be pure. If you have a function with a strongly pure signature, global data can never be passed to it.
 So is the rule that if you pass a non-pure delegate into a pure function 
 it's automatically weak-pure? 

Yes.
 If so, does this not mean we need two 
 versions of pure functions that take delegates, one that takes a pure 
 delegate, and one that takes a non-pure one?  Otherwise, how do you know 
 what's strong and what's weak?  For example, a weak pure function is 
 strong when called from a strong-pure function, so you could say if the 
 calling function is pure, the call is strong-pure.  But wouldn't you 
 need separate generated code for the two cases?

Generally not. Basically, all weak-pure means, is that it's OK for a strongly pure function to call it. And that's pretty much all it means. If it's called from outside a strongly pure function, it's just a normal impure function. Now, there is some potential for a call to a weakly pure function to be considered as strongly pure, even when called from a normal function, if all the parameters are pure. But that's a secondary effect (and the compiler currently never does it): the real value of weakly pure is that it allows a dramatic increase in the number of strongly pure functions.
 I guess you can see from the number of question marks, I'm not sure 
 about this at all, either way :)  If you think it will work, then I 
 trust your judgment.
 
 -Steve

Mar 21 2011
parent Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:
 On Mon, 21 Mar 2011 10:08:56 -0400, Don <nospam nospam.com> wrote:
 
 Steven Schveighoffer wrote:

 Will there not be an expectation that a pure function will not 
 read/write shared data that will be broken (i.e. why did the compiler 
 allow this, I thought I was safe from this!)?

If you pass the address of global or shared data into a weakly pure function, it won't be pure. If you have a function with a strongly pure signature, global data can never be passed to it.

Just on this one point, TLS or __gshared references are indistinguishable from normal references, but shared references are not the same, the type system lets you know it's shared. I thought this was the main reason why we were allowed to create weak-pure functions, because d has this distinction. Are you allowed to declare a function like this? int foo(shared int *x) pure {return *x;}

No.
 
 I was under the impression that a pure function couldn't access shared 
 data -- period.  Is this not true?  

Not necessarily. It only guarantees that it won't access shared data unless you provided it. Clearly, this theoretically
 weak-pure function could not be called from a strong-pure function, so 
 the pure decoration is useless.
 
 But this is very similar to a delegate that does the same thing.
 
 Why does one make sense and the other not?  In other words, if I have a 
 function like this:
 
 int foo(int delegate() x) pure {...}
 
 is this *ever* callable from a strong-pure function?  Or does the 
 delegate have to be declared pure?  It seems to me that either:
 
 1) it's not ever callable from a strong-pure function, making the pure 
 decoration useless or

It will only be callable from a strong-pure function if (a) the delegate is marked as pure; or (b) it's a delegate literal (so its purity can be checked). Lazy is an instance of (b).
 2) it's callable from a strong pure function, but then the compiler 
 needs to generate two copies of the function, one with a pure delegate, 
 one without.

If the delegate isn't pure, it cannot be called at all from a strongly pure function. Pure functions (even weakly pure) cannot call non-pure delegates in any way -- except in the aforementioned delegate literal case.
 
 On item 2, the reason I feel this way is in the case where foo wants to 
 pass the delegate to another pure function, it might optimize that call 
 differently if the delegate is known to be pure.  Or maybe we 
 don't/can't care...
 
 Thanks for having patience, I'm asking all these questions because I 
 want to make sure we do not put in rules that are wrong but hard to 
 remove later when everything uses them -- even though I don't understand 
 everything going on here ;)
 
 -Steve

Mar 22 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 18 Mar 2011 14:35:27 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 04:34:54 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it  
 should be  able to be pure.  Even delegates which modify TLS data  
 should be able to  be pure (weak-pure, but still pure).


TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables.

This, for example, is a weak pure function: void foo(int *n) pure { *n = 5;} Because TLS variables are not shared, you should be able to do this: int x; void bar() { foo(&x); }

Yes, that compiles fine. But bar() is not pure.
  But you are right, there is a huge difference between a local  
 reference to TLS data and directly accessing TLS data -- the latter can  
 be obscured from the compiler, resulting in the compiler thinking the  
 function can be strong pure.
  So I don't know exactly how to mitigate this, but in my mind, it feels  
 like this should work:
  int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
  int x;
  void bar()
 {
    foo(x == 4, x = 5);
 }
  It seems not too different from the above example where you pass the  
 address of x.  But obviously the x = 5 delegate cannot be pure (it  
 modifies TLS data).
  We may have no recourse to get this to work.  It may be a lost cause,  
 and you just can't have lazy variables for pure functions.

It's not a lost cause, it's a two-liner! mtype.c line 5045: if (fparam->storageClass & STClazy) { - error(0, "cannot have lazy parameters to a pure function"); + tf->purity = PUREweak; + break; } This is a bit conservative: it would be possible to allow lazy parameters to be marked as pure, which would allow them to be strongly pure. But that would probably not be worth the extra complexity.

I'm not sure this works. Aren't you allowed to pass in a delegate to a lazy parameter? For example: shared int x; int foo() { return x; } int bar(lazy int n) pure { return n; } void baz() { bar(&foo); } or alternatively: void baz() { bar(x); } The no-shared-data rule prevents you from passing in a shared int reference to a pure function, but how do you stop a delegate from accessing shared data? -Steve
Mar 18 2011
prev sibling next sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
On Fri, 18 Mar 2011 11:02:13 +0100, Kagamin <spam here.lot> wrote:

 Don Wrote:

 Pure functions calling weakly pure functions are also weakly pure and  

I presume you mean "Pure functions calling weakly pure functions *would also be* weakly pure and so on." ?

What's the difference?

Your original statement seems to indicate that strongly pure functions would be reduced to weakly pure if they call weakly pure functions. -- Simen
Mar 18 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 18 Mar 2011 20:06:16 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 14:35:27 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer wrote:
 On Fri, 18 Mar 2011 04:34:54 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it  
 should be  able to be pure.  Even delegates which modify TLS data  
 should be able to  be pure (weak-pure, but still pure).


TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables.

This, for example, is a weak pure function: void foo(int *n) pure { *n = 5;} Because TLS variables are not shared, you should be able to do this: int x; void bar() { foo(&x); }

Yes, that compiles fine. But bar() is not pure.
  But you are right, there is a huge difference between a local  
 reference to TLS data and directly accessing TLS data -- the latter  
 can be obscured from the compiler, resulting in the compiler thinking  
 the function can be strong pure.
  So I don't know exactly how to mitigate this, but in my mind, it  
 feels like this should work:
  int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
  int x;
  void bar()
 {
    foo(x == 4, x = 5);
 }
  It seems not too different from the above example where you pass the  
 address of x.  But obviously the x = 5 delegate cannot be pure (it  
 modifies TLS data).
  We may have no recourse to get this to work.  It may be a lost  
 cause, and you just can't have lazy variables for pure functions.

It's not a lost cause, it's a two-liner! mtype.c line 5045: if (fparam->storageClass & STClazy) { - error(0, "cannot have lazy parameters to a pure function"); + tf->purity = PUREweak; + break; } This is a bit conservative: it would be possible to allow lazy parameters to be marked as pure, which would allow them to be strongly pure. But that would probably not be worth the extra complexity.

a lazy parameter?

Yes.
  For example:
  shared int x;
  int foo()
 {
    return x;
 }
  int bar(lazy int n) pure
 {
    return n;
 }
  void baz()
 {
    bar(&foo);
 }
  or alternatively:
  void baz()
 {
    bar(x);
 }

This compiles just fine. (Well, you need to use bar(foo) not bar(&foo)). But if you try to mark baz() as pure, here's what you get: test0.d(135): Error: pure function 'baz' cannot call impure function 'foo'

But does this make sense? A pure function (bar) is reading a shared integer via foo, I thought that was a big no-no?
 or for the second case:

 test0.d(136): Error: pure function 'baz' cannot access mutable static  
 data 'x'

 bar is just weakly pure.

But I wasn't saying baz is pure, I was saying bar is pure (probably should be more diverse in the names). But I'm concerned about a pure function being able to indirectly read/write shared data. Does this make sense? I guess a weak-pure function acts like a normal function when called from a normal-function. Is that why it's ok? Will there not be an expectation that a pure function will not read/write shared data that will be broken (i.e. why did the compiler allow this, I thought I was safe from this!)? So is the rule that if you pass a non-pure delegate into a pure function it's automatically weak-pure? If so, does this not mean we need two versions of pure functions that take delegates, one that takes a pure delegate, and one that takes a non-pure one? Otherwise, how do you know what's strong and what's weak? For example, a weak pure function is strong when called from a strong-pure function, so you could say if the calling function is pure, the call is strong-pure. But wouldn't you need separate generated code for the two cases? I guess you can see from the number of question marks, I'm not sure about this at all, either way :) If you think it will work, then I trust your judgment. -Steve
Mar 21 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 21 Mar 2011 10:08:56 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer wrote:

 Will there not be an expectation that a pure function will not  
 read/write shared data that will be broken (i.e. why did the compiler  
 allow this, I thought I was safe from this!)?

If you pass the address of global or shared data into a weakly pure function, it won't be pure. If you have a function with a strongly pure signature, global data can never be passed to it.

Just on this one point, TLS or __gshared references are indistinguishable from normal references, but shared references are not the same, the type system lets you know it's shared. I thought this was the main reason why we were allowed to create weak-pure functions, because d has this distinction. Are you allowed to declare a function like this? int foo(shared int *x) pure {return *x;} I was under the impression that a pure function couldn't access shared data -- period. Is this not true? Clearly, this theoretically weak-pure function could not be called from a strong-pure function, so the pure decoration is useless. But this is very similar to a delegate that does the same thing. Why does one make sense and the other not? In other words, if I have a function like this: int foo(int delegate() x) pure {...} is this *ever* callable from a strong-pure function? Or does the delegate have to be declared pure? It seems to me that either: 1) it's not ever callable from a strong-pure function, making the pure decoration useless or 2) it's callable from a strong pure function, but then the compiler needs to generate two copies of the function, one with a pure delegate, one without. On item 2, the reason I feel this way is in the case where foo wants to pass the delegate to another pure function, it might optimize that call differently if the delegate is known to be pure. Or maybe we don't/can't care... Thanks for having patience, I'm asking all these questions because I want to make sure we do not put in rules that are wrong but hard to remove later when everything uses them -- even though I don't understand everything going on here ;) -Steve
Mar 21 2011
prev sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
On Mon, 21 Mar 2011 15:35:58 +0100, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 Why does one make sense and the other not?  In other words, if I have a  
 function like this:

 int foo(int delegate() x) pure {...}

 is this *ever* callable from a strong-pure function?  Or does the  
 delegate have to be declared pure?  It seems to me that either:

 1) it's not ever callable from a strong-pure function, making the pure  
 decoration useless or
 2) it's callable from a strong pure function, but then the compiler  
 needs to generate two copies of the function, one with a pure delegate,  
 one without.

 On item 2, the reason I feel this way is in the case where foo wants to  
 pass the delegate to another pure function, it might optimize that call  
 differently if the delegate is known to be pure.  Or maybe we  
 don't/can't care...

That is a good point. However, pure delegates should (but might not currently) be implicitly castable to impure. Hence, one version, taking an impure delegate, should be enough. Moreover, I believe a pure delegate could only ever be weakly-pure, hence precluding the use of aggressive optimizations. If my first assumption is correct, and the latter is not, the problem should only occur if you have two versions of a function, one taking a pure delegate, the other taking an impure one. -- Simen
Mar 21 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 18 Mar 2011 03:50:17 -0400, Kagamin <spam here.lot> wrote:

 Steven Schveighoffer Wrote:

 As long as the delegate does not access shared/global data, it should be
 able to be pure.  Even delegates which modify TLS data should be able to
 be pure (weak-pure, but still pure).

Pure functions calling weakly pure functions are also weakly pure and so on. This effectively leaves you without purity.

No. Strong-pure functions can call weak-pure functions and still can be strong-pure. That's the huge benefit of weak-pure functions -- you can modularize pure functions without having to change everything to immutable. -Steve
Mar 18 2011