www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Uh... destructors?

reply %u <wfunction hotmail.com> writes:
Hi,

I'm just curious... why is saying something like this:

extern(C)
    private static const pure override final synchronized ~this() { }

allowed?

Thanks!
Feb 21 2011
next sibling parent dsimcha <dsimcha yahoo.com> writes:
On 2/21/2011 11:46 PM, %u wrote:

 Hi,

 I'm just curious... why is saying something like this:

 extern(C)
      private static const pure override final synchronized ~this() { }

 allowed?

 Thanks!


...And when's the last time you needed one?
Feb 21 2011
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 21 February 2011 20:46:56 %u wrote:

 Hi,
 
 I'm just curious... why is saying something like this:
 
 extern(C)
     private static const pure override final synchronized ~this() { }
 
 allowed?


dmd is pretty lax about attributes which don't apply. It generally just ignores 
them. Personally, I think that it should error on invalid attributes, but for 
some reason, that's not how it works. Of course, there could be other bugs in 
play here, but there's every possibility that the end result is completely 
valid.

- Jonathan M Davis
Feb 21 2011
next sibling parent reply %u <wfunction hotmail.com> writes:
 dmd is pretty lax about attributes which don't apply. It generally just
ignores them. Personally,


I think that it should error on invalid attributes, but for some reason, that's
not how it works.
Of course, there could be other bugs in play here, but there's every
possibility that the end
result is completely valid.

Well, the trouble is, pretty much all of these are invalid attributes:
- const and pure make no sense, since destructors (should) change an object's
state
- override and final make no sense, since destructors obviously aren't ever
overridden... they're
always called after the subclass's destructor is called
- static obviously makes no sense
- synchronized is meaningless since there's only one thread ever running the
destructor anyway
- private makes no sense since (unless we're trying to imitate C++ here)
destructors are only
called from the runtime, and nowhere else.
- The only meaningful attribute there is extern(C).

I would agree that DMD should ignore the attributes that are redundant or
optional (e.g. it should
be okay for "static" to be written and/or omitted at the module-level, and DMD
should ignore it)
but I don't see why _wrong_ attributes should be ignored... it confuses the
programmer, opens the
potential for error, and doesn't have any benefits. (The only exception I can
think of to this rule
would be attributes that cannot be removed, like saying "private:" in the
beginning... for general
attributes like those, I guess DMD can ignore them, but for specifically
written attributes like
these, is there any benefit whatsoever to allowing them?)

Thanks!
Feb 21 2011
next sibling parent reply Simen Kjaeraas <simen.kjaras gmail.com> writes:
%u Wrote:

 Well, the trouble is, pretty much all of these are invalid attributes:



 - static obviously makes no sense


And here is where you're wrong. You have defined a static destructor, which is
called with module destructor as the program goes out of scope, rather than
when your struct or class is destroyed.
Feb 22 2011
next sibling parent reply %u <wfunction hotmail.com> writes:
 Well, the trouble is, pretty much all of these are invalid




attributes:

 - static obviously makes no sense


 And here is where you're wrong. You have defined a static


destructor, which is called with module destructor as the program
goes out of scope, rather than when your struct or class is
destroyed.

Oops... I knew that, but I totally forgot about it when I wrote
this; good catch.

So okay, fine... 2 out of about 8. That still doesn't mean the rest
of them should be allowed, though... think about how confusing code
with a "pure" destructor would be.
Feb 22 2011
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-02-22 13:15:03 -0500, %u <wfunction hotmail.com> said:


 So okay, fine... 2 out of about 8. That still doesn't mean the rest
 of them should be allowed, though... think about how confusing code
 with a "pure" destructor would be.


What's the problem with a pure destructor? It only means you can't 
access global variables. If the object holds a pointer to somewhere, 
you can still affect that somewhere.

In fact, if your struct's destructor isn't pure, how can you use it as 
a local variable inside of a pure function?

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Feb 22 2011
parent reply %u <wfunction hotmail.com> writes:
 What's the problem with a pure destructor? It only means you can't access
global variables.


If the object holds a pointer to somewhere, you can still affect that somewhere.


 In fact, if your struct's destructor isn't pure, how can you use it as local
variable inside


of a pure function?

The problem is that a pure destructor makes no sense because "pure" means that
a function has
no side effects. So anything that a pure function does would need to be
strictly reflected only
in its output, and in nothing else... and yet, a destructor has no output (and
no input
either), so it's logically not allowed to perform any action with side
effects... in other
words, it can't do _anything_, just like in functional programming.

Unless I'm misunderstanding the meaning of "pure", I don't see how a destructor
marked as
"pure" would be meaningful. (The same reason applies to const.)
Feb 22 2011
parent reply %u <wfunction hotmail.com> writes:
I just visited Wikipedia (savior of the day) and a quick look at
this article:
http://en.wikipedia.org/wiki/Pure_function

yields the following requirements for a pure function:

1. The function always evaluates the same result value given the
same argument value(s). The function result value cannot depend on
any hidden information or state that may change as program execution
proceeds or between different executions of the program, nor can it
depend on any external input from I/O devices.

2. Evaluation of the result does not cause any semantically
observable side effect or output, such as mutation of mutable
objects or output to I/O devices.

"pure" destructors always fail the first test, because they clearly
cause a state change for the current object, if not for a global
resource like memory. Pure functions need to be timeless and can be
freely reordered; a destructor call, however, is not timeless -- it
depends on the current state of the object, and the changes it makes
are visible to the outside world.

They also usually fail the second test, because if, for example, we
flush a file inside a destructor, that clearly has an observable
output... so I can't see why a destructor would ever be pure.
Feb 22 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 22 Feb 2011 14:31:30 -0500, %u <wfunction hotmail.com> wrote:


 I just visited Wikipedia (savior of the day) and a quick look at
 this article:
 http://en.wikipedia.org/wiki/Pure_function

 yields the following requirements for a pure function:

 1. The function always evaluates the same result value given the
 same argument value(s). The function result value cannot depend on
 any hidden information or state that may change as program execution
 proceeds or between different executions of the program, nor can it
 depend on any external input from I/O devices.

 2. Evaluation of the result does not cause any semantically
 observable side effect or output, such as mutation of mutable
 objects or output to I/O devices.

 "pure" destructors always fail the first test, because they clearly
 cause a state change for the current object, if not for a global
 resource like memory. Pure functions need to be timeless and can be
 freely reordered; a destructor call, however, is not timeless -- it
 depends on the current state of the object, and the changes it makes
 are visible to the outside world.

 They also usually fail the second test, because if, for example, we
 flush a file inside a destructor, that clearly has an observable
 output... so I can't see why a destructor would ever be pure.


D pure functions are significantly different than this definition (as of  
recent times, when weak-pure was added).

Essentially, a pure function cannot access global variables.  However, it  
can access variables referred to via a member of the object instance.

i.e. this is a valid pure function:

class C
{
   int x;
   pure void foo() { x++; }
}

-Steve
Feb 22 2011
parent reply %u <wfunction hotmail.com> writes:
 D pure functions are significantly different than this definition


(as of recent times, when weak-pure was added).

 Essentially, a pure function cannot access global variables.


However, it can access variables referred to via a member of the
object instance.

 i.e. this is a valid pure function:


class C
{
   int x;
   pure void foo() { x++; }
}

I... did not know that. But even in that case, pure wouldn't make much
sense, because doing anything like freeing memory or closing a file
handle affects global variables (whether directly in the runtime or
indirectly in the OS)... right?
Feb 22 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 22 Feb 2011 15:48:42 -0500, %u <wfunction hotmail.com> wrote:


 D pure functions are significantly different than this definition


 (as of recent times, when weak-pure was added).

 Essentially, a pure function cannot access global variables.


 However, it can access variables referred to via a member of the
 object instance.

 i.e. this is a valid pure function:


 class C
 {
    int x;
    pure void foo() { x++; }
 }

 I... did not know that. But even in that case, pure wouldn't make much
 sense, because doing anything like freeing memory or closing a file
 handle affects global variables (whether directly in the runtime or
 indirectly in the OS)... right?


Freeing and allocating memory is fair game for pure functions.  It is one  
of the only exceptions to the rule, because without the ability to  
allocate and free memory, functional programming is quite limited.

On closing a file, you couldn't close that file unless the function to  
close it was marked pure.  I would *hope* that the C call to close a file  
was not marked as pure.

I wasn't advocating that destructors can be pure, I was just pointing out  
that the concept of pure has changed in recent times.

-Steve
Feb 22 2011
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 Freeing and allocating memory is fair game for pure functions.


Allocating arrays and objects is possible in pure D functions, despite the
memory pointers they contain (like the ptr of a returned array) are different
across different calls. This makes those function only formally pure and
requires care from both the programmer and the optimizations done by the
compiler, to avoid some bad bugs :-)

Example: the C calloc() function is not considered pure, despite what it does
is not so far from a pure D function that allocates and returns a dynamic array
of ubytes:

import core.stdc.stdlib: calloc;

// OK
pure ubyte* foo1(int n) {
    auto a = new ubyte[n];
    return a.ptr;
}

// Error: pure function 'foo2' cannot call impure function 'calloc'
pure ubyte* foo2(int n) {
    return cast(ubyte*)calloc(n, 1);
}

void main() {}


Regarding freeing memory in pure functions, I am even less sure.
 


 On closing a file, you couldn't close that file unless the function to  
 close it was marked pure.  I would *hope* that the C call to close a file  
 was not marked as pure.


Changing the state of a file is definitively a not pure operation, even for the
quite relaxed standards of purity of D :-)

Bye,
bearophile
Feb 22 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 22 Feb 2011 16:34:21 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:


 Freeing and allocating memory is fair game for pure functions.


 Allocating arrays and objects is possible in pure D functions, despite  
 the memory pointers they contain (like the ptr of a returned array) are  
 different across different calls. This makes those function only  
 formally pure and requires care from both the programmer and the  
 optimizations done by the compiler, to avoid some bad bugs :-)

 Example: the C calloc() function is not considered pure, despite what it  
 does is not so far from a pure D function that allocates and returns a  
 dynamic array of ubytes:

 import core.stdc.stdlib: calloc;

 // OK
 pure ubyte* foo1(int n) {
     auto a = new ubyte[n];
     return a.ptr;
 }

 // Error: pure function 'foo2' cannot call impure function 'calloc'
 pure ubyte* foo2(int n) {
     return cast(ubyte*)calloc(n, 1);
 }

 void main() {}


I would think malloc and friends should be pure, as well as free.  They  
can easily simply be marked pure, since they are C bindings.


 Regarding freeing memory in pure functions, I am even less sure.


If allocation is pure, freeing should also be pure.  You should be allowed  
to clean up what you created in a pure function.

To draw a parallel, a GC collect cycle should be "pure", even though it  
affects global state.  Otherwise, the GC would have to be disabled during  
pure functions.

-Steve
Feb 22 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 I would think malloc and friends should be pure, as well as free.  They  
 can easily simply be marked pure, since they are C bindings.


D even accepts strongly pure functions like:

pure size_t foo() { 
    auto a = new ubyte[1];
    return cast(size_t)a.ptr;
}

For practical purposes D allows pure functions to return dynamic arrays,
objects and structs allocated inside them. This is a significant hole in the D
idea of purity. I think marking malloc as pure makes the situation worse :-)

This is an idea to patch that hole a little, doing this inside pure functions:
1) Keep disallowing alloca()/malloc()/etc calls;
2) Disallow struct allocations;
3) Keep allowing object and dynamic array allocations;
4) Disallow read and write of the "ptr" fields of dynamic arrays;
5) Disallow casts of object references to something else.

This reduces some flexibility of pure functions, it doesn't fully patch that
hole, and makes language rules a more complex.

The idea under those ideas is that objects and arrays are usually interesting
for the data/semantics they contain, and not for the value of their pointer. If
you ignore the value of their pointer, then the hole vanishes. You can't full
close this hole, but those ideas help avoid the more blatantly impure semantics
inside/from pure functions.

I'd like to patch that hole, but it can't be fully patched. So maybe all this
is useless, or worse it makes the language more rigid and not more safe, so
it's probably better to not adopt those ideas.

Bye,
bearophile
Feb 22 2011
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 22 Feb 2011 18:19:15 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:


 I would think malloc and friends should be pure, as well as free.  They
 can easily simply be marked pure, since they are C bindings.


 D even accepts strongly pure functions like:

 pure size_t foo() {
     auto a = new ubyte[1];
     return cast(size_t)a.ptr;
 }


cast voids all warranties ;)


 For practical purposes D allows pure functions to return dynamic arrays,  
 objects and structs allocated inside them. This is a significant hole in  
 the D idea of purity. I think marking malloc as pure makes the situation  
 worse :-)


Memory allocation has to be allowed inside pure functions.  Otherwise,  
pure functions are too strict and limited.

Allowing malloc is somewhat exceptional because you then must allow free.   
But I see no reason (yet) to disallow free.  If free cannot be pure, then  
malloc cannot be pure.

Note, the 'hole' you refer to is not a bad thing.  A weakly pure function  
allows one to modularize pure functions, whereas prior to this, things  
like sort could not be pure functions, and therefore could not be used  
inside pure functions.  I think the hole allows pure functions to be  
actually usable and easy whereas before they were much too obscure to be  
useful in much code.


 The idea under those ideas is that objects and arrays are usually  
 interesting for the data/semantics they contain, and not for the value  
 of their pointer. If you ignore the value of their pointer, then the  
 hole vanishes. You can't full close this hole, but those ideas help  
 avoid the more blatantly impure semantics inside/from pure functions.


All that is required is to disallow casting.  But casting can allow more  
flexibility, and might actually be necessary in some cases (sometimes, in  
low level languages, the developer knows better than the compiler what  
should be allowed).  A  safe pure function should be what you desire.


 I'd like to patch that hole, but it can't be fully patched. So maybe all  
 this is useless, or worse it makes the language more rigid and not more  
 safe, so it's probably better to not adopt those ideas.


Those rules do not enhance the experience.  The only one which makes sense  
is to disallow casting, but we already have a construct for that:  safe.

-Steve
Feb 23 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/11 3:14 PM, Steven Schveighoffer wrote:

 On Tue, 22 Feb 2011 15:48:42 -0500, %u <wfunction hotmail.com> wrote:


 D pure functions are significantly different than this definition


 (as of recent times, when weak-pure was added).

 Essentially, a pure function cannot access global variables.


 However, it can access variables referred to via a member of the
 object instance.

 i.e. this is a valid pure function:


 class C
 {
 int x;
 pure void foo() { x++; }
 }

 I... did not know that. But even in that case, pure wouldn't make much
 sense, because doing anything like freeing memory or closing a file
 handle affects global variables (whether directly in the runtime or
 indirectly in the OS)... right?


 Freeing and allocating memory is fair game for pure functions.


I don't think freeing memory is pure.

Andrei
Feb 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 08:04:49 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 2/22/11 3:14 PM, Steven Schveighoffer wrote:

 On Tue, 22 Feb 2011 15:48:42 -0500, %u <wfunction hotmail.com> wrote:


 D pure functions are significantly different than this definition


 (as of recent times, when weak-pure was added).

 Essentially, a pure function cannot access global variables.


 However, it can access variables referred to via a member of the
 object instance.

 i.e. this is a valid pure function:


 class C
 {
 int x;
 pure void foo() { x++; }
 }

 I... did not know that. But even in that case, pure wouldn't make much
 sense, because doing anything like freeing memory or closing a file
 handle affects global variables (whether directly in the runtime or
 indirectly in the OS)... right?


 Freeing and allocating memory is fair game for pure functions.


 I don't think freeing memory is pure.


Why not?  If it shouldn't be allowed, it should be easy to show with an  
example of why.

-Steve
Feb 23 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/11 9:00 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 08:04:49 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/22/11 3:14 PM, Steven Schveighoffer wrote:

 On Tue, 22 Feb 2011 15:48:42 -0500, %u <wfunction hotmail.com> wrote:


 D pure functions are significantly different than this definition


 (as of recent times, when weak-pure was added).

 Essentially, a pure function cannot access global variables.


 However, it can access variables referred to via a member of the
 object instance.

 i.e. this is a valid pure function:


 class C
 {
 int x;
 pure void foo() { x++; }
 }

 I... did not know that. But even in that case, pure wouldn't make much
 sense, because doing anything like freeing memory or closing a file
 handle affects global variables (whether directly in the runtime or
 indirectly in the OS)... right?


 Freeing and allocating memory is fair game for pure functions.


 I don't think freeing memory is pure.


 Why not? If it shouldn't be allowed, it should be easy to show with an
 example of why.

 -Steve


free(p) affects data remotely outside the pure function.

Andrei
Feb 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 10:35:26 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 9:00 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 08:04:49 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/22/11 3:14 PM, Steven Schveighoffer wrote:

 On Tue, 22 Feb 2011 15:48:42 -0500, %u <wfunction hotmail.com> wrote:


 D pure functions are significantly different than this definition


 (as of recent times, when weak-pure was added).

 Essentially, a pure function cannot access global variables.


 However, it can access variables referred to via a member of the
 object instance.

 i.e. this is a valid pure function:


 class C
 {
 int x;
 pure void foo() { x++; }
 }

 I... did not know that. But even in that case, pure wouldn't make  
 much
 sense, because doing anything like freeing memory or closing a file
 handle affects global variables (whether directly in the runtime or
 indirectly in the OS)... right?


 Freeing and allocating memory is fair game for pure functions.


 I don't think freeing memory is pure.


 Why not? If it shouldn't be allowed, it should be easy to show with an
 example of why.

 -Steve


 free(p) affects data remotely outside the pure function.


This is allowed however in the new pure regime:

pure void foo(int *x) {(*x)++;}

int x;
foo(&x);

Or do you mean something else?

-Steve
Feb 23 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/11 9:42 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 10:35:26 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 free(p) affects data remotely outside the pure function.


 This is allowed however in the new pure regime:

 pure void foo(int *x) {(*x)++;}

 int x;
 foo(&x);

 Or do you mean something else?

 -Steve


At a level it's clear to me that a function cannot be at the same time 
pure and unsafe. For example:

pure void foo(int *x) { free(x); (*x)++; }

This function essentially breaks any guarantee for the entire program, 
so it would be quite difficult to claim it's pure.


Andrei
Feb 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 10:46:43 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 At a level it's clear to me that a function cannot be at the same time  
 pure and unsafe. For example:

 pure void foo(int *x) { free(x); (*x)++; }

 This function essentially breaks any guarantee for the entire program,  
 so it would be quite difficult to claim it's pure.


I thought  safe was orthogonal to pure?  If this isn't the case, then yes,  
free must be disallowed.  But then malloc must also be, and any construct  
which manages its own memory via malloc/free.

 From what I remember, pure functions:

1. cannot access shared or global non-immutable data
2. cannot call non-pure functions

I don't remember the definition that pure functions must also be  safe.

-Steve
Feb 23 2011
next sibling parent reply %u <wfunction hotmail.com> writes:
 I thought  safe was orthogonal to pure?  If this isn't the case,


then yes, free must be disallowed.  But then malloc must also be,
and any construct which manages its own memory via malloc/free.

 From what I remember, pure functions:
 1. cannot access shared or global non-immutable data
 2. cannot call non-pure functions
 I don't remember the definition that pure functions must also be


 safe.

 -Steve


It seems that you're using the word "pure" as a synonym for the
noalias and/or restrict __declspec keywords. However, they're
different words because they have different meanings. :) If you
really mean noalias, then I think we just just call it that and
introduce the attribute  noalias? Otherwise, it's a misuse of the
term "pure", because D doesn't seem to be really using the term for
its correct purpose.
Feb 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 11:41:52 -0500, %u <wfunction hotmail.com> wrote:


 I thought  safe was orthogonal to pure?  If this isn't the case,


 then yes, free must be disallowed.  But then malloc must also be,
 and any construct which manages its own memory via malloc/free.

 From what I remember, pure functions:
 1. cannot access shared or global non-immutable data
 2. cannot call non-pure functions
 I don't remember the definition that pure functions must also be


  safe.

 -Steve


 It seems that you're using the word "pure" as a synonym for the
 noalias and/or restrict __declspec keywords. However, they're
 different words because they have different meanings. :) If you
 really mean noalias, then I think we just just call it that and
 introduce the attribute  noalias? Otherwise, it's a misuse of the
 term "pure", because D doesn't seem to be really using the term for
 its correct purpose.


Don't know what those keywords are, but pure functions in the academic  
sense are discernible from weak-pure functions via the parameters and  
return values.

A strong pure (one which is able to be optimized/factored) function has  
one additional requirement that it's parameters and return value must all  
be immutable or implicitly convertible to immutable.

To add another keyword to specify this would be wasteful.

Yes, it breaks from tradition, but D's pure functions are far from  
traditional anyways.

-Steve
Feb 23 2011
parent reply %u <wfunction hotmail.com> writes:
 It seems that you're using the word "pure" as a synonym for the noalias and/or
restrict __declspec keywords. However, they're different words because they have




different meanings. :) If you really mean noalias, then I think we just just
call it that and introduce the attribute  noalias? Otherwise, it's a misuse of
the term
"pure", because D doesn't seem to be really using the term for its correct
purpose.


 Don't know what those keywords are, but pure functions in the academic sense
are discernible from weak-pure functions via the parameters and return values.


They're Microsoft-specific keywords. From MSDN:
"noalias means that a function call does not modify or reference visible global
state and only modifies the memory pointed to directly by pointer parameters
(first-level
indirections).
"If a function is annotated as noalias, the optimizer can assume that, in
addition to the parameters themselves, only first-level indirections of pointer
parameters are
referenced or modified inside the function. The visible global state is the set
of all data that is not defined or referenced outside of the compilation scope,
and their
address is not taken."


This is _clearly_ what D is calling "pure", and because of the fact that "pure"
is such a misnomer and that noalias is (almost?) clearly what we mean, I
suggest we use the
latter term instead... does this sound like a good idea?
Feb 23 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, February 23, 2011 09:17:15 %u wrote:

 It seems that you're using the word "pure" as a synonym for the noalias
 and/or restrict __declspec keywords. However, they're different words
 because they have




 
 different meanings. :) If you really mean noalias, then I think we just
 just call it that and introduce the attribute  noalias? Otherwise, it's a
 misuse of the term "pure", because D doesn't seem to be really using the
 term for its correct purpose.
 

 Don't know what those keywords are, but pure functions in the academic
 sense are discernible from weak-pure functions via the parameters and
 return values.


 
 They're Microsoft-specific keywords. From MSDN:
 "noalias means that a function call does not modify or reference visible
 global state and only modifies the memory pointed to directly by pointer
 parameters (first-level indirections).
 "If a function is annotated as noalias, the optimizer can assume that, in
 addition to the parameters themselves, only first-level indirections of
 pointer parameters are referenced or modified inside the function. The
 visible global state is the set of all data that is not defined or
 referenced outside of the compilation scope, and their address is not
 taken."
 
 
 This is _clearly_ what D is calling "pure", and because of the fact that
 "pure" is such a misnomer and that noalias is (almost?) clearly what we
 mean, I suggest we use the latter term instead... does this sound like a
 good idea?


No. pure is what we want. Changing it would break code and contradict TDPL 
(though the addition of weakly pure isn't in TDPL). Strongly pure functions are 
essentially what you'd expect from pure. Weakly pure functions aren't, but 
they're necessary to make pure very useful, and there's no real benefit in
using 
a new keyword or attribute to mark them. pure is the word used because we're 
modeling the idea that multiple calls to the same function with the same 
parameters have the same result - which is indeed the case with strongly pure 
functions. The funny case is weakly pure, but it's _not_ optimized out, and
it's 
necessary for pure to be useful. You can clearly see whether a function is 
weakly or strongly pure by looking at its signature, so I really don't think 
that it's an issue.

- Jonathan M Davis
Feb 23 2011
parent reply %u <wfunction hotmail.com> writes:
 No. pure is what we want. Changing it would break code and contradict TDPL
(though the


addition of weakly pure isn't in TDPL). Strongly pure functions are essentially
what you'd
expect from pure. Weakly pure functions aren't, but they're necessary to make
pure very
useful, and there's no real benefit in using a new keyword or attribute to mark
them. pure is
the word used because we're modeling the idea that multiple calls to the same
function with
the same parameters have the same result - which is indeed the case with
strongly pure
functions. The funny case is weakly pure, but it's _not_ optimized out, and
it's necessary for
pure to be useful. You can clearly see whether a function is weakly or strongly
pure by
looking at its signature, so I really don't think that it's an issue.


You keep on mentioning that weakly pure "is necessary for pure to be useful",
but the problem
is, even if something looks like a duck and walks like a duck, that doesn't
mean it's a duck!
Just because __declspec(noalias) _appears_ to mean the same thing as pure from
the outside
perspective that doesn't bother to peek too much into things doesn't mean that
it's the same
thing (it obviously isn't).
You're mentioning yourself that "weakly pure" isn't in the language specs...
and if DMD is
supposed to be a D compiler, shouldn't it actually follow the exact spec? (If
"weakly pure"
was actually intentional, then shouldn't it actually _be_ in the spec?
Otherwise, the compiler
is clearly contradicting TDPL about a fundamental CS concept, isn't it?)
Feb 23 2011
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 14:28:01 -0500, %u <wfunction hotmail.com> wrote:


 No. pure is what we want. Changing it would break code and contradict  
 TDPL (though the


 addition of weakly pure isn't in TDPL). Strongly pure functions are  
 essentially what you'd
 expect from pure. Weakly pure functions aren't, but they're necessary to  
 make pure very
 useful, and there's no real benefit in using a new keyword or attribute  
 to mark them. pure is
 the word used because we're modeling the idea that multiple calls to the  
 same function with
 the same parameters have the same result - which is indeed the case with  
 strongly pure
 functions. The funny case is weakly pure, but it's _not_ optimized out,  
 and it's necessary for
 pure to be useful. You can clearly see whether a function is weakly or  
 strongly pure by
 looking at its signature, so I really don't think that it's an issue.

 You keep on mentioning that weakly pure "is necessary for pure to be  
 useful", but the problem
 is, even if something looks like a duck and walks like a duck, that  
 doesn't mean it's a duck!
 Just because __declspec(noalias) _appears_ to mean the same thing as  
 pure from the outside
 perspective that doesn't bother to peek too much into things doesn't  
 mean that it's the same
 thing (it obviously isn't).


Jonathan's point is that pure is what we say it is, because that's the  
definition of the language.  If you want Microsoft's version of pure, use  
Microsoft's language/compiler.  D's definition of pure is what we said it  
is.

Note that weak pure does fit within the definition of "having no side  
effects."  That is, it cannot alter things it is not passed via parameters.


 You're mentioning yourself that "weakly pure" isn't in the language  
 specs... and if DMD is
 supposed to be a D compiler, shouldn't it actually follow the exact  
 spec? (If "weakly pure"
 was actually intentional, then shouldn't it actually _be_ in the spec?  
 Otherwise, the compiler
 is clearly contradicting TDPL about a fundamental CS concept, isn't it?)


Strong pure and weak pure functions have no visible difference, and one  
can be used wherever the other is used.  The difference between strong and  
weak pure is only how the compiler deals with them.  So from a user  
perspective, strong and weak pure are the same, both are pure functions  
and have the same restrictions.

-Steve
Feb 23 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, February 23, 2011 11:28:01 %u wrote:

 No. pure is what we want. Changing it would break code and contradict
 TDPL (though the


 
 addition of weakly pure isn't in TDPL). Strongly pure functions are
 essentially what you'd expect from pure. Weakly pure functions aren't, but
 they're necessary to make pure very useful, and there's no real benefit in
 using a new keyword or attribute to mark them. pure is the word used
 because we're modeling the idea that multiple calls to the same function
 with the same parameters have the same result - which is indeed the case
 with strongly pure functions. The funny case is weakly pure, but it's
 _not_ optimized out, and it's necessary for pure to be useful. You can
 clearly see whether a function is weakly or strongly pure by looking at
 its signature, so I really don't think that it's an issue.
 
 
 You keep on mentioning that weakly pure "is necessary for pure to be
 useful", but the problem is, even if something looks like a duck and walks
 like a duck, that doesn't mean it's a duck! Just because
 __declspec(noalias) _appears_ to mean the same thing as pure from the
 outside perspective that doesn't bother to peek too much into things
 doesn't mean that it's the same thing (it obviously isn't).
 You're mentioning yourself that "weakly pure" isn't in the language
 specs... and if DMD is supposed to be a D compiler, shouldn't it actually
 follow the exact spec? (If "weakly pure" was actually intentional, then
 shouldn't it actually _be_ in the spec? Otherwise, the compiler is clearly
 contradicting TDPL about a fundamental CS concept, isn't it?)


I didn't say anything about the language spec. I said that weakly pure wasn't 
mentioned in TDPL, and I'd have to go read the section on pure in TDPL again to 
see how much of it would need to be changed (possibly very little) for it to be 
correct, even if it doesn't mention weakly pure. Weakly pure was added after 
TDPL was released, so it's not in there. It may very well be in the spec at
this 
point. I don't know where the spec is.

Weakly pure was added, because strongly pure functions (which is really all
pure 
was originally) quickly become useless, because so few functions can be
strongly 
pure, and pure functions can only call pure functions. So, pure was expanded
and 
the concept of weakly pure was added. It's still pure though. If you give the 
exact same arguments to a weakly pure function on multiple calls, then the 
results will be exactly the same. It's just that that includes changes to the 
function arguments. As such, those calls can't be optimized out. Strongly pure 
functions, however, do _not_ allow changes to their function arguments, so 
multiple calls to them _can_ be optimized out. All of that is up to the
compiler 
(though it's easy enough to tell whether a function is strongly or weakly pure).

You mark a function as pure because its result is the same every time for the 
same arguments. It's up to the compiler whether it can optimize it or not. And 
in cases where the result includes possible changes to the function's
arguments, 
the function is weakly pure and can't be optimized by the compiler. But that's 
arguably an implementation detail of the compiler.

pure works just fine, and it _is_ following the academic definition in the
sense 
that for the exact same arguments, you get the exact same results every time - 
it's just that the function's arguments are potentially part of that result.

- Jonathan M Davis
Feb 23 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/11 9:57 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 10:46:43 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 At a level it's clear to me that a function cannot be at the same time
 pure and unsafe. For example:

 pure void foo(int *x) { free(x); (*x)++; }

 This function essentially breaks any guarantee for the entire program,
 so it would be quite difficult to claim it's pure.


 I thought  safe was orthogonal to pure? If this isn't the case, then
 yes, free must be disallowed. But then malloc must also be, and any
 construct which manages its own memory via malloc/free.


malloc can be reasonably made pure. It would be a mistake to see malloc 
and free as two symmetrical parts of a whole. The truth of the matter is 
that allocating is fine, deallocating is problematic.


  From what I remember, pure functions:

 1. cannot access shared or global non-immutable data
 2. cannot call non-pure functions

 I don't remember the definition that pure functions must also be  safe.


Perhaps we should amend the definition.


Andrei
Feb 23 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 11:45:48 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 9:57 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 10:46:43 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 At a level it's clear to me that a function cannot be at the same time
 pure and unsafe. For example:

 pure void foo(int *x) { free(x); (*x)++; }

 This function essentially breaks any guarantee for the entire program,
 so it would be quite difficult to claim it's pure.


 I thought  safe was orthogonal to pure? If this isn't the case, then
 yes, free must be disallowed. But then malloc must also be, and any
 construct which manages its own memory via malloc/free.


 malloc can be reasonably made pure. It would be a mistake to see malloc  
 and free as two symmetrical parts of a whole. The truth of the matter is  
 that allocating is fine, deallocating is problematic.


So what happens to programs that just use malloc and don't use free?  I  
think you have to consider that if a pure function doesn't return its  
malloc'd memory, it will leak, and a leaking pure function is just as bad  
as an usafe one IMO.

D's malloc, however, does not require free to be callable since the memory  
will be cleaned up by the GC.


  From what I remember, pure functions:

 1. cannot access shared or global non-immutable data
 2. cannot call non-pure functions

 I don't remember the definition that pure functions must also be  safe.


 Perhaps we should amend the definition.


Why?  We have both attributes, why not just require " safe pure" if you  
want  safe pure functions?

-Steve
Feb 23 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/11 10:52 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 11:45:48 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 9:57 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 10:46:43 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 At a level it's clear to me that a function cannot be at the same time
 pure and unsafe. For example:

 pure void foo(int *x) { free(x); (*x)++; }

 This function essentially breaks any guarantee for the entire program,
 so it would be quite difficult to claim it's pure.


 I thought  safe was orthogonal to pure? If this isn't the case, then
 yes, free must be disallowed. But then malloc must also be, and any
 construct which manages its own memory via malloc/free.


 malloc can be reasonably made pure. It would be a mistake to see
 malloc and free as two symmetrical parts of a whole. The truth of the
 matter is that allocating is fine, deallocating is problematic.


 So what happens to programs that just use malloc and don't use free? I
 think you have to consider that if a pure function doesn't return its
 malloc'd memory, it will leak, and a leaking pure function is just as
 bad as an usafe one IMO.

 D's malloc, however, does not require free to be callable since the
 memory will be cleaned up by the GC.


Yah, that's why safety and purity go hand in hand with GC.


 From what I remember, pure functions:

 1. cannot access shared or global non-immutable data
 2. cannot call non-pure functions

 I don't remember the definition that pure functions must also be  safe.


 Perhaps we should amend the definition.


 Why? We have both attributes, why not just require " safe pure" if you
 want  safe pure functions?

 -Steve


Because a pure unsafe function is useless.


Andrei
Feb 23 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 12:01:15 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 10:52 AM, Steven Schveighoffer wrote:

 Why? We have both attributes, why not just require " safe pure" if you
 want  safe pure functions?


 Because a pure unsafe function is useless.


Just because a function is not marked  safe does not mean it is unsafe.   
It just means you can do things the compiler cannot verify are safe, but  
that you know are actually safe.  I showed you earlier an example of a  
safe pure function that uses malloc and free.

Programmers are allowed to make conceptually safe functions which are not  
marked as  safe, why not the same for pure functions?

-Steve
Feb 23 2011
next sibling parent reply %u <wfunction hotmail.com> writes:
 Programmers are allowed to make conceptually safe functions which


are not marked as  safe, why not the same for pure functions?

Programmers can always shoot themselves in the foot anyway, if they
really want to. Why not just make it easier for them? :) (We could
allow unsafe casts, for instance.)

Sorry, but that's the argument here...
Feb 23 2011
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 12:25:52 -0500, %u <wfunction hotmail.com> wrote:


 Programmers are allowed to make conceptually safe functions which


 are not marked as  safe, why not the same for pure functions?

 Programmers can always shoot themselves in the foot anyway, if they
 really want to. Why not just make it easier for them? :) (We could
 allow unsafe casts, for instance.)


All casts are inherently unsafe.


 Sorry, but that's the argument here...


No, that's not it.  The argument is that the 'compiler knows best' mode  
also known as  safe can get in the way of writing high performance code.   
When I can prove to myself that code is safe, but the compiler can't, I  
have to step into "unsafe" land.  To say pure functions cannot enjoy that  
ability is too limiting.

Note, you can *still* have pure  safe functions if you want to write code  
in that mode.  But pure implying  safe doesn't make sense.

-Steve
Feb 23 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/11 11:16 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:01:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 10:52 AM, Steven Schveighoffer wrote:

 Why? We have both attributes, why not just require " safe pure" if you
 want  safe pure functions?


 Because a pure unsafe function is useless.


 Just because a function is not marked  safe does not mean it is unsafe.
 It just means you can do things the compiler cannot verify are safe, but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


I understand that. My point is that allowing unsafe functions to be pure 
dilutes pure to the point of uselessness.

Andrei
Feb 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:



 Just because a function is not marked  safe does not mean it is unsafe.
 It just means you can do things the compiler cannot verify are safe, but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


 I understand that. My point is that allowing unsafe functions to be pure  
 dilutes pure to the point of uselessness.


And that's not a point.  It's an unsupported opinion.

pure has nothing to do with safety, it has to do with optimization.  Safe  
functions are no more optimizable than unsafe ones.  Safety has to do with  
reducing memory bugs.

The two concepts are orthogonal, I have not been convinced otherwise.

-Steve
Feb 23 2011
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/11 11:47 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:



 Just because a function is not marked  safe does not mean it is unsafe.
 It just means you can do things the compiler cannot verify are safe, but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


 I understand that. My point is that allowing unsafe functions to be
 pure dilutes pure to the point of uselessness.


 And that's not a point. It's an unsupported opinion.


Fine, I agree. If I'll have better arguments in the future, I'll bring 
them up here.

Andrei
Feb 23 2011
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 23/02/2011 17:47, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:



 Just because a function is not marked  safe does not mean it is unsafe.
 It just means you can do things the compiler cannot verify are safe, but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


 I understand that. My point is that allowing unsafe functions to be
 pure dilutes pure to the point of uselessness.


 And that's not a point. It's an unsupported opinion.

 pure has nothing to do with safety, it has to do with optimization. Safe
 functions are no more optimizable than unsafe ones. Safety has to do
 with reducing memory bugs.

 The two concepts are orthogonal, I have not been convinced otherwise.

 -Steve


pure has something to do with  safety. (Also, it has more to do with 
than just optimization, it also affects code readability.)

In order to gain any benefit from calling pure functions (whether the 
benefit is programmer code readability or compiler optimization) it 
needs to be determined from the pure function's signature what is the 
transitively reachable mutable state that the function may access. 
Normally this state is whatever is transitively reachable from the 
parameters. However, if you allow *arbitrary* _pointer arithmetic_ you 
could legally manipulate any mutable data in your program from within 
the pure function. This would make the pure attribute useless because it 
would not offer any additional guarantees whatsoever over an unpure 
function. So such a rule is necessary such that, for example, the 
following function should not be allowed to be pure:

pure int func(int* ptr, int ix) {
   return (ptr + ix)++;
}

I'm not sure if this is what Andrei had in mind with regards to  safety.
It should be noted that none of this implies that free() should be 
disallowed in pure functions. And indeed I think that if malloc() is 
allowerd, free() can and should be allowed as well.


-- 
Bruno Medeiros - Software Engineer
Mar 08 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 08 Mar 2011 14:28:44 -0500, Bruno Medeiros  
<brunodomedeiros+spam com.gmail> wrote:


 On 23/02/2011 17:47, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:



 Just because a function is not marked  safe does not mean it is  
 unsafe.
 It just means you can do things the compiler cannot verify are safe,  
 but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


 I understand that. My point is that allowing unsafe functions to be
 pure dilutes pure to the point of uselessness.


 And that's not a point. It's an unsupported opinion.

 pure has nothing to do with safety, it has to do with optimization. Safe
 functions are no more optimizable than unsafe ones. Safety has to do
 with reducing memory bugs.

 The two concepts are orthogonal, I have not been convinced otherwise.

 -Steve


 pure has something to do with  safety. (Also, it has more to do with  
 than just optimization, it also affects code readability.)

 In order to gain any benefit from calling pure functions (whether the  
 benefit is programmer code readability or compiler optimization) it  
 needs to be determined from the pure function's signature what is the  
 transitively reachable mutable state that the function may access.  
 Normally this state is whatever is transitively reachable from the  
 parameters. However, if you allow *arbitrary* _pointer arithmetic_ you  
 could legally manipulate any mutable data in your program from within  
 the pure function. This would make the pure attribute useless because it  
 would not offer any additional guarantees whatsoever over an unpure  
 function. So such a rule is necessary such that, for example, the  
 following function should not be allowed to be pure:

 pure int func(int* ptr, int ix) {
    return (ptr + ix)++;
 }


If I want to make my own array type, then it will be quite unusable in  
such pure functions.

Assuming that such code is illegal assumes the compiler fully understands  
the semantic meaning of that code.

Keep in mind that an array is semantically understood by the compiler, but  
I can't convey that same semantic knowledge to the compiler for my custom  
type.  I may know that some pointer arithmetic is perfectly safe and pure,  
even when the compiler cannot grok that.  It doesn't make sense that pure  
functions shouldn't be allowed to be streamlined/optimized as I can normal  
functions.  To me the concepts of  safe and pure are still orthogonal.

 safe is not so much about "safety" as it is about "compiler-verified  
safety".  I feel somewhat the words are getting in the way here.  I don't  
mean memory unsafe code, I mean memory safe code that cannot be compiler  
verified via  safe.  I feel that because the compiler can't verify pure  
functions are memory safe doesn't make pure functions unusable or useless.

-Steve
Mar 08 2011
next sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 08/03/2011 20:17, Steven Schveighoffer wrote:

 On Tue, 08 Mar 2011 14:28:44 -0500, Bruno Medeiros
 <brunodomedeiros+spam com.gmail> wrote:


 On 23/02/2011 17:47, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:



 Just because a function is not marked  safe does not mean it is
 unsafe.
 It just means you can do things the compiler cannot verify are
 safe, but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


 I understand that. My point is that allowing unsafe functions to be
 pure dilutes pure to the point of uselessness.


 And that's not a point. It's an unsupported opinion.

 pure has nothing to do with safety, it has to do with optimization. Safe
 functions are no more optimizable than unsafe ones. Safety has to do
 with reducing memory bugs.

 The two concepts are orthogonal, I have not been convinced otherwise.

 -Steve


 pure has something to do with  safety. (Also, it has more to do with
 than just optimization, it also affects code readability.)

 In order to gain any benefit from calling pure functions (whether the
 benefit is programmer code readability or compiler optimization) it
 needs to be determined from the pure function's signature what is the
 transitively reachable mutable state that the function may access.
 Normally this state is whatever is transitively reachable from the
 parameters. However, if you allow *arbitrary* _pointer arithmetic_ you
 could legally manipulate any mutable data in your program from within
 the pure function. This would make the pure attribute useless because
 it would not offer any additional guarantees whatsoever over an unpure
 function. So such a rule is necessary such that, for example, the
 following function should not be allowed to be pure:

 pure int func(int* ptr, int ix) {
 return (ptr + ix)++;
 }


 If I want to make my own array type, then it will be quite unusable in
 such pure functions.


That was just a synthetic example, to show my point. Sure, here we could 
replace the parameters with an array, and then the compiler would be 
able to understand the semantics behind it, and thus also determine the 
actual possible transitive state (which would be the contents of the array).
But there are other situations where such would not be possible.


 Assuming that such code is illegal assumes the compiler fully
 understands the semantic meaning of that code.

 Keep in mind that an array is semantically understood by the compiler,
 but I can't convey that same semantic knowledge to the compiler for my
 custom type. I may know that some pointer arithmetic is perfectly safe
 and pure, even when the compiler cannot grok that. It doesn't make sense
 that pure functions shouldn't be allowed to be streamlined/optimized as
 I can normal functions. To me the concepts of  safe and pure are still
 orthogonal.


I'm not saying all pointer arithmetic and manipulation should be 
illegal. It could be allowed, but only so long as the coder maintains 
the contract of the pure attribute. So this means that you could use 
pointers to manipulate whatever is transitively reachable from the 
function parameters (or stuff that was created inside the pure 
function), but the rest should not be accessed through pointer 
arithmetic, precisely because the compiler would not be able to 
determine that from the function signature.
Note that when I said "illegal" I didn't necessarily mean compiler 
verified illegal code. That might be too complex to implement in the 
language, so instead it might just be a unchecked contract. Breaking 
that contract would result in /undefined behavior/.



  safe is not so much about "safety" as it is about "compiler-verified
 safety". I feel somewhat the words are getting in the way here. I don't
 mean memory unsafe code, I mean memory safe code that cannot be compiler
 verified via  safe. I feel that because the compiler can't verify pure
 functions are memory safe doesn't make pure functions unusable or useless.

 -Steve


Yes, I am feeling we are getting a bit confused by words. Although I 
think I understood what you meant by unsafe code: legal code that cannot 
be verified by the compiler to be " safe". The only other alternative 
(other than  safe code, is simply illegal code, in the runtime sense: 
aka, code that results in /undefined behavior/).

-- 
Bruno Medeiros - Software Engineer
Mar 08 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 08 Mar 2011 18:33:31 -0500, Bruno Medeiros  
<brunodomedeiros+spam com.gmail> wrote:


 I'm not saying all pointer arithmetic and manipulation should be  
 illegal. It could be allowed, but only so long as the coder maintains  
 the contract of the pure attribute. So this means that you could use  
 pointers to manipulate whatever is transitively reachable from the  
 function parameters (or stuff that was created inside the pure  
 function), but the rest should not be accessed through pointer  
 arithmetic, precisely because the compiler would not be able to  
 determine that from the function signature.
 Note that when I said "illegal" I didn't necessarily mean compiler  
 verified illegal code. That might be too complex to implement in the  
 language, so instead it might just be a unchecked contract. Breaking  
 that contract would result in /undefined behavior/.


Then I think we are saying the same thing :)

-Steve
Mar 09 2011
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 09/03/2011 13:32, Steven Schveighoffer wrote:

 On Tue, 08 Mar 2011 18:33:31 -0500, Bruno Medeiros
 <brunodomedeiros+spam com.gmail> wrote:


 I'm not saying all pointer arithmetic and manipulation should be
 illegal. It could be allowed, but only so long as the coder maintains
 the contract of the pure attribute. So this means that you could use
 pointers to manipulate whatever is transitively reachable from the
 function parameters (or stuff that was created inside the pure
 function), but the rest should not be accessed through pointer
 arithmetic, precisely because the compiler would not be able to
 determine that from the function signature.
 Note that when I said "illegal" I didn't necessarily mean compiler
 verified illegal code. That might be too complex to implement in the
 language, so instead it might just be a unchecked contract. Breaking
 that contract would result in /undefined behavior/.


 Then I think we are saying the same thing :)

 -Steve


Cool then :)

-- 
Bruno Medeiros - Software Engineer
Mar 23 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, March 08, 2011 12:17:02 Steven Schveighoffer wrote:

 On Tue, 08 Mar 2011 14:28:44 -0500, Bruno Medeiros
 
 <brunodomedeiros+spam com.gmail> wrote:

 On 23/02/2011 17:47, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu
 
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:

 Just because a function is not marked  safe does not mean it is
 unsafe.
 It just means you can do things the compiler cannot verify are safe,
 but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.
 
 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?
 
 -Steve


 
 I understand that. My point is that allowing unsafe functions to be
 pure dilutes pure to the point of uselessness.


 
 And that's not a point. It's an unsupported opinion.
 
 pure has nothing to do with safety, it has to do with optimization. Safe
 functions are no more optimizable than unsafe ones. Safety has to do
 with reducing memory bugs.
 
 The two concepts are orthogonal, I have not been convinced otherwise.
 
 -Steve


 
 pure has something to do with  safety. (Also, it has more to do with
 than just optimization, it also affects code readability.)
 
 In order to gain any benefit from calling pure functions (whether the
 benefit is programmer code readability or compiler optimization) it
 needs to be determined from the pure function's signature what is the
 transitively reachable mutable state that the function may access.
 Normally this state is whatever is transitively reachable from the
 parameters. However, if you allow *arbitrary* _pointer arithmetic_ you
 could legally manipulate any mutable data in your program from within
 the pure function. This would make the pure attribute useless because it
 would not offer any additional guarantees whatsoever over an unpure
 function. So such a rule is necessary such that, for example, the
 following function should not be allowed to be pure:
 
 pure int func(int* ptr, int ix) {
 
    return (ptr + ix)++;
 
 }


 
 If I want to make my own array type, then it will be quite unusable in
 such pure functions.
 
 Assuming that such code is illegal assumes the compiler fully understands
 the semantic meaning of that code.
 
 Keep in mind that an array is semantically understood by the compiler, but
 I can't convey that same semantic knowledge to the compiler for my custom
 type.  I may know that some pointer arithmetic is perfectly safe and pure,
 even when the compiler cannot grok that.  It doesn't make sense that pure
 functions shouldn't be allowed to be streamlined/optimized as I can normal
 functions.  To me the concepts of  safe and pure are still orthogonal.
 
  safe is not so much about "safety" as it is about "compiler-verified
 safety".  I feel somewhat the words are getting in the way here.  I don't
 mean memory unsafe code, I mean memory safe code that cannot be compiler
 verified via  safe.  I feel that because the compiler can't verify pure
 functions are memory safe doesn't make pure functions unusable or useless.


 safe is for compiler-verified safety.  trusted is for programmer-verified
safety.

- Jonathan M Davis
Mar 08 2011
prev sibling parent reply Don <nospam nospam.com> writes:
Bruno Medeiros wrote:

 On 23/02/2011 17:47, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:



 Just because a function is not marked  safe does not mean it is unsafe.
 It just means you can do things the compiler cannot verify are safe, 
 but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


 I understand that. My point is that allowing unsafe functions to be
 pure dilutes pure to the point of uselessness.


 And that's not a point. It's an unsupported opinion.

 pure has nothing to do with safety, it has to do with optimization. Safe
 functions are no more optimizable than unsafe ones. Safety has to do
 with reducing memory bugs.

 The two concepts are orthogonal, I have not been convinced otherwise.

 -Steve


 
 pure has something to do with  safety. (Also, it has more to do with 
 than just optimization, it also affects code readability.)
 
 In order to gain any benefit from calling pure functions (whether the 
 benefit is programmer code readability or compiler optimization) it 
 needs to be determined from the pure function's signature what is the 
 transitively reachable mutable state that the function may access. 
 Normally this state is whatever is transitively reachable from the 
 parameters. However, if you allow *arbitrary* _pointer arithmetic_ you 
 could legally manipulate any mutable data in your program from within 
 the pure function. This would make the pure attribute useless because it 
 would not offer any additional guarantees whatsoever over an unpure 
 function. So such a rule is necessary such that, for example, the 
 following function should not be allowed to be pure:
 
 pure int func(int* ptr, int ix) {
   return (ptr + ix)++;
 }


I don't think this makes the pure attribute useless, since you still 
only get a violation of purity, if you are smuggling in the address of a 
global via some other parameter (in this case, ix).
You just can't do strong purity optimisation if there are any pointer 
parameters. But that remains true even if you disallow pointer 
arithmetic inside pure functions.

I don't think it can violate weak purity, unless the caller deliberately 
smuggles the address of a global. So I don't know if this needs to be 
prevented, or not.


 I'm not sure if this is what Andrei had in mind with regards to  safety.
 It should be noted that none of this implies that free() should be 
 disallowed in pure functions. And indeed I think that if malloc() is 
 allowerd, free() can and should be allowed as well.
Mar 12 2011
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 12/03/2011 20:21, Don wrote:

 Bruno Medeiros wrote:

 On 23/02/2011 17:47, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 12:28:33 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 2/23/11 11:16 AM, Steven Schveighoffer wrote:



 Just because a function is not marked  safe does not mean it is
 unsafe.
 It just means you can do things the compiler cannot verify are
 safe, but
 that you know are actually safe. I showed you earlier an example of a
 safe pure function that uses malloc and free.

 Programmers are allowed to make conceptually safe functions which are
 not marked as  safe, why not the same for pure functions?

 -Steve


 I understand that. My point is that allowing unsafe functions to be
 pure dilutes pure to the point of uselessness.


 And that's not a point. It's an unsupported opinion.

 pure has nothing to do with safety, it has to do with optimization. Safe
 functions are no more optimizable than unsafe ones. Safety has to do
 with reducing memory bugs.

 The two concepts are orthogonal, I have not been convinced otherwise.

 -Steve


 pure has something to do with  safety. (Also, it has more to do with
 than just optimization, it also affects code readability.)

 In order to gain any benefit from calling pure functions (whether the
 benefit is programmer code readability or compiler optimization) it
 needs to be determined from the pure function's signature what is the
 transitively reachable mutable state that the function may access.
 Normally this state is whatever is transitively reachable from the
 parameters. However, if you allow *arbitrary* _pointer arithmetic_ you
 could legally manipulate any mutable data in your program from within
 the pure function. This would make the pure attribute useless because
 it would not offer any additional guarantees whatsoever over an unpure
 function. So such a rule is necessary such that, for example, the
 following function should not be allowed to be pure:

 pure int func(int* ptr, int ix) {
 return (ptr + ix)++;
 }


 I don't think this makes the pure attribute useless, since you still
 only get a violation of purity, if you are smuggling in the address of a
 global via some other parameter (in this case, ix).


I mean useless for the callers of pure functions (in other words the 
guarantees it offers externally). Inside the pure function it would 
still of some (limited) use, as the purity checks would still be present 
(you could not access a global directly, for example)


 You just can't do strong purity optimisation if there are any pointer
 parameters. But that remains true even if you disallow pointer
 arithmetic inside pure functions.


Why is it the case that it is still true if you disallow pointer arithmetic?
For example:

void main() {
   int[] globalArray = initializedElsewhere();

   int b1 = globalArray[30];
   auto a1 = someComplicatedPureFunction(&globalArray[10], 42);

   int b2 = globalArray[30];

   globalArray[20] = 666;

   auto a2 = someComplicatedPureFunction(&globalArray[10], 42);
}


Can the initial value of b2 be taken from b1, or do we have to access 
the memory again?
Similarly, can the initial value of a2 be taken from a1, or do we have 
to call someComplicatedPureFunction again?

It seems that if we disallow pointer arithmetic, the answer is yes for 
both case, but no otherwise. (well, for the a2 case it also needs to be 
that someComplicatedPureFunction does not return a value that it has 
allocated) Isn't it so?


-- 
Bruno Medeiros - Software Engineer
Mar 23 2011
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-02-23 12:01:15 -0500, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 Because a pure unsafe function is useless.


I disagree. Suppose you have a function which is conceptually pure but 
requires a temporary 100 Mb matrix of doubles. Wouldn't it make sense 
to use malloc/free for this temporary storage instead of using the GC 
and risking the block never being collected because of a false pointer 
somewhere?

Should a function be prevented from being pure just because the 
programmer decided to use some tricks the compiler can't guaranty the 
safety of? There might be legitimate reasons for those tricks like 
optimization, solving GC memory problems, using special hardware for 
some calculation, etc.

 safe has an escape route ( trusted) for when you need to perform these 
things. If pure makes a function  safe by default, we need to have an 
escape route for it too (" trusted pure" perhaps?). My only fear is 
that pure implying  safe needlessly complicates the attribute system.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Feb 23 2011
parent %u <wfunction hotmail.com> writes:
 Because a pure unsafe function is useless.





 I disagree. Suppose you have a function which is conceptually pure but
requires a temporary 100


Mb matrix of doubles. Wouldn't it make sense to use malloc/free for this
temporary storage
instead of using the GC and risking the block never being collected because of
a false pointer
somewhere?

This is _precisely_ why I think the 'delete' keyword will still have uses -- it
lets you delete
memory that you _know_ will no longer be needed, and it gets read if the need
to malloc/free. As
a bonus, it will be much less "special" of a case than excepting the ordinary
malloc/free
functions.
But anyway, I doubt that that's going to convince anyone, so I'll stop talking
about the delete
keyword. :)

Another point is that the problem with false pointers is actually an
implementation flaw in the D
runtime, _not_ an issue with the language. I actually think that even an
extremely small chance
at remote possibility of a false pointer will provide enough reason for people
to avoid adopting
D, notwithstanding the greatness (IMHO) of the language as it currently is. If
D is going to be
adopted on a large scale, I don't think it's going to happen with the
possibility of false
pointers looming in the dark.
It would be an architectural change for the GC (and probably the compiler) to
fix the issue, but
now that it's brought up, I think it might be a good idea to revisit it, since
even if it's not a
great possibility, it's obviously an uneasiness lurking in the back of people's
heads, and a
hindrance to the adoption of the language.
Any thoughts on this? (Should this GC discussion be a separate thread?)
Feb 23 2011
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2011-02-23 11:45:48 -0500, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 On 2/23/11 9:57 AM, Steven Schveighoffer wrote:

 On Wed, 23 Feb 2011 10:46:43 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 

 At a level it's clear to me that a function cannot be at the same time
 pure and unsafe. For example:
 
 pure void foo(int *x) { free(x); (*x)++; }
 
 This function essentially breaks any guarantee for the entire program,
 so it would be quite difficult to claim it's pure.


 
 I thought  safe was orthogonal to pure? If this isn't the case, then
 yes, free must be disallowed. But then malloc must also be, and any
 construct which manages its own memory via malloc/free.


 
 malloc can be reasonably made pure. It would be a mistake to see malloc 
 and free as two symmetrical parts of a whole. The truth of the matter 
 is that allocating is fine, deallocating is problematic.


Deallocating is problematic if after deallocation you continue to keep 
references to the deallocated memory. Generally speaking, it's unsafe 
to keep references to deallocated memory; whether you're in a pure 
function or not does not change that.

There's no question that 'free' should be a  system function because if 
used incorrectly your program will behave erratically. But this has 
nothing to do with purity.

There's no question that malloc/free manipulates the global state. The 
problem to purity I see in that is that the memory address returned by 
malloc will be different each time. We've decided that allocating 
memory with the 'new' keyword was fine, so I don't see any harm in 
allowing 'malloc' too (as long as you can make sure the compiler won't 
fuse two malloc calls in one if they have the same argument). As for 
free, theoretically you should never call it twice with the same 
argument, so I don't see any harm in making it pure.

If you want your pure function to also be safe, it's easy to make it  safe.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Feb 23 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, February 22, 2011 12:48:42 %u wrote:

 D pure functions are significantly different than this definition


 
 (as of recent times, when weak-pure was added).
 

 Essentially, a pure function cannot access global variables.


 
 However, it can access variables referred to via a member of the
 object instance.
 

 i.e. this is a valid pure function:


 class C
 {
    int x;
    pure void foo() { x++; }
 }
 
 I... did not know that. But even in that case, pure wouldn't make much
 sense, because doing anything like freeing memory or closing a file
 handle affects global variables (whether directly in the runtime or
 indirectly in the OS)... right?


Except that newing something up is pure. pure is used in D to allow for 
optimizations. That and being able to say that a function doesn't access
globals 
are pretty much the only real reasons for its existence, I think. A pure 
function does not allow any access to mutable global variables (and it may or 
may not presently allow access to immutable global variables - I don't remember 
- but it could). A strongly pure function (a pure function whose parameters are 
all immutable or implicitly convertible to immutable) can have multiple calls
to 
it with the same values optimized out. A weakly pure function (a pure function 
which is not strongly pure) can't be optimized out, but it _can_ be called from 
a strongly pure function - unlike non-pure functions.

Constructors and destructors are funny cases, since they mess with memory, but 
you have to be able to make them pure (or at least you have to be able to make 
constructors pure) or you can't allocate anything in a pure function You can't 
optimize them out, but they _are_ designed such that you can call them from
pure 
functions if they're pure. So, it would really only matter whether a destructor 
were pure if you called it from a pure function, and you don't normally call 
destructors. So, I don't think that there's necessarily anything wrong with 
marking a destructor as pure, but I'm not sure that there's necessarily much 
point to it either.

- Jonathan M Davis
Feb 22 2011
prev sibling parent reply Ary Manzana <ary esperanto.org.ar> writes:
On 2/22/11 10:36 AM, Simen Kjaeraas wrote:

 %u Wrote:

 Well, the trouble is, pretty much all of these are invalid attributes:



 - static obviously makes no sense


 And here is where you're wrong. You have defined a static destructor, which is
called with module destructor as the program goes out of scope, rather than
when your struct or class is destroyed.


This is why attributes that make no sense must be an error: you don't 
know if an attribute you put is being ignored by the compiler or not 
(like what has just happened here).
Feb 23 2011
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 23/02/2011 18:07, Ary Manzana wrote:

 On 2/22/11 10:36 AM, Simen Kjaeraas wrote:

 %u Wrote:

 Well, the trouble is, pretty much all of these are invalid attributes:



 - static obviously makes no sense


 And here is where you're wrong. You have defined a static destructor, which is
called
 with module destructor as the program goes out of scope, rather than when your
struct or
 class is destroyed.


 This is why attributes that make no sense must be an error: you don't know if
an attribute
 you put is being ignored by the compiler or not (like what has just happened
here).


Uh, that's a total non sequitur.  The point Simen is making is that static
_does_ make 
sense here.

Stewart.
Feb 23 2011
parent Kevin Bealer <kevindangerbealer removedanger.gmail.com> writes:
== Quote from Stewart Gordon (smjg_1998 yahoo.com)'s article

 On 23/02/2011 18:07, Ary Manzana wrote:

 On 2/22/11 10:36 AM, Simen Kjaeraas wrote:

 %u Wrote:

 Well, the trouble is, pretty much all of these are invalid attributes:



 - static obviously makes no sense


 And here is where you're wrong. You have defined a static destructor, which






is called

 with module destructor as the program goes out of scope, rather than when






your struct or

 class is destroyed.


 This is why attributes that make no sense must be an error: you don't know if




an attribute

 you put is being ignored by the compiler or not (like what has just happened




here).

 Uh, that's a total non sequitur.  The point Simen is making is that static


_does_ make

 sense here.
 Stewart.


But if the compiler always rejected the nonsensical ones it would be clear that
the ones that
are not rejected have some meaning.  The ambiguity he is talking about (I think)
is between
the two ideas "the compiler accepted this but its meaningless" and "the compiler
accepted this
so it *must* be meaningful".  If you can't be sure the second is true, then you
don't realize
there is a bug, e.g. if you did not intend the destructor to be a class/module
destructor.

Kevin
Feb 23 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 22 Feb 2011 01:08:38 -0500, %u <wfunction hotmail.com> wrote:


 dmd is pretty lax about attributes which don't apply. It generally just  
 ignores them. Personally,


 I think that it should error on invalid attributes, but for some reason,  
 that's not how it works.
 Of course, there could be other bugs in play here, but there's every  
 possibility that the end
 result is completely valid.

 Well, the trouble is, pretty much all of these are invalid attributes:
 - const and pure make no sense, since destructors (should) change an  
 object's state


const is perfectly valid; From one perspective, you're rarely changing the  
actual object state, just free-ing allocated resources. From another, you  
might be logging something and don't want to accidentally change a value.

pure, especially weak purity, is expected from most destructors (and  
functions for that matter). And given that destructors should be able to  
be run on a GC thread, disabling access to TLS variables is perfectly  
valid.

Now, put the two together and you do end up with your hands mostly tied,  
but asserts are still valid as is (IIRC) freeing memory.


 - override and final make no sense, since destructors obviously aren't  
 ever overridden... they're
 always called after the subclass's destructor is called


Regarding override, all class destructors are implicitly override, in  
which case the extra modifiers make no semantic difference. But module  
destructors are not virtual, so override doesn't make sense.

Regarding final, it also makes sense for classes (i.e. preventing  
inheritance), but again module destructors are not virtual, so final  
doesn't make sense in this context.


 - static obviously makes no sense


As stated by others, this is a module destructor, which are declared using  
the static keyword.


 - synchronized is meaningless since there's only one thread ever running  
 the destructor anyway


Well, synchronized(monitor) would make perfect sense (since module  
destructors are run whenever a thread shuts down), and as there should be  
a way to manually set/get the implicit synchronized monitor, this is valid.


 - private makes no sense since (unless we're trying to imitate C++ here)  
 destructors are only
 called from the runtime, and nowhere else.


Well, this comes from the fact that all functions have a protection  
modifier (private,package,protected,public,etc). So it's perfectly valid  
to stick whichever one you want on it, since none of them apply and one  
must be present.
Feb 22 2011
prev sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 22/02/2011 06:08, %u wrote:
<snip>

 - synchronized is meaningless since there's only one thread ever running the
destructor anyway


As such, destructors are vacuously synchronized.  At least, if you don't have a
collision 
between threads that try to delete the same object at the same time, but that
would 
indicate a bug in the program anyway.


 - private makes no sense since (unless we're trying to imitate C++ here)
destructors are only
 called from the runtime, and nowhere else.
 - The only meaningful attribute there is extern(C).


In what way is extern(C) meaningful for a destructor?


 I would agree that DMD should ignore the attributes that are redundant or
optional (e.g. it should
 be okay for "static" to be written and/or omitted at the module-level, and DMD
should ignore it)
 but I don't see why _wrong_ attributes should be ignored... it confuses the
programmer, opens the
 potential for error, and doesn't have any benefits.


I entirely agree.  It's been discussed quite a bit:
http://d.puremagic.com/issues/show_bug.cgi?id=3118


 (The only exception I can think of to this rule
 would be attributes that cannot be removed, like saying "private:" in the
beginning... for general
 attributes like those, I guess DMD can ignore them, but for specifically
written attributes like
 these, is there any benefit whatsoever to allowing them?)


No.

Stewart.
Feb 22 2011
parent reply %u <wfunction hotmail.com> writes:
 - private makes no sense since (unless we're trying to imitate C++ here)
destructors are only called from




the runtime, and nowhere else.

 - The only meaningful attribute there is extern(C).





 In what way is extern(C) meaningful for a destructor?


I guess it would be logical to specify that, if someone is desperately trying
to get C code to interop with
D. But I don't think it's too useful... it's just not meaningless.



 I would agree that DMD should ignore the attributes that are redundant or
optional (e.g. it should be




okay for "static" to be written and/or omitted at the module-level, and DMD
should ignore it) but I don't
see why _wrong_ attributes should be ignored... it confuses the programmer,
opens the potential for error,
and doesn't have any benefits.


 I entirely agree.  It's been discussed quite a bit:
 http://d.puremagic.com/issues/show_bug.cgi?id=3118


Interesting... thanks for the link!
Feb 22 2011
parent Stewart Gordon <smjg_1998 yahoo.com> writes:
On 23/02/2011 01:22, %u wrote:

 - private makes no sense since (unless we're trying to imitate C++ here)
destructors are only called from




 the runtime, and nowhere else.

 - The only meaningful attribute there is extern(C).





 In what way is extern(C) meaningful for a destructor?


 I guess it would be logical to specify that, if someone is desperately trying
to get C code to interop with
 D. But I don't think it's too useful... it's just not meaningless.


<snip>

To me it doesn't make sense for C code to call a D object's destructor.  For a
start, it 
would bypass the D runtime that ensures that an object is destructed only once.
 How would 
the name be mangled, anyway?

Stewart.
Feb 23 2011
prev sibling parent reply dsimcha <dsimcha yahoo.com> writes:
On 2/22/2011 12:13 AM, Jonathan M Davis wrote:

 On Monday 21 February 2011 20:46:56 %u wrote:

 Hi,

 I'm just curious... why is saying something like this:

 extern(C)
      private static const pure override final synchronized ~this() { }

 allowed?


 dmd is pretty lax about attributes which don't apply. It generally just ignores
 them. Personally, I think that it should error on invalid attributes, but for
 some reason, that's not how it works. Of course, there could be other bugs in
 play here, but there's every possibility that the end result is completely
 valid.

 - Jonathan M Davis


One point noone's apparently made yet:  DMD's ignorance here is 
occasionally a boon for generic programming.  For example, in some 
places in various code I write things like:

void doStuff(C)(scope C callable)

In this case, C can be either a delegate, a function pointer, or a class 
or struct that overloads opCall.  Scope structs and function pointers 
make no sense.  Scope delegates mean that the delegate does not escape 
the scope of doStuff(), so no closure allocation is needed.  If I had to 
write two separate functions to handle cases like these it would be a 
**huge** PITA.
Feb 23 2011
parent reply Steven Wawryk <stevenw acres.com.au> writes:
On 24/02/11 14:47, dsimcha wrote:

 On 2/22/2011 12:13 AM, Jonathan M Davis wrote:

 On Monday 21 February 2011 20:46:56 %u wrote:

 Hi,

 I'm just curious... why is saying something like this:

 extern(C)
 private static const pure override final synchronized ~this() { }

 allowed?


 dmd is pretty lax about attributes which don't apply. It generally
 just ignores
 them. Personally, I think that it should error on invalid attributes,
 but for
 some reason, that's not how it works. Of course, there could be other
 bugs in
 play here, but there's every possibility that the end result is
 completely
 valid.

 - Jonathan M Davis


 One point noone's apparently made yet: DMD's ignorance here is
 occasionally a boon for generic programming. For example, in some places
 in various code I write things like:

 void doStuff(C)(scope C callable)

 In this case, C can be either a delegate, a function pointer, or a class
 or struct that overloads opCall. Scope structs and function pointers
 make no sense. Scope delegates mean that the delegate does not escape
 the scope of doStuff(), so no closure allocation is needed. If I had to
 write two separate functions to handle cases like these it would be a
 **huge** PITA.


Aren't scope parameters deprecated and going to disappear from under 
your feet?
Feb 24 2011
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 24 Feb 2011 19:25:24 -0500, Steven Wawryk <stevenw acres.com.au>  
wrote:


 On 24/02/11 14:47, dsimcha wrote:

 On 2/22/2011 12:13 AM, Jonathan M Davis wrote:

 On Monday 21 February 2011 20:46:56 %u wrote:

 Hi,

 I'm just curious... why is saying something like this:

 extern(C)
 private static const pure override final synchronized ~this() { }

 allowed?


 dmd is pretty lax about attributes which don't apply. It generally
 just ignores
 them. Personally, I think that it should error on invalid attributes,
 but for
 some reason, that's not how it works. Of course, there could be other
 bugs in
 play here, but there's every possibility that the end result is
 completely
 valid.

 - Jonathan M Davis


 One point noone's apparently made yet: DMD's ignorance here is
 occasionally a boon for generic programming. For example, in some places
 in various code I write things like:

 void doStuff(C)(scope C callable)

 In this case, C can be either a delegate, a function pointer, or a class
 or struct that overloads opCall. Scope structs and function pointers
 make no sense. Scope delegates mean that the delegate does not escape
 the scope of doStuff(), so no closure allocation is needed. If I had to
 write two separate functions to handle cases like these it would be a
 **huge** PITA.


 Aren't scope parameters deprecated and going to disappear from under  
 your feet?


No, just scope classes.

-Steve
Feb 24 2011
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
 This is an idea to patch that hole a little, doing this inside pure functions:
 1) Keep disallowing alloca()/malloc()/etc calls;
 2) Disallow struct allocations;
 3) Keep allowing object and dynamic array allocations;
 4) Disallow read and write of the "ptr" fields of dynamic arrays;
 5) Disallow casts of object references to something else.


Note this is about _inside pure functions_.

A safer idea comes from using the type system, but it's hard. The idea is
introducing "pure memory references". Any memory allocation inside a pure
function returns a pointer/reference that can't be _read_ (as value) inside the
pure function itself, or even outside it, if it among the function results. I
think it's possible to design a type system able to enforce this safely, but I
don't think this will happen for D.

Bye,
bearophile
Feb 22 2011
parent bearophile <bearophileHUGS lycos.com> writes:
 A safer idea comes from using the type system, but it's hard. The idea is
introducing "pure memory references".


I think I have just invented the idea of referentially transparent
pointers/references for D, a subtype of normal D pointers/references.

Bye,
bearophile
Feb 22 2011
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 cast voids all warranties ;)


OK. But that idea is unchanged if you remove the cast and return an int* from
that function.



 Memory allocation has to be allowed inside pure functions.  Otherwise,  
 pure functions are too strict and limited.


I agree. But there are different ways to allocate memory, and those differences
are important.



 Allowing malloc is somewhat exceptional because you then must allow free.  
 But I see no reason (yet) to disallow free.  If free cannot be pure, then  
 malloc cannot be pure.


I suggest to disallow both malloc/calloc and free inside pure functions,
because what malloc returns a pointer, that is a value, that is not
deterministic, it changes across different calls to malloc.



 Note, the 'hole' you refer to is not a bad thing.


It's a bad thing that will cause troubles to both D programmers and D compiler
writers :-)



 A weakly pure function  
 allows one to modularize pure functions, whereas prior to this, things  
 like sort could not be pure functions, and therefore could not be used  
 inside pure functions.  I think the hole allows pure functions to be  
 actually usable and easy whereas before they were much too obscure to be  
 useful in much code.


I think this is unrelated to the hole I was talking about.



 All that is required is to disallow casting.


Disallowing casting is not enough here.

I have written two more posts after that, but you may have missed them because
I have broken the tread:
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=130426

Some examples:

class Foo {
  int x;
  bool opEquals(Foo o) { return x == o.x; }
}
pure Foo bar() {
  return new Foo; // OK
}
void main() {
  Foo f1 = bar(); // OK
  Foo f2 = new Foo;
  f1.x = 10; // OK
  assert(f1 != f2); // OK
  f1 = f2; // OK
  assert(f1 is f2); // no  
}

Here bar() allocated memory and it's pure, this is OK. f1 is mutable. You are
allowed to call opEquals. You are allowed to overwrite the reference f1. But
you aren't allowed to read the reference f1, because this breaks the
referential transparency of the results of pure functions.

The idea is a subtype of pointers/references, that at compile-time doesn't
allow to read the value of the pointer/reference itself. I think this is able
to patch the hole I was talking about (a cast is able to punch a hole again in
this, of course).

Bye,
bearophile
Feb 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 13:13:35 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:


 cast voids all warranties ;)


 OK. But that idea is unchanged if you remove the cast and return an int*  
 from that function.


That is allowed, and it's expected that a pure function can return  
different references with identical calls.

Even if the return value is an immutable reference, it is allowed to  
return different references with identical calls.  You should never expect  
that a pure optimization occurs.  All you should expect is that a pure  
function returns the same value for identical calls.  Same value does not  
mean the same bits.


 Allowing malloc is somewhat exceptional because you then must allow  
 free.
 But I see no reason (yet) to disallow free.  If free cannot be pure,  
 then
 malloc cannot be pure.


 I suggest to disallow both malloc/calloc and free inside pure functions,  
 because what malloc returns a pointer, that is a value, that is not  
 deterministic, it changes across different calls to malloc.


pure functions are not necessarily  safe functions, you can access  
pointers.


 A weakly pure function
 allows one to modularize pure functions, whereas prior to this, things
 like sort could not be pure functions, and therefore could not be used
 inside pure functions.  I think the hole allows pure functions to be
 actually usable and easy whereas before they were much too obscure to be
 useful in much code.


 I think this is unrelated to the hole I was talking about.


Then I don't know of the hole you mean.  Pure functions are 100% about  
optimization.  An optimization should *never* be assumed.  That is, given  
a pure function foo that returns a string, it should not be assumed that:

auto s = foo();
auto s2 = foo();

is always factored into

auto s = foo();
auto s2 = s;

It's an optimization, one which the compiler could or could not decide to  
use.

However, it *should* be assumed that:

assert(s == s2);

That is, the values are the same.

If you are going to use casts, then those rules are out the window.


 All that is required is to disallow casting.


 Disallowing casting is not enough here.


What are your expectations for pure functions?


 I have written two more posts after that, but you may have missed them  
 because I have broken the tread:
 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=130426

 Some examples:

 class Foo {
   int x;
   bool opEquals(Foo o) { return x == o.x; }
 }
 pure Foo bar() {
   return new Foo; // OK
 }
 void main() {
   Foo f1 = bar(); // OK
   Foo f2 = new Foo;
   f1.x = 10; // OK
   assert(f1 != f2); // OK
   f1 = f2; // OK
   assert(f1 is f2); // no
 }

 Here bar() allocated memory and it's pure, this is OK. f1 is mutable.  
 You are allowed to call opEquals. You are allowed to overwrite the  
 reference f1. But you aren't allowed to read the reference f1, because  
 this breaks the referential transparency of the results of pure  
 functions.


There is no such guarantee for weakly-pure functions.  There's not even  
such a guarantee for strong-pure functions.  To guarantee this would  
require some sort of memoization, and require optimizations to be followed.


 The idea is a subtype of pointers/references, that at compile-time  
 doesn't allow to read the value of the pointer/reference itself. I think  
 this is able to patch the hole I was talking about (a cast is able to  
 punch a hole again in this, of course).


I don't see any value in disallowing such accesses.  The expectation that  
two pure function calls will necessarily return the *exact* same bits when  
the function is returning references/pointers is incorrect.

-Steve
Feb 23 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 That is allowed, and it's expected that a pure function can return  
 different references with identical calls.


A pointer and a "transparent reference" are two different things. That function
returns a pointer, and a pointer is a value.
If you call a pure function with the same input you need to receive the same
value. Otherwise it's not pure. If you mean something else, then you have
redefined the meaning of "pure" so much that the word "pure" is the wrong one
to use.



 pure functions are not necessarily  safe functions, you can access  
 pointers.


I know what I am currently able to do with pure functions in D. But here we are
talking about what's the right/good way to design purity (or maybe we are even
talking about what's the right way to modify D purity).

I presume you have understood what I have explained about transparent
references.



 Pure functions are 100% about optimization.


Pure means first of all avoiding side effects. A pure function is not allowed
to do some things, so both the programmer and the compiler are able to use such
stronger constrains to perform some extra trasnformations normally not allowed
on impure functions, and the resulting code is safer from certain kinds of bugs.



 An optimization should *never* be assumed.  That is, given  
 a pure function foo that returns a string, it should not be assumed that:
 
 auto s = foo();
 auto s2 = foo();
 
 is always factored into
 
 auto s = foo();
 auto s2 = s;
 
 It's an optimization, one which the compiler could or could not decide to  
 use.
 
 However, it *should* be assumed that:
 
 assert(s == s2);
 
 That is, the values are the same.


I have suggested an extension of the type system that disallows statically code
like:
assert(s.ptr == s2.ptr);

because this may break the transparent nature of references coming out of pure
functions.



 What are your expectations for pure functions?


Allocating memory in a pure function is so useful that I expect D pure function
to allow it. On the other hand I expect pure functions to behave
deterministically, same input means same output. To solve this I have suggested
di disallow reading the value of references/pointers coming out of pure
functions.



 There is no such guarantee for weakly-pure functions.  There's not even  
 such a guarantee for strong-pure functions.


I know. This is something I'd like to add (well, I am not sure it's
implementable, I am not sure it's a good idea, etc, so it's just an idea for
now. And I don't think D will add it).



 To guarantee this would  
 require some sort of memoization, and require optimizations to be followed.


I think no memoization is needed, I think the only change needed is on the type
system (so there are no effects on runtime. This type system just disallows
some programs).

I may call them  transparent pointers/references, they are a supertype of
normal ones:

class Foo() {}
pure  transparent(Foo) bar() {
  return new Foo();
}
Foo bar2() { // not pure
  return new Foo();
}
void main() {
   transparent Foo f1 = bar(); // OK
   transparent Foo f2 = bar2(); // OK
  Foo f3 = bar(); // not allowed, Foo is a subtype of  transparent(Foo)
}


On a  transparent Foo reference you are allowed to overwrite it, modify or use
anything in the class itself, but you are statically allowed to use "is"
operator on it, because it breaks its transparency (if you cast it to a pointer
or not transparent, then you punch a hole in this type part of the type system).

Something similar is acceptable for struct pointers too, and arrays:

struct Spam() {}
 transparent Spam* bar3() {
  return new Spam();
}

struct Spam() {}
 transparent int[] bar4() {
  return new int[10];
}

Bye,
bearophile
Feb 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 14:31:35 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:


 That is allowed, and it's expected that a pure function can return
 different references with identical calls.


 A pointer and a "transparent reference" are two different things. That  
 function returns a pointer, and a pointer is a value.
 If you call a pure function with the same input you need to receive the  
 same value. Otherwise it's not pure. If you mean something else, then  
 you have redefined the meaning of "pure" so much that the word "pure" is  
 the wrong one to use.


A pointer is not a value, it's a pointer.  int is a value.  You should  
expect two calls to a pure function to return the same exact int.

Because memory allocation has side effects, you have to accept that it can  
allow pure functions to not return exactly the same bits.  That is the  
price of allowing memory allocations.

D is a language that allows pointers and comparison of pointers.  There  
are plenty of languages that don't allow this (such as Java), one of those  
might be better suited for your requirements.

I see zero value in disallowing comparing pointers in D.  What kinds of  
problems does pointer comparison cause?  I know of none.  Showing an  
assert that two pointers are not equal is not evidence of an error, it's  
evidence of incorrect expectations.

-Steve
Feb 23 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 A pointer is not a value, it's a pointer.  int is a value.  You should  
 expect two calls to a pure function to return the same exact int.


I don't care of how you want to define what a pointer is. Definitions are
labels for ideas, you are free to use a different label.



 Because memory allocation has side effects, you have to accept that it can  
 allow pure functions to not return exactly the same bits.


I have suggested a way to avoid most of problems caused by that fact.



 D is a language that allows pointers and comparison of pointers.  There  
 are plenty of languages that don't allow this (such as Java), one of those  
 might be better suited for your requirements.


Java too allows to compare the value of references (like the "is" D operator)
so the type system idea I have suggested is applicable in Java too, if Java
wants to add pure functions that allow allocations.



 I see zero value in disallowing comparing pointers in D.


I have not suggested to disallow comparing all pointers. I have suggested to
disallow it only for pointers/references allocated inside a pure function, that
are  transparent.



 What kinds of problems does pointer comparison cause?  I know of none.


If you compare pointers created in a pure function, you are breaking the most
important constraint of pure functions. A pure function is deterministic. D
pure function aren't deterministic, because the value of the memory pointers
they return can be different across different calls. If you leave this hole in
pure functions, then their purity is much less useful, you can't perform
optimizations, you can't reason about code like you are able to do with pure
functions.

Currently you are able to write functions like:

pure bool randomPure() {
    int[] a1 = new int[1];
    int[] a2 = new int[2];
    return a1.ptr > a2.ptr;
}

Is this function pure? It returns a pseudo-random boolean. You are free to
define this function as pure, but this form of purity is much less strong than
what people think of "pure".

With the type system change I have prosed it becomes:

pure bool randomPure() {
     transparent int[] a1 = new int[1];
     transparent int[] a2 = new int[2];
    return a1.ptr > a2.ptr; // compile-time error, their ptr are  transparent,
so they can't be read
}



 Showing an  
 assert that two pointers are not equal is not evidence of an error, it's  
 evidence of incorrect expectations.


One of the main purposes of a type system is indeed to disallow programs based
on incorrect expectations, to help programmers that may not always remeber what
the incorrect expectations are :-)

Bye,
bearophile
Feb 23 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 15:28:32 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:


 I see zero value in disallowing comparing pointers in D.


 I have not suggested to disallow comparing all pointers. I have  
 suggested to disallow it only for pointers/references allocated inside a  
 pure function, that are  transparent.


That's not what your example showed.  It showed comparing two allocated  
pointers *outside* a pure function, and expected them to be equal.  I see  
that as disallowing all pointer comparisons.


 What kinds of problems does pointer comparison cause?  I know of none.


 If you compare pointers created in a pure function, you are breaking the  
 most important constraint of pure functions. A pure function is  
 deterministic. D pure function aren't deterministic, because the value  
 of the memory pointers they return can be different across different  
 calls. If you leave this hole in pure functions, then their purity is  
 much less useful, you can't perform optimizations, you can't reason  
 about code like you are able to do with pure functions.

 Currently you are able to write functions like:

 pure bool randomPure() {
     int[] a1 = new int[1];
     int[] a2 = new int[2];
     return a1.ptr > a2.ptr;
 }


This is the first real example you have made that shows a problem!  It  
uses all valid constructs within pure functions (without casts), and by  
current rules could be considered strong-pure, however, it violates the  
rule of pure that the same parameters should result in the same answer.


 Is this function pure? It returns a pseudo-random boolean. You are free  
 to define this function as pure, but this form of purity is much less  
 strong than what people think of "pure".


I would not define the function as pure.  The questions to answer are:

1. Can the compiler detect valid reference comparisons without annotation?
2. Is this going to be a common problem?

At first glance, I'd say the answer to number 2 is no.  Most people are  
not going to use the randomness of the memory allocator to subvert  
purity.  It's unlikely that you accidentally write code like that.

The answer to number 1, I don't know.  I don't think the compiler can  
determine the origin of allocated memory without annotation.


 With the type system change I have prosed it becomes:

 pure bool randomPure() {
      transparent int[] a1 = new int[1];
      transparent int[] a2 = new int[2];
     return a1.ptr > a2.ptr; // compile-time error, their ptr are  
  transparent, so they can't be read
 }


I can see now the value in this.  I just wonder if it would be worth it.   
Seems like such a rare bug to create a whole new type constructor.

It also has some unpleasant effects.  For example, the object equality  
operator does this:

bool opEquals(Object o1, Object o2)
{
   if(o1 is o2)
     return true;
   ...
}

So this optimization would be unavailable inside pure functions, no?  Or  
require a dangerous cast?

Would it be enough to just require this type of restriction in pure  safe  
functions?

I feel that a new type of reference is likely overkill for this issue.


 Showing an
 assert that two pointers are not equal is not evidence of an error, it's
 evidence of incorrect expectations.


 One of the main purposes of a type system is indeed to disallow programs  
 based on incorrect expectations, to help programmers that may not always  
 remeber what the incorrect expectations are :-)


My point in the above statement is that when you say:

auto a = new int[1];
auto b = new int[1];
assert(a.ptr == b.ptr);

is not evidence of an error :)  This is what you did previously.

-Steve
Feb 23 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 That's not what your example showed.  It showed comparing two allocated  
 pointers *outside* a pure function, and expected them to be equal.  I see  
 that as disallowing all pointer comparisons.


My first examples were not so good, I am sorry :-) Thank you for not ignoring
my posts despite that.

The idea was that if you allocate memory inside a pure function, then the
result of this memory allocation is a  transparent pointer/reference. And that
a  transparent pointer/reference can't be read (but you are allowed to
dereference it, overwrite it, etc).

So this is OK, because the transparency of the pointer is respected:

pure  transparent(int*) foo() {
  return new int; // allocates just one int on the heap
}
void main() {
   transparent int* i = foo(); // OK
  *i = 10; // OK
}



 I just wonder if it would be worth it.


Probably not, but we need to understand what the holes are, before accepting to
keep them in the language :-)



 It also has some unpleasant effects.  For example, the object equality  
 operator does this:
 
 bool opEquals(Object o1, Object o2)
 {
    if(o1 is o2)
      return true;
    ...
 }
 
 So this optimization would be unavailable inside pure functions, no?  Or  
 require a dangerous cast?



Ick.
You can't give a  transparent Object to function that expects an Object,
because transparent references disallow something that you are allowed to do on
a not transparent reference/pointer:


pure  transparent Object foo() {
  return new Object();
}
void opEquals(Object o1, Object o2) {}
void main() {
   transparent Object o = foo();
  opEquals(o, o); // not allowed
}




 Would it be enough to just require this type of restriction in pure  safe  
 functions?


I don't know.

Currently  safe is a wrong name, it means means  memorySafe. So I think that
currently " memorySafe" and "pure" are almost orthogonal concepts.

Bye,
bearophile
Feb 23 2011
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 23 Feb 2011 16:40:22 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:


 That's not what your example showed.  It showed comparing two allocated
 pointers *outside* a pure function, and expected them to be equal.  I  
 see
 that as disallowing all pointer comparisons.


 My first examples were not so good, I am sorry :-) Thank you for not  
 ignoring my posts despite that.

 The idea was that if you allocate memory inside a pure function, then  
 the result of this memory allocation is a  transparent  
 pointer/reference. And that a  transparent pointer/reference can't be  
 read (but you are allowed to dereference it, overwrite it, etc).

 So this is OK, because the transparency of the pointer is respected:

 pure  transparent(int*) foo() {
   return new int; // allocates just one int on the heap
 }
 void main() {
    transparent int* i = foo(); // OK
   *i = 10; // OK
 }


 transparent has little value outside a pure function, because the  
compiler does not expect to be able to perform pure optimizations on  
normal functions.  Right now, the only drawback you have shown of being  
able to compare such references is when that comparison is used as the  
return value for a pure function.  We can't continue to enforce the  
 transparent rules when they don't prevent problems, or else the language  
becomes too burdensome.


 It also has some unpleasant effects.  For example, the object equality
 operator does this:

 bool opEquals(Object o1, Object o2)
 {
    if(o1 is o2)
      return true;
    ...
 }

 So this optimization would be unavailable inside pure functions, no?  Or
 require a dangerous cast?



 Ick.
 You can't give a  transparent Object to function that expects an Object,  
 because transparent references disallow something that you are allowed  
 to do on a not transparent reference/pointer:


 pure  transparent Object foo() {
   return new Object();
 }
 void opEquals(Object o1, Object o2) {}
 void main() {
    transparent Object o = foo();
   opEquals(o, o); // not allowed
 }


Once a reference gets out of a pure function, I think it should implicitly  
cast to a non-transparent reference, or else you have a huge mess.  I  
would not like to define  transparent versions of all functions.

The question is, shouldn't you be able to compare two objects inside a  
pure function?  I would think so.  Note that the comparison inside  
opEquals does not alter purity, it's just an optimization.

Then again, I don't think opEquals would be callable inside a pure  
function, since it's not pure itself.

Would something like o is null be allowed?


 Would it be enough to just require this type of restriction in pure  
  safe
 functions?


 I don't know.

 Currently  safe is a wrong name, it means means  memorySafe. So I think  
 that currently " memorySafe" and "pure" are almost orthogonal concepts.


 safe is supposed to prevent using pointers.  But then again, the  
definition is a moving target.  I have a vague memory that really only  
pointer arithmetic is disallowed, comparison is allowed.

-Steve
Feb 23 2011
prev sibling parent reply Kevin Bealer <kevindangerbealer removedanger.gmail.com> writes:
== Quote from bearophile (bearophileHUGS lycos.com)'s article
...

 Currently you are able to write functions like:
 pure bool randomPure() {
     int[] a1 = new int[1];
     int[] a2 = new int[2];
     return a1.ptr > a2.ptr;
 }


Is it possible to fix this by disallowing using the value of a pointer,
except allowing them to be compared for equality (only)?  In other words,
within a pure function, only 'is' '!is', ==, != are permitted as tests
of pointers?  Other arithmetic forbidden... this sort of matches the Java
model where you can't fiddle with references except to check if they are
the same.

Kevin
Feb 23 2011
parent reply Kevin Bealer <kevindangerbealer removedanger.gmail.com> writes:
== Quote from Kevin Bealer (kevindangerbealer removedanger.gmail.com)'s article

 == Quote from bearophile (bearophileHUGS lycos.com)'s article
 ...

 Currently you are able to write functions like:
 pure bool randomPure() {
     int[] a1 = new int[1];
     int[] a2 = new int[2];
     return a1.ptr > a2.ptr;
 }


 Is it possible to fix this by disallowing using the value of a pointer,
 except allowing them to be compared for equality (only)?  In other words,
 within a pure function, only 'is' '!is', ==, != are permitted as tests
 of pointers?  Other arithmetic forbidden... this sort of matches the Java
 model where you can't fiddle with references except to check if they are
 the same.
 Kevin


I wanted to add something I just realized -- malloc is essentially pure if
you can't compare the pointers it returns.  Of course if you can't do either
casting or arithmetic on these pointers I'm not sure how to use malloc.

Kevin
Feb 23 2011
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/24/11, Kevin Bealer <kevindangerbealer removedanger.gmail.com> wrote:

 I wanted to add something I just realized -- malloc is essentially pure if
 you can't compare the pointers it returns.  Of course if you can't do either
 casting or arithmetic on these pointers I'm not sure how to use malloc.

 Kevin


Some people keep saying: even though having the GC is great, sometimes
it's necessary to manually manage memory. But any time that argument
is raised and someone mentions library support for manual memory
management, the opposition says "if you want to manage memory, do it
on your own, end of story."

Don't ask me why /I'd/ need that, right now I don't. But others have
raised this issue more than once.
Feb 23 2011
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Sorry for the late reply to this post, the topic is not easy and I am not sure
what the right ideas are:
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=130554

Steven Schveighoffer:


  transparent has little value outside a pure function, because the  
 compiler does not expect to be able to perform pure optimizations on  
normal functions.  Right now, the only drawback you have shown of being  
 able to compare such references is when that comparison is used as the  
 return value for a pure function.  We can't continue to enforce the  
  transparent rules when they don't prevent problems, or else the language  
 becomes too burdensome.


You can't drop  transparent (referential transparency attribute, I hae ) until
you are outside the pure subgraph of the program, otherwise:

pure int[] foo() { return new int[1]; }
pure bool rnd() { return foo().ptr > foo.ptr(); }


Do input arguments of pure functions need the  transparent attributes? Is
dropping it ouside the pure subset of the program safe/acceptable? I think so.


Regarding D purity, currently in D there is no way to tag as pure the result of
memoization applied to a pure function. Memoization is just one special case,
but it's important. To solve this the compiler has to implement the memoization
itself, or a  trusted_pure user annotation is needed... I think the second
solution is better, because there is no good way to implement the first, and
it's not so much different from the three trusted/safe/system attributes. If a
good pure-related idea gets implemented (that is, you are allowed to assign the
result of pure functions to const variables with no need of a cast) then
 trusted_pure becomes usable to punch another little hole in the const system.

Elsewhere I have tried to explain why having a good and comprehensive
implementation of purity will be important in D2:
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=132010

Bye,
bearophile
Mar 21 2011
parent bearophile <bearophileHUGS lycos.com> writes:
 Do input arguments of pure functions need the  transparent attributes? Is
dropping it ouside the pure subset of the program safe/acceptable? I think so.


Sorry, in my opinion the answers to those questions are no and yes.

Bye,
bearophile
Mar 21 2011