• bearophile (20/20) Apr 12 2014 Is it possible and a good idea to allow code like the function
• Jonathan M Davis (6/15) Apr 12 2014 Honestly, I would have considered that to be a bug. Converting the retur...
• bearophile (12/20) Apr 12 2014 In foo1 D is working as designed, as this was a desired feature,
• Jonathan M Davis (7/26) Apr 14 2014 Well, it's the first I've heard of it, and I certainly don't like the id...
• Steven Schveighoffer (35/63) Apr 14 2014 Here is how I consider it...
• bearophile (6/8) Apr 14 2014 If you think the proposal is not good, can you please copy what
• Steve Teale (26/29) Apr 15 2014 A number of things perplex me here.
• bearophile (17/41) Apr 15 2014 This rather fastidious limitation was finally removed in the
• Steve Teale (9/27) Apr 15 2014 Since this is D-Learn, I can be indignant, and say that D needs
• bearophile (27/34) Apr 15 2014 There is a nice article on D purity that I suggest you to read.
• Steve Teale (13/47) Apr 15 2014 Like I said, I allowed myself the luxury of being indignant
• Jonathan M Davis (33/42) Apr 16 2014 http://klickverbot.at/blog/2012/05/purity-in-d/
• Daniel Murphy (4/11) Apr 13 2014 It's not a bug, and it's not another matter entirely - it's the same thi...
• Jonathan M Davis (11/24) Apr 14 2014 Doing the conversion on the caller's side enables code that wouldn't oth...
• Daniel Murphy (2/4) Apr 13 2014 It seems reasonable.
• bearophile (5/6) Apr 13 2014 OK, added:

"bearophile" <bearophileHUGS lycos.com> writes:

Is it possible and a good idea to allow code like the function 
foo2?


string foo1(in string s) pure nothrow {
     auto s2 = s.dup;
     s2[0] = 'a';
     return s2; // OK.
}

void foo2(in string s, ref string sOut) pure nothrow {
     auto s2 = s.dup;
     s2[0] = 'a';
     sOut = s2; // Error: cannot implicitly convert
}

void foo3(in string s, out string sOut) pure nothrow {
     auto s2 = s.dup;
     s2[0] = 'a';
     sOut = s2; // Error: cannot implicitly convert
}

void main() {}


Bye,
bearophile

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Saturday, April 12, 2014 21:26:01 bearophile wrote:
 Is it possible and a good idea to allow code like the function
 foo2?
 
 
 string foo1(in string s) pure nothrow {
      auto s2 = s.dup;
      s2[0] = 'a';
      return s2; // OK.
 }

Honestly, I would have considered that to be a bug. Converting the return type 
to a different level of mutability based on purity is one thing. Automatically 
casting the return value just because the function is pure is another matter 
entirely. Clearly, it can work, but it seems incredibly sloppy to me.

- Jonathan M Davis

"bearophile" <bearophileHUGS lycos.com> writes:

Jonathan M Davis:

 Honestly, I would have considered that to be a bug. Converting 
 the return type
 to a different level of mutability based on purity is one 
 thing. Automatically
 casting the return value just because the function is pure is 
 another matter
 entirely. Clearly, it can work, but it seems incredibly sloppy 
 to me.

In foo1 D is working as designed, as this was a desired feature,
it has passed the Kenji and Walter review, and it was implemented
several months ago. It's a very handy way to create immutable
data with pure functions and it's safe, it's safer than
assumeUnique that is just a convention. Very recently Walter has
further improved this feature, allowing more implicit conversion
cases. So it's the opposite of a bug, it saves you from bugs in
three different ways.

But my question was about the successive foo functions :-)

Bye,
bearophile

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Sunday, April 13, 2014 01:52:13 bearophile wrote:
 Jonathan M Davis:
 Honestly, I would have considered that to be a bug. Converting
 the return type
 to a different level of mutability based on purity is one
 thing. Automatically
 casting the return value just because the function is pure is
 another matter
 entirely. Clearly, it can work, but it seems incredibly sloppy
 to me.

 
 In foo1 D is working as designed, as this was a desired feature,
 it has passed the Kenji and Walter review, and it was implemented
 several months ago. It's a very handy way to create immutable
 data with pure functions and it's safe, it's safer than
 assumeUnique that is just a convention. Very recently Walter has
 further improved this feature, allowing more implicit conversion
 cases. So it's the opposite of a bug, it saves you from bugs in
 three different ways.

Well, it's the first I've heard of it, and I certainly don't like the idea, 
but if it's intended and implemented, then that's the way it is, I guess. 
Maybe I'll come to agree after thinking about it more.

 But my question was about the successive foo functions :-)

Well, if I don't like the first example, I'm not about to be in favor of 
making more examples follow suite.

- Jonathan M Davis

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Mon, 14 Apr 2014 06:37:18 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Sunday, April 13, 2014 01:52:13 bearophile wrote:
 Jonathan M Davis:
 Honestly, I would have considered that to be a bug. Converting
 the return type
 to a different level of mutability based on purity is one
 thing. Automatically
 casting the return value just because the function is pure is
 another matter
 entirely. Clearly, it can work, but it seems incredibly sloppy
 to me.

 In foo1 D is working as designed, as this was a desired feature,
 it has passed the Kenji and Walter review, and it was implemented
 several months ago. It's a very handy way to create immutable
 data with pure functions and it's safe, it's safer than
 assumeUnique that is just a convention. Very recently Walter has
 further improved this feature, allowing more implicit conversion
 cases. So it's the opposite of a bug, it saves you from bugs in
 three different ways.

 Well, it's the first I've heard of it, and I certainly don't like the  
 idea,
 but if it's intended and implemented, then that's the way it is, I guess.
 Maybe I'll come to agree after thinking about it more.

Here is how I consider it...

The foo1 function for reference:

 string foo1(in string s) pure nothrow {
      auto s2 = s.dup;
      s2[0] = 'a';
      return s2; // OK.
 }

The compiler accepts only immutable references. Any mutable references  
inside a pure function that only accepts immutable references can ONLY be  
data that is uniquely referenced from this function. In other words,  
there's no possible way that data is referenced outside this function,  
because a pure function cannot access globals.

So it's logical to assume that any mutable data inside the function can be  
implicitly cast to immutable. In fact, I would say in general, mutable  
data can be cast to immutable inside any pure function that only accepts  
immutable parameters. However, it cannot use the mutable references after  
casting. This would confuse the optimizer, which thinks that immutable  
data is still immutable.

For that reason, I would disallow out parameters from casting implicitly.

e.g.:

pure string mkcopy(string s) pure nothrow { return s.idup; }

void foo2(in string s, ref string r, ref string r2) pure nothrow {
     auto s2 = s.dup;
     r2 = s2;
     auto s3 = mkcopy(r2); // should be a pure copy of s
     s2[0] = 'a'; // modified immutable data referenced by r2!
     r = mkcopy(r2); // ???
}

At the line marked ???, note that the compiler has already called mkcopy  
on r2, which hasn't itself changed, and data r2 references is immutable.  
The compiler is fully allowed to change that line to:

r = s3;

which would mean that r != r2, even though the last line says otherwise.

I think the point of restricting to the return value is not only that you  
are sure nothing else can refer to the data, but also that nothing else  
happens to the mutable copy after the cast.

I wonder if scope(exit) usage could compromise the current rule...

-Steve

"bearophile" <bearophileHUGS lycos.com> writes:

Steven Schveighoffer:

 For that reason, I would disallow out parameters from casting 
 implicitly.

If you think the proposal is not good, can you please copy what 
you have written here in the issue thread?
https://issues.dlang.org/show_bug.cgi?id=12573

Bye,
bearophile

"Steve Teale" <steve.teale britseyeview.com> writes:

On Monday, 14 April 2014 at 15:16:07 UTC, bearophile wrote:
 Steven Schveighoffer:

 For that reason, I would disallow out parameters from casting 
 implicitly.


A number of things perplex me here.

1) If I attempt to compile foo2() more or less as presented with 
2.065, the compiler tells me:

Error: '_adDupT' is not nothrow
Error: function 'mu.foo2' is nothrow yet may throw

What version is the discussion about?

2) If I have a simple program as follows, the compiler does not 
complain about me altering the value of global a.

import std.stdio;
string a = "aaa";

void foo2(in string s, ref string sOut) pure {
     auto s2 = s.dup;
     s2[0] = 'a';
     sOut = cast(string) s2; // Error: cannot implicitly convert
}

void main() {
{
    foo2("xyz", a);
    writeln(a);
}

3) Using a ref parameter for a pure function seems to me to be a 
clear indication of intended side effect. Wikipedia on pure 
functions says it's not allowed. Out does seem somehow different 
to me, since it's initial value is by definition throw-away.

Steve

"bearophile" <bearophileHUGS lycos.com> writes:

Steve Teale:

 1) If I attempt to compile foo2() more or less as presented 
 with 2.065, the compiler tells me:

 Error: '_adDupT' is not nothrow
 Error: function 'mu.foo2' is nothrow yet may throw

 What version is the discussion about?

This rather fastidious limitation was finally removed in the 
latest D alpha (and if you look in Bugzilla you see it has 
created few regressions).


 2) If I have a simple program as follows, the compiler does not 
 complain about me altering the value of global a.

 import std.stdio;
 string a = "aaa";

 void foo2(in string s, ref string sOut) pure {
     auto s2 = s.dup;
     s2[0] = 'a';
     sOut = cast(string) s2; // Error: cannot implicitly convert
 }

 void main() {
 {
    foo2("xyz", a);
    writeln(a);
 }

Yes, foo2 is weakly pure, but main is not tagged as pure, so main 
is free to use a global reference. If you mark main pure, your 
code will not compile even if you comment out the writeln. D is 
working as designed here.


 3) Using a ref parameter for a pure function seems to me to be 
 a clear indication of intended side effect.

In D you can also have "const ref". A mutable ref makes the 
function "weakly pure" at best.


 Wikipedia on pure functions says it's not allowed.

D has both strongly pure and weakly pure functions (and later the 
compiler has added "const-ly pure" and another category, all 
invisible.


 Out does seem somehow different to me, since it's initial
 value is by definition throw-away.

Yes, this is true in theory. I don't know if this is also true in 
D in practice, because D out has some problems.

Bye,
bearophile

"Steve Teale" <steve.teale britseyeview.com> writes:

On Tuesday, 15 April 2014 at 09:41:57 UTC, bearophile wrote:
 Yes, foo2 is weakly pure, but main is not tagged as pure, so 
 main is free to use a global reference. If you mark main pure, 
 your code will not compile even if you comment out the writeln. 
 D is working as designed here.


 3) Using a ref parameter for a pure function seems to me to be 
 a clear indication of intended side effect.

 In D you can also have "const ref". A mutable ref makes the 
 function "weakly pure" at best.


 Wikipedia on pure functions says it's not allowed.

 D has both strongly pure and weakly pure functions (and later 
 the compiler has added "const-ly pure" and another category, 
 all invisible.


 Out does seem somehow different to me, since it's initial
 value is by definition throw-away.

 Yes, this is true in theory. I don't know if this is also true 
 in D in practice, because D out has some problems.

 Bye,
 bearophile

Since this is D-Learn, I can be indignant, and say that D needs 
to get its act together, and have a clean definition of 'pure'. 
What you describe is not only undocumented, but also far too 
complicated - pure weak nothrow dontpiss kissmyass  never, and so 
on if the direction continues.

Nontheless, thank you for your assiduous efforets to make D 
internally consistent.

Steve

"bearophile" <bearophileHUGS lycos.com> writes:

Steve Teale:

 Since this is D-Learn, I can be indignant, and say that D needs 
 to get its act together, and have a clean definition of 'pure'. 
 What you describe is not only undocumented, but also far too 
 complicated - pure weak nothrow dontpiss kissmyass  never, and 
 so on if the direction continues.

There is a nice article on D purity that I suggest you to read. 
Unfortunately I don't remember the link.

To design a system language as complex as C++/D you need lot of 
mathematics and theory. Andrei is good in computer science theory 
and in complex programming too, but Walter is less 
mathematically-minded and this could be visible in the design of 
some parts of D. Anyway, everyone is doing their best, Walter is 
very good and he has all my respect. And in a real-world system 
language a clean definition of pure (like the original pure 
implementation of D) is not very useful. The current definition 
of pure of D is good enough and it's a part very near to be 
completely implemented (unlike many other parts of D like 
synchronized, scope, SIMD, vector operations, operator 
overloading, dynamic libraries, GC, and more). So please don't be 
too much indignant, we don't have the research laboratories of 
Microsoft :-)


 Nontheless, thank you for your assiduous efforets to make D 
 internally consistent.

In the end I am not doing much. At best I can only hope to spot 
some sharp corners of the language, and ask for them to be 
smoothed. But it's very hard to make progress. And lately Andrei 
(perhaps rightfully) has raised the bar regarding the acceptable 
breaking changes. And people like Kenji have a mind better than 
mine (able to think about tens of corner cases, able to keep in 
mind many complex interactions, etc) even when they are sleeping 
:-)

Bye,
bearophile

"Steve Teale" <steve.teale britseyeview.com> writes:

On Tuesday, 15 April 2014 at 18:02:00 UTC, bearophile wrote:
 Steve Teale:

 Since this is D-Learn, I can be indignant, and say that D 
 needs to get its act together, and have a clean definition of 
 'pure'. What you describe is not only undocumented, but also 
 far too complicated - pure weak nothrow dontpiss kissmyass 
  never, and so on if the direction continues.

 There is a nice article on D purity that I suggest you to read. 
 Unfortunately I don't remember the link.

 To design a system language as complex as C++/D you need lot of 
 mathematics and theory. Andrei is good in computer science 
 theory and in complex programming too, but Walter is less 
 mathematically-minded and this could be visible in the design 
 of some parts of D. Anyway, everyone is doing their best, 
 Walter is very good and he has all my respect. And in a 
 real-world system language a clean definition of pure (like the 
 original pure implementation of D) is not very useful. The 
 current definition of pure of D is good enough and it's a part 
 very near to be completely implemented (unlike many other parts 
 of D like synchronized, scope, SIMD, vector operations, 
 operator overloading, dynamic libraries, GC, and more). So 
 please don't be too much indignant, we don't have the research 
 laboratories of Microsoft :-)


 Nontheless, thank you for your assiduous efforets to make D 
 internally consistent.

 In the end I am not doing much. At best I can only hope to spot 
 some sharp corners of the language, and ask for them to be 
 smoothed. But it's very hard to make progress. And lately 
 Andrei (perhaps rightfully) has raised the bar regarding the 
 acceptable breaking changes. And people like Kenji have a mind 
 better than mine (able to think about tens of corner cases, 
 able to keep in mind many complex interactions, etc) even when 
 they are sleeping :-)

 Bye,
 bearophile

Like I said, I allowed myself the luxury of being indignant 
because we were hidden away on D Learn. I have total respect for 
Walter. For one thing he is a self-taught programmer like me, 
though much more advanced, and I've known him on and off since 
the Zortech days in 1988. I don't much about Andrei and Kenji, 
except that they are out of my league.

I guess that rather than complain that the compiler does not 
appear to conform to the definition, I should help by trying to 
improve the documentation.

Anyway my view on the original subject is that the implicit 
conversion should be allowed for the out case, but not the ref.

Steve

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Tuesday, April 15, 2014 18:01:59 bearophile wrote:
 Steve Teale:
 Since this is D-Learn, I can be indignant, and say that D needs
 to get its act together, and have a clean definition of 'pure'.
 What you describe is not only undocumented, but also far too
 complicated - pure weak nothrow dontpiss kissmyass  never, and
 so on if the direction continues.

 
 There is a nice article on D purity that I suggest you to read.
 Unfortunately I don't remember the link.

http://klickverbot.at/blog/2012/05/purity-in-d/

The key thing to understand about pure though is that _all_ it guarantees is 
that the function cannot access mutable, global variables or call any other 
functions which are not pure. The only variables that it can access which are 
either global and guaranteed to never change their values after initialization 
(which generally means that they're immutable) and the ones which are passed 
in directly to the function or which can be accessed via the variables that 
are passed in.

So, really, pure has _nothing_ to do with mathematical purity. It's just 
protecting against access to global variables. Now, that restriction allows a 
number of other things to be derived from that, which allows us to do cool 
stuff - including optimizing functions which _are_ pure in the mathematical 
sense. The most basic distinction would be what is typically called weak or 
strong purity, where strongly pure functions have immutable parameters (and 
thus multiple calls with the same arguments can be optimized to a single 
call), and weakly pure functions are pure functions which aren't strongly 
pure. Strongly pure functions generally can be optimized, whereas weakly pure 
functions cannot, but the actual distinctions can get more complicated than 
that, so even talking about weak vs strong purity isn't actually clear enough.

So, it's certainly true that things get a bit complicated when discussing what 
can be inferred due to the fact that a function is pure. And it doesn't always 
come down to strong vs weak purity either. So, I wouldn't expect very many 
people to understand under what circumstances the compiler is able to do 
certain things with a pure function, and that's not necessarily a good thing.

However, the key thing to understand with pure itself is that all it means is 
that the function can't access global, mutable state or call other functions 
which aren't pure. Pretty much everything else about it is essentially either 
a compiler optimization based on that information or an implicit type 
conversion which is made legal based on that information.

Regardless, I'd advise reading David's blog post on the matter, which I linked 
to above.

- Jonathan M Davis

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"Jonathan M Davis"  wrote in message 
news:mailman.112.1397351369.5999.digitalmars-d-learn puremagic.com...

 Honestly, I would have considered that to be a bug. Converting the return 
 type
 to a different level of mutability based on purity is one thing. 
 Automatically
 casting the return value just because the function is pure is another 
 matter
 entirely. Clearly, it can work, but it seems incredibly sloppy to me.

It's not a bug, and it's not another matter entirely - it's the same thing 
as converting a call expression, just on the inside instead of the outside. 

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Sunday, April 13, 2014 21:21:02 Daniel Murphy wrote:
 "Jonathan M Davis"  wrote in message
 news:mailman.112.1397351369.5999.digitalmars-d-learn puremagic.com...
 
 Honestly, I would have considered that to be a bug. Converting the return
 type
 to a different level of mutability based on purity is one thing.
 Automatically
 casting the return value just because the function is pure is another
 matter
 entirely. Clearly, it can work, but it seems incredibly sloppy to me.

 
 It's not a bug, and it's not another matter entirely - it's the same thing
 as converting a call expression, just on the inside instead of the outside.

Doing the conversion on the caller's side enables code that wouldn't otherwise 
work - particularly if you're not intimately familiar with what guarantees 
pure really gives you. Doing the conversion inside the function doesn't really 
buy you much of anything IMHO and promotes being lazy with types. It may not 
be a big deal, but I don't think that it's really a good idea either.

So, while from the perspective of what the compiler can guarantee, it may be 
the same from both sides, I don't think that it's the same at all with regards 
to how it affects the programmer or what code they need to write (or should 
write).

- Jonathan M Davis

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"bearophile"  wrote in message news:hxdrbyqrhwvuochlhclc forum.dlang.org... 

 Is it possible and a good idea to allow code like the function 
 foo2?

It seems reasonable.

"bearophile" <bearophileHUGS lycos.com> writes:

Daniel Murphy:

 It seems reasonable.

OK, added:
https://issues.dlang.org/show_bug.cgi?id=12573

Bye,
bearophile