www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Implicit conversions through purity

reply "bearophile" <bearophileHUGS lycos.com> writes:
Is it possible and a good idea to allow code like the function 
foo2?


string foo1(in string s) pure nothrow {
     auto s2 = s.dup;
     s2[0] = 'a';
     return s2; // OK.
}

void foo2(in string s, ref string sOut) pure nothrow {
     auto s2 = s.dup;
     s2[0] = 'a';
     sOut = s2; // Error: cannot implicitly convert
}

void foo3(in string s, out string sOut) pure nothrow {
     auto s2 = s.dup;
     s2[0] = 'a';
     sOut = s2; // Error: cannot implicitly convert
}

void main() {}


Bye,
bearophile
Apr 12 2014
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, April 12, 2014 21:26:01 bearophile wrote:
 Is it possible and a good idea to allow code like the function
 foo2?
 
 
 string foo1(in string s) pure nothrow {
      auto s2 = s.dup;
      s2[0] = 'a';
      return s2; // OK.
 }
Honestly, I would have considered that to be a bug. Converting the return type to a different level of mutability based on purity is one thing. Automatically casting the return value just because the function is pure is another matter entirely. Clearly, it can work, but it seems incredibly sloppy to me. - Jonathan M Davis
Apr 12 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 Honestly, I would have considered that to be a bug. Converting 
 the return type
 to a different level of mutability based on purity is one 
 thing. Automatically
 casting the return value just because the function is pure is 
 another matter
 entirely. Clearly, it can work, but it seems incredibly sloppy 
 to me.
In foo1 D is working as designed, as this was a desired feature, it has passed the Kenji and Walter review, and it was implemented several months ago. It's a very handy way to create immutable data with pure functions and it's safe, it's safer than assumeUnique that is just a convention. Very recently Walter has further improved this feature, allowing more implicit conversion cases. So it's the opposite of a bug, it saves you from bugs in three different ways. But my question was about the successive foo functions :-) Bye, bearophile
Apr 12 2014
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, April 13, 2014 01:52:13 bearophile wrote:
 Jonathan M Davis:
 Honestly, I would have considered that to be a bug. Converting
 the return type
 to a different level of mutability based on purity is one
 thing. Automatically
 casting the return value just because the function is pure is
 another matter
 entirely. Clearly, it can work, but it seems incredibly sloppy
 to me.
In foo1 D is working as designed, as this was a desired feature, it has passed the Kenji and Walter review, and it was implemented several months ago. It's a very handy way to create immutable data with pure functions and it's safe, it's safer than assumeUnique that is just a convention. Very recently Walter has further improved this feature, allowing more implicit conversion cases. So it's the opposite of a bug, it saves you from bugs in three different ways.
Well, it's the first I've heard of it, and I certainly don't like the idea, but if it's intended and implemented, then that's the way it is, I guess. Maybe I'll come to agree after thinking about it more.
 But my question was about the successive foo functions :-)
Well, if I don't like the first example, I'm not about to be in favor of making more examples follow suite. - Jonathan M Davis
Apr 14 2014
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Apr 2014 06:37:18 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Sunday, April 13, 2014 01:52:13 bearophile wrote:
 Jonathan M Davis:
 Honestly, I would have considered that to be a bug. Converting
 the return type
 to a different level of mutability based on purity is one
 thing. Automatically
 casting the return value just because the function is pure is
 another matter
 entirely. Clearly, it can work, but it seems incredibly sloppy
 to me.
In foo1 D is working as designed, as this was a desired feature, it has passed the Kenji and Walter review, and it was implemented several months ago. It's a very handy way to create immutable data with pure functions and it's safe, it's safer than assumeUnique that is just a convention. Very recently Walter has further improved this feature, allowing more implicit conversion cases. So it's the opposite of a bug, it saves you from bugs in three different ways.
Well, it's the first I've heard of it, and I certainly don't like the idea, but if it's intended and implemented, then that's the way it is, I guess. Maybe I'll come to agree after thinking about it more.
Here is how I consider it... The foo1 function for reference:
 string foo1(in string s) pure nothrow {
      auto s2 = s.dup;
      s2[0] = 'a';
      return s2; // OK.
 }
The compiler accepts only immutable references. Any mutable references inside a pure function that only accepts immutable references can ONLY be data that is uniquely referenced from this function. In other words, there's no possible way that data is referenced outside this function, because a pure function cannot access globals. So it's logical to assume that any mutable data inside the function can be implicitly cast to immutable. In fact, I would say in general, mutable data can be cast to immutable inside any pure function that only accepts immutable parameters. However, it cannot use the mutable references after casting. This would confuse the optimizer, which thinks that immutable data is still immutable. For that reason, I would disallow out parameters from casting implicitly. e.g.: pure string mkcopy(string s) pure nothrow { return s.idup; } void foo2(in string s, ref string r, ref string r2) pure nothrow { auto s2 = s.dup; r2 = s2; auto s3 = mkcopy(r2); // should be a pure copy of s s2[0] = 'a'; // modified immutable data referenced by r2! r = mkcopy(r2); // ??? } At the line marked ???, note that the compiler has already called mkcopy on r2, which hasn't itself changed, and data r2 references is immutable. The compiler is fully allowed to change that line to: r = s3; which would mean that r != r2, even though the last line says otherwise. I think the point of restricting to the return value is not only that you are sure nothing else can refer to the data, but also that nothing else happens to the mutable copy after the cast. I wonder if scope(exit) usage could compromise the current rule... -Steve
Apr 14 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 For that reason, I would disallow out parameters from casting 
 implicitly.
If you think the proposal is not good, can you please copy what you have written here in the issue thread? https://issues.dlang.org/show_bug.cgi?id=12573 Bye, bearophile
Apr 14 2014
parent reply "Steve Teale" <steve.teale britseyeview.com> writes:
On Monday, 14 April 2014 at 15:16:07 UTC, bearophile wrote:
 Steven Schveighoffer:

 For that reason, I would disallow out parameters from casting 
 implicitly.
A number of things perplex me here. 1) If I attempt to compile foo2() more or less as presented with 2.065, the compiler tells me: Error: '_adDupT' is not nothrow Error: function 'mu.foo2' is nothrow yet may throw What version is the discussion about? 2) If I have a simple program as follows, the compiler does not complain about me altering the value of global a. import std.stdio; string a = "aaa"; void foo2(in string s, ref string sOut) pure { auto s2 = s.dup; s2[0] = 'a'; sOut = cast(string) s2; // Error: cannot implicitly convert } void main() { { foo2("xyz", a); writeln(a); } 3) Using a ref parameter for a pure function seems to me to be a clear indication of intended side effect. Wikipedia on pure functions says it's not allowed. Out does seem somehow different to me, since it's initial value is by definition throw-away. Steve
Apr 15 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Steve Teale:

 1) If I attempt to compile foo2() more or less as presented 
 with 2.065, the compiler tells me:

 Error: '_adDupT' is not nothrow
 Error: function 'mu.foo2' is nothrow yet may throw

 What version is the discussion about?
This rather fastidious limitation was finally removed in the latest D alpha (and if you look in Bugzilla you see it has created few regressions).
 2) If I have a simple program as follows, the compiler does not 
 complain about me altering the value of global a.

 import std.stdio;
 string a = "aaa";

 void foo2(in string s, ref string sOut) pure {
     auto s2 = s.dup;
     s2[0] = 'a';
     sOut = cast(string) s2; // Error: cannot implicitly convert
 }

 void main() {
 {
    foo2("xyz", a);
    writeln(a);
 }
Yes, foo2 is weakly pure, but main is not tagged as pure, so main is free to use a global reference. If you mark main pure, your code will not compile even if you comment out the writeln. D is working as designed here.
 3) Using a ref parameter for a pure function seems to me to be 
 a clear indication of intended side effect.
In D you can also have "const ref". A mutable ref makes the function "weakly pure" at best.
 Wikipedia on pure functions says it's not allowed.
D has both strongly pure and weakly pure functions (and later the compiler has added "const-ly pure" and another category, all invisible.
 Out does seem somehow different to me, since it's initial
 value is by definition throw-away.
Yes, this is true in theory. I don't know if this is also true in D in practice, because D out has some problems. Bye, bearophile
Apr 15 2014
parent reply "Steve Teale" <steve.teale britseyeview.com> writes:
On Tuesday, 15 April 2014 at 09:41:57 UTC, bearophile wrote:
 Yes, foo2 is weakly pure, but main is not tagged as pure, so 
 main is free to use a global reference. If you mark main pure, 
 your code will not compile even if you comment out the writeln. 
 D is working as designed here.


 3) Using a ref parameter for a pure function seems to me to be 
 a clear indication of intended side effect.
In D you can also have "const ref". A mutable ref makes the function "weakly pure" at best.
 Wikipedia on pure functions says it's not allowed.
D has both strongly pure and weakly pure functions (and later the compiler has added "const-ly pure" and another category, all invisible.
 Out does seem somehow different to me, since it's initial
 value is by definition throw-away.
Yes, this is true in theory. I don't know if this is also true in D in practice, because D out has some problems. Bye, bearophile
Since this is D-Learn, I can be indignant, and say that D needs to get its act together, and have a clean definition of 'pure'. What you describe is not only undocumented, but also far too complicated - pure weak nothrow dontpiss kissmyass never, and so on if the direction continues. Nontheless, thank you for your assiduous efforets to make D internally consistent. Steve
Apr 15 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Steve Teale:

 Since this is D-Learn, I can be indignant, and say that D needs 
 to get its act together, and have a clean definition of 'pure'. 
 What you describe is not only undocumented, but also far too 
 complicated - pure weak nothrow dontpiss kissmyass  never, and 
 so on if the direction continues.
There is a nice article on D purity that I suggest you to read. Unfortunately I don't remember the link. To design a system language as complex as C++/D you need lot of mathematics and theory. Andrei is good in computer science theory and in complex programming too, but Walter is less mathematically-minded and this could be visible in the design of some parts of D. Anyway, everyone is doing their best, Walter is very good and he has all my respect. And in a real-world system language a clean definition of pure (like the original pure implementation of D) is not very useful. The current definition of pure of D is good enough and it's a part very near to be completely implemented (unlike many other parts of D like synchronized, scope, SIMD, vector operations, operator overloading, dynamic libraries, GC, and more). So please don't be too much indignant, we don't have the research laboratories of Microsoft :-)
 Nontheless, thank you for your assiduous efforets to make D 
 internally consistent.
In the end I am not doing much. At best I can only hope to spot some sharp corners of the language, and ask for them to be smoothed. But it's very hard to make progress. And lately Andrei (perhaps rightfully) has raised the bar regarding the acceptable breaking changes. And people like Kenji have a mind better than mine (able to think about tens of corner cases, able to keep in mind many complex interactions, etc) even when they are sleeping :-) Bye, bearophile
Apr 15 2014
next sibling parent "Steve Teale" <steve.teale britseyeview.com> writes:
On Tuesday, 15 April 2014 at 18:02:00 UTC, bearophile wrote:
 Steve Teale:

 Since this is D-Learn, I can be indignant, and say that D 
 needs to get its act together, and have a clean definition of 
 'pure'. What you describe is not only undocumented, but also 
 far too complicated - pure weak nothrow dontpiss kissmyass 
  never, and so on if the direction continues.
There is a nice article on D purity that I suggest you to read. Unfortunately I don't remember the link. To design a system language as complex as C++/D you need lot of mathematics and theory. Andrei is good in computer science theory and in complex programming too, but Walter is less mathematically-minded and this could be visible in the design of some parts of D. Anyway, everyone is doing their best, Walter is very good and he has all my respect. And in a real-world system language a clean definition of pure (like the original pure implementation of D) is not very useful. The current definition of pure of D is good enough and it's a part very near to be completely implemented (unlike many other parts of D like synchronized, scope, SIMD, vector operations, operator overloading, dynamic libraries, GC, and more). So please don't be too much indignant, we don't have the research laboratories of Microsoft :-)
 Nontheless, thank you for your assiduous efforets to make D 
 internally consistent.
In the end I am not doing much. At best I can only hope to spot some sharp corners of the language, and ask for them to be smoothed. But it's very hard to make progress. And lately Andrei (perhaps rightfully) has raised the bar regarding the acceptable breaking changes. And people like Kenji have a mind better than mine (able to think about tens of corner cases, able to keep in mind many complex interactions, etc) even when they are sleeping :-) Bye, bearophile
Like I said, I allowed myself the luxury of being indignant because we were hidden away on D Learn. I have total respect for Walter. For one thing he is a self-taught programmer like me, though much more advanced, and I've known him on and off since the Zortech days in 1988. I don't much about Andrei and Kenji, except that they are out of my league. I guess that rather than complain that the compiler does not appear to conform to the definition, I should help by trying to improve the documentation. Anyway my view on the original subject is that the implicit conversion should be allowed for the out case, but not the ref. Steve
Apr 15 2014
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, April 15, 2014 18:01:59 bearophile wrote:
 Steve Teale:
 Since this is D-Learn, I can be indignant, and say that D needs
 to get its act together, and have a clean definition of 'pure'.
 What you describe is not only undocumented, but also far too
 complicated - pure weak nothrow dontpiss kissmyass  never, and
 so on if the direction continues.
There is a nice article on D purity that I suggest you to read. Unfortunately I don't remember the link.
http://klickverbot.at/blog/2012/05/purity-in-d/ The key thing to understand about pure though is that _all_ it guarantees is that the function cannot access mutable, global variables or call any other functions which are not pure. The only variables that it can access which are either global and guaranteed to never change their values after initialization (which generally means that they're immutable) and the ones which are passed in directly to the function or which can be accessed via the variables that are passed in. So, really, pure has _nothing_ to do with mathematical purity. It's just protecting against access to global variables. Now, that restriction allows a number of other things to be derived from that, which allows us to do cool stuff - including optimizing functions which _are_ pure in the mathematical sense. The most basic distinction would be what is typically called weak or strong purity, where strongly pure functions have immutable parameters (and thus multiple calls with the same arguments can be optimized to a single call), and weakly pure functions are pure functions which aren't strongly pure. Strongly pure functions generally can be optimized, whereas weakly pure functions cannot, but the actual distinctions can get more complicated than that, so even talking about weak vs strong purity isn't actually clear enough. So, it's certainly true that things get a bit complicated when discussing what can be inferred due to the fact that a function is pure. And it doesn't always come down to strong vs weak purity either. So, I wouldn't expect very many people to understand under what circumstances the compiler is able to do certain things with a pure function, and that's not necessarily a good thing. However, the key thing to understand with pure itself is that all it means is that the function can't access global, mutable state or call other functions which aren't pure. Pretty much everything else about it is essentially either a compiler optimization based on that information or an implicit type conversion which is made legal based on that information. Regardless, I'd advise reading David's blog post on the matter, which I linked to above. - Jonathan M Davis
Apr 16 2014
prev sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Jonathan M Davis"  wrote in message 
news:mailman.112.1397351369.5999.digitalmars-d-learn puremagic.com...

 Honestly, I would have considered that to be a bug. Converting the return 
 type
 to a different level of mutability based on purity is one thing. 
 Automatically
 casting the return value just because the function is pure is another 
 matter
 entirely. Clearly, it can work, but it seems incredibly sloppy to me.
It's not a bug, and it's not another matter entirely - it's the same thing as converting a call expression, just on the inside instead of the outside.
Apr 13 2014
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, April 13, 2014 21:21:02 Daniel Murphy wrote:
 "Jonathan M Davis"  wrote in message
 news:mailman.112.1397351369.5999.digitalmars-d-learn puremagic.com...
 
 Honestly, I would have considered that to be a bug. Converting the return
 type
 to a different level of mutability based on purity is one thing.
 Automatically
 casting the return value just because the function is pure is another
 matter
 entirely. Clearly, it can work, but it seems incredibly sloppy to me.
It's not a bug, and it's not another matter entirely - it's the same thing as converting a call expression, just on the inside instead of the outside.
Doing the conversion on the caller's side enables code that wouldn't otherwise work - particularly if you're not intimately familiar with what guarantees pure really gives you. Doing the conversion inside the function doesn't really buy you much of anything IMHO and promotes being lazy with types. It may not be a big deal, but I don't think that it's really a good idea either. So, while from the perspective of what the compiler can guarantee, it may be the same from both sides, I don't think that it's the same at all with regards to how it affects the programmer or what code they need to write (or should write). - Jonathan M Davis
Apr 14 2014
prev sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"bearophile"  wrote in message news:hxdrbyqrhwvuochlhclc forum.dlang.org... 

 Is it possible and a good idea to allow code like the function 
 foo2?
It seems reasonable.
Apr 13 2014
parent "bearophile" <bearophileHUGS lycos.com> writes:
Daniel Murphy:

 It seems reasonable.
OK, added: https://issues.dlang.org/show_bug.cgi?id=12573 Bye, bearophile
Apr 13 2014