www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Time for Phobos CTFE-ability unittests...right? RIGHT?

reply "Nick Sabalausky" <a a.a> writes:
Yet again, a new DMD release has broken my code, and other people's code, 
too, just because of Phobos functions loosing their CTFE-ability. (strip(), 
toLower(), etc... And yes, I did bring up the strip() regression on the beta 
list, to no effect.)

We praise and promote the hell out of CTFE (and rightly so) for being able 
to take ordinary functions and run them at compile-time. Except, with every 
new release, I become more and more convinced that Phobos should *not* be 
used in CTFE, because I know from experience that if I do, chances are it 
will soon break. So we have to maintain separate CTFE-guaranteed versions of 
whatever we need from Phobos. Obviously, aside from being a royal pain in 
the ass,  that completely defeats the point of CTFE. Worse still, it makes 
us look really, really bad when we go out and promote a CTFE that just plain 
doesn't work as advertised when it comes the language's own standard 
library.

Granted, I *completely* understand and sympathize with the practical issues 
involved (Phobos being heavily in-development, CTFE itself undergoing big 
improvements, etc...) So I fully agree it made perfect sense for this 
"bleeding-edge/unstable" branch of D to not concern itself with CTFE-ability 
regressions in Phobos just yet...

BUT...*Now* D2 has been declared the "main" version of D, suitable and 
recommended for new projects, and is being promoted as such. That changes 
the situation. Our old, previously sensible, approach to Phobos CTFE-ability 
has now become "Breaking regressions in each release."

Therefore, I think *now* is the time when Phobos needs to start having 
regression tests for CTFE-ability. If something doesn't work, then hell, at 
least a quick-and-dirty if(_ctfe) branch would at least be better than 
outright breakage. And if there are worries about that hindering improvement 
of DMD's CTFE mechanism, then a version identifier could be added to turn 
off the CTFE-workaround codepaths.

I realize it's not exactly helpful to say "Hey, you should do X!" instead of 
just simply pitching in and helping. But I'm hoping we can at least agree 
that we've reached a point where it's appropriate, and not premature, for 
this to be done. (One step at a time, right?)
Jul 12 2011
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
I'd like to point out that *normal code* in Phobos is losing
functionality far too often too, like replace() no longer working
on immutable strings as of the last release.

Generalized templates are great, but not at the cost of existing
functionality!
Jul 12 2011
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-12 15:50, Adam D. Ruppe wrote:

 I'd like to point out that *normal code* in Phobos is losing
 functionality far too often too, like replace() no longer working
 on immutable strings as of the last release.
 
 Generalized templates are great, but not at the cost of existing
 functionality!


Please make sure that you report all such regressions. In some cases, they 
probably can't be fixed (especially if it's related to a compiler change which 
fixes something which shouldn't have worked before), but in a lot of such 
cases, it's likely simply because there wasn't a test case for it, and it 
wasn't caught.

As for immutable arrays, they're generally not going to work with range-based 
functions because of how templates work ( 
http://d.puremagic.com/issues/show_bug.cgi?id=6148 ). But if the change 
requested in http://d.puremagic.com/issues/show_bug.cgi?id=6289 is merged in, 
then slicing immutable arrays should work. Arguably however, most functions in 
std.array should work with immutable arrays (since it _is_ std._array_ after 
all), and if you can assume arrays rather than generic ranges, you can 
generally make them work with immutable arrays. So, replace may very well be 
fixed to work with immutable arrays. But if you don't point out such 
regressions, they don't necessarily get caught.

So, please point out such regressions in Phobos. That way they can be fixed 
and unit tests can be added to help ensure that they don't happen again.

- Jonathan M Davis
Jul 12 2011
prev sibling next sibling parent reply bcs <bcs example.com> writes:
== Quote from Adam D. Ruppe (destructionator gmail.com)'s article

 I'd like to point out that *normal code* in Phobos is losing
 functionality far too often too, like replace() no longer working
 on immutable strings as of the last release.
 Generalized templates are great, but not at the cost of existing
 functionality!


The answer to both problems is to have a test suit that is very easy
for the average user to add code to that's run along with the current
auto-tester. What I'm thinking of is somthing like codepad.org where
anyone can submit a code sample and where "interested parties" get informed
when the results for them change. With that in place, when
some one hits a regression it can be added to the (or a) regression
suit with just a few clicks. With a bit more work, you could even let
people "go back in time" and show that a test case user to run.

Heck, I was looking into building exactly that a few months back but
real life got in the way.
Jul 12 2011
parent reply Brad Roberts <braddr puremagic.com> writes:
On 7/12/2011 7:14 PM, bcs wrote:

 == Quote from Adam D. Ruppe (destructionator gmail.com)'s article

 I'd like to point out that *normal code* in Phobos is losing
 functionality far too often too, like replace() no longer working
 on immutable strings as of the last release.
 Generalized templates are great, but not at the cost of existing
 functionality!


 
 The answer to both problems is to have a test suit that is very easy
 for the average user to add code to that's run along with the current
auto-tester. What I'm thinking of is somthing like codepad.org where
 anyone can submit a code sample and where "interested parties" get informed
when the results for them change. With that in place, when
 some one hits a regression it can be added to the (or a) regression
 suit with just a few clicks. With a bit more work, you could even let
 people "go back in time" and show that a test case user to run.
 
 Heck, I was looking into building exactly that a few months back but
 real life got in the way.


Why separate at all?  Add tests to phobos directly like any other unit test.
Jul 12 2011
parent bcs <bcs google.com> writes:
== Quote from Brad Roberts (braddr puremagic.com)'s article

 On 7/12/2011 7:14 PM, bcs wrote:

 == Quote from Adam D. Ruppe (destructionator gmail.com)'s article

 I'd like to point out that *normal code* in Phobos is losing
 functionality far too often too, like replace() no longer working
 on immutable strings as of the last release.
 Generalized templates are great, but not at the cost of existing
 functionality!


 The answer to both problems is to have a test suit that is very




easy

 for the average user to add code to that's run along with the




current auto-tester. What I'm thinking of is somthing like codepad.org
where

 anyone can submit a code sample and where "interested parties" get




informed when the results for them change. With that in place, when

 some one hits a regression it can be added to the (or a)




regression

 suit with just a few clicks. With a bit more work, you could even




let

 people "go back in time" and show that a test case user to run.

 Heck, I was looking into building exactly that a few months back




but

 real life got in the way.


 Why separate at all?  Add tests to phobos directly like any other


unit test.

Because that requires commit privileges, having git installed and
about a dozen other things. I would like to be able to past some code
into a web page, tweak it till it shows what I want and post a URL.

As for moving stuff from there into phobos; well that might work for
selected cases, but for the system I'm thinking of you may end up with
a selection of "interesting" cases (ones the devs care about) that's
several times the size of phobos and a corpus of "active" test (ones
that someone cares about) an order of magnitude larger than that. As
long as the search and monitor tools to handle it are done well, the
more test the better I say.
Jul 12 2011
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday 12 July 2011 22:50:14 Adam D. Ruppe wrote:

 I'd like to point out that *normal code* in Phobos is losing
 functionality far too often too, like replace() no longer working
 on immutable strings as of the last release.
 
 Generalized templates are great, but not at the cost of existing
 functionality!


https://github.com/D-Programming-Language/phobos/pull/143

- Jonathan M Davis
Jul 12 2011
parent Adam Ruppe <destructionator gmail.com> writes:
 https://github.com/D-Programming-Language/phobos/pull/143


Excellent! Thank you.
Jul 13 2011
prev sibling next sibling parent reply Trass3r <un known.com> writes:
I totally agree.

At least simple functions like string manipulation need to be available at  
compile-time to use string mixins properly.

2.051 broke replace(), then it was fixed.
2.052 broke it again by moving it into std.array and changing the  
implementation.
Now with 2.054 toUpper() is broken.

If you have a library that extensively uses CTFE this is a major problem.
Jul 12 2011
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Last time I brought this issue up in bugzilla it was shot down with
"We don't guarantee and don't have to guarantee functions will always
be CTFE-able between releases".

I end up having to put initializers in a module ctor. Not nice!
Jul 12 2011
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Andrej Mitrovic" <andrej.mitrovich gmail.com> wrote in message 
news:mailman.1574.1310511524.14074.digitalmars-d puremagic.com...

 Last time I brought this issue up in bugzilla it was shot down with
 "We don't guarantee and don't have to guarantee functions will always
 be CTFE-able between releases".


Yea, I've seen statements to that effect, too. It was a fairly reasonable 
stance when D2 was the "unstable" branch, but now that D2 is the "main" D, 
it ultimately means that you can't use Phobos in CTFE because it'll likely 
break. And that's a *huge* blow to CTFE in general.
Jul 12 2011
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I don't understand what strip() could be doing to break CTFE anyway?
Jul 12 2011
parent reply Johann MacDonagh <johann.macdonagh..no spam..gmail.com> writes:
On 7/12/2011 7:17 PM, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


I believe a lot of the std.string functionality was modified to use 
routines in druntime in this latest release. The default sc.ini links to 
a static druntime lib as opposed to compiling the source. That means 
CTFE won't have the source for, in strip's example, _aApplycd2 
(rt/aApply.d in druntime). Fixing this (and adding in support for 
std.intrinsic) should fix a ton of bugs where a "small" change in Phobos 
kills CTFE.
Jul 12 2011
parent reply Trass3r <un known.com> writes:
Am 13.07.2011, 01:24 Uhr, schrieb Johann MacDonagh  
<johann.macdonagh..no spam..gmail.com>:


 On 7/12/2011 7:17 PM, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


 I believe a lot of the std.string functionality was modified to use  
 routines in druntime in this latest release. The default sc.ini links to  
 a static druntime lib as opposed to compiling the source. That means  
 CTFE won't have the source for, in strip's example, _aApplycd2  
 (rt/aApply.d in druntime). Fixing this (and adding in support for  
 std.intrinsic) should fix a ton of bugs where a "small" change in Phobos  
 kills CTFE.


That's true. I had several of these "no source available" errors.

Though I think toUpper crashed at:

S toUpper(S)(S s)  trusted pure
     if(isSomeString!S)
{
     foreach (i, dchar cOuter; s)   <---
     {
Jul 12 2011
parent Johann MacDonagh <johann.macdonagh..no spam..gmail.com> writes:
On 7/12/2011 7:35 PM, Trass3r wrote:

 Am 13.07.2011, 01:24 Uhr, schrieb Johann MacDonagh
 <johann.macdonagh..no spam..gmail.com>:


 On 7/12/2011 7:17 PM, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


 I believe a lot of the std.string functionality was modified to use
 routines in druntime in this latest release. The default sc.ini links
 to a static druntime lib as opposed to compiling the source. That
 means CTFE won't have the source for, in strip's example, _aApplycd2
 (rt/aApply.d in druntime). Fixing this (and adding in support for
 std.intrinsic) should fix a ton of bugs where a "small" change in
 Phobos kills CTFE.


 That's true. I had several of these "no source available" errors.

 Though I think toUpper crashed at:

 S toUpper(S)(S s)  trusted pure
 if(isSomeString!S)
 {
 foreach (i, dchar cOuter; s) <---
 {


That's exactly the issue. When the compiler hits a foreach over a string 
that wants to convert its character type to another, it has to insert 
special magic to make that work (UTF decoding, etc...). Clearly this 
stuff shouldn't be in phobos, so it's in druntime. Unfortunately, the 
compiler doesn't have access to druntime source by default.

I'm not sure how it was doing it before. This worked in 2.053.
Jul 12 2011
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-12 16:17, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


Don has been making huge changes to CTFE. Stuff that didn't used to compile, 
now complie. Stuff which compiled but shouldn't have now doesn't compile. 
There's probably stuff which used to compile and should still compile which 
doesn't compile now too. But with all of those changes, I'm not sure that it's 
at all reasonable to expect CTFE-ability to be stable. It should be heading in 
that direction, but I'm not sure how stable Don considers it. Certainly, strip 
could be failing for a perfectly legitimate reason, or it could be a bug. I 
have no idea. But with all of the changes that have been being made to CTFE, 
I'm not at all surprised if stuff has quit working. There's probably more that 
works now that didn't before, but with all of the recent changes, breakage 
doesn't surprise me one bit.

Having tests in Phobos for CTFE would catch many breakages, but if Don isn't 
yet guaranteeing what can and can't be CTFEed by the compiler, then such 
breakages could easily be because of fixes which made it so that stuff which 
wasn't supposed to compiled but did stopped compiling. So, until Don thinks 
that what he's been doing to CTFE is appropriately stable and can make 
guarantees about what will and won't be CTFEable as far as language features 
go, then Phobos can't make any guarantees about CTFEability.

So, basically, a lot of CTFE changes have been happening, and Don has said 
pretty much said that we're not currently making guarantees about what's 
CTFEable and what isn't. And until the changes stabilize and Don is willing to 
make guarantees, Phobos can't guarantee anything about CTFE.

- Jonathan M Davis
Jul 12 2011
next sibling parent David Nadlinger <see klickverbot.at> writes:
On 7/13/11 1:29 AM, Jonathan M Davis wrote:

 So, basically, a lot of CTFE changes have been happening, and Don has said
 pretty much said that we're not currently making guarantees about what's
 CTFEable and what isn't. And until the changes stabilize and Don is willing to
 make guarantees, Phobos can't guarantee anything about CTFE.


Yes, but still I think a comprehensive regression test suite would be 
helpful, also for Don, because it would show him, well, if there are any 
regressions. I mean, it's not like Don would say, »Hey I just work on 
CTFE stuff that is fun to do and don't care about the rest«…

David
Jul 12 2011
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1576.1310513383.14074.digitalmars-d puremagic.com...

 On 2011-07-12 16:17, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


 Don has been making huge changes to CTFE. Stuff that didn't used to 
 compile,
 now complie. Stuff which compiled but shouldn't have now doesn't compile.
 There's probably stuff which used to compile and should still compile 
 which
 doesn't compile now too. But with all of those changes, I'm not sure that 
 it's
 at all reasonable to expect CTFE-ability to be stable. It should be 
 heading in
 that direction, but I'm not sure how stable Don considers it. Certainly, 
 strip
 could be failing for a perfectly legitimate reason, or it could be a bug. 
 I
 have no idea. But with all of the changes that have been being made to 
 CTFE,
 I'm not at all surprised if stuff has quit working. There's probably more 
 that
 works now that didn't before, but with all of the recent changes, breakage
 doesn't surprise me one bit.


I definitely expected that in 2.053, since that's the version that had the 
CTFE overhaul. But it's not being re-overhauled in each version after 
2.053 - just bugfixes and added features. So at this point, I think it's 
reasonable to expect that any CTFE changes that break Phobos code should be 
fixed before release, even if only a temporary fix, in either the CTFE 
engine or in the Phobos code that was broken.
Jul 12 2011
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-12 17:16, Nick Sabalausky wrote:

 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1576.1310513383.14074.digitalmars-d puremagic.com...
 

 On 2011-07-12 16:17, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


 
 Don has been making huge changes to CTFE. Stuff that didn't used to
 compile,
 now complie. Stuff which compiled but shouldn't have now doesn't compile.
 There's probably stuff which used to compile and should still compile
 which
 doesn't compile now too. But with all of those changes, I'm not sure that
 it's
 at all reasonable to expect CTFE-ability to be stable. It should be
 heading in
 that direction, but I'm not sure how stable Don considers it. Certainly,
 strip
 could be failing for a perfectly legitimate reason, or it could be a bug.
 I
 have no idea. But with all of the changes that have been being made to
 CTFE,
 I'm not at all surprised if stuff has quit working. There's probably more
 that
 works now that didn't before, but with all of the recent changes,
 breakage doesn't surprise me one bit.


 
 I definitely expected that in 2.053, since that's the version that had the
 CTFE overhaul. But it's not being re-overhauled in each version after
 2.053 - just bugfixes and added features. So at this point, I think it's
 reasonable to expect that any CTFE changes that break Phobos code should be
 fixed before release, even if only a temporary fix, in either the CTFE
 engine or in the Phobos code that was broken.


Well, regardless of whether CTFE is still having a major overhaul, there's 
still plenty of work being done on it. If you look at the changelog, you'll 
see that there are a lot of CTFE-related changes, and I know that more CTFE 
changes are in the works for 2.055 based on what Don has said. So, it doesn't 
surprise me at all that issues with it are cropping up. And unless Don is 
willing to say that we're going to guarantee the CTFEability of functions, 
then there will be no guarantees of any functions remaining CTFEable, and 
we're definitely not going to test that functions stay CTFEable.

If Don is willing to say that CTFE is stable enough to make guarantees about 
it and that we can make guarantees about Phobos functions being CTFEable, then 
I think that it makes perfect sense to add tests for it and make sure that 
functions remain CTFEable if they're currently CTFEable. But as long as Don is 
not willing to say that, then Phobos is not going to have any such guarantees 
and it makes no sense to test for CTFEability.

Ideally, I think that we would be able to make such guarantees and test for 
them, but Don has been against it, and he's the one managing CTFE at this 
point, so it's really up to him.

- Jonathan M Davis
Jul 12 2011
parent Don <nospam nospam.com> writes:
Jonathan M Davis wrote:

 On 2011-07-12 17:16, Nick Sabalausky wrote:

 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1576.1310513383.14074.digitalmars-d puremagic.com...


 On 2011-07-12 16:17, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


 Don has been making huge changes to CTFE. Stuff that didn't used to
 compile,
 now complie. Stuff which compiled but shouldn't have now doesn't compile.
 There's probably stuff which used to compile and should still compile
 which
 doesn't compile now too. But with all of those changes, I'm not sure that
 it's
 at all reasonable to expect CTFE-ability to be stable. It should be
 heading in
 that direction, but I'm not sure how stable Don considers it. Certainly,
 strip
 could be failing for a perfectly legitimate reason, or it could be a bug.
 I
 have no idea. But with all of the changes that have been being made to
 CTFE,
 I'm not at all surprised if stuff has quit working. There's probably more
 that
 works now that didn't before, but with all of the recent changes,
 breakage doesn't surprise me one bit.


 I definitely expected that in 2.053, since that's the version that had the
 CTFE overhaul. But it's not being re-overhauled in each version after
 2.053 - just bugfixes and added features. So at this point, I think it's
 reasonable to expect that any CTFE changes that break Phobos code should be
 fixed before release, even if only a temporary fix, in either the CTFE
 engine or in the Phobos code that was broken.


 
 Well, regardless of whether CTFE is still having a major overhaul, there's 
 still plenty of work being done on it. If you look at the changelog, you'll 
 see that there are a lot of CTFE-related changes, and I know that more CTFE 
 changes are in the works for 2.055 based on what Don has said. So, it doesn't 
 surprise me at all that issues with it are cropping up. And unless Don is 
 willing to say that we're going to guarantee the CTFEability of functions, 
 then there will be no guarantees of any functions remaining CTFEable, and 
 we're definitely not going to test that functions stay CTFEable.



2.053 fixed the fundamental wrong-code problem: array references didn't 
work AT ALL. But it only fixed local variables, not parameters.

2.054 fixed reference parameters. It also completely changed the 
treatment of slices.
After this, most of the wrong-code issues with CTFE were fixed. So that 
if it compiles in CTFE, it actually works.

2.054 also added support for pointers and heap-allocated structs.
This means that CTFE works in enough cases that a large chunk of Phobos 
should actually work now, and also means that most improvements to 
Phobos functions shouldn't cause CTFE to fail.

My plan is that 2.055 will clean up the smaller issues, which are of two 
types: (1) special cases that were overlooked (hopefully all of these 
will cause internal compiler errors rather than wrong-code); and (2) the 
compiler runtime helper functions.

This will leave the CTFE rules as:
(1) source code must be available;
(2) no access to global or static variables;
(3) no asm;
(4) no classes (and therefore, no interfaces, can't throw exceptions, etc);
(5) no closures.

(4) and (5) are only temporary limitations.
Note that for any function which is pure, we can expect that 
non-CTFEable regressions will happen only when external functions are 
called. Interestingly this is something which could be checked statically.
(CTFE can also use functions which aren't pure, but they're considerably 
more difficult to check, since they are CTFE-callable only for 
particular combinations of parameters).
Jul 21 2011
prev sibling parent reply filgood <filgood somewhere.net> writes:
Jonathan,

Given that D2 is now the main version, would it not be possible - maybe 
in the future - if big changes like the CTFE work by Don are planned to 
bring to do this in a 2.1 release of D (and once stable bump to a 2.2 
release or so)... While the big changes happen in 2.1, the 2 releases 
would only hold bug fixes?

~ filgood


On 13/07/2011 00:29, Jonathan M Davis wrote:

 On 2011-07-12 16:17, Andrej Mitrovic wrote:

 I don't understand what strip() could be doing to break CTFE anyway?


 Don has been making huge changes to CTFE. Stuff that didn't used to compile,
 now complie. Stuff which compiled but shouldn't have now doesn't compile.
 There's probably stuff which used to compile and should still compile which
 doesn't compile now too. But with all of those changes, I'm not sure that it's
 at all reasonable to expect CTFE-ability to be stable. It should be heading in
 that direction, but I'm not sure how stable Don considers it. Certainly, strip
 could be failing for a perfectly legitimate reason, or it could be a bug. I
 have no idea. But with all of the changes that have been being made to CTFE,
 I'm not at all surprised if stuff has quit working. There's probably more that
 works now that didn't before, but with all of the recent changes, breakage
 doesn't surprise me one bit.

 Having tests in Phobos for CTFE would catch many breakages, but if Don isn't
 yet guaranteeing what can and can't be CTFEed by the compiler, then such
 breakages could easily be because of fixes which made it so that stuff which
 wasn't supposed to compiled but did stopped compiling. So, until Don thinks
 that what he's been doing to CTFE is appropriately stable and can make
 guarantees about what will and won't be CTFEable as far as language features
 go, then Phobos can't make any guarantees about CTFEability.

 So, basically, a lot of CTFE changes have been happening, and Don has said
 pretty much said that we're not currently making guarantees about what's
 CTFEable and what isn't. And until the changes stabilize and Don is willing to
 make guarantees, Phobos can't guarantee anything about CTFE.

 - Jonathan M Davis
Jul 13 2011
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-13 13:58, filgood wrote:

 Jonathan,
 
 Given that D2 is now the main version, would it not be possible - maybe
 in the future - if big changes like the CTFE work by Don are planned to
 bring to do this in a 2.1 release of D (and once stable bump to a 2.2
 release or so)... While the big changes happen in 2.1, the 2 releases
 would only hold bug fixes?


D versioning doesn't work that way. Currently, we have 1.069 for D1 and 2.054 
for D2. The next version of D2 will be 2.055. I don't know exactly what 
Walter's plans are, but it looks to me like the idea is that 2.100 will follow 
2.099. But regardless, D is still in development, and CTFE happens to be one 
one of its less stable elements. A lot of work is being done on it, and it 
will probably stabilize relatively soon, but until it does, I don't expect 
that we'll be making any guarantees about what will or won't be CTFEable. 
Certainly, Don has been opposed to such suggestions thus far. I hope that once 
it's more stable, we'll be able to have unit tests in Phobos to guarantee that 
at least some set of functions stay CTFEable if they're already CTFEable, but 
we're just not going to make any such guarantees before CTFE is more stable.

Not to mention, while most stuff in D is relatively stable, and the language 
spec really shouldn't be changing much - if at all - at this point, both the 
compiler and libraries are still under heavy development. Breaking changes 
should be rare, but they do happen. And newer and/or fancier features are the 
ones which are most likely to break. CTFE happens to be one of them. There are 
a couple of others (alias this and inout) which aren't even completely 
implemented yet. So, it's questionable if talking about D2 as if it were 
stable is a good idea. It's getting close. IIRC, in a recent discussion on it, 
Don thought that it would reach that point before the end of the year, but 
some features just aren't yet stable enough to be guaranteed to continue to 
work exactly as they do now. Fortunately, that list should be rather short. 
But CTFE is on it.

So, we're not going to fork D2 at this point and have a 2.0 which only has bug 
fixes and 2.1 which has new features. We're fast approaching the point where 
what we have will be stable and there shouldn't be any big breaking changes 
(big breaking changes are actually pretty rare as it is at this point), but 
we're not going to effectively fork development to provide a more stable 
branch for a feature for which we're not yet giving guarantees about its 
stability.

I _do_ think that having unit tests in Phobos for CTFEability is a good idea, 
but until Don says that CTFE is stable enough for that, we're not going to do 
it. I believe that he's on vacation at the moment (which is presumably why he 
hasn't said anything on this thread), and maybe he can be persuaded once he's 
back, but the last time that he had anything to say about it, he was against 
the idea.

- Jonathan M Davis
Jul 13 2011
prev sibling parent reply bcs <bcs example.com> writes:
== Quote from Andrej Mitrovic (andrej.mitrovich gmail.com)'s article

 Last time I brought this issue up in bugzilla it was shot down with
 "We don't guarantee and don't have to guarantee functions will


always

 be CTFE-able between releases".


Maybe there should be a std.ctfe.* that looks a bit like std.* that IS
guaranteed to work. Ideally it would be nothing but a bunch of alias.


 I end up having to put initializers in a module ctor. Not nice!
Jul 12 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"bcs" <bcs example.com> wrote in message 
news:iviv9h$2ee$1 digitalmars.com...

 == Quote from Andrej Mitrovic (andrej.mitrovich gmail.com)'s article

 Last time I brought this issue up in bugzilla it was shot down with
 "We don't guarantee and don't have to guarantee functions will


 always

 be CTFE-able between releases".


 Maybe there should be a std.ctfe.* that looks a bit like std.* that IS
 guaranteed to work. Ideally it would be nothing but a bunch of alias.


If we do that we may as well just stick their bodies inside the original 
function in an if(_ctfe) block.
Jul 12 2011
parent bcs <bcs google.com> writes:
== Quote from Nick Sabalausky (a a.a)'s article

 "bcs" <bcs example.com> wrote in message
 news:iviv9h$2ee$1 digitalmars.com...

 == Quote from Andrej Mitrovic (andrej.mitrovich gmail.com)'s




article

 Last time I brought this issue up in bugzilla it was shot down






with

 "We don't guarantee and don't have to guarantee functions will


 always

 be CTFE-able between releases".


 Maybe there should be a std.ctfe.* that looks a bit like std.*




that IS

 guaranteed to work. Ideally it would be nothing but a bunch of




alias.


 If we do that we may as well just stick their bodies inside the


original

 function in an if(_ctfe) block.


That only gets a small part of the benefit, the rest is that it would
document to the end user what is CTFE clean and also document to the
dev what functions need to be CTFE clean. Putting if(_ctfe) in the
function ends up making the user look at the code (rather than docs)
and still doesn't give a definitive answer as to if the code works
this time or, given that, if it may break next time around.



 Jul 12 2011



prev sibling  parent reply Don <nospam nospam.com>  writes:


Nick Sabalausky wrote:

 Yet again, a new DMD release has broken my code, and other people's code, 
 too, just because of Phobos functions loosing their CTFE-ability. (strip(), 
 toLower(), etc... And yes, I did bring up the strip() regression on the beta 
 list, to no effect.)


The situation is, that prior to 2.054, use of Phobos in CTFE functions 
has not been supported. No official statements have ever been made that 
it was supported (and I've tried hard to publicize the fact that it was 
not supported). There were some fundamental wrong-code bugs and severe 
limitations in CTFE. And so although some Phobos functions seemed to 
work in CTFE, most of them were not correct.
So, any Phobos function which used to work in CTFE in some old release 
but doesn't any more is not a regression. It was just dumb luck, relying 
on undefined behaviour, if any happened to work in a particular release.

This has changed with 2.054. Although there are still some restrictions 
(no classes, many builtin functions aren't implemented), the fundamental 
wrong code bugs are fixed. So any Phobos function which works in CTFE in 
2.054, but fails in a future release, can be considered a regression.
We are now in a _completely_ different regime.



 Jul 20 2011



 next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On 2011-07-20 15:37, Don wrote:

 Nick Sabalausky wrote:

 Yet again, a new DMD release has broken my code, and other people's code,
 too, just because of Phobos functions loosing their CTFE-ability.
 (strip(), toLower(), etc... And yes, I did bring up the strip()
 regression on the beta list, to no effect.)


 
 The situation is, that prior to 2.054, use of Phobos in CTFE functions
 has not been supported. No official statements have ever been made that
 it was supported (and I've tried hard to publicize the fact that it was
 not supported). There were some fundamental wrong-code bugs and severe
 limitations in CTFE. And so although some Phobos functions seemed to
 work in CTFE, most of them were not correct.
 So, any Phobos function which used to work in CTFE in some old release
 but doesn't any more is not a regression. It was just dumb luck, relying
 on undefined behaviour, if any happened to work in a particular release.
 
 This has changed with 2.054. Although there are still some restrictions
 (no classes, many builtin functions aren't implemented), the fundamental
 wrong code bugs are fixed. So any Phobos function which works in CTFE in
 2.054, but fails in a future release, can be considered a regression.
 We are now in a _completely_ different regime.


If that's the case, then we should look at adding CTFEability tests to at 
least some portion of Phobos.

- Jonathan M Davis



 Jul 20 2011



  parent  "Nick Sabalausky" <a a.a>  writes:


"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1817.1311202371.14074.digitalmars-d puremagic.com...

 On 2011-07-20 15:37, Don wrote:

 Nick Sabalausky wrote:

 Yet again, a new DMD release has broken my code, and other people's 
 code,
 too, just because of Phobos functions loosing their CTFE-ability.
 (strip(), toLower(), etc... And yes, I did bring up the strip()
 regression on the beta list, to no effect.)


 The situation is, that prior to 2.054, use of Phobos in CTFE functions
 has not been supported. No official statements have ever been made that
 it was supported (and I've tried hard to publicize the fact that it was
 not supported). There were some fundamental wrong-code bugs and severe
 limitations in CTFE. And so although some Phobos functions seemed to
 work in CTFE, most of them were not correct.
 So, any Phobos function which used to work in CTFE in some old release
 but doesn't any more is not a regression. It was just dumb luck, relying
 on undefined behaviour, if any happened to work in a particular release.




Ok, I see. I guess I must have missed that.


 This has changed with 2.054. Although there are still some restrictions
 (no classes, many builtin functions aren't implemented), the fundamental
 wrong code bugs are fixed. So any Phobos function which works in CTFE in
 2.054, but fails in a future release, can be considered a regression.
 We are now in a _completely_ different regime.




Glad to hear it!


 If that's the case, then we should look at adding CTFEability tests to at
 least some portion of Phobos.


Totally agree. I think string-processing and anything else likely to be 
needed in string metaprogramming (such as to!string(int) and to!int(string)) 
would be the most important ones.



 Jul 20 2011



prev sibling  parent reply dsimcha <dsimcha yahoo.com>  writes:


Ok, now that the new docs are up, I'm kind of curious about what's a fundamental
limitation vs. a todo.  I assume the following are fundamental limitations,
since
CTFE has no concept of threading or global state and works by interpreting the
code, not executing it directly on the architecture it's being compiled for:

The function source code must be available to the compiler.

Executed expressions may not reference any global or local static variables

No synchronized or asm statements.  (Since CTFE code is always effectively
single-threaded and doesn't allow global state anyhow, could we maybe make
synchronized statements effectively a no-op in CTFE?)

Casts which involve a reinterpretation of the underlying pattern of bits (eg,
from
int[] to float[]) are not permitted

C-style semantics on pointer arithmetic are strictly enforced.

Function parameters may not be C-style variadic


The following look like they are simple matters of programming and could be
implemented eventually:

With statements, scope statements, try-catch-finally statements, throw
statements.

Delete expressions.

Classes and interfaces.


That said, this is a heck of a lot less limitations than CTFE used to have. 
Great
job, Don!

== Quote from Don (nospam nospam.com)'s article

 Nick Sabalausky wrote:

 Yet again, a new DMD release has broken my code, and other people's code,
 too, just because of Phobos functions loosing their CTFE-ability. (strip(),
 toLower(), etc... And yes, I did bring up the strip() regression on the beta
 list, to no effect.)


 The situation is, that prior to 2.054, use of Phobos in CTFE functions
 has not been supported. No official statements have ever been made that
 it was supported (and I've tried hard to publicize the fact that it was
 not supported). There were some fundamental wrong-code bugs and severe
 limitations in CTFE. And so although some Phobos functions seemed to
 work in CTFE, most of them were not correct.
 So, any Phobos function which used to work in CTFE in some old release
 but doesn't any more is not a regression. It was just dumb luck, relying
 on undefined behaviour, if any happened to work in a particular release.
 This has changed with 2.054. Although there are still some restrictions
 (no classes, many builtin functions aren't implemented), the fundamental
 wrong code bugs are fixed. So any Phobos function which works in CTFE in
 2.054, but fails in a future release, can be considered a regression.
 We are now in a _completely_ different regime.





 Jul 26 2011



 next sibling parent  "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


 Ok, now that the new docs are up, I'm kind of curious about what's a
 fundamental limitation vs. a todo. I assume the following are fundamental
 limitations, since CTFE has no concept of threading or global state and
 works by interpreting the code, not executing it directly on the
 architecture it's being compiled for:
 
 The function source code must be available to the compiler.
 
 Executed expressions may not reference any global or local static variables
 
 No synchronized or asm statements. (Since CTFE code is always effectively
 single-threaded and doesn't allow global state anyhow, could we maybe make
 synchronized statements effectively a no-op in CTFE?)
 
 Casts which involve a reinterpretation of the underlying pattern of bits
 (eg, from int[] to float[]) are not permitted
 
 C-style semantics on pointer arithmetic are strictly enforced.
 
 Function parameters may not be C-style variadic
 
 
 The following look like they are simple matters of programming and could be
 implemented eventually:
 
 With statements, scope statements, try-catch-finally statements, throw
 statements.
 
 Delete expressions.
 
 Classes and interfaces.
 
 
 That said, this is a heck of a lot less limitations than CTFE used to have.
 Great job, Don!


Agreed. The situation with regards to CTFE has really been improving. Thanks 
Don!

- Jonathan M Davis



 Jul 26 2011



prev sibling  parent reply Don <nospam nospam.com>  writes:


dsimcha wrote:

 Ok, now that the new docs are up, I'm kind of curious about what's a
fundamental
 limitation vs. a todo.  I assume the following are fundamental limitations,
since
 CTFE has no concept of threading or global state and works by interpreting the
 code, not executing it directly on the architecture it's being compiled for:
 
 The function source code must be available to the compiler.


Fundamental limitation.

 
 Executed expressions may not reference any global or local static variables


Fundamental limitation.


 No synchronized or asm statements.  (Since CTFE code is always effectively
 single-threaded and doesn't allow global state anyhow, could we maybe make
 synchronized statements effectively a no-op in CTFE?)


Asm fundamental, synchronized could be a no-op.


 
 Casts which involve a reinterpretation of the underlying pattern of bits (eg,
from
 int[] to float[]) are not permitted


Not sure. Would be tricky to implement in general.


 C-style semantics on pointer arithmetic are strictly enforced.


Fundamental, and intentional.


 Function parameters may not be C-style variadic


Not fundamental, this could be implemented fairly easily. But probably 
shouldn't be encouraged, since it's not necessary.


 The following look like they are simple matters of programming and could be
 implemented eventually:
 
 With statements, scope statements, try-catch-finally statements, throw
statements.
 
 Delete expressions.
 
 Classes and interfaces.


Correct. If you use them, the error message for all of these is "XXX is 
not yet implemented in CTFE", whereas the others say "cannot be 
interpreted at compile time". They'll all work eventually. (Not sure 
about delete, it might not be good idea).


 
 That said, this is a heck of a lot less limitations than CTFE used to have. 
Great
 job, Don!


Thanks. It's been possible to work on this because there are now so many 
people fixing the critical compiler bugs. In particular, Kennytm has 
been an enormous help. As well as a phenomenal number of patches, he's 
also been posting PERFECT bug reports on CTFE issues.


 
 == Quote from Don (nospam nospam.com)'s article

 Nick Sabalausky wrote:

 Yet again, a new DMD release has broken my code, and other people's code,
 too, just because of Phobos functions loosing their CTFE-ability. (strip(),
 toLower(), etc... And yes, I did bring up the strip() regression on the beta
 list, to no effect.)


 The situation is, that prior to 2.054, use of Phobos in CTFE functions
 has not been supported. No official statements have ever been made that
 it was supported (and I've tried hard to publicize the fact that it was
 not supported). There were some fundamental wrong-code bugs and severe
 limitations in CTFE. And so although some Phobos functions seemed to
 work in CTFE, most of them were not correct.
 So, any Phobos function which used to work in CTFE in some old release
 but doesn't any more is not a regression. It was just dumb luck, relying
 on undefined behaviour, if any happened to work in a particular release.
 This has changed with 2.054. Although there are still some restrictions
 (no classes, many builtin functions aren't implemented), the fundamental
 wrong code bugs are fixed. So any Phobos function which works in CTFE in
 2.054, but fails in a future release, can be considered a regression.
 We are now in a _completely_ different regime.


 





 Jul 30 2011



  parent  Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Saturday 30 July 2011 21:09:48 Don wrote:

 dsimcha wrote:

 The following look like they are simple matters of programming and could
 be implemented eventually:
 
 With statements, scope statements, try-catch-finally statements, throw
 statements.
 
 Delete expressions.
 
 Classes and interfaces.


 
 Correct. If you use them, the error message for all of these is "XXX is
 not yet implemented in CTFE", whereas the others say "cannot be
 interpreted at compile time". They'll all work eventually. (Not sure
 about delete, it might not be good idea).


Isn't delete supposed to be deprecated? If so, then I definitely wouldn't 
expect CTFE to support it.

- Jonathan M Davis



 Jul 30 2011