www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - compile-time explicitness

reply Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
I have a proposal to introduce and optional explicit separation of
compile-time facilities from run-time facilities.
There is a new keyword, called compiletime (or something similar).
The keyword can be used with `is` expression to detect whether the current
scope is executing in compile-time or not.

static if(__traits(compiletime))
{
    // ...
}

The keyword is a storage class for declarations of variables, types and
functions.

A variable, declared with this keyword behaves much like a enum:

compiletime float dmdVersion = 2.055;

This keyword reflects the nature of dmdVersion much better, then the `enum`
keyword, since it does not actually enumerate anything.
dmdVersion does not exist at run-time (hence, cannot be taken address of) just
like enums.

compiletime void myfunc() { }
void myfunc() { }

functions, marked with compiletime keyword do not exist at run-time (cannot be
taken address of, exported from DLLs, etc.). Functions (even with the same set
of arguments) can overload on this storage class, allowing to seamlessly
specialize compile-time version of any function or method. If the function is
marked as compiletime, then all it's parameters are marked as such too (in
current CTFE, they're still deemed run-time, because the function itself
doesn't know that it's running at compile-time), essentially making it a
template, but with no mandatory functional programming style.

compiletime struct MyStyruct {}
compiletime alias MyAlias;
compiletime union MyUnion {}

Type definitions, marked as compiletime do not exist at run-time (have no
TypeInfo, cannot declare run-time variables) and they enforce their instances
to be compiletime as well. This allows to further enhance the expressiveness
of the language:

compiletime alias ulong Version;

void func(int templateParam)()
{
    Version v = (templateParam + 5) / 2 - veryComplexExpression;
    static if(v > 25) // ok, v got compiletime from it's type.
    {
        doTheStuff();
    }
}

Template functions will have a very convenient syntax sugar:

void myFunc(int arg1, compiletime int arg2, char arg3, compiletime char arg4)
{
}

, which is equivalent to:

vodi myFunc(int arg2, int arg3)(int arg1, int arg4)
{
}

functions, which differ only by the difference of compiletime-ness of their
arguments (the number and types are equal) can overload against each other,
allowing the function implementor to specialize functions, so that each
compile-time argument, passed to the function may dramatically increase the
run-time performance of the function (essentially allowing to conditionally
hardcode parts of the function at compile time).

The function parameters can be marked as auto compiletime:

void f(auto compiletime arg)
{
    static if(__traits(compiletime, arg))
    {
        // partially specialize the function right in the scope.
    }
}

, which will allow the programmer to write a single function, where each of
it's arguments can dramatically increase the run-time performance of the
function by being compile-time.

And, of course, if everything else fails, the good ol' CTFE will come to the
rescue.

Now for the best part:

All this (almost) does not break any code!!! The almost stands for the keyword
itself, which may be annotated  compiletime (like  safe) until D3, where the
annotation will either be removed or, everything else will get annotated too.

I will closely study DMD's source code to start implementing this myself if
and when you guys approve this idea.

What do you think?
What application of the keyword did i miss?
Do i deserve a candy for this or not? :-)
Sep 23 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 23 Sep 2011 11:51:35 -0400, Gor F. Gyolchanyan  
<gor.f.gyolchanyan gmail.com> wrote:


 I have a proposal to introduce and optional explicit separation of
 compile-time facilities from run-time facilities.
 There is a new keyword, called compiletime (or something similar).
 The keyword can be used with `is` expression to detect whether the  
 current
 scope is executing in compile-time or not.

 static if(__traits(compiletime))
 {
     // ...
 }


static if(__ctfe) does this already


 The keyword is a storage class for declarations of variables, types and  
 functions.

 A variable, declared with this keyword behaves much like a enum:

 compiletime float dmdVersion = 2.055;

 This keyword reflects the nature of dmdVersion much better, then the  
 `enum`
 keyword, since it does not actually enumerate anything.
 dmdVersion does not exist at run-time (hence, cannot be taken address  
 of) just
 like enums.


This is a long dead horse, enum is here to stay in its current meaning,  
much as a lot of us don't like it.  You're likely to get nowhere in this  
argument.

All of the rest of your points are solved with static if(__ctfe)

-Steve
Sep 23 2011
next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.v19qwjhqeav7ka localhost.localdomain...

 All of the rest of your points are solved with static if(__ctfe)

 -Steve


You can't use __ctfe with static if - you can only use it where a runtime 
variable would be used.
Sep 23 2011
prev sibling parent reply Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
About enum: it still stays. at least until D3 or so.
enum has it's own meaning too. the variable part is the least important part of
what i presented.

Not qite. __ctfe can solve many problems, but, for example, you can't use __ctfe
to determine whether you should use to!string or toStringNow.

__ctfe can't allow you to have both compile-time and run-time versions of the
function body simultaneously. DMD will still report an error, since __ctfe is a
run-time defined value and cannot be used in static if.

__ctfe can't help make non-functional-programming style templates.

__ctfe can't allow you to define temporary locals to store intermediate results
of
computation with template parameters, which can then be used in a static if.
Sep 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 23 Sep 2011 12:20:21 -0400, Gor F. Gyolchanyan  
<gor.f.gyolchanyan gmail.com> wrote:


 Not qite.


I apologize, you are right that __ctfe is a runtime variable, but I find  
that interesting.

An optimizer will (hopefully) remove any if(__ctfe) branches, so even  
though it's "evaluated at runtime", it's still evaluates to the constant 0.

The only difference is, you cannot use it to for instance change the  
structure of a struct during compile time.  And actually, thinking about  
it, this has to be this way.

For instance, what if you had:

struct S
{
    static if(__ctfe)
      int x;
    int y;
}

enum S s = S(1, 2);

The expression is evaluated at compile-time, but then the result can be  
used during runtime.  How does that work?

I think actually __ctfe as defined is the correct thing to have.  static  
if has too much power to change type layouts and would wreak havoc with  
the CTFE model.

So I guess I was wrong, and it's a good thing I was :)


 __ctfe can solve many problems, but, for example, you can't use __ctfe
 to determine whether you should use to!string or toStringNow.


You could:

if(__ctfe)
    to!string(x); // optimized out at runtime
else
    toStringNow(x); // optimized out at compile-time (well doesn't really  
matter)


 __ctfe can't allow you to have both compile-time and run-time versions  
 of the
 function body simultaneously.


The optimizer should remove any if(__ctfe) in the normal runtime function,  
since it's equivalent to if(0).  So yes, you can have two different  
versions.

For an example, see std.array.Appender's methods


 __ctfe can't help make non-functional-programming style templates.


I'm not sure what you mean by this.


 __ctfe can't allow you to define temporary locals to store intermediate  
 results of
 computation with template parameters, which can then be used in a static  
 if.


you can use enum for that already.

-Steve
Sep 23 2011
parent reply Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
It's not a performance issue.
You can't do this:

if(_ctfe)
    to!string(...);
else
    toStringNow!(...);

because the toStringNow!(...) won't compile, because it's argument is not a
compile-time value.

about non-functional style templates:
When you have a complex computation in your templates, you're forced to create
additional private templates to divide the computation and put it together in a
single expression.
Sep 23 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 23 Sep 2011 14:15:25 -0400, Gor F. Gyolchanyan  
<gor.f.gyolchanyan gmail.com> wrote:


 It's not a performance issue.
 You can't do this:

 if(_ctfe)
     to!string(...);
 else
     toStringNow!(...);

 because the toStringNow!(...) won't compile, because it's argument is  
 not a
 compile-time value.


Well, first, I was unfamiliar with toStringNow (I thought it was simply a  
different function), but second, all you need is a ctfe-able toString  
function.  to!string should be ctfe-able for many things, a lot of work is  
going into making things in phobos more compile-time ready and pure-ready.


 about non-functional style templates:
 When you have a complex computation in your templates, you're forced to  
 create
 additional private templates to divide the computation and put it  
 together in a
 single expression.


Can you give an example?  I'm still not understanding why you can't just  
use ctfe.

-Steve
Sep 23 2011
parent reply Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
Of course, if the entire D gets CTFE-able, the __ctfe will be completely
useless.
But i can't see that coming for a long time. Most of the major programming
problems are best solved with classes and that's where CTFE stops.

About templates:

void unpackIntoFunction(alias arrayOfVariants, alias func)()
{
   // you can't generate the necessary mixin strings right here, because it
won't
be compile-time.
   // you need to create a separate function, that returns the required string
to
be mixed in.
   mixin(whatIJustGenerated);
}
Sep 23 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/23/2011 08:47 PM, Gor F. Gyolchanyan wrote:

 Of course, if the entire D gets CTFE-able, the __ctfe will be completely
useless.
 But i can't see that coming for a long time. Most of the major programming
 problems are best solved with classes and that's where CTFE stops.

 About templates:

 void unpackIntoFunction(alias arrayOfVariants, alias func)()
 {
     // you can't generate the necessary mixin strings right here, because it
won't
 be compile-time.
     // you need to create a separate function, that returns the required
string to
 be mixed in.
     mixin(whatIJustGenerated);
 }



mixin({
     string r;
     foreach(i; 0..10) r~=q{writeln("hello world!");};
     return r;
}());
Sep 23 2011
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 23 Sep 2011 14:47:32 -0400, Gor F. Gyolchanyan  
<gor.f.gyolchanyan gmail.com> wrote:


 Of course, if the entire D gets CTFE-able, the __ctfe will be completely  
 useless.
 But i can't see that coming for a long time.


Well, anywhere that it makes sense, it should be CTFE-able.

For example to!string(int) should be CTFE-able, as well as to!int(string)

I think a really cool CTFE feature would be if sort could be used in  
compile time.  Imagine writing a lookup table in whatever order you want,  
then just using:

enum lookupTable = sorted(lookupData);

I can think of a few places that would have been useful in my past.

Places where __ctfe is useful are functions that depend on opaque runtime  
functions.  Then you can write alternatives.  Again, I'll refer you to  
std.array.Appender.


 Most of the major programming
 problems are best solved with classes and that's where CTFE stops.


I'm not sure everybody agrees on that.  Classes are good for certain  
things, and horrible for others.


 About templates:

 void unpackIntoFunction(alias arrayOfVariants, alias func)()
 {
    // you can't generate the necessary mixin strings right here, because  
 it won't
 be compile-time.
    // you need to create a separate function, that returns the required  
 string to
 be mixed in.
    mixin(whatIJustGenerated);
 }


enum + static if signifies compile time.  If you want to use the  
compile-time constructs, you can build it at compile-time.  Otherwise, you  
can either use eponymous templates (functional style) or CTFE-able  
functions (imperative style).

A more complete example will allow better explanation.

-Steve
Sep 23 2011
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Damn I must have missed the discussion where __ctfe was introduced.
What exactly is this new symbol for? It should be put in the docs
unless it's there already (I can't find it).
Sep 23 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 23 Sep 2011 15:38:30 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 Damn I must have missed the discussion where __ctfe was introduced.
 What exactly is this new symbol for? It should be put in the docs
 unless it's there already (I can't find it).


http://www.d-programming-language.org/function.html#interpretation

-Steve
Sep 23 2011
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 9/23/11, Steven Schveighoffer <schveiguy yahoo.com> wrote:

 On Fri, 23 Sep 2011 15:38:30 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 Damn I must have missed the discussion where __ctfe was introduced.
 What exactly is this new symbol for? It should be put in the docs
 unless it's there already (I can't find it).


 http://www.d-programming-language.org/function.html#interpretation

 -Steve


Oh right, Don updated this recently but I forgot to take a look. Thanks.
Sep 23 2011
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 9/23/11, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 Damn I must have missed the discussion where __ctfe was introduced.
 What exactly is this new symbol for? It should be put in the docs
 unless it's there already (I can't find it).


Looks like it's not new, but I didn't know about it. Found this:
http://www.mail-archive.com/digitalmars-d puremagic.com/msg19041.html
Sep 23 2011
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, September 23, 2011 11:15 Gor F. Gyolchanyan wrote:

 It's not a performance issue.
 You can't do this:
 
 if(_ctfe)
 to!string(...);
 else
 toStringNow!(...);
 
 because the toStringNow!(...) won't compile, because it's argument is not a
 compile-time value.


So, use a normal function. The main reason to use _ctfe is because the normal 
function doesn't work with CTFE for one reason or another, and so you use a 
less efficient function which _can_ do it at compile time. Since you're 
operating on a value rather than a type, there's no reason why you can't use a 
function rather than an eponymous template.

- Jonathan M Davis
Sep 23 2011
parent reply Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
But that will lead me to writing the same functions twice (making a template
version, actually). Again, if D becomes entirely CTFE-able, the topic can be
closed for good. :-)
Sep 23 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/23/2011 08:50 PM, Gor F. Gyolchanyan wrote:

 But that will lead me to writing the same functions twice (making a template
 version, actually). Again, if D becomes entirely CTFE-able, the topic can be
 closed for good. :-)


What can template versions compute that normal functions cannot?
Sep 23 2011
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
If you instantiate a variable with a function call at compile-time
then that function shouldn't be compiled in (AFAIK) unless it's
referenced elsewhere. And for disallowing user-code to use such a
function that's what access specifiers are for. The beauty of D is
that you write a function once and only once, and you can (if it's
possible) run it at compile time, or at runtime based on the *call*,
not it's definition.

I think you need to slow down with these proposals. This introduces
nothing but complications into the language imho.
Sep 23 2011
parent Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
About the "write once": i agree, it's a valuable feature of D, regarding CTFE.
But there are many times, where a function can be made to be much faster if some
of it's arguments are known at compile-time.
Writing those kind of functions involves having lots of different kind of
templates with non-uniform syntax. Being able to call a function uniformly and
choose it's implementation, based on it's arguments being compile-time allows to
simplify both the definition and the usage of such functions. How's that gonna
introduce complications?
Sep 23 2011
prev sibling next sibling parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
It's great that you have taken an interest in the development of D!

Some of this is already in the language in other forms (enum, for instance) 
and some has been discussed before (searching the newsgroup archive for 
static arguments should find it) and rejected.  The bar for new language 
features is pretty high at this point, and new features that don't add 
important new functionality or remove serious problems are very unlikely to 
make it in. (With some exceptions)

To make a stronger proposal, try going into more depth about what problems 
this enhancement can solve and why they can't be solved using existing 
features.

As for implementing this feature, it would require a huge amount of work on 
dmd.  I'd suggest having a go at some of the 1000+ existing bugs before 
trying to implement highly complex features in the compiler.

That said, I don't make the decisions with regard to new language features. 
An idea that comes with an implementation always has more chance of getting 
a serious review from the core language designers.
Sep 23 2011
parent reply Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
Thank you so much for an objective reply! Of course i realize, that there are
more
important issues with DMD (i recently suggested not to implement enhancement
requests to free up time to fix bugs). What i suggest is, as you correctly
noticed, a very big task and i realize that it's not gonna be soon. What i
wanted
to take away from this discussion is the general opinion of the idea to rethink
the compile-time facilities of D in the future.
Sep 23 2011
next sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Gor F. Gyolchanyan Wrote:


 Thank you so much for an objective reply! Of course i realize, that there are
more
 important issues with DMD (i recently suggested not to implement enhancement
 requests to free up time to fix bugs).


I was going to say, for someone who wants to stop work on enhancements, you
sure do make a lot of requests.

Here is what I would like to caution you about, even for "future suggestions."
One of the hardest parts of an idea is polishing it. Then finding someone to
implement it, so it can be polished more. Then implementing it. So even ideas
for D3 would be a distraction from getting things done in D2. This means you'll
get many (Walter, Andrei...) that will not be participating in such discussions
and only take quick glances.

The approach I would suggest. Is to assume an idea has already been discussed.
Go to the D.learn section and post a question on how best to solve ... and
include what you'd like to do. This will keep new ideas coming in, but have
them directed through a filter. Even just rephrasing a topic becomes a learning
experience, "why can't I be explicit about what is compile-time?"

Enjoy learning D and I hope you find good uses for it.
Sep 23 2011
parent Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
Thanks! You've been extremely helpful! You're right, asking for existing
solution
before proposing a new one is much better.

I've been studying D2 for several months non-stop. I've been paying close
attention to all discussions on D newsgroups and bugfixes on every DMD release.
It's time for me to stop observing and start acting, so i decided to come here
and
spill out what i got collected in my mind for all this time of living with D. I
guess i should slow down a bit with new stuff and direct my efforts to old stuff
that needs to be done.

I can't stop noticing how great D feels after years of C++ experience.
Sep 23 2011
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, September 23, 2011 11:06 Gor F. Gyolchanyan wrote:

 Thank you so much for an objective reply! Of course i realize, that there
 are more important issues with DMD (i recently suggested not to implement
 enhancement requests to free up time to fix bugs). What i suggest is, as
 you correctly noticed, a very big task and i realize that it's not gonna
 be soon. What i wanted to take away from this discussion is the general
 opinion of the idea to rethink the compile-time facilities of D in the
 future.


They work quite well overall. I think that if we want to find solid ways to 
improve them, we're going to need more code written using what we currently 
have. More experience is needed with the current compile time facilities of D 
to really figure out how best to improve them

- Jonathan M Davis
Sep 23 2011
parent Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
Agreed.
There are lots of dark-magic compile-time D techniques yet to be discovered.

Even C++'s templates, which are decades old, don't stop getting new usage ideas.

I'll stop making enhancement requests until I face a problem with current
language, which not even D.learn can help solve.

On another note, I want to start fixing compiler bugs in DMD's front-end. What
do
you suggest me to start from? What should i read first?
Sep 23 2011
prev sibling parent reply travert phare.normalesup.org (Christophe) writes:
Gor F. Gyolchanyan , dans le message (digitalmars.D:145120), a écrit :

 Template functions will have a very convenient syntax sugar:
 
 void myFunc(int arg1, compiletime int arg2, char arg3, compiletime char arg4)
 {
 }
 
 , which is equivalent to:
 
 vodi myFunc(int arg2, int arg3)(int arg1, int arg4)
 {
 }



 The function parameters can be marked as auto compiletime:
 
 void f(auto compiletime arg)
 {
     static if(__traits(compiletime, arg))
     {
         // partially specialize the function right in the scope.
     }
 }



That sounds really, really great to me. I often wanted to do something 
like that. One of the strong point in D is CTFE, let's get even better 
at this game.

auto compiletime seems to be much better that compiletime:
In many cases, the compiler will be able to optimize if like static if 
without having to write the specialized function at all. When a 
specialized function has to be rewritten, it should be IHMO in the same 
function body. One of the issue with templating a specialized function 
is also that it has to be separated from the function body.

My advice would be to have compiletime be what you described as auto 
compiletime by default. The programmer could be allowed to prevent/not 
recommend the compilation of the runtime version of function with a 
static if/pragma.

BTW, the behavior of auto args with regards to function overloads and 
other templates have to be precised.


 All this (almost) does not break any code!!! The almost stands for the keyword
 itself, which may be annotated  compiletime (like  safe) until D3, where the
 annotation will either be removed or, everything else will get annotated too.


Yes, this feature can be easily tested without breaking anything.

About other usages of the keyword:
- using enums for compiletime is nice, but it seems to me to be a hack 
(we often use enum when we want the variable to be compile-time because 
enum happens to be compile-time, but not because we want enum).
- for this, and because of the usage in function arguments, using the 
compiletime keyword as variable attribute makes the language more 
consistent.

- compiletime as function attribute to prevent using the function in 
runtime is weird. Why would we want to do that ? However, this could be 
an alternative to making all parameters compiletime, after 
compiletime parameters are implemented.



 Do i deserve a candy for this or not? :-)


I think so :)

-- 
Christophe
Sep 26 2011
parent Gor F. Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
Thanks! I thought that idea was worthless after i received lots of criticism
(healthy criticism, to be honest).

The biggest one of all is the auto compiletime, of course.
This alone would allow one to write functions, that take advantage of any
optimization opportunities and perform run-time computation only if absolutely
necessary.
Sep 26 2011