www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE thoughts & functional approach

reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
I like CTFE and the meta programming idea for languages like D.

However, I'm wondering why most (everyone?) is trying to do 
meta-programming using the same language as the one getting compiled. 
IMO the use-cases a pretty different and doing CTFE, code-generation 
etc. needs some other approach. If you look at all the strange template 
syntax, strange hacks etc. it's all far from being obvious.

Why not have a CTL (compile-time-language) that has access to some 
compiler internals, that follows a more functional concept? We are 
evaluating sequences of things to generate code, include / exclude code 
etc.

From my experience with the different approaches, functional thinking 
is much better suited and simpler to use for CTFE goals.

IMO that would really be a big step ahead. Because you know a hammer, 
not everything is a nail...

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Jan 31 2016
next sibling parent reply cym13 <cpicard openmailbox.org> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:

 I like CTFE and the meta programming idea for languages like D.

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.


I see things differently. First of all I don't see everyone trying
to do meta-programming with the same language. C++ for example has
a quite specific syntax between its arcane templates and the
preprocessor.

I see having the same language as a *huge* advantage. If a 
function
doesn't have side effects then it can be used at runtime or at
compile-time and integrated with your logic easily.

D's metaprogramming success is IMHO directly linked to it not 
having
a separate language for it, because it lowers the cost of learning
and using metaprogramming. It's saying "metaprogramming isn't
different from any other kind of programming, you can use the same
tools".


 Why not have a CTL (compile-time-language) that has access to 
 some compiler internals, that follows a more functional 
 concept? We are evaluating sequences of things to generate 
 code, include / exclude code etc.


Having access to some compiler internals is already what is done,
or I don't understand exactly what you mean by that.


 From my experience with the different approaches, functional 
 thinking is much better suited and simpler to use for CTFE 
 goals.


I think you should put some sort of example of how you'd want it,
because right now I don't understand. D has some nice functional
tools and they already show their strength at compile-time.

For example function purity is central to their CTFE-ability, 
using
template recursion instead of loops has often proved being a
winning strategy etc... Yes applying functionnal principles works
well at compile-time, but why should we need another language to 
do
that?
Jan 31 2016
parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-01-31 14:35:13 +0000, cym13 said:


 I see things differently. First of all I don't see everyone trying
 to do meta-programming with the same language. C++ for example has
 a quite specific syntax between its arcane templates and the
 preprocessor.


Well, ok, maybe a "using the same concepts as the underlaying language" 
might hit it better.


 I see having the same language as a *huge* advantage. If a function
 doesn't have side effects then it can be used at runtime or at
 compile-time and integrated with your logic easily.


The thing I mean is not that you shouldn't be able to reference or use 
things from the "target language" but how to write down what you want 
to do. 

Code generation by building strings using the D operators for example 
is something I think is not very elegant. If I could use a list and 
build it up without having to care about the code / data difference, 
that would simplify things a lot.

Imagine we could use Lua during compile time and have access to the AST etc.



 D's metaprogramming success is IMHO directly linked to it not having
 a separate language for it, because it lowers the cost of learning
 and using metaprogramming.


But limits you to a subset that doesn't feel very natural for doing a 
lot of common things in code-generation.


 It's saying "metaprogramming isn't different from any other kind of 
 programming, you can use the same tools".


Yes, and I don't think this statement holds. It's very different 
because the goal is totally different. I need a tool that allows me to 
manipulate my underlying code during compilation. The main aspect is: 
Manipulate D code.


 Why not have a CTL (compile-time-language) that has access to some 
 compiler internals, that follows a more functional concept? We are 
 evaluating sequences of things to generate code, include / exclude code 
 etc.


 
 Having access to some compiler internals is already what is done, or I 
 don't understand exactly what you mean by that.


Sure, but the question is how do to deal with it. IMO it's not very 
straight forward at the moment.


 I think you should put some sort of example of how you'd want it,
 because right now I don't understand. D has some nice functional
 tools and they already show their strength at compile-time.


Ok, here is a simple example: I want to ensure that specific switch 
statements handle all cases for a given enum, list, etc. This is a 
common pattern and often a source of problems because you change a 
collection but miss to update all side-effecting places.

This is an example how I have done it (maybe there is a much better way 
to do it, but I didn't come up with one):

==> BEGIN

import std.conv;
import std.stdio;
import std.string;
import std.traits;

//   example code should work with classes as well
enum A {afoo, bfoo, cfoo};

enum members1 = __traits(allMembers, A); // returns TypeTuple
auto members2 = __traits(allMembers, A);

pragma(msg, typeof(members1));
// pragma(msg, typeid(members1)); // run-time only
// static assert(is(members1 : enum)); // Error: basic type expected, not enu

pragma(msg, typeof(members2));
// pragma(msg, typeid(members2)); // run-time only


// function that generates a string which is used as a mixin at compile time
// result string must conform to syntax as it was hand-written code
string generateEnums(T...)(string type){
    string code = "enum " ~ type ~ " {";

    // this is a static foreach (compile time)
    foreach(m; T){
      debug pragma(msg, m ~ ","); // check what code we get at compile time
      code ~= m ~ ",";
    }

    return(code ~ "}");
}

int main(){
    A switch_var_a;
    final switch(switch_var_a){
      case A.afoo:
      case A.bfoo:
      case A.cfoo:
    }

    string user_input = readln();

    mixin(generateEnums!members1("B"));
    B switch_var_b = chomp(user_input).to!B; // get rid of terminating chars

    final switch (switch_var_b) {
      case B.afoo:
      {
        writeln("a");
        break;
      }
      case B.bfoo: // if commeted will cause a compiler error
      {
        writeln("b");
        break;
      }
      case B.cfoo: {writeln("c");}
    }

    return(0);
}

<== END

How about being able to write something like "ensure_final_switch B;" 
and have this call a CTF that generates the necessary code and has 
access to tool for building D structured code, AST etc.? And has a 
compile-time state I can later access in a upcoming CTF.


-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Feb 02 2016
parent anonymous <anonymous example.com> writes:
On 02.02.2016 09:27, Robert M. Münch wrote:


 ==> BEGIN


[...]

 enum A {afoo, bfoo, cfoo};


(Aside: In D no semicolon is needed here.)


 string generateEnums(T...)(string type){
     string code = "enum " ~ type ~ " {";

     // this is a static foreach (compile time)
     foreach(m; T){
       debug pragma(msg, m ~ ","); // check what code we get at compile time
       code ~= m ~ ",";
     }

     return(code ~ "}");


(Aside: Those parentheses are misleading. return is not a function.)


 }

 int main(){


[...]

     mixin(generateEnums!members1("B"));
     B switch_var_b = chomp(user_input).to!B; // get rid of terminating
 chars


[...]

 }

 <== END


I'm not saying that everything is perfect as it is, but that code can be 
made nicer with what we have right now:

----
template generateEnums(string[] members)
{
     // 'join' variant:
     import std.array: join;
     mixin("enum generateEnums {" ~ members.join(",") ~ "}");

     // 'format' variant:
     // import std.format;
     // mixin(format(q{ enum generateEnums {%-(%s, %)} }, members));
}

void main()
{
     alias B = generateEnums!([members1]);
     B switch_var_b = chomp(readln()).to!B;
     /* ... */
}
----


 How about being able to write something like "ensure_final_switch B;"
 and have this call a CTF that generates the necessary code and has
 access to tool for building D structured code, AST etc.? And has a
 compile-time state I can later access in a upcoming CTF.


So you're asking for AST macros, I suppose. There are two DIPs for them:

http://wiki.dlang.org/DIP50 - AST Macros
http://wiki.dlang.org/DIP78 - AST Macros Lite

I don't know where they stand, as I'm not really interested in the whole 
thing, but maybe one of those matches your vision.

If that's not what you have in mind, please be more concrete about what 
you think of. Maybe show some pseudo code showing how you'd like to be 
able to solve the example of generating enums.
Feb 02 2016
prev sibling next sibling parent reply anonymous <anonymous example.com> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:

 I like CTFE and the meta programming idea for languages like D.

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.


You're conflating CTFE with the other meta programming tools 
here. CTFE is the same language as run-time D, but it doesn't 
have strange template syntax. Templates, static if, __traits, 
etc. have strange syntax, but they're sort of a different 
language already.

Are you maybe wishing for a nicer alternative to templates, etc?


 Why not have a CTL (compile-time-language) that has access to 
 some compiler internals, that follows a more functional 
 concept? We are evaluating sequences of things to generate 
 code, include / exclude code etc.

 From my experience with the different approaches, functional 
 thinking is much better suited and simpler to use for CTFE 
 goals.

 IMO that would really be a big step ahead. Because you know a 
 hammer, not everything is a nail...


I think this is too vague to lead anywhere. At least you should 
identify specific problems with D's toolset. And if you have 
concrete ideas for improvements, you should desribe them in more 
detail, spelling out how they improve upon the status quo.

If you want to have an entirely different meta programming 
system, then you should show how it would look like, and how it 
would be better than the status quo. I don't think anyone can 
make much of "make it more functional". Also, when it's 
fundamentally different from what we have now, then I don't see 
it getting into D at the moment. The language is not in a phase 
of designing fundamentals.
Jan 31 2016
parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-01-31 15:31:59 +0000, anonymous said:


 You're conflating CTFE with the other meta programming tools here. CTFE 
 is the same language as run-time D, but it doesn't have strange 
 template syntax. Templates, static if, __traits, etc. have strange 
 syntax, but they're sort of a different language already.


See my example in the other post. IMO what I want to achieve is a very 
simple example but the implementation I came up with is far from being 
simple.


 Are you maybe wishing for a nicer alternative to templates, etc?


No, my point is that CTFE and meta-programming seems to be much simpler 
and powerful if I can use a more functional programming approach for 
it. Working with powerful lists and data = code and code = data concept 
would simplify this.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Feb 02 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 2 February 2016 at 08:31:07 UTC, Robert M. Münch 
wrote:

 No, my point is that CTFE and meta-programming seems to be much 
 simpler and powerful if I can use a more functional programming 
 approach for it. Working with powerful lists and data = code 
 and code = data concept would simplify this.


Ultimately a restricted deductive programming language would be 
the better option. There is no need for compile time evaluation 
to be turing complete; after all, you want the compiler to finish 
within a fixed time limit.
Feb 02 2016
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.


  I have trouble coming up with a proper use-case to use Regex 
during CTFE; I mean, sure you can do a replacement expansion and 
encode a few bits to expand for a switch statement or something, 
but at the same time you can already do that.
Jan 31 2016
parent cym13 <cpicard openmailbox.org> writes:
On Monday, 1 February 2016 at 00:02:55 UTC, Era Scarecrow wrote:

 On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch 
 wrote:

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.


  I have trouble coming up with a proper use-case to use Regex 
 during CTFE; I mean, sure you can do a replacement expansion 
 and encode a few bits to expand for a switch statement or 
 something, but at the same time you can already do that.


Wrong thread?
Jan 31 2016
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:

 I like CTFE and the meta programming idea for languages like D.

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.

 Why not have a CTL (compile-time-language) that has access to 
 some compiler internals, that follows a more functional 
 concept? We are evaluating sequences of things to generate 
 code, include / exclude code etc.

 From my experience with the different approaches, functional 
 thinking is much better suited and simpler to use for CTFE 
 goals.

 IMO that would really be a big step ahead. Because you know a 
 hammer, not everything is a nail...


I'm not sure what is preventing you from doing that already. 
There is compile time reflection (has access to some compiler 
internals) and D support functional style.

Unless you have some specific in mind, I don't think there is 
anything we can do to help here. t seems you already have the 
pieces you want.
Feb 01 2016
parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-02-01 08:15:11 +0000, deadalnix said:


 I'm not sure what is preventing you from doing that already. There is 
 compile time reflection (has access to some compiler internals) and D 
 support functional style. Unless you have some specific in mind, I 
 don't think there is anything we can do to help here. t seems you 
 already have the pieces you want.


See my other example. Yes, maybe I can do everything already, but it's 
not straightforward. I have to fiddle around a lot to get it to work.

Of course that can be my incompetence with CTFE. However, my experience 
and intuition is that what I would like to use CTFE for 
(code-generation depending on some other parts of the code like enums, 
presence of member functions, etc.) can be done much simpler than at 
the moment.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Feb 02 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 2 February 2016 at 08:34:38 UTC, Robert M. Münch 
wrote:

 On 2016-02-01 08:15:11 +0000, deadalnix said:


 I'm not sure what is preventing you from doing that already. 
 There is compile time reflection (has access to some compiler 
 internals) and D support functional style. Unless you have 
 some specific in mind, I don't think there is anything we can 
 do to help here. t seems you already have the pieces you want.


 See my other example. Yes, maybe I can do everything already, 
 but it's not straightforward. I have to fiddle around a lot to 
 get it to work.

 Of course that can be my incompetence with CTFE. However, my 
 experience and intuition is that what I would like to use CTFE 
 for (code-generation depending on some other parts of the code 
 like enums, presence of member functions, etc.) can be done 
 much simpler than at the moment.


That is definitely true that the compile time API is kind of 
screwy. That's definitively not you. I think the best path 
forward at this stage is to provide nice API as a library on top 
of it.
Feb 02 2016
parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-02-02 08:39:34 +0000, deadalnix said:


 That is definitely true that the compile time API is kind of screwy. 
 That's definitively not you. I think the best path forward at this 
 stage is to provide nice API as a library on top of it.


And if we do this, it's only a small step to add a functional layer to 
deal with this API where I can chain tranformations to generate my code.

My point to discuss was, that I think the combination "compile time 
API" + "compile time suited language" is something different than the 
road currently followed.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Feb 02 2016
prev sibling parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-01-31 13:59:06 +0000, Robert M. Münch said:


 I like CTFE and the meta programming idea for languages like D.
 
 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting compiled. 
 IMO the use-cases a pretty different and doing CTFE, code-generation 
 etc. needs some other approach. If you look at all the strange template 
 syntax, strange hacks etc. it's all far from being obvious.
 
 Why not have a CTL (compile-time-language) that has access to some 
 compiler internals, that follows a more functional concept? We are 
 evaluating sequences of things to generate code, include / exclude code 
 etc.
 
 From my experience with the different approaches, functional thinking 
 is much better suited and simpler to use for CTFE goals.
 
 IMO that would really be a big step ahead. Because you know a hammer, 
 not everything is a nail...


Here is a link http://terralang.org/ where these guys mix Lua and their 
compiled language, to achieve what I was thinking about in the same 
line:

"In this use-case, Lua serves as a powerful meta-programming language. 
You can think of it as a replacement for C++ template metaprogramming3 
or C preprocessor X-Macros4 with better syntax and nicer properties 
such as hygiene5."

Maybe this better explains, where I think it makes sense to seperate 
the two levels: language for the buidling-step, and language for the 
actual solution.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Feb 08 2016