www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Source code output

reply "JS" <js.mdnq gmail.com> writes:
With heavy ctfe code generation usage is it possible to have the 
d compiler output the source code after all mixin templates have 
been "used"? This way it is easier to visually check for errors 
in the generated code.

I imagine one could use pragma in a "special way" to do this but 
I was hoping for something more direct.
Jul 16 2013
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jul 17, 2013 at 05:27:34AM +0200, JS wrote:

 With heavy ctfe code generation usage is it possible to have the d
 compiler output the source code after all mixin templates have been
 "used"? This way it is easier to visually check for errors in the
 generated code.
 
 I imagine one could use pragma in a "special way" to do this but I
 was hoping for something more direct.


Yeah I've been looking for this too. Once you have templates and mixins
nested deeply enough, it can be near impossible to figure out just
exactly *what* is being compiled at the end of it all.

Though I suppose the common approach is to generate a string
representing the code, and then using mixin(str) to compile it; this
lets you use pragma(msg) to output str and thereby see what exactly is
being compiled. But this may not be possible in some cases.


T

-- 
Be in denial for long enough, and one day you'll deny yourself of things you
wish you hadn't.
Jul 16 2013
next sibling parent "JS" <js.mdnq gmail.com> writes:
On Wednesday, 17 July 2013 at 05:29:26 UTC, H. S. Teoh wrote:

 On Wed, Jul 17, 2013 at 05:27:34AM +0200, JS wrote:

 With heavy ctfe code generation usage is it possible to have 
 the d
 compiler output the source code after all mixin templates have 
 been
 "used"? This way it is easier to visually check for errors in 
 the
 generated code.
 
 I imagine one could use pragma in a "special way" to do this 
 but I
 was hoping for something more direct.


 Yeah I've been looking for this too. Once you have templates 
 and mixins
 nested deeply enough, it can be near impossible to figure out 
 just
 exactly *what* is being compiled at the end of it all.

 Though I suppose the common approach is to generate a string
 representing the code, and then using mixin(str) to compile it; 
 this
 lets you use pragma(msg) to output str and thereby see what 
 exactly is
 being compiled. But this may not be possible in some cases.


 T


That is what I use but at some point it will become worthless. I 
was thinking it may be possible to optionally pragma the code 
depending on a symbol like __codegen, which would then trigger 
all the appropriate pragma's, the problem is, the output will be 
a mess because of the nesting(code duplication). Only the most 
outside mixin's pragma needs to be triggered, the one directly 
used in the code(not inside another template).
Jul 16 2013
prev sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 17 July 2013 at 05:29:26 UTC, H. S. Teoh wrote:

 On Wed, Jul 17, 2013 at 05:27:34AM +0200, JS wrote:

 With heavy ctfe code generation usage is it possible to have 
 the d
 compiler output the source code after all mixin templates have 
 been
 "used"? This way it is easier to visually check for errors in 
 the
 generated code.
 
 I imagine one could use pragma in a "special way" to do this 
 but I
 was hoping for something more direct.


 Yeah I've been looking for this too. Once you have templates 
 and mixins
 nested deeply enough, it can be near impossible to figure out 
 just
 exactly *what* is being compiled at the end of it all.

 Though I suppose the common approach is to generate a string
 representing the code, and then using mixin(str) to compile it; 
 this
 lets you use pragma(msg) to output str and thereby see what 
 exactly is
 being compiled. But this may not be possible in some cases.


 T


I would love to have .codeof that can be applied to as many 
things as possible.

Or __traits(code, ...)

Or __traits(code, expansion_depth, ...)

So as to be able to drill down in to the code of something to the 
necessary level but no further.


Btw, this isn't only useful for printing out, it would be a 
powerful compile-time tool in general:

E.g. given a function
void foo(int a, int b)
{ body }

it would then be possible to do:

void fooCompileTimeCurried(int a)(int b)
{
     mixin(stripSig(__traits(code, 1, foo)));
}

to get an optimised version of foo for a == some_ct_value
Jul 17 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-07-17 05:27, JS wrote:

 With heavy ctfe code generation usage is it possible to have the d
 compiler output the source code after all mixin templates have been
 "used"? This way it is easier to visually check for errors in the
 generated code.

 I imagine one could use pragma in a "special way" to do this but I was
 hoping for something more direct.


The Eclipse plugin Descent had a nice compile time view. In general it 
showed how the compiler lowers some features to other lower level 
features. I.e. "scope" is lowered to try-catch-finally. It also showed 
the result of string and template mixins. It was really nice to have. 
Too bad the plugin was abandoned. It never got any real support for D2.

-- 
/Jacob Carlborg
Jul 17 2013
next sibling parent dennis luehring <dl.soluz gmx.net> writes:
Am 17.07.2013 09:33, schrieb Jacob Carlborg:

 On 2013-07-17 05:27, JS wrote:

 With heavy ctfe code generation usage is it possible to have the d
 compiler output the source code after all mixin templates have been
 "used"? This way it is easier to visually check for errors in the
 generated code.

 I imagine one could use pragma in a "special way" to do this but I was
 hoping for something more direct.


 The Eclipse plugin Descent had a nice compile time view. In general it
 showed how the compiler lowers some features to other lower level
 features. I.e. "scope" is lowered to try-catch-finally. It also showed
 the result of string and template mixins. It was really nice to have.
 Too bad the plugin was abandoned. It never got any real support for D2.


some videos of Descent: http://www.youtube.com/user/asterite
Jul 17 2013
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/17/13 4:33 AM, Jacob Carlborg wrote:

 On 2013-07-17 05:27, JS wrote:

 With heavy ctfe code generation usage is it possible to have the d
 compiler output the source code after all mixin templates have been
 "used"? This way it is easier to visually check for errors in the
 generated code.

 I imagine one could use pragma in a "special way" to do this but I was
 hoping for something more direct.


 The Eclipse plugin Descent had a nice compile time view. In general it
 showed how the compiler lowers some features to other lower level
 features. I.e. "scope" is lowered to try-catch-finally. It also showed
 the result of string and template mixins. It was really nice to have.
 Too bad the plugin was abandoned. It never got any real support for D2.


Since Descent used a port of the D compiler to Java, I think the same 
can be done by tweaking dmd. You have the generated AST after semantic 
analysis and you can easily output that. But of course you need to be 
able to convert the AST to string with correct indentation.

In general, I think a compiler should *really* care about setting line 
and column numbers to AST nodes and assisting the user as much as 
possible, like providing a way to output the final representation of the 
program as a String.

For example compiling this program with D:

---
string foo() {
   return "void bar() {\n  return 1 + 2\n}";
}

void main() {
   mixin(foo());
}
---

gives this error:

foo.d(7): Error: expression expected, not ')'
foo.d(8): Error: found '}' when expecting ';' following statement
foo.d(8): Error: found 'EOF' when expecting '}' following compound statement
foo.d(7): Error: + has no effect in expression (1 + 2 + 0)

Can you see where is the error?

I'll show here some ideas which D can use, which we have implemented in 
Crystal.

1. Show errors in expanded macros:

Compiling this (similar to D's string mixins):

---
macro define_method(x)
   "

       x + y + )
   end
   "
end

define_method "hello"
---

Gives this error:

Error in foo.cr:9: macro didn't expand to a valid program, it expanded to:

================================================================================
--------------------------------------------------------------------------------
   1.
   2.   def hello(x, y)
   3.       x + y + )
   4.   end
   5.
--------------------------------------------------------------------------------
Syntax error in expanded macro:3: unexpected token: )

       x + y + )
               ^
================================================================================

define_method "hello"
^~~~~~~~~~~~~

2. The above should also work with semantic errors:

---
macro define_method(x)
   "

     x + y
   end
   "
end

define_method "hello"

hello(1, 'a')
---

gives this error:

Error in /Users/asterite-manas/Projects/crystal/foo.cr:11: instantiating 
'hello(Int32, Char)'

hello(1, 'a')
^~~~~

in macro 'define_method' 
/Users/asterite-manas/Projects/crystal/foo.cr:1, line 3:

   1.
   2.   def hello(x, y)
   3.     x + y
   4.   end
   5.

     x + y
       ^


Overloads are:










Couldn't find overloads for these types:


---

If you don't provide clear errors for metaprogramming, even though it 
can be very powerful, if you don't understand the errors you go slower 
and slower and maybe eventually you decide not to use those features.

For macros (string mixins), associate the AST nodes to virtual files 
whose source code is the expanded source code. Keep track of line *and* 
column numbers (lines alone are not enough when you have long lines).
Jul 17 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-07-17 16:13, Ary Borenszweig wrote:


 Since Descent used a port of the D compiler to Java, I think the same
 can be done by tweaking dmd. You have the generated AST after semantic
 analysis and you can easily output that. But of course you need to be
 able to convert the AST to string with correct indentation.

 In general, I think a compiler should *really* care about setting line
 and column numbers to AST nodes and assisting the user as much as
 possible, like providing a way to output the final representation of the
 program as a String.

 For example compiling this program with D:

 ---
 string foo() {
    return "void bar() {\n  return 1 + 2\n}";
 }

 void main() {
    mixin(foo());
 }
 ---

 gives this error:

 foo.d(7): Error: expression expected, not ')'
 foo.d(8): Error: found '}' when expecting ';' following statement
 foo.d(8): Error: found 'EOF' when expecting '}' following compound
 statement
 foo.d(7): Error: + has no effect in expression (1 + 2 + 0)

 Can you see where is the error?

 I'll show here some ideas which D can use, which we have implemented in
 Crystal.
 [Snip]


That's how error messages should look like :)

-- 
/Jacob Carlborg
Jul 17 2013