www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is it possible to obtain textual representation of an arbitary code?

reply Oleksii Skidan <al.skidan gmail.com> writes:
Hi,

I wonder if it's possible to convert D language code into a 
string at compile time? C/C++ preprocessor has this feature 

argument into a string constant. See the following code snippet 
for example:

```cplusplus
#define ASSERT(EXPR) some_function((EXPR), #EXPR)

// Elsewhere in the code:
void some_function(bool const condition, char const* const expr) {
   if (!condition) {
      printf("Assertion failed for: %s\n", expr);
   }
}

// Usage:
ASSERT(a == b); // Will print "Assertion failed for: a == b"
```

I could imagine a mixin-based solution in D:
```d
// Usage:
ASSERT!"a == b";
```
But it seems a bit alien to me. First of all, it kind of 
stringly-typed one. Secondly, neither IDEs nor advanced text 
editors are able to figure out that the string contains actual D 
code, and so neither syntax highlighting nor code assistance work 
with this approach.

BR,
--
Oleksii
Jan 26 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 11:18:21 Oleksii Skidan via Digitalmars-d-learn 
wrote:

 Hi,

 I wonder if it's possible to convert D language code into a
 string at compile time? C/C++ preprocessor has this feature

 argument into a string constant. See the following code snippet
 for example:

 ```cplusplus
 #define ASSERT(EXPR) some_function((EXPR), #EXPR)

 // Elsewhere in the code:
 void some_function(bool const condition, char const* const expr) {
    if (!condition) {
       printf("Assertion failed for: %s\n", expr);
    }
 }

 // Usage:
 ASSERT(a == b); // Will print "Assertion failed for: a == b"
 ```

 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of
 stringly-typed one. Secondly, neither IDEs nor advanced text
 editors are able to figure out that the string contains actual D
 code, and so neither syntax highlighting nor code assistance work
 with this approach.


You can use stringof on symbols to get their string representation, but you
can't get sections of code that way. D makes it very easy to turn strings
into code using mixin statements, but it doesn't provide ways to take
arbitrary code and turn it into strings. And in many cases, the compiler
wouldn't necessarily have access to the code anyway, just like you wouldn't
in C/C++. C/C++ can do it with macros but not arbitrary code, and D doesn't
have the equivalent of C/C++ macros. Arguably the closest thing that D has
to C/C++ macros is the ability to mixin strings, and if you're mixing in a
string, you already have the code as a string.

And yes, if you use a lot of metaprogramming stuff like string mixins, then
you're not going to have the code to see in your text editor, but that's
what happens when you generate code at compile time rather than having the
code sit in a file.

However, if you want syntax highlighting for a string literal that contains
code, you can use q{} to contain the string literal rather that "" or `` -
e.g. q{auto i = 42;} instead of "auto i = 42;", then most text editors will
highlight the string as if it were just code.

- Jonathan M Davis
Jan 26 2018
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan wrote:


 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of 
 stringly-typed one. Secondly, neither IDEs nor advanced text 
 editors are able to figure out that the string contains actual 
 D code, and so neither syntax highlighting nor code assistance 
 work with this approach.


Token strings are intended for this and editors *should* 
highlight them (don't know if any currently do):

https://dlang.org/spec/lex.html#token_strings
Jan 26 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 11:32:42 Mike Parker via Digitalmars-d-learn 
wrote:

 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan wrote:

 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of
 stringly-typed one. Secondly, neither IDEs nor advanced text
 editors are able to figure out that the string contains actual
 D code, and so neither syntax highlighting nor code assistance
 work with this approach.


 Token strings are intended for this and editors *should*
 highlight them (don't know if any currently do):

 https://dlang.org/spec/lex.html#token_strings


So that's what they're called. I can never remember (though I rarely use
them either, since I actually prefer that strings be highlighted as strings
and not code even if they contain code). vim definitely highlights them as
code, and I would expect most editors that don't understand them to
highlight them as if they were code, since q{} looks like code. If anything,
it would take a text editor that understood D quite well to highlight it as
a string (though really, no editor should be doing that, since the main
point of using token strings is for them to be highlighted as code).

- Jonathan M Davis
Jan 26 2018
prev sibling next sibling parent reply Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:

 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan 
 wrote:


 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of 
 stringly-typed one. Secondly, neither IDEs nor advanced text 
 editors are able to figure out that the string contains actual 
 D code, and so neither syntax highlighting nor code assistance 
 work with this approach.


 Token strings are intended for this and editors *should* 
 highlight them (don't know if any currently do):

 https://dlang.org/spec/lex.html#token_strings


Seems like I have to add some context into this conversation: I'm 
writing a poor man's testing framework, since it's the best and 
easiest way to learn D ;-)

I'm trying to achieve something similar to 
`Catch2``REQUIRE`macro. To be honest, I did not know about toking 
strings until today, and I don't know D much. Here's what I came 
up with so far:

```d
string require(string expr)(string file = __FILE__, int line = 
__LINE__)
{
     import std.array, std.conv;
     return q{
         if (!($expr)) {
             import std.stdio;
             writeln("Test failed  ", `$file`, ":", $line, "\n",
                     "  Expected: `", `$expr`, "` to be 
`true`.\n");
         }
     }.replace("$expr", expr)
      .replace("$file", file)
      .replace("$line", to!string(line));
}

```

That code snippet uses token strings to compose an if statement 
that basically checks whether the given condition holds. That 
looks okay-ish to me, the usage of that function is not pretty 
though:

```d
unittest
{
     mixin(require!q{false}); // This test will fail.
}
```

It would be awesome if I could write something like the this 
instead:

```d
unittest
{
     require!q{false};
}
```

At first glance it seems like I could have moved the `mixin` 
statement into the `require` function itself, but that would not 
really work. Consider the following snippet:

```d
unittest
{
     float value = 3f;
     require!q{value == 3f}; // This line won't compile.
}
```

That code won't even compile, since `value` exists in `unittest` 
scope, which is not visible to the `require` function.

--
Oleksii
Jan 26 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 12:30:03 Oleksii Skidan via Digitalmars-d-learn 
wrote:

 On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:

 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan

 wrote:

 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of
 stringly-typed one. Secondly, neither IDEs nor advanced text
 editors are able to figure out that the string contains actual
 D code, and so neither syntax highlighting nor code assistance
 work with this approach.


 Token strings are intended for this and editors *should*
 highlight them (don't know if any currently do):

 https://dlang.org/spec/lex.html#token_strings


 Seems like I have to add some context into this conversation: I'm
 writing a poor man's testing framework, since it's the best and
 easiest way to learn D ;-)

 I'm trying to achieve something similar to
 `Catch2``REQUIRE`macro. To be honest, I did not know about toking
 strings until today, and I don't know D much. Here's what I came
 up with so far:

 ```d
 string require(string expr)(string file = __FILE__, int line =
 __LINE__)
 {
      import std.array, std.conv;
      return q{
          if (!($expr)) {
              import std.stdio;
              writeln("Test failed  ", `$file`, ":", $line, "\n",
                      "  Expected: `", `$expr`, "` to be
 `true`.\n");
          }
      }.replace("$expr", expr)
       .replace("$file", file)
       .replace("$line", to!string(line));
 }

 ```

 That code snippet uses token strings to compose an if statement
 that basically checks whether the given condition holds. That
 looks okay-ish to me, the usage of that function is not pretty
 though:

 ```d
 unittest
 {
      mixin(require!q{false}); // This test will fail.
 }
 ```

 It would be awesome if I could write something like the this
 instead:

 ```d
 unittest
 {
      require!q{false};
 }
 ```

 At first glance it seems like I could have moved the `mixin`
 statement into the `require` function itself, but that would not
 really work. Consider the following snippet:

 ```d
 unittest
 {
      float value = 3f;
      require!q{value == 3f}; // This line won't compile.
 }
 ```

 That code won't even compile, since `value` exists in `unittest`
 scope, which is not visible to the `require` function.


Why are you using strings for any of this? Printing out the expression is
kind of pointless. If you have the file and line number (which an
AssertError will give you), then you know where the failure is, and you can
see the expression. All of this extra machinery is just going to increase
your compile times for no benefit. So, what you're doing here is objectively
worse than just using assertions.

There might be some value if you had something like

assertEqual(lhs, rhs);

and then on failure, you printed the values that were being compared, since
that's not necessarily information that's in the code, but the expressions
themselves _are_ already in the code, so printing them out doesn't help any.

But even if you have helper functions that take the values separately so
that they can be printed, in my experience, the extra template
instantiations required to use helper functions like that everywhere in unit
tests increases the compilation times (and memory required) enough that it's
not worth it, especially when you consider that once the tests are passing,
all of that extra machinery does you no good whatsoever. Ultimately, it just
costs less to temporarily make an adjustment to the test and rerun it if you
need more information.

If you don't think that simply using assertions for unit tests is good
enough, then I'd suggest that you look at
https://code.dlang.org/packages/unit-threaded

- Jonathan M Davis
Jan 26 2018
next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
wrote:

 If you don't think that simply using assertions for unit tests 
 is good enough, then I'd suggest that you look at 
 https://code.dlang.org/packages/unit-threaded


There's also https://code.dlang.org/packages/fluent-asserts which 
shows detailed error messages.
Jan 26 2018
parent Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 13:25:12 UTC, Seb wrote:

 On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
 wrote:

 If you don't think that simply using assertions for unit tests 
 is good enough, then I'd suggest that you look at 
 https://code.dlang.org/packages/unit-threaded


 There's also https://code.dlang.org/packages/fluent-asserts 
 which shows detailed error messages.


Thanks,

That one look great! I like how it prints the source code of the 
test, so that you have the context within which the test failed.
Jan 26 2018
prev sibling next sibling parent reply Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
wrote:

 Why are you using strings for any of this? Printing out the 
 expression is kind of pointless. If you have the file and line 
 number (which an AssertError will give you), then you know 
 where the failure is, and you can see the expression. All of 
 this extra machinery is just going to increase your compile 
 times for no benefit. So, what you're doing here is objectively 
 worse than just using assertions.

 There might be some value if you had something like

 assertEqual(lhs, rhs);

 and then on failure, you printed the values that were being 
 compared, since that's not necessarily information that's in 
 the code, but the expressions themselves _are_ already in the 
 code, so printing them out doesn't help any.


That's actually what I have:

```
// Usage:
//  mixin(requireEq!q{expected, actual});
// Checks whether the `actual` value is equal to `reauired`.
string requireEq(string expr)(string file = __FILE__, int line = 
__LINE__)
{
     import std.array, std.conv, std.string;

     auto parts = expr.split(",");

     return q{
         {
             bool equal = false;

             static if (__traits(isFloating, $expected) || 
__traits(isFloating, $actual)) {
                 import std.math;
                 equal = approxEqual($expected, $actual);
             } else {
                 equal = $expected == $actual;
             }

             if (!equal) {
                 import std.stdio, std.conv;
                 writeln("Test failed  ", `$file`, ":", $line, 
"\n",
                         "  Expected `", `$actual`, "` to be `", 
to!string($expected), "`, but got `",
                                                                 
to!string($actual), "`\n");
             }
         }
     }.replace("$expected", parts[0].strip())
      .replace("$actual",   parts[1].strip())
      .replace("$file",     file)
      .replace("$line",     to!string(line));
}

```

The sample code I posted before, was way much simpler than this.


 But even if you have helper functions that take the values 
 separately so that they can be printed, in my experience, the 
 extra template instantiations required to use helper functions 
 like that everywhere in unit tests increases the compilation 
 times (and memory required) enough that it's not worth it, 
 especially when you consider that once the tests are passing, 
 all of that extra machinery does you no good whatsoever. 
 Ultimately, it just costs less to temporarily make an 
 adjustment to the test and rerun it if you need more 
 information.


I won't argue against asserts. I agree with your point: they are 
fine for simple cases, and they are definitely faster to compile 
than any other framework.

My ultimate goal is to implement a BDD framework, so that I could 
write tests in the most productive (for myself) way. I'm not 
aiming for a production-ready quality, a toy framework that suits 
my needs would be just fine.


 If you don't think that simply using assertions for unit tests 
 is good enough, then I'd suggest that you look at 
 https://code.dlang.org/packages/unit-threaded


Thanks, I'll look at it.

Also, I have a vague idea that aliases may be the key to the 
desired functionality.

BR
--
Oleksii
Jan 26 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 14:23:20 Oleksii Skidan via Digitalmars-d-learn 
wrote:

 On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis

 wrote:

 Why are you using strings for any of this? Printing out the
 expression is kind of pointless. If you have the file and line
 number (which an AssertError will give you), then you know
 where the failure is, and you can see the expression. All of
 this extra machinery is just going to increase your compile
 times for no benefit. So, what you're doing here is objectively
 worse than just using assertions.

 There might be some value if you had something like

 assertEqual(lhs, rhs);

 and then on failure, you printed the values that were being
 compared, since that's not necessarily information that's in
 the code, but the expressions themselves _are_ already in the
 code, so printing them out doesn't help any.


 That's actually what I have:

 ```
 // Usage:
 //  mixin(requireEq!q{expected, actual});
 // Checks whether the `actual` value is equal to `reauired`.
 string requireEq(string expr)(string file = __FILE__, int line =
 __LINE__)
 {
      import std.array, std.conv, std.string;

      auto parts = expr.split(",");

      return q{
          {
              bool equal = false;

              static if (__traits(isFloating, $expected) ||
 __traits(isFloating, $actual)) {
                  import std.math;
                  equal = approxEqual($expected, $actual);
              } else {
                  equal = $expected == $actual;
              }

              if (!equal) {
                  import std.stdio, std.conv;
                  writeln("Test failed  ", `$file`, ":", $line,
 "\n",
                          "  Expected `", `$actual`, "` to be `",
 to!string($expected), "`, but got `",

 to!string($actual), "`\n");
              }
          }
      }.replace("$expected", parts[0].strip())
       .replace("$actual",   parts[1].strip())
       .replace("$file",     file)
       .replace("$line",     to!string(line));
 }

 ```

 The sample code I posted before, was way much simpler than this.


If that's what you want, there's still no reason to use strings. Just take
the two arguments separately as normal function arguments and print their
values when there's a failure. Why are you using strings and mixins for any
of this? You could do something like

void assertEqual(T, U)(T lhs, U rhs, string file = __FILE__,
                       size_t line = __LINE__)
{
    enforce!AssertError(lhs == rhs,
                        format("assertEqual failed. lhs: %s, rhs: %s",
                               lhs, rhs),
                        file, line);
}

- Jonathan M Davis
Jan 26 2018
prev sibling parent Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
wrote:

 On Friday, January 26, 2018 12:30:03 Oleksii Skidan via 
 Digitalmars-d-learn wrote:

 On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:

 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan

 wrote:

 [...]


 Token strings are intended for this and editors *should* 
 highlight them (don't know if any currently do):

 https://dlang.org/spec/lex.html#token_strings


 Seems like I have to add some context into this conversation: 
 I'm writing a poor man's testing framework, since it's the 
 best and easiest way to learn D ;-)

 I'm trying to achieve something similar to 
 `Catch2``REQUIRE`macro. To be honest, I did not know about 
 toking strings until today, and I don't know D much. Here's 
 what I came up with so far:

 ```d
 string require(string expr)(string file = __FILE__, int line =
 __LINE__)
 {
      import std.array, std.conv;
      return q{
          if (!($expr)) {
              import std.stdio;
              writeln("Test failed  ", `$file`, ":", $line, 
 "\n",
                      "  Expected: `", `$expr`, "` to be
 `true`.\n");
          }
      }.replace("$expr", expr)
       .replace("$file", file)
       .replace("$line", to!string(line));
 }

 ```

 That code snippet uses token strings to compose an if 
 statement that basically checks whether the given condition 
 holds. That looks okay-ish to me, the usage of that function 
 is not pretty though:

 ```d
 unittest
 {
      mixin(require!q{false}); // This test will fail.
 }
 ```

 It would be awesome if I could write something like the this 
 instead:

 ```d
 unittest
 {
      require!q{false};
 }
 ```

 At first glance it seems like I could have moved the `mixin` 
 statement into the `require` function itself, but that would 
 not really work. Consider the following snippet:

 ```d
 unittest
 {
      float value = 3f;
      require!q{value == 3f}; // This line won't compile.
 }
 ```

 That code won't even compile, since `value` exists in 
 `unittest` scope, which is not visible to the `require` 
 function.


 Why are you using strings for any of this? Printing out the 
 expression is kind of pointless. If you have the file and line 
 number (which an AssertError will give you), then you know 
 where the failure is, and you can see the expression. All of 
 this extra machinery is just going to increase your compile 
 times for no benefit. So, what you're doing here is objectively 
 worse than just using assertions.

 There might be some value if you had something like

 assertEqual(lhs, rhs);

 and then on failure, you printed the values that were being 
 compared, since that's not necessarily information that's in 
 the code, but the expressions themselves _are_ already in the 
 code, so printing them out doesn't help any.

 But even if you have helper functions that take the values 
 separately so that they can be printed, in my experience, the 
 extra template instantiations required to use helper functions 
 like that everywhere in unit tests increases the compilation 
 times (and memory required) enough that it's not worth it, 
 especially when you consider that once the tests are passing, 
 all of that extra machinery does you no good whatsoever. 
 Ultimately, it just costs less to temporarily make an 
 adjustment to the test and rerun it if you need more 
 information.

 If you don't think that simply using assertions for unit tests 
 is good enough, then I'd suggest that you look at 
 https://code.dlang.org/packages/unit-threaded

 - Jonathan M Davis


I've just realized that I can actually make the test code more 
pleasant if I use string concatenation. For example, this test:

```
unittest
{
     import testing;

      {
         Particle a = Particle(0.9, 0);
         Particle b = Particle(2.1, 0);
         Constraint c = Constraint(a, b);
         c.distance = 1f;
         c.solve();
         mixin(requireEq!q{1f, a.x});
         mixin(requireEq!q{0f, a.y});
         mixin(requireEq!q{2f, b.x});
         mixin(requireEq!q{0f, b.y});
     }
}
```

could be written as:

```
unittest
{
     import testing;

     {
         Particle a = Particle(0.9, 0);
         Particle b = Particle(2.1, 0);
         Constraint c = Constraint(a, b);
         c.distance = 1f;
         c.solve();
         mixin(
             requireEq!q{1f, a.x} ~
             requireEq!q{0f, a.y} ~
             requireEq!q{2f, b.x} ~
             requireEq!q{0f, b.y}
         );
     }
}
```
That code looks a little bit unusual, but I guess I can get used 
to it. Seems like I can write a test scenario per mixin.
Jan 26 2018
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:

 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan 
 wrote:


 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of 
 stringly-typed one. Secondly, neither IDEs nor advanced text 
 editors are able to figure out that the string contains actual 
 D code, and so neither syntax highlighting nor code assistance 
 work with this approach.


 Token strings are intended for this and editors *should* 
 highlight them (don't know if any currently do):

 https://dlang.org/spec/lex.html#token_strings


LOL, all do that: https://imgur.com/a/UoHpz.
Jan 26 2018
prev sibling parent timotheecour <timothee.cour2 gmail.com> writes:
On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan wrote:

 Hi,

 I wonder if it's possible to convert D language code into a 
 string at compile time? C/C++ preprocessor has this feature 

 argument into a string constant. See the following code snippet 
 for example:



See feature request I suggested here:

https://forum.dlang.org/thread/mailman.2377.1516230845.9493.digital
ars-d puremagic.com __ARGS__ : allow access to (stringified) arguments, as C's
`#arg` macro
Jan 26 2018