www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Validate static asserts

reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
I have bunch of `static assert(<condition>, <message>)` in my 
code and would like to validate that specific code triggers 
specific assert by checking what `<message>` is thrown.

Right now I do `static assert(!__traits(compiles, { <my code> 
}));` but since `<my code>` might not compile due to many 
different reasons, I might not be testing original `static 
assert` and might miss breaking change.

One way to do this is to extract `<condition>` and `<message>` 
into some function and test it outside of `static assert`:
```d
auto check()
{
     return tuple(false, // check result  ('false' is just for 
example)
                  "message");
}

void f()
{
     enum result = check();
     static assert(result.condition, result.message);
}

unittest
{
     enum result = check();
     static assert(result.condition);
     static assert(result.message == "message");
}
```
But I don't like this approach because unit test doesn't really 
test `f()` (it tests duplicated code) so it can't guarantee that 
`f()` works as expected.


Is there a way to validate static asserts in unit tests?
Sep 09 2022
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/9/22 10:35 AM, Andrey Zherikov wrote:

 I have bunch of `static assert(<condition>, <message>)` in my code and 
 would like to validate that specific code triggers specific assert by 
 checking what `<message>` is thrown.
 
 Right now I do `static assert(!__traits(compiles, { <my code> }));` but 
 since `<my code>` might not compile due to many different reasons, I 
 might not be testing original `static assert` and might miss breaking 
 change.
 
 One way to do this is to extract `<condition>` and `<message>` into some 
 function and test it outside of `static assert`:
 ```d
 auto check()
 {
      return tuple(false, // check result  ('false' is just for example)
                   "message");
 }
 
 void f()
 {
      enum result = check();
      static assert(result.condition, result.message);
 }
 
 unittest
 {
      enum result = check();
      static assert(result.condition);
      static assert(result.message == "message");
 }
 ```
 But I don't like this approach because unit test doesn't really test 
 `f()` (it tests duplicated code) so it can't guarantee that `f()` works 
 as expected.
 
 
 Is there a way to validate static asserts in unit tests?


Even this doesn't validate that you get the right message for the 
expected failure.

You can just test the message generation (and use a function for that). 
That's easier than doing some weird tuple thing.

But validating that the correct message comes out of a failed 
compilation can only be done outside compilation.

-Steve
Sep 09 2022
prev sibling next sibling parent Paul Backus <snarwin gmail.com> writes:
On Friday, 9 September 2022 at 14:35:33 UTC, Andrey Zherikov 
wrote:

 I have bunch of `static assert(<condition>, <message>)` in my 
 code and would like to validate that specific code triggers 
 specific assert by checking what `<message>` is thrown.


It sounds like maybe your goal here is to test that attempting to 
compile a specific piece of code will result in a specific error 
message being *shown to the user*.

Unfortunately, the D compiler does not allow you to introspect on 
error messages, so it is impossible to write a `unittest` that 
covers this requirement. Instead, you will have to write an 
external script or program that attempts to compile a test 
program and checks the output for the expected error message.
Sep 09 2022
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/9/22 07:35, Andrey Zherikov wrote:


 might not compile due to many different reasons


I faced a related situation recently: My error string generation was 
buggy, which taught me that the compiler does not even compile the 
string part of 'static assert' in the 'true' case.

The following program compiles! :)

void main() {
     static assert (true, "hello" / WAT);
}


 Is there a way to validate static asserts in unit tests?


I added and removed '&& false' to every 'static assert' condition 
manually one by one. :/

Perhaps a new compiler switch can compile every 'static assert' with an 
automatic 'false' and dump all their text to the output.

Ali
Sep 09 2022
parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Friday, 9 September 2022 at 15:22:30 UTC, Ali Çehreli wrote:

 I added and removed '&& false' to every 'static assert' 
 condition manually one by one. :/


It's not CI-friendly :(


 Perhaps a new compiler switch can compile every 'static assert' 
 with an automatic 'false' and dump all their text to the output.


What's about new `compileOutput` trait that returns compiler 
output?
```d
static assert(__traits(compileOutput, { <my code> }) == 
"message");
```
Sep 09 2022
parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 9 September 2022 at 16:41:54 UTC, Andrey Zherikov 
wrote:

 What's about new `compileOutput` trait that returns compiler 
 output?
 ```d
 static assert(__traits(compileOutput, { <my code> }) == 
 "message");
 ```


As a compiler dev, that sounds terrifying. It would make 
basically every change to dmd a breaking change.
Sep 09 2022
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/9/22 10:35, Dennis wrote:

 On Friday, 9 September 2022 at 16:41:54 UTC, Andrey Zherikov wrote:

 What's about new `compileOutput` trait that returns compiler output?
 ```d
 static assert(__traits(compileOutput, { <my code> }) == "message");
 ```


 As a compiler dev, that sounds terrifying. It would make basically every
 change to dmd a breaking change.


For that very reason, I wrote the function 'assertErrorStringContains()' 
a couple of days ago to ensure *my* strings were in the output:

A precondition:

     void test_1(int i)
     in (i > 0, fooError("The value must be positive", i, 42))
     {
         // ...
     }

A unit test that ensures it fails and checks string pieces appear in the 
output:

     /*
         The .msg text of the error contains both the error string and 
the data
         that is included in the error.
     */
     assertErrorStringContains(() => test_1(-1), [ "The value must be 
positive",
                                                   "-1, 42" ]);
Here is assertErrorStringContains:

     // Assert that the expression throws an Error object and that its 
string
     // representation contains all expected strings.
     void assertErrorStringContains(void delegate() expr, string[] expected)
     {
         bool thrown = false;

         try
         {
             expr();

         }
         catch (Error err)
         {
             thrown = true;

             import std.algorithm : any, canFind, splitter;
             import std.conv : to;
             import std.format : format;

             auto lines = err.to!string.splitter('\n');
             foreach (exp; expected)
             {
                 assert(lines.any!(line => line.canFind(exp)),
                        format!"Failed to find \"%s\" in the output: 
%-(\n  |%s%)"(
                            exp, lines));
             }
         }

         assert(thrown);
     }

Ali
Sep 09 2022
prev sibling parent user1234 <user1234 12.de> writes:
On Friday, 9 September 2022 at 17:35:44 UTC, Dennis wrote:

 On Friday, 9 September 2022 at 16:41:54 UTC, Andrey Zherikov 
 wrote:

 What's about new `compileOutput` trait that returns compiler 
 output?
 ```d
 static assert(__traits(compileOutput, { <my code> }) == 
 "message");
 ```


 As a compiler dev, that sounds terrifying. It would make 
 basically every change to dmd a breaking change.


Ah yes, it is so stupid that error message are part of the 
semantics
Sep 09 2022