www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - unittest "name" {}

reply Dennis <dkorpel gmail.com> writes:
Names on unittests can be useful for documentation, a better 
message on failure, or to selectively run a specific test. D has 
no built-in way to name unittests, but the most popular library 
test runner ([silly](https://code.dlang.org/packages/silly)) uses 
the first string UDA as a name:

```D
 ("Johny")
unittest {
	// This unittest is named Johny
}
```

This isn't too pretty though, and typing parentheses and quotes 
is annoying (using a QWERTY keyboard with US-layout).

I was thinking it could be improved by either allowing 
` "string"` attributes, similar to ` identifier`, or by allowing 
a string literal after the `unittest` keyword:

```
unittest "Johny" {
     // Has  ("Johny") as first UDA
}
```

This would provide a more ergonomic, standardized way of naming 
tests, and it's a very simple parser change (there's currently 
nothing allowed between `unittest {`).

In many other languages, unittests are regular functions with 
names, but 
[Zig-test](https://ziglang.org/documentation/master/#Zig-Test) 
has this syntax for comparison:

```Zig
test "Johny" {
}
```

Question to the community: do you name tests, and do you like the 
idea?
Feb 10 2023
next sibling parent reply ProtectAndHide <ProtectAndHide gmail.com> writes:
On Friday, 10 February 2023 at 21:08:38 UTC, Dennis wrote:

 ...
 Question to the community: do you name tests, and do you like 
 the idea?


Nice idea.

I'd prefer it was consistent with the import statement though:

unittest : mytest
{

}
Feb 10 2023
parent reply ProtectAndHide <ProtectAndHide gmail.com> writes:
On Friday, 10 February 2023 at 21:15:23 UTC, ProtectAndHide wrote:

 On Friday, 10 February 2023 at 21:08:38 UTC, Dennis wrote:

 ...
 Question to the community: do you name tests, and do you like 
 the idea?


 Nice idea.

 I'd prefer it was consistent with the import statement though:

 unittest : mytest
 {

 }


Also maybe a way to tell the compiler what unittest to run 
perhaps (i.e. passing in the name of the unittest (all unittests 
being the default).

unittest : myQuicktest // dmd -unittest:myQuicktest
{

}

unittest : myLongtest // dmd -unittest:myLongtest
{

}
Feb 10 2023
parent reply ProtectAndHide <ProtectAndHide gmail.com> writes:
On Friday, 10 February 2023 at 21:21:30 UTC, ProtectAndHide wrote:

 Also maybe a way to tell the compiler what unittest to run 
 perhaps (i.e. passing in the name of the unittest (all 
 unittests being the default).

 unittest : myQuicktest // dmd -unittest:myQuicktest
 {

 }

 unittest : myLongtest // dmd -unittest:myLongtest
 {

 }


better be consistent here as well:

dmd -unittest                          (as per current. runs all 
unittests)
dmd -unittest=myQuicktest              (runs only that named 
unittest)
dmd -unittest=myQuicktest,myLongtest   (runs these named 
unittests only)
Feb 10 2023
parent Mike Shah <mshah.475 gmail.com> writes:
 Question to the community: do you name tests, and do you like 
 the idea?




Definitely like the ability for named tests.

The output would be much cleaner if we could see per module how 
many unittest blocks passed along with the name of the passing or 
failing test.

It seems the proposal right now is just for a simpler naming 
scheme rather than having to use UDAs?

On Friday, 10 February 2023 at 21:45:29 UTC, ProtectAndHide wrote:


 On Friday, 10 February 2023 at 21:21:30 UTC, ProtectAndHide 
 wrote:

 Also maybe a way to tell the compiler what unittest to run 
 perhaps (i.e. passing in the name of the unittest (all 
 unittests being the default).

 unittest : myQuicktest // dmd -unittest:myQuicktest
 {

 }

 unittest : myLongtest // dmd -unittest:myLongtest
 {

 }


 better be consistent here as well:

 dmd -unittest                          (as per current. runs 
 all unittests)
 dmd -unittest=myQuicktest              (runs only that named 
 unittest)
 dmd -unittest=myQuicktest,myLongtest   (runs these named 
 unittests only)


Agreed, being able to select and run selected subsets of unit 
tests as suggested would be incredibly useful.

Perhaps distinguishing between running 'private unittest' may be 
useful as well.

dmd -unittest             // Runs all unit tests
dmd -unittest=all         // Default, runs all unit tests
dmd -unittest=private     // All private unittest
dmd -unittest=myQuicktest // Runs specific named test
dmd -unittest=myQuicktest,myLongtest // Run two or more tests

While this discussion is happening, I wonder what the most 
popular framework is currently for testing (perhaps 
https://github.com/atilaneves/unit-threaded ?).

Might be some inspiration there (others probably know better than 
I about these frameworks) if any language level changes are made 
for the longer term.
Feb 11 2023
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/10/23 4:08 PM, Dennis wrote:

 Names on unittests can be useful for documentation, a better message on 
 failure, or to selectively run a specific test. D has no built-in way to 
 name unittests, but the most popular library test runner 
 ([silly](https://code.dlang.org/packages/silly)) uses the first string 
 UDA as a name:
 
 ```D
  ("Johny")
 unittest {
      // This unittest is named Johny
 }
 ```
 
 This isn't too pretty though, and typing parentheses and quotes is 
 annoying (using a QWERTY keyboard with US-layout).
 
 I was thinking it could be improved by either allowing ` "string"` 
 attributes, similar to ` identifier`, or by allowing a string literal 
 after the `unittest` keyword:
 
 ```
 unittest "Johny" {
      // Has  ("Johny") as first UDA
 }
 ```


If you are going that route, just `unittest Johnny` would be fine.

But just to warn you, there's no place to store this in the ModuleInfo, 
it stores *one* function pointer for the whole file.

I personally am fine with the requirements to use a UDA.

And I also prefer the simple "first string" method, because one thing 
that is super-annoying is to have to depend on the unittest library for 
normal builds (I recently removed this for mysql-native).

-Steve
Feb 10 2023
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 10 February 2023 at 21:48:00 UTC, Steven Schveighoffer 
wrote:

 I personally am fine with the requirements to use a UDA.

 And I also prefer the simple "first string" method,


My proposal is purely syntactic sugar, it's exactly the same as 
adding a first string UDA.
Feb 10 2023
next sibling parent reply WebFreak001 <d.forum webfreak.org> writes:
On Friday, 10 February 2023 at 22:24:54 UTC, Dennis wrote:

 On Friday, 10 February 2023 at 21:48:00 UTC, Steven 
 Schveighoffer wrote:

 I personally am fine with the requirements to use a UDA.

 And I also prefer the simple "first string" method,


 My proposal is purely syntactic sugar, it's exactly the same as 
 adding a first string UDA.


I like this idea, and I think as  ("") has already become the 
de-facto standard across testing frameworks on DUB we can just 
make it behave like that and everyone will be happy without 
breaking changes + it's all quite an easy change for everyone.

Maybe it's a good idea to do some critical thinking about this, 
but let's not overdo it, I think this solution is quite 
practical. (though I haven't have critically thought about this 
yet very much)

Q: what does `unittest foo {}` do? try to read variable foo? I 
think if we support that a syntax like `unittest(foo) {}` would 
be more consistent, but maybe we should just make it be 
identifiers that are output as-is into a string. (it would look 
like function definitions and we could add duplication checking 
into the compiler)
Feb 10 2023
next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 11 February 2023 at 00:03:50 UTC, WebFreak001 wrote:

 I like this idea, and I think as  ("") has already become the 
 de-facto standard across testing frameworks on DUB we can just 
 make it behave like that and everyone will be happy without 
 breaking changes + it's all quite an easy change for everyone.

 Maybe it's a good idea to do some critical thinking about this, 
 but let's not overdo it, I think this solution is quite 
 practical. (though I haven't have critically thought about this 
 yet very much)


This is kinda constraining, to a single form. If changes are to 
be done to language, then perhaps it would be best to consider 
also parameterized tests, interpolated test names, or test names 
that are not strings at all but something convertible to string.
Feb 11 2023
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 11 February 2023 at 00:03:50 UTC, WebFreak001 wrote:

 On Friday, 10 February 2023 at 22:24:54 UTC, Dennis wrote:

 On Friday, 10 February 2023 at 21:48:00 UTC, Steven 
 Schveighoffer wrote:

 I personally am fine with the requirements to use a UDA.

 And I also prefer the simple "first string" method,


 My proposal is purely syntactic sugar, it's exactly the same 
 as adding a first string UDA.


 I like this idea, and I think as  ("") has already become the 
 de-facto standard across testing frameworks on DUB we can just 
 make it behave like that and everyone will be happy without 
 breaking changes + it's all quite an easy change for everyone.


The reason I used a string UDA initially (and, probably why silly 
does the same thing) is to avoid having to import a symbol to use 
it there. It's the simplest thing that will work and not 
"corrupt" production code.

Don't get me started on version(unittest).
Feb 13 2023
parent reply Bogdan <contact szabobogdan.com> writes:
On Monday, 13 February 2023 at 12:05:05 UTC, Atila Neves wrote:

 On Saturday, 11 February 2023 at 00:03:50 UTC, WebFreak001 
 wrote:

 On Friday, 10 February 2023 at 22:24:54 UTC, Dennis wrote:

 On Friday, 10 February 2023 at 21:48:00 UTC, Steven 
 Schveighoffer wrote:

 I personally am fine with the requirements to use a UDA.

 And I also prefer the simple "first string" method,


 My proposal is purely syntactic sugar, it's exactly the same 
 as adding a first string UDA.


 I like this idea, and I think as  ("") has already become the 
 de-facto standard across testing frameworks on DUB we can just 
 make it behave like that and everyone will be happy without 
 breaking changes + it's all quite an easy change for everyone.


 The reason I used a string UDA initially (and, probably why 
 silly does the same thing) is to avoid having to import a 
 symbol to use it there. It's the simplest thing that will work 
 and not "corrupt" production code.

 Don't get me started on version(unittest).



I think using  ("") is some kind of a hack because if you use a 
documentation generator, you will have to also add a `///` 
comment to have a nice description of the example. But since 
there is already a way to explain what a unit test is doing, and 
I am referring to the `///` comment, why don't you just use that 
comment?

This is what I implemented in `trial`, and it works great. 
Currently, the code is parsed using `libdparse`, but I must admit 
that it would be great if there would be a trait that gives you 
the documentation comment of a symbol. I know there were 
discussions about this a few years ago, and it was decided not to 
have such a compiler feature.
Feb 16 2023
parent Atila Neves <atila.neves gmail.com> writes:
On Thursday, 16 February 2023 at 08:29:44 UTC, Bogdan wrote:

 On Monday, 13 February 2023 at 12:05:05 UTC, Atila Neves wrote:

 On Saturday, 11 February 2023 at 00:03:50 UTC, WebFreak001 
 wrote:

 On Friday, 10 February 2023 at 22:24:54 UTC, Dennis wrote:

 On Friday, 10 February 2023 at 21:48:00 UTC, Steven 
 Schveighoffer wrote:

 I personally am fine with the requirements to use a UDA.

 And I also prefer the simple "first string" method,


 My proposal is purely syntactic sugar, it's exactly the same 
 as adding a first string UDA.


 I like this idea, and I think as  ("") has already become the 
 de-facto standard across testing frameworks on DUB we can 
 just make it behave like that and everyone will be happy 
 without breaking changes + it's all quite an easy change for 
 everyone.


 The reason I used a string UDA initially (and, probably why 
 silly does the same thing) is to avoid having to import a 
 symbol to use it there. It's the simplest thing that will work 
 and not "corrupt" production code.

 Don't get me started on version(unittest).



 I think using  ("") is some kind of a hack


People didn't generally attach string UDAs to tests before, so it 
doesn't intrude and "just works".


 because if you use a documentation generator, you will have to 
 also add a `///` comment to have a nice description of the 
 example.


IMHO that should go on the function, not the test.


 But since there is already a way to explain what a unit test is 
 doing, and I am referring to the `///` comment, why don't you 
 just use that comment?


For me at least, because it'd be incredibly annoying to select 
that test to run in the command line. I think that if you need a 
comment to explain what the test is doing, then you should 
rewrite the test.
Feb 16 2023
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/10/23 5:24 PM, Dennis wrote:

 On Friday, 10 February 2023 at 21:48:00 UTC, Steven Schveighoffer wrote:

 I personally am fine with the requirements to use a UDA.

 And I also prefer the simple "first string" method,


 
 My proposal is purely syntactic sugar, it's exactly the same as adding a 
 first string UDA.


In that case, I'm not in favor of it. I don't think the first form is so 
much worse that it needs adjusting:

```d
// existing
 ("foo") unittest {
    ...
}

// proposed
unittest "foo" {
    ...
}
```

-Steve
Feb 10 2023
prev sibling parent reply IGotD- <nise nise.com> writes:
On Friday, 10 February 2023 at 21:48:00 UTC, Steven Schveighoffer 
wrote:

 If you are going that route, just `unittest Johnny` would be 
 fine.


I'm actually for that it should be unittest "Johnny" instead of 
without the quotations. The reason is that we want something that 
describes the test much as possible and that will often include 
spaces.

For example:

unittest "Johnny download test fail to find server"

instead of

unittest Johnny download test fail to find server

which certainly will confuse the parser.

You can overcome this by having snake case but that it kind of 
ugly

unittest Johnny_download_test_fail_to_find_server


With quotation marks we will have more freedom how to name the 
tests.
Feb 16 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/16/23 5:40 AM, IGotD- wrote:

 On Friday, 10 February 2023 at 21:48:00 UTC, Steven Schveighoffer wrote:

 If you are going that route, just `unittest Johnny` would be fine.


 
 I'm actually for that it should be unittest "Johnny" instead of without 
 the quotations. The reason is that we want something that describes the 
 test much as possible and that will often include spaces.
 
 For example:
 
 unittest "Johnny download test fail to find server"
 
 instead of
 
 unittest Johnny download test fail to find server
 
 which certainly will confuse the parser.
 
 You can overcome this by having snake case but that it kind of ugly
 
 unittest Johnny_download_test_fail_to_find_server
 
 
 With quotation marks we will have more freedom how to name the tests.


I actually prefer without the quotes. Why? Because it matches what we 
currently do:

struct foo
class bar
unittest baz

In addition, if you make it a string, since D has CTFE, this is going to 
get confusing:

sneaky.d:
enum Johnny = "Freddy";

othermodule.d:

unittest Johnny {
}

"unittest Freddy failed"

user: huh?

So you have to come up with a name that has no spaces. Are you a 
programmer or not?

I would couch this by saying, I don't really think we need anything 
here. The  ("Johnny") is good enough for me. In fact, I'm fine without 
labeling unittests at all, and just looking at the file/line of the 
failed tests.

-Steve
Feb 16 2023
next sibling parent reply bachmeier <no spam.net> writes:
On Thursday, 16 February 2023 at 15:36:25 UTC, Steven 
Schveighoffer wrote:


 I would couch this by saying, I don't really think we need 
 anything here. The  ("Johnny") is good enough for me. In fact, 
 I'm fine without labeling unittests at all, and just looking at 
 the file/line of the failed tests.


My opinion is that naming a unit test is the wrong approach. If 
the condition for an `enforce` statement fails, it returns an 
informative message, not a name attached to the `enforce` 
statement. Along those lines, I would prefer this:

```
unittest {
   onFailure("Comparison of transformations failed");
   ...
}
```

Naming unit tests would open up other issues. Would you allow 
recycling of names? There's no reason you couldn't have variable 
foo and unittest foo, but that would be confusing to someone 
learning the language. Would you have to use unique unittest 
names if you import 30 modules written by someone else? This 
could probably be resolved without great difficulty at the level 
of language design, but it's all added complexity that you 
shouldn't deal with when learning a language.
Feb 16 2023
next sibling parent Ben Jones <fake fake.fake> writes:
On Thursday, 16 February 2023 at 17:26:20 UTC, bachmeier wrote:

 On Thursday, 16 February 2023 at 15:36:25 UTC, Steven 
 Schveighoffer wrote:


I don't think I saw this in the thread yet:  seems like a pretty 
simple improvement to the status quo: 
https://github.com/dlang/dmd/pull/14881
Feb 16 2023
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 16 February 2023 at 17:26:20 UTC, bachmeier wrote:

 My opinion is that naming a unit test is the wrong approach. If 
 the condition for an `enforce` statement fails, it returns an 
 informative message, not a name attached to the `enforce` 
 statement. Along those lines, I would prefer this:

 ```
 unittest {
   onFailure("Comparison of transformations failed");
   ...
 }
 ```


```
unittest
{
     import std.stdio;

     scope(failure) writeln("Test failed");
     assert(1 + 1 == 3);
}
```
Feb 16 2023
parent bachmeier <no spam.net> writes:
On Thursday, 16 February 2023 at 23:21:11 UTC, Paul Backus wrote:

 On Thursday, 16 February 2023 at 17:26:20 UTC, bachmeier wrote:

 My opinion is that naming a unit test is the wrong approach. 
 If the condition for an `enforce` statement fails, it returns 
 an informative message, not a name attached to the `enforce` 
 statement. Along those lines, I would prefer this:

 ```
 unittest {
   onFailure("Comparison of transformations failed");
   ...
 }
 ```


 ```
 unittest
 {
     import std.stdio;

     scope(failure) writeln("Test failed");
     assert(1 + 1 == 3);
 }
 ```


That works, but it's verbose, and if you run this program

```
import std.stdio;

unittest {
   assert(3 == 2);
   scope(failure) { writeln("running failure code"); }
}

void main() {}
```

The output is

```
[unittest] Assertion failure
1/1 modules FAILED unittests
```

so it's not exactly the same as an `onFailure` that could go at 
the top or bottom. You can also use version statements if you 
want to selectively run unit tests, so the whole discussion is 
really about convenience.
Feb 16 2023
prev sibling parent reply ProtectAndHide <ProtectAndHide gmail.com> writes:
On Thursday, 16 February 2023 at 17:26:20 UTC, bachmeier wrote:

 My opinion is that naming a unit test is the wrong approach. If 
 the condition for an `enforce` statement fails, it returns an 
 informative message, not a name attached to the `enforce` 
 statement. Along those lines, I would prefer this:

 ```
 unittest {
   onFailure("Comparison of transformations failed");
   ...
 }
 ```

 Naming unit tests would open up other issues. Would you allow 
 recycling of names? There's no reason you couldn't have 
 variable foo and unittest foo, but that would be confusing to 
 someone learning the language. Would you have to use unique 
 unittest names if you import 30 modules written by someone 
 else? This could probably be resolved without great difficulty 
 at the level of language design, but it's all added complexity 
 that you shouldn't deal with when learning a language.


If one wants to identify the purpose of a particular unittest, 
then naming it makes sense.

Searching for some attribute within the unittest itself (e.g 
onFailure("...")) is not the correct approach to solving this 
particular problem.

Imagine your suggested approach as being the approach for a type, 
such as:

class()
{
   classID := "This class is for the purposes of creating a car 
object";
}

No. I prefer to name my class.

I'd prefer to name my unittest also.
Feb 16 2023
parent ProtectAndHide <ProtectAndHide gmail.com> writes:
On Thursday, 16 February 2023 at 23:33:51 UTC, ProtectAndHide 
wrote:

 If one wants to identify the purpose of a particular unittest, 
 then naming it makes sense.

 Searching for some attribute within the unittest itself (e.g 
 onFailure("...")) is not the correct approach to solving this 
 particular problem.

 Imagine your suggested approach as being the approach for a 
 type, such as:

 class()
 {
   classID := "This class is for the purposes of creating a car 
 object";
 }

 No. I prefer to name my class.

 I'd prefer to name my unittest also.


And the naming convention for a unittest should be just as 
straight forward.

ie. the same as for a class/variable.

There is no need to allow spaces (for example). Of you need a 
sentence to describe your unittest, that is more about 
documentation, than naming something.

unittest testThisClass
{

}

unittest testThatClass
{

}
Feb 16 2023
prev sibling next sibling parent reply Guillaume Piolat <first.last spam.org> writes:
On Thursday, 16 February 2023 at 15:36:25 UTC, Steven 
Schveighoffer wrote:

 I actually prefer without the quotes. Why? Because it matches 
 what we currently do:

 struct foo
 class bar
 unittest baz


OTOH you give name to structs and classes _because_ you need the 
names elsewere. What is this elsewhere that needs unittests 
identifiers? I'm still not getting it.
Feb 16 2023
next sibling parent Guillaume Piolat <first.last spam.org> writes:
On Thursday, 16 February 2023 at 23:16:14 UTC, Guillaume Piolat 
wrote:

 OTOH you give name to structs and classes _because_ you need 
 the names elsewere. What is this elsewhere that needs unittests 
 identifiers? I'm still not getting it.


Else you go... nameless inline classes, or nameless structs (in C)
Feb 16 2023
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/16/23 6:16 PM, Guillaume Piolat wrote:

 On Thursday, 16 February 2023 at 15:36:25 UTC, Steven Schveighoffer wrote:

 I actually prefer without the quotes. Why? Because it matches what we 
 currently do:

 struct foo
 class bar
 unittest baz


 
 OTOH you give name to structs and classes _because_ you need the names 
 elsewere. What is this elsewhere that needs unittests identifiers? I'm 
 still not getting it.


The only legitimate reason I can see is to run specific unittests by 
naming them at runtime. Or for the stack trace to look pretty.

But I'm ok with running all the unittests in a module. And you can do 
that without any language changes today.

Like I said, I don't think anything needs to change. But if such a 
feature absolutely has to be added, I'd vote for a symbol and not a string.

-Steve
Feb 16 2023
prev sibling parent ProtectAndHide <ProtectAndHide gmail.com> writes:
On Thursday, 16 February 2023 at 15:36:25 UTC, Steven 
Schveighoffer wrote:

 I actually prefer without the quotes. Why? Because it matches 
 what we currently do:

 struct foo
 class bar
 unittest baz

 In addition, if you make it a string, since D has CTFE, this is 
 going to get confusing:

 sneaky.d:
 enum Johnny = "Freddy";

 othermodule.d:

 unittest Johnny {
 }

 "unittest Freddy failed"

 user: huh?

 So you have to come up with a name that has no spaces. Are you 
 a programmer or not?

 I would couch this by saying, I don't really think we need 
 anything here. The  ("Johnny") is good enough for me. In fact, 
 I'm fine without labeling unittests at all, and just looking at 
 the file/line of the failed tests.

 -Steve


The litmus test for how to go about naming unittests should be 
what you would do if you didn't know how to do it.

int myInt
struct myStruct
class myClass
unittest myUnitTest (most people would automatically default to 
this, is my guess)

unittest  myUnitTest (on what basis would people automatically 
think that this is how to do it?

unittest "my unit test" (again, on what basis would people 
automatically think that this is how to do it?

It seems like a simple proposition with a simple for the 
programmer to do it.

Now make the language work for the programmer please, and not the 
other way around.
Feb 16 2023
prev sibling next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
I'm ok with ``unittest "name" {}`` syntax. It should also support 
identifiers too.

We'll need to think about how to handle this at the ModuleInfo level though.
Feb 10 2023
prev sibling next sibling parent reply max haughton <maxhaton gmail.com> writes:
On Friday, 10 February 2023 at 21:08:38 UTC, Dennis wrote:

 Names on unittests can be useful for documentation, a better 
 message on failure, or to selectively run a specific test. D 
 has no built-in way to name unittests, but the most popular 
 library test runner 
 ([silly](https://code.dlang.org/packages/silly)) uses the first 
 string UDA as a name:

 [...]


I don't think it's worth breaking tooling / grammars over.

That being said I am in favour of the language dictating that a 
string in a UDA on a unittest has special meaning (i.e. is a name 
or description)
Feb 11 2023
parent reply Bradley Chatha <sealabjaster gmail.com> writes:
On Saturday, 11 February 2023 at 20:05:38 UTC, max haughton wrote:

 That being said I am in favour of the language dictating that a 
 string in a UDA on a unittest has special meaning (i.e. is a 
 name or description)


Wouldn't that only really be useful if the compiler had better 
support for named unittests, e.g. `-test-filer=yada`, or 
`-test-output=pretty (similar to silly or something like that)`?

Or am I missing the point entirely here :D
Feb 11 2023
next sibling parent reply Andrew <andrewlalisofficial gmail.com> writes:
On Saturday, 11 February 2023 at 21:10:15 UTC, Bradley Chatha 
wrote:

 On Saturday, 11 February 2023 at 20:05:38 UTC, max haughton 
 wrote:

 That being said I am in favour of the language dictating that 
 a string in a UDA on a unittest has special meaning (i.e. is a 
 name or description)


 Wouldn't that only really be useful if the compiler had better 
 support for named unittests, e.g. `-test-filer=yada`, or 
 `-test-output=pretty (similar to silly or something like that)`?

 Or am I missing the point entirely here :D


In my opinion, the compiler doesn't really need to be responsible 
for all the bells and whistles in a modern testing framework; it 
just compiles code or it complains when it can't.

The issue is that we as a community haven't rallied around a 
single best testing framework which expands upon the basic 
`unittest` functionality. Atila's "unit-threaded" might be the 
best contender so far, but it's not unanimous like pytest for 
python or junit for java.
Feb 11 2023
parent IGotD- <nise nise.com> writes:
On Saturday, 11 February 2023 at 21:25:50 UTC, Andrew wrote:

 In my opinion, the compiler doesn't really need to be 
 responsible for all the bells and whistles in a modern testing 
 framework; it just compiles code or it complains when it can't.

 The issue is that we as a community haven't rallied around a 
 single best testing framework which expands upon the basic 
 `unittest` functionality. Atila's "unit-threaded" might be the 
 best contender so far, but it's not unanimous like pytest for 
 python or junit for java.


One of the best things with D is the unit test support and how 
easy it is to add it so I think it should definitely be in the 
compiler or at least it should appear to be integrated.

Also what is opinionated is if the unit tests should be in the 
same file or separate. I'm of the opinion that it should be in 
the same file as then you are encouraged to create unit tests as 
well changes to the code can also easily be reflected in unit 
test. This is a matter of opinion and also if you want to use a 
third party unit test framework, then you are free to do so.

I welcome named unit tests and it took a long time for it to even 
be considered.
Feb 12 2023
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/11/23 4:10 PM, Bradley Chatha wrote:

 On Saturday, 11 February 2023 at 20:05:38 UTC, max haughton wrote:

 That being said I am in favour of the language dictating that a string 
 in a UDA on a unittest has special meaning (i.e. is a name or 
 description)


 
 Wouldn't that only really be useful if the compiler had better support 
 for named unittests, e.g. `-test-filer=yada`, or `-test-output=pretty 
 (similar to silly or something like that)`?
 
 Or am I missing the point entirely here :D


All this is possible. Let me talk a little bit about how this all works.

The compiler takes all unittest blocks compiled in a given module, and 
creates individual functions from all of them.

Then, the compiler creates *one* master unittest function for each 
module, and has that function just call all the individual functions. 
This is the function pointer that is stored in the ModuleInfo.

That's all that is done from the compiler side. None of this is defined 
explicitly in the spec or language, so we have free reign to do this 
however we want.

Now, on the library side, all it does is search all moduleinfos and find 
any that have unittests. Then it runs those unittests using a function 
in druntime (or you can register a custom runner if you want). All that 
library function has accessible in the moduleinfo is the function 
pointer to run all unittests for that module. The unittest runner 
returns a structure that indicates how many tests were run (modules), 
and how many passed. We could easily modify that structure to say how 
many individual unittests were run (we can do whatever we want in the 
test running function, it has no explicit specification), and have the 
test runner collect which tests failed based either on file/line number 
by default, or use a UDA string if it's provided.

We could easily modify the moduleinfo to contain an array of unittest 
functions, and run that instead. Each function might even be a tuple of 
unittest name and function pointer. Then we have the capability to 
specify which unittests are run.

You can already do this at the module level if desired without any 
changes (e.g. --runUnittestModule=foo.bar can be acted on properly).

All of it is possible. We are very much unrestricted on how unittests 
actually run, since the only requirement is that the library run them, 
and report what happened.

The only thing about this is, it's all doable *already* with the test 
frameworks in code.dlang.org. It takes a bit more effort, but it's all 
available there. You aren't using the ModuleInfo system, but rather 
compile-time introspection to find unittests. But so what?

So really, it's a matter of how much we want to complicate the existing 
system to achieve something that is already somewhat achievable. That is 
the question we have to answer.

-Steve
Feb 11 2023
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
SDC supports the following syntax:

```d
unittest testname {
   // Test things here....
}
```
Feb 11 2023
parent Max Samukha <maxsamukha gmail.com> writes:
On Sunday, 12 February 2023 at 00:15:22 UTC, deadalnix wrote:

 SDC supports the following syntax:

 ```d
 unittest testname {
   // Test things here....
 }
 ```


This.
Feb 11 2023
prev sibling next sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Friday, 10 February 2023 at 21:08:38 UTC, Dennis wrote:

 Names on unittests can be useful for documentation, a better 
 message on failure, or to selectively run a specific test.


I'd prefer `unittest name {}`. For documentation, there are doc 
comments. For error messages, we could have a standard 
 description attribute not specific to unittests (or treat an 
unqualified UDA string as such, or have a special doc section).
Feb 11 2023
prev sibling next sibling parent reply Guillaume Piolat <first.last spam.org> writes:
On Friday, 10 February 2023 at 21:08:38 UTC, Dennis wrote:

 Question to the community: do you name tests, and do you like 
 the idea?


Personally I'm not sure if I could find an interesting name for 
tests, but I can certainly express their intent in a string 
literal like in Zig.
Feb 16 2023
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 17/02/2023 2:20 AM, Guillaume Piolat wrote:

 Personally I'm not sure if I could find an interesting name for tests, 
 but I can certainly express their intent in a string literal like in Zig.


Identifier support should probably be only syntax level deep, and be 
treated as if it was a string in the implementation.

So for rapid prototyping and debugging it may be desirable to just use 
an identifier rather than a string if you need it.

Thoughts?
Feb 16 2023
parent reply Guillaume Piolat <first.last spam.org> writes:
On Thursday, 16 February 2023 at 13:29:48 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 So for rapid prototyping and debugging it may be desirable to 
 just use an identifier rather than a string if you need it.

 Thoughts?


Can certainly live with either (this is a decision for core 
people) but what is the purpose of an identifier there?
Feb 16 2023
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 17/02/2023 2:58 AM, Guillaume Piolat wrote:

 On Thursday, 16 February 2023 at 13:29:48 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:

 So for rapid prototyping and debugging it may be desirable to just use 
 an identifier rather than a string if you need it.

 Thoughts?


 
 Can certainly live with either (this is a decision for core people) but 
 what is the purpose of an identifier there?


It wouldn't be either or, you'd do both. After all you'd just convert 
the identifier to a string during parsing no extra semantics.
Feb 16 2023
prev sibling next sibling parent reply apz28 <home home.com> writes:
On Friday, 10 February 2023 at 21:08:38 UTC, Dennis wrote:

 Names on unittests can be useful for documentation, a better 
 message on failure, or to selectively run a specific test. D 
 has no built-in way to name unittests, but the most popular 
 library test runner 
 ([silly](https://code.dlang.org/packages/silly)) uses the first 
 string UDA as a name:

 ```D
  ("Johny")
 unittest {
 	// This unittest is named Johny
 }
 ```

 This isn't too pretty though, and typing parentheses and quotes 
 is annoying (using a QWERTY keyboard with US-layout).

 I was thinking it could be improved by either allowing 
 ` "string"` attributes, similar to ` identifier`, or by 
 allowing a string literal after the `unittest` keyword:

 ```
 unittest "Johny" {
     // Has  ("Johny") as first UDA
 }
 ```


module sample;

class X { void foo() {} }

Refer below form -> it can be expanded with addition attribute(s) 
in future with consistent structure
unittest(name="good", document="class X")
{}
unittest(name="good", document="class X.foo")
{}

name = name of unittest
document = generate document for class X
            generate document for function class X.foo


for running test
unitttest=name:"good"
unittest=module:"sample"

name = only run unittest with matching name "good"
module = only run unittest(s) for module "sample"
Feb 16 2023
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 17/02/2023 3:22 AM, apz28 wrote:

 document = generate document for class X
             generate document for function class X.foo


We could easily generate such strings in this example.

We also have access to core.attributes to make the compiler understand 
other information, if its desired.

There are solutions here that don't require making this more 
complicated. So we'd need a concrete attributes above name of easily 
extracted from the AST.
Feb 16 2023
prev sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 10 February 2023 at 21:08:38 UTC, Dennis wrote:

 I was thinking it could be improved by either allowing 
 ` "string"` attributes, similar to ` identifier`, or by 
 allowing a string literal after the `unittest` keyword:

 ```
 unittest "Johny" {
     // Has  ("Johny") as first UDA
 }
 ```

 This would provide a more ergonomic, standardized way of naming 
 tests, and it's a very simple parser change (there's currently 
 nothing allowed between `unittest {`).

 In many other languages, unittests are regular functions with 
 names, but 
 [Zig-test](https://ziglang.org/documentation/master/#Zig-Test) 
 has this syntax for comparison:

 ```Zig
 test "Johny" {
 }
 ```

 Question to the community: do you name tests, and do you like 
 the idea?


Personal vote: We write unittest names with spaces in, so 
`unittest "the test" { }` would be very nice.

The place that the string gets used is when invoking the test 
directly: `build/TestRunner 'foo.bar.the test'`.
Feb 19 2023