www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - How can one reliably run unittests

reply deadalnix <deadalnix gmail.com> writes:
D's unittest block is a great idea. However, I find that in 
practice, I fail to make good use of them and have been 
neglecting them. Sometime by using an external test suite 
instead, sometime neglecting testing altogether. How come?

Well, the main reason is that, as far as I can tell, it is near 
damn impossible to run them at scale in any sensible way. Let me 
explain.

Ideally, one would want a module to be a unit. Declaration, 
implementation, but also tests are part of the unit. During the 
dev cycle, it is desired to be able to runt he test on modules to 
verify that everything works as advertised. While doing so is 
easy on a pet project,a s the projects grows, contains several 
libraries and executable, doing this become very challenging.

It is not really possible to build a monster executable that 
contains everything. First, this would kill build times, but, 
with several main around it won't link. So let's split into 
component.

Some of these component are libraries, some are executable. 
Adding a main for libraries is required, or it won't link, but 
adding one to executable is going to cause a link error. This in 
itself is a major main in the ass, because that means there is no 
one consistent way to unittest a module without knowing if that 
module has a main. In addition, everything needs to be built 
twice now, which is really undesirable.

Maybe one could rdmd each modules to runt he tests? That seems 
like the best approach to me, considering one doesn't want to 
actually produce an artifact for the tests, simply run them. this 
also has the main/no main problem, but in addition, it often 
fails with incomprehensible linker errors. For some reason, rdmd 
is not able to include all dependencies consistently, but, even 
better, it doesn't take the same standard flags as other linkers 
do, so it is not possible to reuse existing build infrastructure 
to feed all the correct flags to rdmd.

This may seems like it wouldn't be that big of a deal if you 
manage all your flags by yourself, but very quickly turns into a 
nightmare once you have to use 3rd party libraries that ship with 
their own set of flags.

At this point, I would just wish that one could rdmd --unittest 
modulename.d and just pass it a couple of -I, -L and -l flags and 
have all of it work. I have no idea how to achieve that.

This kind of orthogonality is important as a project scale. One 
cannot just doctor all the unitests call to all the modules. You 
need to be able to tell your build system something akin to "hey, 
every time you encounter a D module, please implicitly add this 
unitest target. Thanks." This cannot be made to work if running 
unit tests is done in a way that depends on the content of the 
module.

I don't think my case is isolated, every single substantial D 
project I encountered has some kind of test suite or something 
similar instead of relying on the unitests within modules. This 
is frankly a failure, and it is 100% due to poor UX as the 
language feature itself is great.

If someone has some bandwidth, it is IMO a high leverage task 
that solve real problem for real people and 90% of the work is 
already there. The feature is there, the runtime support &al is 
there. just provide a consistent UX so its use can be integrated 
properly in larger systems.

Thanks in advance.
Aug 24 2021
next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 24 August 2021 at 12:21:41 UTC, deadalnix wrote:

 D's unittest block is a great idea. However, I find that in 
 practice, I fail to make good use of them and have been 
 neglecting them. Sometime by using an external test suite 
 instead, sometime neglecting testing altogether. How come?

 Well, the main reason is that, as far as I can tell, it is near 
 damn impossible to run them at scale in any sensible way. Let 
 me explain.

 Ideally, one would want a module to be a unit. Declaration, 
 implementation, but also tests are part of the unit. During the 
 dev cycle, it is desired to be able to runt he test on modules 
 to verify that everything works as advertised. While doing so 
 is easy on a pet project,a s the projects grows, contains 
 several libraries and executable, doing this become very 
 challenging.

 It is not really possible to build a monster executable that 
 contains everything. First, this would kill build times, but, 
 with several main around it won't link. So let's split into 
 component.

 Some of these component are libraries, some are executable. 
 Adding a main for libraries is required, or it won't link, but 
 adding one to executable is going to cause a link error. This 
 in itself is a major main in the ass, because that means there 
 is no one consistent way to unittest a module without knowing 
 if that module has a main. In addition, everything needs to be 
 built twice now, which is really undesirable.

 Maybe one could rdmd each modules to runt he tests? That seems 
 like the best approach to me, considering one doesn't want to 
 actually produce an artifact for the tests, simply run them. 
 this also has the main/no main problem, but in addition, it 
 often fails with incomprehensible linker errors. For some 
 reason, rdmd is not able to include all dependencies 
 consistently, but, even better, it doesn't take the same 
 standard flags as other linkers do, so it is not possible to 
 reuse existing build infrastructure to feed all the correct 
 flags to rdmd.

 This may seems like it wouldn't be that big of a deal if you 
 manage all your flags by yourself, but very quickly turns into 
 a nightmare once you have to use 3rd party libraries that ship 
 with their own set of flags.

 At this point, I would just wish that one could rdmd --unittest 
 modulename.d and just pass it a couple of -I, -L and -l flags 
 and have all of it work. I have no idea how to achieve that.

 This kind of orthogonality is important as a project scale. One 
 cannot just doctor all the unitests call to all the modules. 
 You need to be able to tell your build system something akin to 
 "hey, every time you encounter a D module, please implicitly 
 add this unitest target. Thanks." This cannot be made to work 
 if running unit tests is done in a way that depends on the 
 content of the module.

 I don't think my case is isolated, every single substantial D 
 project I encountered has some kind of test suite or something 
 similar instead of relying on the unitests within modules. This 
 is frankly a failure, and it is 100% due to poor UX as the 
 language feature itself is great.

 If someone has some bandwidth, it is IMO a high leverage task 
 that solve real problem for real people and 90% of the work is 
 already there. The feature is there, the runtime support &al is 
 there. just provide a consistent UX so its use can be 
 integrated properly in larger systems.

 Thanks in advance.


Interesting.

My ideal scenario wouldn't involve linkers at all - I don't want 
to spend time waiting for my code to link, especially since, the 
way I write code means that unit tests never need optimising or 
native code generated. I'd go as far as to say that if one's unit 
tests *do* need any of that, they need to rethink their 
mini-integra... err, unit tests. But I digress.

The way I'd like it to work personally would be to CTFE unit 
tests in a module and have them run whenever I save the file. No 
linker, no main function, just that.

Back to the issue at hand. The problem with wanting to pass flags 
to the rdmd is that unless the project is trivial it'll have dub 
dependencies, so doing that manually is a no-no. This is the same 
thing that happens all the time in compile-as-you-type plugins, 
and why I maintain two separate emacs packages to make that done 
automatically (one for CMake/C++ and one for dub/D). Which means 
asking dub. Which means a build system.

Which means you just gave me an idea for an awesome reggae rule 
to generate a library with all of the dependencies of a module, a 
main function to run the tests, and then voilà. It'd have to do 
this for each module, and the first time said all-dependencies 
library is built it'd take time, but after that it'd work.

You'd still have to wait for the linker though, and if for some 
reason you're not using lld (or worse, you're on Windows), the 
wait will be looooong.
Aug 24 2021
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 24 August 2021 at 12:39:47 UTC, Atila Neves wrote:

 My ideal scenario wouldn't involve linkers at all - I don't 
 want to spend time waiting for my code to link, especially 
 since, the way I write code means that unit tests never need 
 optimising or native code generated. I'd go as far as to say 
 that if one's unit tests *do* need any of that, they need to 
 rethink their mini-integra... err, unit tests. But I digress.


This is not doable in practice ATM. You use an exception from 
another module? you are screwed already, you need to link that 
other module in. This is a UX disaster, though.


 The way I'd like it to work personally would be to CTFE unit 
 tests in a module and have them run whenever I save the file. 
 No linker, no main function, just that.


Yes, or if there is a linker behind, I don't want to know about 
it. Slow link time can be a problem, but making it work at all, 
even with slow link time, would be a tremendous step forward.


 Back to the issue at hand. The problem with wanting to pass 
 flags to the rdmd is that unless the project is trivial it'll 
 have dub dependencies, so doing that manually is a no-no. This 
 is the same thing that happens all the time in 
 compile-as-you-type plugins, and why I maintain two separate 
 emacs packages to make that done automatically (one for 
 CMake/C++ and one for dub/D). Which means asking dub. Which 
 means a build system.


yes, dub makes this whole problem exponentially worse.


 Which means you just gave me an idea for an awesome reggae rule 
 to generate a library with all of the dependencies of a module, 
 a main function to run the tests, and then voilà. It'd have to 
 do this for each module, and the first time said 
 all-dependencies library is built it'd take time, but after 
 that it'd work.


Yes. One problem you'll run into is the `main` issue, where you 
need to use different flags. It is also unfortunate that building 
system have to pay the cost to paper over bad command line UI. 
This is bound to create poor integration in existing toolchains - 
which we see is the case already.


 You'd still have to wait for the linker though, and if for some 
 reason you're not using lld (or worse, you're on Windows), the 
 wait will be looooong.


The alternative is the feature not being usable at all, so I 
don't really care. I can throw hardware at that problem.
Aug 24 2021
prev sibling next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Tuesday, 24 August 2021 at 12:39:47 UTC, Atila Neves wrote:

 Interesting.

 My ideal scenario wouldn't involve linkers at all - I don't 
 want to spend time waiting for my code to link, especially 
 since, the way I write code means that unit tests never need 
 optimising or native code generated. I'd go as far as to say 
 that if one's unit tests *do* need any of that, they need to 
 rethink their mini-integra... err, unit tests. But I digress.

 The way I'd like it to work personally would be to CTFE unit 
 tests in a module and have them run whenever I save the file. 
 No linker, no main function, just that.

 Back to the issue at hand. The problem with wanting to pass 
 flags to the rdmd is that unless the project is trivial it'll 
 have dub dependencies, so doing that manually is a no-no. This 
 is the same thing that happens all the time in 
 compile-as-you-type plugins, and why I maintain two separate 
 emacs packages to make that done automatically (one for 
 CMake/C++ and one for dub/D). Which means asking dub. Which 
 means a build system.

 Which means you just gave me an idea for an awesome reggae rule 
 to generate a library with all of the dependencies of a module, 
 a main function to run the tests, and then voilà. It'd have to 
 do this for each module, and the first time said 
 all-dependencies library is built it'd take time, but after 
 that it'd work.

 You'd still have to wait for the linker though, and if for some 
 reason you're not using lld (or worse, you're on Windows), the 
 wait will be looooong.


Imho, there should be a well known api for fetching unit tests at 
compile time and at runtime, then external or builtin unit test 
runner could use the api to properly select which unit tests to 
run. Second thing that compile time api should offer is the 
ability to force compiler ignore unit tests from modules or 
packages or annotations on unit test itself (it is also a 
nightmare to compile unit tests from a third party source library 
that takes ages to compile due to template bloat), or in general 
provide some kind of hooks into unittest config and running, 
similar to how junit does in java.

Best regards,
Alexandru.
Aug 24 2021
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/24/2021 5:39 AM, Atila Neves wrote:

 The way I'd like it to work personally would be to CTFE unit tests in a module 
 and have them run whenever I save the file. No linker, no main function, just
that.


Interestingly, a lot of the tests for ImportC are in the form of CTFE and 
_Static_assert(). It works so well, I plan on using that method wherever I can.
Aug 25 2021
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 24 August 2021 at 12:21:41 UTC, deadalnix wrote:

 Some of these component are libraries, some are executable. 
 Adding a main for libraries is required, or it won't link, but 
 adding one to executable is going to cause a link error. This 
 in itself is a major main in the ass, because that means there 
 is no one consistent way to unittest a module without knowing 
 if that module has a main. In addition, everything needs to be 
 built twice now, which is really undesirable.


That particular problem has a well known workaround -- prepend 
the `main` function with `version` statements to exclude it from 
`unittest` builds:

```D
version (unittest) {} else
void main ()
{
     ...
}
```

Obviously that assumes you control all the codebases that define 
a `main`, and it doesn't solve the problem of building everything 
twice, but it should cover a lot of use-cases.
Aug 24 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 24 August 2021 at 15:54:25 UTC, Joseph Rushton 
Wakeling wrote:

 That particular problem has a well known workaround -- prepend 
 the `main` function with `version` statements to exclude it 
 from `unittest` builds:

 ```D
 version (unittest) {} else
 void main ()
 {
     ...
 }
 ```

 Obviously that assumes you control all the codebases that 
 define a `main`, and it doesn't solve the problem of building 
 everything twice, but it should cover a lot of use-cases.


True, and I'm sure that with an endless bag of trick, this could 
be made to work. this is missing the point, though.

The compiler KNOWS if there is a main or not. The compiler KNOWS 
if I want unittests or not. Yet it's going to do retarded stuff 
unless one uses a bunch of workarounds.

It's like we have this car, the engine works, the break are in 
order and so on, except the steering wheel is at the back seat 
and everybody tells you that you can still drive by passing your 
head through the window and look forward this way.

This just doesn't make sense.
Aug 24 2021
parent reply Mathias LANG <geod24 gmail.com> writes:
On Tuesday, 24 August 2021 at 17:45:20 UTC, deadalnix wrote:

 The compiler KNOWS if there is a main or not. The compiler 
 KNOWS if I want unittests or not. Yet it's going to do retarded 
 stuff unless one uses a bunch of workarounds.


This. I think the original implementers had a different idea of 
how things would be used. For a long time, unittests would run 
first, then the main would be called. We all know how annoying 
that was, (most) people do not want this, they wanted two 
different modes, clearly separated. I say most, because there are 
still a few hold out I think.

Personally, I'd like my unittest binary to generate something 
that has a much better UX than what's currently in druntime. You 
know, timing statistics, colors, ability to run a single test, 
the whole thing.

I'm aware that there are frameworks out there, but it needs to be 
built-in.
Aug 25 2021
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 25 August 2021 at 07:50:50 UTC, Mathias LANG wrote:

 This. I think the original implementers had a different idea of 
 how things would be used. For a long time, unittests would run 
 first, then the main would be called. We all know how annoying 
 that was, (most) people do not want this, they wanted two 
 different modes, clearly separated. I say most, because there 
 are still a few hold out I think.


I think there's also a complexity that the compiler cannot know 
which main to ignore (e.g. application main) versus which main to 
keep (e.g. a custom-written main inserted for running unittests, 
as some frameworks do).

That's surely readily solvable but it suggests that there should 
be a ready way to distinguish between entry points for running an 
app versus entry points for running tests (so that test-suite 
implementations can override the latter and have it Just Work).

I fully agree with you and Amaury that this _should_ be solved 
and that the user shouldn't have to bother with workarounds like 
the one that I shared above.
Aug 25 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/25/21 3:50 AM, Mathias LANG wrote:

 Personally, I'd like my unittest binary to generate something that has a 
 much better UX than what's currently in druntime. You know, timing 
 statistics, colors, ability to run a single test, the whole thing.
 


You can get almost all that via registering a [unittest handler 
function](https://dlang.org/phobos/core_runtime.html#.Runtime.extend
dModuleUnitTester). 
The only thing that isn't provided is running individual tests, but that 
is a compiler issue (the compiler combines all unittests in a module 
into one callable).


 I'm aware that there are frameworks out there, but it needs to be built-in.


Disagree. I'm fine with the simple unittest experience, and avoiding 
putting all this kind of bloat into the runtime.

-Steve
Aug 25 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Wednesday, 25 August 2021 at 11:11:24 UTC, Steven 
Schveighoffer wrote:

 The only thing that isn't provided is running individual tests, 
 but that is a compiler issue (the compiler combines all 
 unittests in a module into one callable).


You can run individual tests:

```d
module tester1;

unittest { assert(true); }
unittest { assert(!!true); }
unittest { assert(1 != 1); }
unittest { assert(1 > 0); }

version (unittest) {
     bool tester() {
         import std.meta : AliasSeq;
         import std.stdio : writef, writeln;

         alias tests = AliasSeq!(__traits(getUnitTests, tester1));
         static foreach (i; 0 .. tests.length) {
             writef!"Test %d/%d ..."(i + 1, tests.length);
             try {
                 tests[i]();
                 writeln("ok");
             } catch (Throwable t) {
                 writeln("failed");
             }
         }
         return false;
     }

     shared static this() {
         import core.runtime : Runtime;

         Runtime.moduleUnitTester = &tester;
     }
}

void main() {
     assert(false); // this doesn't get run
}
```

usage:

```
$ dmd -unittest -run tester1.d
Test 1/4 ...ok
Test 2/4 ...ok
Test 3/4 ...failed
Test 4/4 ...ok
```

I found this out in 
https://forum.dlang.org/post/bukhjtbxouadyunqwdih forum.dlang.org 
, which has a tester that calls tests differently depending on 
their having a UDA
Aug 25 2021
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/25/21 7:46 AM, jfondren wrote:

 On Wednesday, 25 August 2021 at 11:11:24 UTC, Steven Schveighoffer wrote:

 The only thing that isn't provided is running individual tests, but 
 that is a compiler issue (the compiler combines all unittests in a 
 module into one callable).


 
 You can run individual tests:


Well, yes. You can do everything yourself (you don't need to use the 
unittest system for this BTW). What I meant to say is, if you want to 
run all unit tests linked into the program, regardless of compile-time 
introspection, you need to rely on the compiler/linker to gather those 
for you (they are all provided as part of the ModuleInfo items), the 
compiler does not store references to individual tests.

The things you *could* do are:

1. Print out messages for each module run, even if they pass
2. Colorize the output of those messages
3. Run unittests of specified modules only.


 I found this out in 
 https://forum.dlang.org/post/bukhjtbxouadyunqwdih forum.dlang.org , 
 which has a tester that calls tests differently depending on their 
 having a UDA


UDAs are also not provided to the ModuleInfo, so there isn't a way to 
affect the standard unittest framework that way.

-Steve
Aug 25 2021
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 25 August 2021 at 12:44:54 UTC, Steven 
Schveighoffer wrote:

 [snip]


Would it be possible to use an external unit testing framework to 
get all the tests when compiling with -betterC automatically 
without manually listing all the modules?
Aug 25 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/25/21 10:44 AM, jmh530 wrote:

 On Wednesday, 25 August 2021 at 12:44:54 UTC, Steven Schveighoffer wrote:

 [snip]


 
 Would it be possible to use an external unit testing framework to get 
 all the tests when compiling with -betterC automatically without 
 manually listing all the modules?
 


betterC does not create ModuleInfo. So I don't think so.

Potentially you could register tests via `pragma(crt_constructor)` 
calls. It would be messy.

Nothing would be as nice as the way the compiler currently sets up the 
moduleinfo.

-Steve
Aug 25 2021
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 25 August 2021 at 14:50:04 UTC, Steven 
Schveighoffer wrote:

 On 8/25/21 10:44 AM, jmh530 wrote:

 On Wednesday, 25 August 2021 at 12:44:54 UTC, Steven 
 Schveighoffer wrote:

 [snip]


 
 Would it be possible to use an external unit testing framework 
 to get all the tests when compiling with -betterC 
 automatically without manually listing all the modules?
 


 betterC does not create ModuleInfo. So I don't think so.

 Potentially you could register tests via 
 `pragma(crt_constructor)` calls. It would be messy.

 Nothing would be as nice as the way the compiler currently sets 
 up the moduleinfo.

 -Steve


Thanks.
Aug 25 2021
prev sibling next sibling parent reply wjoe <invalid example.com> writes:
On Wednesday, 25 August 2021 at 11:46:18 UTC, jfondren wrote:

 On Wednesday, 25 August 2021 at 11:11:24 UTC, Steven 
 Schveighoffer wrote:

 The only thing that isn't provided is running individual 
 tests, but that is a compiler issue (the compiler combines all 
 unittests in a module into one callable).


 You can run individual tests:

 ```d
 module tester1;

 unittest { assert(true); }
 unittest { assert(!!true); }
 unittest { assert(1 != 1); }
 unittest { assert(1 > 0); }

 version (unittest) {
     bool tester() {
         import std.meta : AliasSeq;
         import std.stdio : writef, writeln;

         alias tests = AliasSeq!(__traits(getUnitTests, 
 tester1));
         static foreach (i; 0 .. tests.length) {
             writef!"Test %d/%d ..."(i + 1, tests.length);
             try {
                 tests[i]();
                 writeln("ok");
             } catch (Throwable t) {
                 writeln("failed");
             }
         }
         return false;
     }

     shared static this() {
         import core.runtime : Runtime;

         Runtime.moduleUnitTester = &tester;
     }
 }

 void main() {
     assert(false); // this doesn't get run
 }
 ```

 usage:

 ```
 $ dmd -unittest -run tester1.d
 Test 1/4 ...ok
 Test 2/4 ...ok
 Test 3/4 ...failed
 Test 4/4 ...ok
 ```

 I found this out in 
 https://forum.dlang.org/post/bukhjtbxouadyunqwdih forum.dlang.org , which has
a tester that calls tests differently depending on their having a UDA


But due to the fact that all unittests are compiled into one 
callable, i.e. it's all in the same process, doesn't that mean 
that after catching a throwable that the program is in UB ? And 
isn't that the reason why the runtime unittester aborts on the 
1st failure ?
So shouldn't it be like every module has its own callable which 
can then run in separate processes which then wouldn't affect 
others if one of them failed ? Also, that way all the modules 
could be tested in parallel.
Aug 25 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 25 August 2021 at 13:23:22 UTC, wjoe wrote:

 But due to the fact that all unittests are compiled into one 
 callable, i.e. it's all in the same process, doesn't that mean 
 that after catching a throwable that the program is in UB ? And 
 isn't that the reason why the runtime unittester aborts on the 
 1st failure ?
 So shouldn't it be like every module has its own callable which 
 can then run in separate processes which then wouldn't affect 
 others if one of them failed ? Also, that way all the modules 
 could be tested in parallel.


Errors thrown by assertions in unit tests are a special case:


 Individual tests are specified in the unit test using 
 AssertExpressions. Unlike AssertExpressions used elsewhere, the 
 assert is not assumed to hold, and upon assert failure the 
 program is still in a defined state.


Source: https://dlang.org/spec/unittest.html
Aug 25 2021
next sibling parent reply wjoe <invalid example.com> writes:
On Wednesday, 25 August 2021 at 13:25:22 UTC, Paul Backus wrote:

 On Wednesday, 25 August 2021 at 13:23:22 UTC, wjoe wrote:

 [...]


 Errors thrown by assertions in unit tests are a special case:


 [...]


 Source: https://dlang.org/spec/unittest.html


That's good to know but I was specifically referring to the 
catch(Throwable) part in the above example.
Aug 25 2021
parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 25 August 2021 at 13:28:40 UTC, wjoe wrote:

 On Wednesday, 25 August 2021 at 13:25:22 UTC, Paul Backus wrote:

 On Wednesday, 25 August 2021 at 13:23:22 UTC, wjoe wrote:

 [...]


 Errors thrown by assertions in unit tests are a special case:


 [...]


 Source: https://dlang.org/spec/unittest.html


 That's good to know but I was specifically referring to the 
 catch(Throwable) part in the above example.


Good point. It should probably be catch(AssertError), although 
even that is technically too broad.
Aug 25 2021
prev sibling parent reply wjoe <invalid example.com> writes:
On Wednesday, 25 August 2021 at 13:25:22 UTC, Paul Backus wrote:

 On Wednesday, 25 August 2021 at 13:23:22 UTC, wjoe wrote:

 [...]


 Errors thrown by assertions in unit tests are a special case:


 [...]


 Source: https://dlang.org/spec/unittest.html


Also, does that hold for something like assertThrown!SomeError ?
Aug 25 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 25 August 2021 at 13:36:27 UTC, wjoe wrote:

 On Wednesday, 25 August 2021 at 13:25:22 UTC, Paul Backus wrote:

 On Wednesday, 25 August 2021 at 13:23:22 UTC, wjoe wrote:

 [...]


 Errors thrown by assertions in unit tests are a special case:


 [...]


 Source: https://dlang.org/spec/unittest.html


 Also, does that hold for something like assertThrown!SomeError ?


Officially, according to what's written in the spec? No. In 
practice? Yes. And enough people rely on custom assertions like 
assertThrown and the ones in unit-threaded [1] that this is 
unlikely to change.

[1] 
https://unit-threaded.dpldocs.info/unit_threaded.assertions.html
Aug 25 2021
parent wjoe <invalid example.com> writes:
On Wednesday, 25 August 2021 at 13:39:32 UTC, Paul Backus wrote:

 On Wednesday, 25 August 2021 at 13:36:27 UTC, wjoe wrote:

 On Wednesday, 25 August 2021 at 13:25:22 UTC, Paul Backus 
 wrote:

 On Wednesday, 25 August 2021 at 13:23:22 UTC, wjoe wrote:

 [...]


 Errors thrown by assertions in unit tests are a special case:


 [...]


 Source: https://dlang.org/spec/unittest.html


 Also, does that hold for something like assertThrown!SomeError 
 ?


 Officially, according to what's written in the spec? No. In 
 practice? Yes. And enough people rely on custom assertions like 
 assertThrown and the ones in unit-threaded [1] that this is 
 unlikely to change.

 [1] 
 https://unit-threaded.dpldocs.info/unit_threaded.assertions.html


Thanks. I wondered about that for some time.
Aug 25 2021
prev sibling parent user1234 <user1234 12.de> writes:
On Wednesday, 25 August 2021 at 11:46:18 UTC, jfondren wrote:


 [...]
         alias tests = AliasSeq!(__traits(getUnitTests, [...]


drop that useless AliasSeq please ;)
Aug 25 2021
prev sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Wednesday, 25 August 2021 at 11:11:24 UTC, Steven 
Schveighoffer wrote:

 Disagree. I'm fine with the simple unittest experience, and 
 avoiding putting all this kind of bloat into the runtime.

 -Steve


What bloat ? Is it the amount of code needed, or the binary size 
that bothers you ?

For the former, we already have many framework doing this, so 
there's a real interest in having it, and having it in a single 
place means that improvements will be centralized, instead of 
being duplicated. So, less code bloat when you look at the big 
picture.

For the later, I've never seen anyone bother about unittest 
binary size. I don't think we should. And if the unittest 
framework code ends up in the binary when `-unittest` is not 
used, we're doing it wrong.
Aug 25 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/26/21 12:12 AM, Mathias LANG wrote:

 On Wednesday, 25 August 2021 at 11:11:24 UTC, Steven Schveighoffer wrote:

 Disagree. I'm fine with the simple unittest experience, and avoiding 
 putting all this kind of bloat into the runtime.


 
 What bloat ? Is it the amount of code needed, or the binary size that 
 bothers you ?


unittesting code is in the binary whether unittests are run or not. 
Whether unittests should be run is all decided at runtime.

But also, what I meant is, if you are going to include all the parts to 
colorize things, that now needs to be part of druntime.


 For the former, we already have many framework doing this, so there's a 
 real interest in having it, and having it in a single place means that 
 improvements will be centralized, instead of being duplicated. So, less 
 code bloat when you look at the big picture.


This is different, because the language doesn't have to include the 
framework to do it. You can write this separately.


 For the later, I've never seen anyone bother about unittest binary size. 
 I don't think we should. And if the unittest framework code ends up in 
 the binary when `-unittest` is not used, we're doing it wrong.


unittest framework code exists whether we are unittesting or not. And 
yet more seemingly unrelated "druntime" modules need to be built.

-Steve
Aug 26 2021
parent reply Mathias LANG <geod24 gmail.com> writes:
On Friday, 27 August 2021 at 00:44:03 UTC, Steven Schveighoffer 
wrote:

 unittesting code is in the binary whether unittests are run or 
 not. Whether unittests should be run is all decided at runtime.


And that is the root of the problem. Very few people expect or 
want this.
Let's just make `-unittest` its own, separate compilation mode, 
with a different entry point, like it should have been done 15 
years ago.


 But also, what I meant is, if you are going to include all the 
 parts to colorize things, that now needs to be part of druntime.


Considering the above is fixed, this isn't a problem anymore. 
Either via `--gc-sections` or similar, or simply by making it a 
compiler-instantiated template.


 This is different, because the language doesn't have to include 
 the framework to do it. You can write this separately.


I like that D allows me to do pretty much anything the way I want 
it. I can override operators to the extent that I can event 
replicate the special case that is associative array first 
assignment. That's pure, plain amazing. And it comes without 
bloat. No runtime indirection, things are "pay-as-you-go".

But when I throw in the `-unittest` switch (or the `-cov` or 
`-profile` for that matter), I want all the bells and whistles. 
Give me colors, an ncurses interface, an HTTP server even! Give 
me the full battery pack. Don't expect everyone and their mother 
to implement their own testing framework, that's just bonkers.
Aug 27 2021
parent reply Johan <j j.nl> writes:
On Friday, 27 August 2021 at 07:48:40 UTC, Mathias LANG wrote:

 But when I throw in the `-unittest` switch (or the `-cov` or 
 `-profile` for that matter), I want all the bells and whistles. 
 Give me colors, an ncurses interface, an HTTP server even! Give 
 me the full battery pack. Don't expect everyone and their 
 mother to implement their own testing framework, that's just 
 bonkers.


Why would you expect a _compiler_ to implement a testing 
framework?
If the compiler implements the testing framework, any improvement 
of the testing framework would require an update of the compiler, 
which is a _very_ large hurdle. (You want colors? You get 50 new 
language semantic changes for free!)
I think it's much better if the testing framework is 
implementated separate from anything else. That doesn't mean 
everybody has to implement it from scratch, just that they should 
download it from some other location than the compiler website.
Separation of concerns.

(by the way, I'm convinced `unittest` is a misfeature of the 
language. Besides the clear downside that is apparent from this 
thread, it hurts legibility and because of that it actually 
discourages extensive unittesting)

-Johan
Aug 27 2021
next sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Friday, 27 August 2021 at 10:33:02 UTC, Johan wrote:

 On Friday, 27 August 2021 at 07:48:40 UTC, Mathias LANG wrote:

 But when I throw in the `-unittest` switch (or the `-cov` or 
 `-profile` for that matter), I want all the bells and 
 whistles. Give me colors, an ncurses interface, an HTTP server 
 even! Give me the full battery pack. Don't expect everyone and 
 their mother to implement their own testing framework, that's 
 just bonkers.


 Why would you expect a _compiler_ to implement a testing 
 framework?


I don't. I expect the _runtime_ to do it. And to give me the tool 
to override it if I have needs that aren't covered by it. 
Currently it only does the later.
Aug 27 2021
parent reply Johan <j j.nl> writes:
On Friday, 27 August 2021 at 10:59:54 UTC, Mathias LANG wrote:

 On Friday, 27 August 2021 at 10:33:02 UTC, Johan wrote:

 On Friday, 27 August 2021 at 07:48:40 UTC, Mathias LANG wrote:

 But when I throw in the `-unittest` switch (or the `-cov` or 
 `-profile` for that matter), I want all the bells and 
 whistles. Give me colors, an ncurses interface, an HTTP 
 server even! Give me the full battery pack. Don't expect 
 everyone and their mother to implement their own testing 
 framework, that's just bonkers.


 Why would you expect a _compiler_ to implement a testing 
 framework?


 I don't. I expect the _runtime_ to do it. And to give me the 
 tool to override it if I have needs that aren't covered by it. 
 Currently it only does the later.


For this discussion I consider the compiler+runtime to be a 
single entity, but OK: same question, why should the language 
runtime do it? I find it a large stretch to include full 
unittesting framework inside the scope of a language runtime's 
core functionality.
What is so special about unittesting that requires the 
implementation to be inside the runtime? The downsides are clear 
to me (and large), what advantage is gained by having it in the 
language runtime?

-Johan
Aug 27 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/27/21 7:19 AM, Johan wrote:

 On Friday, 27 August 2021 at 10:59:54 UTC, Mathias LANG wrote:

 On Friday, 27 August 2021 at 10:33:02 UTC, Johan wrote:

 On Friday, 27 August 2021 at 07:48:40 UTC, Mathias LANG wrote:

 But when I throw in the `-unittest` switch (or the `-cov` or 
 `-profile` for that matter), I want all the bells and whistles. Give 
 me colors, an ncurses interface, an HTTP server even! Give me the 
 full battery pack. Don't expect everyone and their mother to 
 implement their own testing framework, that's just bonkers.


 Why would you expect a _compiler_ to implement a testing framework?


 I don't. I expect the _runtime_ to do it. And to give me the tool to 
 override it if I have needs that aren't covered by it. Currently it 
 only does the later.


 
 For this discussion I consider the compiler+runtime to be a single 
 entity, but OK: same question, why should the language runtime do it? I 
 find it a large stretch to include full unittesting framework inside the 
 scope of a language runtime's core functionality.
 What is so special about unittesting that requires the implementation to 
 be inside the runtime? The downsides are clear to me (and large), what 
 advantage is gained by having it in the language runtime?


separate compilation. The compiler doesn't know all the modules that 
have been built with unittests. Only the runtime does.

The compiler embeds unittests into the ModuleInfo, which means after the 
linking is done, then you can run unittests if there are any.

But the unittest "framework" right now is really simple (it's in fact 
[this 
function](https://github.com/dlang/druntime/blob/e6caaab9d359198b760c698dcb6d253afb3f81f6/src/core/r
ntime.d#L544-L655). 
I think it should stay that way, and if you want it more complex, use 
your own test system. I spent a good deal of time making it easier to 
install your own handler (in addition to NOT running main after 
unittests by default).

One thing that could be useful is a user-specified way to generate 
unittest metadata. Right now, you just get a single function pointer in 
ModuleInfo. You lose any UDAs or references to individual unittest 
functions. Allowing one to specify a metadata collector on unittests 
would be very useful in separating the unittest framework from the 
dependencies, and making it much easier to select a different one, 
regardless of your dependency preferences.

-Steve
Aug 27 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 27 August 2021 at 14:13:56 UTC, Steven Schveighoffer 
wrote:

 separate compilation. The compiler doesn't know all the modules 
 that have been built with unittests. Only the runtime does.


The is another extremely weird design.

When you run the unitests, you want to run the unitests for the 
module you specifically asked for, not for half the galaxy.
Aug 31 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Aug 31, 2021 at 11:08:46AM +0000, deadalnix via Digitalmars-d wrote:

 On Friday, 27 August 2021 at 14:13:56 UTC, Steven Schveighoffer wrote:

 separate compilation. The compiler doesn't know all the modules that
 have been built with unittests. Only the runtime does.
 


 
 The is another extremely weird design.
 
 When you run the unitests, you want to run the unitests for the module
 you specifically asked for, not for half the galaxy.


Yeah, this problem has been brought up before, but nobody had the
solution.

The problem is that when you compile with -unittest, it turns on
unittests for ALL modules, including Phobos, 3rd party libraries,
*everything*.  This is rarely what you want -- Phobos, for example,
contains a bunch of extremely heavy duty unittests that end users don't
ever want to run.  Because of this, the version=StdUnittest hack was
implemented in Phobos.  But this is not scalable: every library and his
neighbour's dog would have to implement their own unittest hack version
identifier for this scheme to work.  And if some 3rd party library
author neglected to do this, you're left out in the cold.

I think it makes much more sense to only compile unittests for modules
explicitly specified on the command-line. After all, the user is
unlikely to be interested in unittesting 3rd party libraries; his
primary interest is to unittest his *own* code.  However, this
simplistic approach falls flat in the face of `dmd -i`.  Schemes like
`-unittest=mod1,mod2,...` have been proposed before, but AFAIK never
implemented.


T

-- 
If it's green, it's biology, If it stinks, it's chemistry, If it has numbers
it's math, If it doesn't work, it's technology.
Aug 31 2021
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/31/21 12:22 PM, H. S. Teoh wrote:

 On Tue, Aug 31, 2021 at 11:08:46AM +0000, deadalnix via Digitalmars-d wrote:

 On Friday, 27 August 2021 at 14:13:56 UTC, Steven Schveighoffer wrote:

 separate compilation. The compiler doesn't know all the modules that
 have been built with unittests. Only the runtime does.


 The is another extremely weird design.

 When you run the unitests, you want to run the unitests for the module
 you specifically asked for, not for half the galaxy.




This is exactly how it works. You ask to build unittests for a module by 
specifying `-unittest` on the build line for that module. The linker 
hooks up the module infos into a data segment, and then the runtime 
looks through those looking for ones with unittests to run.

So for instance, if you link against library X, and you didn't actually 
build library X with -unittest, then you won't run library X's 
unittests, only the ones in your project.

But my point is, the reason it's done this way is because D uses the 
build system that C uses -- which has a compiler and a linker as 
separate steps.

It's also the reason that cycle detection can't be run until program 
startup.


 
 Yeah, this problem has been brought up before, but nobody had the
 solution.
 
 The problem is that when you compile with -unittest, it turns on
 unittests for ALL modules, including Phobos, 3rd party libraries,
 *everything*.  This is rarely what you want -- Phobos, for example,
 contains a bunch of extremely heavy duty unittests that end users don't
 ever want to run.  Because of this, the version=StdUnittest hack was
 implemented in Phobos.  But this is not scalable: every library and his
 neighbour's dog would have to implement their own unittest hack version
 identifier for this scheme to work.  And if some 3rd party library
 author neglected to do this, you're left out in the cold.


There is some misunderstandings with what you say here.

When you compile with `-unittest`, all unittests are turned on BUT only 
unittests in modules given on the command line are semantically analyzed 
and put into object files. So you don't get all the unittests from 3rd 
party libraries in your build. However, it does turn on the 
`version(unittest)`, which might make some libraries alter how their 
types are defined.

It also will include unittests inside templates that you instantiate 
(because it doesn't know if those were already done somewhere). This is 
why I very much dislike unittests inside templates.

The `version(StdUnittest)` was added so that extra imports, altered and 
extra types were not included, as well as removing unittests inside 
templates. It wasn't added to exclude standard unittests (which was 
already happening).

-Steve
Aug 31 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 31 August 2021 at 16:50:07 UTC, Steven Schveighoffer 
wrote:

 So for instance, if you link against library X, and you didn't 
 actually build library X with -unittest, then you won't run 
 library X's unittests, only the ones in your project.


So I was curious at to what was meant by this. Let me try to make 
it explicit why the current behavior is nonsense. I ran something 
akin to:

```
$ dmd -m64 -Isrc -w -debug -g -main -unittest -i -run 
src/somemodule.d
3 modules passed unittests
```

It is clear that this is running random unitests I don't care 
about. Module that are brought in via the -i flag are also 
compiled with their unittests, which is nonsense.

On the other hand, I made an interesting discovery:

```
$ dmd -m64 -Isrc -w -debug -g -main -unittest -i -run 
src/module1.d src/module2.d
```

Will run exclusively the tests for module1.d and ignore module2.d 
, which is considered an argument to be passed to the main that 
don't exist and is never run.

This is complete nonsense (and means that I simply run a fraction 
of my unitests and have no idea how to fix my build to runt hem 
all).

You'll note that I didn't find a way to get rdmd to do the right 
thing here, so I'm recompiling everything all the time for no 
good reason when i want to run the tests, which is also a giant 
pain in the ass.


 But my point is, the reason it's done this way is because D 
 uses the build system that C uses -- which has a compiler and a 
 linker as separate steps.


I don't think this is a good reason. There is nothing in the C 
style build.link model that prevent us from doing the right thing.


 It's also the reason that cycle detection can't be run until 
 program startup.


Cool, this can be done at startup from module infos. The problem 
is not at this level. The compiler provide set of flags that 
provide a sensible behavior. it's all nonsense.
Sep 02 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 2 September 2021 at 17:26:46 UTC, deadalnix wrote:

 ```
 $ dmd -m64 -Isrc -w -debug -g -main -unittest -i -run 
 src/somemodule.d
 3 modules passed unittests
 ```

 It is clear that this is running random unitests I don't care 
 about. Module that are brought in via the -i flag are also 
 compiled with their unittests, which is nonsense.


This is the only reasonable outcome from this combination of 
flags. Not everything that ever surprises you is nonsense. If 
you're not familiar with how dmd works, then being surprised is 
just part of learning how it works.


 On the other hand, I made an interesting discovery:

 ```
 $ dmd -m64 -Isrc -w -debug -g -main -unittest -i -run 
 src/module1.d src/module2.d
 ```

 Will run exclusively the tests for module1.d and ignore 
 module2.d , which is considered an argument to be passed to the 
 main that don't exist and is never run.


Yep. That's what -run does. It passes the remaining commandline 
to the program to be run.


 This is complete nonsense


The only *conceivable* alternative to this "complete nonsense" is 
to always require a -- separator, which dub does, which is more 
an artifact of getopt-style commandline processing than the 
outcome of a thoughtfully designed CLI.

dub requiring -- just means I never use it directly. Here's my 
two-line ~/.local/bin/dfmt:

```

dub run -q --skip-registry=all dfmt -- -i --brace_style=otbs "$ "
```


 (and means that I simply run a fraction of my unitests and have 
 no idea how to fix my build to runt hem all).


```
$ grep . *.d
alsotest.d:unittest {
alsotest.d:    assert(!0);
alsotest.d:}
donttest.d:unittest {
donttest.d:    assert(0);
donttest.d:}
pleasetest.d:unittest {
pleasetest.d:    assert(true);
pleasetest.d:}
$ dmd -c donttest.d; dmd -c -unittest pleasetest.d alsotest.d
$ dmd -main -of=main *.o
$ ./main
2 modules passed unittests
```
Sep 02 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 2 September 2021 at 17:51:10 UTC, jfondren wrote:

 This is the only reasonable outcome from this combination of 
 flags. Not everything that ever surprises you is nonsense. If 
 you're not familiar with how dmd works, then being surprised is 
 just part of learning how it works.


No, it clearly isn't. In fact, there are exactly 0 use case for 
what this combination of flag is doing and 0 combination of flags 
for use cases that exist in the wild.

If you don't see the problem, I don't know what to tell you. 
Maybe "Please do not design any software that is meant to be used 
by anyone ever, thanks".


 Yep. That's what -run does. It passes the remaining commandline 
 to the program to be run.


 This is complete nonsense


 The only *conceivable* alternative to this "complete nonsense" 
 is to always require a -- separator, which dub does, which is 
 more an artifact of getopt-style commandline processing than 
 the outcome of a thoughtfully designed CLI.


Once again, snap out of whatever mindset you are in. This is 
*OBVIOUSLY* nonsense because there is *NO PROGRAM TO BE RUN*. The 
compiler know this because I had to pass the -main flag to begin 
with.

There is *NO MAIN* there is *NO EXPECTATION THAT ANY MAIN IS RUN* 
so passing argument to it simply does not make sense.

As Pauli would say, it's not even wrong.


Think of it this way. There is a unitest feature in the language. 
There are literally no way to tell the compiler "please run the 
unitests for this set of module". Or if there is one, it is so 
bizantine that nobody seems to be able to figure it out. This is 
a 7 pages long thread about it, full of D experts, and yet, 
nobody knows how to do this simple task.
Sep 02 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 2 September 2021 at 18:01:34 UTC, deadalnix wrote:

 On Thursday, 2 September 2021 at 17:51:10 UTC, jfondren wrote:

 This is the only reasonable outcome from this combination of 
 flags. Not everything that ever surprises you is nonsense. If 
 you're not familiar with how dmd works, then being surprised 
 is just part of learning how it works.


 No, it clearly isn't. In fact, there are exactly 0 use case for 
 what this combination of flag is doing and 0 combination of 
 flags for use cases that exist in the wild.


1 use case: run all the tests in all the imported modules. The 
thing it does.


 This is *OBVIOUSLY* nonsense because there is *NO PROGRAM TO BE 
 RUN*. The compiler know this because I had to pass the -main 
 flag to begin with.


You want -run to treat the commandline completely differently 
depending on whether -main has been passed? That would help you 
to have not been surprised in this one case, but by making the 
CLI more complex it's just going to be more confusing to more 
people in the long run.

Bug report: I removed -main and suddenly was only testing a 
single module instead of the several I'd been testing before.


 Think of it this way. There is a unitest feature in the 
 language. There are literally no way to tell the compiler 
 "please run the unitests for this set of module".


This is fair. There's a lot of machinery set up *around* testing 
but there isn't a one-shot command. 'dub test' also isn't such a 
command, as it's package-oriented rather than module-oriented.


 Or if there is one, it is so bizantine that nobody seems to be 
 able to figure it out. This is a 7 pages long thread about it, 
 full of D experts, and yet, nobody knows how to do this simple 
 task.


I just demonstrated compiling three modules to only test two of 
them. This 7 pages long thread has all kinds of talk in it; it is 
not full of people asking "I have these files; how do I run only 
these tests in them?" or it'd be full of as many answers. The 
machinery built up *around* testing is very good and can be 
persuaded to work any way you want.
Sep 02 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 2 September 2021 at 18:15:17 UTC, jfondren wrote:

 You want -run to treat the commandline completely differently 
 depending on whether -main has been passed? That would help you 
 to have not been surprised in this one case, but by making the 
 CLI more complex it's just going to be more confusing to more 
 people in the long run.


No.

I want a way to run the unitests in a set of module, and only 
these modules. I don't care about `-run`, `-main` or whatever. I 
care about the fact that there is no sensible combination of 
these flags that can achieve this.


 I just demonstrated compiling three modules to only test two of 
 them. This 7 pages long thread has all kinds of talk in it; it 
 is not full of people asking "I have these files; how do I run 
 only these tests in them?" or it'd be full of as many answers. 
 The machinery built up *around* testing is very good and can be 
 persuaded to work any way you want.


This is certainly possible. Nobody ever argued this wasn't 
possible. What was argued is that you pretty much have to rollout 
your own test framework to do this, and that it doesn't compose 
well when you import 3rd party code, which your demonstration 
confirms.

The machinery built around testing is absolutely not very good. 
You can claim it is, but the very fact that you have to raise 
scafolding that do not compose to do the most basic thing is a 
demonstration that it isn't. You simply can't argue with facts.

The fact is that all the functionality that are desired to do 
something sensible exist. There is just no way to plug them into 
each other in any sensible way because all the utilities and 
flags provided do some weird shit that is kinda useful but not 
really.
Sep 02 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 2 September 2021 at 18:35:48 UTC, deadalnix wrote:

 The fact is that all the functionality that are desired to do 
 something sensible exist. There is just no way to plug them 
 into each other in any sensible way because all the utilities 
 and flags provided do some weird shit that is kinda useful but 
 not really.


The fact is that dmd is a compiler and not a build system. It 
therefore has flags that have simple, dumb, easy to compose 
behaviors rather than too-smart ones that would ever entertain 
considerations like "There is NO MAIN there is NO EXPECTATION 
THAT ANY MAIN IS RUN so ---" or "who would ever want what this 
combination of flags clearly does?"

When you pass -main, dmd adds a module to your build that 
includes a main function, job done. When you pass -unittest, dmd 
compiles in unittests and (as of recent versions) even has a 
somewhat nice default testing framework that provides feedback 
and skips running main(). When you pass --version, dmd prints out 
the version number without even asking if the commandline also 
asked for other stuff to be done.

The alternative to simple dumb flags is not a paradise where dmd 
always does what everyone expects; it's its own confusing hell 
where people are constantly surprised by behaviors that are too 
complex for the outcome to ever be predicted without a week-long 
course, instead of a quick check of the current manpage where 
each flag only needs a single sentence for a description.

I think there's an argument to be made that maybe a 'dmd test' 
should be added to run the tests of a single module. This 
wouldn't make anything easy that wasn't easy already, but it's a 
defensible bit of streamlining and it'd support one of D's 
advantages as a language. I think the very first complaint that 
someone would raise about it, in all caps, is that the use case 
of "dmd test a; dmd test b; dmd test c" would incur a lot of 
recompilation when these modules depend on each other, e.g. c 
might be built twice without unit tests and then a third time 
with them.
Sep 02 2021
next sibling parent jfondren <julian.fondren gmail.com> writes:
On Thursday, 2 September 2021 at 19:16:38 UTC, jfondren wrote:

 The fact is that dmd is a compiler and not a build system.


...

 I think there's an argument to be made that maybe a 'dmd test' 
 should be added to run the tests of a single module.


Nah, this discussion is meandering due to some frustration and 
some specific complaints taking the place of the original 
complaint. If anyone's following this thread, this is a good time 
to go back to the original post: 
https://forum.dlang.org/post/rgalgunhtubgeoyrfwxd forum.dlang.org

Where the use case is roughly

1. the developer is knee-deep in a big D project with a bunch of 
dependencies and modules
2. the developer is doing work on a single module
3. the developer wants to run that module's tests to confirm that 

4. the developer wants to do this without incurring a "testing!" 
variant of https://xkcd.com/303/ and without needing a bunch of 
boilerplate like Unittest_ThisModule version blocks in each 
module.
Sep 02 2021
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 2 September 2021 at 19:16:38 UTC, jfondren wrote:

 The alternative to simple dumb flags is not a paradise where 
 dmd always does what everyone expects; it's its own confusing 
 hell where people are constantly surprised by behaviors that 
 are too complex for the outcome to ever be predicted without a 
 week-long course, instead of a quick check of the current 
 manpage where each flag only needs a single sentence for a 
 description.


This is not the alternative. This what exists today.
Sep 02 2021
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 2 September 2021 at 20:46:57 UTC, deadalnix wrote:

 On Thursday, 2 September 2021 at 19:16:38 UTC, jfondren wrote:

 The alternative to simple dumb flags is not a paradise where 
 dmd always does what everyone expects; it's its own confusing 
 hell where people are constantly surprised by behaviors that 
 are too complex for the outcome to ever be predicted without a 
 week-long course, instead of a quick check of the current 
 manpage where each flag only needs a single sentence for a 
 description.


 This is not the alternative. This what exists today.


Case in point: https://issues.dlang.org/show_bug.cgi?id=22268
Sep 02 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/2/21 1:26 PM, deadalnix wrote:

 On Tuesday, 31 August 2021 at 16:50:07 UTC, Steven Schveighoffer wrote:

 So for instance, if you link against library X, and you didn't 
 actually build library X with -unittest, then you won't run library 
 X's unittests, only the ones in your project.


 
 So I was curious at to what was meant by this. Let me try to make it 
 explicit why the current behavior is nonsense. I ran something akin to:
 
 ```
 $ dmd -m64 -Isrc -w -debug -g -main -unittest -i -run src/somemodule.d
 3 modules passed unittests
 ```
 
 It is clear that this is running random unitests I don't care about. 
 Module that are brought in via the -i flag are also compiled with their 
 unittests, which is nonsense.


The -i flag finds all dependencies and builds them all at once as if you 
weren't linking in the object code that is in a library. That's not what 
I was saying.

What I was saying is, you have a library, you built it without -unittest 
(on its own build line, or just use dub because it does the right 
thing), and then you build your main application with -unittest, linking 
in the already-built library (or its object files). The runtime runs all 
unittests it finds, and so it's running just the modules that were built 
with the -unittest flag.

When you use -i, it means *ALL MODULES AND IMPORTS* are built with the 
-unittest flag, and so they all run.


 
 Will run exclusively the tests for module1.d and ignore module2.d , 
 which is considered an argument to be passed to the main that don't 
 exist and is never run.
 
 This is complete nonsense (and means that I simply run a fraction of my 
 unitests and have no idea how to fix my build to runt hem all).


-i is going to run all unittests. How else would it interpret the 
command line? If you want to build some modules differently than others
(i.e. some with unittests and some without), you can't use -i.


 But my point is, the reason it's done this way is because D uses the 
 build system that C uses -- which has a compiler and a linker as 
 separate steps.


 
 I don't think this is a good reason. There is nothing in the C style 
 build.link model that prevent us from doing the right thing.


What is the "right thing"? Reading what you expect in your mind is not 
what a compiler can do. Define how it will work. I think unittests work 
acceptably the way they are now, but I'm sure others would be keen to 
have some more bells and whistles. Just make a proposal and get it 
merged in.

Start by saying "when I type `dmd -m64 -Isrc -w -debug -g -main 
-unittest -i -run src/somemodule.d`, here is what I expect to happen: 
...", and then define rules that will make that happen.

-Steve
Sep 02 2021
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 2 September 2021 at 23:00:06 UTC, Steven 
Schveighoffer wrote:

 When you use -i, it means *ALL MODULES AND IMPORTS* are built 
 with the -unittest flag, and so they all run.


I know. That's dumb.


 -i is going to run all unittests. How else would it interpret 
 the command line? If you want to build some modules differently 
 than others
 (i.e. some with unittests and some without), you can't use -i.


Is this groundhog day? There are no combination of flags that do 
the sensible thing. I can't use -i, no problem. I can't use 
anything else either, big problem.

The proof is in the pudding, you just told me that I need a full 
build infra to run unittests for something that use anything else 
than phobos.


 What is the "right thing"? Reading what you expect in your mind 
 is not what a compiler can do. Define how it will work. I think 
 unittests work acceptably the way they are now, but I'm sure 
 others would be keen to have some more bells and whistles. Just 
 make a proposal and get it merged in.


The right thing is being able to run the unitests of a god damn 
module easily. not the unit tests of that module and all of its 
dependencies. Not using a complex test framework. Just running 
the unit tests of a module.

This is the sensible thing. This is the simplest thing. This is 
the "right thing".

It doesn't matter if the module has a main or not. It doesn't 
matter if the module depends on phobos or half the solar system.


 Start by saying "when I type `dmd -m64 -Isrc -w -debug -g -main 
 -unittest -i -run src/somemodule.d`, here is what I expect to 
 happen: ...", and then define rules that will make that happen.


Here is what I expect.

$ dmd --run-the-god-damn-unittest mymodule.d

And it run the unit tests of mymodule.d . Replace 
`--run-the-god-damn-unittest` by any set of flags that pleases 
you, for as long as there is one.
Sep 02 2021
next sibling parent reply Adam Ruppe <destructionator gmail.com> writes:
The -unittest flag should just accept a pattern, same as -i, so 
you can explicitly add or remove things that get the unittest 
build.

Then you can be like dmd -unittest=foo.bar -i main.d and it will 
build foo.bar with unitest and skip the rest while still 
auto-following imports.
Sep 02 2021
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Sep 02, 2021 at 11:48:25PM +0000, Adam Ruppe via Digitalmars-d wrote:

 The -unittest flag should just accept a pattern, same as -i, so you
 can explicitly add or remove things that get the unittest build.
 
 Then you can be like dmd -unittest=foo.bar -i main.d and it will build
 foo.bar with unitest and skip the rest while still auto-following
 imports.


Haha, I just wrote a post describing pretty much the same thing.  Great
minds think alike. ;-)


T

-- 
You are only young once, but you can stay immature indefinitely. -- azephrahel
Sep 02 2021
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Sep 02, 2021 at 11:22:57PM +0000, deadalnix via Digitalmars-d wrote:
[...]

 Here is what I expect.
 
 $ dmd --run-the-god-damn-unittest mymodule.d
 
 And it run the unit tests of mymodule.d . Replace
 `--run-the-god-damn-unittest` by any set of flags that pleases you,
 for as long as there is one.


I propose to extend the -unittest flag to take an optional list of
identifiers:

	-unittest=ident1,ident2,...

Each identifier is either a module or package, in which case it means
compile unittests for the listed module(s) or all modules whose FQN has
a prefix matching the identifier; or one of these special identifiers:

	.		Compile unittests only for modules whose source
			files are found in the current working directory
			or one of its descendents;

	default		Compile *all* unittests in all modules being
			compiled (the current default behaviour).

	module		Compile unittests for all modules explicitly
			specified on the command-line.

(I deliberately picked reserved keywords and symbols which cannot be
confused with D module names.)

So what Amaury wants would be accomplished by:

	dmd -unittest=module -main -i -run mymodule.d

To compile and run local unittests (i.e., everything contained in the
current directory or one of its recursive subdirectories, excluding
things outside like Phobos or 3rd party dub dependencies):

	dmd -unittest=. -main -i -run myprogram.d

To compile and run unittests for a specific packages/modules (including
a 3rd party library if for whatever strange reason you need to):

	dmd -unittest=path.to.package -i -run myprogram.d

To compile and run unittests for multiple packages/modules:

	dmd -unittest=std,thirdparty.pkg,mysubproject -i -run myprogram.d

Finally, if no argument is given to -unittest, then `-unittest=default`
is assumed, which behaves like things do today.

Notes: The above specified filters on modules only apply to the set of
modules currently being compiled; that's why -i is necessary if your
module imports anything else.

Would this satisfy everybody?


T

-- 
"Hi." "'Lo."
Sep 02 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 3 September 2021 at 00:09:37 UTC, H. S. Teoh wrote:

 Would this satisfy everybody?


1/ make module the default. This is really the sensible behavior. 
Changing imports really change the list of test that are run, 
especially not recursively.
2/ the `-main` problem remains.
3/ it still isn't possible to run the unit tests of several 
modules in one go by passing a list of modules to the compiler 
(in the same way you can compile several modules in one go and 
generate an object file for all of them at once).
Sep 02 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Sep 03, 2021 at 01:12:13AM +0000, deadalnix via Digitalmars-d wrote:

 On Friday, 3 September 2021 at 00:09:37 UTC, H. S. Teoh wrote:

 Would this satisfy everybody?
 


 
 1/ make module the default. This is really the sensible behavior.
 Changing imports really change the list of test that are run,
 especially not recursively.


Making module the default would break existing usage. But whatever, the
important thing is to implement -unittest=xxx first, then changing the
default is a 1-line change later.



 2/ the `-main` problem remains.


Even though it's kinda related, -main is really an orthogonal issue.
I've also run into the problem myself where -main is needed when your
modules don't contain main() but must be suppressed or worked around
when it does declare main().  This should be fixed so that -main
*optionally* inserts an empty main(), e.g., as a weak symbol, or
whatever, so that the existence of a real main() doesn't cause a linker
error.



 3/ it still isn't possible to run the unit tests of several modules in
 one go by passing a list of modules to the compiler (in the same way
 you can compile several modules in one go and generate an object file
 for all of them at once).


Of course you can.  Under my proposal:

	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d


T

-- 
Democracy: The triumph of popularity over principle. -- C.Bond
Sep 02 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 3 September 2021 at 03:06:27 UTC, H. S. Teoh wrote:

 2/ the `-main` problem remains.


 Even though it's kinda related, -main is really an orthogonal 
 issue. I've also run into the problem myself where -main is 
 needed when your modules don't contain main() but must be 
 suppressed or worked around when it does declare main().  This 
 should be fixed so that -main *optionally* inserts an empty 
 main(), e.g., as a weak symbol, or whatever, so that the 
 existence of a real main() doesn't cause a linker error.


Well, yes and no. It can be solved in an orthogonal fashion, but 
this really is the same problem: the flag you can pass to the 
compiler are doing nonsensical stuff and are not amendable to 
common use cases.


 3/ it still isn't possible to run the unit tests of several 
 modules in one go by passing a list of modules to the compiler 
 (in the same way you can compile several modules in one go and 
 generate an object file for all of them at once).


 Of course you can.  Under my proposal:

 	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d


No, this will run the code in mod1.d , passing ["mod2.d" "mod3.d" 
"main.d"] as command line arguments.
Sep 03 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Sep 03, 2021 at 08:39:35AM +0000, deadalnix via Digitalmars-d wrote:
[...]

 3/ it still isn't possible to run the unit tests of several
 modules in one go by passing a list of modules to the compiler (in
 the same way you can compile several modules in one go and
 generate an object file for all of them at once).


 
 Of course you can.  Under my proposal:
 
 	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d
 


 
 No, this will run the code in mod1.d , passing ["mod2.d" "mod3.d"
 "main.d"] as command line arguments.


Oh right, this is the other bogonity of -run: it expects the following
argument to be the module that gets run.  I really don't understand the
logic of this: surely the compiler ought to be able to know which module
main() is declared in, so why does it need the user to select one
specific module to be run?  This should be automated within the compiler
itself.


T

-- 
Never criticize a man until you've walked a mile in his shoes. Then when you do
criticize him, you'll be a mile away and he won't have his shoes.
Sep 05 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Sunday, 5 September 2021 at 15:04:58 UTC, H. S. Teoh wrote:

 On Fri, Sep 03, 2021 at 08:39:35AM +0000, deadalnix via 
 Digitalmars-d wrote: [...]

 3/ it still isn't possible to run the unit tests of 
 several modules in one go by passing a list of modules to 
 the compiler (in the same way you can compile several 
 modules in one go and generate an object file for all of 
 them at once).


 
 Of course you can.  Under my proposal:
 
 	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d
 


 
 No, this will run the code in mod1.d , passing ["mod2.d" 
 "mod3.d" "main.d"] as command line arguments.


 I really don't understand the logic of this: surely the 
 compiler ought to be able to know which module main() is 
 declared in, so why does it need the user to select one 
 specific module to be run?


```
$ grep -H . mod?.d
mod1.d:void main(string[] args) {
mod1.d:    import std.stdio : writeln;
mod1.d:    writeln(args[1 .. $]);
mod1.d:}
$ dmd -run mod1.d mod2.d mod3.d
["mod2.d", "mod3.d"]
```

mod1.d is a program that prints its args that I'd like to run 
with some arguments. mod2.d and mod3.d are not modules that even 
exist; I just want them printed out like that. Distinguishing 
between dmd and program arguments isn't a matter of compiler 
smarts.
Sep 05 2021
next sibling parent jfondren <julian.fondren gmail.com> writes:
On Sunday, 5 September 2021 at 15:52:51 UTC, jfondren wrote:

 On Sunday, 5 September 2021 at 15:04:58 UTC, H. S. Teoh wrote:

 I really don't understand the logic of this: surely the 
 compiler ought to be able to know which module main() is 
 declared in, so why does it need the user to select one 
 specific module to be run?


 ```
 $ grep -H . mod?.d
 mod1.d:void main(string[] args) {
 mod1.d:    import std.stdio : writeln;
 mod1.d:    writeln(args[1 .. $]);
 mod1.d:}
 $ dmd -run mod1.d mod2.d mod3.d
 ["mod2.d", "mod3.d"]
 ```

 mod1.d is a program that prints its args that I'd like to run 
 with some arguments. mod2.d and mod3.d are not modules that 
 even exist; I just want them printed out like that. 
 Distinguishing between dmd and program arguments isn't a matter 
 of compiler smarts.


More:

```
$ grep -H . mod?.d
mod1.d:void main(string[] args) {
mod1.d:    import std.stdio : writeln;
mod1.d:    writeln(args[1 .. $]);
mod1.d:}
mod2.d:shared static this() {
mod2.d:    import std.stdio : writeln;
mod2.d:    writeln("module init: mod2.d");
mod2.d:}
mod3.d:shared static this() {
mod3.d:    import std.stdio : writeln;
mod3.d:    writeln("module init: mod3.d");
mod3.d:}
$ dmd -run mod1.d mod2.d mod3.d
["mod2.d", "mod3.d"]
$ dmd mod2.d mod3.d -run mod1.d mod2.d mod3.d
module init: mod2.d
module init: mod3.d
["mod2.d", "mod3.d"]
$ dmd mod2.d mod1.d -run mod3.d mod2.d mod1.d
module init: mod2.d
module init: mod3.d
["mod2.d", "mod1.d"]
$ echo 'import mod2, mod3;' >> mod1.d
$ dmd -i -run mod1.d
module init: mod2.d
module init: mod3.d
[]
```

In particular:

```
$ dmd mod2.d mod3.d -run mod1.d mod2.d mod3.d
$ dmd mod2.d mod1.d -run mod3.d mod2.d mod1.d
```

dmd links all the objects together to make an executable and runs 
it, and the behavior's the same with main()-having mod1.d and 
main()-lacking mod3.d as the argument after -run.
Sep 05 2021
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Sep 05, 2021 at 03:52:51PM +0000, jfondren via Digitalmars-d wrote:

 On Sunday, 5 September 2021 at 15:04:58 UTC, H. S. Teoh wrote:

 On Fri, Sep 03, 2021 at 08:39:35AM +0000, deadalnix via Digitalmars-d
 wrote: [...]

 	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d




 
 No, this will run the code in mod1.d , passing ["mod2.d" "mod3.d"
 "main.d"] as command line arguments.


 
 I really don't understand the logic of this: surely the compiler
 ought to be able to know which module main() is declared in, so why
 does it need the user to select one specific module to be run?


 
 ```
 $ grep -H . mod?.d
 mod1.d:void main(string[] args) {
 mod1.d:    import std.stdio : writeln;
 mod1.d:    writeln(args[1 .. $]);
 mod1.d:}
 $ dmd -run mod1.d mod2.d mod3.d
 ["mod2.d", "mod3.d"]
 ```
 
 mod1.d is a program that prints its args that I'd like to run with
 some arguments. mod2.d and mod3.d are not modules that even exist; I
 just want them printed out like that. Distinguishing between dmd and
 program arguments isn't a matter of compiler smarts.


This is seriously a wrong conflation of the program arguments with a
program module.  If the whole point is to support specifying the
command-line to the program, then the argument list should appear
immediately after -run; the first argument after -run should not be
conflated with argv[0]. (For example, what if I'm writing an analogue of
busybox, and need to specify a different argv[0] from the name of the
executable?)

The current behaviour is counterintuitive, doesn't cover all use cases,
and has inconsistencies that lead to confusion.  It's poor design.


T

-- 
I am a consultant. My job is to make your job redundant. -- Mr Tom
Sep 05 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Sunday, 5 September 2021 at 16:12:26 UTC, H. S. Teoh wrote:

 This is seriously a wrong conflation of the program arguments 
 with a program module.  If the whole point is to support 
 specifying the command-line to the program, then the argument 
 list should appear immediately after -run; the first argument 
 after -run should not be conflated with argv[0]. (For example, 
 what if I'm writing an analogue of busybox, and need to specify 
 a different argv[0] from the name of the executable?)

 The current behaviour is counterintuitive, doesn't cover all 
 use cases, and has inconsistencies that lead to confusion.  
 It's poor design.


The current behavior is entirely intuitive, it covers all cases, 
it's consistent, it's fine. The only difference from it and a 
lazy getopt solution is lacking a -- before the arguments. The 
only difference between it and your proposed solution is that 
-run would no longer require an argument.

Listen to yourself:

```



```

It is absolutely not the case that one of these is wretched bad 
design and the other is pure and great design. These are 
completely identical designs. From the perspective of someone 
who's never seen -run used, it's a coinflip whether they'll guess 
it works one way or another, and it's no longer a coinflip after 
they read the manpage or have seen it used--it's just memory at 
that point, and either design is justifiable and easy to remember.

If the current design were counterintuitive, then people would 
occasionally screw it up even after learning it. I use -run all 
the time, almost more often than rdmd --eval, and I've never once 
had a problem. "I want to -run this" even follows SVO grammar.

I don't think I'd even be bothered--or notice--if a PR went 
through that made -run nonpositional, but you obviously don't use 
this feature all the time, what's with the ludicrously 
over-the-top vitriol about it?


 the first argument after -run should not be conflated with 
 argv[0]. (For example, what if I'm writing an analogue of 
 busybox, and need to specify a different argv[0] from the name 
 of the executable?)


argv[0] is provided by the kernel, not userspace. `man execve` 
and `man posix_spawn` and such will have an argv parameter, but 
that's for subsequent arguments. The most busybox can do is tell 
the kernel what to run with one path or another, in the extreme 
by creating a temporary symlink.
Sep 05 2021
parent jfondren <julian.fondren gmail.com> writes:
On Sunday, 5 September 2021 at 16:38:01 UTC, jfondren wrote:

 I don't think I'd even be bothered--or notice--if a PR went 
 through that made -run nonpositional, but you obviously don't 
 use this feature all the time, what's with the ludicrously 
 over-the-top vitriol about it?


Regrettably, avoiding criticizing something after 5 seconds of 
thought isn't an adequate replacement for thinking it through, 
either. Because -run still needs to know where the program's 
parameters begin, it can't be completely non-positional.
Sep 05 2021
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 5 September 2021 at 16:12:26 UTC, H. S. Teoh wrote:

 The current behaviour is counterintuitive, doesn't cover all 
 use cases, and has inconsistencies that lead to confusion.  
 It's poor design.


I'm glad that, one by one, people are waking up to this fact. It 
took 80+ messages, but we're getting there.

This is especially bonkers in the unittest case, because we are 
not even planning to run main.
Sep 05 2021
prev sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 2 September 2021 at 23:22:57 UTC, deadalnix wrote:

 The right thing is being able to run the unitests of a god damn 
 module easily.


A patch for silly.d that boils down to

```
string targetmodule;
// add "test a single module" to existing getopt flags
if (targetmodule && moduleName!module_ != targetmodule) continue;
```

permits this usage:

```
$ time dub -q test -- --module=testme
  ✓ testme __unittest_L1_C1

Summary: 1 passed, 0 failed in 0 ms

real    0m0.190s
user    0m0.129s
sys     0m0.059s
```

Which tests that single module and ignores the others. This still 
compiles in all the other modules of your project, still compiles 
in their unittest blocks, still runs their module initialization, 
still might rebuild a dub dependency, but in the end it runs only 
the unittest blocks of the chosen module.

Anything short of that is going to be a hacky once-off solution 
that relies on package-specific knowledge that a module's 
unittests don't really depend on that other stuff happening. At 
that level you may as well run 'dmd -main -unittest -run module'.

I think this is the best you're going to get. You can try this 
now by cloning the silly repo, patching it, running `dub 
add-override silly 1.1.1 .` in the repo, and trying out the usage 
above in your dub package that's already set up to use silly.
There's really nothing to the patch but it's here: 
https://run.dlang.io/is/smXgof


Is this groundhog day?


Well, there's someone in the thread who keeps complaining about 
random dmd flags, which both draws defenses of those flags and 
proposals to change them that won't actually give you the 
workflow you want (like the -unittest= proposal, that still 
requires you to tell dmd about where all your modules are, which 
would still be irritating for dub dependencies).
Sep 02 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 3 September 2021 at 00:34:01 UTC, jfondren wrote:

 Well, there's someone in the thread who keeps complaining about 
 random dmd flags, which both draws defenses of those flags and 
 proposals to change them that won't actually give you the 
 workflow you want (like the -unittest= proposal, that still 
 requires you to tell dmd about where all your modules are, 
 which would still be irritating for dub dependencies).


I'll be blunt, but this is needed because this whole thread is 
retarded.

The current flag's behavior make no sense. They are not useful. 
They don't so something sensible. They don't serve typical use 
cases. The whole thing is completely retarded.

Yes, improving unittest as some suggest would be an improvement. 
But it doesn't address the core problem. The `-main` situation 
remains retarded. It is still not possible to run the unittest of 
several modules this way, etc...

And here is the root problem: something is complete trash and 
yet, not only I'm told this is normal and expected, but I'm 
supposed to be happy with it. By yourself, by Walter, by Steven. 
You guys are obviously smart, but you aren't looking at this 
straight or even at all. Or you are drowning in so much trash you 
don't even see it anymore, like a fish in water. In don't know. 
But this is alarming.

In fact, people went through the length of migrating the whole 
standard library to use the StdUnittest version to work around 
the problem I'm pointing. Think about it. people would rather 
change how all the tests of the standard lib work rather than fix 
this shit. This is insane. Literally.

WAKE UP!

This is bad. This is really bad. Stop making excuses. It's time 
to ask, how did this get this way and what can be done so that 
this never ever happens again.

Consider the following. You work in a car company. The eng team 
just came up with this new design with the engine bolted on the 
roof of the car. You tell them "Guys, this does not make sense, 
nobody is going to want a car with the engine on the roof." and 
they respond to you "Well, this is how it is, you can get better 
cooling this way and yada yada yada...", what would you think? 
That these people need a technical solution? No, these people 
need an intervention. They are building a car for people to use, 
right?

Well, jfondren, you need an intervention. Stop. Take a break and 
think. You need to look at this with a fresh eye. You have been 
in this crap for too long and can't see straight anymore.

The set of flags the compiler propose is complete nonsense that 
do not serve users, it has cause major rewrite in the standard 
library, and motivated the creating of a new test framework per 
large project. This needs to stop.
Sep 02 2021
next sibling parent jfondren <julian.fondren gmail.com> writes:
On Friday, 3 September 2021 at 01:08:25 UTC, deadalnix wrote:

 you are drowning in so much trash you don't even see it anymore


...

 WAKE UP!


...

 This is bad. This is really bad. Stop making excuses. It's time 
 to ask, how did this get this way and what can be done so that 
 this never ever happens again.

 Consider the following.


Nah.
Sep 02 2021
prev sibling next sibling parent Dennis <dkorpel gmail.com> writes:
On Friday, 3 September 2021 at 01:08:25 UTC, deadalnix wrote:

 The current flag's behavior make no sense. They are not useful. 
 They don't so something sensible. They don't serve typical use 
 cases.


The behavior is good for [custom test 
runners](https://forum.dlang.org/post/pmniziegqhwbaqwfrbqz forum.dlang.org) or
[incremental compilation of
unittests](https://github.com/dlang/phobos/blob/d8c702dd54e9a5fd2d914df1335b664f0fd9a
20/posix.mak#L362). Dub projects can use
[silly](https://code.dlang.org/packages/silly) to filter tests, and maybe the
default dub test runner should be able to do the same. That wouldn't require a
change to the `-unittest` flag though.


 And here is the root problem: something is complete trash and 
 yet, not only I'm told this is normal and expected, but I'm 
 supposed to be happy with it.


I'm sorry to hear you're frustrated with the current situation, 
but I can't just open a dmd pull request breaking the CLI and 
dismissing everyone else's use cases and get that merged. If you 
want to rant a bit to vent your frustration, that's fine by me, 
but if you want this thread to result in an improvement, you'll 
need to come up with a specific, feasible proposal.


 In fact, people went through the length of migrating the whole 
 standard library to use the StdUnittest version to work around 
 the problem I'm pointing.


No, see [Steven's 
post](https://forum.dlang.org/post/sglmk0$33m$1 digitalmars.com).
Sep 03 2021
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/2/21 9:08 PM, deadalnix wrote:

 By yourself, by Walter, by Steven. You guys are obviously smart, but you 
 aren't looking at this straight or even at all. Or you are drowning in 
 so much trash you don't even see it anymore, like a fish in water.


Or I haven't thought about or cared about a problem with unittests ever. 
I just use `dub test` or `dub -b unittest`, and it works as I expect -- 
all unittests in my project are run, no unittests in dependencies are 
run. No trash to drown in.

I'm not too concerned with your use cases, as the current system runs 
just fine for me, and if you don't like it, you can use a fancier test 
runner that provides more control. I don't see a problem with the 
existence of external unittest systems. The basic system is designed to 
allow you complete control if you don't like the default, and the 
default is fine for most people.

I'm kinda done with this thread anyway, about to kill it...

-Steve
Sep 03 2021
prev sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 3 September 2021 at 01:08:25 UTC, deadalnix wrote:

 WAKE UP!

 This is bad. This is really bad. Stop making excuses. It's time 
 to ask, how did this get this way and what can be done so that 
 this never ever happens again.

 The set of flags the compiler propose is complete nonsense that 
 do not serve users, it has cause major rewrite in the standard 
 library, and motivated the creating of a new test framework per 
 large project. This needs to stop.


First of all, I fully agree.

Second, I was digging out my DMD bug 18843 ("-deps -unittest 
causes cataclysmic memory usage"), just to throw some more fuel 
on this trash fire of a debate, and good news: `dmd -deps 
-unittest helloworld.d` no longer causes cataclysmic memory usage:

```
$ cat helloworld.d
import std.stdio; void main() { writeln("Hello World"); }
$ dmd --version
DMD64 D Compiler v2.097.2
Copyright (C) 1999-2021 by The D Language Foundation, All Rights 
Reserved written by Walter Bright
$ dmd -deps -unittest helloworld.d
dmd2/linux/bin64/../../src/phobos/std/stdio.d(578): Error: 
undefined identifier `testFilename`
*snip 20 more semi-identical errors*
```

I don't think I need to add anything here...
Sep 03 2021
parent deadalnix <deadalnix gmail.com> writes:
On Friday, 3 September 2021 at 12:49:05 UTC, FeepingCreature 
wrote:

 Second, I was digging out my DMD bug 18843 ("-deps -unittest 
 causes cataclysmic memory usage"), just to throw some more fuel 
 on this trash fire of a debate, and good news: `dmd -deps 
 -unittest helloworld.d` no longer causes cataclysmic memory 
 usage:

 [...]

 I don't think I need to add anything here...


Well, we could add that even though it cause cataclysmic memory 
usage, the dependency tracking is not done properly, at least for 
rdmd: https://issues.dlang.org/show_bug.cgi?id=22268
Sep 03 2021
prev sibling parent reply James Blachly <james.blachly gmail.com> writes:
On 9/2/21 7:00 PM, Steven Schveighoffer wrote:

 What I was saying is, you have a library, you built it without -unittest 
 (on its own build line, or just use dub because it does the right 
 thing), and then you build your main application with -unittest, linking 
 in the already-built library (or its object files). The runtime runs all 
 unittests it finds, and so it's running just the modules that were built 
 with the -unittest flag.


Steve:

Sorry to reply a few days late and in the middle of a gargantuan thread, 
but I wanted some clarification from you on dub just doing the right 
thing --

Let's suppose I maintain a library L (written in D; has unit tests). 
Suppose further I have client program P, which uses L.

As far as I can tell, if P's dubfile includes library L, and the user 
execeutes `dub test` then the unit tests from the library L as well as 
program P are built and executed.

We had to wrap our library's unittests in `debug(library_unittest)` to 
escape this behavior.

Can you provide some clarity because it sounds like you are saying dub 
will build P's unit tests, but not library L's? Not at all my 
experience, unless it was updated recently.
Sep 08 2021
parent jfondren <julian.fondren gmail.com> writes:
On Thursday, 9 September 2021 at 00:13:00 UTC, James Blachly 
wrote:

 Let's suppose I maintain a library L (written in D; has unit 
 tests). Suppose further I have client program P, which uses L.

 As far as I can tell, if P's dubfile includes library L, and 
 the user execeutes `dub test` then the unit tests from the 
 library L as well as program P are built and executed.

 We had to wrap our library's unittests in 
 `debug(library_unittest)` to escape this behavior.

 Can you provide some clarity because it sounds like you are 
 saying dub will build P's unit tests, but not library L's? Not 
 at all my experience, unless it was updated recently.


Either one of the stated behaviors can be the case.

Consider these pair of libraries:

```
$ grep -R .
ell/dub.sdl:name "ell"
ell/dub.sdl:description "A minimal D application."
ell/dub.sdl:authors "Julian Fondren"
ell/dub.sdl:copyright "Copyright © 2021, Julian Fondren"
ell/dub.sdl:license "MIT"
ell/source/ell.d:module ell;
ell/source/ell.d:public const int x = 5;
ell/source/ell.d:unittest {
ell/source/ell.d:    assert(x != 5);
ell/source/ell.d:}
pea/dub.sdl:name "pea"
pea/dub.sdl:description "A minimal D application."
pea/dub.sdl:authors "Julian Fondren"
pea/dub.sdl:copyright "Copyright © 2021, Julian Fondren"
pea/dub.sdl:license "MIT"
pea/dub.sdl:dependency "ell" version="*"
pea/source/pea.d:module pea;
pea/source/pea.d:import ell : x;
pea/source/pea.d:public const int n = 5;
pea/source/pea.d:unittest {
pea/source/pea.d:    assert(n == x);
pea/source/pea.d:}

```

ell has tests that always fail. pea has tests that always 
succeed. pea depends on ell.

```
$ dub add-local ./ell
Registered package: ell (version: ~master)
$ cd pea
pea$ dub -q test
1 modules passed unittests
```

In this configuration, that ell fails its unittests doesn't 
matter at all to pea. The library and its value `x` are still 
used in pea.

And ell definitely fails its unittests:

```
pea$ cd ../ell; dub -q test
core.exception.AssertError source/ell.d(6): unittest failure
...
... a lot of output
...
1/1 modules FAILED unittests
Program exited with code 1
```

But let's add a single line to ell's dub configuration...

```
ell$ echo 'targetType "sourceLibrary"' >> dub.sdl
ell$ cat dub.sdl
name "ell"
description "A minimal D application."
authors "Julian Fondren"
copyright "Copyright © 2021, Julian Fondren"
license "MIT"
targetType "sourceLibrary"
```

...and we see the behavior that you noted with P and L:

```
ell$ cd ../pea
pea$ dub -q test
core.exception.AssertError ../ell/source/ell.d(6): unittest 
failure
...
... a lot of output
...
1/2 modules FAILED unittests
```


length it needs to be. (Meanwhile other problems with dub only 

does a bunch of git and repository stuff with undesirable 
slowdowns.)
Sep 08 2021
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Tuesday, 31 August 2021 at 16:22:17 UTC, H. S. Teoh wrote:

 On Tue, Aug 31, 2021 at 11:08:46AM +0000, deadalnix via 
 Digitalmars-d wrote:

 On Friday, 27 August 2021 at 14:13:56 UTC, Steven 
 Schveighoffer wrote:

 separate compilation. The compiler doesn't know all the 
 modules that have been built with unittests. Only the 
 runtime does.
 


 
 The is another extremely weird design.
 
 When you run the unitests, you want to run the unitests for 
 the module you specifically asked for, not for half the galaxy.


 Yeah, this problem has been brought up before, but nobody had 
 the solution.

 The problem is that when you compile with -unittest, it turns 
 on unittests for ALL modules, including Phobos, 3rd party 
 libraries, *everything*.  This is rarely what you want -- 
 Phobos, for example, contains a bunch of extremely heavy duty 
 unittests that end users don't ever want to run.  Because of 
 this, the version=StdUnittest hack was implemented in Phobos.  
 But this is not scalable: every library and his neighbour's dog 
 would have to implement their own unittest hack version 
 identifier for this scheme to work.  And if some 3rd party 
 library author neglected to do this, you're left out in the 
 cold.

 I think it makes much more sense to only compile unittests for 
 modules explicitly specified on the command-line. After all, 
 the user is unlikely to be interested in unittesting 3rd party 
 libraries; his primary interest is to unittest his *own* code.  
 However, this simplistic approach falls flat in the face of 
 `dmd -i`.  Schemes like `-unittest=mod1,mod2,...` have been 
 proposed before, but AFAIK never implemented.


 T


Then perhaps have some exlude-unittest argument that accepts 
packages instead?
That would allow to exclude unittest from entire set of modules.

Also why do we even make an executable when we compile with 
-unittest flag?
Can't we just make a shared lib, and then have a unit test runner 
loading this lib and run the tests in it?
Imho this would be a lot more flexible. Dmd could come with an 
default runner, and if someone doesn't like it, he could just 
replace runner with his own implementation, that would do all the 
prettyfying he wants.

The only thing needed here is well built unit test api exposed to 
the runner and reflection api at runtime, to allow that said 
runner inspect underlying tests for additional information, such 
as annotations, to do more advanced logic than default runner 
would do.

Regards,
Alexandru.
Aug 31 2021
parent jfondren <julian.fondren gmail.com> writes:
On Tuesday, 31 August 2021 at 16:52:59 UTC, Alexandru Ermicioi 
wrote:

 Imho this would be a lot more flexible. Dmd could come with an 
 default runner, and if someone doesn't like it, he could just 
 replace runner with his own implementation, that would do all 
 the prettyfying he wants.


This is already the case. Replacing the default runner is what 
silly's doing here: 
https://gitlab.com/AntonMeep/silly/-/blob/master/silly.d#L29


 The only thing needed here is well built unit test api exposed 
 to the runner and reflection api at runtime, to allow that said 
 runner inspect underlying tests for additional information, 
 such as annotations, to do more advanced logic than default 
 runner would do.


"at runtime" aside, this is already possible. I give an example 
of a test runner that runs tests differently depending on its 
annotations in 
https://forum.dlang.org/post/bukhjtbxouadyunqwdih forum.dlang.org
Aug 31 2021
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 31 August 2021 at 16:22:17 UTC, H. S. Teoh wrote:

 However, this simplistic approach falls flat in the face of 
 `dmd -i`.  Schemes like `-unittest=mod1,mod2,...` have been 
 proposed before, but AFAIK never implemented.


 T


Why is that? couldn't dmd -i simply not compile the module that 
are "autoloaded" rather than explicitly provided without 
unittests?
Aug 31 2021
parent Dennis <dkorpel gmail.com> writes:
On Tuesday, 31 August 2021 at 17:37:34 UTC, deadalnix wrote:

 Why is that? couldn't dmd -i simply not compile the module that 
 are "autoloaded" rather than explicitly provided without 
 unittests?


It could, but then `dmd -i -betterC -unittest -run testrunner.d` 
would not work anymore, where testrunner.d looks like:

```D
module testrunner;

static {
	import moda;
	import modb;
	import modc;
}

alias Seq(T...) = T;

extern(C) int main(int argc, const(char)** argv) {
	foreach(mod; Seq!(moda, modb, modc)) {
		static foreach (test; __traits(getUnitTests, mod)) {
			test();
		}
	}
	return 0;
}
```
Aug 31 2021
prev sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Friday, 27 August 2021 at 10:33:02 UTC, Johan wrote:

 On Friday, 27 August 2021 at 07:48:40 UTC, Mathias LANG wrote:

 But when I throw in the `-unittest` switch (or the `-cov` or 
 `-profile` for that matter), I want all the bells and 
 whistles. Give me colors, an ncurses interface, an HTTP server 
 even! Give me the full battery pack. Don't expect everyone and 
 their mother to implement their own testing framework, that's 
 just bonkers.


 Why would you expect a _compiler_ to implement a testing 
 framework?
 If the compiler implements the testing framework, any 
 improvement of the testing framework would require an update of 
 the compiler,


It makes for a better default experience, before someone's picked 
a testing framework and before code's gotten much investment. 
unittest{} is such a lightweight feature that a single module 
with a single function may as well have a unittest{} block.

Not too long ago dmd was completely silent on a successful 
-unittest run. Now it says "1 modules passed unittests" at the 
end. This improved the default experience and no proper testing 
frameworks were harmed.


 I think it's much better if the testing framework is 
 implementated separate from anything else.


That's already the case. silly, "The better test runner for D", 

https://code.dlang.org/ and has been in the top five for as long 
as I can remember.

It checks off everyone's complaints in this thread:
1. don't want to care about main? for historical reasons you 
still have to version(unittest) it out, but as dub is updated 
that won't be necessary either.
2. want colors? It has red and green unicode symbols.
3. don't want unittests{} blocks cluttering your code? put them 
in their own modules under tests/ and add "tests/*" to 
excludedSourceFiles of non-unittest configurations.
4. with minimal configuration following the README.md, all you 
have to do is run 'dub test' and add tests over time. It's not 
the case that everyone needs to write their own testing framework.
5. it doesn't bloat the normal build. (if you don't move the 
dependency on silly into your unittest configuration, the empty 
module adds 88 bytes to your normal build).

silly's another good example of hooking into the runtime's 
testing support: 
https://gitlab.com/AntonMeep/silly/-/blob/master/silly.d#L29


 That doesn't mean everybody has to implement it from scratch, 
 just that they should download it from some other location than 
 the compiler website.
 Separation of concerns.

 (by the way, I'm convinced `unittest` is a misfeature of the 
 language. Besides the clear downside that is apparent from this 
 thread,


What clear downside is that? A minor nit about -main? That 
there's a discussion happening at all?

I fixed the betterC bug that I noticed earlier in 
https://github.com/dlang/dmd/pull/13025 and progress is being 
made on deadalnix's complaint in 
https://github.com/dlang/dmd/pull/13026 , and for years people 
have been running 'dub test' without worrying about this or 
running 'dmd -unittest -run module', oops, 'dmd -main -unittest 
-run module', and getting more of the very gentle on-ramp to unit 
tests than they ever lost to the oops.

For the usual internet reasons this discussion is occasionally 
about 15x as heated as warranted, but since there's no fixing 
that, you shouldn't sweat it either.
Aug 27 2021
parent reply Johan <j j.nl> writes:
On Friday, 27 August 2021 at 11:15:44 UTC, jfondren wrote:

 On Friday, 27 August 2021 at 10:33:02 UTC, Johan wrote:

 (by the way, I'm convinced `unittest` is a misfeature of the 
 language. Besides the clear downside that is apparent from 
 this thread,


 What clear downside is that? A minor nit about -main? That 
 there's a discussion happening at all?


The downside that people expect it to work well.

You may be right that the thread is diverging too far from the 
OP, so I'll stop :)

-Johan
Aug 27 2021
parent jfondren <julian.fondren gmail.com> writes:
On Friday, 27 August 2021 at 11:33:14 UTC, Johan wrote:

 On Friday, 27 August 2021 at 11:15:44 UTC, jfondren wrote:

 On Friday, 27 August 2021 at 10:33:02 UTC, Johan wrote:

 (by the way, I'm convinced `unittest` is a misfeature of the 
 language. Besides the clear downside that is apparent from 
 this thread,


 What clear downside is that? A minor nit about -main? That 
 there's a discussion happening at all?


 The downside that people expect it to work well.


Look at this:

```d

/++ dub.sdl:
     configuration "release" {
         targetType "executable"
     }
     configuration "unittest" {
         targetType "library"
         dependency "silly" version="~>1.1.1"
     }
+/

T factorial(T)(T number) {
     if (number <= 1)
         return 1;
     else
         return number * factorial!T(number - 1);
}

 ("!5") unittest {
     assert(factorial(5) == 120);
}

 ("!0 and !1") unittest {
     assert(factorial(0) == 1);
     assert(factorial(1) == 1);
}

version (unittest) {
} else {
     void main(string[] args) {
         import std.conv : to;
         import std.stdio : writeln;

         writeln(args[1].to!int.factorial);
     }
}
```

Use it like a script:

```
$ ./fact.d 5; ./fact.d 10
120
3628800
```

Run its tests (colors lost in translation):

```
$ dub -q test --single fact.d
  ✓ fact !5
  ✓ fact !0 and !1

Summary: 2 passed, 0 failed in 0 ms
```

That's not too bad, is it? It can definitely be improved, but 
none of my Perl scripts have tests in them.

Nevermind the factorial, it's just something I picked out of the 
pile.
Aug 27 2021
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Aug 24, 2021 at 12:21:41PM +0000, deadalnix via Digitalmars-d wrote:
[...]

 Some of these component are libraries, some are executable. Adding a
 main for libraries is required, or it won't link, but adding one to
 executable is going to cause a link error. This in itself is a major
 main in the ass, because that means there is no one consistent way to
 unittest a module without knowing if that module has a main. In
 addition, everything needs to be built twice now, which is really
 undesirable.


There's the `-main` flag for adding an empty main() to the compiled
sources.  But yeah, if there's already a main() it will die with a
compile error.  The hack solution for this is to prepend this to your
main(), like this:

	version(unittest){} else
	void main() {
		...
	}


The point about building everything twice is legit, and is a pain point
I myself have come across.  In the old days, -unittest used to just run
unittests before invoking main(). Then somebody changed this so that
-unittest causes main() not to run.  So when coding/compiling/debugging,
I now need to build the program twice: once with -unittest, once
without.

Technically this isn't avoidable (no) thanks to `version(unittest)`: the
object files may have very different contents so compiling twice is
almost unavoidable. But in the normal, common case, there isn't *that*
much of a difference, and it would be nice to only have to compile once
(like in the old days).



 Maybe one could rdmd each modules to runt he tests? That seems like
 the best approach to me, considering one doesn't want to actually
 produce an artifact for the tests, simply run them. this also has the
 main/no main problem, but in addition, it often fails with
 incomprehensible linker errors. For some reason, rdmd is not able to
 include all dependencies consistently, but, even better, it doesn't
 take the same standard flags as other linkers do, so it is not
 possible to reuse existing build infrastructure to feed all the
 correct flags to rdmd.


Honestly, these days I find little need for rdmd. I can just run `dmd
-unittest -main -i -run main.d` and it will compile unittests, insert an
empty main(), and pull in all imports, run the resulting executable, and
clean up afterwards. Just a single command line takes care of it all.

Well, modulo the `version(unittest) else` hack for main().



 This may seems like it wouldn't be that big of a deal if you manage
 all your flags by yourself, but very quickly turns into a nightmare
 once you have to use 3rd party libraries that ship with their own set
 of flags.


Which means we need dub support for this. :-D



 At this point, I would just wish that one could rdmd --unittest
 modulename.d and just pass it a couple of -I, -L and -l flags and have
 all of it work. I have no idea how to achieve that.


[...]

As I already said:

	dmd -unittest -main -i -run main.d


T

-- 
Only boring people get bored. -- JM
Aug 24 2021
parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 24 August 2021 at 17:03:36 UTC, H. S. Teoh wrote:

 As I already said:

 	dmd -unittest -main -i -run main.d


 T


I was not aware of this feature. This is pretty good and a good 
step forward. thanks for letting me know.
Aug 25 2021
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/24/2021 5:21 AM, deadalnix wrote:

 Adding a main for libraries is required, or it won't link, but adding one to 
 executable is going to cause a link error.


This is indeed a real problem. It was solved with the -main switch to dmd,
which 
will just stick one in for you. I use it all the time for unit testing modules.
Aug 25 2021
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 25 August 2021 at 20:49:32 UTC, Walter Bright wrote:

 On 8/24/2021 5:21 AM, deadalnix wrote:

 Adding a main for libraries is required, or it won't link, but 
 adding one to executable is going to cause a link error.


 This is indeed a real problem. It was solved with the -main 
 switch to dmd, which will just stick one in for you. I use it 
 all the time for unit testing modules.


The problem with -main is that it's too blunt an instrument: it 
will add a main function whether the code you're compiling 
already has one or not. That means you have to keep track of 
which modules have main functions and which ones don't, so that 
you (or more realistically, your build system) can add -main only 
when it's necessary.

If we had something like -main=ifMissing that added an empty main 
function only if the modules being compiled didn't already have 
one, then it would be possible to unit-test any D module with a 
command line like

     dmd -i -unittest -main=ifMissing -run mymodule.d

Being able to use the same command for everything makes 
automation much easier--you can write it in a makefile rule, put 
it in a script, etc.
Aug 25 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Wednesday, 25 August 2021 at 21:05:42 UTC, Paul Backus wrote:

 If we had something like -main=ifMissing that added an empty 
 main function only if the modules being compiled didn't already 
 have one, then it would be possible to unit-test any D module 
 with a command line like

     dmd -i -unittest -main=ifMissing -run mymodule.d

 Being able to use the same command for everything makes 
 automation much easier--you can write it in a makefile rule, 
 put it in a script, etc.


Although at this point, the -main=ifMissing functionality should 
just be part of -unittest
Aug 25 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Wednesday, 25 August 2021 at 21:17:54 UTC, jfondren wrote:

 On Wednesday, 25 August 2021 at 21:05:42 UTC, Paul Backus wrote:

 If we had something like -main=ifMissing that added an empty 
 main function only if the modules being compiled didn't 
 already have one, then it would be possible to unit-test any D 
 module with a command line like

     dmd -i -unittest -main=ifMissing -run mymodule.d

 Being able to use the same command for everything makes 
 automation much easier--you can write it in a makefile rule, 
 put it in a script, etc.


 Although at this point, the -main=ifMissing functionality 
 should just be part of -unittest


Or rather, -unittest -run ? Since you could dmd -c -unittest a 
bunch of modules and then link them together.
Aug 25 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 25 August 2021 at 21:19:36 UTC, jfondren wrote:

 On Wednesday, 25 August 2021 at 21:17:54 UTC, jfondren wrote:

 On Wednesday, 25 August 2021 at 21:05:42 UTC, Paul Backus 
 wrote:

 If we had something like -main=ifMissing that added an empty 
 main function only if the modules being compiled didn't 
 already have one, then it would be possible to unit-test any 
 D module with a command line like

     dmd -i -unittest -main=ifMissing -run mymodule.d

 Being able to use the same command for everything makes 
 automation much easier--you can write it in a makefile rule, 
 put it in a script, etc.


 Although at this point, the -main=ifMissing functionality 
 should just be part of -unittest


 Or rather, -unittest -run ? Since you could dmd -c -unittest a 
 bunch of modules and then link them together.


Even if we have a rule like "-unittest -run implies 
-main=ifMissing", I think there should still be a separate flag 
for it--for documentation if nothing else.
Aug 25 2021
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/25/2021 2:46 PM, Paul Backus wrote:

     dmd -i -unittest -main=ifMissing -run mymodule.d








The trouble with adding switches like that is it takes a while to understand 
them and what they do, and they'll inevitably create yet another problem which 
will require another switch.

It's better to use one of the existing methods shown in this thread which use 
already well-understood behavior.
Aug 25 2021
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/25/2021 2:05 PM, Paul Backus wrote:

 The problem with -main is that it's too blunt an instrument: it will add a
main 
 function whether the code you're compiling already has one or not. That means 
 you have to keep track of which modules have main functions and which ones 
 don't, so that you (or more realistically, your build system) can add -main
only 
 when it's necessary.


That problem has never occurred to me, and has never happened to me, yet I use 
-main all the time.

One way to resolve it is just put your main() in a separate module.
Aug 25 2021
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Aug 25, 2021 at 03:44:40PM -0700, Walter Bright via Digitalmars-d wrote:

 On 8/25/2021 2:05 PM, Paul Backus wrote:

 The problem with -main is that it's too blunt an instrument: it will
 add a main function whether the code you're compiling already has
 one or not. That means you have to keep track of which modules have
 main functions and which ones don't, so that you (or more
 realistically, your build system) can add -main only when it's
 necessary.


 
 That problem has never occurred to me, and has never happened to me,
 yet I use -main all the time.
 
 One way to resolve it is just put your main() in a separate module.


This always happens to me, so much so that I've come to insert the
following as boilerplate to all my code:

	version(unittest){} else
	int main(string[] args) { ... }


T

-- 
LINUX = Lousy Interface for Nefarious Unix Xenophobes.
Aug 25 2021
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/25/2021 3:48 PM, H. S. Teoh wrote:

 This always happens to me, so much so that I've come to insert the
 following as boilerplate to all my code:
 
 	version(unittest){} else
 	int main(string[] args) { ... }



That works, too.
Aug 25 2021
prev sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 25 August 2021 at 22:44:40 UTC, Walter Bright wrote:

 On 8/25/2021 2:05 PM, Paul Backus wrote:

 The problem with -main is that it's too blunt an instrument: 
 it will add a main function whether the code you're compiling 
 already has one or not. That means you have to keep track of 
 which modules have main functions and which ones don't, so 
 that you (or more realistically, your build system) can add 
 -main only when it's necessary.


 That problem has never occurred to me, and has never happened 
 to me, yet I use -main all the time.

 One way to resolve it is just put your main() in a separate 
 module.


What if you are dealing with something that is a single file?

If you run below on run.dlang.org with either `-unittest` or 
`-unittest -main`, then you get a message that only one main is 
allowed. If you remove the dependency, then it compiles without 
error with `-unittest` but gets the same error with `-unittest 
-main`. The difference is (IIRC) when you include a dependency it 
is also calling `-single`. If you had something like 
`-main=ifmissing` that others describe, then there would be a 
consistent solution that could work well with run.dlang.org.

```d
/+dub.sdl:
dependency "mir-algorithm" versio="*"
+/

unittest {}
void main() {}
```
Aug 25 2021
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 25 August 2021 at 20:49:32 UTC, Walter Bright wrote:

 On 8/24/2021 5:21 AM, deadalnix wrote:

 Adding a main for libraries is required, or it won't link, but 
 adding one to executable is going to cause a link error.


 This is indeed a real problem. It was solved with the -main 
 switch to dmd, which will just stick one in for you. I use it 
 all the time for unit testing modules.


No it wasn't, because it add a main no matter what, which means 
that when there is a main in the module being tested, you get a 
link error.

The root of the problem, really, is that I don't cares about main 
when running unit tests. I don't want to know if it is there or 
not, I don't want it to run, nothing. If the compiler needs one 
to link under the hood, then good for it, but this is an 
implementation detail.

When running the unit tests, one wants to know if the unit tests 
pass. Nothing more, nothing less.

The compiler has 100% of the information it needs to make this 
happen, yet, for some reason, it is requesting them from me. 
Which i can provide too, but this is fragile, and it doesn't 
scale, as I can't get my build system to automate it for me. 
Which in turn lead to most projects rolling out their own 
solution.

I can work with having to pass the -main flag. This is not ideal, 
because what else could I possibly want? A link error? I'm not 
trying to link and run main. I care about unittests in that case. 
But having main declared twice is some special level of nonsense. 
In what case could I possibly want two main?
Aug 26 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 26 August 2021 at 09:38:54 UTC, deadalnix wrote:

 The root of the problem, really, is that I don't cares about 
 main when running unit tests. I don't want to know if it is 
 there or not, I don't want it to run, nothing. If the compiler 
 needs one to link under the hood, then good for it, but this is 
 an implementation detail.


OK. Create a bugzilla issue. There are some related bugs that 


and -of).

This is about the extent of the current support, in 
src/dmd/mars.d's tryMain:

```d
     if (params.addMain)
         files.push("__main.d");
     // Create Modules
     Modules modules = createModules(files, libmodules);
     // Read files
     // Start by "reading" the special files (__main.d, __stdin.d)
     foreach (m; modules)
     {
         if (params.addMain && m.srcfile.toString() == "__main.d")
         {
             auto data = arraydup("int main(){return 
0;}\0\0\0\0"); // need 2 trailing nulls for sentinel and 2 for 
lexer
             m.srcBuffer = new FileBuffer(cast(ubyte[]) data[0 .. 
$-4]);
         }
```

and src/dmd/glue.d:

```d
private bool onlyOneMain(Loc loc)
{
     __gshared Loc lastLoc;
     __gshared bool hasMain = false;
     if (hasMain)
     {
         const(char)* msg = "";
         if (global.params.addMain)
             msg = ", -main switch added another `main()`";
```

It's not surprising that such a lightweight implementation 
doesn't result in world-class ergonomics. It's something I use 
all the time to test individual modules at the CLI, and I'd much 
rather have this than nothing. But for bells and whistles I'd 
look to dub.

In a pinch you could write a dmd wrapper that calls dmd, sees if 
it gets a linker complaint about main, and if so calls dmd again 
with -main. An obvious performance hit, probably not that bad in 
most cases where you would consider this, but now you don't have 
care about main.

```d

import std.process : execute, spawnProcess, wait;
import std.algorithm : canFind;
import std.stdio : write;

int main(string[] args) {
     auto result = execute(["dmd"] ~ args[1..$]);
     if (result.status != 0 && result.output.canFind("undefined 
reference to `main'")) {
         return spawnProcess(["dmd", "-main"] ~ args[1..$]).wait;
     }
     write(result.output);
     return result.status;
}
```

usage:

```
$ cat nomain.d
unittest {
     import std.stdio : writeln;

     writeln("no main");
}
$ cat hasmain.d
unittest {
     import std.stdio : writeln;

     writeln("has main");
}

void main() {}
$ ./automain.d -unittest -run nomain.d
no main
1 modules passed unittests
$ ./automain.d -unittest -run hasmain.d
1 modules passed unittests
has main
```
Aug 26 2021
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 26 August 2021 at 11:32:06 UTC, jfondren wrote:

 On Thursday, 26 August 2021 at 09:38:54 UTC, deadalnix wrote:

 The root of the problem, really, is that I don't cares about 
 main when running unit tests. I don't want to know if it is 
 there or not, I don't want it to run, nothing. If the compiler 
 needs one to link under the hood, then good for it, but this 
 is an implementation detail.


 OK. Create a bugzilla issue. There are some related bugs that 


 with -c and -of).

 This is about the extent of the current support, in 
 src/dmd/mars.d's tryMain:

 ```d
     if (params.addMain)
         files.push("__main.d");
     // Create Modules
     Modules modules = createModules(files, libmodules);
     // Read files
     // Start by "reading" the special files (__main.d, 
 __stdin.d)
     foreach (m; modules)
     {
         if (params.addMain && m.srcfile.toString() == 
 "__main.d")
         {
             auto data = arraydup("int main(){return 
 0;}\0\0\0\0"); // need 2 trailing nulls for sentinel and 2 for 
 lexer
             m.srcBuffer = new FileBuffer(cast(ubyte[]) data[0 
 .. $-4]);
         }
 ```

 and src/dmd/glue.d:

 ```d
 private bool onlyOneMain(Loc loc)
 {
     __gshared Loc lastLoc;
     __gshared bool hasMain = false;
     if (hasMain)
     {
         const(char)* msg = "";
         if (global.params.addMain)
             msg = ", -main switch added another `main()`";
 ```

 It's not surprising that such a lightweight implementation 
 doesn't result in world-class ergonomics. It's something I use 
 all the time to test individual modules at the CLI, and I'd 
 much rather have this than nothing. But for bells and whistles 
 I'd look to dub.

 In a pinch you could write a dmd wrapper that calls dmd, sees 
 if it gets a linker complaint about main, and if so calls dmd 
 again with -main. An obvious performance hit, probably not that 
 bad in most cases where you would consider this, but now you 
 don't have care about main.

 [...]


Congratulations, you just wrote the embryo of a test framework. 
It will work, and indeed, most projects ou there get one past a 
certain size.

It' very unfortunate though, because the compiler has all the 
info it needs to do the right thing, and yet refuses to do it.

Note that you could likely get better perfs out of this by 
linking in a weak definition of _Dmain, so it won't clash, but 
the general problem remains.
Aug 26 2021
prev sibling next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 26 August 2021 at 09:38:54 UTC, deadalnix wrote:

 But having main declared twice is some special level of 
 nonsense. In what case could I possibly want two main?


When there's both a main function and the `-main` flag, should 
`-main` do nothing or override the main function with an empty 
one?
Aug 26 2021
next sibling parent jfondren <julian.fondren gmail.com> writes:
On Thursday, 26 August 2021 at 23:29:20 UTC, Dennis wrote:

 On Thursday, 26 August 2021 at 09:38:54 UTC, deadalnix wrote:

 But having main declared twice is some special level of 
 nonsense. In what case could I possibly want two main?


 When there's both a main function and the `-main` flag, should 
 `-main` do nothing or override the main function with an empty 
 one?


In the -unittest usage, and without some code overriding the 
runtime's testrunner, these are equivalent outcomes since dmd no 
longer runs main after unittests:

```d
import std.stdio;
unittest { writeln("unittest"); }
void main() { writeln("main"); }
```

contrast:

```
$ dmd -unittest -run ignoremain.d
unittest
1 modules passed unittests

$ gdc -funittest -o not-ignored ignoremain.d
$ ./not-ignored
unittest
main
```
Aug 26 2021
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 26 August 2021 at 23:29:20 UTC, Dennis wrote:

 On Thursday, 26 August 2021 at 09:38:54 UTC, deadalnix wrote:

 But having main declared twice is some special level of 
 nonsense. In what case could I possibly want two main?


 When there's both a main function and the `-main` flag, should 
 `-main` do nothing or override the main function with an empty 
 one?


This is the wrong question. When one runs the unittests, they 
don't care about main. The existance of the main flag is dubious 
to begin with.
Aug 27 2021
parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 27 August 2021 at 10:30:58 UTC, deadalnix wrote:

 This is the wrong question. When one runs the unittests, they 
 don't care about main. The existance of the main flag is 
 dubious to begin with.


Saying "-unittest should have been designed differently" doesn't 
help us forward. Currently it's possible to have a -betterC test 
runner in main, or set `UnitTestResult.runmain = true` from 
`core.runtime`, or to create a unittest build with incremental 
separate compilation, so changing the semantics of the -unittest 
flag is a very disruptive change.

However, like you said, there is no use case for having two 
`main` functions in one build, so I was looking into improving 
that (see https://github.com/dlang/dmd/pull/13026), which is how 
I got to that very real implementation question.

jfondren pointed out that the default test runner doesn't run 
main, so I think it's best to make -main only insert main when 
there isn't already one being compiled in. Then you should be 
able to do `dmd -unittest -main -run somemodule.d` on any 
(self-contained) d file with consistent results.
Aug 27 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/27/21 9:35 AM, Dennis wrote:

 On Friday, 27 August 2021 at 10:30:58 UTC, deadalnix wrote:

 This is the wrong question. When one runs the unittests, they don't 
 care about main. The existance of the main flag is dubious to begin with.


 
 Saying "-unittest should have been designed differently" doesn't help us 
 forward. Currently it's possible to have a -betterC test runner in main, 
 or set `UnitTestResult.runmain = true` from `core.runtime`, or to create 
 a unittest build with incremental separate compilation, so changing the 
 semantics of the -unittest flag is a very disruptive change.
 
 However, like you said, there is no use case for having two `main` 
 functions in one build, so I was looking into improving that (see 
 https://github.com/dlang/dmd/pull/13026), which is how I got to that 
 very real implementation question.
 
 jfondren pointed out that the default test runner doesn't run main, so I 
 think it's best to make -main only insert main when there isn't already 
 one being compiled in. Then you should be able to do `dmd -unittest 
 -main -run somemodule.d` on any (self-contained) d file with consistent 
 results.


The default test runner doesn't run main *by default*, but it also *can* 
run main if passed a runtime option. So it still needs a main to run if 
requested.

I think a "-nomain" option is something worth looking at (basically, the 
generated `extern(C) main` function doesn't call a missing `dmain` 
function). It can't be tied to `-unittest` though.

-Steve
Aug 27 2021
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/26/2021 2:38 AM, deadalnix wrote:

 I can work with having to pass the -main flag. This is not ideal, because what 
 else could I possibly want?


Then we'll get a bug report where the user compiled a module, and it compiled 
and linked without error, then he runs the program and nothing happens because 
the compiler inserted an empty main().
Aug 27 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 28 August 2021 at 00:47:44 UTC, Walter Bright wrote:

 On 8/26/2021 2:38 AM, deadalnix wrote:

 I can work with having to pass the -main flag. This is not 
 ideal, because what else could I possibly want?


 Then we'll get a bug report where the user compiled a module, 
 and it compiled and linked without error, then he runs the 
 program and nothing happens because the compiler inserted an 
 empty main().


No, because it'll run the unittests, and the user will see "X 
test passed" or something similar when running the executable, 
understand that they ran the uni tests and go ahead and run the 
executable that has been built without the unitests flag.

If the user doesn't have any unitests in the executable, then 
saying "0 test run, 0 passed, 0 failed" is enough to indicated to 
the user that this was a unitests build.
Aug 31 2021
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/31/2021 4:04 AM, deadalnix wrote:

 No, because it'll run the unittests, and the user will see "X test passed" or 
 something similar when running the executable, understand that they ran the
uni 
 tests and go ahead and run the executable that has been built without the 
 unitests flag.
 
 If the user doesn't have any unitests in the executable, then saying "0 test 
 run, 0 passed, 0 failed" is enough to indicated to the user that this was a 
 unitests build.


The unittest feature doesn't do any of those things unless you code it to.
Sep 05 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 5 September 2021 at 21:00:15 UTC, Walter Bright wrote:

 On 8/31/2021 4:04 AM, deadalnix wrote:

 No, because it'll run the unittests, and the user will see "X 
 test passed" or something similar when running the executable, 
 understand that they ran the uni tests and go ahead and run 
 the executable that has been built without the unitests flag.
 
 If the user doesn't have any unitests in the executable, then 
 saying "0 test run, 0 passed, 0 failed" is enough to indicated 
 to the user that this was a unitests build.


 The unittest feature doesn't do any of those things unless you 
 code it to.


It already output a very similar message: "15 modules passed 
unittests"

The specifics of the message do not matter here.
Sep 05 2021
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/5/2021 3:00 PM, deadalnix wrote:

 On Sunday, 5 September 2021 at 21:00:15 UTC, Walter Bright wrote:

 The unittest feature doesn't do any of those things unless you code it to.


 
 It already output a very similar message: "15 modules passed unittests"


That must be a fairly recent change I wasn't aware of.
Sep 06 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Monday, 6 September 2021 at 16:35:03 UTC, Walter Bright wrote:

 On 9/5/2021 3:00 PM, deadalnix wrote:

 On Sunday, 5 September 2021 at 21:00:15 UTC, Walter Bright 
 wrote:

 The unittest feature doesn't do any of those things unless 
 you code it to.


 
 It already output a very similar message: "15 modules passed 
 unittests"


 That must be a fairly recent change I wasn't aware of.


It came in with https://github.com/dlang/druntime/pull/1685 , 
which also added a lot more control over how the tests could be 
run.

Old behavior: adding -unittest causes unittest functions to be 
compiled in and to run before main, and that's it: the program 
only exited before main on error because that's how assert() 
failures are handled, and the program continued to run if there 
were no errors. This perfectly fits the use case of "I want a 
build of my program with a bunch of extra testing, ala a debug or 
a non-release build" but required some workarounds to fit the use 
case of "I want to run all of my unittests, *only*"

Much of this thread can be listed under complaints about a third 
use case, "I want to run a single module's tests, *only*"
Sep 06 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 6 September 2021 at 16:52:38 UTC, jfondren wrote:

 It came in with https://github.com/dlang/druntime/pull/1685 , 
 which also added a lot more control over how the tests could be 
 run.


https://github.com/dlang/druntime/pull/1685#issuecomment-256795118

Damn, I don't know who that guy is, but it seems like he foresaw 
the current mess that we are in.
Sep 06 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Monday, 6 September 2021 at 16:55:30 UTC, deadalnix wrote:

 On Monday, 6 September 2021 at 16:52:38 UTC, jfondren wrote:

 It came in with https://github.com/dlang/druntime/pull/1685 , 
 which also added a lot more control over how the tests could 
 be run.


 https://github.com/dlang/druntime/pull/1685#issuecomment-256795118

 Damn, I don't know who that guy is, but it seems like he 
 foresaw the current mess that we are in.


-main wasn't and still isn't a problem for the use case the PR 
was authored for, of "I want to run all my unit tests, *only*", 
and the approach of the PR approach was completely successful at 
making druntime much more friendly to that use case. It had zero 
impact on your use case, but you didn't sell it very well by only 
mentioning what for other uses was a very trivial inconvenience.
Sep 06 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Monday, 6 September 2021 at 22:53:50 UTC, deadalnix wrote:

 On Monday, 6 September 2021 at 17:12:50 UTC, jfondren wrote:

 -main wasn't and still isn't a problem for the use case the PR 
 was authored for, of "I want to run all my unit tests, 
 *only*", and the approach of the PR approach was completely 
 successful at making druntime much more friendly to that use 
 case. It had zero impact on your use case, but you didn't sell 
 it very well by only mentioning what for other uses was a very 
 trivial inconvenience.


 [something deadalnix typed a while ago and has been waiting to 
 post]


The blindness you have to precisely how people intend to use 
features, "use case blindness", has resulted in a lot of trouble 
for you:

1. you earn pointless animosity by describing existing features, 
which other people use all the time in comfort, as entirely 
broken and obviously badly designed.

2. you keep derailing your own thread about your own use case, 
because you don't think it's important to keep the focus on 
*your* use case.

3. even when you talk about problems that directly relate to 
pretty severe inconveniences for your use case, you're unable to 
interest other people in them, because you don't think to connect 
the problems to a use case where they *are* severe inconveniences.

You keep appealing to experiences that are actually personal to 
you and not generally felt. I've mostly enjoyed this thread and 
appreciate that you made it; I dug a lot into unit tests in D as 
a result of following it. The author of the PR that you think has 
helped 'lead to this mess' has said he's happy fine with the 
state of D unit testing. Just like other people are not all 
constantly slapping their foreheads over how irritating -main is 
to use, and just like the other participants of that PR didn't 
think "it doesn't fix -main" was a showstopper for that PR, other 
people also do not all think that "General has a 80 post thread 
about unit tests" is an argument in itself than unit tests have a 
problem.

I think a still likely outcome is that a general interest in 
unittests will result in a lot of improvements, but only to other 
people's use cases. If that's the case in a few months, give 
Discord a try.
Sep 06 2021
parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Monday, 6 September 2021 at 23:57:05 UTC, jfondren wrote:

 The blindness you have to precisely how people intend to use 
 features, "use case blindness", has resulted in a lot of 
 trouble for you:

 1. you earn pointless animosity by describing existing 
 features, which other people use all the time in comfort, as 
 entirely broken and obviously badly designed.


Doesn't mean that it should not be improved, or properly fixed. 
Oh and if someone doesn't complain about them, doesn't mean that 
they have good architecture.


 2. you keep derailing your own thread about your own use case, 
 because you don't think it's important to keep the focus on 
 *your* use case.


Selecting just a few modules to compile in a large project will 
affect not only him, but all projects that use source libraries 
that have long compile time of the unit tests.


 3. even when you talk about problems that directly relate to 
 pretty severe inconveniences for your use case, you're unable 
 to interest other people in them, because you don't think to 
 connect the problems to a use case where they *are* severe 
 inconveniences.


Why are you trying to infer what he thinks, and post this as his 
thoughts?
He did interest at least me, since I had experience of waiting 
unit tests compile for a source library that I didn't need to in 
first place. And to tell the truth it wasn't a nice experience, 
given D advertises fast compilation cycles.

Best regards,
Alexandru.
Sep 07 2021
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Monday, 6 September 2021 at 16:55:30 UTC, deadalnix wrote:

 Damn, I don't know who that guy is, but it seems like he 
 foresaw the current mess that we are in.


The comment reads:


 I'm not convinced this is the right approach. The thing will 
 still fail to link when no main function is provided.

 IMO, it is better to do this as proposed by basil. I plan to do 
 a DMD PR.


So what happened to that plan? What's basil's proposal?
Sep 06 2021
parent deadalnix <deadalnix gmail.com> writes:
On Monday, 6 September 2021 at 20:19:02 UTC, Dennis wrote:

 I'm not convinced this is the right approach. The thing will 
 still fail to link when no main function is provided.

 IMO, it is better to do this as proposed by basil. I plan to 
 do a DMD PR.


 So what happened to that plan? What's basil's proposal?


I have no idea, I do not even remember having posted that comment.
Sep 06 2021
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 26 August 2021 at 09:38:54 UTC, deadalnix wrote:

 On Wednesday, 25 August 2021 at 20:49:32 UTC, Walter Bright 
 wrote:

 This is indeed a real problem. It was solved with the -main 
 switch to dmd, which will just stick one in for you. I use it 
 all the time for unit testing modules.


 No it wasn't, because it add a main no matter what, which means 
 that when there is a main in the module being tested, you get a 
 link error.


Good news: this is now fixed!

https://github.com/dlang/dmd/pull/13057

Grab a 
[nightly](https://github.com/dlang/dmd/releases/tag/nightly) and 
try out: `dmd -unittest -main -run`
Sep 11 2021
parent deadalnix <deadalnix gmail.com> writes:
On Saturday, 11 September 2021 at 21:23:07 UTC, Dennis wrote:

 Good news: this is now fixed!

 https://github.com/dlang/dmd/pull/13057

 Grab a 
 [nightly](https://github.com/dlang/dmd/releases/tag/nightly) 
 and try out: `dmd -unittest -main -run`


https://github.com/dlang/dmd/pull/13057#issuecomment-918061875
Sep 13 2021