www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP82: static unittest blocks

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
This DIP provides a way to handle unittest blocks inside of 
templates which works with ddoc without compiling the unittest 
blocks into each instantiation.

http://wiki.dlang.org/DIP82


At present, we really can't put unittest blocks - including 
ddoc-ed unittest blocks - inside of templated types, which has 
been a problem for Phobos and came up again just the other day 
when someone was helpful and went and tried to turn the examples 
in std.datetime's *Interval types in ddoc-ed unit tests, but we 
had to reject the changes, because it would have resulted in 
those tests being compiled into every instantiation of those 
templated types, polluting the code of anyone who uses them as 
well as making the Phobos unit tests take longer to run for no 
extra benefit whatsoever.

So, I wrote up this DIP in the hopes of solving the problem in a 
clean and simple manner which fits in well with the existing 
language features.

- Jonathan M Davis
Sep 26 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-09-27 07:01, Jonathan M Davis wrote:
 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.

 http://wiki.dlang.org/DIP82
How would this work in the compiler? Currently it only lexes and parses uninstantiated templates. I think that Andrei also wants to make the lexing and parsing lazy. -- /Jacob Carlborg
Sep 27 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 09:24:29 UTC, Jacob Carlborg 
wrote:
 On 2015-09-27 07:01, Jonathan M Davis wrote:
 This DIP provides a way to handle unittest blocks inside of 
 templates
 which works with ddoc without compiling the unittest blocks 
 into each
 instantiation.

 http://wiki.dlang.org/DIP82
How would this work in the compiler? Currently it only lexes and parses uninstantiated templates. I think that Andrei also wants to make the lexing and parsing lazy.
I don't know the details of how this would be implemented in the compiler, but clearly what that the compiler currently does is enough to generate documentation from the ddoc comments inside of templates, and I wouldn't think that detecting a static unittest would be much more complicated than that. It can then do full analysis of the static unittest if it's the module with the template that's being compiled, and it's being compiled with -unittest, and otherwise, it can pretty much ignore it (or at least, ignore it as much as any other unittest block gets ignored when -unittest isn't used). But we have a real need here if we want to be able to actually have unit tests inside of templated types next to what they're unit testing. So, I would certainly hope that it's reasonable to implement something like this without complicating the compiler much. Certainly, the fact that ddoc works inside of templates makes it seem like this should be feasible. - Jonathan M Davis
Sep 27 2015
prev sibling next sibling parent reply Kenji Hara via Digitalmars-d <digitalmars-d puremagic.com> writes:
2015-09-27 14:01 GMT+09:00 Jonathan M Davis via Digitalmars-d <
digitalmars-d puremagic.com>:

 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.

 http://wiki.dlang.org/DIP82


 At present, we really can't put unittest blocks - including ddoc-ed
 unittest blocks - inside of templated types, which has been a problem for
 Phobos and came up again just the other day when someone was helpful and
 went and tried to turn the examples in std.datetime's *Interval types in
 ddoc-ed unit tests, but we had to reject the changes, because it would have
 resulted in those tests being compiled into every instantiation of those
 templated types, polluting the code of anyone who uses them as well as
 making the Phobos unit tests take longer to run for no extra benefit
 whatsoever.

 So, I wrote up this DIP in the hopes of solving the problem in a clean and
 simple manner which fits in well with the existing language features.
Interesting proposal. Quick questions: 1. I think the token `static` should be placed at the immediate front of `unittest` static unittest { ... } static nothrow unittest { ... } // NG It's consistent with the behavior of `static this()` family. 2. Currently the members of template won't be semantically analyzed until it's instantiated. So, when the `static unittest` is enclosed in other blocks, how it works? template Foo(T) { // inside conditional compilation blocks: static if (is(T == int)) { static unittest { ... } } version (UserDefinedVersion) { static unittest { ... } } debug (UserDefinedDebugId) { static unittest { ... } } // inside block/labeled attributes nothrow { static unittest { ... } } safe: static unittest { ... } mixin("static unittest { ... }"); mixin(makeStaticUnittest()); } string makeStaticUnittest() { return "static unittest { ... }"; } Kenji Hara
Sep 27 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 10:32:00 UTC, Kenji Hara wrote:
 2015-09-27 14:01 GMT+09:00 Jonathan M Davis via Digitalmars-d < 
 digitalmars-d puremagic.com>:

 This DIP provides a way to handle unittest blocks inside of 
 templates which works with ddoc without compiling the unittest 
 blocks into each instantiation.

 http://wiki.dlang.org/DIP82


 At present, we really can't put unittest blocks - including 
 ddoc-ed unittest blocks - inside of templated types, which has 
 been a problem for Phobos and came up again just the other day 
 when someone was helpful and went and tried to turn the 
 examples in std.datetime's *Interval types in ddoc-ed unit 
 tests, but we had to reject the changes, because it would have 
 resulted in those tests being compiled into every 
 instantiation of those templated types, polluting the code of 
 anyone who uses them as well as making the Phobos unit tests 
 take longer to run for no extra benefit whatsoever.

 So, I wrote up this DIP in the hopes of solving the problem in 
 a clean and simple manner which fits in well with the existing 
 language features.
Interesting proposal. Quick questions: 1. I think the token `static` should be placed at the immediate front of `unittest` static unittest { ... } static nothrow unittest { ... } // NG It's consistent with the behavior of `static this()` family.
I confess that I've always found it to be inconsistent that static constructors required that the static be immediately before rather than being allowed before or after and among other attributes like would occur with any other function. But if we're going to retain that inconsistency with static constructors, it doesn't really make things much worse to do the same with unnittest blocks.
 2. Currently the members of template won't be semantically 
 analyzed until it's instantiated. So, when the `static 
 unittest` is enclosed in other blocks, how it works?

     template Foo(T) {
         // inside conditional compilation blocks:
         static if (is(T == int)) {
             static unittest { ... }
         }
         version (UserDefinedVersion) {
             static unittest { ... }
         }
         debug (UserDefinedDebugId) {
             static unittest { ... }
         }

         // inside block/labeled attributes
         nothrow {
             static unittest { ... }
         }
          safe:
             static unittest { ... }

         mixin("static unittest { ... }");
         mixin(makeStaticUnittest());
     }

     string makeStaticUnittest() { return "static unittest { ... 
 }"; }
Well, my first question would be what happens with a ddoc comment on a symbol within a template? It works to put ddoc comments on symbols inside of templates, so clearly we have to have rules of some kind which deal with each of the cases that you have here, though I'm not as familiar as I probably should be with which of those results in a symbol having its ddoc comment in the documentation and which doesn't. Regardles, it does indeed get a bit interesting. Ideally, I would think that all of them save for the static if and mixins would act exactly like they would outside of a template, since nothing about this is specific to a template instantiation, but if no semantic analysis is currently done on a template until it's instantiated, then supporting a static unittest inside of a version block, debug block, a block for an attribute, or after an attribute used like a label would mean that at least some amount of semantic analysis would have to occur that doesn't necessarily happen now (though again, I wonder how that works with ddoc comments). Maybe no semantic analysis is done if no static unittest is found inside of them, and if one or more static unittest is found inside of them, the minimal amount of semantic analysis required to determine whether they get compiled in is done? As for the static if, I don't see how it can work if it depends on any template arguments. I would think that it would work easily enough like the version blocks and debug blocks and whatnot if the condition did not depend on the template arguments, but I think that it's clear that if the condition depends on the template arguments at all, then any static unittest blocks in there aren't going to work and probably should be considered errors (even if it's just when the template is instantiated and the static if's condition is true). Now, as to the mixins... It would be nice if they worked, but I have a hard time believing that it's reasonable to make them work. We would like to be able to have mixins work with ddoc comments, since without that, we can't put documentation on code that's mixed in. So, if we ever implement that, then it might be reasonable to do the same with static unittest blocks. But other than that, the result would be that every mixin inside of a template would have to be generated just to see whether it had a static unittest in it, which is kind of ugly. If the argument to mixin required a template argument to generate it, then it could clearly be skipped, but if not, then we'd have to generate it just to check, which would increase the compilation times for any template which did that, which wouldn't exactly be pretty. It probably wouldn't be a big deal in general, since in most cases, if you're going to mix in code like that, you're probably going to want to do it based on one or more template arguments, in which case, the compiler would be able to see that it didn't need to generate the mixin. Still, it would probably be better to just not support mixins like this (at least initially) and possibly to make it an error if a static unittest gets mixed in, since it would only be mixed in when the template was instantiated. But even if we did something simple and made it so that static unittest blocks couldn't legally go inside of a version block, debug block, etc. and that all attributes on it had to be applied to it directly in order to have any effect, it would still be an improvement over what we have now. I would hope though that we could do something similar to what we do with ddoc comments inside of a template and that the rules that that uses would work for this too. But I don't know exactly what those rules are, so I don't know how well that will work. - Jonathan M Davis
Sep 27 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 15:06:28 UTC, Jonathan M Davis 
wrote:
 On Sunday, 27 September 2015 at 10:32:00 UTC, Kenji Hara wrote:
 2. Currently the members of template won't be semantically 
 analyzed until it's instantiated. So, when the `static 
 unittest` is enclosed in other blocks, how it works?
[snip] Another possibility is that because a static unittest block only matters when the module that it's in is compiled - and only when compiled with -unittest - we could make it so that a template has full semantic analysis done on it (or at least whatever is necessary to compile static unittest blocks) when it's the module that it's in which is being compiled (and -unittest is used), whereas when the module is merely imported, or -unittest is not used, only what's done currently gets done, and static unittest blocks can be ignored when the template is instantiated or -unittest is not used. And doing that would even work with mixins. It still wouldn't work to have static unittest blocks inside of static if blocks or mixins that required any template arguments, but the rest should work as long as the semantic analysis is done when the module that the template is in is being compiled with -unittest, and the static unittest blocks inside of code that can't be semantically analyzed without instantiating the template can be errors when the template gets instantiated (since that would be better than having them silently do nothing). That would increase compilation times for templates when compiling the module that they're in with -unittest, but it would only be when compiling that module with -untitest and not for anyone importing the module, so I think that the additional cost would be worth it - particularly in light of how it harms maintenance to have to put all of the unittest blocks outside of the template like we do now. - Jonathan M Davis
Sep 27 2015
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Sunday, 27 September 2015 at 05:01:48 UTC, Jonathan M Davis 
wrote:
 This DIP provides a way to handle unittest blocks inside of 
 templates which works with ddoc without compiling the unittest 
 blocks into each instantiation.
I understand that this DIP solves a genuine problem, so understand that this bike-sheding comment is not a dismissal of the idea. I think it's a bad idea to further confuse what static means, as it currently has no less than five different distinct uses. * static ifs * static array construction * static assert * static/shared static this * static functions/data Maybe a new keyword or some attribute would make this clearer.
Sep 27 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 14:10:06 UTC, Jack Stouffer wrote:
 On Sunday, 27 September 2015 at 05:01:48 UTC, Jonathan M Davis 
 wrote:
 This DIP provides a way to handle unittest blocks inside of 
 templates which works with ddoc without compiling the unittest 
 blocks into each instantiation.
I understand that this DIP solves a genuine problem, so understand that this bike-sheding comment is not a dismissal of the idea. I think it's a bad idea to further confuse what static means, as it currently has no less than five different distinct uses. * static ifs * static array construction * static assert * static/shared static this * static functions/data Maybe a new keyword or some attribute would make this clearer.
static in this case is consistent with how static is used on constructors or any symbol at the class or struct level where it makes it so that the symbol is for the class or struct as a whole rather than per instance. For static unittest blocks, it makes it so that the unittest is for the template as a whole and not for a particular instantiation. So, this definitely in line with how static works already, and I think that it would be much worse to add a new keyword or attribute for this. static is the obvious choice IMHO. - Jonathan M Davis
Sep 27 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-09-27 07:01, Jonathan M Davis wrote:
 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.

 http://wiki.dlang.org/DIP82
I assume you won't have access to the template parameters? -- /Jacob Carlborg
Sep 28 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 28 September 2015 at 09:06:52 UTC, Jacob Carlborg 
wrote:
 On 2015-09-27 07:01, Jonathan M Davis wrote:
 This DIP provides a way to handle unittest blocks inside of 
 templates
 which works with ddoc without compiling the unittest blocks 
 into each
 instantiation.

 http://wiki.dlang.org/DIP82
I assume you won't have access to the template parameters?
No. It wouldn't make sense to, since it's not an instantiation of the template and thus there are no template arguments. Basically, aside from ddoc, this should be the same as if the unittest block were outside of the template, but by having a unittest block inside of the template, it can be immediately after the function that it's testing, allowing it to be used for generating the examples for ddoc as well as making it easier to find and maintain the tests that go with each function. It would be kind of like how with a static function in a class/struct, you don't have access to the this pointer/reference, since you're not dealing with an actual instance. Only in this case, it's the template arguments which you wouldn't have access to, not the this pointer/reference, because you're not dealing with an actual instantiation of the template. - Jonathan M Davis
Sep 28 2015
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Sep 27, 2015 at 05:01:45AM +0000, Jonathan M Davis via Digitalmars-d
wrote:
 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.
 
 http://wiki.dlang.org/DIP82
Thanks, Jonathan, for writing this up. I have felt a need for this for a long time now, both in Phobos and in my own code. Overall, I'm very much in favor of this proposal, particularly as it lets us make use of ddoc'd unittests within templated types without the associated template bloat. This is quite important for Phobos as otherwise, we either risk code examples in the docs bit-rotting (user annoyance when examples don't compile, etc.), or user code compiled with -unittest unnecessarily paying the penalty of multiple identical unittests just because they use Phobos. One minor issue with the DIP as currently written, though: it should be stipulated that inside a static unittest block, it is illegal to reference template arguments to the enclosing template block. Otherwise, you may run into pathological cases where it's not clear what the correct semantics should be: template MyTmpl(T) { /// void method() { ... } /// static unittest { // Unclear what should be the type of t in the // single instance of this unittest: T t; } } // (... since MyTmpl likely has multiple different // instantiations:) alias T1 = MyTmpl!int; alias T2 = MyTmpl!string; alias T3 = MyTmpl!float; T -- 2+2=4. 2*2=4. 2^2=4. Therefore, +, *, and ^ are the same operation.
Oct 05 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 5 October 2015 at 19:32:00 UTC, H. S. Teoh wrote:
 One minor issue with the DIP as currently written, though: it 
 should be stipulated that inside a static unittest block, it is 
 illegal to reference template arguments to the enclosing 
 template block. Otherwise, you may run into pathological cases 
 where it's not clear what the correct semantics should be:
Given how the rest of the DIP is written, I don't see how it would even be possible for it to be legal to reference any template arguments. That's implied by the other semantics involved. But I should update the DIP to make that explicit. - Jonathan M Davis
Oct 05 2015