www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP82: static unittest blocks

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
This DIP provides a way to handle unittest blocks inside of 
templates which works with ddoc without compiling the unittest 
blocks into each instantiation.

http://wiki.dlang.org/DIP82


At present, we really can't put unittest blocks - including 
ddoc-ed unittest blocks - inside of templated types, which has 
been a problem for Phobos and came up again just the other day 
when someone was helpful and went and tried to turn the examples 
in std.datetime's *Interval types in ddoc-ed unit tests, but we 
had to reject the changes, because it would have resulted in 
those tests being compiled into every instantiation of those 
templated types, polluting the code of anyone who uses them as 
well as making the Phobos unit tests take longer to run for no 
extra benefit whatsoever.

So, I wrote up this DIP in the hopes of solving the problem in a 
clean and simple manner which fits in well with the existing 
language features.

- Jonathan M Davis
Sep 26 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-09-27 07:01, Jonathan M Davis wrote:

 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.

 http://wiki.dlang.org/DIP82


How would this work in the compiler? Currently it only lexes and parses 
uninstantiated templates. I think that Andrei also wants to make the 
lexing and parsing lazy.

-- 
/Jacob Carlborg
Sep 27 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 09:24:29 UTC, Jacob Carlborg 
wrote:

 On 2015-09-27 07:01, Jonathan M Davis wrote:

 This DIP provides a way to handle unittest blocks inside of 
 templates
 which works with ddoc without compiling the unittest blocks 
 into each
 instantiation.

 http://wiki.dlang.org/DIP82


 How would this work in the compiler? Currently it only lexes 
 and parses uninstantiated templates. I think that Andrei also 
 wants to make the lexing and parsing lazy.


I don't know the details of how this would be implemented in the 
compiler, but clearly what that the compiler currently does is 
enough to generate documentation from the ddoc comments inside of 
templates, and I wouldn't think that detecting a static unittest 
would be much more complicated than that. It can then do full 
analysis of the static unittest if it's the module with the 
template that's being compiled, and it's being compiled with 
-unittest, and otherwise, it can pretty much ignore it (or at 
least, ignore it as much as any other unittest block gets ignored 
when -unittest isn't used).

But we have a real need here if we want to be able to actually 
have unit tests inside of templated types next to what they're 
unit testing. So, I would certainly hope that it's reasonable to 
implement something like this without complicating the compiler 
much. Certainly, the fact that ddoc works inside of templates 
makes it seem like this should be feasible.

- Jonathan M Davis
Sep 27 2015
prev sibling next sibling parent reply Kenji Hara via Digitalmars-d <digitalmars-d puremagic.com> writes:
2015-09-27 14:01 GMT+09:00 Jonathan M Davis via Digitalmars-d <
digitalmars-d puremagic.com>:


 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.

 http://wiki.dlang.org/DIP82


 At present, we really can't put unittest blocks - including ddoc-ed
 unittest blocks - inside of templated types, which has been a problem for
 Phobos and came up again just the other day when someone was helpful and
 went and tried to turn the examples in std.datetime's *Interval types in
 ddoc-ed unit tests, but we had to reject the changes, because it would have
 resulted in those tests being compiled into every instantiation of those
 templated types, polluting the code of anyone who uses them as well as
 making the Phobos unit tests take longer to run for no extra benefit
 whatsoever.

 So, I wrote up this DIP in the hopes of solving the problem in a clean and
 simple manner which fits in well with the existing language features.



Interesting proposal.

Quick questions:

1. I think the token `static` should be placed at the immediate front of
`unittest`

    static unittest { ... }
    static nothrow unittest { ... }   // NG

It's consistent with the behavior of `static this()` family.

2. Currently the members of template won't be semantically analyzed until
it's instantiated. So, when the `static unittest` is enclosed in other
blocks, how it works?

    template Foo(T) {
        // inside conditional compilation blocks:
        static if (is(T == int)) {
            static unittest { ... }
        }
        version (UserDefinedVersion) {
            static unittest { ... }
        }
        debug (UserDefinedDebugId) {
            static unittest { ... }
        }

        // inside block/labeled attributes
        nothrow {
            static unittest { ... }
        }
         safe:
            static unittest { ... }

        mixin("static unittest { ... }");
        mixin(makeStaticUnittest());
    }

    string makeStaticUnittest() { return "static unittest { ... }"; }

Kenji Hara
Sep 27 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 10:32:00 UTC, Kenji Hara wrote:

 2015-09-27 14:01 GMT+09:00 Jonathan M Davis via Digitalmars-d < 
 digitalmars-d puremagic.com>:


 This DIP provides a way to handle unittest blocks inside of 
 templates which works with ddoc without compiling the unittest 
 blocks into each instantiation.

 http://wiki.dlang.org/DIP82


 At present, we really can't put unittest blocks - including 
 ddoc-ed unittest blocks - inside of templated types, which has 
 been a problem for Phobos and came up again just the other day 
 when someone was helpful and went and tried to turn the 
 examples in std.datetime's *Interval types in ddoc-ed unit 
 tests, but we had to reject the changes, because it would have 
 resulted in those tests being compiled into every 
 instantiation of those templated types, polluting the code of 
 anyone who uses them as well as making the Phobos unit tests 
 take longer to run for no extra benefit whatsoever.

 So, I wrote up this DIP in the hopes of solving the problem in 
 a clean and simple manner which fits in well with the existing 
 language features.



 Interesting proposal.

 Quick questions:

 1. I think the token `static` should be placed at the immediate 
 front of `unittest`

     static unittest { ... }
     static nothrow unittest { ... }   // NG

 It's consistent with the behavior of `static this()` family.


I confess that I've always found it to be inconsistent that 
static constructors required that the static be immediately 
before rather than being allowed before or after and among other 
attributes like would occur with any other function. But if we're 
going to retain that inconsistency with static constructors, it 
doesn't really make things much worse to do the same with 
unnittest blocks.


 2. Currently the members of template won't be semantically 
 analyzed until it's instantiated. So, when the `static 
 unittest` is enclosed in other blocks, how it works?

     template Foo(T) {
         // inside conditional compilation blocks:
         static if (is(T == int)) {
             static unittest { ... }
         }
         version (UserDefinedVersion) {
             static unittest { ... }
         }
         debug (UserDefinedDebugId) {
             static unittest { ... }
         }

         // inside block/labeled attributes
         nothrow {
             static unittest { ... }
         }
          safe:
             static unittest { ... }

         mixin("static unittest { ... }");
         mixin(makeStaticUnittest());
     }

     string makeStaticUnittest() { return "static unittest { ... 
 }"; }


Well, my first question would be what happens with a ddoc comment 
on a symbol within a template? It works to put ddoc comments on 
symbols inside of templates, so clearly we have to have rules of 
some kind which deal with each of the cases that you have here, 
though I'm not as familiar as I probably should be with which of 
those results in a symbol having its ddoc comment in the 
documentation and which doesn't.

Regardles, it does indeed get a bit interesting. Ideally, I would 
think that all of them save for the static if and mixins would 
act exactly like they would outside of a template, since nothing 
about this is specific to a template instantiation, but if no 
semantic analysis is currently done on a template until it's 
instantiated, then supporting a static unittest inside of a 
version block, debug block, a block for an attribute, or after an 
attribute used like a label would mean that at least some amount 
of semantic analysis would have to occur that doesn't necessarily 
happen now (though again, I wonder how that works with ddoc 
comments). Maybe no semantic analysis is done if no static 
unittest is found inside of them, and if one or more static 
unittest is found inside of them, the minimal amount of semantic 
analysis required to determine whether they get compiled in is 
done?

As for the static if, I don't see how it can work if it depends 
on any template arguments. I would think that it would work 
easily enough like the version blocks and debug blocks and 
whatnot if the condition did not depend on the template 
arguments, but I think that it's clear that if the condition 
depends on the template arguments at all, then any static 
unittest blocks in there aren't going to work and probably should 
be considered errors (even if it's just when the template is 
instantiated and the static if's condition is true).

Now, as to the mixins... It would be nice if they worked, but I 
have a hard time believing that it's reasonable to make them 
work. We would like to be able to have mixins work with ddoc 
comments, since without that, we can't put documentation on code 
that's mixed in. So, if we ever implement that, then it might be 
reasonable to do the same with static unittest blocks. But other 
than that, the result would be that every mixin inside of a 
template would have to be generated just to see whether it had a 
static unittest in it, which is kind of ugly. If the argument to 
mixin required a template argument to generate it, then it could 
clearly be skipped, but if not, then we'd have to generate it 
just to check, which would increase the compilation times for any 
template which did that, which wouldn't exactly be pretty. It 
probably wouldn't be a big deal in general, since in most cases, 
if you're going to mix in code like that, you're probably going 
to want to do it based on one or more template arguments, in 
which case, the compiler would be able to see that it didn't need 
to generate the mixin. Still, it would probably be better to just 
not support mixins like this (at least initially) and possibly to 
make it an error if a static unittest gets mixed in, since it 
would only be mixed in when the template was instantiated.

But even if we did something simple and made it so that static 
unittest blocks couldn't legally go inside of a version block, 
debug block, etc. and that all attributes on it had to be applied 
to it directly in order to have any effect, it would still be an 
improvement over what we have now. I would hope though that we 
could do something similar to what we do with ddoc comments 
inside of a template and that the rules that that uses would work 
for this too. But I don't know exactly what those rules are, so I 
don't know how well that will work.

- Jonathan M Davis
Sep 27 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 15:06:28 UTC, Jonathan M Davis 
wrote:

 On Sunday, 27 September 2015 at 10:32:00 UTC, Kenji Hara wrote:



 2. Currently the members of template won't be semantically 
 analyzed until it's instantiated. So, when the `static 
 unittest` is enclosed in other blocks, how it works?




[snip]

Another possibility is that because a static unittest block only 
matters when the module that it's in is compiled - and only when 
compiled with -unittest - we could make it so that a template has 
full semantic analysis done on it (or at least whatever is 
necessary to compile static unittest blocks) when it's the module 
that it's in which is being compiled (and -unittest is used), 
whereas when the module is merely imported, or -unittest is not 
used, only what's done currently gets done, and static unittest 
blocks can be ignored when the template is instantiated or 
-unittest is not used. And doing that would even work with mixins.

It still wouldn't work to have static unittest blocks inside of 
static if blocks or mixins that required any template arguments, 
but the rest should work as long as the semantic analysis is done 
when the module that the template is in is being compiled with 
-unittest, and the static unittest blocks inside of code that 
can't be semantically analyzed without instantiating the template 
can be errors when the template gets instantiated (since that 
would be better than having them silently do nothing).

That would increase compilation times for templates when 
compiling the module that they're in with -unittest, but it would 
only be when compiling that module with -untitest and not for 
anyone importing the module, so I think that the additional cost 
would be worth it - particularly in light of how it harms 
maintenance to have to put all of the unittest blocks outside of 
the template like we do now.

- Jonathan M Davis
Sep 27 2015
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Sunday, 27 September 2015 at 05:01:48 UTC, Jonathan M Davis 
wrote:

 This DIP provides a way to handle unittest blocks inside of 
 templates which works with ddoc without compiling the unittest 
 blocks into each instantiation.


I understand that this DIP solves a genuine problem, so 
understand that this bike-sheding comment is not a dismissal of 
the idea.

I think it's a bad idea to further confuse what static means, as 
it currently has no less than five different distinct uses.

* static ifs
* static array construction
* static assert
* static/shared static this
* static functions/data

Maybe a new keyword or some  attribute would make this clearer.
Sep 27 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 27 September 2015 at 14:10:06 UTC, Jack Stouffer wrote:

 On Sunday, 27 September 2015 at 05:01:48 UTC, Jonathan M Davis 
 wrote:

 This DIP provides a way to handle unittest blocks inside of 
 templates which works with ddoc without compiling the unittest 
 blocks into each instantiation.


 I understand that this DIP solves a genuine problem, so 
 understand that this bike-sheding comment is not a dismissal of 
 the idea.

 I think it's a bad idea to further confuse what static means, 
 as it currently has no less than five different distinct uses.

 * static ifs
 * static array construction
 * static assert
 * static/shared static this
 * static functions/data

 Maybe a new keyword or some  attribute would make this clearer.


static in this case is consistent with how static is used on 
constructors or any symbol at the class or struct level where it 
makes it so that the symbol is for the class or struct as a whole 
rather than per instance. For static unittest blocks, it makes it 
so that the unittest is for the template as a whole and not for a 
particular instantiation. So, this definitely in line with how 
static works already, and I think that it would be much worse to 
add a new keyword or attribute for this. static is the obvious 
choice IMHO.

- Jonathan M Davis
Sep 27 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-09-27 07:01, Jonathan M Davis wrote:

 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.

 http://wiki.dlang.org/DIP82


I assume you won't have access to the template parameters?

-- 
/Jacob Carlborg
Sep 28 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 28 September 2015 at 09:06:52 UTC, Jacob Carlborg 
wrote:

 On 2015-09-27 07:01, Jonathan M Davis wrote:

 This DIP provides a way to handle unittest blocks inside of 
 templates
 which works with ddoc without compiling the unittest blocks 
 into each
 instantiation.

 http://wiki.dlang.org/DIP82


 I assume you won't have access to the template parameters?


No. It wouldn't make sense to, since it's not an instantiation of 
the template and thus there are no template arguments. Basically, 
aside from ddoc, this should be the same as if the unittest block 
were outside of the template, but by having a unittest block 
inside of the template, it can be immediately after the function 
that it's testing, allowing it to be used for generating the 
examples for ddoc as well as making it easier to find and 
maintain the tests that go with each function.

It would be kind of like how with a static function in a 
class/struct, you don't have access to the this 
pointer/reference, since you're not dealing with an actual 
instance. Only in this case, it's the template arguments which 
you wouldn't have access to, not the this pointer/reference, 
because you're not dealing with an actual instantiation of the 
template.

- Jonathan M Davis
Sep 28 2015
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Sep 27, 2015 at 05:01:45AM +0000, Jonathan M Davis via Digitalmars-d
wrote:

 This DIP provides a way to handle unittest blocks inside of templates
 which works with ddoc without compiling the unittest blocks into each
 instantiation.
 
 http://wiki.dlang.org/DIP82


Thanks, Jonathan, for writing this up.  I have felt a need for this for
a long time now, both in Phobos and in my own code.  Overall, I'm very
much in favor of this proposal, particularly as it lets us make use of
ddoc'd unittests within templated types without the associated template
bloat.  This is quite important for Phobos as otherwise, we either risk
code examples in the docs bit-rotting (user annoyance when examples
don't compile, etc.), or user code compiled with -unittest unnecessarily
paying the penalty of multiple identical unittests just because they use
Phobos.

One minor issue with the DIP as currently written, though: it should be
stipulated that inside a static unittest block, it is illegal to
reference template arguments to the enclosing template block.
Otherwise, you may run into pathological cases where it's not clear what
the correct semantics should be:

	template MyTmpl(T) {
		///
		void method() { ... }

		///
		static unittest {
			// Unclear what should be the type of t in the
			// single instance of this unittest:
			T t;
		}
	}

	// (... since MyTmpl likely has multiple different
	// instantiations:)
	alias T1 = MyTmpl!int;
	alias T2 = MyTmpl!string;
	alias T3 = MyTmpl!float;


T

-- 
2+2=4. 2*2=4. 2^2=4. Therefore, +, *, and ^ are the same operation.
Oct 05 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 5 October 2015 at 19:32:00 UTC, H. S. Teoh wrote:

 One minor issue with the DIP as currently written, though: it 
 should be stipulated that inside a static unittest block, it is 
 illegal to reference template arguments to the enclosing 
 template block. Otherwise, you may run into pathological cases 
 where it's not clear what the correct semantics should be:


Given how the rest of the DIP is written, I don't see how it 
would even be possible for it to be legal to reference any 
template arguments. That's implied by the other semantics 
involved. But I should update the DIP to make that explicit.

- Jonathan M Davis
Oct 05 2015