www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - dunit r247

reply Christopher Wright <dhasenan gmail.com> writes:
Hi all,

I've released a new version of dunit, r247. (I have finally decided on
subversion revision numbers as a simple, unambiguous versioning scheme).
It's recommended for immediate use and is certified alpha-quality software.

Documentation and Downloads
---------------------------
Documentation is included in the .zip file and also on the wiki.
Docs: http://dsource.org/projects/dmocks/wiki/DUnit
Download: 
http://dsource.org/projects/dmocks/browser/downloads/dunit.r247.zip?format=raw

Assertions
----------
The major production-ready element of dunit is a greatly-tweaked
assertion system. There are minor API changes; in general, you can convert:
	expect(value).constraint;
to:
	expect(value, constraint);

Additionally, you can combine constraints using | or &:
expect(6, equals(3) | greaterThan(5));

Additionally, it is much more reasonable to add additional messages to
an assertion:
expect(foo(), lessThan(5), "foo is being mean again!");

The assertions system is now extensible; deriving from
dunit.assertions.model.Constraint will allow you to use your constraint
as if it were a regular, builtin constraint.

Parameterized tests
-------------------
A parameterized test is one that uses data generated externally. Dunit
now supports parameterized testing. The builtin parameterization systems
are combinatorial and sequential, with test data generators for random
and sequential integers and floating point values.
Mar 18 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Christopher Wright:

Having other testing frameworks/tools for D is good. There are many kinds of
testing, and the built-in one isn't supposed to implement them all.

Regarding the issues of unit testing with unittest{}, I think the built-in
unittesting has to be improved, to removed some of such issues. I am not
looking for an universal and perfect built-in unittesting, and I think the
built-in unittesting has to be kept simple, but the following things have to be
fixed, maybe Walter will eventually understand why they are important:
- Unittests are not labeled.
- There is no output that specifically indicates that the tests were run.
- A failing test will prevent any other tests from running. 
- There is no indication of which test failed, if any.
Such things are bare-bone functionality for any unit testing system.
And I'd like to add a way to unittest at compile time too, to test types,
templates, etc. (Until few weeks ago I didn't know any way at all to do this,
then someone has given me a hint).


What's the advantage of:
expect(foo(5), equals(3) | greaterThan(5));
Compared to:
expect(foo(5) == 3 | foo(5) > 5);
Or:
auto aux5 = foo(5);
expect(aux5 == 3 | aux5 > 5);
?


If you write Dunit tests in separate modules, while your production code
doesn't include dunit, you cannot test private methods. Dunit encourages the
practice of separating tests and modules.<


For me it's often better to keep tests very close to the things they test. It
helps me spot and fix bugs faster, to avoid jumping across files, and when I
quickly move a block of code (function, class, template, etc) when I reorganize
the code it is less likely for me to lose its tests along the way. I think
tests are a part of a function/class/template, just like its ddocs.

Bye,
bearophile
Mar 19 2009
next sibling parent "Nick Sabalausky" <a a.a> writes:
"bearophile" <bearophileHUGS lycos.com> wrote in message 
news:gptoms$bmm$1 digitalmars.com...

 Christopher Wright:

 Having other testing frameworks/tools for D is good. There are many kinds 
 of testing, and the built-in one isn't supposed to implement them all.

 Regarding the issues of unit testing with unittest{}, I think the built-in 
 unittesting has to be improved, to removed some of such issues. I am not 
 looking for an universal and perfect built-in unittesting, and I think the 
 built-in unittesting has to be kept simple, but the following things have 
 to be fixed, maybe Walter will eventually understand why they are 
 important:
 - Unittests are not labeled.
 - There is no output that specifically indicates that the tests were run.
 - A failing test will prevent any other tests from running.
 - There is no indication of which test failed, if any.
 Such things are bare-bone functionality for any unit testing system.
 And I'd like to add a way to unittest at compile time too, to test types, 
 templates, etc. (Until few weeks ago I didn't know any way at all to do 
 this, then someone has given me a hint).


 What's the advantage of:
 expect(foo(5), equals(3) | greaterThan(5));
 Compared to:
 expect(foo(5) == 3 | foo(5) > 5);
 Or:
 auto aux5 = foo(5);
 expect(aux5 == 3 | aux5 > 5);
 ?


If you write Dunit tests in separate modules, while your production code 
doesn't include dunit, you cannot test private methods. Dunit encourages 
the practice of separating tests and modules.<


 For me it's often better to keep tests very close to the things they test. 
 It helps me spot and fix bugs faster, to avoid jumping across files, and 
 when I quickly move a block of code (function, class, template, etc) when 
 I reorganize the code it is less likely for me to lose its tests along the 
 way. I think tests are a part of a function/class/template, just like its 
 ddocs.


Seconded, with the additional reason that I've found unittests to be an 
incredibly useful form of documentation-by-example. There have been numerous 
occasions where I've read documentation that still left me with questions, 
so I opened up the actual source to see what was going on, saw the 
unittests, and thought "A-ha! So that's how it's used! And I *know* it's 
correct because it's part of the unittest."
Mar 19 2009
prev sibling next sibling parent reply Gide Nwawudu <gide btinternet.com> writes:
On Thu, 19 Mar 2009 11:34:52 -0400, bearophile
<bearophileHUGS lycos.com> wrote:


Christopher Wright:

Having other testing frameworks/tools for D is good. There are many kinds of
testing, and the built-in one isn't supposed to implement them all.

Regarding the issues of unit testing with unittest{}, I think the built-in
unittesting has to be improved, to removed some of such issues. I am not
looking for an universal and perfect built-in unittesting, and I think the
built-in unittesting has to be kept simple, but the following things have to be
fixed, maybe Walter will eventually understand why they are important:
- Unittests are not labeled.
- There is no output that specifically indicates that the tests were run.
- A failing test will prevent any other tests from running. 
- There is no indication of which test failed, if any.
Such things are bare-bone functionality for any unit testing system.
And I'd like to add a way to unittest at compile time too, to test types,
templates, etc. (Until few weeks ago I didn't know any way at all to do this,
then someone has given me a hint).


I think that nestable named unittest would be nice. I'll raise it as
an enhancement request.

unittest ("XML") {
	unittest("elements") {
	    assert(isValidXml("<aaa />"));
	    assert(isValidXml("<aaa/>"));
	    assert(isValidXml("<aaa></aaa>"));
            ...
	}
	unittest("attributes") {
	    assert(isValidXml("<aaa abc="\x\"/>"));
	    assert(isValidXml("<aaa abc=\"x\" def=\"y\"/>"));
            ...
	}
	unittest("encoding") {
	    assert(encode("hello") is "hello");
	    assert(encode("a > b") == "a &gt; b");
	    ...
	}		
}

Gide
Mar 19 2009
parent Gide Nwawudu <gide btinternet.com> writes:
On Thu, 19 Mar 2009 18:26:32 +0000, Gide Nwawudu <gide btinternet.com>
wrote:


On Thu, 19 Mar 2009 11:34:52 -0400, bearophile
<bearophileHUGS lycos.com> wrote:


Christopher Wright:

Having other testing frameworks/tools for D is good. There are many kinds of
testing, and the built-in one isn't supposed to implement them all.

Regarding the issues of unit testing with unittest{}, I think the built-in
unittesting has to be improved, to removed some of such issues. I am not
looking for an universal and perfect built-in unittesting, and I think the
built-in unittesting has to be kept simple, but the following things have to be
fixed, maybe Walter will eventually understand why they are important:
- Unittests are not labeled.
- There is no output that specifically indicates that the tests were run.
- A failing test will prevent any other tests from running. 
- There is no indication of which test failed, if any.
Such things are bare-bone functionality for any unit testing system.
And I'd like to add a way to unittest at compile time too, to test types,
templates, etc. (Until few weeks ago I didn't know any way at all to do this,
then someone has given me a hint).


I think that nestable named unittest would be nice. I'll raise it as
an enhancement request.


Added http://d.puremagic.com/issues/show_bug.cgi?id=2749

Gide
Mar 19 2009
prev sibling next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
bearophile wrote:

 Christopher Wright:
 
 Having other testing frameworks/tools for D is good. There are many kinds of
testing, and the built-in one isn't supposed to implement them all.
 
 Regarding the issues of unit testing with unittest{}, I think the built-in
unittesting has to be improved, to removed some of such issues. I am not
looking for an universal and perfect built-in unittesting, and I think the
built-in unittesting has to be kept simple, but the following things have to be
fixed, maybe Walter will eventually understand why they are important:
 - Unittests are not labeled.
 - There is no output that specifically indicates that the tests were run.
 - A failing test will prevent any other tests from running. 
 - There is no indication of which test failed, if any.
 Such things are bare-bone functionality for any unit testing system.
 And I'd like to add a way to unittest at compile time too, to test types,
templates, etc. (Until few weeks ago I didn't know any way at all to do this,
then someone has given me a hint).


What are you using for this? __traits(compiles) works for d2, to an 
extent, and for d1, is (typeof(expression)). But for templates that have 
to be mixed into some context, that's more tricky.


 What's the advantage of:
 expect(foo(5), equals(3) | greaterThan(5));
 Compared to:
 expect(foo(5) == 3 | foo(5) > 5);


What error message should that give?

The former gives:
Expected: equal to 3 or greater than 5
But was: <whatever value foo(5) returned>

The latter gives:
Assertion error


 If you write Dunit tests in separate modules, while your production code
doesn't include dunit, you cannot test private methods. Dunit encourages the
practice of separating tests and modules.<


 
 For me it's often better to keep tests very close to the things they test. It
helps me spot and fix bugs faster, to avoid jumping across files, and when I
quickly move a block of code (function, class, template, etc) when I reorganize
the code it is less likely for me to lose its tests along the way. I think
tests are a part of a function/class/template, just like its ddocs.


I believe that there is a benefit to keeping the tests close to the 
tested code. I have not noticed a significant lack, however, when using 
junit or nunit.


 Bye,
 bearophile
Mar 19 2009
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Christopher Wright" <dhasenan gmail.com> wrote in message 
news:gpuibc$1nrv$1 digitalmars.com...

 bearophile wrote:

 What's the advantage of:
 expect(foo(5), equals(3) | greaterThan(5));
 Compared to:
 expect(foo(5) == 3 | foo(5) > 5);


 What error message should that give?

 The former gives:
 Expected: equal to 3 or greater than 5
 But was: <whatever value foo(5) returned>

 The latter gives:
 Assertion error


The nice thing about D is how often it lets you, as I prefer to word it, 
"Eat your cake and then still have it."

Attached is the assert-alternative that I use (D1/Tango). It's used like 
this:

---------------------
module nonFatalAssertTest.main;

import semitwist.util.nonFatalAssert;

void main()
{
    FatalizeAsserts();
    // Main program code here
}

unittest
{
    int foo = 2;

    // *REALLY* need a way for a template to automatically get
    // the file/line of instantiation.
    // Improvement to mixin syntax would also be nice
    // Also, my editor doesn't know that backticks indicate a string,
    // so it's still properly highlighted as code :)
    mixin(NonFatalAssert!(__LINE__, __FILE__, `foo == 3 || foo > 5`, "foo is 
bad"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `2 + 2 == 4`, "Basic 
arithmetic"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `false`));
}
---------------------

Output of that is:

---------------------
nonFatalAssertTest\main.d(16): Assert Failure (foo == 3 || foo > 5): foo is 
bad
nonFatalAssertTest\main.d(18): Assert Failure (false)
tango.core.Exception.AssertException semitwist\util\nonFatalAssert.d(62): 2 
Assert Failures
---------------------

Granted, this doesn't currently output foo's actual value, but it probably 
wouldn't be too hard to modify it to do so via std.algorithm-style trickery. 


begin 666 nonFatalAssert.d
M+R\ 4V5M:51W:7-T($QI8G)A<GD-"B\O(%=R:71T96X :6X =&AE($0 <')O



M9'5L92!S96UI='=I<W0N=71I;"YN;VY&871A;$%S<V5R=#L-" T*:6UP;W)T
M('1A;F=O+FEO+E-T9&]U=#L-"FEM<&]R="!T86YG;RYU=&EL+D-O;G9E<G0[



M='!U='1I;F< =&AE(&9I<G-T(&]N92!A;F0 =&AE;B!S=&]P<&EN9RX-"BHO


M<V5R="AL;VYG(&QI;F4L(&-H87);72!F:6QE+"!C:&%R6UT 8V]N9"P 8VAA
M<EM=(&US9STB(BD-"GL-" EC;VYS="!C:&%R6UT 3F]N1F%T86Q!<W-E<G0 
M/0T*"2)[7&XB? T*"2)B;V]L(%].;VY&871A;$%S<V5R=%]R97-U;'0 /2 B

M9V]F?B(L(")^9FEL92YS=')I;F=O9GXB+" B?F-O;F0N<W1R:6YG;V9^(BP 
M(GYM<V<N<W1R:6YG;V9^(BDH7TYO;D9A=&%L07-S97)T7W)E<W5L="D[(GX-


M;&]N9R!L:6YE+"!C:&%R6UT 9FEL92P 8VAA<EM=(&-O;F0L(&-H87);72!M



M:6QU<F4 *'M]*7M](BP-" D)"2  ("  ("  ("  ("  ("!F:6QE+"!L:6YE
M+"!C;VYD+ T*"0D)"0D)("  (&US9ST](B( /R B(B Z("(Z("( ?B!M<V<I


M07-S97)T0V]U;G0H*0T*>PT*"7)E='5R;B!N;VY&871A;$%S<V5R=$-O=6YT
M.PT*?0T*=F]I9"!R97-E=$YO;D9A=&%L07-S97)T0V]U;G0H*0T*>PT*"6YO



M;R$H8VAA<EM=*2AG971.;VY&871A;$%S<V5R=$-O=6YT*"DI('X-" D)"2( 
M07-S97)T($9A:6QU<F4B('X-" D)"2AG971.;VY&871A;$%S<V5R=$-O=6YT
B*"D /3T ,2 _("(B(#H (G,B*0T*"0DI.PT*"7T-"GT-" ``
`
end
Mar 19 2009
parent "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:gpunvp$20u8$1 digitalmars.com...

 Granted, this doesn't currently output foo's actual value, but it probably
 wouldn't be too hard to modify it to do so via std.algorithm-style 
 trickery.


Done <g>. New alternate-assert module is attached.

Sample usage:

--------------
module nonFatalAssertTest.main;

import semitwist.util.nonFatalAssert;

void main()
{
    FatalizeAsserts();
    // Main program code here
}

unittest
{
    int foo = 2;
    char[] bar = "hello";

    // *REALLY* need a way for a template to automatically get
    // the file/line of instantiation.
    // Improvement to mixin syntax would also be nice
    // Also, my editor doesn't know that backticks indicate a string,
    // so it's still properly highlighted as code :)
    mixin(NonFatalAssert!(__LINE__, __FILE__, `foo == 3 || foo > 5`, "foo is 
bad"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `2 + 2 == 4`, "Basic 
arithmetic"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `false`));

    mixin(NonFatalEnsure!(__LINE__, __FILE__, foo, `_ == 3 || _ > 5`, 
"ensure foo failed"));
    mixin(NonFatalEnsure!(__LINE__, __FILE__, foo, `_ > 0`));
    mixin(NonFatalEnsure!(__LINE__, __FILE__, bar, `_ == "hola"`));
}
--------------

Output:

--------------
nonFatalAssertTest\main.d(21): Assert Failed (foo == 3 || foo > 5): foo is 
bad
nonFatalAssertTest\main.d(23): Assert Failed (false)
nonFatalAssertTest\main.d(25): Ensure Failed: ensure foo failed
Value 'foo':
Expected: _ == 3 || _ > 5
Actual: 2
nonFatalAssertTest\main.d(27): Ensure Failed
Value 'bar':
Expected: _ == "hola"
Actual: hello
tango.core.Exception.AssertException semitwist\util\nonFatalAssert.d(86): 4 
Assert Failures
--------------

Sorry if this comes across as thread hijacking, that wasn't my intent. My 
main point is that in D, we don't have to choose between having detailed 
assert messages and having natural syntax for assert's conditional. We can 
have both. 


begin 666 nonFatalAssert.d
M+R\ 4V5M:51W:7-T($QI8G)A<GD-"B\O(%=R:71T96X :6X =&AE($0 <')O



M9'5L92!S96UI='=I<W0N=71I;"YN;VY&871A;$%S<V5R=#L-" T*:6UP;W)T
M('1A;F=O+FEO+E-T9&]U=#L-"FEM<&]R="!T86YG;RYU=&EL+D-O;G9E<G0[



M='!U='1I;F< =&AE(&9I<G-T(&]N92!A;F0 =&AE;B!S=&]P<&EN9RX-"BHO


M<V5R="AL;VYG(&QI;F4L(&-H87);72!F:6QE+"!C:&%R6UT 8V]N9%-T<BP 
M8VAA<EM=(&US9STB(BD-"GL-" EC;VYS="!C:&%R6UT 3F]N1F%T86Q!<W-E
M<G0 /0T*"2)[7&XB? T*"2( ("  8F]O;"!?3F]N1F%T86Q!<W-E<G1?8V]N
M9%)E<W5L=" ](" B?F-O;F13=')^(BD[7&XB? T*"2( ("  7TYO;D9A=&%L
M07-S97)T(2 B?FQI;F4N<W1R:6YG;V9^(BP (GYF:6QE+G-T<FEN9V]F?B(L
M(")^8V]N9%-T<BYS=')I;F=O9GXB+" B?FUS9RYS=')I;F=O9GXB*2A?3F]N
M1F%T86Q!<W-E<G1?8V]N9%)E<W5L="D[7&XB? T*"2)]7&XB.PT*"2\O<')A
M9VUA*&US9RP (DYO;D9A=&%L07-S97)T.B B?DYO;D9A=&%L07-S97)T*3L-
M"GT-" T*8F]O;"!?3F]N1F%T86Q!<W-E<G0H;&]N9R!L:6YE+"!C:&%R6UT 
M9FEL92P 8VAA<EM=(&-O;F13='(L(&-H87);72!M<V<](B(I*&)O;VP 8V]N

M07-S97)T0V]U;G0K*SL-" D)4W1D;W5T+F9O<FUA=&QN*")[?2A[?2DZ($%S
M<V5R="!&86EL960 *'M]*7M](BP-" D)"0D)"69I;&4L(&QI;F4L(&-O;F13


M16YS=7)E*&QO;F< ;&EN92P 8VAA<EM=(&9I;&4L(&%L:6%S('9A;'5E+"!C
M:&%R6UT 8V]N9%-T<BP 8VAA<EM=(&US9STB(BD-"GL-" EC;VYS="!C:&%R
M6UT 3F]N1F%T86Q%;G-U<F4 /0T*"2)[7&XB? T*"2( ("  (GYT>7!E;V8H
M=F%L=64I+G-T<FEN9V]F?B( 7R ](" B?G9A;'5E+G-T<FEN9V]F?B(I.UQN
M(GX-" DB("  (&)O;VP 7TYO;D9A=&%L16YS=7)E7V-O;F1297-U;'0 /2 H
M(GYC;VYD4W1R?B(I.UQN(GX-" DB("  (%].;VY&871A;$5N<W5R92$H(GYL
M:6YE+G-T<FEN9V]F?B(L(")^9FEL92YS=')I;F=O9GXB+"! (GYV86QU92YS
M=')I;F=O9GXB8"P (GYC;VYD4W1R+G-T<FEN9V]F?B(L(")^='EP96]F*'9A
M;'5E*2YS=')I;F=O9GXB+" B?FUS9RYS=')I;F=O9GXB*2A?+"!?3F]N1F%T
M86Q%;G-U<F5?8V]N9%)E<W5L="D[7&XB? T*"2)]7&XB.PT*"2\O<')A9VUA
M*&US9RP (DYO;D9A=&%L16YS=7)E.B B?DYO;D9A=&%L16YS=7)E*3L-"GT-
M" T*8F]O;"!?3F]N1F%T86Q%;G-U<F4H;&]N9R!L:6YE+"!C:&%R6UT 9FEL
M92P 8VAA<EM=('9A;'5E4W1R+"!C:&%R6UT 8V]N9%-T<BP 5"P 8VAA<EM=
M(&US9STB(BDH5"!V86QU95)E<W5L="P 8F]O;"!C;VYD4F5S=6QT*0T*>PT*

M.PT*"0E3=&1O=70N9F]R;6%T;&XH(GM]*'M]*3H 16YS=7)E($9A:6QE9'M]
M7&XB? T*"0D ("  ("  ("  ("  ("  (E9A;'5E("=[?2<Z7&XB? T*"0D)
M"0D)(D5X<&5C=&5D.B![?5QN(GX-" D)"0D)"2)!8W1U86PZ('M](BP-" D)

M"0D)"0EV86QU95-T<BP 8V]N9%-T<BP =F%L=65297-U;'0I.PT*"7T-" D-
M" ER971U<FX 8V]N9%)E<W5L=#L-"GT-"G!R:79A=&4 =6EN="!N;VY&871A

M*0T*>PT*"7)E='5R;B!N;VY&871A;$%S<V5R=$-O=6YT.PT*?0T*=F]I9"!R
M97-E=$YO;D9A=&%L07-S97)T0V]U;G0H*0T*>PT*"6YO;D9A=&%L07-S97)T



M971.;VY&871A;$%S<V5R=$-O=6YT*"DI('X-" D)"2( 07-S97)T($9A:6QU
M<F4B('X-" D)"2AG971.;VY&871A;$%S<V5R=$-O=6YT*"D /3T ,2 _("(B
6(#H (G,B*0T*"0DI.PT*"7T-"GT-" ``
`
end
Mar 19 2009
prev sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Christopher Wright wrote:

 bearophile wrote:

 What's the advantage of:
 expect(foo(5), equals(3) | greaterThan(5));
 Compared to:
 expect(foo(5) == 3 | foo(5) > 5);


 
 What error message should that give?
 
 The former gives:
 Expected: equal to 3 or greater than 5
 But was: <whatever value foo(5) returned>
 
 The latter gives:
 Assertion error


What about this syntax?

check!("% == 3 || % == 5")(foo(5));

This could display:

Expected: (% == 3 || % == 5) for <value of foo(5)>.

Then you don't have to dick about writing classes when you want to test
something.  Rule one of a good API: get out of my way. :D

  -- Daniel
Mar 19 2009
parent Christopher Wright <dhasenan gmail.com> writes:
Daniel Keep wrote:

 What about this syntax?
 
 check!("% == 3 || % == 5")(foo(5));
 
 This could display:
 
 Expected: (% == 3 || % == 5) for <value of foo(5)>.
 
 Then you don't have to dick about writing classes when you want to test
 something.  Rule one of a good API: get out of my way. :D
 
   -- Daniel


On the face of it, that seems reasonable. I'm not certain what I will do 
in that regard.
Mar 19 2009
prev sibling parent Robert Fraser <fraserofthenight gmail.com> writes:
bearophile wrote:

 For me it's often better to keep tests very close to the things they test. It
helps me spot and fix bugs faster, to avoid jumping across files, and when I
quickly move a block of code (function, class, template, etc) when I reorganize
the code it is less likely for me to lose its tests along the way. I think
tests are a part of a function/class/template, just like its ddocs.


Well, you can always put the Dunit tests in the same module; this was 
just a suggestion.
Mar 19 2009