www.digitalmars.com         C & C++   DMDScript  

D - Assertions, unit testing

reply Mike Swieton <mike swieton.net> writes:
I really like the built in unit testing in D, but I think that there are
points that could be improved upon.

In no particular order:

If assert were a method, then I could override it and extend it. This would
allow us to add, easily, things like assertEquals and the usual crew there,
and also custom ones, such as (for example) assertConnected, or better error
reporting or improved messages.

I see no way to do this now: I could do things like wrap assert with another
method, but then the wrong file/line number will be reported.

Anotehr, and rather minor, gripe is this: there is no way to really name
tests. I know I can just have a name in a comment before the block, but that
really isn't the same thing. Nitpicking, really, but I feel it ought to be
there.

Basically, I think D's unit tests are not powerful enough. If it were in a
library, I'd say, OK, write a new library, but as they are part of the
standard, and one of the defining features of the language, I think it's
important to do it well. Things like more asserts, with messages are
significant. Another large omission is the setup/teardown methods. Also, there
is no way at all, that I can see, to hook this into a reporting framework.

I think its important that these things be there, even though I think it will
necessitate some language changes. Thoughts on this?

Mike Swieton
__
...Unix, MS-DOS, and Windows NT (also known as the Good, the Bad, and the
Ugly)
	- Matt Welsh
Mar 23 2004
next sibling parent reply Manfred Nowak <svv1999 hotmail.com> writes:
Mike Swieton wrote:

[...]

 I could do things like wrap assert with another
 method, but then the wrong file/line number will be reported.


[...]

To obtain filename and line number à la

  myassert( 0, std.compiler.currLine)

would suffice.

So long!
Mar 23 2004
parent reply Mike Swieton <mike swieton.net> writes:
On Wed, 24 Mar 2004 08:53:23 +0100, Manfred Nowak wrote:


 Mike Swieton wrote:



 To obtain filename and line number à la
 
   myassert( 0, std.compiler.currLine)
 
 would suffice.


Did not know that, that's helpful. But it still isn't quite there. Here's the
thing: I don't want to type std.compiler.currLine each time. And D doesn't
have macros, does it? In C I'd just macro-out the real assert.

Mike Swieton
__
I've developed a new philosophy. I only dread one day at a time.
	- Charlie Brown
Mar 24 2004
parent Manfred Nowak <svv1999 hotmail.com> writes:
Mike Swieton wrote:

[...]

 But it still isn't quite there.


Right. It is a suggestion.

[...]

 I don't want to type


[...]

Use a macro editor.


 And D doesn't have macros, does it?


Right. No macros for good reasons, but aliases and debug statements.


[...]
So long!
Mar 24 2004
prev sibling next sibling parent reply "Ben Hinkle" <bhinkle4 juno.com> writes:
"Mike Swieton" <mike swieton.net> wrote in message
news:pan.2004.03.24.06.10.37.841395 swieton.net...

 I really like the built in unit testing in D, but I think that there are
 points that could be improved upon.

 In no particular order:

 If assert were a method, then I could override it and extend it. This


would

 allow us to add, easily, things like assertEquals and the usual crew


there,

 and also custom ones, such as (for example) assertConnected, or better


error

 reporting or improved messages.

 I see no way to do this now: I could do things like wrap assert with


another

 method, but then the wrong file/line number will be reported.


It's possible to suplement the built-in unittesting/assert with custom
testing
tools. For instance make a subclass of object that all your classes subclass
that have the methods you want and then call those instead of assert.
If it is generic enough and robust enough and useful enough it could make
it into phobos or something like that.


 Anotehr, and rather minor, gripe is this: there is no way to really name
 tests. I know I can just have a name in a comment before the block, but


that

 really isn't the same thing. Nitpicking, really, but I feel it ought to be
 there.


I haven't tried it and maybe it already exists somewhere but is it possible
to write an "assertStr" that throws a custom object? See the file
std.asserterror.d for how the built-in assert is implemented.


 Basically, I think D's unit tests are not powerful enough. If it were in a
 library, I'd say, OK, write a new library, but as they are part of the
 standard, and one of the defining features of the language, I think it's
 important to do it well. Things like more asserts, with messages are
 significant. Another large omission is the setup/teardown methods. Also,


there

 is no way at all, that I can see, to hook this into a reporting framework.


I assume Walter wanted to build in basic unit testing and leave "full
fledged"
unit testing to a library (which doesn't exist yet). Since there are endless
variations and features to add to testing infrastructure it would be
impossible
to get it totally "right" in the core language.


 I think its important that these things be there, even though I think it


will

 necessitate some language changes. Thoughts on this?

 Mike Swieton
 __
 ...Unix, MS-DOS, and Windows NT (also known as the Good, the Bad, and the
 Ugly)
 - Matt Welsh
Mar 24 2004
parent reply Mike Swieton <mike swieton.net> writes:
On Wed, 24 Mar 2004 14:57:44 -0500, Ben Hinkle wrote:

 I assume Walter wanted to build in basic unit testing and leave "full
 fledged" unit testing to a library (which doesn't exist yet). Since there
 are endless variations and features to add to testing infrastructure it
 would be impossible to get it totally "right" in the core language.


I think that that is the right way to do it. What I am trying to say here is
that I don't think that what is exposed is very extendable, for example what I
mentioned about assert not being a real function (generated at compile-time as
far as I can tell).

What I'm asking is this: is it possible to add features to the unit testing
framework, such as reporting, suites, messages on asserts, other kinds of
asserts, and so on, and still use the built-in unit test framework? These
other features could be a seperate library (and that's where they belong, in
fact). But: can it be done on top of the built-in one?

It's been suggested that I use a macro editor (external tool; the
toolkit/language/framework/whatever should do that for me), or that I write a
class with custom asserts and inherit from it (ditto), and so on. But this is
making the user implement the test framework, when a framework should already
exist. I'm not trying to argue that the language be changed; I'd be delighted
to find out that everything I am talking about can be built on top of the
existing facilities.

But it can't really be said that D has unit testing, if I have to throw out
everything it provides if I want to get half the features that every other
framework for any other language has.

Please, correct me if I'm wrong, point me to the features that could be used
to write such a framework. But if the above is correct, it ought to be
addressed.

Mike Swieton
__
Computer Science is the only discipline in which we view adding a new wing to
a building as being maintenance.
	- Jim Horning
Mar 24 2004
parent Ben Hinkle <bhinkle4 juno.com> writes:
 But it can't really be said that D has unit testing, if I have to throw out
 everything it provides if I want to get half the features that every other
 framework for any other language has.


I agree the assert expression is weak if you want to get line
numbers *and* user messages/data. But saying the whole thing
needs to be tossed is stating the case too strongly. I just
played around for about 15 minutes or so and cooked up the
following "test infrastructure" on top of the existing
functionality. The ugly part of the user code is having to
wrap the asserts in try catch blocks and rethrow. But is that
ugly enough to throw out the whole thing? That's a style
question that each person can choose for themselves. I certainly
wouldn't mind seeing a more flexible assert and AssertError.

One way you can help your plight is to propose something.
What would make D extensible enough?

Anyway, the code (compile with -unittest flag):

import std.asserterror;

// put this in some shared module
struct MyTestHarness {
   char [] fMsg; // current test message, if any

   bit myTest(int a, int b, char[] msg) {
      // perform logging, report generation, etc.
      fMsg = msg;
      return a==b;
   }
}

class MyError {
   this(MyTestHarness harness, AssertError err) {
      fErr = err;
      fHarness = harness;
   }

   MyTestHarness fHarness;
   AssertError fErr;
      
   void print()
   {
      // perform logging, report generation, etc.
      printf("%.*s\n", fHarness.fMsg);
      fErr.print();
   }
}


// this is user code
unittest {
   MyTestHarness harness;
   try {
      assert(harness.myTest(1,1,"just checking if 1==1"));
      assert(harness.myTest(1,2,"just checking if 1==2"));
   } catch (AssertError e) {
      throw new MyError(harness,e);
   }
}

int
main()
{
   return 0;
}


Save it to test.d, run it and it prints
Error: just checking if 1==2
AssertError Failure test(37)
Mar 24 2004
prev sibling parent J C Calvarese <jcc7 cox.net> writes:
Mike Swieton wrote:

 I really like the built in unit testing in D, but I think that there are
 points that could be improved upon.
 


[snip...]

 Anotehr, and rather minor, gripe is this: there is no way to really name
 tests. I know I can just have a name in a comment before the block, but that
 really isn't the same thing. Nitpicking, really, but I feel it ought to be
 there.


[snip...]

This probably won't satisfy you, but you can use nested functions to add 
some structure to unittests:

void main()
{
     printf("The actual program.\n");
}


unittest
{
     void strings()
     {
         assert("a" ~ "b" == "ab");
         assert("a" < "b");
     }

     void numbers()
     {
         assert(1 + 1 == 2);
         assert(10 * 10 == 100);
     }

     printf("Start unittests...\n");

     strings();
     numbers();

     printf("Unittests successful.\n");
}


Compile with:
 echo off
dmd nested.d -unittest
nested.exe
pause

-- 
Justin
http://jcc_7.tripod.com/d/
Mar 24 2004