www.digitalmars.com         C & C++   DMDScript  

D - Unittests

reply Dario <Dario_member pathlink.com> writes:
I would be nice that unittests were automatically launched by the compiler as
soon as it has finished the code-generation phase and that unittests weren't
really included in the object file at last.
This way you'll know if a unittest failed as soon as you compile the code, and
the -unittest option would be active by default when -release is not enabled.
What do you think?
-Dario
Jul 12 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Dario" <Dario_member pathlink.com> wrote in message
news:bepjq1$1q68$1 digitaldaemon.com...

 I would be nice that unittests were automatically launched by the compiler


as

 soon as it has finished the code-generation phase and that unittests


weren't

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the code,


and

 the -unittest option would be active by default when -release is not


enabled.

 What do you think?


It's an intriguing idea, but the compiler has no way of executing code in a
.obj file without doing a full link. Doing a full link gets you where you
are today anyway with running unittests <g>.
Jul 13 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:

 
 "Dario" <Dario_member pathlink.com> wrote in message
 news:bepjq1$1q68$1 digitaldaemon.com...

 I would be nice that unittests were automatically launched by the compiler


 as

 soon as it has finished the code-generation phase and that unittests


 weren't

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the code,


 and

 the -unittest option would be active by default when -release is not


 enabled.

 What do you think?


 
 It's an intriguing idea, but the compiler has no way of executing code in a
 .obj file without doing a full link. Doing a full link gets you where you
 are today anyway with running unittests <g>.


I think unittests should:

   - always tell which test(s) exactly have failed.

   - never stop on an error but always run through all tests.
     This is important if you break code while refactoring. You must have
     the complete information on the total effects.
     Typically this is accomplished by forking the process and reporting
     the state of the process back to the main unittest code.

   - should have a visual display to show how many tests have already been 
     run and how many have failed (this is important for refactoring) because
     unittests may run (on more complex) systems up to hours.

   - run only once from an completed exe (an app.exe could create an marker
     file app.uok whose date-time-stamp is tested against app.exe)
   
--
Helmut Leitner    leitner hls.via.at   
Graz, Austria   www.hls-software.com
Jul 13 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F11B6D6.EFEB8CFD chello.at...

 Walter wrote:

 "Dario" <Dario_member pathlink.com> wrote in message
 news:bepjq1$1q68$1 digitaldaemon.com...

 I would be nice that unittests were automatically launched by the






compiler

 as

 soon as it has finished the code-generation phase and that unittests


 weren't

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the






code,

 and

 the -unittest option would be active by default when -release is not


 enabled.

 What do you think?


 It's an intriguing idea, but the compiler has no way of executing code




in a

 .obj file without doing a full link. Doing a full link gets you where




you

 are today anyway with running unittests <g>.


 I think unittests should:

    - always tell which test(s) exactly have failed.


It does already, with the file/line of the failed contract. Of course, you
can put any code you want in the unittest blocks, not just assert()'s.



    - never stop on an error but always run through all tests.
      This is important if you break code while refactoring. You must have
      the complete information on the total effects.
      Typically this is accomplished by forking the process and reporting
      the state of the process back to the main unittest code.


My experience usually is that if one test fails, subsequent tests fail as
well because of cascading errors from the first fault.


    - should have a visual display to show how many tests have already been
      run and how many have failed (this is important for refactoring)


because

      unittests may run (on more complex) systems up to hours.


You can do this by writing such code into the unittest blocks.



    - run only once from an completed exe (an app.exe could create an


marker

      file app.uok whose date-time-stamp is tested against app.exe)


That's an interesting approach. I've used self-patching exe's in the past to
do initializations; I had to stop the practice because anti-virus programs
had fits about it.
Jul 13 2003
next sibling parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:

 I think unittests should:

    - always tell which test(s) exactly have failed.


 
 It does already, with the file/line of the failed contract. Of course, you
 can put any code you want in the unittest blocks, not just assert()'s.


We had an fault during ICFP-contest where it didn't. 
 

    - never stop on an error but always run through all tests.
      This is important if you break code while refactoring. You must have
      the complete information on the total effects.
      Typically this is accomplished by forking the process and reporting
      the state of the process back to the main unittest code.


 
 My experience usually is that if one test fails, subsequent tests fail as
 well because of cascading errors from the first fault.


This may or may not be the case. But that's not important.

Important is that the people that invented unittests and write
books about unittests and advocate test driven development
expect it - I think with good reasons - to work that way.
 

    - should have a visual display to show how many tests have already been
      run and how many have failed (this is important for refactoring)


 because

      unittests may run (on more complex) systems up to hours.


 
 You can do this by writing such code into the unittest blocks.


It should soon become a phobos module.
 
--
Helmut Leitner    leitner hls.via.at   
Graz, Austria   www.hls-software.com
Jul 13 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F1247B1.2788A5B chello.at...

 Walter wrote:

 I think unittests should:
    - always tell which test(s) exactly have failed.


 It does already, with the file/line of the failed contract. Of course,




you

 can put any code you want in the unittest blocks, not just assert()'s.


 We had an fault during ICFP-contest where it didn't.


What happened? It sounds like it could have been a GP fault, not an assert
fail.
Jul 14 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:

 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F1247B1.2788A5B chello.at...

 Walter wrote:

 I think unittests should:
    - always tell which test(s) exactly have failed.


 It does already, with the file/line of the failed contract. Of course,




 you

 can put any code you want in the unittest blocks, not just assert()'s.


 We had an fault during ICFP-contest where it didn't.


 
 What happened? It sounds like it could have been a GP fault, not an assert
 fail.


Yes, something like that, or a DIV/0 fault. I think it was with a sin/cos 
function that faulted in my local sandbox. I got rid of it by updating DMD to
0.67.
I think that Burton Radons knows better, what the real reason was, he seemed
to recognize the problem.

A unittest has to report an error position (and continue) no matter what caused
it.

--
Helmut Leitner    leitner hls.via.at   
Graz, Austria   www.hls-software.com
Jul 14 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F13934D.C12084C8 chello.at...

 Walter wrote:

 What happened? It sounds like it could have been a GP fault, not an




assert

 fail.


 Yes, something like that, or a DIV/0 fault. I think it was with a sin/cos
 function that faulted in my local sandbox. I got rid of it by updating DMD


to 0.67.

 I think that Burton Radons knows better, what the real reason was, he


seemed

 to recognize the problem.

 A unittest has to report an error position (and continue) no matter what


caused it.

The position of a GPF will be revealed by running it under a debugger. You
can write contracts that continue by just using something other than the
built-in assert:

Print message and stop:
    assert(succeeded);

Print message and continue:
    if (!succeeded) printf("test xyz failed\n");
Jul 15 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:

 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F13934D.C12084C8 chello.at...

 Walter wrote:

 What happened? It sounds like it could have been a GP fault, not an




 assert

 fail.


 Yes, something like that, or a DIV/0 fault. I think it was with a sin/cos
 function that faulted in my local sandbox. I got rid of it by updating DMD


 to 0.67.

 I think that Burton Radons knows better, what the real reason was, he


 seemed

 to recognize the problem.

 A unittest has to report an error position (and continue) no matter what


 caused it.
 
 The position of a GPF will be revealed by running it under a debugger. You
 can write contracts that continue by just using something other than the
 built-in assert:
 
 Print message and stop:
     assert(succeeded);
 
 Print message and continue:
     if (!succeeded) printf("test xyz failed\n");


If you can't locate a GPF by a unittest, then it's not a real unittest.
Real unittests fork the program and and GPF won't kill the main process.

--
Helmut Leitner    leitner hls.via.at   
Graz, Austria   www.hls-software.com
Jul 15 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F1472BF.99DE612 chello.at...

 If you can't locate a GPF by a unittest, then it's not a real unittest.
 Real unittests fork the program and and GPF won't kill the main process.


I see we have different ideas on what unit tests should be!
Jul 16 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:

 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F1472BF.99DE612 chello.at...

 If you can't locate a GPF by a unittest, then it's not a real unittest.
 Real unittests fork the program and and GPF won't kill the main process.


 
 I see we have different ideas on what unit tests should be!


You have just implemented - and I can understand that - what's 
easy to implement. At some point you will have to live up to the
expectations of test-oriented people used to unittest frameworks.

Even if you use a free open source C testing framework under 
windows, you
   - catch all errors (including GPFs)
   - run through all tests no matter how many errors
     and what errors occur

I think it would not even be a problem to supply sample code 
how to accomplish that.

But obviously when you say "have different ideas on what unit tests
should be" then the master has spoken his final words...

--
Helmut Leitner    leitner hls.via.at   
Graz, Austria   www.hls-software.com
Jul 16 2003
next sibling parent reply "Matthew Wilson" <matthew stlsoft.org> writes:
Surely if a test fails, then other subsequent tests cannot be guaranteed
their correct pre-conditions (not to mention a stable host) so their results
may be meaningless? A little like relying on later compiler errors when you
should be fixing the first one.



"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F163019.BDDD344E chello.at...

 Walter wrote:

 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F1472BF.99DE612 chello.at...

 If you can't locate a GPF by a unittest, then it's not a real






unittest.

 Real unittests fork the program and and GPF won't kill the main






process.

 I see we have different ideas on what unit tests should be!


 You have just implemented - and I can understand that - what's
 easy to implement. At some point you will have to live up to the
 expectations of test-oriented people used to unittest frameworks.

 Even if you use a free open source C testing framework under
 windows, you
    - catch all errors (including GPFs)
    - run through all tests no matter how many errors
      and what errors occur

 I think it would not even be a problem to supply sample code
 how to accomplish that.

 But obviously when you say "have different ideas on what unit tests
 should be" then the master has spoken his final words...

 --
 Helmut Leitner    leitner hls.via.at
 Graz, Austria   www.hls-software.com
Jul 16 2003
parent reply Helmut Leitner <leitner hls.via.at> writes:
Matthew Wilson wrote:

 
 Surely if a test fails, then other subsequent tests cannot be guaranteed
 their correct pre-conditions (not to mention a stable host) so their results
 may be meaningless? A little like relying on later compiler errors when you
 should be fixing the first one.


Just look at the way unittests are written. 
They typically don't build on each other. 
They are not like the logical flow of a program.

-- 
Helmut Leitner    leitner hls.via.at
Graz, Austria   www.hls-software.com
Jul 17 2003
parent reply "Luna Kid" <lunakid neuropolis.org> writes:
To the "stop or not to stop" debate: would it be
OK adding a compiler (err... linker??) switch like
"do not abort on failed asserts"? (I can imagine
that it may not be as trivial as it sounds, but
I don't know the D implementation of assert().)

Sz.
Jul 18 2003
parent "Walter" <walter digitalmars.com> writes:
"Luna Kid" <lunakid neuropolis.org> wrote in message
news:bf8ae1$18bu$1 digitaldaemon.com...

 To the "stop or not to stop" debate: would it be
 OK adding a compiler (err... linker??) switch like
 "do not abort on failed asserts"? (I can imagine
 that it may not be as trivial as it sounds, but
 I don't know the D implementation of assert().)


You can add an exception handler to catch them and continue. In fact, that's
what the startup code does in order to catch & print any out,
phobos\dmain2.d:

    try
    {
        _moduleCtor();
        _moduleUnitTests();

        for (i = 0; i < argc; i++)
        {
            int len = strlen(argv[i]);
            am[i] = argv[i][0 .. len];
        }

        args = am[0 .. argc];

        result = main(args);
    }

    catch (Object o)
    {
        printf("Error: ");
        o.print();
        result = EXIT_FAILURE;
    }
Jul 20 2003
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F163019.BDDD344E chello.at...

 You have just implemented - and I can understand that - what's
 easy to implement. At some point you will have to live up to the
 expectations of test-oriented people used to unittest frameworks.


I agree that what D provides is basic. However, it is way, *way* ahead of
what any other language provides, which is nothing. In my own use of D unit
tests, it has proven its value time and again.


 Even if you use a free open source C testing framework under
 windows, you
    - catch all errors (including GPFs)
    - run through all tests no matter how many errors
      and what errors occur

 I think it would not even be a problem to supply sample code
 how to accomplish that.

 But obviously when you say "have different ideas on what unit tests
 should be" then the master has spoken his final words...


D doesn't prevent unittests from operating the way you want them too - but
you'd need to write some additional code to implement that.
Jul 17 2003
parent reply Helmut Leitner <leitner hls.via.at> writes:
Walter wrote:

 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F163019.BDDD344E chello.at...

 You have just implemented - and I can understand that - what's
 easy to implement. At some point you will have to live up to the
 expectations of test-oriented people used to unittest frameworks.


 
 I agree that what D provides is basic. However, it is way, *way* ahead of
 what any other language provides, which is nothing. In my own use of D unit
 tests, it has proven its value time and again.
 

 Even if you use a free open source C testing framework under
 windows, you
    - catch all errors (including GPFs)
    - run through all tests no matter how many errors
      and what errors occur

 I think it would not even be a problem to supply sample code
 how to accomplish that.

 But obviously when you say "have different ideas on what unit tests
 should be" then the master has spoken his final words...


 
 D doesn't prevent unittests from operating the way you want them too - but
 you'd need to write some additional code to implement that.


Only if you provide an interface to get at the unittest function units.

Lets say any unittest is like a 
  void unittest_xx();
then there should be an array of pointers in any module compiled for unittests
  array_unitest_modulexyz (pointer, codeline)
and some datastructure (like a linked list or similar)
  list_unittest_arrays (arraypointer, modulename)

There should be a way to compile the unittests without actually 
being called at the program start.

It should be guaranteed that different unittest blocks within one module
are not combined into one functional unit (I don't know if you do).

-- 
Helmut Leitner    leitner hls.via.at
Graz, Austria   www.hls-software.com
Jul 17 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <leitner hls.via.at> wrote in message
news:3F165C52.F2624CD1 hls.via.at...


 D doesn't prevent unittests from operating the way you want them too -




but

 you'd need to write some additional code to implement that.


 Only if you provide an interface to get at the unittest function units.

 Lets say any unittest is like a
   void unittest_xx();
 then there should be an array of pointers in any module compiled for


unittests

   array_unitest_modulexyz (pointer, codeline)
 and some datastructure (like a linked list or similar)
   list_unittest_arrays (arraypointer, modulename)


You can see how they are called in phobos/moduleinit.d. It's equivalent to
as you suggest.


 There should be a way to compile the unittests without actually
 being called at the program start.


You can do this by modifying phobos/moduleinit.d.


 It should be guaranteed that different unittest blocks within one module
 are not combined into one functional unit (I don't know if you do).


Check out this code:

unittest
{
 printf("1\n");
}

unittest
{
 printf("2\n");
}

void main()
{
}

which compiles to:

_DATA   segment
        db      031h,00ah,000h,032h,00ah,000h,074h,065h
        db      073h,074h,000h,000h
__ModuleInfo_test:
        db      000h,000h,000h,000h
        db      000h,000h,000h,000h,004h,000h,000h,000h
        dd      offset FLAT:_DATA[6]
        db      000h,000h,000h,000h
        db      000h,000h,000h,000h,000h,000h,000h,000h
        db      000h,000h,000h,000h,000h,000h,000h,000h
        db      000h,000h,000h,000h,000h,000h,000h,000h
        dd      offset FLAT:__modtest_test
_DATA   ends
CONST   segment
CONST   ends
_BSS    segment
_BSS    ends
FMB     segment
FMB     ends
FM      segment
        dd      offset FLAT:__ModuleInfo_test
FM      ends
FME     segment
FME     ends
_Dtest_unittest0_FZv    comdat
        assume  CS:_Dtest_unittest0_FZv
L0:             push    offset FLAT:_DATA
                call    near ptr _printf
                add     ESP,4
                ret
_Dtest_unittest0_FZv    ends
_Dtest_unittest1_FZv    comdat
        assume  CS:_Dtest_unittest1_FZv
L0:             push    offset FLAT:_DATA[3]
                call    near ptr _printf
                add     ESP,4
                ret
_Dtest_unittest1_FZv    ends
__Dmain comdat
        assume  CS:__Dmain
                ret
__Dmain ends
__modtest_test  comdat
        assume  CS:__modtest_test
L0:             call    near ptr _Dtest_unittest0_FZv
                call    near ptr _Dtest_unittest1_FZv
                ret
__modtest_test  ends
        end
Jul 20 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Thank you for these hints. They may or may not be sufficient.

The problem is that unittests are implemented on top of almost
any programming language. Most of the time they are separate
testing frameworks.

The D claim to support unittests would make a lot of sense 
if - and only if - the support is out of the box. Otherwise 
it means nothing.

I'll try to build on the hooks you gave me, but it will only
make sense when reintegrated into D.

Walter wrote:

 
 "Helmut Leitner" <leitner hls.via.at> wrote in message
 news:3F165C52.F2624CD1 hls.via.at...
 

 D doesn't prevent unittests from operating the way you want them too -




 but

 you'd need to write some additional code to implement that.


 Only if you provide an interface to get at the unittest function units.

 Lets say any unittest is like a
   void unittest_xx();
 then there should be an array of pointers in any module compiled for


 unittests

   array_unitest_modulexyz (pointer, codeline)
 and some datastructure (like a linked list or similar)
   list_unittest_arrays (arraypointer, modulename)


 
 You can see how they are called in phobos/moduleinit.d. It's equivalent to
 as you suggest.
 

 There should be a way to compile the unittests without actually
 being called at the program start.


 
 You can do this by modifying phobos/moduleinit.d.
 

 It should be guaranteed that different unittest blocks within one module
 are not combined into one functional unit (I don't know if you do).


 
 Check out this code:
 
 unittest
 {
  printf("1\n");
 }
 
 unittest
 {
  printf("2\n");
 }
 
 void main()
 {
 }
 
 which compiles to:
 
 _DATA   segment
         db      031h,00ah,000h,032h,00ah,000h,074h,065h
         db      073h,074h,000h,000h
 __ModuleInfo_test:
         db      000h,000h,000h,000h
         db      000h,000h,000h,000h,004h,000h,000h,000h
         dd      offset FLAT:_DATA[6]
         db      000h,000h,000h,000h
         db      000h,000h,000h,000h,000h,000h,000h,000h
         db      000h,000h,000h,000h,000h,000h,000h,000h
         db      000h,000h,000h,000h,000h,000h,000h,000h
         dd      offset FLAT:__modtest_test
 _DATA   ends
 CONST   segment
 CONST   ends
 _BSS    segment
 _BSS    ends
 FMB     segment
 FMB     ends
 FM      segment
         dd      offset FLAT:__ModuleInfo_test
 FM      ends
 FME     segment
 FME     ends
 _Dtest_unittest0_FZv    comdat
         assume  CS:_Dtest_unittest0_FZv
 L0:             push    offset FLAT:_DATA
                 call    near ptr _printf
                 add     ESP,4
                 ret
 _Dtest_unittest0_FZv    ends
 _Dtest_unittest1_FZv    comdat
         assume  CS:_Dtest_unittest1_FZv
 L0:             push    offset FLAT:_DATA[3]
                 call    near ptr _printf
                 add     ESP,4
                 ret
 _Dtest_unittest1_FZv    ends
 __Dmain comdat
         assume  CS:__Dmain
                 ret
 __Dmain ends
 __modtest_test  comdat
         assume  CS:__modtest_test
 L0:             call    near ptr _Dtest_unittest0_FZv
                 call    near ptr _Dtest_unittest1_FZv
                 ret
 __modtest_test  ends
         end


--
Helmut Leitner    leitner hls.via.at   
Graz, Austria   www.hls-software.com
Jul 26 2003
parent "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F226F64.EC1B9CCC chello.at...

 Thank you for these hints. They may or may not be sufficient.

 The problem is that unittests are implemented on top of almost
 any programming language. Most of the time they are separate
 testing frameworks.

 The D claim to support unittests would make a lot of sense
 if - and only if - the support is out of the box. Otherwise
 it means nothing.

 I'll try to build on the hooks you gave me, but it will only
 make sense when reintegrated into D.


Let me know how it goes. -Walter
Aug 12 2003
prev sibling parent reply "Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> writes:
Hi,

    Comments embedded.

"Walter" <walter digitalmars.com> escreveu na mensagem
news:besod7$1qb8$1 digitaldaemon.com...

 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F11B6D6.EFEB8CFD chello.at...

 Walter wrote:

 "Dario" <Dario_member pathlink.com> wrote in message
 news:bepjq1$1q68$1 digitaldaemon.com...

 I would be nice that unittests were automatically launched by the






 compiler

 as

 soon as it has finished the code-generation phase and that unittests


 weren't

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the






 code,

 and

 the -unittest option would be active by default when -release is not


 enabled.

 What do you think?


 It's an intriguing idea, but the compiler has no way of executing code




 in a

 .obj file without doing a full link. Doing a full link gets you where




 you

 are today anyway with running unittests <g>.






    I think a make-like utility is essential in D projects, I usually have a
dummy "main" for unittests that just return 0, so I just type "make tests"
and everything is fine. You shouldn't compile and link with your real "main"
using -unittest.


 I think unittests should:

    - always tell which test(s) exactly have failed.


 It does already, with the file/line of the failed contract. Of course, you
 can put any code you want in the unittest blocks, not just assert()'s.


    It isn't enough for a couple of reasons: assert just give me the
file/line of the assert failed, not the call stack. If I have a contract
failing inside a method after twenty calls from my unittest the error
message don't give me any context to work with. A very good solution would
be making the contract failure info Eiffel-like, dumping the call stack and
the variables in stack at the time (so we can figure out the context). The
current state is really crude, I usually resort to insert several lines of
printf to pinpoint the error. Of course I could debug instead of this, but I
use TDD and in any other language the tests are enough.


    - never stop on an error but always run through all tests.
      This is important if you break code while refactoring. You must




have

      the complete information on the total effects.
      Typically this is accomplished by forking the process and reporting
      the state of the process back to the main unittest code.


 My experience usually is that if one test fails, subsequent tests fail as
 well because of cascading errors from the first fault.


    Today we have units of unittests so I think each unit would be
independent. If I have three "unittest" blocks they are independent (they
are test fixtures, and each must be independent). When I change something
it's important to know that the "insert" operation failed but everything
else is still fine. Today it's all or nothing.


    - should have a visual display to show how many tests have already




been

      run and how many have failed (this is important for refactoring)


 because

      unittests may run (on more complex) systems up to hours.


 You can do this by writing such code into the unittest blocks.


    If unittests had names (like "unittest testFindString { ... }") using
reflective libraries we could automate this process and create different
test reporters for different environments (e.g. text, GUI, XML, etc.) like
any of the xUnit tools make. Walter, how much do you know about the xUnit
tools (e.g. jUnit, sUnit)? There's a lot of info in www.junit.org about how
they could develop a powerful, extensible, tool using Java's reflexive
mechanisms. I'll make a summary of jUnit basic features:

- It has classes representing test cases, test suites, test results and test
runners.
- Test cases are just classes extending TestCase containing methods named
testXXXXXXXX and two special methods: setUp and tearDown. When the test case
is executed the test runner uses reflection to find all methods named
testXXXXXX, and for each one executes the TestCase setUp method, the current
test method and finally calls the TestCase tearDown method. If an assertion
fails inside a TestCase test method, it raises an exception that is used to
report the failure.
- TestSuites are a bunch of TestCases bundled together.
- TestResults encapsulate the details of the executed tests, including which
failed, which succeded and which raised uncatched exceptions.
- TestRunners run the tests and report the failures in a given medium (e.g.
text, GUI, XML, e-mail to the developers, etc.).

   This is just a quick look of how it works, but almost every programmer
coming to D will look after these features. I use jUnit all the time in
Java, and D unittest support is pale in comparison. If we need some kind of
house-keeping operations (i.e. setUp and tearDown) we have to remember
calling them explicitly, if we need some kind of alternative medium (GUI is
much better than text) we must explicit code each every time. I'm tempted to
rewrite jUnit in D, actually I'm doing that right now, just to have these
features.


    - run only once from an completed exe (an app.exe could create an


 marker

      file app.uok whose date-time-stamp is tested against app.exe)


 That's an interesting approach. I've used self-patching exe's in the past


to

 do initializations; I had to stop the practice because anti-virus programs
 had fits about it.


    Having unittests as first class objects is nice, but if the unittesting
support is incomplete regarding other languages, it won't be necessary and
probably discarded in larger projects. jUnit is an example of how it should
be, for other frameworks you can check
http://www.artima.com/suiterunner/index.html , http://www.fitnesse.org/ ,
http://fit.c2.com and http://c2.com/cgi/wiki?FrameworkForIntegratedTest

    Best regards,
    Daniel Yokomiso.

A: Top-posters.
Q: What is the most annoying thing on Usenet?


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.502 / Virus Database: 300 - Release Date: 18/7/2003
Jul 23 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> wrote in message
news:bfn8lg$2hm2$1 digitaldaemon.com...

 It does already, with the file/line of the failed contract. Of course,




you

 can put any code you want in the unittest blocks, not just assert()'s.


     It isn't enough for a couple of reasons: assert just give me the
 file/line of the assert failed, not the call stack. If I have a contract
 failing inside a method after twenty calls from my unittest the error
 message don't give me any context to work with. A very good solution would
 be making the contract failure info Eiffel-like, dumping the call stack


and

 the variables in stack at the time (so we can figure out the context). The
 current state is really crude, I usually resort to insert several lines of
 printf to pinpoint the error. Of course I could debug instead of this, but


I

 use TDD and in any other language the tests are enough.


The stack trace is a good idea. I use it regularly in a debugger, which
prints it out just fine. I've thought of some ways to implement it in
general for D exceptions, which would solve your issue.



     Today we have units of unittests so I think each unit would be
 independent. If I have three "unittest" blocks they are independent (they
 are test fixtures, and each must be independent). When I change something
 it's important to know that the "insert" operation failed but everything
 else is still fine. Today it's all or nothing.


You can continue by wrapping the tests you want to continue with a
try-catch.



     If unittests had names (like "unittest testFindString { ... }") using
 reflective libraries we could automate this process and create different
 test reporters for different environments (e.g. text, GUI, XML, etc.) like
 any of the xUnit tools make. Walter, how much do you know about the xUnit
 tools (e.g. jUnit, sUnit)?


Nothing.


 There's a lot of info in www.junit.org about how
 they could develop a powerful, extensible, tool using Java's reflexive
 mechanisms. I'll make a summary of jUnit basic features:

 - It has classes representing test cases, test suites, test results and


test

 runners.
 - Test cases are just classes extending TestCase containing methods named
 testXXXXXXXX and two special methods: setUp and tearDown. When the test


case

 is executed the test runner uses reflection to find all methods named
 testXXXXXX, and for each one executes the TestCase setUp method, the


current

 test method and finally calls the TestCase tearDown method. If an


assertion

 fails inside a TestCase test method, it raises an exception that is used


to

 report the failure.
 - TestSuites are a bunch of TestCases bundled together.
 - TestResults encapsulate the details of the executed tests, including


which

 failed, which succeded and which raised uncatched exceptions.
 - TestRunners run the tests and report the failures in a given medium


(e.g.

 text, GUI, XML, e-mail to the developers, etc.).

    This is just a quick look of how it works, but almost every programmer
 coming to D will look after these features. I use jUnit all the time in
 Java, and D unittest support is pale in comparison. If we need some kind


of

 house-keeping operations (i.e. setUp and tearDown) we have to remember
 calling them explicitly, if we need some kind of alternative medium (GUI


is

 much better than text) we must explicit code each every time. I'm tempted


to

 rewrite jUnit in D, actually I'm doing that right now, just to have these
 features.


Sounds like a good idea. Such a "dUnit" be a great tool to add. Keep us
posted on the progress on it!
Aug 12 2003
parent "Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> writes:
Hi,

    Comments embedded.

"Walter" <walter digitalmars.com> escreveu na mensagem
news:bhbk0a$1p9k$1 digitaldaemon.com...
[snip]

 The stack trace is a good idea. I use it regularly in a debugger, which
 prints it out just fine. I've thought of some ways to implement it in
 general for D exceptions, which would solve your issue.


    Great! It's one of the most annying things today.


     Today we have units of unittests so I think each unit would be
 independent. If I have three "unittest" blocks they are independent




(they

 are test fixtures, and each must be independent). When I change




something

 it's important to know that the "insert" operation failed but everything
 else is still fine. Today it's all or nothing.


 You can continue by wrapping the tests you want to continue with a
 try-catch.


    I guess this is a matter of personal taste. I usually write orthogonal
tests, so a failure in one doesn't imply in failures of others. The xUnit
tools use this metaphor too, and many programmers use it, but I don't need
it this way.


     If unittests had names (like "unittest testFindString { ... }")




using

 reflective libraries we could automate this process and create different
 test reporters for different environments (e.g. text, GUI, XML, etc.)




like

 any of the xUnit tools make. Walter, how much do you know about the




xUnit

 tools (e.g. jUnit, sUnit)?


 Nothing.


    I think it'll be very good for you to read papers and code about these
tools. They'll give you some perspective of the "standards" in unit testing.


 There's a lot of info in www.junit.org about how
 they could develop a powerful, extensible, tool using Java's reflexive
 mechanisms. I'll make a summary of jUnit basic features:

 - It has classes representing test cases, test suites, test results and


 test

 runners.
 - Test cases are just classes extending TestCase containing methods




named

 testXXXXXXXX and two special methods: setUp and tearDown. When the test


 case

 is executed the test runner uses reflection to find all methods named
 testXXXXXX, and for each one executes the TestCase setUp method, the


 current

 test method and finally calls the TestCase tearDown method. If an


 assertion

 fails inside a TestCase test method, it raises an exception that is used


 to

 report the failure.
 - TestSuites are a bunch of TestCases bundled together.
 - TestResults encapsulate the details of the executed tests, including


 which

 failed, which succeded and which raised uncatched exceptions.
 - TestRunners run the tests and report the failures in a given medium


 (e.g.

 text, GUI, XML, e-mail to the developers, etc.).

    This is just a quick look of how it works, but almost every




programmer

 coming to D will look after these features. I use jUnit all the time in
 Java, and D unittest support is pale in comparison. If we need some kind


 of

 house-keeping operations (i.e. setUp and tearDown) we have to remember
 calling them explicitly, if we need some kind of alternative medium (GUI


 is

 much better than text) we must explicit code each every time. I'm




tempted

 to

 rewrite jUnit in D, actually I'm doing that right now, just to have




these

 features.


 Sounds like a good idea. Such a "dUnit" be a great tool to add. Keep us
 posted on the progress on it!


    Looks like this week I'll resume coding in D, but most of the dUnit tool
depends on the stack trace information.

    Best regards,
    Daniel Yokomiso.

"I drink to make other people interesting."
 - Groucho Marx



---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.510 / Virus Database: 307 - Release Date: 14/8/2003
Aug 22 2003