www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Liskov principle and unittest

reply deadalnix <deadalnix gmail.com> writes:
I recently faced a problem in java. I have looooooooots of classes 
inheriting from the same superclass. Thoses classes has to conform to 
some behaviour expected when manipulation the superclass as Liskov 
substitution principle says.

This is definitvelly hard to achieve. Solution promoted on the web are 
frankly not satisfying. Most of them suggest to inherit jUnit tests too, 
bit this is unable to handle classes implementing multiple interfaces, 
and require to write a test class for each subclasses. So if somebody 
don't do thoses tests, the subclass isn't checked (which is predictible) 
but the superclass tests are not ran either, which make things even 
worse. This innability to check that easily makes a lazy dev even more 
dangerous, which isn't what we want.

Back to D, which has integrated unittest, I wonder how I could implement 
such a check. I think the D unittest system suffer the same flaw as 
jUint (previously exposed). What do you think ?

deadalnix
Sep 23 2011
next sibling parent reply Tobias Pankrath <tobias pankrath.net> writes:
The only thing that D unittest do, is executing code if you
pass -unittest to the compiler. What you do in this code is entirely
up to you. 

So, I think the answer is no. No you can't do this with
the current language and standard library. 

When I started learning D, I thought the way D handles unit tests is
very limited compared to libraries in other languages. However D makes
it straight forward and simple to build something more elaborate.

So what I'm doing now, is to register delegates as unit tests, so I can
get a statistic telling me, how many pass and how many fail.

I think it is doable in an similar way to associate test cases with
classes and for every subclass look if a test case should be inherited
from the baseclass using Ds compile time reflection capabilities.
Sep 23 2011
parent mta`chrono <chrono mta-international.net> writes:
class Superclass
{
     abstract int do_some_stuff();
     abstract int do_some_other_stuff();
}

class Foo1 : Superclass....
class Foo2 : Superclass....
class Foo3 : Superclass....


unittest
{
      // get an instance of all classes
      Superclass[] instances = [new Foo1(), new Foo2(), new Foo3()];

      // test their behavoir
      foreach(instance; instances)
      {
           assert(instance.do_some_stuff() > 0, "whoops!");
           assert(instance.do_some_other_stuff() < 100, "whoops!");
      }
}
Sep 23 2011
prev sibling parent reply Max Klyga <max.klyga gmail.com> writes:
On 2011-09-23 12:55:25 +0300, deadalnix said:


 I recently faced a problem in java. I have looooooooots of classes 
 inheriting from the same superclass. Thoses classes has to conform to 
 some behaviour expected when manipulation the superclass as Liskov 
 substitution principle says.
 
 This is definitvelly hard to achieve. Solution promoted on the web are 
 frankly not satisfying. Most of them suggest to inherit jUnit tests 
 too, bit this is unable to handle classes implementing multiple 
 interfaces, and require to write a test class for each subclasses. So 
 if somebody don't do thoses tests, the subclass isn't checked (which is 
 predictible) but the superclass tests are not ran either, which make 
 things even worse. This innability to check that easily makes a lazy 
 dev even more dangerous, which isn't what we want.
 
 Back to D, which has integrated unittest, I wonder how I could 
 implement such a check. I think the D unittest system suffer the same 
 flaw as jUint (previously exposed). What do you think ?
 
 deadalnix


D has a wonderful feature that is suitable for such use cases - Design 
By Contract.
You specify invariants and in/out contracts in your superclass and they 
will be automatically checked for subclasses.

Read more here - http://www.d-p-l.org/dbc.html
Sep 23 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/23/2011 10:17 PM, Max Klyga wrote:

 On 2011-09-23 12:55:25 +0300, deadalnix said:


 I recently faced a problem in java. I have looooooooots of classes
 inheriting from the same superclass. Thoses classes has to conform to
 some behaviour expected when manipulation the superclass as Liskov
 substitution principle says.

 This is definitvelly hard to achieve. Solution promoted on the web are
 frankly not satisfying. Most of them suggest to inherit jUnit tests
 too, bit this is unable to handle classes implementing multiple
 interfaces, and require to write a test class for each subclasses. So
 if somebody don't do thoses tests, the subclass isn't checked (which
 is predictible) but the superclass tests are not ran either, which
 make things even worse. This innability to check that easily makes a
 lazy dev even more dangerous, which isn't what we want.

 Back to D, which has integrated unittest, I wonder how I could
 implement such a check. I think the D unittest system suffer the same
 flaw as jUint (previously exposed). What do you think ?

 deadalnix


 D has a wonderful feature that is suitable for such use cases - Design
 By Contract.


It's called Contract Programming actually, afaik because Design By 
Contract is trademarked by Eiffel Software. :o)


 You specify invariants and in/out contracts in your superclass and they
 will be automatically checked for subclasses.

 Read more here - http://www.d-p-l.org/dbc.html


I think what he wants is to have automatically generated unittests for 
derived classes that check if the unittests of the parent still pass for 
them. I don't know of a nice way to do that without requiring some 
minimal effort by the implementor of the derived class. Contracts 
quickly detect and report otherwise potentially unnoticed bugs during 
test runs, but they don't provide the robustness of running actual unit 
tests. The two features can be combined very well though.
Sep 23 2011
parent reply deadalnix <deadalnix gmail.com> writes:
 I think what he wants is to have automatically generated unittests for
 derived classes that check if the unittests of the parent still pass for
 them. I don't know of a nice way to do that without requiring some
 minimal effort by the implementor of the derived class. Contracts
 quickly detect and report otherwise potentially unnoticed bugs during
 test runs, but they don't provide the robustness of running actual unit
 tests. The two features can be combined very well though.


Yes, that's it.

Actually, I'm not blaming D, as I explained, the same problem exists 
with others languages.

Contract is a part of the solution, but not really the same as unittest. 
Indeed, contract are very compatible with Liskov's principle, but 
unittest isn't.

I just wanted to open up the discution on that.
Sep 25 2011
parent reply Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sun, Sep 25, 2011 at 20:02, deadalnix <deadalnix gmail.com> wrote:

 I think what he wants is to have automatically generated unittests for
 derived classes that check if the unittests of the parent still pass for
 them. I don't know of a nice way to do that without requiring some
 minimal effort by the implementor of the derived class.




IIUC, I might have a partial solution:

1- put your unit test in a CTE function template that, given a type,
will generate the unit test you want.
2- given a class type, find all classes derived from it in a module.
(That's doable. I think I could even code something that determine the
inheritance tree in a module. Dunno for multi-module hierarchy,
though).
3- iterate on the found classes, generating the
unitests-as-strings-to-be-mixed-in
4- mix them in.

Client code would look like this:

class Origin {}
class Derived1 : Origin {}
...
class Derived1000 : Origin /* or any other Derived */ {}

string generateUnitTest(Type)()
{
    return "auto t = new(" ~ Type.stringof ~ ")(...);
               /* some more unit test code*/
               ";
}

mixin(testDerived!(Origin, generateUnitTest);


where testDerived is :

string testDerived(Type, alias generator)()
{
    alias GetDerived!Origin Children;
    string result;
    foreach(i, Child; Children)
        result ~= "unittest { " ~ generator!Child ~ " }\n";
    return result;
}

So you write the generator once, and mix testDerived in. And you should be done.
Is that what you want? (Still have to code GetDerived, of course)

The limitation is I'm not sure that would work for class templates hierarchies.
Sep 25 2011
parent reply deadalnix <deadalnix gmail.com> writes:
Le 25/09/2011 21:30, Philippe Sigaud a écrit :

 On Sun, Sep 25, 2011 at 20:02, deadalnix<deadalnix gmail.com>  wrote:

 I think what he wants is to have automatically generated unittests for
 derived classes that check if the unittests of the parent still pass for
 them. I don't know of a nice way to do that without requiring some
 minimal effort by the implementor of the derived class.




 IIUC, I might have a partial solution:

 1- put your unit test in a CTE function template that, given a type,
 will generate the unit test you want.
 2- given a class type, find all classes derived from it in a module.
 (That's doable. I think I could even code something that determine the
 inheritance tree in a module. Dunno for multi-module hierarchy,
 though).
 3- iterate on the found classes, generating the
 unitests-as-strings-to-be-mixed-in
 4- mix them in.

 Client code would look like this:

 class Origin {}
 class Derived1 : Origin {}
 ...
 class Derived1000 : Origin /* or any other Derived */ {}

 string generateUnitTest(Type)()
 {
      return "auto t = new(" ~ Type.stringof ~ ")(...);
                 /* some more unit test code*/
                 ";
 }

 mixin(testDerived!(Origin, generateUnitTest);


 where testDerived is :

 string testDerived(Type, alias generator)()
 {
      alias GetDerived!Origin Children;
      string result;
      foreach(i, Child; Children)
          result ~= "unittest { " ~ generator!Child ~ " }\n";
      return result;
 }

 So you write the generator once, and mix testDerived in. And you should be
done.
 Is that what you want? (Still have to code GetDerived, of course)

 The limitation is I'm not sure that would work for class templates hierarchies.


Sounds nice ! I'm pretty sure this has some limitations, but that 
definitively a step forward.

I'll do some tests about that.

If the getDerived isn't devellopped, then the whole stuff will fail at 
compile time, so this ok. We can be sure that tests are performed for 
every derived classes, and so, even if the devellopper of the derived 
class forgot to do unittest or don't want to.
Sep 26 2011
parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Mon, Sep 26, 2011 at 14:05, deadalnix <deadalnix gmail.com> wrote:

 The limitation is I'm not sure that would work for class templates
 hierarchies.


 Sounds nice ! I'm pretty sure this has some limitations, but that
 definitively a step forward.

 I'll do some tests about that.

 If the getDerived isn't devellopped, then the whole stuff will fail at
 compile time, so this ok. We can be sure that tests are performed for every
 derived classes, and so, even if the devellopper of the derived class forgot
 to do unittest or don't want to.


Here is the missing code:

/** finds all local (module-level) classes */
template LocalClasses()
{
    mixin("alias TypeTuple!" ~ localClasses() ~ " LocalClasses;");
}

string localClasses()
{
    string result;
    foreach(i, symbol;mixin("__traits(allMembers, " ~ .stringof[7..$] ~ ")"))
        mixin("static if (is(" ~ symbol ~ " == class))
            result ~= symbol ~ \",\";");

    return "(" ~ result[0..$-1] ~ ")";
}

/** Given a class type, returns the local (module-level)
 children classes. */
template GetDerived(Type) if (is(Type == class))
{
    mixin("alias TypeTuple!(" ~ getDerivedFrom!Type ~ ") GetDerived;");
}

string getDerivedFrom(Type)() if (is(Type == class))
{
    string result;
    foreach(i, Class; LocalClasses!())
        static if (is(Class : Type))
            result ~=  Class.stringof ~ ",";
    return result;
}


string testDerived(Origin, alias generator)() if (is(Origin == class))
{
    alias GetDerived!Origin Children;
    string result;
    foreach(i, Child; Children)
        result ~= "unittest\n{\n" ~ generator!Child ~ "\n}\n";
    return result;
}

/** Stub unit-test, just generating a pragma(msg, ...) call */
string genUT(Type)()
{
    return "pragma(msg, \"testing for \", " ~ Type.stringof ~ ");";
}


// Usage:
class C1 {}

class C2 {}
class C3: C2 {}
class C4 : C2 {}
class C5 : C4 {}

mixin(testDerived!(C2, genUT));


This will find that C2 derived classes are C3, C4 and C5 and will
generate stub unit-tests for each of them. Not for C1, because it's
not derived from C2.

Hey, I even found a way to statically reconstitute the entire class
tree. It feels good to code in D again!

Limitations :
- not tested with multi-modules hierarchies. I know how to get the
module (not package) imports from a module name, so I could recurse on
them. But packages are problematic.
- class templates may cause problems.
Sep 26 2011