digitalmars.D.announce - Unit Threaded - a unit testing library for D
- Atila Neves (24/24) Aug 27 2013 My very first D project is now feature-complete as far as I can
- Meta (4/4) Aug 27 2013 On Tuesday, 27 August 2013 at 13:07:02 UTC, Atila Neves wrote:
- Tobias Pankrath (19/19) Aug 27 2013 "But doesn't D have built-in unittest blocks"? Yes, and they're
- Andrej Mitrovic (18/23) Aug 27 2013 Quoting you but responding to OP:
- Atila Neves (7/25) Aug 27 2013 They're easy to implement and I did to test the library
- Dicebot (5/10) Aug 27 2013 By the, way, can we currently in 2.064 attach UDA's to unittest
- Andrej Mitrovic (31/33) Aug 27 2013 Yes. E.g.:
- Jacob Carlborg (5/9) Aug 27 2013 Yes, and unit tests for CTFE:
- Nick Sabalausky (10/19) Aug 27 2013 Someone actually crated all of those and made a pull request to Phobos,
- linkrope (3/5) Aug 27 2013 Where is your 'assertOp' from the comments of issue 4653?
- Andrej Mitrovic (2/7) Aug 27 2013 It appears I didn't add it! Thanks for reminding me.
- Atila Neves (9/16) Aug 27 2013 I haven't had a problem with compile times. Ideally the test code
- Russel Winder (17/23) Aug 27 2013 The trend in other languages, especially the Java-verse, is to use the
- Atila Neves (19/29) Aug 27 2013 Not at all. In fact, I was playing with using it to do
- Atila Neves (3/28) Nov 04 2013 I finally got around to putting it up in the dub registry.
My very first D project is now feature-complete as far as I can see (alpha) and I'd like to share it with the community: https://github.com/atilaneves/unit-threaded There are more details on github but here are the highlights: 1. Automatic registration of unit tests via compile-time reflection 2. Unit tests can be classes or functions, the latter for minimal boilerplate 3. Support for D's unittest blocks 4. Runs in multiple threads by default, possible to run in one thread 5. Manual selection of tests to run at the command-line (runs all tests by default) I've used it myself to test my other D projects (2 other for now) to make sure it did what I wanted it to. I basically wrote this for myself knowing what I'd want from a unit testing library, but I'm sure it can be useful for the general D public. Feedback is of course more than welcome! Atila P.S. I totally expect this to break for large real-world projects in its current state. It can test some modules in phobos but had problems with others, for instance. My own other D projects are tiny and even if I had a large D codebase the tests I'd write would reflect the biases that go into this library.
Aug 27 2013
On Tuesday, 27 August 2013 at 13:07:02 UTC, Atila Neves wrote: I cannot tell you how handy this will be. I've got a couple of projects that badly need this as is, and I can see myself using it for all future projects of any notable size.
Aug 27 2013
"But doesn't D have built-in unittest blocks"? Yes, and they're massively useful. Even short scripts can benefit from them with 0 effort and setup. In fact, I use them to test this library. However, for larger projects it lacks some functionality: If all tests pass, great. If one fails, it's hard to know why. The only tool is assert, and you have to write your own assert messages (no assertEqual, assertNull, etc.) No possibility to run just one particular test Only runs in one thread. ------------- That's exactly the reason I did something like that, too. Especially the first and third reason. However I use a different approach, which I find easier to handle: I'm using the build-in unittest blocks to register delegates as tests in a central table. These than get executed by a test runner in main(). This way you can register multiple tests using i.e. a loop over sample data or something similar. And it's very easy to implement and has almost no impact on compile time.
Aug 27 2013
On 8/27/13, Tobias Pankrath <tobias pankrath.net> wrote:If all tests pass, great. If one fails, it's hard to know why.Quoting you but responding to OP: There was a pull I made to make assert print out some more info on failure, but it never passed the test-suite. Pull: https://github.com/D-Programming-Language/dmd/pull/1426The only tool is assert, and you have to write your own assert messages (no assertEqual, assertNull, etc.)Currently we have assertThrown/assertNotThrown, but assertEqual and friends are easy to implement and we thought about including them. However it needs more community support before it's included in Phobos (i.e. more people need to want these). I have some of these functions here: https://github.com/AndrejMitrovic/minilib/blob/master/src/minilib/core/test.dNo possibility to run just one particular testE.g. test only one unittest block rather than the entire module at once? I'd imagine this will be possible with the new getUnitTests trait in 2.064.Only runs in one thread.getUnitTests should help here, although it's still based on compile-time introspection just like your library. I guess if nothing else, this trait will make your library implementation simpler. :)
Aug 27 2013
Currently we have assertThrown/assertNotThrown, but assertEqual and friends are easy to implement and we thought about including them. However it needs more community support before it's included in Phobos (i.e. more people need to want these).They're easy to implement and I did to test the library itself. But it's either reimplement them everytime for each project or format assert messages manually. Neither appealed to me.Yes.No possibility to run just one particular testE.g. test only one unittest block rather than the entire module at once?I'd imagine this will be possible with the new getUnitTests trait in 2.064.Hadn't heard of that, I'm going to have to take a look.getUnitTests should help here, although it's still based on compile-time introspection just like your library. I guess if nothing else, this trait will make your library implementation simpler. :)Yay? :)
Aug 27 2013
On Tuesday, 27 August 2013 at 15:42:28 UTC, Andrej Mitrovic wrote:getUnitTests should help here, although it's still based on compile-time introspection just like your library. I guess if nothing else, this trait will make your library implementation simpler. :)By the, way, can we currently in 2.064 attach UDA's to unittest blocks? Together with getUnitTest that will allow to have same test code base that acts in both "old-school" mode and gets used by some fancy introspection library.
Aug 27 2013
On 8/27/13, Dicebot <public dicebot.lv> wrote:By the way, can we currently in 2.064 attach UDA's to unittest blocks?Yes. E.g.: ----- alias Seq(T...) = T; struct RunTest { } struct SkipTest { } RunTest unittest { } SkipTest unittest { } void main() { alias Tests = Seq!(__traits(getUnitTests, test)); // http://d.puremagic.com/issues/show_bug.cgi?id=7804 workaround alias Attrib1 = Seq!(__traits(getAttributes, Tests[0]))[0]; // ditto alias Attrib2 = Seq!(__traits(getAttributes, Tests[1]))[0]; static assert(is(Attrib1 == RunTest)); static assert(is(Attrib2 == SkipTest)); } ----- The Seq() nonsense is to avoid running into Issue 7804. Working with attributes is generally very quirky, I had to define a whole set of __traits(compiles) versions of templates just in order to be able to extract various attributes from symbols (an attribute can be a type, an instance, a template declaration, a template instance, a literal, etc...). Here's some of the work I've had to do to work with attributes: Helper functions: https://github.com/AndrejMitrovic/minilib/blob/master/src/minilib/core/attributes.d And I use the attributes to enable things like (nevermind the ironic name "simple" :p): https://github.com/AndrejMitrovic/minilib/blob/master/src/minilib/parser/simple_ini/tests.d
Aug 27 2013
On 2013-08-27 18:59, Dicebot wrote:By the, way, can we currently in 2.064 attach UDA's to unittest blocks? Together with getUnitTest that will allow to have same test code base that acts in both "old-school" mode and gets used by some fancy introspection library.Yes, and unit tests for CTFE: http://forum.dlang.org/thread/ks1brj$1l6c$1 digitalmars.com -- /Jacob Carlborg
Aug 27 2013
On Tue, 27 Aug 2013 17:42:19 +0200 Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:Someone actually crated all of those and made a pull request to Phobos, but it was rejected because people decided it was better to improve the regular assert to be able to display such information automatically. Predictably, these hypothetical improvements to assert never actually got made (it's been probably around a couple years now), so that's why we've still got nothing like that in the stock DMD/Phobos. I do wish we weren't in the habit of letting "perfect" get in the way of progress.The only tool is assert, and you have to write your own assert messages (no assertEqual, assertNull, etc.)Currently we have assertThrown/assertNotThrown, but assertEqual and friends are easy to implement and we thought about including them. However it needs more community support before it's included in Phobos (i.e. more people need to want these).
Aug 27 2013
On Tuesday, 27 August 2013 at 15:42:28 UTC, Andrej Mitrovic wrote:I have some of these functions here: https://github.com/AndrejMitrovic/minilib/blob/master/src/minilib/core/test.dWhere is your 'assertOp' from the comments of issue 4653? I recently added it to https://github.com/linkrope/dunit
Aug 27 2013
On 8/28/13, linkrope <linkrope github.com> wrote:On Tuesday, 27 August 2013 at 15:42:28 UTC, Andrej Mitrovic wrote:It appears I didn't add it! Thanks for reminding me.I have some of these functions here: https://github.com/AndrejMitrovic/minilib/blob/master/src/minilib/core/test.dWhere is your 'assertOp' from the comments of issue 4653? I recently added it to https://github.com/linkrope/dunit
Aug 27 2013
I haven't had a problem with compile times. Ideally the test code should be small and well-separated into modules so I'm not too worried about that (the compile-time scanning only happens on the test modules). As for the registration, it was one of my goals to not have to do any. You could still conceivably loop at compile-time with a foreach on a tuple and generate the required test functions / classes. In my experience there's usually no need for that kind of thing, but YMMV.However I use a different approach, which I find easier to handle: I'm using the build-in unittest blocks to register delegates as tests in a central table. These than get executed by a test runner in main(). This way you can register multiple tests using i.e. a loop over sample data or something similar. And it's very easy to implement and has almost no impact on compile time.
Aug 27 2013
On Tue, 2013-08-27 at 15:07 +0200, Atila Neves wrote:My very first D project is now feature-complete as far as I can=20 see (alpha) and I'd like to share it with the community: =20 https://github.com/atilaneves/unit-threaded =20 There are more details on github but here are the highlights:The trend in other languages, especially the Java-verse, is to use the same testing framework for unit, integration and some system testing. By using the name "Unit Threaded", and talking of unit testing, you seem to be stating that this framework is no good for integration and system testing, just unit testing. Is this the case? Thanks. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Aug 27 2013
Not at all. In fact, I was playing with using it to do integration testing at work for our C++ code (it has a C interface). I guess I was just focussed on unit testing because I got annoyed at how long our unit tests* take to run, which was what led me to write the C++11 version to begin with. After your post I've got half a mind to try and speed up all aspects of testing at work. Thanks! I guess I or someone else will just have to try and use the library for other types of testing and fix anything that might need to be fixed. I even mentioned at work that if I started a new company or large project that I'd use D to write tests for anything with a C interface. There's less glue to write (even with Swig, getting the Python bindings to work can be a pain), it compiles lightning fast and is just as nice as writing, say, Python. Atila * Even though part of the reason they take so long to run is that quite a few of them are unit tests in name only, being mini-integration tests in practice.The trend in other languages, especially the Java-verse, is to use the same testing framework for unit, integration and some system testing. By using the name "Unit Threaded", and talking of unit testing, you seem to be stating that this framework is no good for integration and system testing, just unit testing. Is this the case? Thanks.
Aug 27 2013
I finally got around to putting it up in the dub registry. http://code.dlang.org/packages/unit-threaded On Tuesday, 27 August 2013 at 13:07:02 UTC, Atila Neves wrote:My very first D project is now feature-complete as far as I can see (alpha) and I'd like to share it with the community: https://github.com/atilaneves/unit-threaded There are more details on github but here are the highlights: 1. Automatic registration of unit tests via compile-time reflection 2. Unit tests can be classes or functions, the latter for minimal boilerplate 3. Support for D's unittest blocks 4. Runs in multiple threads by default, possible to run in one thread 5. Manual selection of tests to run at the command-line (runs all tests by default) I've used it myself to test my other D projects (2 other for now) to make sure it did what I wanted it to. I basically wrote this for myself knowing what I'd want from a unit testing library, but I'm sure it can be useful for the general D public. Feedback is of course more than welcome! Atila P.S. I totally expect this to break for large real-world projects in its current state. It can test some modules in phobos but had problems with others, for instance. My own other D projects are tiny and even if I had a large D codebase the tests I'd write would reflect the biases that go into this library.
Nov 04 2013