digitalmars.D - Unit testing with asserts: Why is assertHandler required to throw?
- Trip Volpe (11/11) Jan 31 2010 I recently began porting an existing C++ project of mine (a compiler/int...
- Nick Sabalausky (12/57) Jan 31 2010 The deferAssert module (possible name change and other API improvements
- Trip Volpe (3/7) Jan 31 2010 Good to know I'm not the only one who's had to working around this. :-P ...
- Nick Sabalausky (9/21) Jan 31 2010 Yea, that's one of the reasons I like to push for a better string mixin
- Lutger (29/40) Jan 31 2010 You can use line and file info with default arguments, this works
- Nick Sabalausky (3/5) Jan 31 2010 That must be a D2 thing, because I know it doesn't work in D1.
- Lutger (3/9) Jan 31 2010 Indeed, here is the change:
- Trip Volpe (11/29) Jan 31 2010 Ah ha! I din't know that D had __LINE__ and __FILE__. I thought those ha...
- bearophile (4/16) Jan 31 2010 It looks like the bad special cased magic you can find in Perl :-)
- =?UTF-8?B?UGVsbGUgTcOlbnNzb24=?= (12/23) Jan 31 2010 I think the reason for this is that assert is used by the compiler for
- Trip Volpe (3/16) Jan 31 2010 Heh, yeah, if I had just noticed std.contracts in the first place, I wou...
I recently began porting an existing C++ project of mine (a compiler/interpreter for a dynamic language) to D. In the process I found that the built-in unit testing support, while an awesome concept, was a little bit sparse. In particular, assert() is fairly useless for unit tests, since it throws on a failure and prevents all subsequent tests from running. Unfortunately, it's the only facility that seems to give you access to the current file and line number, very important bits of info for unit tests in a any non-trivial project. I was contemplating just abandoning D and sticking with C++ and Googletest, but after doing a bit of digging through the source, I found setAssertHandler() in core.exception. Perfect! I could use my own assert handler that just records the error and allows future asserts to still be reached. So I wrote it up and gave it a try, but pretty quickly started getting access violation errors. I did a bit of searching, and found a bug report on the issue: http://d.puremagic.com/issues/show_bug.cgi?id=3208 In the comments it's indicated that this is intentional, that the compiler expects the assertion handler to throw. So I guess I have a few questions: 1. Why is this the expected behavior? It seems to me that there are relatively few useful things you can do with a custom assert handler unless it is possible to refrain from throwing. 2. Is this going to be fixed any time soon? It seems fairly important to me; it really kills the value of built-in unit tests to be forced to choose between not having any line number information and being able to recognize only one assert failure per test run(!!!). 3. Are there any current workarounds for the problem? I could just use my own assertion function, but 1) I can't call it "assert" because that's a reserved word, and 2) most importantly, it won't have any way of indicating in its diagnostic output which source line the failure occurred on. In C++, of course, I could use preprocessor macros to do that, but D omits a preprocessor (for very good reasons). Unfortunately, this seems like a case where D hasn't provided replacement functionality. But maybe there's a sneaky fix I haven't thought of? I also found Runtime.moduleUnitTester() in core.runtime, which is very useful in itself, but provides an _almost_ useful improvement: I can catch the AssertErrors on a module-by-module basis, which would at least allow the unit tests for all modules to be run regardless of a failure in one of them. However, a single module could easily have dozens of tests in it, each of which could contain many individual asserts. Stopping the test early because one assert failed makes no sense. It would be _almost_ acceptable to at least be sure of running all the tests in each module; is there any way to poke inside or otherwise override ModuleInfo's unitTest() function? Thanks in advance for any advice or comments! D is a completely awesome language, but I found this issue a bit strange.
Jan 31 2010
"Trip Volpe" <mraccident gmail.com> wrote in message news:hk4pql$1ehf$1 digitalmars.com...I recently began porting an existing C++ project of mine (a compiler/interpreter for a dynamic language) to D. In the process I found that the built-in unit testing support, while an awesome concept, was a little bit sparse. In particular, assert() is fairly useless for unit tests, since it throws on a failure and prevents all subsequent tests from running. Unfortunately, it's the only facility that seems to give you access to the current file and line number, very important bits of info for unit tests in a any non-trivial project. I was contemplating just abandoning D and sticking with C++ and Googletest, but after doing a bit of digging through the source, I found setAssertHandler() in core.exception. Perfect! I could use my own assert handler that just records the error and allows future asserts to still be reached. So I wrote it up and gave it a try, but pretty quickly started getting access violation errors. I did a bit of searching, and found a bug report on the issue: http://d.puremagic.com/issues/show_bug.cgi?id=3208 In the comments it's indicated that this is intentional, that the compiler expects the assertion handler to throw. So I guess I have a few questions: 1. Why is this the expected behavior? It seems to me that there are relatively few useful things you can do with a custom assert handler unless it is possible to refrain from throwing. 2. Is this going to be fixed any time soon? It seems fairly important to me; it really kills the value of built-in unit tests to be forced to choose between not having any line number information and being able to recognize only one assert failure per test run(!!!). 3. Are there any current workarounds for the problem? I could just use my own assertion function, but 1) I can't call it "assert" because that's a reserved word, and 2) most importantly, it won't have any way of indicating in its diagnostic output which source line the failure occurred on. In C++, of course, I could use preprocessor macros to do that, but D omits a preprocessor (for very good reasons). Unfortunately, this seems like a case where D hasn't provided replacement functionality. But maybe there's a sneaky fix I haven't thought of? I also found Runtime.moduleUnitTester() in core.runtime, which is very useful in itself, but provides an _almost_ useful improvement: I can catch the AssertErrors on a module-by-module basis, which would at least allow the unit tests for all modules to be run regardless of a failure in one of them. However, a single module could easily have dozens of tests in it, each of which could contain many individual asserts. Stopping the test early because one assert failed makes no sense. It would be _almost_ acceptable to at least be sure of running all the tests in each module; is there any way to poke inside or otherwise override ModuleInfo's unitTest() function? Thanks in advance for any advice or comments! D is a completely awesome language, but I found this issue a bit strange.The deferAssert module (possible name change and other API improvements pending) of my SemiTwist D Tools library ( http://www.dsource.org/projects/semitwist ) is designed to get around those problems. Here's a sample app with it's own output in the comments at the bottom: http://www.dsource.org/projects/semitwist/browser/trunk/src/semitwist/apps/tests/deferAssertTest/main.d And the closest thing I have to an API reference at this point: http://www.dsource.org/projects/semitwist/browser/trunk/src/semitwist/util/deferAssert.d (BTW, contrary to the names, deferAssert and deferEnsure don't actually defer anything other than throwing the fatal assert exception)
Jan 31 2010
Nick Sabalausky Wrote:The deferAssert module (possible name change and other API improvements pending) of my SemiTwist D Tools library ( http://www.dsource.org/projects/semitwist ) is designed to get around those problems.Good to know I'm not the only one who's had to working around this. :-P Considering how D has unit testing and contract programming built-in already, I'm still a bit surprised it doesn't have assertions and expectations that are actually useful for people doing heavy unit testing. deferAssert is interesting, but it seems to require a lot of boilerplate. The combination of mixin and template syntax is a bit kludgy, heh.
Jan 31 2010
"Trip Volpe" <mraccident gmail.com> wrote in message news:hk583n$28eb$1 digitalmars.com...Nick Sabalausky Wrote:Yea, that's one of the reasons I like to push for a better string mixin syntax whenever the topic comes up. But it's the best I've been able to do in D1 without resorting to JUnit-style garbage like "expectTrue(a.logicalOr(b).logicalAnd(c.greaterThan(d)))" or whatever the actual JUnit gibberish is (and I don't even have to abuse operator overloading either). Though, it's still a lot better than what I'd be able to do in most languages.The deferAssert module (possible name change and other API improvements pending) of my SemiTwist D Tools library ( http://www.dsource.org/projects/semitwist ) is designed to get around those problems.Good to know I'm not the only one who's had to working around this. :-P Considering how D has unit testing and contract programming built-in already, I'm still a bit surprised it doesn't have assertions and expectations that are actually useful for people doing heavy unit testing. deferAssert is interesting, but it seems to require a lot of boilerplate. The combination of mixin and template syntax is a bit kludgy, heh.
Jan 31 2010
On 01/31/2010 09:39 PM, Trip Volpe wrote:I recently began porting an existing C++ project of mine (a compiler/interpreter for a dynamic language) to D. In the process I found that the built-in unit testing support, while an awesome concept, was a little bit sparse. In particular, assert() is fairly useless for unit tests, since it throws on a failure and prevents all subsequent tests from running. Unfortunately, it's the only facility that seems to give you access to the current file and line number, very important bits of info for unit tests in a any non-trivial project. I was contemplating just abandoning D and sticking with C++ and Googletest, but after doing a bit of digging through the source, I found setAssertHandler() in core.exception. Perfect! I could use my own assert handler that just records the error and allows future asserts to still be reached. So I wrote it up and gave it a try, but pretty quickly started getting access violation errors. I did a bit of searching, and found a bug report on the issue: http://d.puremagic.com/issues/show_bug.cgi?id=3208 In the comments it's indicated that this is intentional, that the compiler expects the assertion handler to throw. So I guess I have a few questions: 1. Why is this the expected behavior? It seems to me that there are relatively few useful things you can do with a custom assert handler unless it is possible to refrain from throwing. 2. Is this going to be fixed any time soon? It seems fairly important to me; it really kills the value of built-in unit tests to be forced to choose between not having any line number information and being able to recognize only one assert failure per test run(!!!). 3. Are there any current workarounds for the problem? I could just use my own assertion function, but 1) I can't call it "assert" because that's a reserved word, and 2) most importantly, it won't have any way of indicating in its diagnostic output which source line the failure occurred on. In C++, of course, I could use preprocessor macros to do that, but D omits a preprocessor (for very good reasons). Unfortunately, this seems like a case where D hasn't provided replacement functionality. But maybe there's a sneaky fix I haven't thought of? I also found Runtime.moduleUnitTester() in core.runtime, which is very useful in itself, but provides an _almost_ useful improvement: I can catch the AssertErrors on a module-by-module basis, which would at least allow the unit tests for all modules to be run regardless of a failure in one of them. However, a single module could easily have dozens of tests in it, each of which could contain many individual asserts. Stopping the test early because one assert failed makes no sense. It would be _almost_ acceptable to at least be sure of running all the tests in each module; is there any way to poke inside or otherwise override ModuleInfo's unitTest() function? Thanks in advance for any advice or comments! D is a completely awesome language, but I found this issue a bit strange.You can use line and file info with default arguments, this works (special case). I just hack around the default unittest system, something like this: void test(string testName)(void delegate () testClosure, int line = __LINE__, string file = __FILE__) { // register test start, run testClosure, end test } void expectEquals(A, B)(A a, B b, int line = __LINE__, string file = __FILE__) { // record outcome of assertion } unittest { test!"truth of it"( { expectEquals(true, true); } ); } or slightly abusive: unittest { test!"truth of it" = { expectEquals(true, true); }; }
Jan 31 2010
"Lutger" <lutger.blijdestijn gmail.com> wrote in message news:hk4r35$1hpg$1 digitalmars.com...You can use line and file info with default arguments, this works (special case).That must be a D2 thing, because I know it doesn't work in D1.
Jan 31 2010
On 01/31/2010 10:04 PM, Nick Sabalausky wrote:"Lutger"<lutger.blijdestijn gmail.com> wrote in message news:hk4r35$1hpg$1 digitalmars.com...Indeed, here is the change: http://www.digitalmars.com/d/2.0/changelog.html#new2_013You can use line and file info with default arguments, this works (special case).That must be a D2 thing, because I know it doesn't work in D1.
Jan 31 2010
Lutger Wrote:You can use line and file info with default arguments, this works (special case). I just hack around the default unittest system, something like this: void test(string testName)(void delegate () testClosure, int line = __LINE__, string file = __FILE__) { // register test start, run testClosure, end test } void expectEquals(A, B)(A a, B b, int line = __LINE__, string file = __FILE__) { // record outcome of assertion }Ah ha! I din't know that D had __LINE__ and __FILE__. I thought those had been done away with along with the preprocessor. It looks like they're a lot smarter than in C as well, since this works just fine: test.d: 1 void main() { 2 printlocation(); // prints "test.d:2" (as opposed to "test.d:5") 3 } 4 5 void printlocation( string file = __FILE__, int line = __LINE__ ) { 6 writefln( "%s:%d", file, line ); 7 } This still feels a little more hacky than I'd like, but it should do. Thanks! :-)
Jan 31 2010
Trip Volpe:Ah ha! I din't know that D had __LINE__ and __FILE__. I thought those had been done away with along with the preprocessor. It looks like they're a lot smarter than in C as well, since this works just fine: test.d: 1 void main() { 2 printlocation(); // prints "test.d:2" (as opposed to "test.d:5") 3 } 4 5 void printlocation( string file = __FILE__, int line = __LINE__ ) { 6 writefln( "%s:%d", file, line ); 7 } This still feels a little more hacky than I'd like, but it should do. Thanks! :-)It looks like the bad special cased magic you can find in Perl :-) Bye, bearophile
Jan 31 2010
On 01/31/2010 09:39 PM, Trip Volpe wrote:I recently began porting an existing C++ project of mine (a compiler/interpreter for a dynamic language) to D. In the process I found that the built-in unit testing support, while an awesome concept, was a little bit sparse. In particular, assert() is fairly useless for unit tests, since it throws on a failure and prevents all subsequent tests from running. Unfortunately, it's the only facility that seems to give you access to the current file and line number, very important bits of info for unit tests in a any non-trivial project. I was contemplating just abandoning D and sticking with C++ and Googletest, but after doing a bit of digging through the source, I found setAssertHandler() in core.exception. Perfect! I could use my own assert handler that just records the error and allows future asserts to still be reached. So I wrote it up and gave it a try, but pretty quickly started getting access violation errors. I did a bit of searching, and found a bug report on the issue: http://d.puremagic.com/issues/show_bug.cgi?id=3208 In the comments it's indicated that this is intentional, that the compiler expects the assertion handler to throw. So I guess I have a few questions: 1. Why is this the expected behavior? It seems to me that there are relatively few useful things you can do with a custom assert handler unless it is possible to refrain from throwing. 2. Is this going to be fixed any time soon? It seems fairly important to me; it really kills the value of built-in unit tests to be forced to choose between not having any line number information and being able to recognize only one assert failure per test run(!!!). 3. Are there any current workarounds for the problem? I could just use my own assertion function, but 1) I can't call it "assert" because that's a reserved word, and 2) most importantly, it won't have any way of indicating in its diagnostic output which source line the failure occurred on. In C++, of course, I could use preprocessor macros to do that, but D omits a preprocessor (for very good reasons). Unfortunately, this seems like a case where D hasn't provided replacement functionality. But maybe there's a sneaky fix I haven't thought of? I also found Runtime.moduleUnitTester() in core.runtime, which is very useful in itself, but provides an _almost_ useful improvement: I can catch the AssertErrors on a module-by-module basis, which would at least allow the unit tests for all modules to be run regardless of a failure in one of them. However, a single module could easily have dozens of tests in it, each of which could contain many individual asserts. Stopping the test early because one assert failed makes no sense. It would be _almost_ acceptable to at least be sure of running all the tests in each module; is there any way to poke inside or otherwise override ModuleInfo's unitTest() function? Thanks in advance for any advice or comments! D is a completely awesome language, but I found this issue a bit strange.I think the reason for this is that assert is used by the compiler for some things, like assert(0) at the end of functions. An assertion handler should be created for use in unittests. Preferably with colorized output. :) I stole and changed slightly from std.contracts.enforce (untested) T test(T, string file=__FILE__, int line=__LINE__) (T value, string message="Test failed") { if (!value) writeln(file, "(", line, "): ", message); return value; } Use just like an assert, but will just write on errors.
Jan 31 2010
Pelle MÃ¥nsson Wrote:An assertion handler should be created for use in unittests. Preferably with colorized output. :) I stole and changed slightly from std.contracts.enforce (untested) T test(T, string file=__FILE__, int line=__LINE__) (T value, string message="Test failed") { if (!value) writeln(file, "(", line, "): ", message); return value; } Use just like an assert, but will just write on errors.Heh, yeah, if I had just noticed std.contracts in the first place, I would have realized that I still had __FILE__ and __LINE__ to play around with. Ah well -- D's usual thoughtfulness pulls through again! And I'm a sucker for bright colors too, so I'm building colorized output into my set of unit test augmentations. :-)
Jan 31 2010