• Jonathan M Davis (27/27) Nov 05 2010 I'm proposing a possible new module for phobos which would be called
• Nick Sabalausky (32/76) Nov 05 2010 I made a very similar thing in my SemiTwist D Tools library, so I can
• Jonathan M Davis (25/107) Nov 06 2010 I don't really see the benefit of this. As it is, assertOpComp!() clearl...
• Nick Sabalausky (7/58) Nov 06 2010 I see, I didn't know that.
• retard (3/12) Nov 06 2010 I thought the keyword 'macro' was reserved for these purposes. Just wait...
• Jacob Carlborg (24/51) Nov 06 2010 I think that the problems that this module tries to fix should instead
• Masahiro Nakagawa (7/53) Nov 06 2010 I don't know this module's usability, but is getExcMsg useful?
• Jonathan M Davis (8/13) Nov 06 2010 It's used to test unittests.d itself, but I don't think that I'm using i...
• Nick Sabalausky (5/20) Nov 06 2010 I'm not suggesting getExcMsg be removed, but if you ever need to verify ...
• Jonathan M Davis (12/35) Nov 06 2010 The reason I created it originally IIRC was to verify stuff like the fac...
• Tomek =?UTF-8?B?U293acWEc2tp?= (7/25) Nov 06 2010 If the language can't help, helpers are definitely of value. In Java I r...

Jonathan M Davis <jmdavisProg gmx.com> writes:

I'm proposing a possible new module for phobos which would be called 
std.unittests. The code and DDoc file can be found here: http://is.gd/gLH9Q

Or you can look at the code here directly (though it has poor highlighting): 
http://ideone.com/EOlod

The module contains a set of helper functions for unit tests which are an 
improvement over simply using assert. I have them to be very helpful in the
unit 
tests in my proposal for std.datetime. My unit tests are more concise, easier
to 
read, and much easier to debug because of them. assertEqual() and 
assertExcThrown() have been particularly useful.

assert is a great tool, but simply telling you that an assertion failed is not 
as useful as telling you _how_ it failed, and a function like assertExcThrown() 
(which, as you may guess, asserts that a particular exception was thrown by a 
particular function call) really reduces the boiler plate code that you have to 
write for testing what it tests. So, I find such helper functions are extremely 
useful.

Because I have used them so extensively in std.datetime, they are pretty much 
either going to have to be added as a module to Phobos or added as private 
helper functions in std.datetime. Personally,  I think that they (or something 
very similar to them) merit being added as a new module in Phobos, which can be 
expanded upon as others come up with particularly useful unit testing functions.

So, I'm requesting that the folks here on the list have a look at my module and 
review it for possible inclusion in Phobos. I don't know what Andrei's position 
on them is (he rightly focused his review of my datetime code on the date/time 
functionality and API rather than its unit tests and didn't mention my unit 
testing functions at all), but I think that such a module would be of great 
worth and would like other's opinions on it.

- Jonathan M Davis

"Nick Sabalausky" <a a.a> writes:

"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.125.1289018234.21107.digitalmars-d puremagic.com...
 I'm proposing a possible new module for phobos which would be called
 std.unittests. The code and DDoc file can be found here: 
 http://is.gd/gLH9Q

 Or you can look at the code here directly (though it has poor 
 highlighting):
 http://ideone.com/EOlod

 The module contains a set of helper functions for unit tests which are an
 improvement over simply using assert. I have them to be very helpful in 
 the unit
 tests in my proposal for std.datetime. My unit tests are more concise, 
 easier to
 read, and much easier to debug because of them. assertEqual() and
 assertExcThrown() have been particularly useful.

 assert is a great tool, but simply telling you that an assertion failed is 
 not
 as useful as telling you _how_ it failed, and a function like 
 assertExcThrown()
 (which, as you may guess, asserts that a particular exception was thrown 
 by a
 particular function call) really reduces the boiler plate code that you 
 have to
 write for testing what it tests. So, I find such helper functions are 
 extremely
 useful.

 Because I have used them so extensively in std.datetime, they are pretty 
 much
 either going to have to be added as a module to Phobos or added as private
 helper functions in std.datetime. Personally,  I think that they (or 
 something
 very similar to them) merit being added as a new module in Phobos, which 
 can be
 expanded upon as others come up with particularly useful unit testing 
 functions.

 So, I'm requesting that the folks here on the list have a look at my 
 module and
 review it for possible inclusion in Phobos. I don't know what Andrei's 
 position
 on them is (he rightly focused his review of my datetime code on the 
 date/time
 functionality and API rather than its unit tests and didn't mention my 
 unit
 testing functions at all), but I think that such a module would be of 
 great
 worth and would like other's opinions on it.

I made a very similar thing in my SemiTwist D Tools library, so I can 
definitely attest to D's need for all of this.

My comments on this one, based on a breif skim of the doc:

I would try to change it so this:

assertOpCmp!("<")(myfunc(), 7, "My test failed!");

Can be written more like these:

assert!("_ < 7", "myfunc()", "My test failed!")();
assert!("_1 < _2", "myfunc()", "7", "My test failed!")();
assert!("_1 < _2", "myfunc()", "somethingElse()", "My test failed!")();

Aside from being more flexible, this way when it fails it can report much 
more relevent information that can actually compete with JUnit/NUnit. For 
instance, even with the custom message omitted, it could still give info 
like:

Failed: 'myfunc() < 7'
'myfunc()' was '13'

It may require it be used like 'mixin(assert!(blah blah))', but that's a big 
part of why I've been saying for a loooong time that it needs to be possible 
to make CTFE's intended for string mixins to be mixed in *without* the 
"mixin(...)" mess. Because as it is, if you present an interface that 
requires it be wrapped in "mixin(...)", people will bitch about the extra 
noise and refuse to use it, rendering string mixins minimally useful. (Last 
time I suggested this we got the ability to make *template* mixins that are 
usable without "mixin", which is caparitively useless.)

Also, I'd say if file/line info can't be inferred automatically because of 
varargs, then the interface needs to lose the vararg ability. It'd be nice 
to have, but not needing to provide line/file info manually is much nicer. 
Although, we could probably have both by using the suggestion above (ie, 
make the whole thing generate a string to be mixed-in).

Do the 'assertEqual', 'assertOpCmp', etc, all report if an exception was 
thrown (and if so, what exception)? If not, they should.

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Friday 05 November 2010 22:54:20 Nick Sabalausky wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.125.1289018234.21107.digitalmars-d puremagic.com...
 
 I'm proposing a possible new module for phobos which would be called
 std.unittests. The code and DDoc file can be found here:
 http://is.gd/gLH9Q
 
 Or you can look at the code here directly (though it has poor
 highlighting):
 http://ideone.com/EOlod
 
 The module contains a set of helper functions for unit tests which are an
 improvement over simply using assert. I have them to be very helpful in
 the unit
 tests in my proposal for std.datetime. My unit tests are more concise,
 easier to
 read, and much easier to debug because of them. assertEqual() and
 assertExcThrown() have been particularly useful.
 
 assert is a great tool, but simply telling you that an assertion failed
 is not
 as useful as telling you _how_ it failed, and a function like
 assertExcThrown()
 (which, as you may guess, asserts that a particular exception was thrown
 by a
 particular function call) really reduces the boiler plate code that you
 have to
 write for testing what it tests. So, I find such helper functions are
 extremely
 useful.
 
 Because I have used them so extensively in std.datetime, they are pretty
 much
 either going to have to be added as a module to Phobos or added as
 private helper functions in std.datetime. Personally,  I think that they
 (or something
 very similar to them) merit being added as a new module in Phobos, which
 can be
 expanded upon as others come up with particularly useful unit testing
 functions.
 
 So, I'm requesting that the folks here on the list have a look at my
 module and
 review it for possible inclusion in Phobos. I don't know what Andrei's
 position
 on them is (he rightly focused his review of my datetime code on the
 date/time
 functionality and API rather than its unit tests and didn't mention my
 unit
 testing functions at all), but I think that such a module would be of
 great
 worth and would like other's opinions on it.

 
 I made a very similar thing in my SemiTwist D Tools library, so I can
 definitely attest to D's need for all of this.
 
 My comments on this one, based on a breif skim of the doc:
 
 I would try to change it so this:
 
 assertOpCmp!("<")(myfunc(), 7, "My test failed!");
 
 Can be written more like these:
 
 assert!("_ < 7", "myfunc()", "My test failed!")();
 assert!("_1 < _2", "myfunc()", "7", "My test failed!")();
 assert!("_1 < _2", "myfunc()", "somethingElse()", "My test failed!")();
 
 Aside from being more flexible, this way when it fails it can report much
 more relevent information that can actually compete with JUnit/NUnit. For
 instance, even with the custom message omitted, it could still give info
 like:
 
 Failed: 'myfunc() < 7'
 'myfunc()' was '13'

I don't really see the benefit of this. As it is, assertOpComp!() clearly tells 
you what the actual result was and what the values were. It also reports the
file 
and line number of the failed test. What more do you need? Seeing the exact 
expressions that resulted in the values to be tested doesn't strike me as 
necessary. You have the file and line number. You know what the exact result
was. 
You can look at that exact line to see what produced the values that were 
tested. It seems to me like you have all of the information you need.

 Also, I'd say if file/line info can't be inferred automatically because of
 varargs, then the interface needs to lose the vararg ability. It'd be nice
 to have, but not needing to provide line/file info manually is much nicer.
 Although, we could probably have both by using the suggestion above (ie,
 make the whole thing generate a string to be mixed-in).

We need the ability to call arbitrary functions with some of the unit test 
functions (the ones where you have to pass LineInfo()). I don't see how you 
could do that with a string mixin unless you made the test itself a mixin 
instead of a function call, which strikes me as quite unpleasant. I'm certainly 
up for suggestions, but what I have is the best solution that I've seen. It is 
rather annoying to have to pass LineInfo() as the first argument, but it's 
straightforward. It works. And it's not like you can forget to do it, since it 
won't compile if you do.

 Do the 'assertEqual', 'assertOpCmp', etc, all report if an exception was
 thrown (and if so, what exception)? If not, they should.

They throw just like assert would. Even assertExcThrown() throws the exception 
if it's not the type of exception that you were testing for. I see no benefit
in 
having them catch stray exceptions. That's not what they're testing for, and I 
believe that just like with assert, if a unit test function fails, that should 
be the end of that unit test. The exception gets reported just as it would be 
with an assert that threw an exception. So, it seems to me to be the correct 
behavior. The tests test for exactly what you tell them to and let other 
exceptions through to be reported by the runtime.

- Jonathan M Davis

"Nick Sabalausky" <a a.a> writes:

"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.129.1289031502.21107.digitalmars-d puremagic.com...
 On Friday 05 November 2010 22:54:20 Nick Sabalausky wrote:
 I made a very similar thing in my SemiTwist D Tools library, so I can
 definitely attest to D's need for all of this.

 My comments on this one, based on a breif skim of the doc:

 I would try to change it so this:

 assertOpCmp!("<")(myfunc(), 7, "My test failed!");

 Can be written more like these:

 assert!("_ < 7", "myfunc()", "My test failed!")();
 assert!("_1 < _2", "myfunc()", "7", "My test failed!")();
 assert!("_1 < _2", "myfunc()", "somethingElse()", "My test failed!")();

 Aside from being more flexible, this way when it fails it can report much
 more relevent information that can actually compete with JUnit/NUnit. For
 instance, even with the custom message omitted, it could still give info
 like:

 Failed: 'myfunc() < 7'
 'myfunc()' was '13'

 I don't really see the benefit of this. As it is, assertOpComp!() clearly 
 tells
 you what the actual result was and what the values were.

I see, I didn't know that.


 Also, I'd say if file/line info can't be inferred automatically because 
 of
 varargs, then the interface needs to lose the vararg ability. It'd be 
 nice
 to have, but not needing to provide line/file info manually is much 
 nicer.
 Although, we could probably have both by using the suggestion above (ie,
 make the whole thing generate a string to be mixed-in).

 We need the ability to call arbitrary functions with some of the unit test
 functions (the ones where you have to pass LineInfo()). I don't see how 
 you
 could do that with a string mixin unless you made the test itself a mixin
 instead of a function call, which strikes me as quite unpleasant.

No more unpleasant than LineInfo(), and it also opens the door for other 
improvements.


 Do the 'assertEqual', 'assertOpCmp', etc, all report if an exception was
 thrown (and if so, what exception)? If not, they should.

 They throw just like assert would. Even assertExcThrown() throws the 
 exception
 if it's not the type of exception that you were testing for. I see no 
 benefit in
 having them catch stray exceptions. That's not what they're testing for, 
 and I
 believe that just like with assert, if a unit test function fails, that 
 should
 be the end of that unit test.

STRONGLY disagree here. (Mentioned more about it in a different branch of 
this thread.)

retard <re tard.com.invalid> writes:

Sat, 06 Nov 2010 01:54:20 -0400, Nick Sabalausky wrote:

 It may require it be used like 'mixin(assert!(blah blah))', but that's a
 big part of why I've been saying for a loooong time that it needs to be
 possible to make CTFE's intended for string mixins to be mixed in
 *without* the "mixin(...)" mess. Because as it is, if you present an
 interface that requires it be wrapped in "mixin(...)", people will bitch
 about the extra noise and refuse to use it, rendering string mixins
 minimally useful. (Last time I suggested this we got the ability to make
 *template* mixins that are usable without "mixin", which is caparitively
 useless.)

I thought the keyword 'macro' was reserved for these purposes. Just wait 
for D3.. It's possible then.

Jacob Carlborg <doob me.com> writes:

On 2010-11-06 05:37, Jonathan M Davis wrote:
 I'm proposing a possible new module for phobos which would be called
 std.unittests. The code and DDoc file can be found here: http://is.gd/gLH9Q

 Or you can look at the code here directly (though it has poor highlighting):
 http://ideone.com/EOlod

 The module contains a set of helper functions for unit tests which are an
 improvement over simply using assert. I have them to be very helpful in the
unit
 tests in my proposal for std.datetime. My unit tests are more concise, easier
to
 read, and much easier to debug because of them. assertEqual() and
 assertExcThrown() have been particularly useful.

 assert is a great tool, but simply telling you that an assertion failed is not
 as useful as telling you _how_ it failed, and a function like assertExcThrown()
 (which, as you may guess, asserts that a particular exception was thrown by a
 particular function call) really reduces the boiler plate code that you have to
 write for testing what it tests. So, I find such helper functions are extremely
 useful.

 Because I have used them so extensively in std.datetime, they are pretty much
 either going to have to be added as a module to Phobos or added as private
 helper functions in std.datetime. Personally,  I think that they (or something
 very similar to them) merit being added as a new module in Phobos, which can be
 expanded upon as others come up with particularly useful unit testing
functions.

 So, I'm requesting that the folks here on the list have a look at my module and
 review it for possible inclusion in Phobos. I don't know what Andrei's position
 on them is (he rightly focused his review of my datetime code on the date/time
 functionality and API rather than its unit tests and didn't mention my unit
 testing functions at all), but I think that such a module would be of great
 worth and would like other's opinions on it.

 - Jonathan M Davis

I think that the problems that this module tries to fix should instead 
properly be fixed in the compiler. An assert error should be able to 
show what expression failed and one failing unit test in a module should 
now stop the renaming unit tests. The failing unit test would of course 
stop running but not the renaming ones.

An idea is to have a function that takes a delegate and wrap the whole 
test in the delegate, like this:

unittest
{
     test("possibly a message here as well", {
         assert(/*some condition*/);
     });
}

"test" would catch all AssertErrors, collect them and then at the end 
print out the result of all the tests. If one assert throws in a 
delegate the whole test (delegate) will stop running but the renaming 
tests in the unit test will continue to run. Since the assert error will 
contain the file and line number one could also read in the file and 
print out the failing line and possibly a couple of surrounding lines.

Of course it would want the compiler to do this automatically instead of 
inventing a library solution.

-- 
/Jacob Carlborg

"Masahiro Nakagawa" <repeatedly gmail.com> writes:

On Sat, 06 Nov 2010 13:37:02 +0900, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 I'm proposing a possible new module for phobos which would be called
 std.unittests. The code and DDoc file can be found here:  
 http://is.gd/gLH9Q

 Or you can look at the code here directly (though it has poor  
 highlighting):
 http://ideone.com/EOlod

 The module contains a set of helper functions for unit tests which are an
 improvement over simply using assert. I have them to be very helpful in  
 the unit
 tests in my proposal for std.datetime. My unit tests are more concise,  
 easier to
 read, and much easier to debug because of them. assertEqual() and
 assertExcThrown() have been particularly useful.

 assert is a great tool, but simply telling you that an assertion failed  
 is not
 as useful as telling you _how_ it failed, and a function like  
 assertExcThrown()
 (which, as you may guess, asserts that a particular exception was thrown  
 by a
 particular function call) really reduces the boiler plate code that you  
 have to
 write for testing what it tests. So, I find such helper functions are  
 extremely
 useful.

 Because I have used them so extensively in std.datetime, they are pretty  
 much
 either going to have to be added as a module to Phobos or added as  
 private
 helper functions in std.datetime. Personally,  I think that they (or  
 something
 very similar to them) merit being added as a new module in Phobos, which  
 can be
 expanded upon as others come up with particularly useful unit testing  
 functions.

 So, I'm requesting that the folks here on the list have a look at my  
 module and
 review it for possible inclusion in Phobos. I don't know what Andrei's  
 position
 on them is (he rightly focused his review of my datetime code on the  
 date/time
 functionality and API rather than its unit tests and didn't mention my  
 unit
 testing functions at all), but I think that such a module would be of  
 great
 worth and would like other's opinions on it.

 - Jonathan M Davis

I don't know this module's usability, but is getExcMsg useful?
I can't imagine useful situation.
In addition, I grepped your datetime.d and getExcMsg not found :(

If possible, I want the mock, stub and related features.


Masahiro

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Saturday 06 November 2010 12:28:02 Masahiro Nakagawa wrote:
 I don't know this module's usability, but is getExcMsg useful?
 I can't imagine useful situation.
 In addition, I grepped your datetime.d and getExcMsg not found :(
 
 If possible, I want the mock, stub and related features.

It's used to test unittests.d itself, but I don't think that I'm using it in 
datetime.d at the moment. The idea is to use it if you want to verify that an 
exception says exactly what you want it to say (especially if you could get 
multiple exceptions of the same type but with different messages from a 
function). I will grant you, however, that it's one of the less useful 
functions.

- Jonathan M Davis

"Nick Sabalausky" <a a.a> writes:

"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.138.1289084125.21107.digitalmars-d puremagic.com...
 On Saturday 06 November 2010 12:28:02 Masahiro Nakagawa wrote:
 I don't know this module's usability, but is getExcMsg useful?
 I can't imagine useful situation.
 In addition, I grepped your datetime.d and getExcMsg not found :(

 If possible, I want the mock, stub and related features.

 It's used to test unittests.d itself, but I don't think that I'm using it 
 in
 datetime.d at the moment. The idea is to use it if you want to verify that 
 an
 exception says exactly what you want it to say (especially if you could 
 get
 multiple exceptions of the same type but with different messages from a
 function). I will grant you, however, that it's one of the less useful
 functions.

I'm not suggesting getExcMsg be removed, but if you ever need to verify an 
exception's message is exactly matches a specific string, then you're almost 
certainly using exceptions the wrong way to begin with.

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Saturday 06 November 2010 18:35:29 Nick Sabalausky wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.138.1289084125.21107.digitalmars-d puremagic.com...
 
 On Saturday 06 November 2010 12:28:02 Masahiro Nakagawa wrote:
 I don't know this module's usability, but is getExcMsg useful?
 I can't imagine useful situation.
 In addition, I grepped your datetime.d and getExcMsg not found :(
 
 If possible, I want the mock, stub and related features.

 
 It's used to test unittests.d itself, but I don't think that I'm using it
 in
 datetime.d at the moment. The idea is to use it if you want to verify
 that an
 exception says exactly what you want it to say (especially if you could
 get
 multiple exceptions of the same type but with different messages from a
 function). I will grant you, however, that it's one of the less useful
 functions.

 
 I'm not suggesting getExcMsg be removed, but if you ever need to verify an
 exception's message is exactly matches a specific string, then you're
 almost certainly using exceptions the wrong way to begin with.

The reason I created it originally IIRC was to verify stuff like the fact that 
passing an invalid hour to TimeOfDay complained about the hour being incorrect 
rather than the minute or the second. It _is_ arguably error-prone, because
it's 
testing the _exact_ error message, but if you're trying to verify that the 
correct exception was thrown and multiple exceptions of the same type can be 
thrown from a function, I don't really see any other way to do it. I ended up 
not using it in std.datetime (probably because I consolidated most of the code 
which threw DateTimeExceptions, so there was less need for it), but I do think 
that it's at least potentially useful if you want to verify that your
exceptions 
have the correct error messages.

- Jonathan M Davis

Tomek =?UTF-8?B?U293acWEc2tp?= <just ask.me> writes:

Jonathan M Davis napisał:

 I'm proposing a possible new module for phobos which would be called
 std.unittests. The code and DDoc file can be found here:
 http://is.gd/gLH9Q
 
 Or you can look at the code here directly (though it has poor
 highlighting): http://ideone.com/EOlod
 
 The module contains a set of helper functions for unit tests which are an
 improvement over simply using assert. I have them to be very helpful in
 the unit tests in my proposal for std.datetime. My unit tests are more
 concise, easier to read, and much easier to debug because of them.
 assertEqual() and assertExcThrown() have been particularly useful.

If the language can't help, helpers are definitely of value. In Java I rarely
use the built-in assert statement and resort to JUnit 
functions where possible.

 assert is a great tool, but simply telling you that an assertion failed is
 not as useful as telling you _how_ it failed, and a function like
 assertExcThrown() (which, as you may guess, asserts that a particular
 exception was thrown by a particular function call) really reduces the
 boiler plate code that you have to write for testing what it tests. So, I
 find such helper functions are extremely useful.

From a different angle: if assert is already part of the language, and the
compiler has access to the AST of the tested 
expression, perhaps it could also automatically generate code that
pretty-prints the expression?

-- 
Tomek