www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CURL Wrapper: Congratulations Next up: std.serialize

reply "dsimcha" <dsimcha yahoo.com> writes:
By a vote of 14-0, Jonas Drewsen's CURL wrapper (std.net.curl) 
has been accepted into Phobos.  Thanks to Jonas for his hard work 
and his persistence through the multiple rounds of review that it 
took to get this module up to Phobos's high and increasing 
quality standard.

Keep the good work coming.  Next in line, if it's ready, is Jacob 
Carlborg's std.serialize.  Jacob, please post here when you've 
got something ready to go.
Dec 26 2011
next sibling parent "jdrewsen" <jdrewsen nospam.com> writes:
On Tuesday, 27 December 2011 at 02:01:51 UTC, dsimcha wrote:

 By a vote of 14-0, Jonas Drewsen's CURL wrapper (std.net.curl) 
 has been accepted into Phobos.  Thanks to Jonas for his hard 
 work and his persistence through the multiple rounds of review 
 that it took to get this module up to Phobos's high and 
 increasing quality standard.

 Keep the good work coming.  Next in line, if it's ready, is 
 Jacob Carlborg's std.serialize.  Jacob, please post here when 
 you've got something ready to go.


I'll go fix the last issues mentioned in the final review thread 
and the do a pull request.

Thanks to all who provided valuable feedback.
Dec 27 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-27 03:01, dsimcha wrote:

 By a vote of 14-0, Jonas Drewsen's CURL wrapper (std.net.curl) has been
 accepted into Phobos. Thanks to Jonas for his hard work and his
 persistence through the multiple rounds of review that it took to get
 this module up to Phobos's high and increasing quality standard.

 Keep the good work coming. Next in line, if it's ready, is Jacob
 Carlborg's std.serialize. Jacob, please post here when you've got
 something ready to go.


Project page (two tutorials): http://dsource.org/projects/orange
Repository: https://github.com/jacob-carlborg/orange

Documentation:

http://dl.dropbox.com/u/18386187/orange_docs/orange.serialization.Serializer.html

(Don't forget the "Package" tab)

Unit tests are available in the "tests" directory. These unit tests are 
not like regular unit tests that test individual functions. These unit 
tests are on a higher level.

The most important part to review is the "serialization" package. The 
rest is mostly utility modules.

Running the unit tests:
./unittest.sh

Use "make" to compile the library or create an executable using rdmd.

A few things to think about that need to be resolved:

* This is quite a large library and I really don't want to put it all 
into one module. I'm hoping it will be OK with a package

* I would really like to keep the unit tests in their own modules 
because they're quite large and the modules are already large without 
the unit tests in them

* The unit tests use a kind of mini-unit test framework. Should that be 
kept or removed?

Note:

The documentation is generate using D1, I don't think that should make a 
difference though.

-- 
/Jacob Carlborg
Dec 28 2011
next sibling parent reply "dsimcha" <dsimcha yahoo.com> writes:
On Wednesday, 28 December 2011 at 16:01:50 UTC, Jacob Carlborg 
wrote:

 Running the unit tests:
 ./unittest.sh

 Use "make" to compile the library or create an executable using 
 rdmd.

 A few things to think about that need to be resolved:

 * This is quite a large library and I really don't want to put 
 it all into one module. I'm hoping it will be OK with a package


So the package would be std.serialize?


 * I would really like to keep the unit tests in their own 
 modules because they're quite large and the modules are already 
 large without the unit tests in them


Sounds reasonable.  It goes against the Phobos convention, but it 
sounds like you have a good reason to.


 * The unit tests use a kind of mini-unit test framework. Should 
 that be kept or removed?


I haven't looked at it yet, but if it's generally useful, maybe 
it should be extracted and exposed as part of Phobos.  I'd say 
keep it for now but keep it private, and later make a proposal 
for a full review to make it a public, official part of Phobos.


 Note:

 The documentation is generate using D1, I don't think that 
 should make a difference though.
Dec 28 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-28 19:43, dsimcha wrote:

 On Wednesday, 28 December 2011 at 16:01:50 UTC, Jacob Carlborg wrote:

 Running the unit tests:
 ./unittest.sh

 Use "make" to compile the library or create an executable using rdmd.

 A few things to think about that need to be resolved:

 * This is quite a large library and I really don't want to put it all
 into one module. I'm hoping it will be OK with a package


 So the package would be std.serialize?


Yeah, or std.serialization.


 * I would really like to keep the unit tests in their own modules
 because they're quite large and the modules are already large without
 the unit tests in them


 Sounds reasonable. It goes against the Phobos convention, but it sounds
 like you have a good reason to.


 * The unit tests use a kind of mini-unit test framework. Should that
 be kept or removed?


 I haven't looked at it yet, but if it's generally useful, maybe it
 should be extracted and exposed as part of Phobos. I'd say keep it for
 now but keep it private, and later make a proposal for a full review to
 make it a public, official part of Phobos.


I think it is, don't know what others think. What it does is it catches 
AssertErrors so other unit tests can continue to run and then gives a 
nice report at the end.

-- 
/Jacob Carlborg
Dec 28 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, December 28, 2011 23:07:51 Jacob Carlborg wrote:

 I think it is, don't know what others think. What it does is it catches
 AssertErrors so other unit tests can continue to run and then gives a
 nice report at the end.


I'm against it. I think that the compiler/runtime should be fixed so that each 
unit test block is run in a module even if one fails. That would solve the 
problem quite nicely IMHO, and that's already _supposed_ to be how it works. 
It just isn't properly implemented in that regard yet. And I'm against 
unittest blocks running any code after a single failure. So, I don't think 
that any additional unit testing framework is necessary.

- Jonathan M Davis
Dec 28 2011
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-29 06:11, Jonathan M Davis wrote:

 On Wednesday, December 28, 2011 23:07:51 Jacob Carlborg wrote:

 I think it is, don't know what others think. What it does is it catches
 AssertErrors so other unit tests can continue to run and then gives a
 nice report at the end.


 I'm against it. I think that the compiler/runtime should be fixed so that each
 unit test block is run in a module even if one fails. That would solve the
 problem quite nicely IMHO, and that's already _supposed_ to be how it works.
 It just isn't properly implemented in that regard yet. And I'm against
 unittest blocks running any code after a single failure. So, I don't think
 that any additional unit testing framework is necessary.

 - Jonathan M Davis


Then the compiler need as well to collect all possible asserts and then 
somehow present them to the user. My library implementation already does 
this. Since this is completely implemented in library code it would be 
possible to have different formatters, a basic for outputting to the 
console and a more advanced with HTML output.

Less important but can be quite useful sometimes is that with my 
framework you add contexts to the unit tests, just like RSpec for those 
familiar with it.

If you haven't already, I suggest you take a look at the documentation 
for the unit test framework:

http://dl.dropbox.com/u/18386187/orange_docs/orange.test.UnitTester.html



For example:

import orange.test.UnitTester;

int sum (int x, int y)
{
     return x * y;
}

unittest ()
{
     describe("sum") in {
          it("should return the sum of the two given arguments") in {
               assert(sum(1, 2) == 3);
          }
     }
}

void main ()
{
     run;
}

If a test fails the framework will print out the context, the stack 
trace and a snippet from the failing test, something like this:

  sum
    - should return the sum of the given arguments

  Failures:
      1) sum should return the sum of the given arguments

         Stack trace:
         tango.core.Exception.AssertException main(44): Assertion failure


  describe("sum") in {
  	it("should return the sum of the given arguments") in {
  		assert(sum(1, 2) == 3);
  	};
  };

  1 test, 1 failure

-- 
/Jacob Carlborg
Dec 29 2011
parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Thursday, 29 December 2011 at 12:49:55 UTC, Jacob Carlborg 
wrote:

 For example:

 import orange.test.UnitTester;

 int sum (int x, int y)
 {
    return x * y;
 }

 unittest ()
 {
    describe("sum") in {
         it("should return the sum of the two given arguments") 
 in {
              assert(sum(1, 2) == 3);
         }
    }
 }

 void main ()
 {
    run;
 }

 If a test fails the framework will print out the context, the 
 stack trace and a snippet from the failing test, something like 
 this:

 sum
   - should return the sum of the given arguments

 Failures:
     1) sum should return the sum of the given arguments

        Stack trace:
        tango.core.Exception.AssertException main(44): Assertion 
 failure


 describe("sum") in {
 	it("should return the sum of the given arguments") in {
 		assert(sum(1, 2) == 3);
 	};
 };

 1 test, 1 failure


Operator overloading abuse, ahoy!
Dec 29 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-12-29 13:59, Jakob Ovrum wrote:

 On Thursday, 29 December 2011 at 12:49:55 UTC, Jacob Carlborg wrote:

 For example:

 import orange.test.UnitTester;

 int sum (int x, int y)
 {
 return x * y;
 }

 unittest ()
 {
 describe("sum") in {
 it("should return the sum of the two given arguments") in {
 assert(sum(1, 2) == 3);
 }
 }
 }

 void main ()
 {
 run;
 }

 If a test fails the framework will print out the context, the stack
 trace and a snippet from the failing test, something like this:

 sum
 - should return the sum of the given arguments

 Failures:
 1) sum should return the sum of the given arguments

 Stack trace:
 tango.core.Exception.AssertException main(44): Assertion failure


 describe("sum") in {
 it("should return the sum of the given arguments") in {
 assert(sum(1, 2) == 3);
 };
 };

 1 test, 1 failure


 Operator overloading abuse, ahoy!


Yeah, I know. The "describe" and "it" functions can be changed to take 
the delegate as an argument instead.

-- 
/Jacob Carlborg
Dec 29 2011
prev sibling parent reply Tobias Pankrath <tobias pankrath.net> writes:
When I first came to D, I read "unittest support" and thought that would 
be nice. But after I tried it, I realized it sucks and wrote something
similar to Jacobs unittest framework.


 I'm against it. I think that the compiler/runtime should be fixed so that
 each unit test block is run in a module even if one fails. That would
 solve the problem quite nicely IMHO, 


Which problem? That you can't get a good summary or backtrace? No 
descriptions of the unit test that failed? That other unit tests in the 
module will not run after an error? That you can't run some
specific unit-tests only?


 And
 I'm against unittest blocks running any code after a single failure. 


On the other hand, I think this is absolutely necessary, especially if
you have big modules. 

Imagine a high level unit-tests fails and you can't see the failure in the
low level helper functions that nails down the error. However every library
implemented unit test framework could just stop after the first failure, if
configured properly. Should not be the default though.


 So, I
 don't think that any additional unit testing framework is necessary.


I really think it is and will use one for my D code. Since both worlds
could live together peacefully there is absolutely no reason not to include
one in phobos.
Dec 30 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, December 30, 2011 13:41:37 Tobias Pankrath wrote:

 When I first came to D, I read "unittest support" and thought that would
 be nice. But after I tried it, I realized it sucks and wrote something
 similar to Jacobs unittest framework.
 

 I'm against it. I think that the compiler/runtime should be fixed so
 that
 each unit test block is run in a module even if one fails. That would
 solve the problem quite nicely IMHO,


 
 Which problem? That you can't get a good summary or backtrace? No
 descriptions of the unit test that failed? That other unit tests in the
 module will not run after an error? That you can't run some
 specific unit-tests only?


You get a proper error message backtrace on unit test failure. The only 
problem is that once a single unittest block within a module fails, no further 
unittest blocks within that module are run (unittest blocks from subsequent 
modules are run but no more from the one with the failure). As such, once you 
get a single failure within a module, you can't see any further failures. That 
_is_ a problem, but it's the only one that I'm aware of. And that needs to be 
fixed in the runtime, _not_ by adding more complex library support. It's a 
known issue which requires some additional support by the compiler (additional 
hooks of some kind IIRC) for the runtime to be fixed, but it is the runtime 
that should be fixed rather than trying to fix it with a library.


 And
 I'm against unittest blocks running any code after a single failure.


 
 On the other hand, I think this is absolutely necessary, especially if
 you have big modules.
 
 Imagine a high level unit-tests fails and you can't see the failure in the
 low level helper functions that nails down the error. However every library
 implemented unit test framework could just stop after the first failure, if
 configured properly. Should not be the default though.
 

 So, I
 don't think that any additional unit testing framework is necessary.


 
 I really think it is and will use one for my D code. Since both worlds
 could live together peacefully there is absolutely no reason not to include
 one in phobos.


It's one thing to use a fancier framework on your own. It's quite another to 
put it in the standard library. D's unit testing framework is designed to be 
straightforward and simple. On the whole, it does the job quite well. And once 
the issue of not running subsequent unittest blocks within a module after a 
failure in that module is fixed, I see no benefit from adding any additional 
library support. It just complicates things further.

- Jonathan M Davis
Dec 30 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-30 19:49, Jonathan M Davis wrote:

 On Friday, December 30, 2011 13:41:37 Tobias Pankrath wrote:

 I really think it is and will use one for my D code. Since both worlds
 could live together peacefully there is absolutely no reason not to include
 one in phobos.


 It's one thing to use a fancier framework on your own. It's quite another to
 put it in the standard library. D's unit testing framework is designed to be
 straightforward and simple. On the whole, it does the job quite well. And once
 the issue of not running subsequent unittest blocks within a module after a
 failure in that module is fixed, I see no benefit from adding any additional
 library support. It just complicates things further.

 - Jonathan M Davis


Will that be able to give a proper report of all failed tests in a nice 
format?

-- 
/Jacob Carlborg
Dec 30 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, December 30, 2011 21:38:07 Jacob Carlborg wrote:

 On 2011-12-30 19:49, Jonathan M Davis wrote:

 On Friday, December 30, 2011 13:41:37 Tobias Pankrath wrote:

 I really think it is and will use one for my D code. Since both worlds
 could live together peacefully there is absolutely no reason not to
 include one in phobos.


 
 It's one thing to use a fancier framework on your own. It's quite
 another to put it in the standard library. D's unit testing framework
 is designed to be straightforward and simple. On the whole, it does the
 job quite well. And once the issue of not running subsequent unittest
 blocks within a module after a failure in that module is fixed, I see
 no benefit from adding any additional library support. It just
 complicates things further.
 
 - Jonathan M Davis


 
 Will that be able to give a proper report of all failed tests in a nice
 format?


I think that the AssertError's message (which includes the file and line number 
of the failure) and its stack trace are plenty. It's exactly what you need and 
nothing else.

- Jonathan M Davis
Dec 30 2011
next sibling parent reply Tobias Pankrath <tobias pankrath.net> writes:
 
 I think that the AssertError's message (which includes the file and line
 number of the failure) and its stack trace are plenty. It's exactly what
 you need and nothing else.
 
 - Jonathan M Davis


I want to have such a summary. What's about running only certain unittests?
Dec 31 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the file and line
 number of the failure) and its stack trace are plenty. It's exactly what
 you need and nothing else.
 
 - Jonathan M Davis


 
 I want to have such a summary.


I don't see any reason to put that in the standard library. There's nothing 
wrong with 3rd party solutions which give additional functionality, but D's 
unit test framework is designed to be minimilistic, and I don't think that 
adding anything beyond what it does now in terms of summary makes any sense. 
The only major issue in that regard IMHO is the fact that no further unittest 
blocks within a module are run after one fails. Even if it did, I still don't 
think that a fancier summary would be worth having - especially in the 
standard library.

If you want that sort of summary, you probably want it printing stuff out on 
success too, and that definitely goes against how the built-in framework works 
(since it follows the typical unix approach of failure printing out stuff and 
success printing nothing). So, I think that that really makes more sense as a 
3rd party solution rather than as part of the standard library. And in 
general, 3rd party solutions are more likely to be customizable in a way you'd 
like rather than picking a single way of doing things.


 What's about running only certain unittests?


D's unit test framework isn't designed that way at this point. You need named 
unit tests for that to really make sense. It could theoretically be added and 
would be nice, but that would require changes to the language (though 
fortunately, they would be backwards compatible changes). So, we may see that 
eventually but not right now. At this point, the closest that you get to that 
is to unit test each of your modules separately rather than all at once. And 
actually, even major unit testing frameworks such as JUnit often end up 
running all of the unit tests within a module/file when you tell them to run a 
single unit test (probably in part since one unit test can theoretically affect 
the ones that follow it, and probably in part due to how they're implemented). 
So, I'm not sure how common being able to really run a single unit test is 
anyway. It would be a nice addition, and we may get it eventually, but it's 
not going to happen right now.

- Jonathan M Davis
Dec 31 2011
next sibling parent Tobias Pankrath <tobias pankrath.net> writes:
Jonathan M Davis wrote:


 On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the file and
 line number of the failure) and its stack trace are plenty. It's
 exactly what you need and nothing else.
 
 - Jonathan M Davis


 
 I want to have such a summary.


 
 I don't see any reason to put that in the standard library. 


I do see any reason not to put in the standard library. On the contrary:
not having one in the standard library prohibits phobos modules to
use an advanced solution.


 If you want that sort of summary, you probably want it printing stuff out
 on success too, and that definitely goes against how the built-in
 framework works (since it follows the typical unix approach of failure
 printing out stuff and success printing nothing). 


If I want to run the program only for the units tests, yes. But this does
not touch the questions, whether or not to put a unit test framework in
the stdlib.


 So, I think that that
 really makes more sense as a 3rd party solution rather than as part of the
 standard library. 


Since unit tests found their way in the language, I really see no reason,
why they are no fit for the standard lib.

What does a library qualifiy for phobos?


 And in general, 3rd party solutions are more likely to
 be customizable in a way you'd like rather than picking a single way of
 doing things.


Quality of other standard libraries shouldn't be an upper bound for phobos.


 D's unit test framework isn't designed that way at this point. 


I wouldn't say that D has an unit test framework right now. I has merely a 
way to run code at startup. 


 You need
 named unit tests for that to really make sense. It could theoretically be
 added and would be nice, but that would require changes to the language
 (though fortunately, they would be backwards compatible changes). So, we
 may see that eventually but not right now. At this point, the closest that
 you get to that is to unit test each of your modules separately rather
 than all at once. 


Thats my line, we are lacking features and we could have a good library 
solution for this.


 So, I'm not sure how common being
 able to really run a single unit test is anyway. 


I always run all of them and if one fails, I'll want to run those that 
failed to examine why they failed.

I agree that we shouldn't add complexitiy to the buildin language unit 
tests, apart from some additions that you mention. But we should define
a way, how an advanced unit test framework should operate with the
core feature and of course they should work together.

And since it is something many want to use (look at unit tests frameworks
in other languages), it does have its place in the stdlib.


PS: My newsreader (the KDE newsreader from kontact) seems to kill threading. 
Does anyone know how to change this without changing the newsreader?
Dec 31 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-31 11:37, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the file and line
 number of the failure) and its stack trace are plenty. It's exactly what
 you need and nothing else.

 - Jonathan M Davis


 I want to have such a summary.


 I don't see any reason to put that in the standard library. There's nothing
 wrong with 3rd party solutions which give additional functionality, but D's
 unit test framework is designed to be minimilistic, and I don't think that
 adding anything beyond what it does now in terms of summary makes any sense.
 The only major issue in that regard IMHO is the fact that no further unittest
 blocks within a module are run after one fails. Even if it did, I still don't
 think that a fancier summary would be worth having - especially in the
 standard library.

 If you want that sort of summary, you probably want it printing stuff out on
 success too, and that definitely goes against how the built-in framework works
 (since it follows the typical unix approach of failure printing out stuff and
 success printing nothing). So, I think that that really makes more sense as a
 3rd party solution rather than as part of the standard library. And in
 general, 3rd party solutions are more likely to be customizable in a way you'd
 like rather than picking a single way of doing things.


 What's about running only certain unittests?


 D's unit test framework isn't designed that way at this point. You need named
 unit tests for that to really make sense. It could theoretically be added and
 would be nice, but that would require changes to the language (though
 fortunately, they would be backwards compatible changes). So, we may see that
 eventually but not right now. At this point, the closest that you get to that
 is to unit test each of your modules separately rather than all at once. And
 actually, even major unit testing frameworks such as JUnit often end up
 running all of the unit tests within a module/file when you tell them to run a
 single unit test (probably in part since one unit test can theoretically affect
 the ones that follow it, and probably in part due to how they're implemented).
 So, I'm not sure how common being able to really run a single unit test is
 anyway. It would be a nice addition, and we may get it eventually, but it's
 not going to happen right now.

 - Jonathan M Davis


It would be possible to implement named unit tests only in library code. 
It would not have as nice syntax as if it was implemented in the 
language but still possible.

In Ruby on Rails I run single unit tests all the time. Why would I run 
all the unit tests, which can take five minutes, when I just can run one 
unit test and it takes just one second?

When your doing test/behavior driven development (T/BDD) it's certainly 
nice to be able to run single unit tests, because you run it all the time.

-- 
/Jacob Carlborg
Dec 31 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, December 31, 2011 16:04:12 Jacob Carlborg wrote:

 It would be possible to implement named unit tests only in library code.
 It would not have as nice syntax as if it was implemented in the
 language but still possible.
 
 In Ruby on Rails I run single unit tests all the time. Why would I run
 all the unit tests, which can take five minutes, when I just can run one
 unit test and it takes just one second?
 
 When your doing test/behavior driven development (T/BDD) it's certainly
 nice to be able to run single unit tests, because you run it all the time.


Yes. I agree that it would be nice, but for it to be done at all cleanly, the 
language, compiler, and druntime need to be improved to make it possible. 
However, at least syntactically, such changes should be completely backwards 
compatible, so they can be added at a future date. Regardless, I don't think 
that it's a problem that Phobos should be trying to solve.

- Jonathan M Davis
Dec 31 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-31 22:01, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 16:04:12 Jacob Carlborg wrote:

 It would be possible to implement named unit tests only in library code.
 It would not have as nice syntax as if it was implemented in the
 language but still possible.

 In Ruby on Rails I run single unit tests all the time. Why would I run
 all the unit tests, which can take five minutes, when I just can run one
 unit test and it takes just one second?

 When your doing test/behavior driven development (T/BDD) it's certainly
 nice to be able to run single unit tests, because you run it all the time.


 Yes. I agree that it would be nice, but for it to be done at all cleanly, the
 language, compiler, and druntime need to be improved to make it possible.
 However, at least syntactically, such changes should be completely backwards
 compatible, so they can be added at a future date. Regardless, I don't think
 that it's a problem that Phobos should be trying to solve.

 - Jonathan M Davis


Ok, if you would rather have all this in the language I would say no do 
that. But I know other people in the community that usually prefer to do 
a library solution if possible.

-- 
/Jacob Carlborg
Jan 01 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, January 01, 2012 15:35:00 Jacob Carlborg wrote:

 Ok, if you would rather have all this in the language I would say no do
 that. But I know other people in the community that usually prefer to do
 a library solution if possible.


If all we're talking about is named unit tests and running all of the unittest 
blocks within a module even if one fails, those aren't huge changes to the 
language. Both have been discussed before, and the only reason that the latter 
hasn't been implemented yet is that it requires changes to the compiler. So, I 
don't see any reason to make those a library solution. If we're talking 
something massively more complicated than that, then yes, a library solution 
starts making more sense. I also think that it then doesn't necessarily make 
sense to put it in the standard library.

If the issue is how the tests print out on failure, that could probably be 
done in the language as well, depending on what you were talking about 
changing. If you want to do something massively complicated, however, then 
you'd probably have to come up with a library solution, but again, I see no 
reason to have it in the standard library if you're doing anything fancy with 
how the test failures print out.

- Jonathan M Davis
Jan 01 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-01-02 00:14, Jonathan M Davis wrote:

 On Sunday, January 01, 2012 15:35:00 Jacob Carlborg wrote:

 Ok, if you would rather have all this in the language I would say no do
 that. But I know other people in the community that usually prefer to do
 a library solution if possible.


 If all we're talking about is named unit tests and running all of the unittest
 blocks within a module even if one fails, those aren't huge changes to the
 language. Both have been discussed before, and the only reason that the latter
 hasn't been implemented yet is that it requires changes to the compiler. So, I
 don't see any reason to make those a library solution. If we're talking
 something massively more complicated than that, then yes, a library solution
 starts making more sense. I also think that it then doesn't necessarily make
 sense to put it in the standard library.

 If the issue is how the tests print out on failure, that could probably be
 done in the language as well, depending on what you were talking about
 changing. If you want to do something massively complicated, however, then
 you'd probably have to come up with a library solution, but again, I see no
 reason to have it in the standard library if you're doing anything fancy with
 how the test failures print out.

 - Jonathan M Davis


I see it as three things I want done.

* Named unit tests
* Continue to run unit tests after a failed one
* A nice report at the end, preferably configurable

It would be nice if the first two were implemented in the language. If 
the third one is implemented in the runtime it could perhaps be 
configurable. One implements an interface, or similar, and can output 
the unit test report how he/she wants.

-- 
/Jacob Carlborg
Jan 01 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-31 11:37, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the file and line
 number of the failure) and its stack trace are plenty. It's exactly what
 you need and nothing else.

 - Jonathan M Davis


 I want to have such a summary.


 I don't see any reason to put that in the standard library. There's nothing
 wrong with 3rd party solutions which give additional functionality, but D's
 unit test framework is designed to be minimilistic, and I don't think that
 adding anything beyond what it does now in terms of summary makes any sense.
 The only major issue in that regard IMHO is the fact that no further unittest
 blocks within a module are run after one fails. Even if it did, I still don't
 think that a fancier summary would be worth having - especially in the
 standard library.


BTW, what would be so wrong if the unit tests for the standard library 
displayed a nice report when finished?

-- 
/Jacob Carlborg
Dec 31 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, December 31, 2011 16:06:49 Jacob Carlborg wrote:

 On 2011-12-31 11:37, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the file and
 line number of the failure) and its stack trace are plenty. It's
 exactly what you need and nothing else.
 
 - Jonathan M Davis


 
 I want to have such a summary.


 
 I don't see any reason to put that in the standard library. There's
 nothing wrong with 3rd party solutions which give additional
 functionality, but D's unit test framework is designed to be
 minimilistic, and I don't think that adding anything beyond what it
 does now in terms of summary makes any sense. The only major issue in
 that regard IMHO is the fact that no further unittest blocks within a
 module are run after one fails. Even if it did, I still don't think
 that a fancier summary would be worth having - especially in the
 standard library.


 
 BTW, what would be so wrong if the unit tests for the standard library
 displayed a nice report when finished?


My primary issue here is that I don't think that we should be adding stuff to 
Phobos which is essentially a new unit test framework on top of the built in 
one. If 3rd party stuff wants to do that. Fine. But the standard library should 
use the standard facilities. If the standard facilities aren't sufficient, then 
they should be improved.

As for a "nice report," I don't see anything wrong with just using the stack 
traces (which include the file, line number, and error message of the assertion 
failure). That's all the information that's needed. Anything else is 
superfluous IMHO. Now, if there were something nicer that could be generally 
agreed upon and added to druntime such that the standard unit test facilities 
used it, then fine. I don't see any point to it, but at least in that case, the 
standard library is still using the standard unit test framework. What I 
really don't want to see is Phobos essentially building a new unit test 
framework on top of the existing one. Any issues that need to be addressed 
with the unit test framework for the standard library should be addressed in 
the standard framework. Any additional framework stuff should be left to 3rd 
parties.

- Jonathan M Davis
Dec 31 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-31 21:56, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 16:06:49 Jacob Carlborg wrote:

 BTW, what would be so wrong if the unit tests for the standard library
 displayed a nice report when finished?


 My primary issue here is that I don't think that we should be adding stuff to
 Phobos which is essentially a new unit test framework on top of the built in
 one. If 3rd party stuff wants to do that. Fine. But the standard library should
 use the standard facilities. If the standard facilities aren't sufficient, then
 they should be improved.


First, that's how programming works. You build new abstractions on top 
of existing ones. Second, I would called the built in support for unit 
testing in D a "unit test framework". If add a unit test framework to 
Phobos it would be standard facilities and it wouldn't be a problem.


 As for a "nice report," I don't see anything wrong with just using the stack
 traces (which include the file, line number, and error message of the assertion
 failure). That's all the information that's needed. Anything else is
 superfluous IMHO. Now, if there were something nicer that could be generally
 agreed upon and added to druntime such that the standard unit test facilities
 used it, then fine. I don't see any point to it, but at least in that case, the
 standard library is still using the standard unit test framework. What I
 really don't want to see is Phobos essentially building a new unit test
 framework on top of the existing one. Any issues that need to be addressed
 with the unit test framework for the standard library should be addressed in
 the standard framework. Any additional framework stuff should be left to 3rd
 parties.

 - Jonathan M Davis


What's wrong with being able to run the unit tests from your editor, 
have the unit test framework output HTML (or similar), displayed in your 
editor and then you can click on links in the stack trace to get to the 
source code. If you don't see why that's useful that we can just end 
this discussion now.

-- 
/Jacob Carlborg
Jan 01 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, January 01, 2012 15:31:18 Jacob Carlborg wrote:

 What's wrong with being able to run the unit tests from your editor,
 have the unit test framework output HTML (or similar), displayed in your
 editor and then you can click on links in the stack trace to get to the
 source code. If you don't see why that's useful that we can just end
 this discussion now.


If you want fancier unit test facilities in your own code. Fine. I just don't 
think that they should be in the standard library. I think that on the whole, 
D's built-in unit test framework works fine as it is (barring a few tweaks such 
as making it so that all unittest blocks within a module run), and that 
further additions are a needless complication to the standard unit testing 
facilities and better left to 3rd party solutions where you can do whatever 
you want with them.

- Jonathan M Davis
Jan 01 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-01-02 00:28, Jonathan M Davis wrote:

 On Sunday, January 01, 2012 15:31:18 Jacob Carlborg wrote:

 What's wrong with being able to run the unit tests from your editor,
 have the unit test framework output HTML (or similar), displayed in your
 editor and then you can click on links in the stack trace to get to the
 source code. If you don't see why that's useful that we can just end
 this discussion now.


 If you want fancier unit test facilities in your own code. Fine. I just don't
 think that they should be in the standard library. I think that on the whole,
 D's built-in unit test framework works fine as it is (barring a few tweaks such
 as making it so that all unittest blocks within a module run), and that
 further additions are a needless complication to the standard unit testing
 facilities and better left to 3rd party solutions where you can do whatever
 you want with them.

 - Jonathan M Davis


I'm ending the discussion here.

-- 
/Jacob Carlborg
Jan 01 2012
prev sibling parent reply Andrew Wiley <wiley.andrew.j gmail.com> writes:
On Sat, Dec 31, 2011 at 2:56 PM, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Saturday, December 31, 2011 16:06:49 Jacob Carlborg wrote:

 On 2011-12-31 11:37, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the file and
 line number of the failure) and its stack trace are plenty. It's
 exactly what you need and nothing else.

 - Jonathan M Davis


 I want to have such a summary.


 I don't see any reason to put that in the standard library. There's
 nothing wrong with 3rd party solutions which give additional
 functionality, but D's unit test framework is designed to be
 minimilistic, and I don't think that adding anything beyond what it
 does now in terms of summary makes any sense. The only major issue in
 that regard IMHO is the fact that no further unittest blocks within a
 module are run after one fails. Even if it did, I still don't think
 that a fancier summary would be worth having - especially in the
 standard library.


 BTW, what would be so wrong if the unit tests for the standard library
 displayed a nice report when finished?


 My primary issue here is that I don't think that we should be adding stuff to
 Phobos which is essentially a new unit test framework on top of the built in
 one. If 3rd party stuff wants to do that. Fine. But the standard library should
 use the standard facilities. If the standard facilities aren't sufficient, then
 they should be improved.


The counterargument is that the language doesn't really provide a
framework - it actually provides anonymous parameterless global
functions that will be run before main is invoked if code is compiled
with -unittest. That isn't considered a framework in any language I've
ever used, but it adds just enough functionality to allow a
well-integrated fully-featured library solution. Would making such a
library solution part of the standard library really be a problem?
I'm mostly ambivalent on this issue because I haven't had time to look
closely at the proposed framework, but your argument seems to be that
all unittesting functionality needs to be built into the language. I
don't think that should be necessary or required.



 As for a "nice report," I don't see anything wrong with just using the stack
 traces (which include the file, line number, and error message of the assertion
 failure). That's all the information that's needed. Anything else is
 superfluous IMHO. Now, if there were something nicer that could be generally
 agreed upon and added to druntime such that the standard unit test facilities
 used it, then fine. I don't see any point to it, but at least in that case, the
 standard library is still using the standard unit test framework. What I
 really don't want to see is Phobos essentially building a new unit test
 framework on top of the existing one. Any issues that need to be addressed
 with the unit test framework for the standard library should be addressed in
 the standard framework. Any additional framework stuff should be left to 3rd
 parties.


As I said above, I wouldn't consider what we have to be a framework,
but it's definitely enough to build an excellent library solution on
top of.
As for a report, the problem is that an assertion error isn't what you
want when you're running, say, a continuous integration server (or,
say, a pull request tester). What you really want is a detailed
explanation of what unittests broke, what the tests were testing, and
how the result differed from what was expected. You want to be able to
have a reasonable idea of what went wrong *without* having to look at
someone else's code and figure out exactly what they're testing every
time.
Dec 31 2011
parent Jacob Carlborg <doob me.com> writes:
On 2012-01-01 01:57, Andrew Wiley wrote:

 On Sat, Dec 31, 2011 at 2:56 PM, Jonathan M Davis<jmdavisProg gmx.com>  wrote:

 On Saturday, December 31, 2011 16:06:49 Jacob Carlborg wrote:

 On 2011-12-31 11:37, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the file and
 line number of the failure) and its stack trace are plenty. It's
 exactly what you need and nothing else.

 - Jonathan M Davis


 I want to have such a summary.


 I don't see any reason to put that in the standard library. There's
 nothing wrong with 3rd party solutions which give additional
 functionality, but D's unit test framework is designed to be
 minimilistic, and I don't think that adding anything beyond what it
 does now in terms of summary makes any sense. The only major issue in
 that regard IMHO is the fact that no further unittest blocks within a
 module are run after one fails. Even if it did, I still don't think
 that a fancier summary would be worth having - especially in the
 standard library.


 BTW, what would be so wrong if the unit tests for the standard library
 displayed a nice report when finished?


 My primary issue here is that I don't think that we should be adding stuff to
 Phobos which is essentially a new unit test framework on top of the built in
 one. If 3rd party stuff wants to do that. Fine. But the standard library should
 use the standard facilities. If the standard facilities aren't sufficient, then
 they should be improved.


 The counterargument is that the language doesn't really provide a
 framework - it actually provides anonymous parameterless global
 functions that will be run before main is invoked if code is compiled
 with -unittest. That isn't considered a framework in any language I've
 ever used, but it adds just enough functionality to allow a
 well-integrated fully-featured library solution. Would making such a
 library solution part of the standard library really be a problem?
 I'm mostly ambivalent on this issue because I haven't had time to look
 closely at the proposed framework, but your argument seems to be that
 all unittesting functionality needs to be built into the language. I
 don't think that should be necessary or required.



 As for a "nice report," I don't see anything wrong with just using the stack
 traces (which include the file, line number, and error message of the assertion
 failure). That's all the information that's needed. Anything else is
 superfluous IMHO. Now, if there were something nicer that could be generally
 agreed upon and added to druntime such that the standard unit test facilities
 used it, then fine. I don't see any point to it, but at least in that case, the
 standard library is still using the standard unit test framework. What I
 really don't want to see is Phobos essentially building a new unit test
 framework on top of the existing one. Any issues that need to be addressed
 with the unit test framework for the standard library should be addressed in
 the standard framework. Any additional framework stuff should be left to 3rd
 parties.


 As I said above, I wouldn't consider what we have to be a framework,
 but it's definitely enough to build an excellent library solution on
 top of.
 As for a report, the problem is that an assertion error isn't what you
 want when you're running, say, a continuous integration server (or,
 say, a pull request tester). What you really want is a detailed
 explanation of what unittests broke, what the tests were testing, and
 how the result differed from what was expected. You want to be able to
 have a reasonable idea of what went wrong *without* having to look at
 someone else's code and figure out exactly what they're testing every
 time.


Exactly, I agree.

-- 
/Jacob Carlborg
Jan 01 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-31 04:35, Jonathan M Davis wrote:

 On Friday, December 30, 2011 21:38:07 Jacob Carlborg wrote:

 On 2011-12-30 19:49, Jonathan M Davis wrote:

 On Friday, December 30, 2011 13:41:37 Tobias Pankrath wrote:

 I really think it is and will use one for my D code. Since both worlds
 could live together peacefully there is absolutely no reason not to
 include one in phobos.


 It's one thing to use a fancier framework on your own. It's quite
 another to put it in the standard library. D's unit testing framework
 is designed to be straightforward and simple. On the whole, it does the
 job quite well. And once the issue of not running subsequent unittest
 blocks within a module after a failure in that module is fixed, I see
 no benefit from adding any additional library support. It just
 complicates things further.

 - Jonathan M Davis


 Will that be able to give a proper report of all failed tests in a nice
 format?


 I think that the AssertError's message (which includes the file and line number
 of the failure) and its stack trace are plenty. It's exactly what you need and
 nothing else.

 - Jonathan M Davis


Yes but what happens when there are many failed tests, i.e. may 
AssertErrors that have been thrown? It will just print all after each 
other and you have to count them yourself if you want to know how many 
failed tests there are?

-- 
/Jacob Carlborg
Dec 31 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, December 31, 2011 15:48:16 Jacob Carlborg wrote:

 Yes but what happens when there are many failed tests, i.e. may
 AssertErrors that have been thrown? It will just print all after each
 other and you have to count them yourself if you want to know how many
 failed tests there are?


What does the number of failures really matter? You just need to know which 
ones failed and where. The AssertErrors give you that.

- Jonathan M Davis
Dec 31 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-31 21:57, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 15:48:16 Jacob Carlborg wrote:

 Yes but what happens when there are many failed tests, i.e. may
 AssertErrors that have been thrown? It will just print all after each
 other and you have to count them yourself if you want to know how many
 failed tests there are?


 What does the number of failures really matter? You just need to know which
 ones failed and where. The AssertErrors give you that.

 - Jonathan M Davis


It can be nice for statistics. And again, how will it be displayed, just 
all AssertErrors after each other?

-- 
/Jacob Carlborg
Jan 01 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, January 01, 2012 15:32:34 Jacob Carlborg wrote:

 On 2011-12-31 21:57, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 15:48:16 Jacob Carlborg wrote:

 Yes but what happens when there are many failed tests, i.e. may
 AssertErrors that have been thrown? It will just print all after each
 other and you have to count them yourself if you want to know how many
 failed tests there are?


 
 What does the number of failures really matter? You just need to know
 which ones failed and where. The AssertErrors give you that.
 
 - Jonathan M Davis


 
 It can be nice for statistics. And again, how will it be displayed, just
 all AssertErrors after each other?


Yes.

- Jonathan M Davis
Jan 01 2012
prev sibling parent reply Brad Anderson <eco gnuk.net> writes:
On Wed, Dec 28, 2011 at 10:11 PM, Jonathan M Davis <jmdavisProg gmx.com>wrote:


 On Wednesday, December 28, 2011 23:07:51 Jacob Carlborg wrote:

 I think it is, don't know what others think. What it does is it catches
 AssertErrors so other unit tests can continue to run and then gives a
 nice report at the end.


 I'm against it. I think that the compiler/runtime should be fixed so that
 each
 unit test block is run in a module even if one fails. That would solve the
 problem quite nicely IMHO, and that's already _supposed_ to be how it
 works.
 It just isn't properly implemented in that regard yet. And I'm against
 unittest blocks running any code after a single failure. So, I don't think
 that any additional unit testing framework is necessary.

 - Jonathan M Davis


Forgive me if this is a silly question but a conversation in IRC got me
wondering if compiler could check for shared/__gshared use (and any other
thread unsafe operation) in each unittest and run those that aren't using
them concurrently? Obviously not all at once but at least more than one at
a time (perhaps set through user configuration).

Regards,
Brad
Dec 28 2011
parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Thursday, 29 December 2011 at 07:07:10 UTC, Brad Anderson 
wrote:

 Forgive me if this is a silly question but a conversation in 
 IRC got me
 wondering if compiler could check for shared/__gshared use (and 
 any other
 thread unsafe operation) in each unittest and run those that 
 aren't using
 them concurrently? Obviously not all at once but at least more 
 than one at
 a time (perhaps set through user configuration).

 Regards,
 Brad


The unittest code can ultimately call into both C and D code 
which source is not known. Such code would need to cause the 
unittest to be excluded from concurrent execution, in light of 
this I think it's questionable if such an analysis would be worth 
implementing. Perhaps look at some example unittest suites that 
take a long time and look at whether these examples can be proven 
to be thread-safe.
Dec 28 2011
prev sibling parent reply "jdrewsen" <jdrewsen nospam.com> writes:
On Wednesday, 28 December 2011 at 16:01:50 UTC, Jacob Carlborg 
wrote:

 On 2011-12-27 03:01, dsimcha wrote:

 By a vote of 14-0, Jonas Drewsen's CURL wrapper (std.net.curl) 
 has been
 accepted into Phobos. Thanks to Jonas for his hard work and his
 persistence through the multiple rounds of review that it took 
 to get
 this module up to Phobos's high and increasing quality 
 standard.

 Keep the good work coming. Next in line, if it's ready, is 
 Jacob
 Carlborg's std.serialize. Jacob, please post here when you've 
 got
 something ready to go.


 Project page (two tutorials): http://dsource.org/projects/orange
 Repository: https://github.com/jacob-carlborg/orange

 Documentation:

 http://dl.dropbox.com/u/18386187/orange_docs/orange.serialization.Serializer.html

 (Don't forget the "Package" tab)

 Unit tests are available in the "tests" directory. These unit 
 tests are not like regular unit tests that test individual 
 functions. These unit tests are on a higher level.

 The most important part to review is the "serialization" 
 package. The rest is mostly utility modules.


After I quick look:

I think that all other modules/packages under the orange package 
should either be integrated into other existing phobos modules or 
as new
phobos modules since they are really not serialization specific.

Furthermore I think that all suppport for tango in the code 
should be stripped before going into phobos.

/Jonas
Dec 28 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-28 22:19, jdrewsen wrote:

 On Wednesday, 28 December 2011 at 16:01:50 UTC, Jacob Carlborg wrote:

 On 2011-12-27 03:01, dsimcha wrote:

 By a vote of 14-0, Jonas Drewsen's CURL wrapper (std.net.curl) has been
 accepted into Phobos. Thanks to Jonas for his hard work and his
 persistence through the multiple rounds of review that it took to get
 this module up to Phobos's high and increasing quality standard.

 Keep the good work coming. Next in line, if it's ready, is Jacob
 Carlborg's std.serialize. Jacob, please post here when you've got
 something ready to go.


 Project page (two tutorials): http://dsource.org/projects/orange
 Repository: https://github.com/jacob-carlborg/orange

 Documentation:

 http://dl.dropbox.com/u/18386187/orange_docs/orange.serialization.Serializer.html


 (Don't forget the "Package" tab)

 Unit tests are available in the "tests" directory. These unit tests
 are not like regular unit tests that test individual functions. These
 unit tests are on a higher level.

 The most important part to review is the "serialization" package. The
 rest is mostly utility modules.


 After I quick look:

 I think that all other modules/packages under the orange package should
 either be integrated into other existing phobos modules or as new
 phobos modules since they are really not serialization specific.


I agree with that.


 Furthermore I think that all suppport for tango in the code should be
 stripped before going into phobos.

 /Jonas


That is obvious. I don't know if this have been clear or not but I've 
requested a pre-review. A review that should be as a regular review but 
before I do a Phobos integration. If it gets accepted into Phobos I'll 
do the necessary modifications (remove all Tango related code and so on) 
to integrate it into Phobos and then we can have a second review. I 
don't want to waste time on removing the Tango support if I don't know 
for sure that it will be accepted.

-- 
/Jacob Carlborg
Dec 28 2011
parent "jdrewsen" <jdrewsen nospam.com> writes:
On Wednesday, 28 December 2011 at 22:21:09 UTC, Jacob Carlborg 
wrote:

 On 2011-12-28 22:19, jdrewsen wrote:

 On Wednesday, 28 December 2011 at 16:01:50 UTC, Jacob Carlborg 
 wrote:

 On 2011-12-27 03:01, dsimcha wrote:

 By a vote of 14-0, Jonas Drewsen's CURL wrapper 
 (std.net.curl) has been
 accepted into Phobos. Thanks to Jonas for his hard work and 
 his
 persistence through the multiple rounds of review that it 
 took to get
 this module up to Phobos's high and increasing quality 
 standard.

 Keep the good work coming. Next in line, if it's ready, is 
 Jacob
 Carlborg's std.serialize. Jacob, please post here when 
 you've got
 something ready to go.


 Project page (two tutorials): 
 http://dsource.org/projects/orange
 Repository: https://github.com/jacob-carlborg/orange

 Documentation:

 http://dl.dropbox.com/u/18386187/orange_docs/orange.serialization.Serializer.html


 (Don't forget the "Package" tab)

 Unit tests are available in the "tests" directory. These unit 
 tests
 are not like regular unit tests that test individual 
 functions. These
 unit tests are on a higher level.

 The most important part to review is the "serialization" 
 package. The
 rest is mostly utility modules.


 After I quick look:

 I think that all other modules/packages under the orange 
 package should
 either be integrated into other existing phobos modules or as 
 new
 phobos modules since they are really not serialization 
 specific.


 I agree with that.


 Furthermore I think that all suppport for tango in the code 
 should be
 stripped before going into phobos.

 /Jonas


 That is obvious. I don't know if this have been clear or not 
 but I've requested a pre-review. A review that should be as a 
 regular review but before I do a Phobos integration. If it gets 
 accepted into Phobos I'll do the necessary modifications 
 (remove all Tango related code and so on) to integrate it into 
 Phobos and then we can have a second review. I don't want to 
 waste time on removing the Tango support if I don't know for 
 sure that it will be accepted.


Ok that wasn't clear to me. I've updated the review queue:

http://prowiki.org/wiki4d/wiki.cgi?ReviewQueue

/Jonas
Dec 29 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, December 29, 2011 00:06:57 Brad Anderson wrote:

 Forgive me if this is a silly question but a conversation in IRC got me
 wondering if compiler could check for shared/__gshared use (and any other
 thread unsafe operation) in each unittest and run those that aren't using
 them concurrently? Obviously not all at once but at least more than one at
 a time (perhaps set through user configuration).


That could result in non-deterministic failures. Granted, unittest blocks 
should usually be independent, but there's nothing that guarantees that they 
are, so running them in a non-derministic order could result in non-
deterministic failures.

Also, what about unit tests for stuff like std.pararellism or std.concurrency - 
anything that actually specifically uses threads in its implementation? Running 
tests in their own threads could cause issues with that.

Each module in Phobos has its own executable for running its unit tests, and 
there has been talk of making _those_ parallelizable, and I think that 
parallelizing things on that level makes a lot more sense than trying to build 
it into the language.

- Jonathan M Davis
Dec 28 2011
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-12-27 03:01, dsimcha wrote:

 By a vote of 14-0, Jonas Drewsen's CURL wrapper (std.net.curl) has been
 accepted into Phobos. Thanks to Jonas for his hard work and his
 persistence through the multiple rounds of review that it took to get
 this module up to Phobos's high and increasing quality standard.

 Keep the good work coming. Next in line, if it's ready, is Jacob
 Carlborg's std.serialize. Jacob, please post here when you've got
 something ready to go.


If we're going with my serialization library as the next item to review, 
should we have a review manager and a new thread?

-- 
/Jacob Carlborg
Dec 29 2011
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, December 31, 2011 18:57:14 Andrew Wiley wrote:

 On Sat, Dec 31, 2011 at 2:56 PM, Jonathan M Davis <jmdavisProg gmx.com> 


wrote:

 On Saturday, December 31, 2011 16:06:49 Jacob Carlborg wrote:

 On 2011-12-31 11:37, Jonathan M Davis wrote:

 On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:

 I think that the AssertError's message (which includes the
 file and
 line number of the failure) and its stack trace are plenty.
 It's
 exactly what you need and nothing else.
 
 - Jonathan M Davis


 
 I want to have such a summary.


 
 I don't see any reason to put that in the standard library.
 There's
 nothing wrong with 3rd party solutions which give additional
 functionality, but D's unit test framework is designed to be
 minimilistic, and I don't think that adding anything beyond what
 it
 does now in terms of summary makes any sense. The only major issue
 in
 that regard IMHO is the fact that no further unittest blocks
 within a
 module are run after one fails. Even if it did, I still don't
 think
 that a fancier summary would be worth having - especially in the
 standard library.


 
 BTW, what would be so wrong if the unit tests for the standard library
 displayed a nice report when finished?


 
 My primary issue here is that I don't think that we should be adding
 stuff to Phobos which is essentially a new unit test framework on top
 of the built in one. If 3rd party stuff wants to do that. Fine. But the
 standard library should use the standard facilities. If the standard
 facilities aren't sufficient, then they should be improved.


 
 The counterargument is that the language doesn't really provide a
 framework - it actually provides anonymous parameterless global
 functions that will be run before main is invoked if code is compiled
 with -unittest. That isn't considered a framework in any language I've
 ever used, but it adds just enough functionality to allow a
 well-integrated fully-featured library solution. Would making such a
 library solution part of the standard library really be a problem?
 I'm mostly ambivalent on this issue because I haven't had time to look
 closely at the proposed framework, but your argument seems to be that
 all unittesting functionality needs to be built into the language. I
 don't think that should be necessary or required.


For the most part, I don't think that we need anything more. It should be fixed 
so that all unittest blocks within a module are run even if one fails. This 
should be fixed in the compiler and runtime rather than having to hack it 
together with a library solution, since it's supposed to work that way in the 
first place but doesn't because of some changes that need to be made to the 
compiler (some hooks of some kind are needed IIRC). Named unit tests would 
also be nice, but it would be _far_ cleaner to add those to the language than 
try and add them with a library solution (e.g. unittest(test_name){}). It 
would also be nice if it were possible to hook in something that would run 
specific unit tests, but again, such hooks would then need to be provided by 
the compiler and/or runtime - unless you want to simply create a function for 
each unit test and have each unittest block call a specific function? That's an 
ugly solution IMHO.

All of those changes would be fairly minor in terms of how they affect how unit 
tests function and would be far cleaner IMHO to have in the language. I think 
that on some level, the unit test framework in the language has failed if we 
have to add library solutions on top of it to get such basic functionality.


 As for a "nice report," I don't see anything wrong with just using the
 stack traces (which include the file, line number, and error message of
 the assertion failure). That's all the information that's needed.
 Anything else is superfluous IMHO. Now, if there were something nicer
 that could be generally agreed upon and added to druntime such that the
 standard unit test facilities used it, then fine. I don't see any point
 to it, but at least in that case, the standard library is still using
 the standard unit test framework. What I really don't want to see is
 Phobos essentially building a new unit test framework on top of the
 existing one. Any issues that need to be addressed with the unit test
 framework for the standard library should be addressed in the standard
 framework. Any additional framework stuff should be left to 3rd
 parties.


 
 As I said above, I wouldn't consider what we have to be a framework,
 but it's definitely enough to build an excellent library solution on
 top of.
 As for a report, the problem is that an assertion error isn't what you
 want when you're running, say, a continuous integration server (or,
 say, a pull request tester). What you really want is a detailed
 explanation of what unittests broke, what the tests were testing, and
 how the result differed from what was expected. You want to be able to
 have a reasonable idea of what went wrong *without* having to look at
 someone else's code and figure out exactly what they're testing every
 time.


I don't really get what you'd need beyond knowing which test failed. The 
AssertErrors tell you that. It would definitely be nicer if the unittest blocks 
had proper names to go with them, but they're not necessary for knowing which 
test failed. The file and line number give you that. I don't see how you're 
going to know what actually failed unless you actually look at the test. If 
the tests have names and are well named, then that gives you a better idea of 
what isn't working before you look at the code, but ultimately, you have to 
look at the code to know what went wrong. And if you can name unittest blocks, 
then that problem is solved anyway, since then you'd have a name for the unit 
test failure to give in any report that you wanted to create.

- Jonathan M Davis
Dec 31 2011
parent Tobias Pankrath <tobias pankrath.net> writes:
Jonathan M Davis wrote:


 I think
 that on some level, the unit test framework in the language has failed if
 we have to add library solutions on top of it to get such basic
 functionality.


That's what we are saying: The unit test framework fails for us and a 
library solution is perfectly possible, even one that lets everyone use
the build in functionality if he wants.

You don't find it useful, but others do. I'd say: many others do. If 
something is considered useful by many it should have a place in the 
standard library.
Jan 01 2012