www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: Showing unittest in documentation (Was Re: std.unittests

reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:


 On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:
 

 On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:

 On 1/24/11 1:50 PM, Jens Mueller wrote:

 Jonathan M Davis wrote:

 I think that it's been discussed a time or two, but nothing has been
 done about
 it. It wouldn't be entirely straightforward to do. Essentially,  
 either a
 unittest block would have to be generated from the Examples section
 in the
 documentation, or you'd have to have some way to indicate that a
 particular
 unittest block got put into the documentation as an Examples section.
 It's
 certainly true that it would be ideal to have a way to avoid the
 duplication,
 but we don't have one at the moment, and it hasn't yet been a high
 enough
 priority to sort out how to do it and implement it.


 I see. I understand that it does not have high priority. Just wondered
 whether ...

 Jens


 The change is much simpler than what Jonathan suggests. A change can be
 made such that any unittest preceded by a documentation comment is
 automatically considered an example.

 /**
 Example:
 */
 unittest
 {
 writeln("This is how it works.");
 }


 Andrei


 BTW I consider this a very important topic. We have _plenty_ of examples  
 that don't work and are not mechanically verifiable. The reasons range  
 from minor typos to language changes to implementation limitations.  
 Generally this is what they call "documentation rot". This is terrible  
 PR for the language.

 Changing ddoc to recognize documentation unittests would fix this matter  
 once and forever.

 Last but not least, the "----" separators for code samples are awful  
 because no editor recognizes them for anything - they confuse the hell  
 out of Emacs for one thing.


 
 This only makes sense if:
 
 1. The unit test immediately follows the item being documented
 2. The unit test *only* tests that item.
 
 The second one could be pretty annoying.  Consider cases where several  
 functions interact (I've seen this many times on Microsoft's  
 Documentation), and it makes sense to make one example that covers all of  
 them.  Having them 'testable' means creating several identical unit tests.
 
 One way to easily fix this is to allow an additional parameter to the  
 comment:
 
 /**
 Example(Foo.foo(int), Foo.bar(int)):
 */
 unittest
 {
     auto foo = new Foo;
     foo.foo(5);
     foo.bar(6);
     assert(foo.toString() == "bazunga!");
 }
 
 The above means, copy the example to both Foo.foo(int) and Foo.bar(int)
 
 An alternative that is more verbose, but probably more understandable:
 
 /**
 Example:
 Covers Foo.foo(int)
 Covers Foo.bar(int)
 */
 
 Of course, a lack of target just means it applies to the item just  
 documented.
 
 One other thing, using writefln is considered bad form in unit tests (you  
 want *no* output if the unit test works).  But many examples might want to  
 demonstrate how e.g. an object interacts with writefln.  Any suggestions?   
 The assert line above is not very pretty for example...
 
 -Steve


Unit-tests are defined on a module level, not a function level, hence I would
expect to see the unit-tests that serve as examples to appear in an examples
section on the page the documents the module itself. In the online docs I would
expect the function names used in the example to be links to the individual
function doc and for each function have a list of links to examples it
participated in. 
This should be automatic and the user shouldn't need to provide the "list of
functions documented in this example". 

Just my two cents..
Jan 24 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 24 January 2011 15:27:08 foobar wrote:

 Steven Schveighoffer Wrote:

 On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu
 
 <SeeWebsiteForEmail erdani.org> wrote:

 On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:

 On 1/24/11 1:50 PM, Jens Mueller wrote:

 Jonathan M Davis wrote:

 I think that it's been discussed a time or two, but nothing has been
 done about
 it. It wouldn't be entirely straightforward to do. Essentially,
 either a
 unittest block would have to be generated from the Examples section
 in the
 documentation, or you'd have to have some way to indicate that a
 particular
 unittest block got put into the documentation as an Examples
 section. It's
 certainly true that it would be ideal to have a way to avoid the
 duplication,
 but we don't have one at the moment, and it hasn't yet been a high
 enough
 priority to sort out how to do it and implement it.


 
 I see. I understand that it does not have high priority. Just
 wondered whether ...
 
 Jens


 
 The change is much simpler than what Jonathan suggests. A change can
 be made such that any unittest preceded by a documentation comment is
 automatically considered an example.
 
 /**
 Example:
 */
 unittest
 {
 writeln("This is how it works.");
 }
 
 
 Andrei


 
 BTW I consider this a very important topic. We have _plenty_ of
 examples that don't work and are not mechanically verifiable. The
 reasons range from minor typos to language changes to implementation
 limitations. Generally this is what they call "documentation rot".
 This is terrible PR for the language.
 
 Changing ddoc to recognize documentation unittests would fix this
 matter once and forever.
 
 Last but not least, the "----" separators for code samples are awful
 because no editor recognizes them for anything - they confuse the hell
 out of Emacs for one thing.


 
 This only makes sense if:
 
 1. The unit test immediately follows the item being documented
 2. The unit test *only* tests that item.
 
 The second one could be pretty annoying.  Consider cases where several
 functions interact (I've seen this many times on Microsoft's
 Documentation), and it makes sense to make one example that covers all of
 them.  Having them 'testable' means creating several identical unit
 tests.
 
 One way to easily fix this is to allow an additional parameter to the
 comment:
 
 /**
 Example(Foo.foo(int), Foo.bar(int)):
 */
 unittest
 {
 
     auto foo = new Foo;
     foo.foo(5);
     foo.bar(6);
     assert(foo.toString() == "bazunga!");
 
 }
 
 The above means, copy the example to both Foo.foo(int) and Foo.bar(int)
 
 An alternative that is more verbose, but probably more understandable:
 
 /**
 Example:
 Covers Foo.foo(int)
 Covers Foo.bar(int)
 */
 
 Of course, a lack of target just means it applies to the item just
 documented.
 
 One other thing, using writefln is considered bad form in unit tests (you
 want *no* output if the unit test works).  But many examples might want
 to demonstrate how e.g. an object interacts with writefln.  Any
 suggestions? The assert line above is not very pretty for example...
 
 -Steve


 
 Unit-tests are defined on a module level, not a function level, hence I
 would expect to see the unit-tests that serve as examples to appear in an
 examples section on the page the documents the module itself. In the
 online docs I would expect the function names used in the example to be
 links to the individual function doc and for each function have a list of
 links to examples it participated in. This should be automatic and the
 user shouldn't need to provide the "list of functions documented in this
 example".
 
 Just my two cents..


Examples almost always go with functions. The whole point is to avoid having to 
have the function examples both in the documentation and in a unittest block.
It 
would be a huge mess to try and put all of the examples in the module 
documentation. I shudder to think what that would look like for std.algorithm - 
or even worse (_far_ worse), std.datetime. The simple syntax of

/++ Example +/
unittest
{
}


making that unittest block go in the documentation of the preceding function 
should work just fine. We already have /++ Ditto +/ which puts a function in
with 
the preceding function's documentation. So, having /++ Example +/ on top of
that 
isn't much of a stretch.

- Jonathan M Davis
Jan 24 2011
parent reply foobar <foo bar.com> writes:
Jonathan M Davis Wrote:


 On Monday 24 January 2011 15:27:08 foobar wrote:


<snip>

 
 Unit-tests are defined on a module level, not a function level, hence I
 would expect to see the unit-tests that serve as examples to appear in an
 examples section on the page the documents the module itself. In the
 online docs I would expect the function names used in the example to be
 links to the individual function doc and for each function have a list of
 links to examples it participated in. This should be automatic and the
 user shouldn't need to provide the "list of functions documented in this
 example".
 
 Just my two cents..


 
 Examples almost always go with functions. The whole point is to avoid having
to 
 have the function examples both in the documentation and in a unittest block.
It 
 would be a huge mess to try and put all of the examples in the module 
 documentation. I shudder to think what that would look like for std.algorithm
- 
 or even worse (_far_ worse), std.datetime. The simple syntax of
 
 /++ Example +/
 unittest
 {
 }
 
 
 making that unittest block go in the documentation of the preceding function 
 should work just fine. We already have /++ Ditto +/ which puts a function in
with 
 the preceding function's documentation. So, having /++ Example +/ on top of
that 
 isn't much of a stretch.
 
 - Jonathan M Davis


Depends on what you mean by example.
It seems to me that by "Example" you mean something like a line or two that
just show how to call a function. I meant by "Example" a more complicated small
program that exercises several functions in the module. 
something more like a tutorial for using the module.

For my kind of example you need to provide just one or two examples for a
module, it makes more sense to put it at the module level and also in a
unit-test block. 

Your kind of example looks less useful to me and I personally wouldn't bother
with unit-testing it. 

I guess it's just personal style and preferences.
Jan 25 2011
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday 25 January 2011 02:10:49 foobar wrote:

 Jonathan M Davis Wrote:

 On Monday 24 January 2011 15:27:08 foobar wrote:


 <snip>
 

 Unit-tests are defined on a module level, not a function level, hence I
 would expect to see the unit-tests that serve as examples to appear in
 an examples section on the page the documents the module itself. In
 the online docs I would expect the function names used in the example
 to be links to the individual function doc and for each function have
 a list of links to examples it participated in. This should be
 automatic and the user shouldn't need to provide the "list of
 functions documented in this example".
 
 Just my two cents..


 
 Examples almost always go with functions. The whole point is to avoid
 having to have the function examples both in the documentation and in a
 unittest block. It would be a huge mess to try and put all of the
 examples in the module documentation. I shudder to think what that would
 look like for std.algorithm - or even worse (_far_ worse), std.datetime.
 The simple syntax of
 
 /++ Example +/
 unittest
 {
 }
 
 
 making that unittest block go in the documentation of the preceding
 function should work just fine. We already have /++ Ditto +/ which puts
 a function in with the preceding function's documentation. So, having
 /++ Example +/ on top of that isn't much of a stretch.
 
 - Jonathan M Davis


 
 Depends on what you mean by example.
 It seems to me that by "Example" you mean something like a line or two that
 just show how to call a function. I meant by "Example" a more complicated
 small program that exercises several functions in the module. something
 more like a tutorial for using the module.
 
 For my kind of example you need to provide just one or two examples for a
 module, it makes more sense to put it at the module level and also in a
 unit-test block.
 
 Your kind of example looks less useful to me and I personally wouldn't
 bother with unit-testing it.
 
 I guess it's just personal style and preferences.


Look at std.algorithm if you want to see typical examples of examples (no pun 
intended). _Those_ are the kind of examples that we're worried about. And yes, 
they're generally examples per function. And if you _don't_ unit test them,
then 
you risk them being incorrect code. By having the be in a unittest block which 
gets turned into an example, you know that the code is valid and that it works. 
And you don't have to worry about the example and the unit test being of sync.

Very little in Phobos would benefit from a module-level example as you
describe. 
Most modules hold functions which are loosely related at best (std.algorithm
and 
std.string being prime examples). So, having a module-level example wouldn't 
really be worth much.

- Jonathan M Davis
Jan 25 2011
prev sibling parent spir <denis.spir gmail.com> writes:
On 01/25/2011 11:10 AM, foobar wrote:

 The simple syntax of

  /++ Example +/
  unittest
  {
  }


  making that unittest block go in the documentation of the preceding function
  should work just fine. We already have /++ Ditto +/ which puts a function in
with
  the preceding function's documentation. So, having /++ Example +/ on top of
that
  isn't much of a stretch.

  - Jonathan M Davis




 Depends on what you mean by example.
 It seems to me that by "Example" you mean something like a line or two that
just show how to call a function. I meant by "Example" a more complicated small
program that exercises several functions in the module.
 something more like a tutorial for using the module.

 For my kind of example you need to provide just one or two examples for a
module, it makes more sense to put it at the module level and also in a
unit-test block.

 Your kind of example looks less useful to me and I personally wouldn't bother
with unit-testing it.

 I guess it's just personal style and preferences.


Seems doc meta-languages are usually too complicated and this complication 
never helps meeting practicle needs. Think at Walter' repeted argument that 
unittests are highly used precisely because they are that simple. The same 
applies to doc, imo. I would thus favor an extremely simple doc feature, along 
with use recommandations, and stricter guidelines for public/std packages.

About the current issue of examples applying to several elements of a module, a 
simple solution would be to have an example section at a standard location (eg 
at the bottom, or just below the general module doc). Each example there would 
be /named/, so that function docs could point to them. A refinement would make 
example names link targets and pointers to them link pointers.
In the same vein: named unittests would allow pointing to them, and possibly 
generate link targets in the html doc, under their own name.

	/** blah
	* blah blah blah
	* blah blah blah
	*
	* Example:
	*     ...
	* See also:
	*     example $(target MapFilterCombination)
	*     unittest $(target testFilter)
	*/

	T[] filter (T) (T[] elements, bool function (T) predicate)
	{
	...
	}

Denis
-- 
_________________
vita es estrany
spir.wikidot.com
Jan 25 2011
prev sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Jonathan M Davis wrote:

 On Monday 24 January 2011 15:27:08 foobar wrote:

 Steven Schveighoffer Wrote:

 
 This only makes sense if:
 
 1. The unit test immediately follows the item being documented
 2. The unit test *only* tests that item.
 
 The second one could be pretty annoying.  Consider cases where several
 functions interact (I've seen this many times on Microsoft's
 Documentation), and it makes sense to make one example that covers all of
 them.  Having them 'testable' means creating several identical unit
 tests.
 
 One way to easily fix this is to allow an additional parameter to the
 comment:
 
 /**
 Example(Foo.foo(int), Foo.bar(int)):
 */
 unittest
 {
 
     auto foo = new Foo;
     foo.foo(5);
     foo.bar(6);
     assert(foo.toString() == "bazunga!");
 
 }
 
 The above means, copy the example to both Foo.foo(int) and Foo.bar(int)
 
 An alternative that is more verbose, but probably more understandable:
 
 /**
 Example:
 Covers Foo.foo(int)
 Covers Foo.bar(int)
 */
 
 Of course, a lack of target just means it applies to the item just
 documented.
 
 One other thing, using writefln is considered bad form in unit tests (you
 want *no* output if the unit test works).  But many examples might want
 to demonstrate how e.g. an object interacts with writefln.  Any
 suggestions? The assert line above is not very pretty for example...
 
 -Steve


 
 Unit-tests are defined on a module level, not a function level, hence I
 would expect to see the unit-tests that serve as examples to appear in an
 examples section on the page the documents the module itself. In the
 online docs I would expect the function names used in the example to be
 links to the individual function doc and for each function have a list of
 links to examples it participated in. This should be automatic and the
 user shouldn't need to provide the "list of functions documented in this
 example".
 
 Just my two cents..


 
 Examples almost always go with functions. The whole point is to avoid having
to 
 have the function examples both in the documentation and in a unittest block.
It 
 would be a huge mess to try and put all of the examples in the module 
 documentation. I shudder to think what that would look like for std.algorithm
- 
 or even worse (_far_ worse), std.datetime. The simple syntax of
 
 /++ Example +/
 unittest
 {
 }
 
 
 making that unittest block go in the documentation of the preceding function 
 should work just fine. We already have /++ Ditto +/ which puts a function in
with 
 the preceding function's documentation. So, having /++ Example +/ on top of
that 
 isn't much of a stretch.


I feel the same.
When I propose how it should be and how it is right to me I feel a bit
bad. Because I have not contributed anything besides talking. That's why
in the end I always will follow the people's advice who actually do the
work. I mean if Walter decides to leave it as it is, it's his decision
until somebody provides a patch. He does the work. Who am I to tell him
what he should do.

Jens
Jan 25 2011