digitalmars.D - Testing package proposed for Phobos
- Walter Bright (30/30) Feb 08 2015 More and more, D code is written as a component that takes a type parame...
- Jakob Ovrum (3/4) Feb 08 2015 Is this idea different from the one in
- Walter Bright (2/5) Feb 08 2015 I wasn't aware of the existence of that. Will check it out.
- H. S. Teoh via Digitalmars-d (8/15) Feb 08 2015 And while we're at it, we might want to look at consolidating some of
- Walter Bright (54/60) Feb 08 2015 Cursory review:
- John Colvin (3/59) Feb 09 2015 This looks like a good bugzilla enhancement request. As you often
- Walter Bright (3/5) Feb 10 2015 It needs to be rethought from the ground up, and introduced as a new mod...
- Jonathan M Davis via Digitalmars-d (16/22) Feb 10 2015 It's a nice start, but it's far too limited IMHO, since it only defines ...
- Craig Dillabaugh (3/36) Feb 08 2015 Google Summer of Code project? Are you interested in mentoring?
- Russel Winder via Digitalmars-d (22/26) Feb 09 2015 On Mon, 2015-02-09 at 01:54 +0000, Craig Dillabaugh via Digitalmars-d
- Atila Neves (3/11) Feb 09 2015 https://github.com/atilaneves/unit-threaded
- Walter Bright (3/3) Feb 08 2015 I.e. by providing standard mockups for components, and standard tests fo...
- H. S. Teoh via Digitalmars-d (15/18) Feb 08 2015 Yes, I've suggested this a long time ago but it seemed to have fallen by
- CraigDillabaugh (2/27) Feb 09 2015 Was the Mark Twain quote intentional?
- ketmar (2/6) Feb 09 2015 it's a Fortune Wisdom. ;-)=
- Brad Anderson (3/10) Feb 10 2015 If I recall correctly, H.S. Teoh has...assisted the randomness of
- H. S. Teoh via Digitalmars-d (6/18) Feb 10 2015 Yes I have been guilty of that in the past... but this particular
- ketmar (4/16) Feb 10 2015 but his actions depends of such many things that we can consider that=20
- Tobias Pankrath (16/40) Feb 09 2015 Great idea. It might even be better to split the two concerns:
- Walter Bright (54/54) Feb 10 2015 To get things started, I created an InputRange and a function to test an...
- Walter Bright (4/10) Feb 10 2015 import std.range;
- Joseph Rushton Wakeling via Digitalmars-d (2/6) Feb 10 2015 Why only a template static array? Why not allow for an an array wrapper...
- Walter Bright (2/3) Feb 10 2015 Good point. See my followup.
- Walter Bright (3/9) Feb 11 2015 Here's another stab at it:
- FG (2/4) Feb 11 2015 I see four lights, err... spaces. There are four spaces. :)
More and more, D code is written as a component that takes a type parameters
that are ranges. Unit testing becomes inconvenient, as types must be mocked up
to call them. Using an array of data often is inadequate, because the component
may have specializations for an array, or specifies a range that is a subset of
what an array provides.
For example, here's a test range I wrote once for a unittest:
struct Adapter
{
this(ubyte[] r) { this.r = r; }
property bool empty() { return r.length == 0; }
property ubyte front() { return r[0]; }
void popFront() { r = r[1 .. $]; }
property size_t length() { return r.length; }
private:
ubyte[] r;
}
I propose a std.test.ranges package, which contains templates defining each of
the range types (InputRange, BiDirectionalRange, etc.). Each accepts an
argument
which is a static array of T, and then implements exactly the range interface
indicated by its name, nothing more. The range will also have asserts to
guarantee they are used correctly, i.e. front() cannot be called before empty()
returns false.
By default, those mock range templates will be safe. They can be configured to
be system.
Each range type will also have a corresponding testXXXRange!R(R r) function,
that tests the interface to range R to ensure that it follows the protocol
indicated by its name (there was some confusion a while back about exactly what
the protocol entailed, having a standard test function for it will help user
defined ranges be protocol correct).
Anyone interested in taking up this flag?
Feb 08 2015
On Monday, 9 February 2015 at 01:41:33 UTC, Walter Bright wrote:Anyone interested in taking up this flag?Is this idea different from the one in std.internal.test.dummyrange?
Feb 08 2015
On 2/8/2015 5:52 PM, Jakob Ovrum wrote:On Monday, 9 February 2015 at 01:41:33 UTC, Walter Bright wrote:I wasn't aware of the existence of that. Will check it out.Anyone interested in taking up this flag?Is this idea different from the one in std.internal.test.dummyrange?
Feb 08 2015
"H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> On Sun, Feb 08, 2015 at 06:08:31PM -0800, Walter Bright via Digitalmars-d wrote:On 2/8/2015 5:52 PM, Jakob Ovrum wrote:And while we're at it, we might want to look at consolidating some of the other unittest-specific stuff that's sprinkled throughout Phobos modules, such as in std.algorithm.package (search for "rndstuff" in package.d). T -- Why waste time reinventing the wheel, when you could be reinventing the engine? -- Damian ConwayOn Monday, 9 February 2015 at 01:41:33 UTC, Walter Bright wrote:I wasn't aware of the existence of that. Will check it out.Anyone interested in taking up this flag?Is this idea different from the one in std.internal.test.dummyrange?
Feb 08 2015
On 2/8/2015 6:08 PM, Walter Bright wrote:On 2/8/2015 5:52 PM, Jakob Ovrum wrote:Cursory review: 1. Yes, it's what I was asking for, at least in part. 2. Being in std.internal.test implies it is not for general usage. It needs to be for general usage. 3. It's used here: grep -r std.internal.test *.d algorithm\comparison.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\mutation.d: import std.internal.test.dummyrange; algorithm\mutation.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; internal\test\dummyrange.d:module std.internal.test.dummyrange; range\constraints.d: import std.internal.test.dummyrange; range\interfaces.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\primitives.d: import std.internal.test.dummyrange; I'd like to see it used much more pervasively in Phobos - essentially with every range that is defined, and every component that takes a range as a parameter, unless there's a good reason not to on a case-by-case basis. 4. It's only for uint data. This should be parameterized. 5. It's terribly underdocumented. I have no idea what ReturnBy, Length, or RangeType are without reading the implementation code. 6. It does not contain any protocol verification - for example, front() does not check that empty() was called beforehand. 7. There's not a single unittest in it. 8. I don't see any connection between DummyRange and ReferenceInputRange. 9. There are no test functions which accept an input range and test that the protocol at least exists and does not crash. 10. No system types. 11. No output ranges. I.e. it's a good starting point, but there's a lot more that can be done.On Monday, 9 February 2015 at 01:41:33 UTC, Walter Bright wrote:I wasn't aware of the existence of that. Will check it out.Anyone interested in taking up this flag?Is this idea different from the one in std.internal.test.dummyrange?
Feb 08 2015
On Monday, 9 February 2015 at 04:16:26 UTC, Walter Bright wrote:std.internal.test.dummyrange Cursory review: 1. Yes, it's what I was asking for, at least in part. 2. Being in std.internal.test implies it is not for general usage. It needs to be for general usage. 3. It's used here: grep -r std.internal.test *.d algorithm\comparison.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\iteration.d: import std.internal.test.dummyrange; algorithm\mutation.d: import std.internal.test.dummyrange; algorithm\mutation.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; algorithm\searching.d: import std.internal.test.dummyrange; internal\test\dummyrange.d:module std.internal.test.dummyrange; range\constraints.d: import std.internal.test.dummyrange; range\interfaces.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\package.d: import std.internal.test.dummyrange; range\primitives.d: import std.internal.test.dummyrange; I'd like to see it used much more pervasively in Phobos - essentially with every range that is defined, and every component that takes a range as a parameter, unless there's a good reason not to on a case-by-case basis. 4. It's only for uint data. This should be parameterized. 5. It's terribly underdocumented. I have no idea what ReturnBy, Length, or RangeType are without reading the implementation code. 6. It does not contain any protocol verification - for example, front() does not check that empty() was called beforehand. 7. There's not a single unittest in it. 8. I don't see any connection between DummyRange and ReferenceInputRange. 9. There are no test functions which accept an input range and test that the protocol at least exists and does not crash. 10. No system types. 11. No output ranges.This looks like a good bugzilla enhancement request. As you often say: it will get lost here.
Feb 09 2015
On 2/9/2015 3:03 AM, John Colvin wrote:This looks like a good bugzilla enhancement request. As you often say: it will get lost here.It needs to be rethought from the ground up, and introduced as a new module. I'm looking for someone to carry the flag on this.
Feb 10 2015
On Sunday, February 08, 2015 18:08:31 Walter Bright via Digitalmars-d wrote:On 2/8/2015 5:52 PM, Jakob Ovrum wrote:It's a nice start, but it's far too limited IMHO, since it only defines a very restricted subset of range types (though it's certainly better than nothing). I started implementing a more full-featured solution (one which would allow you to define any combination of range types - bidirectional with length, without length, infinite, etc.), but I ran into either compiler bugs or library bugs IIRC, and it fell by the wayside. Regardless, I think that we should create a general solution which allows you to instantiate the full combination of range types (as well as specifying a reasonable subsets for those who want decent coverage but don't want to test every combination) and which can be used in anyone's code rathear than just in Phobos. And perhaps, a package containing a variety of unit test helpers is in order, where a module handling ranges would be just one of the modules therein. - Jonathan M DavisOn Monday, 9 February 2015 at 01:41:33 UTC, Walter Bright wrote:I wasn't aware of the existence of that. Will check it out.Anyone interested in taking up this flag?Is this idea different from the one in std.internal.test.dummyrange?
Feb 10 2015
On Monday, 9 February 2015 at 01:41:33 UTC, Walter Bright wrote:
More and more, D code is written as a component that takes a
type parameters that are ranges. Unit testing becomes
inconvenient, as types must be mocked up to call them. Using an
array of data often is inadequate, because the component may
have specializations for an array, or specifies a range that is
a subset of what an array provides.
For example, here's a test range I wrote once for a unittest:
struct Adapter
{
this(ubyte[] r) { this.r = r; }
property bool empty() { return r.length == 0; }
property ubyte front() { return r[0]; }
void popFront() { r = r[1 .. $]; }
property size_t length() { return r.length; }
private:
ubyte[] r;
}
I propose a std.test.ranges package, which contains templates
defining each of the range types (InputRange,
BiDirectionalRange, etc.). Each accepts an argument which is a
static array of T, and then implements exactly the range
interface indicated by its name, nothing more. The range will
also have asserts to guarantee they are used correctly, i.e.
front() cannot be called before empty() returns false.
By default, those mock range templates will be safe. They can
be configured to be system.
Each range type will also have a corresponding testXXXRange!R(R
r) function, that tests the interface to range R to ensure that
it follows the protocol indicated by its name (there was some
confusion a while back about exactly what the protocol
entailed, having a standard test function for it will help user
defined ranges be protocol correct).
Anyone interested in taking up this flag?
Google Summer of Code project? Are you interested in mentoring?
(Sorry, I have a bit of a one-track mind)
Feb 08 2015
On Mon, 2015-02-09 at 01:54 +0000, Craig Dillabaugh via Digitalmars-d wrote: [=E2=80=A6]=20 Google Summer of Code project? Are you interested in mentoring? =20 (Sorry, I have a bit of a one-track mind)A point came up at the London D Users meeting: Having unittest integrated into module structure and compiler is fine for small unit tests, but what is the D support for integration tests, end-to-end tests, etc. What is the equivalent in D of Catch, TestNG, Spock, Cucumber,=E2=80=A6 If there is one, I should have known it. If there isn't then GSoC 2015 would be a good place to get a start with Catch and Spock being the things to use for inspiration. I'd be happy to mentor. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Feb 09 2015
A point came up at the London D Users meeting: Having unittest integrated into module structure and compiler is fine for small unit tests, but what is the D support for integration tests, end-to-end tests, etc. What is the equivalent in D of Catch, TestNG, Spock, Cucumber,…https://github.com/atilaneves/unit-threaded https://github.com/atilaneves/unencumbered Atila
Feb 09 2015
I.e. by providing standard mockups for components, and standard tests for component protocols, we can significantly improve the unit testing and standard conformance of user designed components.
Feb 08 2015
On Sun, Feb 08, 2015 at 06:06:59PM -0800, Walter Bright via Digitalmars-d wrote:I.e. by providing standard mockups for components, and standard tests for component protocols, we can significantly improve the unit testing and standard conformance of user designed components.Yes, I've suggested this a long time ago but it seemed to have fallen by the wayside. It would be good to provide standard testing stuff for: - Non-forward input ranges. - Forward but non-bidirectional ranges. - Bidirectional but not random access ranges. - Infinite ranges vs. finite ranges. - Ranges with reference semantics vs. value semantics. - Array-like ranges that are not arrays. Ideally, every range-based function in Phobos should be tested with each of these variations, to ensure correct semantics under all representative use cases. T -- Everybody talks about it, but nobody does anything about it! -- Mark Twain
Feb 08 2015
On Monday, 9 February 2015 at 04:40:36 UTC, H. S. Teoh wrote:On Sun, Feb 08, 2015 at 06:06:59PM -0800, Walter Bright via Digitalmars-d wrote:Was the Mark Twain quote intentional?I.e. by providing standard mockups for components, and standard tests for component protocols, we can significantly improve the unit testing and standard conformance of user designed components.Yes, I've suggested this a long time ago but it seemed to have fallen by the wayside. It would be good to provide standard testing stuff for: - Non-forward input ranges. - Forward but non-bidirectional ranges. - Bidirectional but not random access ranges. - Infinite ranges vs. finite ranges. - Ranges with reference semantics vs. value semantics. - Array-like ranges that are not arrays. Ideally, every range-based function in Phobos should be tested with each of these variations, to ensure correct semantics under all representative use cases. T -- Everybody talks about it, but nobody does anything about it! -- Mark Twain
Feb 09 2015
On Mon, 09 Feb 2015 13:48:55 +0000, CraigDillabaugh wrote:it's a Fortune Wisdom. ;-)=Everybody talks about it, but nobody does anything about it! -- Mark Twain=20 Was the Mark Twain quote intentional?
Feb 09 2015
On Monday, 9 February 2015 at 14:47:09 UTC, ketmar wrote:On Mon, 09 Feb 2015 13:48:55 +0000, CraigDillabaugh wrote:If I recall correctly, H.S. Teoh has...assisted the randomness of his signature in the past :P.it's a Fortune Wisdom. ;-)Everybody talks about it, but nobody does anything about it! -- Mark TwainWas the Mark Twain quote intentional?
Feb 10 2015
On Tue, Feb 10, 2015 at 05:46:20PM +0000, Brad Anderson via Digitalmars-d wrote:On Monday, 9 February 2015 at 14:47:09 UTC, ketmar wrote:Yes I have been guilty of that in the past... but this particular instance is solely the genius of my Perl script. :-) T -- Век живи - век учись. А дураком помрёшь.On Mon, 09 Feb 2015 13:48:55 +0000, CraigDillabaugh wrote:If I recall correctly, H.S. Teoh has...assisted the randomness of his signature in the past :P.it's a Fortune Wisdom. ;-)Everybody talks about it, but nobody does anything about it! -- Mark TwainWas the Mark Twain quote intentional?
Feb 10 2015
On Tue, 10 Feb 2015 17:46:20 +0000, Brad Anderson wrote:On Monday, 9 February 2015 at 14:47:09 UTC, ketmar wrote:but his actions depends of such many things that we can consider that=20 help as random action too. maybe just not cryptographically strong=20 randomness. ;-)=On Mon, 09 Feb 2015 13:48:55 +0000, CraigDillabaugh wrote:=20 If I recall correctly, H.S. Teoh has...assisted the randomness of his signature in the past :P.it's a Fortune Wisdom. ;-)Everybody talks about it, but nobody does anything about it! -- Mark Twain=20 Was the Mark Twain quote intentional?
Feb 10 2015
On Monday, 9 February 2015 at 01:41:33 UTC, Walter Bright wrote:
More and more, D code is written as a component that takes a
type parameters that are ranges. Unit testing becomes
inconvenient, as types must be mocked up to call them. Using an
array of data often is inadequate, because the component may
have specializations for an array, or specifies a range that is
a subset of what an array provides.
For example, here's a test range I wrote once for a unittest:
struct Adapter
{
this(ubyte[] r) { this.r = r; }
property bool empty() { return r.length == 0; }
property ubyte front() { return r[0]; }
void popFront() { r = r[1 .. $]; }
property size_t length() { return r.length; }
private:
ubyte[] r;
}
I propose a std.test.ranges package, which contains templates
defining each of the range types (InputRange,
BiDirectionalRange, etc.). Each accepts an argument which is a
static array of T, and then implements exactly the range
interface indicated by its name, nothing more. The range will
also have asserts to guarantee they are used correctly, i.e.
front() cannot be called before empty() returns false.
Great idea. It might even be better to split the two concerns:
Mock ranges taking the static array as data and testing wrappers
that enforce correct usage of wrapped ranges.
So that I can
TokenRange lex() { ... } // returns a range of token
// check that the TokenRange is a valid forward range
testIsForwardRange!TokenRange
// parser
auto parse(TR)(TR r) { ... }
// normal use
auto result = parse(lex());
// testing that parse uses the TR correctly
auto result = parse(assertingForwardRange(lex()));
This way it's easier to craft input for testing the correctness
of parse.
Feb 09 2015
To get things started, I created an InputRange and a function to test any
InputRange. Compiling it:
dmd inputrange -unittest -main -cov
shows 100% coverage.
-------------
struct InputRange(T : T[])
{
this(T[] r)
{
this.r = r;
}
property bool empty()
{
hasData = (r.length != 0);
return !hasData;
}
property T front()
{
assert(hasData);
return r[0];
}
void popFront()
{
assert(hasData);
r = r[1 .. $];
hasData = false;
}
private:
T[] r;
bool hasData;
}
void testInputRange(R)(R r, int n)
{
foreach (i; 0 .. n)
{
if (!r.empty)
{
if (i & 1)
{
auto e = r.front;
auto e2 = r.front;
assert(e == e2);
}
r.popFront();
}
}
}
unittest
{
string s = "hello";
auto r = InputRange!string(s);
r.testInputRange(10);
}
-------------
Feb 10 2015
On 2/10/2015 2:45 PM, Walter Bright wrote:unittest { string s = "hello"; auto r = InputRange!string(s); r.testInputRange(10);import std.range; assert(isInputRange!(typeof(r)));}Forgot to add that. Exactly why this needs to be in a standard module!
Feb 10 2015
On 09/02/15 02:41, Walter Bright via Digitalmars-d wrote:I propose a std.test.ranges package, which contains templates defining each of the range types (InputRange, BiDirectionalRange, etc.). Each accepts an argument which is a static array of T, and then implements exactly the range interface indicated by its name, nothing more.Why only a template static array? Why not allow for an an array wrapper?
Feb 10 2015
On 2/10/2015 7:03 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:Why only a template static array? Why not allow for an an array wrapper?Good point. See my followup.
Feb 10 2015
On 2/8/2015 5:41 PM, Walter Bright wrote:I propose a std.test.ranges package, which contains templates defining each of the range types (InputRange, BiDirectionalRange, etc.). Each accepts an argument which is a static array of T, and then implements exactly the range interface indicated by its name, nothing more. The range will also have asserts to guarantee they are used correctly, i.e. front() cannot be called before empty() returns false.Here's another stab at it: http://digitalmars.com/sargon/asinputrange.html
Feb 11 2015
On 2015-02-11 at 11:58, Walter Bright wrote:Here's another stab at it: http://digitalmars.com/sargon/asinputrange.htmlI see four lights, err... spaces. There are four spaces. :)
Feb 11 2015