www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DStress thoughts

reply Stewart Gordon <smjg_1998 yahoo.com> writes:
1. .html files are served as text/plain, presumably to cause the HTML 
code to be displayed in the browser rather than rendered as an HTML 
page.  Shouldn't .d files have the same treatment?  With them as 
text/x-d-source, Mozilla just assumes it's a format it can't do anything 
with, and so I have to use the view-source: trick to see it.

2. The expected/actual results could be refined a bit.  A possible set 
of expected results:

- Run without error
- Exception (could include AssertError)
- Compile only (for tests that it doesn't make any sense to try running, 
e.g. my submission http://svn.kuehne.cn/dstress/compile/bug_e2ir_1158_A.d)
- Linker error
- Compiler error

Actual results would include as well as the above:

- Pass
- Runtime crash
- Compiler internal error/crash/segfault

There would also be 'expected output' and 'actual output' stuff where 
applicable, be it actual output or compiler errors.  A result of 'pass' 
would mean that the actual output matches the expected output exactly, 
whereas (expected result == actual result) could be true if they don't 
match.

For that matter, are we supposed to be able to find the PASS, XFAIL, 
etc. results that we're told about somewhere?

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Dec 06 2004
parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:

 1. .html files are served as text/plain


All html files should be served as text/html - the only exception are
some html_encoding* / html_line_ending* tests.

a test:
wget http://svn.kuehne.cn/dstress/run/html_comment_01.html








Please provide me with the URLs of any html file that are served incorrectly.


 With them as text/x-d-source, Mozilla just assumes it's a format it
 can't do anything with, and so I have to use the view-source: trick to
 see it.


On Linux, Mac, Window you can define a system wide mime and attach a
default application to it - e.g IDE, editor etc.

I'm not a regular Mozilla user but will inquire.


 2. The expected/actual results could be refined a bit.  A possible set 
 of expected results:

 - Run without error
 - Exception (could include AssertError)
 - Compile only (for tests that it doesn't make any sense to try running, 
 e.g. my submission http://svn.kuehne.cn/dstress/compile/bug_e2ir_1158_A.d)
 - Linker error
 - Compiler error

 Actual results would include as well as the above:

 - Pass
 - Runtime crash
 - Compiler internal error/crash/segfault

 There would also be 'expected output' and 'actual output' stuff where 
 applicable, be it actual output or compiler errors.  A result of 'pass' 
 would mean that the actual output matches the expected output exactly, 
 whereas (expected result == actual result) could be true if they don't 
 match.


see summary / readme:
http://svn.kuehne.cn/dstress/readme.txt
http://svn.kuehne.cn/dstress/www/dstress.html

To keep it simple:
* should compile - http://svn.kuehne.cn/dstress/compile/
* shouldn't compile - http://svn.kuehne.cn/dstress/nocompile/
* should run -  http://svn.kuehne.cn/dstress/run/
* shouldn't run - http://svn.kuehne.cn/dstress/norun/


 For that matter, are we supposed to be able to find the PASS, XFAIL, 
 etc. results that we're told about somewhere?


combined chart - http://svn.kuehne.cn/dstress/www/dstress.html

individual listing - http://svn.kuehne.cn/dstress/raw_results/

A hint: 
Use Subversion to stay up to date, see log messages etc.

Thomas

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.9.13 (GNU/Linux)

iD8DBQFBtFNV3w+/yD4P9tIRAo7/AJ4iqKCBGKjg8TDXxF8USTBByYTv4gCgzAOm
jEbRG54CQKXz6OHF4JfUXZs=
=8uBd
-----END PGP SIGNATURE-----
Dec 06 2004
next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Thomas Kuehne wrote:

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:
 

1. .html files are served as text/plain


 
 All html files should be served as text/html - the only exception are
 some html_encoding* / html_line_ending* tests.


<snip>

 Please provide me with the URLs of any html file that are served incorrectly.


Not sure if these count:

http://svn.kuehne.cn/dstress/nocompile/html_empty_entity_01.html
http://svn.kuehne.cn/dstress/nocompile/html_empty_entity_02.html
http://svn.kuehne.cn/dstress/nocompile/html_empty_entity_03.html

But you're right, the rest seem to be either served as text/html or 
encoding issues.


With them as text/x-d-source, Mozilla just assumes it's a format it
can't do anything with, and so I have to use the view-source: trick to
see it.


 
 
 On Linux, Mac, Window you can define a system wide mime and attach a
 default application to it - e.g IDE, editor etc.


ATM I can't see this as enabling a link I click in Mozilla to be 
displayed there and then.  But maybe it's a Mozilla problem rather a 
DStress problem....

<snip>

 To keep it simple:
 * should compile - http://svn.kuehne.cn/dstress/compile/
 * shouldn't compile - http://svn.kuehne.cn/dstress/nocompile/
 * should run -  http://svn.kuehne.cn/dstress/run/
 * shouldn't run - http://svn.kuehne.cn/dstress/norun/


I see.


For that matter, are we supposed to be able to find the PASS, XFAIL, 
etc. results that we're told about somewhere?


 
 
 combined chart - http://svn.kuehne.cn/dstress/www/dstress.html
 
 individual listing - http://svn.kuehne.cn/dstress/raw_results/


<snip>

Excellent.  But it would be helpful to work out where there are 
platform-dependent failures - I've noticed that you have one or two Mac 
results but no Windows results.  Maybe I'll take your tests home and try 
them....

BTW I noticed that you somehow miscopied my name on a few tests (at 
least the abstract ones).

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Dec 06 2004
next sibling parent reply Brad Anderson <brad dsource.dot.org> writes:
Stewart Gordon wrote:

 Thomas Kuehne wrote:
 

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1

 Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:


 1. .html files are served as text/plain



 All html files should be served as text/html - the only exception are
 some html_encoding* / html_line_ending* tests.




This may help, unless the html files are meant to be text/plain for 
testing purposes.  It will recursively (current dir) set the mime-type 
to text/html in Subversion for all html files.



BA
Dec 06 2004
parent Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brad Anderson schrieb am Mon, 06 Dec 2004 09:12:01 -0600:

 1. .html files are served as text/plain



 All html files should be served as text/html - the only exception are
 some html_encoding* / html_line_ending* tests.




 This may help, unless the html files are meant to be text/plain for 
 testing purposes.  It will recursively (current dir) set the mime-type 
 to text/html in Subversion for all html files.




Thanks, I just somehow missed them when marking all the file properties.

Thomas

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.9.13 (GNU/Linux)

iD8DBQFBtI8+3w+/yD4P9tIRAuMdAKC1gFNKkPx+zcH6OTzLPDHzO8HaJgCeLTtQ
LIPTaEzUf8pf8kVwOrRR+mc=
=HEcJ
-----END PGP SIGNATURE-----
Dec 06 2004
prev sibling parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stewart Gordon schrieb am Mon, 06 Dec 2004 14:10:36 +0000:

With them as text/x-d-source, Mozilla just assumes it's a format it
can't do anything with, and so I have to use the view-source: trick to
see it.


 
 
 On Linux, Mac, Window you can define a system wide mime and attach a
 default application to it - e.g IDE, editor etc.


 ATM I can't see this as enabling a link I click in Mozilla to be 
 displayed there and then.  But maybe it's a Mozilla problem rather a 
 DStress problem....


https://bugzilla.mozilla.org/show_bug.cgi?id=57342
https://bugzilla.mozilla.org/show_bug.cgi?id=196078


 But it would be helpful to work out where there are 
 platform-dependent failures - I've noticed that you have one or two Mac 
 results but no Windows results.  Maybe I'll take your tests home and try 
 them....


You are welcome! 

There are several problems:

1) There is no Windows box within my reach *g*

2) The Makefile uses lot's of tiny GNU tools(grep, cat, ...). JCC has started
to write a bat generator (http://svn.kuehne.cn/dstress/beta/dstresstest.d) to
solve parts of this problem.

3) By producing an error pop-up the segfaulting test cases pose a hurdle for
automated bat files.



 BTW I noticed that you somehow miscopied my name on a few tests (at 
 least the abstract ones).


Ops ...

Thomas

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.9.13 (GNU/Linux)

iD8DBQFBtI7f3w+/yD4P9tIRAvLsAKCO3Mv2cyOS0sweZs83SWpmiHCLlgCfZdDi
IC0eVoQV787jQRZrMKHZ++Q=
=EiKm
-----END PGP SIGNATURE-----
Dec 06 2004
parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 6 Dec 2004 17:54:56 +0100, Thomas Kuehne  
<thomas-dloop kuehne.thisisspam.cn> wrote:

 There are several problems:

 1) There is no Windows box within my reach *g*


Have you heard of:
http://www.vmware.com/products/desktop/ws_features.html

If you can get your hands on a copy of that I recommend it.

Basically it allows you to install any OS into a file on your HDD and then  
you can boot and run that OS any time you want. You can run several at  
once, and telnet to/from/between them all.

Regan
Dec 06 2004
prev sibling next sibling parent reply Georg Wrede <Georg_member pathlink.com> writes:
In article <mj0e82-kt4.ln1 kuehne.cn>, Thomas Kuehne says...

Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:

 1. .html files are served as text/plain


All html files should be served as text/html - the only exception are
some html_encoding* / html_line_ending* tests.


I wish the D tests there were in html, formatted with nice colors.
It is unreasonable to have each person reading that stuff to adjust
the browser on each machine he uses (whether regularly, or just once
to show a specific point to somebody at her computer).

What makes this request the more reasonable is that D boasts being
"html-compatible". So anybody could just download the pretty html
and the D compiler will accept that as input.

Oh, and I'd like the motivation for having xpass and xfail to be 
explained somewhere prominently. Like, what the "unexpected" is?
Not per test, but in general. There are now test that are xpass
version after version, and the general reader might wonder how 
come this remains unexpected.  .-)

(I'm talking about http://svn.kuehne.cn/dstress/www/dstress.html)
Dec 06 2004
parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Wrede schrieb am Mon, 6 Dec 2004 18:11:28 +0000 (UTC):

 I wish the D tests there were in html, formatted with nice colors.
 It is unreasonable to have each person reading that stuff to adjust
 the browser on each machine he uses (whether regularly, or just once
 to show a specific point to somebody at her computer).

 What makes this request the more reasonable is that D boasts being
 "html-compatible". So anybody could just download the pretty html
 and the D compiler will accept that as input.


Have a look at the HTML tests:
http://svn.kuehne.cn/dstress/www/dstress.html#html_comment_01

and scroll down.

FYI I have send Walter several patches that solve most of the HTML FAIL and
XPASS tests.


 Oh, and I'd like the motivation for having xpass and xfail to be 
 explained somewhere prominently. Like, what the "unexpected" is?
 Not per test, but in general.


Are you talking about an abstract like in the readme?

If not, suggest some general explanations.


 There are now test that are xpass version after version, and the
 general reader might wonder how come this remains unexpected.  .-)


How come there are long standing issues with almost all software projects?

Some ideas:
* The documentation isn't inline with the implementation.
* Fixes with scores of dependencies / implications.
* Bugs that aren't on the priority list.
...

The D frontend is open source and the gcc backend is open source too,
thus:

If you feel you need a bug fix and it hasn't be tackled by the
maintainer(big bright W) you might:
a) fix it yourself and send a patch
b) pay someone to do the work for you
c) send a petition *g*

Thomas


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.9.13 (GNU/Linux)

iD8DBQFBtLDE3w+/yD4P9tIRAlNaAJ4keqMh9LNUz7gyDsQoRGADJwYDogCeILLD
1PSNZkRKtGJojbVc005rcv8=
=aNax
-----END PGP SIGNATURE-----
Dec 06 2004
next sibling parent Georg Wrede <Georg_member pathlink.com> writes:
In article <5vne82-dk2.ln1 kuehne.cn>, Thomas Kuehne says...

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Wrede schrieb am Mon, 6 Dec 2004 18:11:28 +0000 (UTC):

 I wish the D tests there were in html, formatted with nice colors.
 It is unreasonable to have each person reading that stuff to adjust
 the browser on each machine he uses (whether regularly, or just once
 to show a specific point to somebody at her computer).

 What makes this request the more reasonable is that D boasts being
 "html-compatible". So anybody could just download the pretty html
 and the D compiler will accept that as input.


Have a look at the HTML tests:
http://svn.kuehne.cn/dstress/www/dstress.html#html_comment_01

and scroll down.


Yes! If only we could get those above it into html too. :-)


FYI I have send Walter several patches that solve most of the HTML FAIL and
XPASS tests.


Few can have missed the value your work brings to D! 


 Oh, and I'd like the motivation for having xpass and xfail to be 
 explained somewhere prominently. Like, what the "unexpected" is?
 Not per test, but in general.


Are you talking about an abstract like in the readme?

If not, suggest some general explanations.


I formulated myself rather badly, sorry.

On the same page, under subheading "^Legend", I'd like to see a
definition of what constitutes pass, xpass, fail, and xfail.
The others (error, and not tested) are self-explanatory.

There is a reference to the DejaGnu test framework, but a smallish
forage there gave nothing concrete in hand. Also the POSIX standard
1003.3 seems to be for paid-only readers.

Pass and fail were easy if they were the only ones. But definition
of "unexpected" (for both) is what I'd like to see right there.

To sum it up, all that's needed is an answer to "why do we have
the category xpass, and why the category xfail?"
Dec 07 2004
prev sibling parent reply Georg Wrede <Georg_member pathlink.com> writes:
In article <5vne82-dk2.ln1 kuehne.cn>, Thomas Kuehne says...

-----BEGIN PGP SIGNED MESSAGE-----


..

Georg Wrede schrieb am Mon, 6 Dec 2004 18:11:28 +0000 (UTC):

 Oh, and I'd like the motivation for having xpass and xfail to be 
 explained somewhere prominently. Like, what the "unexpected" is?
 Not per test, but in general.




Maybe I'm getting slow, but I haven't figured out why xfail is
light GREEN?

One would assume that the colors were in order. So, out of the 4
alternatives (pass, fail, xpass, xfail) the worst ( = most bad)
result would be VERY RED. The next "baddest" RED, and the least
bad would be LIGHT RED, or orange. Pass of course has to be green.

Oh, and the point-of-view here should be that of the compiler
writer (and the community at large), not that of the test writer.
(No disrespect intended!) The reason being that they are the ones
who (hopefully) choose what to tackle next.

I don't feel qualified to even suggest a particular ordering here,
be it by difficulty of fixing, "unexpectedness"(?), priority for
D 1.0, or whatever else.

-----

ps, using the same prefix for expected and unexpected may go
unnoticed for quite some time -- for readers like me, at least.

While writing this, I took another hard look at the coloring,
and I started to suspect that the light green for expected-
failure is because the test writer is happy: the test gave
the result he intended. (So presumably at least the test itself
is ok.) I sure hope I'm wrong.
Dec 07 2004
parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Wrede schrieb am Tue, 7 Dec 2004 08:57:02 +0000 (UTC):

 One would assume that the colors were in order. So, out of the 4
 alternatives (pass, fail, xpass, xfail) the worst ( = most bad)
 result would be VERY RED. The next "baddest" RED, and the least
 bad would be LIGHT RED, or orange. Pass of course has to be green.


That's the intention.


 Oh, and the point-of-view here should be that of the compiler
 writer (and the community at large), not that of the test writer.
 (No disrespect intended!) The reason being that they are the ones
 who (hopefully) choose what to tackle next.


I'm trying to use community standards.

* http://www.cs.utah.edu/dept/old/texinfo/dejagnu/dejagnu_2.html

* http://gcc.gnu.org/install/test.html#TOC3


 (search for XFAIL)


 While writing this, I took another hard look at the coloring,
 and I started to suspect that the light green for expected-
 failure is because the test writer is happy: the test gave
 the result he intended. (So presumably at least the test itself
 is ok.) I sure hope I'm wrong.


When all hope was gone only the Suspicion remained *g*

Thomas

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.9.13 (GNU/Linux)

iD8DBQFBtYBc3w+/yD4P9tIRAosYAKC7iMMY/W0+aVdJqkGlhtV0IRxC/QCeLQFy
L2nWXcEyodp1zi/rDil/Vl4=
=F4DS
-----END PGP SIGNATURE-----
Dec 07 2004
parent Georg Wrede <Georg_member pathlink.com> writes:
In article <srbg82-a93.ln1 kuehne.cn>, Thomas Kuehne says...

When all hope was gone only the Suspicion remained *g*


Thanks, now the Legend seems much clearer!!

georg
Dec 10 2004
prev sibling parent Mike Parker <aldacron71 yahoo.com> writes:
Thomas Kuehne wrote:

 On Linux, Mac, Window you can define a system wide mime and attach a
 default application to it - e.g IDE, editor etc.


But don't you find it much more convenient when you can see a source 
file in your browser? It's rather annoying to switch between apps. 
ViewCVS serves source files as plain text, many personal pages I've seen 
serve source files as plain text. I always get peeved at Sun because 
some Java sources on their site are served as plain text while others 
aren't.

I suppose it's a matter of just configuring the mime type to be used 
with the browser rather than an editor. But it seems to me much simpler 
to just serve it as plain text, and if someone wants to open it 
somewhere other than the browser they can right-click->saveas.
Dec 06 2004