• safety0ff (21/21) Jan 05 2017 While trying to make a few minor contributions during the

safety0ff <safety0ff.dev gmail.com> writes:

While trying to make a few minor contributions during the 
holidays, it seemed like every other PR was hitting a "random" 
unit test failure.

So I solved the few that I noticed [1-3] (the last is still 
waiting review / merge.)

However finding failures that have gone unnoticed or unreported 
is a manual process. So I wrote a simple script that scrapes the 
auto tester website, process the logs of "interesting" failures, 
groups identical failures and outputs them.


I'm wondering if there is interest in having the auto tester 
detect & track the sporadic failures to make them more prominent 
and easier to find & fix.

I perceive that the attitude towards "random" test failures is 
widespread apathy. Failures seldom get filed in the bug tracker. 
While bad tests are often the cause, sometimes there are 
significant issues:

For example, [1] was a significant implementation flaw in 
std.experimental.allocator that was present for ~1.25 years.