www.digitalmars.com         C & C++   DMDScript  

D.gnu - Defuzzed: a fuzzer for D compilers

reply Sebastien Alaiwan <ace17 free.fr> writes:
Hi guys,

I just started a D fuzzer. The goal is to randomly generate input 
source files in an attempt to crash the compiler. It's in very 
early stage at this point (less than 300 loc), but it already can 
crash gdc.

https://github.com/Ace17/defuzzed

Please let me know if you think this could be a valuable tool.
( I already know about dsmith (a fork of csmith), but I wasn't 
able to crash neither gdc nor dmd with it. I also aim for a 
simpler tool, and for a one written in D )
Apr 02 2016
next sibling parent "Iain Buclaw via D.gnu" <d.gnu puremagic.com> writes:
On 2 Apr 2016 7:30 pm, "Sebastien Alaiwan via D.gnu" <d.gnu puremagic.com>
wrote:

 Hi guys,

 I just started a D fuzzer. The goal is to randomly generate input source


files in an attempt to crash the compiler. It's in very early stage at this
point (less than 300 loc), but it already can crash gdc.

 https://github.com/Ace17/defuzzed

 Please let me know if you think this could be a valuable tool.
 ( I already know about dsmith (a fork of csmith), but I wasn't able to


crash neither gdc nor dmd with it. I also aim for a simpler tool, and for a
one written in D )


I expect many bug reports.  :-)



 Apr 02 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 2 April 2016 at 17:29:53 UTC, Sebastien Alaiwan 
wrote:

 Hi guys,

 I just started a D fuzzer. The goal is to randomly generate 
 input source files in an attempt to crash the compiler. It's in 
 very early stage at this point (less than 300 loc), but it 
 already can crash gdc.

 https://github.com/Ace17/defuzzed

 Please let me know if you think this could be a valuable tool.
 ( I already know about dsmith (a fork of csmith), but I wasn't 
 able to crash neither gdc nor dmd with it. I also aim for a 
 simpler tool, and for a one written in D )


there is also generated from Brain Schott aka. Hackerpilot.
It may be useful to take a peek there.



 Apr 03 2016



  parent reply Sebastien Alaiwan <ace17 free.fr>  writes:


On Sunday, 3 April 2016 at 12:08:32 UTC, Stefan Koch wrote:

 there is also generated from Brain Schott aka. Hackerpilot.
 It may be useful to take a peek there.


Thanks, this definitely is going to be a source of inspiration!
(I'm also looking at AFL (American Fuzzy Lop) - lots of great 
stuff in this tool).

Unlike 'generated', I aim to produce semantically valid random 
programs.

I might want to allow some degree of invalidity, as the same 
error could manifest itself as both 'ice-on-valid' and 
'ice-on-invalid', but might be more easily revealed through 
'ice-on-invalid'.

My long-term goal is the complete eradication of "ice-on-valid" 
errors from gdc :-)
(I plan to keep the tool compiler-agnostic).

Your ideas/contributions are welcome!



 Apr 03 2016



  parent  "Brad Roberts via D.gnu" <d.gnu puremagic.com>  writes:


On 4/3/2016 5:48 AM, Sebastien Alaiwan via D.gnu wrote:

 On Sunday, 3 April 2016 at 12:08:32 UTC, Stefan Koch wrote:

 there is also generated from Brain Schott aka. Hackerpilot.
 It may be useful to take a peek there.


 Thanks, this definitely is going to be a source of inspiration!
 (I'm also looking at AFL (American Fuzzy Lop) - lots of great stuff in
 this tool).

 Unlike 'generated', I aim to produce semantically valid random programs.

 I might want to allow some degree of invalidity, as the same error could
 manifest itself as both 'ice-on-valid' and 'ice-on-invalid', but might
 be more easily revealed through 'ice-on-invalid'.

 My long-term goal is the complete eradication of "ice-on-valid" errors
 from gdc :-)
 (I plan to keep the tool compiler-agnostic).

 Your ideas/contributions are welcome!


Also, given the shared frontend among dmd, gdc, and ldc, most of the 
bugs will likely be in that shared code and should result in bugs going 
to issues.dlang.org.

I wouldn't worry too much about valid vs invalid.  ICE's in general tend 
to be fairly easy to fix and have a pretty big negative impact on user 
experience.  So, both are very worth discovering and reporting.

Later,
Brad



 Apr 03 2016



prev sibling  parent reply David Nadlinger <code klickverbot.at>  writes:


On Saturday, 2 April 2016 at 17:29:53 UTC, Sebastien Alaiwan 
wrote:

 Please let me know if you think this could be a valuable tool.


I am really thrilled to see effort being put towards more testing 
tools, and I'm sure that most other LDC devs are as well.

There are so many opportunities which remain unexplored simply 
because nobody took the time to actively try and break the 
current D compilers yet. ICE fuzz-testing like this is a first 
step, and I'm pretty sure that csmith-style comparison of 
executable results between the three compilers will yield lots of 
low-hanging fruit as well.

Please do keep the LDC community posted about your progress too!

  — David



 Apr 03 2016



  parent  Johan Engelen <j j.nl>  writes:


On Sunday, 3 April 2016 at 17:05:56 UTC, David Nadlinger wrote:

 On Saturday, 2 April 2016 at 17:29:53 UTC, Sebastien Alaiwan 
 wrote:

 Please let me know if you think this could be a valuable tool.


 I am really thrilled to see effort being put towards more 
 testing tools, and I'm sure that most other LDC devs are as 
 well.

 There are so many opportunities which remain unexplored simply 
 because nobody took the time to actively try and break the 
 current D compilers yet. ICE fuzz-testing like this is a first 
 step, and I'm pretty sure that csmith-style comparison of 
 executable results between the three compilers will yield lots 
 of low-hanging fruit as well.

 Please do keep the LDC community posted about your progress too!


+1 !

There is a "dsmith" tool out there?! Didn't know about it, that's 
great news!



 Apr 03 2016