[HN Gopher] Dr. Chaos - A structured fuzzing framework in Nim
       ___________________________________________________________________
        
       Dr. Chaos - A structured fuzzing framework in Nim
        
       Author : planetis
       Score  : 56 points
       Date   : 2022-08-28 12:38 UTC (10 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | cjbprime wrote:
       | Surprisingly, structured fuzzing (mutating inputs to a grammar's
       | generator) has proven less effective than optimized traditional
       | random fuzzing, so far.
       | 
       | https://google.github.io/fuzzbench/ shows that grammar fuzzers
       | (like AFLSmart) find less coverage per minute than standard
       | fuzzers like aflplusplus, even for the cases where the grammar
       | fuzzer has an extremely comprehensive grammar for that format.
        
         | planetis wrote:
         | Interesting, how does FormatFuzzer or LPM rank? The one is a
         | generator based format fuzzer and the other a mutator based
         | structured fuzzer. And which benchmarks compare any of those
         | two?
        
           | cjbprime wrote:
           | I'm not sure! Maybe someone can encourage the authors to
           | submit to Fuzzbench.
           | 
           | My intuition for the results so far is that (a) converting
           | generator functions to bitstrings is very computationally
           | expensive (since the comparison is coverage found per unit of
           | time), and (b) perhaps most memory safety vulnerabilities may
           | be occuring in technically invalid or grammatically
           | inconsistent inputs that the grammar will never find.
        
             | planetis wrote:
             | I think we need fuzzers for more uses than file formats,
             | like method fuzzing and in that case garbage data just
             | doesn't work.
        
               | planetis wrote:
               | It's just sound weird to me because depending of who you
               | ask, they will tell you generator fuzzers work the best.
               | However defining what's a good benchmark and what to
               | measure other than code coverage is even more
               | complicated. But I guess we are talking about google they
               | might have figured it out.
        
               | cjbprime wrote:
               | Yes, that's why I described the empirical results as
               | surprising.
        
               | cjbprime wrote:
               | This is also a testable claim! File formats and RPC do
               | not seem that dissimilar to me, especially when using a
               | fuzzer like aflplusplus, which will automatically build a
               | symbol dictionary by pulling strings out of the binary,
               | and perform some light symbolic transformation (laf-intel
               | and cmplog), see
               | https://lafintel.wordpress.com/2016/08/15/circumventing-
               | fuzz...
        
       ___________________________________________________________________
       (page generated 2022-08-28 23:01 UTC)