[HN Gopher] Go Fuzz Testing
___________________________________________________________________
Go Fuzz Testing
Author : andrei
Score : 88 points
Date : 2022-03-29 15:23 UTC (7 hours ago)
(HTM) web link (blog.fuzzbuzz.io)
(TXT) w3m dump (blog.fuzzbuzz.io)
| Thaxll wrote:
| Curious if fuzzing is a common thing in network programming, does
| fuzzing plays nicely with binary inputs?
| masklinn wrote:
| Along side file parsers it's a pretty major fuzzing target, as
| it tends to be exposed to malicious inputs.
|
| Fuzzing works _primarily_ on binary data, "structured" fuzzing
| is somewhat rarer.
| andrei wrote:
| It does! Fuzzing actually started off as a tool built by
| security researchers to find vulnerabilities in parsers, and
| other complex codebases, usually written in C/C++ (looking for
| memory bugs). So anything that deals with untrusted binary data
| is a prime candidate for fuzz testing.
|
| Go's fuzzing framework supports `[]byte` arguments as well as
| all of the standard Go primitives, so you should be able to
| test netcode this way.
|
| If you're looking for a C/C++ solution, my recommendation is
| libfuzzer [0]. We've also built our own C/C++ fuzzing engine at
| Fuzzbuzz [1].
|
| [0] https://llvm.org/docs/LibFuzzer.html
|
| [1] https://docs.fuzzbuzz.io/docs/getting-started-in-c-or-c++
| rendall wrote:
| Neat, Andrei. Good luck with this venture!
| monkpit wrote:
| What's the rationale behind bundling a fuzzing library in the
| language instead of as a separate library?
| masklinn wrote:
| To do guided fuzzing (graybox, like afl or libfuzzer) you need
| to instrument the binary. To instrument the binary, you need to
| be part of the building process.
|
| Since Go has a bespoke compilation toolchains and AFAIK doesn't
| have compiler plugins, external fuzzing tools had to either
| fork the toolchain or perform extensive pre and post processing
| (couldn't tell you what go-fuzz did but many article about go-
| fuzz note that the building process can take a while).
|
| As such, building fuzzing into the standard toolchain and
| maintaining it as part of the project makes a lot of sense. It
| also gives fuzzing a much higher level of visibility (because
| sadly there will always be a population for whom an external /
| third-party tool will be suspicious).
| LukeShu wrote:
| It does instrumented fuzzing. The older
| https://github.com/dvyukov/go-fuzz would rewrite your sources
| to inject the instrumentation and pass the rewritten sources to
| the compiler, but it didn't really work with Go modules. This
| is something that probably makes sense to integrate with the
| compiler toolchain, same as `go test`'s coverage testing.
| spicybright wrote:
| Probably to encourage a standard fuzzing library for the
| language. Like the same way go fmt is part of the language.
| jiehong wrote:
| It's a bit surprising to have fuzzing as part of the standard
| tool chain, but not property based testing.
|
| I guess we still need to rely on something like gopter.
| kardianos wrote:
| Use table driven tests.
|
| func TestFoo(t *testing.T) { list := []struct {
| want string input string expectedErr error
| }{ {item1}, {item2}, } runtest
| lalaithion wrote:
| go test fuzz v1 string("000000000000000000000000000000O00
| 000000000000000000000000000") rune('\u0083')
| int(60)
|
| Interesting that the minimization engine wasn't able to shrink
| this further.
| evmunro wrote:
| I noticed that as well - most fuzzers will have a maximum
| duration or number of iterations they're allowed to attempt
| when minimizing so as not to starve out actual inputs. It could
| be that the fuzzer hit that limit, or potentially prioritizes
| readable inputs over small inputs.
| lalaithion wrote:
| For string inputs, some form of binary search ("Check if the
| bug exists in the first half or second half of the string")
| would be able to reduce this example to "O" in only a few
| iterations. Not sure if this just isn't implemented, or
| whether there's something more complex going on.
|
| There's also the fact that I'd expect a fuzzer that knows
| about Unicode and UTF-8 strings to have a known list of weird
| behavior hardcoded as seed values, and certainly two-byte
| runes would be on that list.
|
| Of course, this is only the first release with the fuzzer,
| and it already looks really amazing - all I'm really saying
| here is that I can't wait for these to be features of the
| fuzzer in the future!
| disposedtrolley wrote:
| You might be interested in reading about delta debugging:
| https://en.m.wikipedia.org/wiki/Delta_debugging
|
| The algorithm has similar complexity as binary search, but
| is a bit smarter on deciding how to split the test input at
| each iteration.
|
| I've been studying this in my masters, and we've recently
| had to write a Java implementation. I'm keen to start on a
| Go package soon that might work well with fuzz testing.
| evmunro wrote:
| I agree - I took a look at the minimization algorithm[0]
| and it seems like it loops through a few basic options,
| with the last one basically normalizing all possible bytes
| to something readable (like "0"). Part of the issue with
| trying to be as generic as possible is you sometimes can't
| find the best solution to every problem, this might be one
| of those situations.
|
| I know the goal of 1.18 was to get the UX down, so I'm
| interested to see how it improves for 1.19.
|
| [0] https://github.com/golang/go/blob/master/src/internal/f
| uzz/m...
| spencerwgreene wrote:
| I'm pretty sure fmt.Println("HELLO", str, len(str), len(result),
| n) isn't supposed to be in the final solution? :)
| andrei wrote:
| Good catch :) fixed!
___________________________________________________________________
(page generated 2022-03-29 23:01 UTC)