go-fuzz

About go-fuzz

This large README is shown as a plain-text preview. Read the full README at the source

# go-fuzz: randomized testing for Go

Go-fuzz is a coverage-guided [fuzzing solution](http://en.wikipedia.org/wiki/Fuzz_testing) for testing of Go packages.
Fuzzing is mainly applicable to packages that parse complex inputs (both text
and binary), and is especially useful for hardening of systems that parse inputs
from potentially malicious users (e.g. anything accepted over a network).

**Note:** go-fuzz has recently added preliminary support for fuzzing [Go Modules](https://github.com/golang/go/wiki/Modules).  See the [section below](https://github.com/dvyukov/go-fuzz/blob/master/README.md#modules-support) for more details. 
If you encounter a problem with modules, please file an issue with details. A workaround might be to disable modules via `export GO111MODULE=off`.

## Usage

First, you need to write a test function of the form:
```go
func Fuzz(data []byte) int
```
Data is a random input generated by go-fuzz, note that in most cases it is
invalid. The function must return 1 if the fuzzer should increase priority
of the given input during subsequent fuzzing (for example, the input is
lexically correct and was parsed successfully); -1 if the input must not be
added to corpus even if gives new coverage; and 0 otherwise; other values are
reserved for future use.

The `Fuzz` function must be in a package that `go-fuzz` can import. This means
the code you want to test can't be in package `main`.  Fuzzing `internal`
packages is supported, however.

In its basic form the Fuzz function just parses the input, and
go-fuzz ensures that it does not panic, crash the program, allocate insane
amount of memory nor hang. Fuzz function can also do application-level checks,
which will make testing more efficient (discover more bugs). For example,
Fuzz function can serialize all inputs that were successfully deserialized,
thus ensuring that serialization can handle everything deserialization can
produce. Or, Fuzz function can deserialize-serialize-deserialize-serialize
and check that results of first and second serialization are equal. Or, Fuzz
function can feed the input into two different implementations (e.g. dumb and
optimized) and check that the output is equal. To communicate application-level
bugs Fuzz function should panic (os.Exit(1) will work too, but panic message
contains more info). Note that Fuzz function should not output to stdout/stderr,
it will slow down fuzzing and nobody will see the output anyway. The exception
is printing info about a bug just before panicking.

Here is an example of a simple Fuzz function for image/png package:
```go
package png

import (
	"bytes"
	"image/png"
)

func Fuzz(data []byte) int {
	png.Decode(bytes.NewReader(data))
	return 0
}
```

A more useful Fuzz function would look like:
```go
func Fuzz(data []byte) int {
	img, err := png.Decode(bytes.NewReader(data))
	if err != nil {
		if img != nil {
			panic("img != nil on error")
		}
		return 0
	}
	var w bytes.Buffer
	err = png.Encode(&w, img)
	if err != nil {
		panic(err)
	}
	return 1
}
```

The second step is collection of initial input corpus. Ideally, files in the
corpus are as small as possible and as diverse as possible. You can use inputs
used by unit tests and/or generate them. For example, for an image decoding
package you can encode several small bitmaps (black, random noise, white with
few non-white pixels) with different levels of compressions and use that as the
initial corpus. Go-fuzz will deduplicate and minimize the inputs. So throwing in
a thousand of inputs is fine, diversity is more important.

Put the initial corpus into the workdir/corpus directory (in our case
```examples/png/corpus```). Go-fuzz will add own inputs to the corpus directory.
Consider committing the generated inputs to your source control system, this
will allow you to restart go-fuzz without losing previous work.

The [go-fuzz-corpus repository](https://github.com/dvyukov/go-fuzz-corpus) contains 
a bunch of examples of test functions and initial input corpuses for various packages.

The next step is to get go-fuzz:

```
$ go install github.com/dvyukov/go-fuzz/go-fuzz@latest github.com/dvyukov/go-fuzz/go-fuzz-build@latest
```

Then, download the corpus and build the test program with necessary instrumentation:
```
$ git clone https://github.com/dvyukov/go-fuzz-corpus.git
$ cd go-fuzz-corpus
$ cd png
$ go-fuzz-build
```
This will produce png-fuzz.zip archive.

Now we are ready to go:
```
$ go-fuzz
```

Go-fuzz will generate and test various inputs in an infinite loop. Workdir is
used to store persistent data like current corpus and crashers, it allows fuzzer
to continue after restart. Discovered bad inputs are stored in workdir/crashers
dir; where file without a suffix contains binary input, file with .quoted suffix
contains quoted input that can be directly copied into a reproducer program or a
test, file with .output suffix contains output of the test on this input. Every
few seconds go-fuzz prints logs to stderr of the form:
```
2015/04/25 12:39:53 workers: 500, corpus: 186 (42s ago), crashers: 3,
     restarts: 1/8027, execs: 12009519 (121224/sec), cover: 2746, uptime: 1m39s
```
Where ```workers``` means number of tests running in parallel (set with -procs
flag). ```corpus``` is current number of interesting inputs the fuzzer has
discovered, time in brackets says when the last interesting input was
discovered. ```crashers``` is number of discovered bugs (check out
workdir/crashers dir). ```restarts``` is the rate with which the fuzzer restarts
test processes. The rate should be close to 1/10000 (which is the planned
restart rate); if it is considerably higher than 1/10000, consider fixing already
discovered bugs which lead to frequent restarts. ```execs``` is total number of
test executions, and the number in brackets is the average speed of test
executions. ```cover``` is number of bits set in a hashed coverage bitmap, if this number
grows fuzzer uncovers new lines of code; size of the bitmap is 64K; ideally ```cover```
value should be less than ~5000, otherwise fuzzer can miss new interesting inputs
due to hash collisions. And finally ```uptime``` is uptime of the process. This same
information is also served via http (see the ```-http``` flag).

## Modules support

go-fuzz has preliminary support for fuzzing [Go Modules](https://github.com/golang/go/wiki/Modules). 
go-fuzz respects the standard `GO111MODULE` environment variable, which can be set to `on`, `off`, or `auto`. 

go-fuzz-build will add a `require` for `github.com/dvyukov/go-fuzz` to your go.mod. If desired, you may remove this once the build is complete.

Vendoring with modules is not yet supported. A `vendor` directory will be ignored, and go-fuzz will report an error if `GOFLAGS=-mod=vendor` is set.

Note that while modules are used to prepare the build, the final instrumented build is still done in GOPATH mode.
For most modules, this should not matter.

## libFuzzer support

go-fuzz-build can also generate an archive file
that can be used with [libFuzzer](https://llvm.org/docs/LibFuzzer.html)
instead of go-fuzz (requires linux).

Sample usage:

```
$ cd $GOPATH/src/github.com/dvyukov/go-fuzz-corpus/fmt
$ go-fuzz-build -libfuzzer  # produces fmt.a
$ clang -fsanitize=fuzzer fmt.a -o fmt.libfuzzer
$ ./fmt.libfuzzer
```

When run with `-libfuzzer`, go-fuzz-build adds the additional build tag
`gofuzz_libfuzzer` when building code.

## Continuous Fuzzing

Just as unit-testing, fuzzing is better done continuously.

Currently there are 2 services that offer continuous fuzzing based on go-fuzz:

- [gitlab.com](https://gitlab.com) ([tutorial](https://docs.gitlab.com/ee/user/application_security/coverage_fuzzing/))
- [fuzzbuzz.io](https://fuzzbuzz.io/) ([tutorial](https://docs.fuzzbuzz.io/getting-started/find-your-first-bug-in-go))

## Random Notes

go-fuzz-build builds the program with gofuzz build tag, this allows to put the
Fuzz function implementation directly into the tested package, but exclude it
from normal builds with ```// +build gofuzz``` directive.

If your inputs contain a checksum, it can make sense to append/update the checksum
in the ```Fuzz``` function. The chances that go-fuzz will generate the correct
checksum are very low, so most work will be in vain otherwise.

Go-fuzz can utilize several machines. To do this, start the coordinator process
separately:
```
$ go-fuzz -workdir=examples/png -coordinator=127.0.0.1:8745
```
It will manage persistent corpus and crashers and coordinate work of worker processes.
Then run one or more worker processes as:
```
$ go-fuzz -bin=./png-fuzz.zip -worker=127.0.0.1:8745 -procs=10
```

## External Articles

- [go-fuzz github.com/arolek/ase](https://medium.com/@dgryski/go-fuzz-github-com-arolek-ase-3c74d5a3150c): A step-by-step tutorial
- [DNS parser, meet Go fuzzer](https://blog.cloudflare.com/dns-parser-meet-go-fuzzer/): A success story with suggestions on how to write the ```Fuzz``` function
- [Automated Testing with go-fuzz](https://speakerdeck.com/filosottile/automated-testing-with-go-fuzz)
- [Going down the rabbit hole with go-fuzz](https://mijailovic.net/2017/07/29/go-fuzz/)
- [Fuzzing markdown parser with go-fuzz](https://blog.kowalczyk.info/article/n/fuzzing-markdown-parser-written-in-go.html)

## History rewrite

go-fuzz repository history was recently rewritten to exclude examples directory
to reduce total repository size and download time (see
[#88](https://github.com/dvyukov/go-fuzz/issues/88),
[#114](https://github.com/dvyukov/go-fuzz/issues/114) and
https://github.com/dvyukov/go-fuzz-corpus). Unfortunately, that means that
`go get -u` command will fail if you had a previous version installed.
Please remove $GOPATH/github.com/dvyukov/go-fuzz before running `go get` again.

## Credits and technical details

Go-fuzz fuzzing logic is heavily based on [american fuzzy lop](http://lcamtuf.coredump.cx/afl/),
so refer to [AFL readme](http://lcamtuf.coredump.cx/afl/README.txt) if you are
interested in technical details. AFL is written and maintained by
[Michal Zalewski](http://lcamtuf.coredump.cx/). Some of the mutations employed
by go-fuzz are inspired by work done by Mateusz Jurczyk, Gynvael Coldwind and
[Felix Gröbert](https://twitter.com/fel1x).

## Trophies

- [spec: non-integral constant can be converted to int](https://github.com/golang/go/issues/11350) **fixed**
- [cmd/compile: out of fixed registers](https://github.com/golang/go/issues/11352) **fixed**
- [cmd/compile: truncates constants](https://github.com/golang/go/issues/11326) **fixed**
- [cmd/compile: overflow in int -> string](https://github.com/golang/go/issues/11330) **fixed**
- [cmd/compile: bad HMUL](https://github.com/golang/go/issues/11358) **fixed**
- [cmd/compile: treecopy Name](https://github.com/golang/go/issues/11361) **fixed**
- [cmd/compile: accepts invalid identifiers](https://github.com/golang/go/issues/11359) **fixed**
- [cmd/compile: hangs compiling hex fp constant](https://github.com/golang/go/issues/11364) **fixed**
- [cmd/compile: mishandles int->complex conversion](https://github.com/golang/go/issues/11365) **fixed**
- [cmd/compile: allows to define blank methods on builtin types](https://github.com/golang/go/issues/11366) **fixed**
- [cmd/compile: mis-calculates a constant](https://github.com/golang/go/issues/11369) **fixed**
- [cmd/compile: interface conversion panic](https://github.com/golang/go/issues/11540) **fixed**
- [cmd/compile: nil pointer dereference](https://github.com/golang/go/issues/11588) **fixed**
- [cmd/compile: nil pointer dereference (2)](https://github.com/golang/go/issues/11666) **fixed**
- [cmd/compile: internal compiler error: plain block b3 len(Succs)==2, want 1](https://github.com/golang/go/issues/11589) **fixed**
- [cmd/compile: internal compiler error: b3.Succs has duplicate block b3](https://github.com/golang/go/issues/11593) **fixed**
- [cmd/compile: internal compiler error: newname nil](https://github.com/golang/go/issues/11610) **fixed**
- [cmd/compile: accepts invalid function type](https://github.com/golang/go/issues/11595) **fixed**
- [cmd/compile: internal compiler error: getinarg: not a func int](https://github.com/golang/go/issues/11596) **fixed**
- [cmd/compile: hangs converting int const to complex64](https://github.com/golang/go/issues/11597) **fixed**
- [cmd/compile: nil deref in error message](https://github.com/golang/go/issues/11614) **fixed**
- [cmd/compile: use of untyped nil in switch](https://github.com/golang/go/issues/11668) **fixed**
- [cmd/compile: implicitly converts complex constant to integer](https://github.com/golang/go/issues/11669) **fixed**
- [cmd/compile: assignment to entry in nil map](https://github.com/golang/go/issues/11670) **fixed**
- [cmd/compile: does not diagnose constant division by zero](https://github.com/golang/go/issues/11674) **fixed**
- [cmd/compile: does not detect a missing return](https://github.com/golang/go/issues/11698) **fixed**
- [cmd/compile: symbol ""._.args_stackmap listed multiple times](https://github.com/golang/go/issues/11699) **fixed**
- [cmd/compile: "0"[0] should not be a constant](https://github.com/golang/go/issues/11370) **fixed**
- [cmd/compile: unexpected %!(NOVERB)](https://github.com/golang/go/issues/13266) **fixed**
- [cmd/compile: wrong line number in error message](https://github.com/golang/go/issues/13267) **fixed**
- [cmd/compile: not-deterministic output](https://github.com/golang/go/issues/13268) **fixed**
- [cmd/compile: parsing problem](https://github.com/golang/go/issues/13270) **fixed**
- [cmd/compile: compiles incorrect program](https://github.com/golang/go/issues/13272) **fixed**
- [cmd/compile: does not compile correct program](https://github.com/golang/go/issues/13273)
- [cmd/compile: compiles incorrect program (2)](https://github.com/golang/go/issues/13274) **fixed**
- [cmd/compile: internal compiler error: want FUNC, but have int](https://github.com/golang/go/issues/17038) **fixed**
- [cmd/compile: nil deref](https://github.com/golang/go/issues/24447) **fixed**
- [cmd/asm: index out of range](https://github.com/golang/go/issues/11759) **fixed**
- [cmd/asm: index out of range (2)](https://github.com/golang/go/issues/12466) **fixed**
- [cmd/asm: index out of range (3)](https://github.com/golang/go/issues/12467) **fixed**
- [cmd/asm: index out of range (4)](https://github.com/golang/go/issues/12657) **fixed**
- [cmd/asm: slice bounds out of range](https://github.com/golang/go/issues/11760) **fixed**
- [cmd/asm: hang](https://github.com/golang/go/issues/11764) **fixed**
- [cmd/asm: hang (2)](https://github.com/golang/go/issues/12469) **fixed**
- [cmd/asm: hang (3)](https://github.com/golang/go/issues/12656) **fixed**
- [cmd/asm: nil deref](https://github.com/golang/go/issues/11765) **fixed**
- [cmd/asm: nil deref (2)](https://github.com/golang/go/issues/12468) **fixed**
- [cmd/asm: nil deref (3)](https://github.com/golang/go/issues/12614) **fixed**
- [cmd/asm: nil deref (4)](https://github.com/golang/go/issues/12627) **fixed**
- [cmd/asm: nil deref (5)](https://github.com/golang/go/issues/12655) **fixed**
- [cmd/asm: cannot happen: slice col](https://github.com/golang/go/issues/12654) **fixed**
- [cmd/asm: unactionable "invalid local variable type 0"](https://github.com/golang/go/issues/12658)
- [internal/trace: index out of range](https://github.com/golang/go/issues/11766) **fixed**
- [internal/trace: index out of range (2)](https://github.com/golang/go/issues/11769) **fixed**
- [internal/trace: nil deref](https://github.com/golang/go/issues/11767) **fixed**
- [internal/trace: nil deref (2)](https://github.com/golang/go/issues/11770) **fixed**
- [fmt: Printf loops on invalid verb spec](https://github.com/golang/go/issues/10674) **fixed**
- [fmt: incorrect overflow detection](https://github.com/golang/go/issues/10695) **fixed**
- [fmt: index out of range](https://github.com/golang/go/issues/10675) **fixed**
- [fmt: index out of range (2)](https://github.com/golang/go/issues/10745) **fixed**
- [fmt: index out of range (3)](https://github.com/golang/go/issues/10770) **fixed**
- [fmt: index out of range (4)](https://github.com/golang/go/issues/10771) **fixed**
- [fmt: index out of range (5)](https://github.com/golang/go/issues/10945) **fixed**
- [fmt: index out of range (6)](https://github.com/golang/go/issues/11376) **fixed**
- [regexp: slice bounds out of range](https://github.com/golang/go/issues/11176) **fixed**
- [regexp: slice bounds out of range (2)](https://github.com/golang/go/issues/11178) **fixed**
- [regexp: LiteralPrefix lies about completeness](https://github.com/golang/go/issues/11172)
- [regexp: LiteralPrefix lies about completeness (2)](https://github.com/golang/go/issues/11175)
- [regexp: POSIX regexp takes 4 seconds to execute](https://github.com/golang/go/issues/11181)
- [regexp: confusing behavior on invalid utf-8 sequences](https://github.com/golang/go/issues/11185)
- [regexp: considers "\Q\E*" as valid regexp](https://github.com/golang/go/issues/11187) **fixed**
- [time: allows signs for year/tz in format string](https://github.com/golang/go/issues/11128)
- [time: RFC3339 time.Parse can not parse string that come from time.Format](https://github.com/golang/go/issues/20555)
- [time: Parse panic: runtime error: index out of range](https://github.com/golang/go/issues/21113) **fixed**
- [math/big: incorrect string->Float conversion](https://github.com/golang/go/issues/11341) **fixed**
- [math/big: MakeFromLiteral with 0 mantissa and large exponent hangs](https://github.com/golang/go/issues/16176) **fixed**
- [net/http: can't send star request](https://github.com/golang/go/issues/11202) **fixed**
- [net/http: allows empty header names](https://github.com/golang/go/issues/11205) **fixed**
- [net/http: allows invalid characters in header values](https://github.com/golang/go/issues/11207) **fixed**
- [net/http: allows %-encoding after \[\]](https://github.com/golang/go/issues/11208) **fixed**
- [net/mail: ParseAddress/String corrupt address](https://github.com/golang/go/issues/11292) **fixed**
- [net/mail: parses invalid address](https://github.com/golang/go/issues/11293) **fixed**
- [net/mail: fails to escape address](https://github.com/golang/go/issues/11294) **fixed**
- [net/textproto: fails to trim header value](https://github.com/golang/go/issues/11204) **fixed**
- [archive/zip: cap out of range](https://github.com/golang/go/issues/10956) **fixed**
- [archive/zip: bad file size](https://github.com/golang/go/issues/10957) **fixed**
- [archive/zip: unexpected EOF](https://github.com/golang/go/issues/11144) **fixed**
- [archive/zip: file with wrong checksum is successfully decompressed](https://github.com/golang/go/issues/11146) **fixed**
- [archive/zip: unexpected EOF when reading archive](https://github.com/golang/go/issues/12449) **fixed**
- [archive/tar: slice bounds out of range](https://github.com/golang/go/issues/10959) **fixed**
- [archive/tar: slice bounds out of range (2)](https://github.com/golang/go/issues/10960) **fixed**
- [archive/tar: slice bounds out of range (3)](https://github.com/golang/go/issues/10966) **fixed**
- [archive/tar: slice bounds out of range (4)](https://github.com/golang/go/issues/10967) **fixed**
- [archive/tar: slice bounds out of range (5)](https://github.com/golang/go/issues/11167) **fixed**
- [archive/tar: deadly hang](https://github.com/golang/go/issues/10968) **fixed**
- [archive/tar: invalid memory address or nil pointer dereference](https://github.com/golang/go/issues/11168) **fixed**
- [archive/tar: invalid memory address or nil pointer dereference (2)](https://github.com/golang/go/issues/12435) **fixed**
- [archive/tar: Reader.Next returns nil header](https://github.com/golang/go/issues/11169) **fixed**
- [archive/tar: Writer incorrectly encodes header data](https://github.com/golang/go/issues/11171) **fixed**
- [archive/tar: incorrectly claims huge file size](https://github.com/golang/go/issues/12434)

Frequently Asked Questions