go-mutesting
Mutation testing for Go with CI quality gates, coverage-aware MSI, baseline tracking, and git-diff filtering
About go-mutesting
This large README is shown as a plain-text preview. Read the full README at the source
> **This repository is being superseded.**
> Development has moved to [quality-gates/mutago](https://github.com/quality-gates/mutago).
> This fork will be made read-only once the transition is complete.
> Please use **mutago** for new projects.
# go-mutesting [](https://pkg.go.dev/github.com/jonbaldie/go-mutesting/v2) [](https://github.com/jonbaldie/go-mutesting/actions/workflows/mutation.yml) [](LICENSE) [](https://go.dev)
go-mutesting is a mutation testing tool for Go. It tweaks your code in small ways and checks whether your tests catch the change. If they don't, that's a gap in your test suite worth closing.
## Features
Beyond finding escaped mutants, go-mutesting can enforce quality gates in CI — failing builds below a mutation score threshold, filtering to changed lines only, and ignoring previously-accepted survivors.
| Feature | Flag |
| :--- | :--- |
| Quality gates — fail CI below a mutation score | `--min-msi`, `--min-covered-msi` |
| Coverage-aware MSI — score covered lines separately | `--coverage` |
| Baseline file — only fail on *new* escapes | `--baseline`, `--update-baseline` |
| Git diff filter — only mutate changed lines in a PR | `--git-diff-lines` |
| Per-test filtering — run only covering tests per mutant | `--per-test` |
| Parallel execution (all CPUs by default) | `--workers N` |
| LLM-ready escaped-mutant report | `--logger-agentic-json` |
| GitHub Actions annotations | `--logger-github` |
| GitLab Code Quality report | `--logger-gitlab` |
| Compact stats JSON for badges/dashboards | `--logger-summary-json` |
| Per-mutator allowlist / denylist in config | `enable_mutators`, `disable_mutators` |
| Extra flags for every `go test` call | `--test-flags` |
| Vet disabled for built-in `go test` runs (`-vet=off`) | `--test-flags=-vet=all` to re-enable |
| Mutant runs stop at the first failing test (`-failfast`) | `--test-flags=-failfast=false` to run every test |
| Fine-grained output filter | `--output-statuses` |
| Quiet mode — suppress killed/skip noise | `--quiet` |
| Suppress diff output | `--no-diffs` |
| Dry-run mode — count mutations without running tests | `--dry-run` |
| Baseline pre-flight check — fail fast if tests already broken | default (was `--noop`) |
| Fail on any escape without a score threshold | `--fail-on-escaped` |
| Run a single mutant by stable ID | `--run-mutant-id` |
| Scale per-mutation timeout by baseline run time | `--timeout-coefficient` |
| Live progress display | automatic on TTY |
```bash
go install github.com/jonbaldie/go-mutesting/v2/cmd/go-mutesting@latest
go-mutesting --version
```
Full documentation: https://jonbaldie.github.io/go-mutesting/
Forked from [avito-tech/go-mutesting](https://github.com/avito-tech/go-mutesting), itself a fork of [zimmski/go-mutesting](https://github.com/zimmski/go-mutesting).
## Custom mutators
Register your own operators with `mutator.Register`, then blank-import their
package in a full go-mutesting clone and build a custom binary. Copying only
`cmd/go-mutesting/main.go` into another module will not build because it uses internal
packages. See the [custom mutator guide](docs/custom-mutators.md) for a walkthrough.
## Quick example
The following command mutates the go-mutesting project with all available mutators.
```bash
go-mutesting github.com/jonbaldie/go-mutesting/v2/...
```
For each mutation the tool prints whether the tests caught it. If they didn't, the source code patch is printed so the mutation can be investigated. The following shows an example patch.
```diff
for _, d := range opts.Mutator.DisableMutators {
pattern := strings.HasSuffix(d, "*")
- if (pattern && strings.HasPrefix(name, d[:len(d)-2])) || (!pattern && name == d) {
+ if (pattern && strings.HasPrefix(name, d[:len(d)-2])) || false {
continue MUTATOR
}
}
```
The example shows that the right term `(!pattern && name == d)` of the `||` operator is made irrelevant by substituting it with `false`. Since this source code change is not detected by the test suite (the tests did not fail), we can mark it as untested code.
## <a name="table-of-content"></a>Table of contents
- [What is mutation testing?](#what-is-mutation-testing)
- [How do I use go-mutesting?](#how-do-i-use-go-mutesting)
- [Blacklist false positives](#black-list-false-positives)
- [Quality gates](#quality-gates)
- [Git diff filtering](#git-diff)
- [Baseline — only fail on new escapes](#baseline)
- [LLM-ready report](#agentic-json)
- [Live progress](#progress)
- [CI integration (GitHub Actions)](#ci-integration)
- [JSON output schemas](#json-outputs)
- [Mutation control via annotations](#mutation-annotations)
- [How do I write my own mutation exec commands?](#write-mutation-exec-commands)
- [Which mutators are implemented?](#list-of-mutators)
- [Other mutation testing tools for Go](#other-projects)
- [Can I make feature requests and report bugs and problems?](#feature-request)
## <a name="what-is-mutation-testing"></a>What is mutation testing?
The definition of mutation testing is best quoted from Wikipedia:
> Mutation testing (or Mutation analysis or Program mutation) is used to design new software tests and evaluate the quality of existing software tests. Mutation testing involves modifying a program in small ways. Each mutated version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant. This is called killing the mutant. Test suites are measured by the percentage of mutants that they kill. New tests can be designed to kill additional mutants.
> <br/>-- <cite>[https://en.wikipedia.org/wiki/Mutation_testing](https://en.wikipedia.org/wiki/Mutation_testing)</cite>
> Tests can be created to verify the correctness of the implementation of a given software system, but the creation of tests still poses the question whether the tests are correct and sufficiently cover the requirements that have originated the implementation.
> <br/>-- <cite>[https://en.wikipedia.org/wiki/Mutation_testing](https://en.wikipedia.org/wiki/Mutation_testing)</cite>
Although the definition focuses on finding code paths not covered by tests, other flaws can be found too. Mutation testing can for example uncover dead and unneeded code.
Mutation testing is also especially interesting for comparing automatically generated test suites with manually written ones.
It is also one of the strongest tools available for keeping AI-generated code honest. AI tools write plausible-looking code that often slips past code review. Mutation testing checks whether your tests would actually catch a bug — not just whether the code looks right.
## <a name="how-do-i-use-go-mutesting"></a>How do I use go-mutesting?
Install the binary with `go install`:
```bash
go install -v github.com/jonbaldie/go-mutesting/v2/...
```
The binary's help can be invoked by executing the binary without arguments or with the `--help` argument.
```bash
go-mutesting --help
```
> **Note**: This README describes only a few of the available arguments. It is therefore advisable to examine the output of the `--help` argument.
The targets of the mutation testing can be defined as arguments to the binary. Every target can be either a Go source file, a directory or a package. Directories and packages can also include the `...` wildcard pattern which will search recursively for Go source files. Test source files with the suffix `_test` are excluded, since this would interfere with the testing process most of the time.
The following example gathers all Go files from the given targets and generates mutations with all available mutators.
```bash
go-mutesting parse.go example/ github.com/jonbaldie/go-mutesting/v2/mutator/...
```
Every mutation has to be tested using an [exec command](#write-mutation-exec-commands). By default the built-in exec command is used, which tests a mutation using the following steps:
- Replace the original file with the mutation.
- Execute all tests of the package of the mutated file, with `go vet` disabled (`-vet=off`) so a vet diagnostic on mutated code is not miscounted as a kill. Pass your own `-vet` via `--test-flags` to re-enable it. The run uses `-failfast`, so no new tests start after the first failure.
- Report if the mutation was killed.
Alternatively the `--exec` argument can be used to invoke an external exec command. The [/scripts/exec](/scripts/exec) directory holds basic exec commands for Go projects. The [test-mutated-package.sh](/scripts/exec/test-mutated-package.sh) script implements all steps and almost all features of the built-in exec command. It can be for example used to test the [github.com/jonbaldie/go-mutesting/v2/example](/example) package.
```bash
go-mutesting --exec "$GOPATH/src/github.com/jonbaldie/go-mutesting/scripts/exec/test-mutated-package.sh" github.com/jonbaldie/go-mutesting/v2/example
```
The execution will print the following output.
```diff
KILLED example/example.go:18 (statement/remove)
KILLED example/example.go:22 (branch/if)
KILLED example/example.go:24 (numbers/incrementer)
--- Original
+++ New
@@ -16,7 +16,7 @@
}
if n < 0 {
- n = 0
+
}
n++
ESCAPED example/example.go:17 (statement/remove)
KILLED example/example.go:26 (arithmetic/base)
KILLED example/example.go:28 (expression/remove)
--- Original
+++ New
@@ -24,7 +24,6 @@
n += bar()
bar()
- bar()
return n
}
ESCAPED example/example.go:25 (statement/remove)
KILLED example/example.go:30 (branch/if)
The mutation score is 75.00% (6 killed, 2 escaped, 0 errored, 0 not covered, 0 skipped, 8 total)
The covered-code mutation score is 0.00%
```
The output shows eight mutations. Six were killed (tests detected the mutation — shown as `KILLED`). Two escaped (tests didn't catch them — shown as `ESCAPED`), and their diffs are printed so you can write a test to cover the gap.
The summary shows the **mutation score** (MSI): killed / total. For the example above, 6/8 = 75.00%. A score of 100% means every mutation was caught.
### <a name="black-list-false-positives"></a>Blacklist false positives
Mutation testing can produce false positives when the mutated code path is never reachable, or when the unoptimized path produces the same result as the optimized one. These cases are not bugs in your tests — they just aren't worth tracking down.
Use `--blacklist` with a file that lists the MD5 checksum of each mutation to ignore (one per line). Checksums are derived from the mutated file, the position of the change, and only the lines that actually changed, so they stay valid when unrelated code in the same file is edited. Checksums generated by go-mutesting v2.10.1 or earlier are not valid for v2.10.2 and later.
To get the checksum for a mutation, run go-mutesting with `--debug` and copy the 32-character hex string printed next to the mutation. For example, a result line looks like this:
```
ESCAPED example/example.go:25 (statement/remove) [checksum: a1b2c3d4e5f60123456789abcdef0123]
```
Copy only the value after `checksum:` into a blacklist file:
```
a1b2c3d4e5f60123456789abcdef0123
```
The blacklist file, which is named `example.blacklist` in this example, can then be used to invoke go-mutesting.
```bash
go-mutesting --blacklist example.blacklist github.com/jonbaldie/go-mutesting/v2/example
```
The execution will print the following output.
```diff
KILLED example/example.go:18 (statement/remove)
KILLED example/example.go:22 (branch/if)
KILLED example/example.go:24 (numbers/incrementer)
--- Original
+++ New
@@ -16,7 +16,7 @@
}
if n < 0 {
- n = 0
+
}
n++
ESCAPED example/example.go:17 (statement/remove)
KILLED example/example.go:26 (arithmetic/base)
KILLED example/example.go:28 (expression/remove)
KILLED example/example.go:30 (branch/if)
The mutation score is 85.71% (6 killed, 1 escaped, 0 errored, 0 not covered, 0 skipped, 7 total)
The covered-code mutation score is 0.00%
```
By comparing this output to the original output we can state that we now have 7 mutations instead of 8.
### <a name="skip-make-args"></a>Skipping make() arguments mutation
Before this filter, numeric arguments in make() calls for slices, maps, and channels were mutated by incrementer/decrementer mutators, leading to false positives or invalid code:
```go
// Original code
slice := make([]int, 0) // Capacity argument (0) was mutated
channel := make(chan int, 0) // Buffer argument (0) was mutated
// Mutated versions
slice := make([]int, 1) // Incrementer mutation
slice := make([]int, -1) // Decrementer mutation
```
These mutations are almost always irrelevant because:
1. They don't affect logical correctness
2. Capacity/length arguments are typically intentional
3. Tests rarely validate exact allocation sizes
The filter prevents mutations in make() arguments for slices, maps, and channels.
### <a name="quality-gates"></a>Quality gates
Use `--min-msi` and `--min-covered-msi` to fail CI if mutation scores drop below a threshold. The tool exits with code 4 when a gate isn't met.
```bash
go-mutesting --min-msi 60 --min-covered-msi 80 ./...
```
Add `--coverage` to generate a coverage profile first. Mutants on uncovered lines skip test execution, are marked "not covered", and are excluded from the covered-MSI denominator (so you're not penalised for code your tests don't reach at all). If the coverage run fails, go-mutesting stops with exit code 3 rather than using a partial profile.
```bash
go-mutesting --coverage --min-msi 50 --min-covered-msi 75 ./...
```
The final summary includes a per-mutator breakdown so you can see which mutation types your tests are weakest against.
By default go-mutesting runs the test suite once without any mutations first (the baseline pre-flight check). If the clean suite already fails — the package does not compile, or a test is red — go-mutesting exits immediately with a tool error (exit 3) rather than producing meaningless results. The `--noop` flag is kept for backward compatibility but has no effect; the check is always on for normal runs (it is skipped under `--coverage`, `--no-exec`, `--dry-run`, or a custom `--exec`).
If a generated mutant does not compile, it is skipped rather than counted as killed by a test.
You can run go-mutesting from outside the targets' module, such as from a parent directory or from another module. It runs `go` in the targets' module, so the result matches a run from inside it. Targets that span more than one module, or are not in a module, are a tool error (exit 3); run go-mutesting once per module. This check is skipped when a `go.work` workspace is in use, and when a custom `--exec` runs the tests.
Use `--timeout-coefficient` to scale the per-mutation timeout relative to an uncached baseline test-suite run (e.g. `--timeout-coefficient 3` allows each mutation up to 3× the clean run). The clean pre-flight run uses a generous cap so a suite slower than `--exec-timeout` can still be measured; the derived timeout then applies to mutant runs. go-mutesting adds `-count=1` unless `--test-flags` already contains a positive `-count=N`. More reliable than a fixed `--exec-timeout` on machines with variable load.
When `--coverage` is enabled, a failed clean coverage run stops go-mutesting with exit code 3. A partial coverage profile from failed tests is never used to classify mutants.
### <a name="git-diff"></a>Git diff filtering (CI mode)
`--git-diff-lines` limits mutation to lines changed since a given git ref. The comparison is made against the **merge-base** of that ref and your current branch, so it mutates exactly the lines a pull request shows — even when your branch is behind the target. Commits that landed on the target branch after you branched off are not attributed to your work. Combine it with `--ignore-msi-with-no-mutations` so the gate passes cleanly on PRs that touch no mutable code.
```bash
go-mutesting \
--git-diff-lines \
--git-diff-base master \
--ignore-msi-with-no-mutations \
--min-msi 80 \
./...
```
Add `--logger-github` to emit escaped mutants as GitHub Actions `::warning` annotations, so they show up inline on the PR diff.
### <a name="baseline"></a>Baseline — only fail on new escapes
If you have existing mutants that survive and your team has accepted them, you can record them in a baseline file. Future runs only fail when a *new* mutant escapes — not an already-known one.
```bash
# First run: record the current survivors and exit 0
go-mutesting --update-baseline ./...
# Normal CI run: fail only if something new escapes
go-mutesting --fail-on-escaped --baseline go-mutesting-baseline.json ./...
```
Commit `go-mutesting-baseline.json` to your repo. The baseline uses stable mutant IDs — they survive refactors that shift line numbers without changing the actual code. When one mutator makes the same change to identical lines in one file, each mutant gets its own ID, numbered in source order, so accepting one does not accept the others.
### <a name="agentic-json"></a>LLM-ready report
`--logger-agentic-json` writes `go-mutesting-agentic.json`. Each escaped mutant gets a stable ID, the diff, surrounding context lines, nearby test file paths, a plain-English description of what the mutator did, and a hint for writing a killing test. Feed it to an LLM to get targeted test suggestions.
```bash
go-mutesting --logger-agentic-json --quiet ./...
```
Use `--run-mutant-id` to re-run a single mutant by its stable ID (copy the `id` field from `go-mutesting-agentic.json`). Other mutants are not classified or counted, and `--dry-run` counts only that mutant. Useful for iterating on a specific test gap without waiting for the full suite. A missing ID is reported as a tool error and exits with code 3; valid-ID runs suppress the summary and quality gates.
### <a name="progress"></a>Live progress
When running in a terminal, go-mutesting shows a live progress line on stderr (killed / escaped / skip counts). It clears automatically before the final summary. It is suppressed in `--verbose`, `--debug`, and silent mode.
### <a name="ci-integration"></a>CI integration (GitHub Actions)
A minimal workflow that gates on mutation score. Adjust thresholds to match your project's baseline.
```yaml
name: Mutation Testing
on:
push:
branches: [master]
pull_request:
jobs:
mutating:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/setup-go@v6
with:
go-version: stable
- run: go build -o /tmp/go-mutesting ./cmd/go-mutesting
- run: |
/tmp/go-mutesting \
--coverage \
--min-msi 70 \
--min-covered-msi 80 \
--logger-github \
./...
```
`--logger-github` emits escaped mutants as `::warning` annotations that appear inline on the PR diff. `--coverage` excludes untested code from the covered-MSI denominator so you aren't penalised for dead code your tests never reach.
For GitLab, replace `--logger-github` with `--logger-gitlab`. This writes `go-mutesting-gitlab.json` in GitLab Code Quality format, which GitLab CI picks up as a code quality report on merge requests.
**Adopting gates on a legacy codebase**: record the current survivors first, then only fail on new regressions.
```bash
# Run once to capture current state
go-mutesting --update-baseline ./...
git add go-mutesting-baseline.json
git commit -m "chore: establish mutation baseline"
# CI: only fail if something new escapes
go-mutesting --baseline go-mutesting-baseline.json --fail-on-escaped ./...
```
### <a name="json-outputs"></a>JSON output schemas
Frequently Asked Questions
What is go-mutesting?
go-mutesting is a Testing library for the Go programming language. Mutation testing for Go with CI quality gates, coverage-aware MSI, baseline tracking, and git-diff filtering
How do I install go-mutesting?
Install go-mutesting with the Go module system using `go get jonbaldie/go-mutesting`. Check the repository for the current installation instructions.
What category does go-mutesting belong to?
go-mutesting is listed under Testing, specifically Testing Frameworks.