testifylint

About testifylint

This large README is shown as a plain-text preview. Read the full README at the source

# testifylint

![Latest release](https://img.shields.io/github/v/release/Antonboom/testifylint)
[![CI](https://github.com/Antonboom/testifylint/actions/workflows/ci.yml/badge.svg)](https://github.com/Antonboom/testifylint/actions/workflows/ci.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/Antonboom/testifylint)](https://goreportcard.com/report/github.com/Antonboom/testifylint?dummy=unused)
[![Coverage Status](https://coveralls.io/repos/github/Antonboom/testifylint/badge.svg?branch=master)](https://coveralls.io/github/Antonboom/testifylint?branch=master&dummy=unused)
[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/Antonboom/testifylint/blob/master/CONTRIBUTING.md#Open-for-contribution)

Checks usage of [github.com/stretchr/testify](https://github.com/stretchr/testify).

## Table of Contents

* [Problem statement](#problem-statement)
* [Installation & usage](#installation--usage)
* [Configuring](#configuring)
* [Checkers](#checkers)
* [Chain of warnings](#chain-of-warnings)
* [testify v2](#testify-v2)

## Problem statement

Tests are also program code and the requirements for them should not differ much from the requirements
for the code under tests 🙂

We should try to maintain the consistency of tests, increase their readability, reduce the chance of bugs
and speed up the search for a problem.

[testify](https://github.com/stretchr/testify) is the most popular Golang testing framework* in recent years.
But it has a terrible ambiguous API in places, and **the purpose of this linter is to protect you from annoying
mistakes**.

_* JetBrains "The State of Go Ecosystem" reports
[2021](https://www.jetbrains.com/lp/devecosystem-2021/go/#Go_which-testing-frameworks-do-you-use-regularly-if-any)
and [2022](https://www.jetbrains.com/lp/devecosystem-2022/go/#which-testing-frameworks-do-you-use-regularly-if-any-)._

## Installation & usage

```
$ go install github.com/Antonboom/testifylint@latest
$ testifylint -h
$ testifylint ./...
```

### Fixing

```
$ testifylint --fix ./...
```

Fixing with `golangci-lint` is currently **unavailable** due to
[golangci/golangci-lint#1779](https://github.com/golangci/golangci-lint/issues/1779).

Be aware that there may be unused imports after the fix, run `go fmt`.

## Configuring

### CLI

```bash
# Enable all checkers.
$ testifylint --enable-all ./...

# Enable specific checkers only.
$ testifylint --disable-all --enable=empty,error-is-as ./...

# Disable specific checkers only.
$ testifylint --enable-all --disable=empty,error-is-as ./...

# Checkers configuration.
$ testifylint --bool-compare.ignore-custom-types ./...
$ testifylint --expected-actual.pattern="^wanted$" ./...
$ testifylint --formatter.check-format-string --formatter.require-f-funcs --formatter.require-string-msg ./...
$ testifylint --go-require.ignore-http-handlers ./...
$ testifylint --require-error.fn-pattern="^(Errorf?|NoErrorf?)$" ./...
$ testifylint --suite-extra-assert-call.mode=require ./...
$ testifylint --time-compare.suppress-calls-pattern="UTC|Round" ./...
```

### golangci-lint

https://golangci-lint.run/docs/linters/configuration/#testifylint

## Checkers

- ✅ – yes
- ❌ – no
- 🤏 – partially

| Name                                                | Enabled By Default | Autofix |
|-----------------------------------------------------|--------------------|---------|
| [blank-import](#blank-import)                       | ✅                  | ❌       |
| [bool-compare](#bool-compare)                       | ✅                  | ✅       |
| [compares](#compares)                               | ✅                  | ✅       |
| [contains](#contains)                               | ✅                  | 🤏      |
| [empty](#empty)                                     | ✅                  | ✅       |
| [encoded-compare](#encoded-compare)                 | ✅                  | ✅       |
| [equal-values](#equal-values)                       | ✅                  | ✅       |
| [error-is-as](#error-is-as)                         | ✅                  | 🤏      |
| [error-nil](#error-nil)                             | ✅                  | ✅       |
| [expected-actual](#expected-actual)                 | ✅                  | ✅       |
| [float-compare](#float-compare)                     | ✅                  | ❌       |
| [formatter](#formatter)                             | ✅                  | 🤏      |
| [go-require](#go-require)                           | ✅                  | ❌       |
| [len](#len)                                         | ✅                  | ✅       |
| [mock-expect](#mock-expect)                         | ✅                  | 🤏      |
| [negative-positive](#negative-positive)             | ✅                  | ✅       |
| [nil-compare](#nil-compare)                         | ✅                  | ✅       |
| [regexp](#regexp)                                   | ✅                  | ✅       |
| [require-error](#require-error)                     | ✅                  | ❌       |
| [suite-broken-parallel](#suite-broken-parallel)     | ✅                  | ✅       |
| [suite-dont-use-pkg](#suite-dont-use-pkg)           | ✅                  | ✅       |
| [suite-extra-assert-call](#suite-extra-assert-call) | ✅                  | ✅       |
| [suite-method-signature](#suite-method-signature)   | ✅                  | ❌       |
| [suite-subtest-run](#suite-subtest-run)             | ✅                  | ❌       |
| [suite-thelper](#suite-thelper)                     | ❌                  | ✅       |
| [useless-assert](#useless-assert)                   | ✅                  | ❌       |
| [zero](#zero)                                       | ✅                  | ✅       |

> ⚠️ Also look at open for contribution [checkers](CONTRIBUTING.md#open-for-contribution)

---

### blank-import

```go
❌
import (
    "testing"

    _ "github.com/stretchr/testify"
    _ "github.com/stretchr/testify/assert"
    _ "github.com/stretchr/testify/http"
    _ "github.com/stretchr/testify/mock"
    _ "github.com/stretchr/testify/require"
    _ "github.com/stretchr/testify/suite"
)

✅
import (
    "testing"
)
```

**Autofix**: false. <br>
**Enabled by default**: true. <br>
**Reason**: `testify` doesn't do any `init()` magic, so these imports as `_` do nothing and considered useless.

---

### bool-compare

```go
❌
assert.Equal(t, false, result)
assert.EqualValues(t, false, result)
assert.Exactly(t, false, result)
assert.NotEqual(t, true, result)
assert.NotEqualValues(t, true, result)
assert.False(t, !result)
assert.True(t, result == true)
// And other variations...

✅
assert.True(t, result)
assert.False(t, result)
```

**Autofix**: true. <br>
**Enabled by default**: true. <br>
**Reason**: Code simplification.

Also `bool-compare` supports user defined types like

```go
type Bool bool
```

And fixes assertions via casting variable to builtin `bool`:

```go
var predicate Bool
❌ assert.Equal(t, false, predicate)
✅ assert.False(t, bool(predicate))
```

To turn off this behavior use the `--bool-compare.ignore-custom-types` flag.

---

### compares

```go
❌
assert.True(t, a == b)
assert.True(t, a != b)
assert.True(t, a > b)
assert.True(t, a >= b)
assert.True(t, a < b)
assert.True(t, a <= b)
assert.False(t, a == b)
// And so on...

✅
assert.Equal(t, a, b)
assert.NotEqual(t, a, b)
assert.Greater(t, a, b)
assert.GreaterOrEqual(t, a, b)
assert.Less(t, a, b)
assert.LessOrEqual(t, a, b)
```

**Autofix**: true. <br>
**Enabled by default**: true. <br>
**Reason**: More appropriate `testify` API with clearer failure message.

If `a` and `b` are pointers then `assert.Same`/`NotSame` is required instead,
due to the inappropriate recursive nature of `assert.Equal` (based on
[reflect.DeepEqual](https://pkg.go.dev/reflect#DeepEqual)).

Compares also detects and simplifies equivalent `time.Time` comparisons, like

```go
❌
assert.True(t, t1.After(t2))
assert.Greater(t, t1.Compare(t2), 0)

✅
assert.Greater(t, t1, t2)
```

For `assert.Equal(t, 0, t1.Compare(t2))` case `compares` suggests `WithinDuration` as the assertion with the most readable
message.

---

### contains

```go
❌
assert.True(t, strings.Contains(a, "abc123"))
assert.False(t, !strings.Contains(a, "abc123"))
assert.False(t, strings.Contains(a, "abc123"))
assert.True(t, !strings.Contains(a, "abc123"))
assert.Contains(t, arr, 1, 2)
assert.NotContains(t, arr, 1, 2)

✅
assert.Contains(t, a, "abc123")
assert.NotContains(t, a, "abc123")
assert.Subset(t, arr, 1, 2)
assert.NotSubset(t, arr, 1, 2)
```

**Autofix**: partially. <br>
**Enabled by default**: true. <br>
**Reason**: Protection from bugs, code simplification and more appropriate `testify` API with clearer failure message.

#### An example of bug in Kubernetes tests

<details>

<summary>Click to expand...</summary>

```go
❌
assert.Contains(t, cadvisorInfoToUserDefinedMetrics(&cInfo), 
    statsapi.UserDefinedMetric{
        UserDefinedMetricDescriptor: statsapi.UserDefinedMetricDescriptor{
            Name:  "qos",
            Type:  statsapi.MetricGauge,
            Units: "per second",
        },
        Time:  metav1.NewTime(timestamp2),
        Value: 100,
    },
    statsapi.UserDefinedMetric{
        UserDefinedMetricDescriptor: statsapi.UserDefinedMetricDescriptor{
            Name:  "cpuLoad",
            Type:  statsapi.MetricCumulative,
            Units: "count",
        },
        Time:  metav1.NewTime(timestamp2),
        Value: 2.1,
    })

✅
assert.Subset(t, cadvisorInfoToUserDefinedMetrics(&cInfo), []statsapi.UserDefinedMetric{
    {
        UserDefinedMetricDescriptor: statsapi.UserDefinedMetricDescriptor{
            Name:  "qos",
            Type:  statsapi.MetricGauge,
            Units: "per second",
        },
        Time:  metav1.NewTime(timestamp2),
        Value: 100,
    },
    {
        UserDefinedMetricDescriptor: statsapi.UserDefinedMetricDescriptor{
            Name:  "cpuLoad",
            Type:  statsapi.MetricCumulative,
            Units: "count",
        },
        Time:  metav1.NewTime(timestamp2),
        Value: 2.1,
    }})
```

</details>

#### An example of bug in Prometheus tests

<details>

<summary>Click to expand...</summary>

```go
❌
logLines := getLogLines(t, ql1File)
/*
   {
       "time": "2025-09-14T07:49:51.652292+03:00",
       "params": { "query": "test statement" },
       "error": "failure",
       "spanID": "0000000000000000",
       "foo": "bar"
   }
 */
require.Contains(t, logLines[0], "error",  errors.New("failure"))
require.Contains(t, logLines[0], "params", map[string]interface{}{"query": "test statement"})

✅
logLines := getLogLines(t, ql1File)
require.Contains(t, logLines[0], `"error":"failure"`)
require.Contains(t, logLines[0], `"params":{"query":"test statement"}`)
```

</details>

---

### empty

```go
❌
assert.Len(t, arr, 0)
assert.Zero(t, str)
assert.Zero(t, len(arr))
assert.Equal(t, 0, len(arr))
assert.EqualValues(t, 0, len(arr))
assert.Exactly(t, 0, len(arr))
assert.LessOrEqual(t, len(arr), 0)
assert.GreaterOrEqual(t, 0, len(arr))
assert.Less(t, len(arr), 1)
assert.Greater(t, 1, len(arr))
assert.Equal(t, "", str)
assert.EqualValues(t, "", str)
assert.Exactly(t, "", str)
assert.Equal(t, ``, str)
assert.EqualValues(t, ``, str)
assert.Exactly(t, ``, str)

assert.Positive(t, len(arr))
assert.NotZero(t, str)
assert.NotZero(t, len(arr))
assert.NotEqual(t, 0, len(arr))
assert.NotEqualValues(t, 0, len(arr))
assert.Greater(t, len(arr), 0)
assert.Less(t, 0, len(arr))
assert.NotEqual(t, "", str)
assert.NotEqualValues(t, "", str)
assert.NotEqual(t, ``, str)
assert.NotEqualValues(t, ``, str)

✅
assert.Empty(t, arr)
assert.NotEmpty(t, arr)
```

**Autofix**: true. <br>
**Enabled by default**: true. <br>
**Reason**: More appropriate `testify` API with clearer failure message.

Also `empty` removes extra `len` call in `*Emtpy` assertions:

```go
❌
assert.Empty(t, len(arr))
assert.NotEmpty(t, len(arr))

✅
assert.Empty(t, arr)
assert.NotEmpty(t, arr)
```

P.S. `empty` does not remove the string conversion and keeps as is:

- `string(bytes)` – to keep assert failure message readability;
- `string(str)` – in favor of [unconvert](https://golangci-lint.run/usage/linters/#unconvert).

---

### encoded-compare

```go
❌
assert.Equal(t, `{"foo": "bar"}`, body)
assert.EqualValues(t, `{"foo": "bar"}`, body)
assert.Exactly(t, `{"foo": "bar"}`, body)
assert.Equal(t, expectedJSON, resultJSON)
assert.Equal(t, expBodyConst, w.Body.String())
assert.Equal(t, fmt.Sprintf(`{"value":"%s"}`, hexString), result)
assert.Equal(t, "{}", json.RawMessage(resp))
assert.Equal(t, expJSON, strings.Trim(string(resultJSONBytes), "\n")) // + Replace, ReplaceAll, TrimSpace

assert.Equal(t, expectedYML, conf)

✅
assert.JSONEq(t, `{"foo": "bar"}`, body)
assert.YAMLEq(t, expectedYML, conf)
```

**Autofix**: true. <br>
**Enabled by default**: true. <br>
**Reason**: Protection from bugs and more appropriate `testify` API with clearer failure message.

`encoded-compare` detects valid JSON string constants (usable in `fmt.Sprintf` also) and JSON-style/YAML-style named
variables. If variable is converted to `json.RawMessage`, then it is considered JSON unconditionally.

When fixing, `encoded-compare` removes unnecessary conversions to `[]byte`, `string`, `json.RawMessage` and calls of
`strings.Replace`, `strings.ReplaceAll`, `strings.Trim`, `strings.TrimSpace`, and adds a conversion to `string` when
needed.

---

### equal-values

```go
❌
assert.EqualValues(t, 42, result.IntField)
assert.NotEqualValues(t, 42, result.IntField)
// And other variations with similar types (strings, numerics, structs, etc.)...

✅
assert.Equal(t, 42, result.IntField)
assert.NotEqual(t, 42, result.IntField)
```

**Autofix**: true. <br>
**Enabled by default**: true. <br>
**Reason**: Using more appropriate and more error-proof `testify` API.

In additional:

1. Overflow-underflow issues are [covered by testify itself](https://github.com/stretchr/testify/pull/1531).
2. Nil comparisons are covered by [nil-compare](#nil-compare).

---

### error-is-as

```go
❌
assert.Error(t, err, errSentinel) // Typo, errSentinel hits `msgAndArgs`.
assert.NoError(t, err, errSentinel)
assert.IsType(t, err, errSentinel)
assert.IsType(t, (*http.MaxBytesError)(nil), err)
assert.IsNotType(t, err, errSentinel)
assert.IsNotType(t, store.NotFoundError{}, err)
assert.True(t, errors.Is(err, errSentinel))
assert.False(t, errors.Is(err, errSentinel))
assert.True(t, errors.As(err, &target))
assert.False(t, errors.As(err, &target))

✅
assert.ErrorIs(t, err, errSentinel)
assert.NotErrorIs(t, err, errSentinel)
assert.ErrorAs(t, err, &target)
assert.NotErrorAs(t, err, &target)
```

**Autofix**: partially. <br>
**Enabled by default**: true. <br>
**Reason**: In the first two cases, a common mistake that leads to hiding the incorrect wrapping of sentinel errors.
In the rest cases – more appropriate `testify` API with clearer failure message.

Also `error-is-as` repeats `go vet`'s
[errorsas check](https://cs.opensource.google/go/x/tools/+/master:go/analysis/passes/errorsas/errorsas.go)
logic, but without autofix.

`assert.ErrorAs` cheatsheet:
```go
assert.ErrorAs(t, err, new(*http.MaxBytesError))
assert.ErrorAs(t, err, store.NotFoundError{})

mbErr := new(http.MaxBytesError)
require.ErrorAs(t, err, &mbErr)
assert.Equal(t, 100, mbErr.Limit)

if mbErr := new(http.MaxBytesError); assert.ErrorAs(t, err, &mbErr) {
    assert.Equal(t, 100, mbErr.Limit)
}
```

---

### error-nil

```go
❌
assert.Nil(t, err)
assert.Empty(t, err)
assert.Zero(t, err)
assert.Equal(t, nil, err)
assert.EqualValues(t, nil, err)
assert.Exactly(t, nil, err)
assert.ErrorIs(t, err, nil)
assert.IsType(t, err, nil)

assert.NotNil(t, err)
assert.NotEmpty(t, err)
assert.NotZero(t, err)
assert.NotEqual(t, nil, err)
assert.NotEqualValues(t, nil, err)
assert.NotErrorIs(t, err, nil)
assert.IsNotType(t, err, nil)

✅
assert.NoError(t, err)
assert.Error(t, err)
```

**Autofix**: true. <br>
**Enabled by default**: true. <br>
**Reason**: More appropriate `testify` API with clearer failure message.

---

### expected-actual

```go
❌
assert.Equal(t, result, expected)
assert.Equal(t, result, len(expected))
assert.Equal(t, len(resultFields), len(expectedFields))
assert.EqualExportedValues(t, resultObj, User{Name: "Rob"})
assert.EqualValues(t, result, 42)
assert.Exactly(t, result, int64(42))
assert.JSONEq(t, result, `{"version": 3}`)
assert.InDelta(t, result, 42.42, 1.0)
assert.InDeltaMapValues(t, result, map[string]float64{"score": 0.99}, 1.0)
assert.InDeltaSlice(t, result, []float64{0.98, 0.99}, 1.0)
assert.InEpsilon(t, result, 42.42, 0.0001)
assert.InEpsilonSlice(t, result, []float64{0.9801, 0.9902}, 0.0001)
assert.IsType(t, result, (*User)(nil))
assert.NotEqual(t, result, "expected")
assert.NotEqualValues(t, result, "expected")
assert.NotSame(t, resultPtr, &value)
assert.Same(t, resultPtr, &value)
assert.WithinDuration(t, resultTime, time.Date(2023, 01, 12, 11, 46, 33, 0, nil), time.Second)
assert.YAMLEq(t, result, "version: '3'")

✅
assert.Equal(t, expected, result)
assert.Equal(t, len(expected), result)
assert.Equal(t, len(expectedFields), len(resultFields))
assert.EqualExportedValues(t, User{Name: "Rob"}, resultObj)
assert.EqualValues(t, 42, result)
// And so on...
```

**Autofix**: true. <br>
**Enabled by default**: true. <br>
**Reason**: A common mistake that makes it harder to understand the reason of failed test.

The checker considers the expected value to be a basic literal, constant, or variable whose name matches the pattern
(`--expected-actual.pattern` flag).

It is
planned [to change the order of assertion arguments](https://github.com/stretchr/testify/issues/1089#Argument_order) to
more natural (actual, expected) in `v2` of `testify`.

---

### float-compare

```go
❌
assert.Equal(t, 42.42, result)
assert.EqualValues(t, 42.42, result)
assert.Exactly(t, 42.42, result)
assert.True(t, result == 42.42)
assert.False(t, result != 42.42)

✅
assert.InEpsilon(t, 42.42, result, 0.0001) // Or assert.InDelta
```

**Autofix**: false. <br>
**Enabled by default**: true. <br>
**Reason**: Do not forget about [floating point rounding issues](https://floating-point-gui.de/errors/comparison/).

This checker is similar to the [floatcompare](https://github.com/golangci/golangci-lint/pull/2608) linter.

---

### formatter

```go
❌
assert.ElementsMatch(t, certConfig.Org, csr.Subject.Org, "organizations not equal")
assert.Error(t, err, fmt.Sprintf("Profile %s should not be valid", test.profile))
assert.Errorf(t, err, fmt.Sprintf("test %s", test.testName))
assert.Truef(t, targetTs.Equal(ts), "the timestamp should be as expected (%s) but was %s", targetTs)
// And other go vet's printf checks...

✅
assert.ElementsMatchf(t, certConfig.Org, csr.Subject.Org, "organizations not equal")
assert.Errorf(t, err, "Profile %s should not be valid", test.profile)
assert.Errorf(t, err, "test %s", test.testName)
assert.Truef(t, targetTs.Equal(ts), "the timestamp should be as expected (%s) but was %s", targetTs, ts)
```

**Autofix**: partially. <br>
**Enabled by default**: true. <br>
**Reason**: Code simplification, protection from bugs, following the "Go Way".

The `formatter` checker has the following features:

#### 1)

Detecting unnecessary `fmt.Sprintf` in assertions. Somewhat reminiscent of the
[staticcheck's S1028 check](https://staticcheck.dev/docs/checks/#S1028).

#### 2)

Validating consistency of assertions format strings and corresponding arguments, using a patched fork of `go vet`'s
[printf](https://cs.opensource.google/go/x/tools/+/master:go/analysis/passes/printf/printf.go) analyzer. To
disable this feature, use `--formatter.check-format-string=false` flag.

#### 3)

Requirement of the f-assertions (e.g. assert.Equal**f**) if format string is used. Disabled by default, use
`--formatter.require-f-funcs` flag to enable. <br>

Frequently Asked Questions