Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Antonboom/testifylint
The Golang linter that checks usage of github.com/stretchr/testify.
https://github.com/Antonboom/testifylint
go golang linter static-analysis testify
Last synced: about 1 month ago
JSON representation
The Golang linter that checks usage of github.com/stretchr/testify.
- Host: GitHub
- URL: https://github.com/Antonboom/testifylint
- Owner: Antonboom
- License: mit
- Created: 2022-06-10T06:41:27.000Z (almost 2 years ago)
- Default Branch: master
- Last Pushed: 2024-04-20T07:14:54.000Z (about 2 months ago)
- Last Synced: 2024-04-21T07:26:49.210Z (about 1 month ago)
- Topics: go, golang, linter, static-analysis, testify
- Language: Go
- Homepage: https://github.com/stretchr/testify
- Size: 6.23 MB
- Stars: 64
- Watchers: 3
- Forks: 6
- Open Issues: 17
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Lists
- awesome-go - testifylint
- awesome-go-cn - testifylint
- awesome-go-stars - testifylint
- awesome-go - testifylint
- awesome-go-with-stars - testifylint
- awesome-go - testifylint
- awesome-go - testifylint
README
# testifylint
[](https://github.com/Antonboom/testifylint/actions/workflows/ci.yml)
[](https://goreportcard.com/report/github.com/Antonboom/testifylint?dummy=unused)
[](https://coveralls.io/github/Antonboom/testifylint?branch=master&dummy=unused)
[](LICENSE)
[](https://github.com/Antonboom/testifylint/blob/master/CONTRIBUTING.md#Open-for-contribution)
Checks usage of [github.com/stretchr/testify](https://github.com/stretchr/testify).
## Table of Contents
* [Problem statement](#problem-statement)
* [Installation & usage](#installation--usage)
* [Configuring](#configuring)
* [Checkers](#checkers)
* [Chain of warnings](#chain-of-warnings)
* [testify v2](#testify-v2)
## Problem statement
Tests are also program code and the requirements for them should not differ much from the requirements
for the code under tests 🙂
We should try to maintain the consistency of tests, increase their readability, reduce the chance of bugs
and speed up the search for a problem.
[testify](https://github.com/stretchr/testify) is the most popular Golang testing framework* in recent years.
But it has a terrible ambiguous API in places, and **the purpose of this linter is to protect you from annoying mistakes**.
Most checkers are stylistic, but checkers like [error-is-as](#error-is-as), [require-error](#require-error),
[expected-actual](#expected-actual), [float-compare](#float-compare) are really helpful.
_* JetBrains "The State of Go Ecosystem" reports [2021](https://www.jetbrains.com/lp/devecosystem-2021/go/#Go_which-testing-frameworks-do-you-use-regularly-if-any)
and [2022](https://www.jetbrains.com/lp/devecosystem-2022/go/#which-testing-frameworks-do-you-use-regularly-if-any-)._
## Installation & usage
```
$ go install github.com/Antonboom/testifylint@latest
$ testifylint -h
$ testifylint ./...
```
### Fixing
```
$ testifylint --fix ./...
```
Fixing with `golangci-lint` is currently **unavailable** due to
[golangci/golangci-lint#1779](https://github.com/golangci/golangci-lint/issues/1779).
## Configuring
### CLI
```bash
# Enable all checkers.
$ testifylint --enable-all ./...
# Enable specific checkers only.
$ testifylint --disable-all --enable=empty,error-is-as ./...
# Disable specific checkers only.
$ testifylint --enable-all --disable=empty,error-is-as ./...
# Checkers configuration.
$ testifylint --bool-compare.ignore-custom-types ./...
$ testifylint --expected-actual.pattern=^wanted$ ./...
$ testifylint --require-error.fn-pattern="^(Errorf?|NoErrorf?)$" ./...
$ testifylint --suite-extra-assert-call.mode=require ./...
```
### golangci-lint
https://golangci-lint.run/usage/linters/#testifylint
## Checkers
| Name | Enabled By Default | Autofix |
|-----------------------------------------------------|--------------------|---------|
| [blank-import](#blank-import) | ✅ | ❌ |
| [bool-compare](#bool-compare) | ✅ | ✅ |
| [compares](#compares) | ✅ | ✅ |
| [empty](#empty) | ✅ | ✅ |
| [error-is-as](#error-is-as) | ✅ | ✅ |
| [error-nil](#error-nil) | ✅ | ✅ |
| [error-nil](#error-nil) | ✅ | ✅ |
| [expected-actual](#expected-actual) | ✅ | ✅ |
| [float-compare](#float-compare) | ✅ | ❌ |
| [go-require](#go-require) | ✅ | ❌ |
| [len](#len) | ✅ | ✅ |
| [negative-positive](#negative-positive) | ✅ | ✅ |
| [nil-compare](#nil-compare) | ✅ | ✅ |
| [require-error](#require-error) | ✅ | ❌ |
| [suite-dont-use-pkg](#suite-dont-use-pkg) | ✅ | ✅ |
| [suite-extra-assert-call](#suite-extra-assert-call) | ✅ | ✅ |
| [suite-thelper](#suite-thelper) | ❌ | ✅ |
| [useless-assert](#useless-assert) | ✅ | ❌ |
> ⚠️ Also look at open for contribution [checkers](CONTRIBUTING.md#open-for-contribution)
---
### blank-import
```go
❌
import (
"testing"
_ "github.com/stretchr/testify"
_ "github.com/stretchr/testify/assert"
_ "github.com/stretchr/testify/http"
_ "github.com/stretchr/testify/mock"
_ "github.com/stretchr/testify/require"
_ "github.com/stretchr/testify/suite"
)
✅
import (
"testing"
)
```
**Autofix**: false.
**Enabled by default**: true.
**Reason**: `testify` doesn't do any `init()` magic, so these imports as `_` do nothing and considered useless.
---
### bool-compare
```go
❌ assert.Equal(t, false, result)
assert.EqualValues(t, false, result)
assert.Exactly(t, false, result)
assert.NotEqual(t, true, result)
assert.NotEqualValues(t, true, result)
assert.False(t, !result)
assert.True(t, result == true)
// And other variations...
✅ assert.True(t, result)
assert.False(t, result)
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: Code simplification.
Also `bool-compare` supports user defined types like
```go
type Bool bool
```
And fixes assertions via casting variable to builtin `bool`:
```go
var predicate Bool
❌ assert.Equal(t, false, predicate)
✅ assert.False(t, bool(predicate))
```
To turn off this behavior use the `--bool-compare.ignore-custom-types` flag.
---
### compares
```go
❌ assert.True(t, a == b)
assert.True(t, a != b)
assert.True(t, a > b)
assert.True(t, a >= b)
assert.True(t, a < b)
assert.True(t, a <= b)
assert.False(t, a == b)
// And so on...
✅ assert.Equal(t, a, b)
assert.NotEqual(t, a, b)
assert.Greater(t, a, b)
assert.GreaterOrEqual(t, a, b)
assert.Less(t, a, b)
assert.LessOrEqual(t, a, b)
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: More appropriate `testify` API with clearer failure message.
---
### empty
```go
❌ assert.Len(t, arr, 0)
assert.Equal(t, 0, len(arr))
assert.EqualValues(t, 0, len(arr))
assert.Exactly(t, 0, len(arr))
assert.LessOrEqual(t, len(arr), 0)
assert.GreaterOrEqual(t, 0, len(arr))
assert.Less(t, len(arr), 0)
assert.Greater(t, 0, len(arr))
assert.Less(t, len(arr), 1)
assert.Greater(t, 1, len(arr))
assert.NotEqual(t, 0, len(arr))
assert.NotEqualValues(t, 0, len(arr))
assert.Less(t, 0, len(arr))
assert.Greater(t, len(arr), 0)
✅ assert.Empty(t, arr)
assert.NotEmpty(t, err)
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: More appropriate `testify` API with clearer failure message.
---
### error-is-as
```go
❌ assert.Error(t, err, errSentinel) // Typo, errSentinel hits `msgAndArgs`.
assert.NoError(t, err, errSentinel)
assert.True(t, errors.Is(err, errSentinel))
assert.False(t, errors.Is(err, errSentinel))
assert.True(t, errors.As(err, &target))
✅ assert.ErrorIs(t, err, errSentinel)
assert.NotErrorIs(t, err, errSentinel)
assert.ErrorAs(t, err, &target)
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: In the first two cases, a common mistake that leads to hiding the incorrect wrapping of sentinel errors.
In the rest cases – more appropriate `testify` API with clearer failure message.
Also `error-is-as` repeats `go vet`'s [errorsas check](https://cs.opensource.google/go/x/tools/+/master:go/analysis/passes/errorsas/errorsas.go)
logic, but without autofix.
---
### error-nil
```go
❌ assert.Nil(t, err)
assert.NotNil(t, err)
assert.Equal(t, nil, err)
assert.EqualValues(t, nil, err)
assert.Exactly(t, nil, err)
assert.ErrorIs(t, err, nil)
assert.NotEqual(t, nil, err)
assert.NotEqualValues(t, nil, err)
assert.NotErrorIs(t, err, nil)
✅ assert.NoError(t, err)
assert.Error(t, err)
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: More appropriate `testify` API with clearer failure message.
---
### expected-actual
```go
❌ assert.Equal(t, result, expected)
assert.EqualExportedValues(t, resultObj, User{Name: "Rob"})
assert.EqualValues(t, result, 42)
assert.Exactly(t, result, int64(42))
assert.JSONEq(t, result, `{"version": 3}`)
assert.InDelta(t, result, 42.42, 1.0)
assert.InDeltaMapValues(t, result, map[string]float64{"score": 0.99}, 1.0)
assert.InDeltaSlice(t, result, []float64{0.98, 0.99}, 1.0)
assert.InEpsilon(t, result, 42.42, 0.0001)
assert.InEpsilonSlice(t, result, []float64{0.9801, 0.9902}, 0.0001)
assert.IsType(t, result, (*User)(nil))
assert.NotEqual(t, result, "expected")
assert.NotEqualValues(t, result, "expected")
assert.NotSame(t, resultPtr, &value)
assert.Same(t, resultPtr, &value)
assert.WithinDuration(t, resultTime, time.Date(2023, 01, 12, 11, 46, 33, 0, nil), time.Second)
assert.YAMLEq(t, result, "version: '3'")
✅ assert.Equal(t, expected, result)
assert.EqualExportedValues(t, User{Name: "Rob"}, resultObj)
assert.EqualValues(t, 42, result)
// And so on...
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: A common mistake that makes it harder to understand the reason of failed test.
The checker considers the expected value to be a basic literal, constant, or variable whose name matches the pattern
(`--expected-actual.pattern` flag).
It is planned [to change the order of assertion arguments](https://github.com/stretchr/testify/issues/1089#Argument_order) to more natural
(actual, expected) in `v2` of `testify`.
---
### float-compare
```go
❌ assert.Equal(t, 42.42, result)
assert.EqualValues(t, 42.42, result)
assert.Exactly(t, 42.42, result)
assert.True(t, result == 42.42)
assert.False(t, result != 42.42)
✅ assert.InEpsilon(t, 42.42, result, 0.0001) // Or assert.InDelta
```
**Autofix**: false.
**Enabled by default**: true.
**Reason**: Do not forget about [floating point rounding issues](https://floating-point-gui.de/errors/comparison/).
This checker is similar to the [floatcompare](https://github.com/golangci/golangci-lint/pull/2608) linter.
---
### go-require
```go
go func() {
conn, err = lis.Accept()
require.NoError(t, err) ❌
if assert.Error(err) {
assert.FailNow(t, msg) ❌
}
}()
```
**Autofix**: false.
**Enabled by default**: true.
**Reason**: Incorrect use of functions.
This checker is a radically improved analogue of `go vet`'s
[testinggoroutine](https://cs.opensource.google/go/x/tools/+/master:go/analysis/passes/testinggoroutine/doc.go) check.
The point of the check is that, according to the [documentation](https://pkg.go.dev/testing#T),
functions leading to `t.FailNow` (essentially to `runtime.GoExit`) must only be used in the goroutine that runs the test.
Otherwise, they will not work as declared, namely, finish the test function.
You can disable the `go-require` checker and continue to use `require` as the current goroutine finisher, but this could lead
1. to possible resource leaks in tests;
2. to increasing of confusion, because functions will be not used as intended.
Typically, any assertions inside goroutines are a marker of poor test architecture.
Try to execute them in the main goroutine and distribute the data necessary for this into it
([example](https://github.com/ipfs/kubo/issues/2043#issuecomment-164136026)).
Also a bad solution would be to simply replace all `require` in goroutines with `assert`
(like [here](https://github.com/gravitational/teleport/pull/22567/files#diff-9f5fd20913c5fe80c85263153fa9a0b28dbd1407e53da4ab5d09e13d2774c5dbR7377))
– this will only mask the problem.
The checker is enabled by default, because `testinggoroutine` is enabled by default in `go vet`.
P.S. Related `testify`'s [thread](https://github.com/stretchr/testify/issues/772).
---
### len
```go
❌ assert.Equal(t, 3, len(arr))
assert.EqualValues(t, 3, len(arr))
assert.Exactly(t, 3, len(arr))
assert.True(t, len(arr) == 3)
✅ assert.Len(t, arr, 3)
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: More appropriate `testify` API with clearer failure message.
---
### negative-positive
```go
❌ assert.Less(t, a, 0)
assert.Greater(t, 0, a)
assert.True(t, a < 0)
assert.True(t, 0 > a)
assert.False(t, a >= 0)
assert.False(t, 0 <= a)
assert.Greater(t, a, 0)
assert.Less(t, 0, a)
assert.True(t, a > 0)
assert.True(t, 0 < a)
assert.False(t, a <= 0)
assert.False(t, 0 >= a)
✅ assert.Negative(t, a)
assert.Positive(t, a)
```
**Autofix**: true.
**Enabled by default**: true
**Reason**: More appropriate `testify` API with clearer failure message.
---
### nil-compare
```go
❌ assert.Equal(t, nil, value)
assert.EqualValues(t, nil, value)
assert.Exactly(t, nil, value)
assert.NotEqual(t, nil, value)
assert.NotEqualValues(t, nil, value)
✅ assert.Nil(t, value)
assert.NotNil(t, value)
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: Protection from bugs and more appropriate `testify` API with clearer failure message.
Using untyped `nil` in the functions above along with a non-interface type does not make sense:
```go
assert.Equal(t, nil, eventsChan) // Always fail.
assert.NotEqual(t, nil, eventsChan) // Always pass.
```
The right way:
```go
assert.Equal(t, (chan Event)(nil), eventsChan)
assert.NotEqual(t, (chan Event)(nil), eventsChan)
```
But in the case of `Equal`, `NotEqual` and `Exactly` it still doesn't work for the function type.
The best option here is to just use `Nil` / `NotNil` (see [details](https://github.com/stretchr/testify/issues/1524)).
---
### require-error
```go
❌ assert.Error(t, err) // s.Error(err), s.Assert().Error(err)
assert.ErrorIs(t, err, io.EOF)
assert.ErrorAs(t, err, &target)
assert.EqualError(t, err, "end of file")
assert.ErrorContains(t, err, "end of file")
assert.NoError(t, err)
assert.NotErrorIs(t, err, io.EOF)
✅ require.Error(t, err) // s.Require().Error(err), s.Require().Error(err)
require.ErrorIs(t, err, io.EOF)
require.ErrorAs(t, err, &target)
// And so on...
```
**Autofix**: false.
**Enabled by default**: true.
**Reason**: Such "ignoring" of errors leads to further panics, making the test harder to debug.
[testify/require](https://pkg.go.dev/github.com/stretchr/testify@master/require#hdr-Assertions) allows
to stop test execution when a test fails.
By default `require-error` only checks the `*Error*` assertions, presented above.
You can set `--require-error.fn-pattern` flag to limit the checking to certain calls (but still from the list above).
For example, `--require-error.fn-pattern="^(Errorf?|NoErrorf?)$"` will only check `Error`, `Errorf`, `NoError`, and `NoErrorf`.
Also, to minimize the number of false positives, `require-error` ignores:
- assertion in the `if` condition;
- the entire `if-else[-if]` block, if there is an assertion in any `if` condition;
- the last assertion in the block, if there are no methods/functions calls after it;
- assertions in an explicit goroutine;
- assertions in an explicit testing cleanup function or suite teardown methods;
- sequence of `NoError` assertions.
---
### suite-dont-use-pkg
```go
import "github.com/stretchr/testify/assert"
func (s *MySuite) TestSomething() {
❌ assert.Equal(s.T(), 42, value)
✅ s.Equal(42, value)
}
```
**Autofix**: true.
**Enabled by default**: true.
**Reason**: More simple and uniform code.
---
### suite-extra-assert-call
By default, the checker wants you to remove unnecessary `Assert()` calls:
```go
func (s *MySuite) TestSomething() {
❌ s.Assert().Equal(42, value)
✅ s.Equal(42, value)
}
```
But sometimes, on the contrary, people want consistency with `s.Assert()` and `s.Require()`:
```go
func (s *MySuite) TestSomething() {
// ...
❌
s.Require().NoError(err)
s.Equal(42, value)
✅
s.Require().NoError(err)
s.Assert().Equal(42, value)
}
```
You can enable such behavior through `--suite-extra-assert-call.mode=require`.
**Autofix**: true.
**Enabled by default**: true, in the `remove` mode.
**Reason**: More simple or uniform code.
---
### suite-thelper
```go
❌
func (s *RoomSuite) assertRoomRound(roundID RoundID) {
s.Equal(roundID, s.getRoom().CurrentRound.ID)
}
✅
func (s *RoomSuite) assertRoomRound(roundID RoundID) {
s.T().Helper()
s.Equal(roundID, s.getRoom().CurrentRound.ID)
}
```
**Autofix**: true.
**Enabled by default**: false.
**Reason**: Consistency with non-suite test helpers. Explicit markup of helper methods.
`s.T().Helper()` call is not important actually because `testify` prints full `Error Trace`
[anyway](https://github.com/stretchr/testify/blob/882382d845cd9780bd93c1acc8e1fa2ffe266ca1/assert/assertions.go#L317).
The checker rather acts as an example of a [checkers.AdvancedChecker](https://github.com/Antonboom/testifylint/blob/676324836555445fded4e9afc004101ec6f597fe/internal/checkers/checker.go#L56).
---
### useless-assert
Currently the checker guards against assertion of the same variable:
```go
❌ assert.Equal(t, tt.value, tt.value)
assert.ElementsMatch(t, users, users)
// And so on...
```
More complex cases are [open for contribution](CONTRIBUTING.md#useless-assert).
**Autofix**: false.
**Enabled by default**: true.
**Reason**: Protection from bugs and possible dead code.
---
## Chain of warnings
Linter does not automatically handle the "evolution" of changes.
And in some cases may be dissatisfied with your code several times, for example:
```go
assert.True(err == nil) // compares: use assert.Equal
assert.Equal(t, err, nil) // error-nil: use assert.NoError
assert.NoError(t, err) // require-error: for error assertions use require
require.NoError(t, err)
```
Please [contribute](./CONTRIBUTING.md) if you have ideas on how to make this better.
## testify v2
The second version of `testify` [promises](https://github.com/stretchr/testify/issues/1089) more "pleasant" API and
makes some above checkers irrelevant.
In this case, the possibility of supporting `v2` in the linter is not excluded.
But at the moment it looks like we are [extremely far](https://github.com/stretchr/testify/issues/1089#issuecomment-1812734472)
from `v2`. Related milestone [here](https://github.com/stretchr/testify/milestone/4).