https://github.com/blampe/shard

A quick hack to shard large Go test suites.
https://github.com/blampe/shard

cicd go

Last synced: 1 day ago
JSON representation

A quick hack to shard large Go test suites.

• Host: GitHub
• URL: https://github.com/blampe/shard
• Owner: blampe
• Created: 2024-08-14T22:47:28.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-11-16T00:15:23.000Z (about 2 months ago)
• Last Synced: 2024-11-16T00:20:20.838Z (about 2 months ago)
• Topics: cicd, go
• Language: Go
• Homepage:
• Size: 6.84 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Shard 🔪

A quick hack to shard large Go test suites.

Example usage with GitHub Actions:

```

jobs:

  tests:

    runs-on: ubuntu-latest

    strategy:

      matrix:

        shard: [0, 1, 2, 3]

  steps:

    - name: Shard tests

      run: go run github.com/blampe/shard --total ${{ strategy.job-total }} --index ${{ strategy.job-index }} > tests

    - name: Run tests

      run: go test $(cat tests)

```

## Why?

Strategies like `go test ./... -list .` compile your code in order to discover test cases.

This can be especially slow in CI environments depending on the state of your build cache.

We can discover test cases significantly faster by essentially `grep`-ing for patterns like `Test..`, `Fuzz...`, etc.

## Caveats

* A test can potentially be executed more than once if another package shares a test with the same name.

  Renaming your tests to be globally unique is currently the best workaround if you want to guarantee a single execution per test function.

  You can discover tests with name collisions by running `shard --total 1 --index 0`.

* Benchmarks aren't currently collected so running with `-bench` will not have any effect.