{"id":30240411,"url":"https://github.com/hypothesis/testpilot","last_synced_at":"2026-02-25T17:04:24.180Z","repository":{"id":38012211,"uuid":"494060051","full_name":"hypothesis/testpilot","owner":"hypothesis","description":"Format, lint and test only the files that have changed on the current branch.","archived":false,"fork":false,"pushed_at":"2025-06-13T12:44:48.000Z","size":162,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-06-13T13:29:49.465Z","etag":null,"topics":["python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hypothesis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-05-19T12:22:18.000Z","updated_at":"2025-06-13T12:19:20.000Z","dependencies_parsed_at":"2023-02-13T21:40:17.342Z","dependency_job_id":"22887108-2bd7-4756-bac7-1ef15993b2ea","html_url":"https://github.com/hypothesis/testpilot","commit_stats":null,"previous_names":[],"tags_count":56,"template":false,"template_full_name":null,"purl":"pkg:github/hypothesis/testpilot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hypothesis%2Ftestpilot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hypothesis%2Ftestpilot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hypothesis%2Ftestpilot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hypothesis%2Ftestpilot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hypothesis","download_url":"https://codeload.github.com/hypothesis/testpilot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hypothesis%2Ftestpilot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270524432,"owners_count":24600195,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python"],"created_at":"2025-08-15T04:38:26.557Z","updated_at":"2026-02-25T17:04:19.140Z","avatar_url":"https://github.com/hypothesis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://github.com/hypothesis/testpilot/actions/workflows/ci.yml?query=branch%3Amain\"\u003e\u003cimg src=\"https://img.shields.io/github/actions/workflow/status/hypothesis/testpilot/ci.yml?branch=main\"\u003e\u003c/a\u003e\n\u003ca href=\"https://pypi.org/project/testpilot/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/testpilot\"\u003e\u003c/a\u003e\n\u003ca\u003e\u003cimg src=\"https://img.shields.io/badge/python-3.12 | 3.11 | 3.10 | 3.9-success\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/hypothesis/testpilot/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-BSD--2--Clause-success\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/hypothesis/cookiecutters/tree/main/pypackage\"\u003e\u003cimg src=\"https://img.shields.io/badge/cookiecutter-pypackage-success\"\u003e\u003c/a\u003e\n\u003ca href=\"https://black.readthedocs.io/en/stable/\"\u003e\u003cimg src=\"https://img.shields.io/badge/code%20style-black-000000\"\u003e\u003c/a\u003e\n\n# Test Pilot\n\nFormat, lint and test only the files that have changed on the current branch.\n\nSee also: [Test Pilot presentation slides](https://docs.google.com/presentation/d/1zyO8ebsDnz_2j98kBq3JV4oIarhJefCCt8N8qLvuoxw/)\n\nTest Pilot formats, lints and tests the files that have changed on your current\nbranch compared to main:\n\n```terminal\n$ testpilot\ntestpilot=\u003e Formatting\ntestpilot=\u003e Linting\ntestpilot=\u003e Running unit tests\ntestpilot=\u003e Printing coverage report\n...\nName                                       Stmts   Miss Branch BrPart     Cover   Missing\n-----------------------------------------------------------------------------------------\nlms/app.py                                    44      0      0      0   100.00%\nlms/assets.py                                  8      0      0      0   100.00%\nlms/new_file_committed_on_this_branch.py       0      0      0      0   100.00%\nlms/new_untracked_file.py                      0      0      0      0   100.00%\ntests/unit/lms/app_test.py                    15      0      4      0   100.00%\ntests/unit/lms/assets_test.py                  5      0      0      0   100.00%\n-----------------------------------------------------------------------------------------\nTOTAL                                         72      0      4      0   100.00%\ntestpilot=\u003e Running functional tests\n.\n```\n\nIt compares your branch to main and finds:\n\n* New, modified and deleted files\n* Committed, staged and untracked changes\n* Source, test and functional test files\n* If you've modified a source file (e.g. `src/foo/bar.py`) Test Pilot will find\n  its corresponding unit test file (e.g. `tests/unit/foo/bar_test.py`) and will\n  run those tests (as well as formatting and linting the test file).\n  This also works the other way round: if you modify a unit test file then its\n  corresponding source file will be formatted and linted.\n\nThis is **much** faster than running `make sure` and produces much less output,\nbut it'll still catch 99% of problems on your branch. Just run the full `make\nsure` once before sending your pull request.\n\n## Only for Hypothesis projects\n\nFor now, **Test Pilot only works with Hypothesis projects.**\nIt uses all sorts of knowledge and assumptions about how the Hypothesis\ndevelopment environment works, what formatting, testing and linting tools we use,\nhow we organize our test files, etc.\nTest Pilot won't work at all with non-Hypothesis projects unless they happen to\ndo everything _exactly_ as our projects do.\n\n## Test Pilot bypasses tox\n\nBy default Test Pilot bypasses tox and runs commands directly.\nFor example it runs `.tox/tests/bin/pytest` directly instead of `tox -e tests`.\nThis is faster but it means tox won't install or update the dependencies\nin the virtualenv for you if they aren't already installed and up to date.\nOther tox things like setting environment variables etc won't happen either.\nIf it doesn't seem to be working try running `testpilot` with\n`-t` / `--tox` (alias: `-s` / `--slower`) and it'll run all the commands\nthrough tox instead:\n\n```terminal\n$ testpilot --tox\n```\n\nAfter doing this once you can usually go back to running `testpilot` without `-t`.\n\nAlthough `testpilot` without `-t` won't keep the dependencies in your virtualenv\nup to date it *will* detect if your virtualenv doesn't exist at all and call tox\nto create it.\nIt'll also detect if the project's version of Python isn't installed and call\npyenv to install it:\n\n```terminal\n$ testpilot\ntestpilot=\u003e Installing Python\nDownloading Python-3.8.12.tar.xz...\n-\u003e https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tar.xz\nInstalling Python-3.8.12...\n...\nInstalled Python-3.8.12 to /home/seanh/.pyenv/versions/3.8.12\n\ntestpilot=\u003e Formatting\ntestpilot=\u003e It looks like .tox/format/bin/black doesn't exist, running tox to install it\n.tox recreate: /home/seanh/Projects/lms/.tox/.tox\n...\n```\n\n## Debugging\n\nYou can use `-d` / `--debug` to get Test Pilot to print out exactly what\ncommands it's running and a few other details:\n\n```terminal\n$ testpilot --debug\ntestpilot=\u003e Running git rev-parse --is-inside-work-tree\ntestpilot=\u003e Project class: \u003cclass 'testpilot.app.StandardProject'\u003e\ntestpilot=\u003e Installing Python\ntestpilot=\u003e Running bin/install-python\ntestpilot=\u003e Running git branch --format %(refname:lstrip=-1) -l main master\ntestpilot=\u003e Project's main branch: 'master'\ntestpilot=\u003e Running git diff origin/master --name-only\ntestpilot=\u003e Running git ls-files --others --exclude-standard\ntestpilot=\u003e Formatting\ntestpilot=\u003e Running pyenv exec .tox/format/bin/black --quiet lms/app.py lms/assets.py lms/new_file_committed_on_this_branch.py lms/new_untracked_file.py tests/functional/lti_certification/v13/grading/test_assignment_and_grading.py tests/unit/lms/app_test.py tests/unit/lms/assets_test.py\n...\n```\n\n## Help\n\nSee `testpilot --help` for the rest of the command line options:\n\n```terminal\n$ testpilot --help\n```\n\n## Installing\n\nWe recommend using [pipx](https://pypa.github.io/pipx/) to install\nTest Pilot.\nFirst [install pipx](https://pypa.github.io/pipx/#install-pipx) then run:\n\n```terminal\npipx install testpilot\n```\n\nYou now have Test Pilot installed! For some help run:\n\n```\ntestpilot --help\n```\n\n## Upgrading\n\nTo upgrade to the latest version run:\n\n```terminal\npipx upgrade testpilot\n```\n\nTo see what version you have run:\n\n```terminal\ntestpilot --version\n```\n\n## Uninstalling\n\nTo uninstall run:\n\n```\npipx uninstall testpilot\n```\n\n## Setting up Your Test Pilot Development Environment\n\nFirst you'll need to install:\n\n* [Git](https://git-scm.com/).\n  On Ubuntu: `sudo apt install git`, on macOS: `brew install git`.\n* [GNU Make](https://www.gnu.org/software/make/).\n  This is probably already installed, run `make --version` to check.\n* [pyenv](https://github.com/pyenv/pyenv).\n  Follow the instructions in pyenv's README to install it.\n  The **Homebrew** method works best on macOS.\n  The **Basic GitHub Checkout** method works best on Ubuntu.\n  You _don't_ need to set up pyenv's shell integration (\"shims\"), you can\n  [use pyenv without shims](https://github.com/pyenv/pyenv#using-pyenv-without-shims).\n\nThen to set up your development environment:\n\n```terminal\ngit clone https://github.com/hypothesis/testpilot.git\ncd testpilot\nmake help\n```\n\n## Releasing a New Version of the Project\n\n1. First, to get PyPI publishing working you need to go to:\n   \u003chttps://github.com/organizations/hypothesis/settings/secrets/actions/PYPI_TOKEN\u003e\n   and add testpilot to the `PYPI_TOKEN` secret's selected\n   repositories.\n\n2. Now that the testpilot project has access to the `PYPI_TOKEN` secret\n   you can release a new version by just [creating a new GitHub release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository).\n   Publishing a new GitHub release will automatically trigger\n   [a GitHub Actions workflow](.github/workflows/pypi.yml)\n   that will build the new version of your Python package and upload it to\n   \u003chttps://pypi.org/project/testpilot/\u003e.\n\n## Changing the Project's Python Versions\n\nTo change what versions of Python the project uses:\n\n1. Change the Python versions in the\n   [cookiecutter.json](.cookiecutter/cookiecutter.json) file. For example:\n\n   ```json\n   \"python_versions\": \"3.10.4, 3.9.12\",\n   ```\n\n2. Re-run the cookiecutter template:\n\n   ```terminal\n   make template\n   ```\n\n3. Commit everything to git and send a pull request\n\n## Changing the Project's Python Dependencies\n\nTo change the production dependencies in the `setup.cfg` file:\n\n1. Change the dependencies in the [`.cookiecutter/includes/setuptools/install_requires`](.cookiecutter/includes/setuptools/install_requires) file.\n   If this file doesn't exist yet create it and add some dependencies to it.\n   For example:\n\n   ```\n   pyramid\n   sqlalchemy\n   celery\n   ```\n\n2. Re-run the cookiecutter template:\n\n   ```terminal\n   make template\n   ```\n\n3. Commit everything to git and send a pull request\n\nTo change the project's formatting, linting and test dependencies:\n\n1. Change the dependencies in the [`.cookiecutter/includes/tox/deps`](.cookiecutter/includes/tox/deps) file.\n   If this file doesn't exist yet create it and add some dependencies to it.\n   Use tox's [factor-conditional settings](https://tox.wiki/en/latest/config.html#factors-and-factor-conditional-settings)\n   to limit which environment(s) each dependency is used in.\n   For example:\n\n   ```\n   lint: flake8,\n   format: autopep8,\n   lint,tests: pytest-faker,\n   ```\n\n2. Re-run the cookiecutter template:\n\n   ```terminal\n   make template\n   ```\n\n3. Commit everything to git and send a pull request\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhypothesis%2Ftestpilot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhypothesis%2Ftestpilot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhypothesis%2Ftestpilot/lists"}