{"id":51625816,"url":"https://github.com/nao1215/atago","last_synced_at":"2026-07-13T01:01:54.934Z","repository":{"id":368957513,"uuid":"1286019322","full_name":"nao1215/atago","owner":"nao1215","description":"Tests CLI behavior from plain YAML: exit codes, output, files, snapshots, and interactive terminals (PTY/TUI)","archived":false,"fork":false,"pushed_at":"2026-07-11T04:01:59.000Z","size":5284,"stargazers_count":8,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-11T06:03:52.928Z","etag":null,"topics":["ci","cli","command-line","cross-platform","e2e-testing","go","golang","integration-testing","pty","snapshot-testing","terminal","test-automation","test-runner","testing","testing-tools","tui","yaml"],"latest_commit_sha":null,"homepage":"https://nao1215.github.io/atago/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nao1215.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":".github/SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"nao1215"}},"created_at":"2026-07-01T11:14:00.000Z","updated_at":"2026-07-11T04:02:03.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/nao1215/atago","commit_stats":null,"previous_names":["nao1215/atago"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/nao1215/atago","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nao1215%2Fatago","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nao1215%2Fatago/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nao1215%2Fatago/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nao1215%2Fatago/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nao1215","download_url":"https://codeload.github.com/nao1215/atago/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nao1215%2Fatago/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35406584,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-12T02:00:06.386Z","response_time":87,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci","cli","command-line","cross-platform","e2e-testing","go","golang","integration-testing","pty","snapshot-testing","terminal","test-automation","test-runner","testing","testing-tools","tui","yaml"],"created_at":"2026-07-13T01:01:54.453Z","updated_at":"2026-07-13T01:01:54.924Z","avatar_url":"https://github.com/nao1215.png","language":"Go","funding_links":["https://github.com/sponsors/nao1215"],"categories":[],"sub_categories":[],"readme":"\u003c!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --\u003e\n[![All Contributors](https://img.shields.io/badge/all_contributors-1-orange.svg?style=flat-square)](#contributors-)\n\u003c!-- ALL-CONTRIBUTORS-BADGE:END --\u003e\n\n![Coverage](https://raw.githubusercontent.com/nao1215/octocovs-central-repo/main/badges/nao1215/atago/coverage.svg)\n[![Build](https://github.com/nao1215/atago/actions/workflows/build.yml/badge.svg)](https://github.com/nao1215/atago/actions/workflows/build.yml)\n[![UnitTest](https://github.com/nao1215/atago/actions/workflows/unit_test.yml/badge.svg)](https://github.com/nao1215/atago/actions/workflows/unit_test.yml)\n[![reviewdog](https://github.com/nao1215/atago/actions/workflows/reviewdog.yml/badge.svg)](https://github.com/nao1215/atago/actions/workflows/reviewdog.yml)\n[![Go Reference](https://pkg.go.dev/badge/github.com/nao1215/atago.svg)](https://pkg.go.dev/github.com/nao1215/atago)\n![GitHub](https://img.shields.io/github/license/nao1215/atago)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./doc/img/atago-logo.jpg\" alt=\"atago logo\" width=\"400\" /\u003e\n\u003c/p\u003e\n\natago tests real CLI behavior from plain YAML: commands, files, snapshots, services, and interactive terminals. It runs your actual binary — in any language — and asserts what a user observes. No test code, no shell DSL.\n\nDocumentation: **https://nao1215.github.io/atago/**\n\n![demo](./doc/img/demo.gif)\n\n## Try it in 30 seconds\n\nNo install, no fictional binary — if you have Go, paste this. It records a real\nrun of a command you already have, then replays it as a test:\n\n```shell\ngo run github.com/nao1215/atago@latest record --out demo.atago.yaml -- git --version\ngo run github.com/nao1215/atago@latest run demo.atago.yaml\n```\n\n```text\n.\n\nPASSED  1 scenario: 1 passed, 0 failed, 0 errored, 0 skipped\n```\n\n`record` runs `git --version` once and writes a spec from what it observed — the\nexit code, the version line on stdout, an empty stderr. `run` replays it. Open\n`demo.atago.yaml` and you have a real test you can tighten, not YAML you wrote\nfrom scratch. (Swap `git --version` for any command you have: `go version`,\n`jq --version`, `ls -la`.)\n\nThen point it at your own tool:\n\n```shell\natago record --out mytool.atago.yaml -- mytool convert input.txt  # turn a real run into a spec\natago run mytool.atago.yaml                                       # replay it as a test\natago run --report junit specs/                                   # or run a whole suite in CI\n```\n\n## Why atago?\n\nPick the tool that owns your layer:\n\n| You are testing | Use |\n|-----------------|-----|\n| An HTTP/gRPC API server — scenario-based API testing | [runn](https://github.com/k1LoW/runn) |\n| A whole platform — integration suites across HTTP, gRPC, Kafka, databases, and more | [venom](https://github.com/ovh/venom) |\n| Shell functions and scripts — BDD-style unit tests | [ShellSpec](https://shellspec.info/) |\n| Bash scripts — TAP-style tests | [Bats](https://github.com/bats-core/bats-core) |\n| A CLI product — exit codes, output, generated files, snapshots, interactive prompts and TUIs | atago |\n\nIf the server or the platform is the system under test, use runn or venom. atago points the other way: the CLI is the product, and HTTP, database, SSH, gRPC, browser, and mock servers appear only as peers your CLI talks to.\n\n## Install\n\n```shell\ngo install github.com/nao1215/atago@latest\n```\n\nOn macOS, Homebrew works too:\n\n```shell\nbrew install --cask nao1215/tap/atago\n```\n\nOn Arch Linux, install the [`atago-bin`](https://aur.archlinux.org/packages/atago-bin) package from the AUR:\n\n```shell\nyay -S atago-bin   # or: paru -S atago-bin\n```\n\nThe [release page](https://github.com/nao1215/atago/releases) contains prebuilt binary archives for Linux, macOS, and Windows (amd64/arm64; `.tar.gz`, or `.zip` on Windows), plus `.deb`, `.rpm`, and `.apk` packages for Linux. Requires Go 1.26 or later when building from source.\n\nRuns on Linux, macOS, and Windows (CI tests all three).\n\n## Getting started\n\n### Start from a real run\n\nYou don't write the first spec — your tool does. `atago record -- \u003ccommand\u003e` runs it once and generates a spec from what it observed (exit code, output, created files):\n\n```shell\n$ atago record --out mytool.atago.yaml -- mytool convert input.txt\nrecorded: exit 0, 2 stdout line(s), 1 file(s) created\nwrote mytool.atago.yaml\n$ atago run mytool.atago.yaml\n.\n\nPASSED  1 scenario: 1 passed, 0 failed, 0 errored, 0 skipped (12ms)\n```\n\nInteractive tools record too: `atago record --pty -- \u003ccommand\u003e` runs it in a real terminal, lets you drive one session by hand, and writes a `pty:` step that replays your keystrokes as expect/send pairs. It works on Linux, macOS, and Windows (a ConPTY); on POSIX a password prompt becomes an `${env:...}` placeholder automatically, while on Windows — where a ConPTY exposes no echo state — you convert a secret send to `${env:...}` by hand. A `--pty` session is bounded by `--timeout` (default 30s): if the program never exits, atago kills it, writes whatever was captured, and fails instead of hanging forever:\n\n```shell\n$ atago record --pty --out wizard.atago.yaml -- mytool init\n```\n\nPrefer a blank template? `atago init` scaffolds one. Either way, the shape is always the same: declare a command, run it, assert on what it produced.\n\n### 1. Check exit code, stdout, and stderr\n\n```yaml\nversion: \"1\"\nsuite:\n  name: example\nscenarios:\n  - name: echo greets the world\n    steps:\n      - run:\n          shell: true            # portable: echo is a shell builtin on Windows\n          command: echo \"hello atago\"\n      - assert:\n          exit_code: 0\n          stdout:\n            contains: atago\n          stderr:\n            empty: true\n```\n\n`atago run` accepts spec files and directories (searched recursively for `*.atago.yaml`; the `*.atago.yml` spelling is accepted too). Each scenario runs in its own temporary directory, and progress streams as a dot per scenario (`.` pass, `F` fail, `E` error, `s` skip):\n\n```shell\n$ atago run ./specs\n...............................................\n\nPASSED  47 scenarios: 47 passed, 0 failed, 0 errored, 0 skipped (1.2s)\n```\n\nScenarios run concurrently by default (`--parallel N`, defaulting to your CPU count; set `--parallel 1` to serialize). Workdirs are isolated, but the host network is shared — so if two scenarios each start a background `service:`, give them distinct ports, or one scenario's requests can reach the other's server.\n\nWhen a check fails, atago prints exactly what was expected and what happened; multi-line mismatches render a colorized unified diff:\n\n```text\nFAILED: demo / greeting matches its golden\n\nStep:\n  assert stdout snapshot\n\nDiff (-expected +actual):\n  --- snapshot (golden)\n  +++ actual\n  @@ -1,3 +1,3 @@\n   hello\n  -WORLD\n  +world\n   bye\n\nHint:\n  stdout did not match snapshot \"snaps/greeting.txt\" (update with --update-snapshots if intended)\n```\n\n### 2. Check generated files and snapshots\n\n`fixture:` writes input files into the isolated workdir; `file:`/`dir:` assertions check what the command produced, and `snapshot:` pins output to a committed golden file (volatile details like temp paths, UUIDs, and timestamps are normalized). A fixture's source is one of `content:` (inline text), `base64:` (inline bytes), `from:` (copy an existing file), or `symlink:` (link to a target):\n\n```yaml\nscenarios:\n  - name: the generator writes the expected files\n    steps:\n      - run:\n          command: mytool generate --out site\n      - assert:\n          file:\n            path: site/index.html\n            contains:\n              - \"\u003chtml\"\n      - assert:\n          stdout:\n            snapshot: snapshots/generate.txt   # record/refresh with `atago snapshot update`\n```\n\nSee [files_and_fixtures](examples/files_and_fixtures.atago.yaml), [snapshot](examples/snapshot.atago.yaml), and [dir_tree](examples/dir_tree.atago.yaml) for whole-tree golden manifests.\n\n### 3. Drive interactive prompts and TUIs\n\nA `pty` step runs the command in a real pseudo-terminal and drives it with a declarative expect/send session — wizards, REPLs, and TTY-detection branches, no `expect(1)` scripting:\n\n```yaml\nscenarios:\n  - name: the init wizard completes\n    steps:\n      - pty:\n          command: mytool init\n          session:\n            - expect: \"Project name:\"\n            - send: \"demo\\n\"\n            - expect: \"created demo/\"\n      - assert:\n          exit_code: 0\n```\n\nNamed keys (`send: {key: enter}`) and asserts on the RENDERED terminal screen cover full TUIs — see [pty](examples/pty.atago.yaml), [pty_screen](examples/pty_screen.atago.yaml), and the cross-platform [pty_portable](examples/pty_portable.atago.yaml). `pty` steps and `atago record --pty` run on Linux, macOS, and Windows (where they drive a ConPTY pseudo-console); only `signal:` stays POSIX-only. The `pty`/`pty_screen` examples skip on Windows because their inner commands (`[ -t 0 ]`, `cat -v`, a SIGINT trap) are POSIX-specific, not because the `pty` mechanism is.\n\n### When your CLI talks to a server\n\nThe same YAML also drives HTTP, database, SSH, gRPC, headless-browser, and offline mock-server peers — as dependencies of the CLI under test. `atago init --template \u003cname\u003e` scaffolds each:\n\n```shell\n$ atago init --list-templates\nbrowser   drive a headless Chrome; assert page content (needs Chrome on PATH)\ncli       run a command; assert exit code/stdout/stderr (runs as-is)\ndb        run SQL; assert on rows via bundled SQLite (runs as-is)\ngrpc      call a unary gRPC method via server reflection (edit target first)\nhttp      call an HTTP API; assert status and JSON body (edit base_url first)\nmock      stub an HTTP API offline and assert what the client sent (needs curl on PATH)\nservices  test against a background server: readiness, retry, teardown (runs as-is)\nssh       run a command on a remote host over SSH (edit host/user first)\n```\n\n## Examples\n\nEvery feature has a commented, runnable spec under [examples/](examples/), tested in CI on Linux, macOS, and Windows. The **[cookbook](https://nao1215.github.io/atago/cookbook/)** collects 50+ copyable recipes for common jobs — converting images, driving prompts and TUIs, simulating API failures offline, proving idempotency — and indexes every spec by task and by feature.\n\nSelection flags compose with any spec: `--filter NAME` (repeatable, and comma-separated for OR — `--filter a,b` or `--filter a --filter b` runs scenarios whose name contains `a` or `b`), `--tag T` and `--skip-tag T` (tags match exactly, not by substring — `atago list` shows the available tags), `--parallel N`, `--fail-fast`, and `--rerun-failed`. While authoring, `--verbose` traces every command, capture, and assertion verdict — for passing scenarios too.\n\n## Use it in CI\n\nReal E2E suites flake (timing, ports, external tools). `--retry-failed N` re-runs failed scenarios in a fresh workdir and reports recovered ones as flaky — green for the exit code, but loud in every report format; silent retries are explicitly a non-goal. `--repeat N` does the opposite job: run each scenario N times to detect flakiness before it reaches CI.\n\n```shell\natago run --ci --retry-failed 2 ./specs          # keep CI green, report instability loudly\natago run --repeat 20 --filter \"race prone\" ./specs   # flake detection\n```\n\n[setup-atago](https://github.com/nao1215/setup-atago) installs a released binary:\n\n```yaml\nname: behavior-specs\non: [push, pull_request]\njobs:\n  atago:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: nao1215/setup-atago@v0\n      - run: atago run --ci --report gha ./specs\n```\n\nOn GitLab CI (or any CI that starts from a container image), use the published\nGHCR image:\n\n```yaml\nimage: ghcr.io/nao1215/atago:latest\n\nstages: [test]\n\nbehavior-specs:\n  stage: test\n  script:\n    - atago run --ci --report junit ./specs \u003e junit.xml\n  artifacts:\n    when: always\n    reports:\n      junit: junit.xml\n```\n\nThe image contains `atago` and `ca-certificates`; if your scenarios drive `git`,\n`jq`, a browser, or your own CLI binary, build FROM `ghcr.io/nao1215/atago:latest`\nand layer those tools on top.\n\n- `--report json|junit|gha|tap` picks the report format; the JSON shape is stable and versioned.\n- `--ci` enables deterministic, color-free output. It also turns an empty selection into a hard error: a `--filter`/`--tag`/`--skip-tag` that matches no scenario fails the run (exit 3) instead of passing an empty suite, so a typo cannot silently disable your specs. Without `--ci` the same case is a warning that still exits 0.\n- `--artifacts-dir DIR` persists the exact payloads a failed assertion compared — plus, for a failed scenario, its background services' logs and each mock server's recorded requests — so a failure stays reviewable after the job ends.\n- Environment variable names listed under `secrets:` are masked as `***` in all reports and snapshots.\n\n## Review specs without running them\n\n`explain` describes what a spec does, `doc` generates Markdown (with fixtures, expected payloads, and golden files inlined), `manifest` emits a stable JSON summary for tooling, and `list` shows scenarios, tags, and artifacts. All of them load and validate the spec first — exit code 2 on a schema error — so any of them doubles as a lint step in CI:\n\n![review](./doc/img/review.gif)\n\n```shell\natago explain spec.atago.yaml\natago doc --out docs/specs.md ./specs\natago manifest ./specs\natago list ./specs\n```\n\n## Snapshot testing\n\n`snapshot` matchers compare output against committed golden files; ANSI colors, temp paths, UUIDs, timestamps, ports, and CRLF are normalized so snapshots stay stable across machines. Record or refresh them with:\n\n![snapshot](./doc/img/snapshot.gif)\n\n```shell\natago snapshot update spec.atago.yaml\n```\n\nFor volatile patterns the built-ins do not cover — auto-increment IDs, request identifiers, epoch times — declare spec-wide `scrub:` rules that rewrite each regex match to a placeholder before the compare (applied after `secrets:` masking):\n\n```yaml\nscrub:\n  - {pattern: 'id=\\d+', placeholder: 'id=\u003cID\u003e'}\n```\n\nSee [scrub](examples/scrub.atago.yaml).\n\n## Editor support (JSON Schema)\n\nA JSON Schema lives at [schema/atago.schema.json](schema/atago.schema.json). With the YAML language server you get completion and validation as you type — step types, every matcher, and the `${workdir}` / `${env:NAME}` / `${name}` / `$${...}` expansion rules. `atago init` and `atago record` already emit this header as the first line of every generated spec, so scaffolded specs get completion out of the box. To add it to an existing spec, use the absolute URL (it resolves in any project, unlike a repo-relative path):\n\n```yaml\n# yaml-language-server: $schema=https://raw.githubusercontent.com/nao1215/atago/main/schema/atago.schema.json\nversion: \"1\"\n```\n\n## Shell completion\n\n`atago completion \u003cbash|zsh|fish|powershell\u003e` prints a completion script for your shell.\n\n## Exit codes\n\n| Code | Meaning                   |\n|------|---------------------------|\n| `0`  | all scenarios passed      |\n| `1`  | one or more failed        |\n| `2`  | spec error (YAML syntax or schema/semantic validation) |\n| `3`  | CLI-invocation error (unknown subcommand, bad flag, or no matching spec files) |\n| `4`  | execution error           |\n| `5`  | internal error            |\n| `6`  | security policy violation |\n\n`Ctrl-C`/`SIGTERM` stops the run cleanly: in-flight processes, services, and sessions are torn down, partial results are reported, and the run exits `4`.\n\n## Real CLIs tested with atago\n\nThese suites run real programs of every shape: the author's Go tools (atago tests itself) and unmodified third-party binaries — git and jq, interactive TUIs (fzf, htop), the python3 REPL, servers driven as scenario services (redis, gitea, grafana, prometheus), cloud and IaC CLIs tested offline (aws-cli, terraform, ecspresso), crypto tools (openssl, age, sops), and document/media pipelines (pandoc, ffmpeg). Most were migrated from ShellSpec. [Real CLIs tested with atago](https://nao1215.github.io/atago/real-world/) lists all 40+ with specs and generated behavior docs.\n\n## The name\n\natago (愛宕) is named for Mount Atago in Kyoto, whose shrine enshrines a deity of fire prevention. A test runner should do the same job: catch the sparks so a project never catches fire.\n\n## Contributing\n\nIssues and pull requests are welcome; see [CONTRIBUTING.md](./CONTRIBUTING.md). Contributions are not only about code: a GitHub Star also motivates development.\n\n## LICENSE\n\nThe atago project is licensed under the terms of [MIT LICENSE](./LICENSE).\n\n## Contributors ✨\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://debimate.jp/\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/22737008?v=4?s=75\" width=\"75px;\" alt=\"CHIKAMATSU Naohiro\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eCHIKAMATSU Naohiro\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/nao1215/atago/commits?author=nao1215\" title=\"Code\"\u003e💻\u003c/a\u003e \u003ca href=\"https://github.com/nao1215/atago/commits?author=nao1215\" title=\"Documentation\"\u003e📖\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnao1215%2Fatago","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnao1215%2Fatago","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnao1215%2Fatago/lists"}