{"id":20528240,"url":"https://github.com/manticoresoftware/clt","last_synced_at":"2025-08-17T11:08:19.465Z","repository":{"id":156115496,"uuid":"626830281","full_name":"manticoresoftware/clt","owner":"manticoresoftware","description":"Command Line Tester","archived":false,"fork":false,"pushed_at":"2025-08-07T07:27:39.000Z","size":52423,"stargazers_count":6,"open_issues_count":20,"forks_count":3,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-08-07T09:26:00.358Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Svelte","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/manticoresoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-04-12T08:43:31.000Z","updated_at":"2025-08-07T07:27:42.000Z","dependencies_parsed_at":"2023-10-11T05:50:55.659Z","dependency_job_id":"ec4956a0-3e1b-4b96-88a5-61177f8e1050","html_url":"https://github.com/manticoresoftware/clt","commit_stats":null,"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"purl":"pkg:github/manticoresoftware/clt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fclt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fclt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fclt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fclt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/manticoresoftware","download_url":"https://codeload.github.com/manticoresoftware/clt/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fclt/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270837583,"owners_count":24654391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-17T02:00:09.016Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T23:24:09.338Z","updated_at":"2025-08-17T11:08:19.448Z","avatar_url":"https://github.com/manticoresoftware.png","language":"Svelte","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Command Line Tester (CLT)\n\nA powerful testing tool that allows you to execute and record commands inside a Docker container, replay them, and validate whether your results match expected outputs. CLT helps automate testing of command-line applications with consistent environments.\n\n## Features\n\n- Record interactive command sessions in reproducible test files\n- Replay recorded sessions to verify consistent behavior\n- Intelligent output comparison with regex pattern matching\n- Reusable test blocks for modular testing\n- GitHub Action support for CI/CD integration\n- Cross-platform compatibility via Docker containers\n\n## Installation\n\nInstallation is straightforward. Simply clone the repository and start using it:\n\n```bash\ngit clone https://github.com/manticoresoftware/clt.git\ncd clt\n./clt help\n```\n\n### Prebuilt Binaries\n\nCLT provides prebuilt binaries for Linux environments (both amd64 and arm64), which are automatically detected based on your platform. While binaries for macOS and Windows are not currently available, they can be built if required.\n\nNOTE: CLT requires Docker to be installed on your machine as all tests run in a container environment.\n\n## Usage\n\n### Recording Tests\n\n1. Start recording a test session with the `record` command:\n\n   ```bash\n   ./clt record centos:7\n   ```\n\n   This starts a recording session using the specified Docker image. You'll see the path to the recording file displayed:\n\n   ```bash\n   Recording data to file: ./tests/centos:7_20230714_161043.rec\n   ```\n\n2. Perform your commands in the interactive shell. All inputs and outputs will be recorded.\n\n3. When finished, press `^D` (Ctrl+D) once to stop recording and save your test.\n\n### Replaying and Validating Tests\n\nTo validate and replay a recorded test:\n\n```bash\n./clt test -t ./tests/centos:7_20230714_161043.rec -d centos:7\necho $?\n```\n\n- `-t` specifies the test file to replay\n- `-d` enables debug output that shows diff information if results don't match\n\nThe exit code will be 0 if the test passed (outputs match) or 1 if there are differences.\n\n## Web-based UI\n\nCLT includes a web-based user interface for editing and running tests. The UI provides an intuitive interface for managing test files, with real-time pattern matching and diff comparison. To learn more about the UI, its features, and how to set it up, see the [UI documentation](ui/README.md).\n\nKey UI features include:\n- File tree explorer for navigating test files\n- Real-time comparison with pattern matching\n- Docker image configuration for test validation\n- GitHub authentication for access control\n\nWe utilize bash to initiate an interactive environment when you record a test. It's important to note that we reset the environment to ensure maximum compatibility with various operating systems. As of now, there is no option to pass environment variables from outside into the test environment.\n\n### Refining Tests\n\nAfter recording, you can refine your test to handle dynamic outputs:\n\n```bash\n./clt refine -t ./tests/centos:7_20230714_161043.rec centos:7\n```\n\nThis will open your configured editor to modify test outputs and add patterns where needed.\n\n## Test File Syntax\n\nCLT test files (`.rec`) use a special syntax to differentiate between commands, outputs, and other sections.\n\n### Supported Sections\n\n**Note:** CLT uses en dashes (`–`) in its section markers, not regular hyphens (`-`). The code samples below show the correct syntax with en dashes.\n\n| Section | Syntax | Description |\n|---------|--------|-------------|\n| `Input` | `––– input –––` | Marks the beginning of a command input section |\n| `Output` | `––– output –––` | Marks the beginning of a command output section |\n| `Output with Checker` | `––– output: checker-name –––` | Like output but uses a custom checker program |\n| `Block` | `––– block: path/to/block –––` | Includes a reusable test block from a `.recb` file |\n| `Comment` | `––– comment –––` | Marks a comment section (ignored during test execution) |\n\n**Note:** The `duration` section (`––– duration: 123ms (45.67%) –––`) is automatically generated by the test system and should not be added manually to `.rec` files. It appears in the resulting `.rep` files after test execution.\n\n### Basic Structure\n\nA test file typically consists of repeating input and output sections:\n\n```\n––– input –––\necho \"Hello, world!\"\n––– output –––\nHello, world!\n```\n\nEach input section contains commands exactly as they were typed, and each output section contains the expected output that should be matched during test replay.\n\n## Pattern Matching\n\nFor outputs that change between runs (timestamps, PIDs, etc.), CLT supports regex pattern matching.\n\n### Using Regular Expressions\n\nPlace regex patterns between `#!/` and `/!#` markers in your test output:\n\n```\n#!/[0-9]+/!#  # Matches any number\n```\n\n\n### Using Named Patterns\n\nCLT provides predefined patterns in the `.clt/patterns` file. Use them with the syntax `%{PATTERN_NAME}`:\n\n```\n%{SEMVER}  # Matches semantic version numbers\n```\n\nYou can define custom patterns by adding them to your project's `.clt/patterns` file using the format:\n\n```\nPATTERN_NAME REGEX_PATTERN\n```\n\nExample `.clt/patterns` file:\n\n```\nSEMVER [0-9]+\\.[0-9]+\\.[0-9]+\nYEAR [0-9]{4}\nIPADDR [0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+\nCOMMITDATE [a-z0-9]{7}@[0-9]{6}\n```\n\n**Note:** CLT first loads the global patterns from `$PROJECT_DIR/.clt/patterns` and then appends your project-specific patterns from `.clt/patterns` for extended functionality.\n\n## Reusable Blocks\n\nCreate modular test components by extracting flows into `.recb` files placed in the same directory as your test files:\n\n```\n––– block: login-sequence –––\n```\n\nThis will include the contents of the `login-sequence.recb` file located in the same directory as your test. You can also organize blocks in subdirectories using relative paths:\n\n```\n––– block: auth/admin-login –––\n```\n\nThe path is always relative to the location of the `.rec` file that's including the block. Blocks can be nested, allowing you to compose complex test scenarios from reusable components.\n\n### Block Internal Steps (UI Feature)\n\nWhen using the CLT UI, you can add steps directly inside block references. These internal steps are automatically saved to the corresponding `.recb` file:\n\n- **Block with internal steps**: Steps added inside a block in the UI are saved to the `.recb` file\n- **Path resolution**: Block paths like `../base/block.recb` are resolved relative to the test file directory\n- **Automatic file creation**: `.recb` files are created/updated automatically when blocks contain internal steps\n- **Backward compatibility**: Existing block references without internal steps continue to work as before\n- **Status propagation**: Block internal steps now properly inherit status from test execution results\n- **UI feedback**: Interface provides visual feedback during test runs with opacity effects\n\n## Custom Output Checkers\n\nFor advanced output validation beyond regex patterns, CLT supports custom checker programs. These are specified in the output statement:\n\n```\n––– output: custom-checker –––\n```\n\n### How Custom Checkers Work\n\n1. Place your checker executable in the `.clt/checkers/` directory\n2. The checker will receive two arguments:\n   - First argument: Path to a temp file containing expected output (from .rec file)\n   - Second argument: Path to a temp file containing actual output (from replay)\n3. The checker should return:\n   - Exit code 0 if outputs match (test passes)\n   - Non-zero exit code if there are differences (test fails)\n4. Any stdout/stderr output from the checker will be displayed when there are differences\n\n### Creating Custom Checkers\n\nCheckers can be written in any language, but must be executable. Here's an example of a simple bash checker that verifies if a specific string exists in the output:\n\n```bash\n#!/bin/bash\n# simple-contains-checker.sh - A checker that verifies if output contains a specific string\n\nEXPECTED_FILE=\"$1\"  # Path to expected output from .rec file\nACTUAL_FILE=\"$2\"    # Path to actual output from test run\n\n# Get the string to search for from the first line of expected file\nSEARCH_STRING=$(head -n 1 \"$EXPECTED_FILE\")\n\n# Check if the actual output contains this string\nif grep -q \"$SEARCH_STRING\" \"$ACTUAL_FILE\"; then\n  echo \"✓ Found expected string: $SEARCH_STRING\"\n  exit 0  # Success\nelse\n  echo \"✗ Missing expected string: $SEARCH_STRING\"\n  exit 1  # Failure\nfi\n```\n\nThis provides great flexibility for validating complex outputs without needing regex patterns.\n\n## Environment Variables\n\nCLT's behavior can be customized through several environment variables:\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `CLT_DEBUG` | Enable debug mode to show detailed output | `0` |\n| `CLT_DIFF_INLINE` | Display differences in-line instead of side-by-side | `0` |\n| `CLT_EDITOR` | Custom editor for refining tests | Auto-detected (`nano` or `vim`) |\n| `CLT_PROMPTS` | Array of additional prompts to detect (e.g., `CLT_PROMPTS=(\"mysql\u003e \")`) | `(\"clt\u003e \")` |\n| `CLT_NO_COLOR` | Disable colored output | Not set |\n| `DEFAULT_DELAY` | Default delay in ms between each command in the test | `5` |\n| `CLT_RUN_ARGS` | Additional arguments to pass to `docker run` | Not set |\n\n## GitHub Actions Integration\n\nCLT provides a ready-to-use GitHub action to run tests in your CI/CD pipeline. You can customize the action with several parameters:\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `image` | Docker image to run the tests in | Required |\n| `test_prefix` | Filter tests by prefix in tests folder | `tests/` |\n| `run_args` | Additional arguments to pass to `docker run` | `''` |\n| `init_code` | Code to execute before running tests | `''` |\n| `timeout` | Allowed timeout in minutes for tests | `5` |\n| `artifact` | Artifact to download for local built docker image | Not set |\n| `repository` | Repository to checkout | Not set |\n| `ref` | Branch name or ref to checkout | Default branch |\n\n### Example Workflow\n\n```yaml\nname: CLT tests\n\non:\n  push:\n    branches: [ main ]\n  pull_request:\n    branches: [ main ]\n\njobs:\n  clt:\n    name: Run CLT tests\n    strategy:\n      fail-fast: false\n      matrix:\n        image: [ \"ubuntu:bionic\", \"ubuntu:focal\", \"ubuntu:jammy\", \"debian:buster\", \"debian:bullseye\", \"debian:bookworm\" ]\n    runs-on: ubuntu-22.04\n    steps:\n      - uses: manticoresoftware/clt@v3\n        with:\n          image: ${{ matrix.image }}\n          test_prefix: test/clt-tests/\n          run_args: -e TELEMETRY=0\n          timeout: 10\n          init_code: |\n            apt-get update\n            apt-get install -y curl\n```\n\n## Project Structure\n\nCLT uses a specific directory structure for its key files:\n\n```\n.clt/\n├── checkers/     # Custom output checker executables\n└── patterns      # Pattern definitions for regex matching\ntests/\n├── *.rec         # Test recording files\n└── *.recb        # Reusable test blocks\n```\n\nThe `.clt/` directory contains CLT configuration and special-purpose files, while the actual test files and reusable blocks are stored in the tests directory or other directories specified for your tests.\n\n## File Extensions\n\nCLT uses several file extensions:\n\n| Extension | Description |\n|-----------|--------------|\n| `.rec` | Original record of inputs and outputs, may include links to block files |\n| `.recb` | Reusable block file that can be included in `.rec` files |\n| `.rep` | Replay file containing results of replaying a `.rec` file |\n\n## Current Limitations\n\n- Use `^D` (Ctrl+D) only once to close the CLT environment; for other exits, use the `exit` command\n- Avoid using `^C`, `^V`, `^Z`, and other control characters as they may not function correctly\n- Reverse search (`^R`) is currently unsupported\n- Complete tests in the `clt\u003e ` shell and press `^D` to terminate the session\n- Use a simple terminal like iTerm; VS Code terminal may cause unusual behavior\n- Each command in the `.rec` test must have an output that contains a newline\n\n### Supported Keystrokes\n\nOnly the following keystrokes are fully supported:\n\n- Standard input characters\n- Left and right arrow keys\n- Backspace and delete keys\n- CTRL+a (beginning of line) and CTRL+e (end of line)\n\n## Developers Section\n\n### Building Tools\n\nTo build the `rec` and `cmp` tools for both aarch64 and amd64 Linux:\n\n```bash\n./bin/cross-build\ngit add .\ngit commit -m 'Update binaries'\n```\n\n### The Replay Flow\n\nDuring test execution, CLT:\n\n1. Finds `.rec` files and compiles them into a ready-to-use version on the fly\n2. Executes each command in sequence, generating a `.rep` file\n3. Uses the `cmp` tool to compare the results with the compiled version of the `.rec` files\n\n### Using Custom Prompts\n\nIf you need to work with additional command prompts (e.g., `mysql\u003e` prompt), configure them as follows:\n\n```bash\nCLT_PROMPTS=(\"mysql\u003e \")\n```\n\nThis helps CLT correctly detect command completion in different interactive shells.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanticoresoftware%2Fclt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanticoresoftware%2Fclt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanticoresoftware%2Fclt/lists"}