{"id":34937274,"url":"https://github.com/zwave-js/remote-ctt-tests","last_synced_at":"2026-03-17T15:28:06.880Z","repository":{"id":329094800,"uuid":"1105834615","full_name":"zwave-js/remote-ctt-tests","owner":"zwave-js","description":"Automated Z-Wave Certification tests for Z-Wave JS","archived":false,"fork":false,"pushed_at":"2025-12-19T22:20:24.000Z","size":66346,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-20T21:56:24.967Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Tcl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zwave-js.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-28T08:21:12.000Z","updated_at":"2025-12-18T14:00:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/zwave-js/remote-ctt-tests","commit_stats":null,"previous_names":["zwave-js/remote-ctt-tests"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/zwave-js/remote-ctt-tests","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwave-js%2Fremote-ctt-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwave-js%2Fremote-ctt-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwave-js%2Fremote-ctt-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwave-js%2Fremote-ctt-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zwave-js","download_url":"https://codeload.github.com/zwave-js/remote-ctt-tests/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zwave-js%2Fremote-ctt-tests/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30626813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-17T14:16:03.965Z","status":"ssl_error","status_checked_at":"2026-03-17T14:16:03.380Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-26T18:33:24.032Z","updated_at":"2026-03-17T15:28:06.869Z","avatar_url":"https://github.com/zwave-js.png","language":"Tcl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Automation of Z-Wave Certification Tests\r\n\r\nThis project provides a complete framework for running Z-Wave CTT certification tests against a Controller DUT with device emulation based on the \"Open Source\" Z-Wave stack.\r\n\r\n## Prerequisites\r\n\r\nAs long as CTT-Remote does not run on Linux, we use WSL (Windows Subsystem for Linux) to run the Z-Wave stack binaries in a Linux environment while controlling them from Windows.\r\n\r\n- **Windows 10/11** or **Windows Server**\r\n- **WSL with Ubuntu** installed\r\n- **.NET Framework 4.8** (for CTT-Remote)\r\n\r\nIn addition, the test orchestrator requires:\r\n\r\n- **Node.js 24** or later\r\n- The `gh` CLI tool for downloading the Z-Wave stack binaries from GitHub and authentication\r\n\r\n\r\n## Architecture\r\n\r\n```\r\n┌──────────────────────────────────────────────────────────┐\r\n│                  Windows (Local or CI)                   │\r\n│                                                          │\r\n│  ┌────────────────────────────────────────────────────┐  │\r\n│  │ WSL (Ubuntu) - CTT Devices                         │  │\r\n│  │                                                    │  │\r\n│  │  • Controller 2 → TCP :5001 (CTT Controller1)      │  │\r\n│  │  • Controller 3 → TCP :5002 (CTT Controller3)      │  │\r\n│  │  • End Device 1 → TCP :5003 (CTT EndDevice1)       │  │\r\n│  │  • End Device 2 → TCP :5004 (CTT EndDevice2)       │  │\r\n│  │  • Zniffer      → TCP :4905 (CTT Zniffer)          │  │\r\n│  └────────────────────────────────────────────────────┘  │\r\n│                          ↑                               │\r\n│  ┌────────────────────────────────────────────────────┐  │\r\n│  │ CTT-Remote.exe                                     │  │\r\n│  │ Connects to controllers/devices on ports 5001-5004 │  │\r\n│  └────────────────────────────────────────────────────┘  │\r\n│                          ↑                               │\r\n│                    WebSocket :4712                       │\r\n│                          ↓                               │\r\n│  ┌────────────────────────────────────────────────────┐  │\r\n│  │ Orchestrator                                       │  │\r\n│  │ Controls test execution and coordinates components │  │\r\n│  └────────────────────────────────────────────────────┘  │\r\n│                          ↑                               │\r\n│                    WebSocket :4713                       │\r\n│                          ↓                               │\r\n│  ┌────────────────────────────────────────────────────┐  │\r\n│  │ DUT Runner                                         │  │\r\n│  │ Manages DUT lifecycle and handles CTT prompts      │  │\r\n│  │ Controls DUT device                                │  │\r\n│  └────────────────────────────────────────────────────┘  │\r\n│                          ↓                               │\r\n│  ┌────────────────────────────────────────────────────┐  │\r\n│  │ WSL (Ubuntu) - DUT Device                          │  │\r\n│  │                                                    │  │\r\n│  │  • Controller 1 → TCP :5000 (DUT)                  │  │\r\n│  └────────────────────────────────────────────────────┘  │\r\n└──────────────────────────────────────────────────────────┘\r\n```\r\n\r\n## Installation\r\n\r\n### 1. Clone the Repository\r\n\r\n```bash\r\ngit clone \u003crepository-url\u003e\r\ncd remote-ctt-tests\r\n```\r\n\r\n### 2. Install Dependencies\r\n\r\n```bash\r\nnpm install\r\n```\r\n\r\n### 3. One-Time WSL Setup\r\n\r\nRun these commands in WSL (open WSL terminal with `wsl`):\r\n\r\n```bash\r\nsudo dpkg --add-architecture i386\r\nsudo apt-get update\r\nsudo apt-get install -y libc6:i386 libstdc++6:i386\r\n```\r\n\r\nThese commands:\r\n- Enable 32-bit architecture support\r\n- Install 32-bit C/C++ libraries required by the Z-Wave binaries\r\n- Make the Z-Wave binaries executable\r\n\r\n\r\n## Getting Started\r\n\r\n### Step 1: Install CTT\r\n\r\nCTT is not included in this repository. You need to install it separately. It will later be bundled so it can be used in CI.\r\nIf the location differs from the default install location, set the `CTT_PATH` environment variable.\r\n\r\n### Step 2: Download Z-Wave Stack\r\n\r\n```bash\r\npowershell .\\setup\\download-zwave-stack.ps1\r\n```\r\n\r\n\u003e **Note:** This setup assumes the DUT is a **controller**, which connects to the emulated controller on port 5000. Testing sample applications (end devices) should also be possible but requires copying additional files from the stack binaries, and updating the `zwave_stack/run.sh` script accordingly.\r\n\r\n### Step 3: Run Emulated Devices\r\n\r\n```bash\r\nnpm run devices\r\n```\r\n\r\nStarts 6 Z-Wave binaries in WSL:\r\n\r\n| Device | Port | Purpose |\r\n|--------|------|---------|\r\n| Controller 1 | 5000 | **Your DUT connects here** |\r\n| Controller 2 | 5001 | CTT |\r\n| Controller 3 | 5002 | CTT |\r\n| End Device 1 | 5003 | CTT |\r\n| End Device 2 | 5004 | CTT |\r\n| Zniffer | 4905 | CTT packet capture |\r\n\r\nThese can now be used to set up the CTT project.\r\n\r\n### Step 4: Configure CTT Project\r\n\r\n1. Create a new project in CTT GUI\r\n2. Set up 5 IP-based devices:\r\n   - 2x Controller: `127.0.0.1:5001` and `:5002`\r\n   - 2x End Device: `127.0.0.1:5003` and `:5004`\r\n   - 1x Zniffer: `127.0.0.1:4905`\r\n\r\n### Step 5: Set Up CTT Network with DUT\r\n\r\nConfigure DUT to connect to `tcp://127.0.0.1:5000`, then establish the test network:\r\n\r\n- **Option A:** DUT includes CTT devices into its network\r\n- **Option B:** CTT includes DUT into its network\r\n\r\nTo test both scenarios, you'll need separate CTT projects.\r\n\r\nMake sure to finish creation of the network, including the query for DUT capabilities.\r\n\r\n### Step 6: Copy CTT Project Files\r\n\r\nCopy from CTT's project folder to `ctt/project/`:\r\n\r\n```\r\nctt/project/\r\n├── Config/                            # All config files\r\n├── json/                              # JSON configurations\r\n├── \u003cyour-project\u003e.cttsln                    # Project file\r\n└── ZWave_CTT_CommandClasses.cttxml    # Command classes definition\r\n```\r\n\r\n### Step 7: Create DUT Runner Script\r\n\r\nImplement the IPC protocol (JSON-RPC 2.0 over WebSocket):\r\n\r\n- Connect to port 4713 (or `RUNNER_IPC_PORT` env var)\r\n- Required methods:\r\n  - `ready` notification (on connect)\r\n  - `start` (initialize DUT with controllerUrl and security keys)\r\n  - `stop` (shutdown DUT)\r\n  - `handleCttPrompt` (respond to CTT prompts)\r\n\r\nSee [dut/zwave-js/run.ts](dut/zwave-js/run.ts) for a reference implementation and [docs/ipc-protocol.md](docs/ipc-protocol.md) for the full protocol specification.\r\n\r\n### Step 8: Update config.json\r\n\r\n```json\r\n{\r\n  \"dut\": {\r\n    \"name\": \"Your DUT Name\",\r\n    \"runnerPath\": \"your-dut/run.ts\",\r\n    \"homeId\": \"d34db33f\",\r\n    \"storageDir\": \"your-dut/storage\",\r\n    \"storageFileFilter\": [\"%HOME_ID_LOWER%.jsonl\"]\r\n  }\r\n}\r\n```\r\n\r\n**Field explanations:**\r\n\r\n- `runnerPath`: Path to your DUT runner script. Supports Node.js (TypeScript/JavaScript), Python, or any executable that your system can handle running directly, e.g. with a shebang.\r\n- `homeId`: Must match the Home ID of your test network (from CTT setup)\r\n- `storageDir` / `storageFileFilter`: Used to transfer DUT network state to GitHub for automated CI testing. The filter patterns support placeholders:\r\n  - `%HOME_ID_LOWER%` - homeId in lowercase\r\n  - `%HOME_ID_UPPER%` - homeId in uppercase\r\n\r\n### Step 9: Pack Archives\r\n\r\n```bash\r\npowershell .\\setup\\pack-ctt-archive.ps1\r\npowershell .\\setup\\pack-network-state-archive.ps1\r\n```\r\n\r\nBoth archives are required for CI/automated testing. They will need to be regenerated if the CTT setup or project changes.\r\n\r\n- **ctt-setup.zip**: Contains CTT (closed-source, vendored) keys, appdata, and configuration\r\n- **network-state.zip**: Contains network state for emulated Z-Wave devices and DUT storage\r\n\r\n### Step 10: Git Commit\r\n\r\n**Check in:**\r\n\r\n- `config.json`\r\n- DUT runner script (`your-dut/run.ts`)\r\n- CTT project files (`ctt/project/`)\r\n- Setup archives (`setup/ctt-setup.zip`, `setup/network-state.zip`)\r\n\r\n**Update .gitignore:**\r\n\r\nThe default `.gitignore` excludes common paths, but may need modifications depending on how your DUT stores state. \r\nAdd exclusions for your DUT's storage directory (e.g., `your-dut/storage/`). This should be included in the `network-state.zip` archive instead.\r\n\r\n## Testing and CI/CD\r\n\r\nTo test locally, run:\r\n\r\n```bash\r\nnpm run start -- [options]\r\n```\r\n\r\n### Options\r\n\r\n| Option | Description |\r\n|--------|-------------|\r\n| `--discover` | List all available test cases grouped by category |\r\n| `--test=\u003cname\u003e` | Run a specific test by name |\r\n| `--test=\u003cn1\u003e,\u003cn2\u003e` | Run multiple tests (comma-separated) |\r\n| `--category=\u003ccat\u003e` | Run all tests in a category (partial match, case-insensitive) |\r\n| `--category=\u003cc1\u003e,\u003cc2\u003e` | Run tests from multiple categories |\r\n| `--group=\u003cgrp\u003e` | Run tests in a group (`Automatic` or `Interactive`) |\r\n| `--group=\u003cg1\u003e,\u003cg2\u003e` | Run tests from multiple groups |\r\n| `--dut=\u003cpath\u003e` | Path to DUT runner (defaults to `config.json` runner path) |\r\n| `--devices-only` | Only start emulated Z-Wave devices, without CTT or the DUT runner |\r\n| `--verbose` | Show CTT-Remote log output |\r\n\r\n### Environment Variables\r\n\r\n| Variable | Default | Description |\r\n|----------|---------|-------------|\r\n| `CTT_PATH` | `C:\\Program Files (x86)\\Z-Wave Alliance\\Z-Wave CTT 3` | Path to CTT installation |\r\n\r\n### Examples\r\n\r\n```bash\r\n# List all available tests\r\nnpm run start -- --discover\r\n\r\n# Run a single test\r\nnpm run start -- --test=CC_Binary_Switch_Set\r\n\r\n# Run multiple specific tests\r\nnpm run start -- --test=CC_Binary_Switch_Set,CC_Binary_Switch_Get\r\n\r\n# Run all automatic tests (for CI)\r\nnpm run start -- --group=Automatic\r\n\r\n# Run all tests in a category\r\nnpm run start -- --category=Binary\r\n\r\n# Combine filters (category AND group)\r\nnpm run start -- --category=Binary --group=Automatic\r\n\r\n# Start only the emulated devices (for manual testing)\r\nnpm run start -- --devices-only\r\n\r\n# Run with verbose CTT output\r\nnpm run start -- --test=CC_Binary_Switch_Set --verbose\r\n```\r\n\r\nThe project comes with a ready-to-use GitHub Actions workflow for running CTT tests in CI using WSL. For now, only tests from the \"Automatic\" group are supported, because they don't require manual interaction.\r\n\r\nTo use the workflow, configure a repository secret named `ZW_STACK_TOKEN` with a GitHub PAT that has **Contents: read** permission for the [Z-Wave-Alliance/z-wave-stack-binaries](https://github.com/Z-Wave-Alliance/z-wave-stack-binaries) repository.\r\n\r\n## Troubleshooting\r\n\r\n### CTT fails with `Error - RequestNodeInfo failed!`**\r\n\r\nMake sure that the DUT has all command classes set (factory reset may be required) before joining the CTT network\r\n\r\n### CTT fails to communicate securely with the DUT\r\n\r\nMake sure that:\r\n- The DUT storage contains the correct files\r\n- The home ID is configured correctly everywhere:\r\n    - `config.json`\r\n    - `ctt/project/Config/TestCaseStaticController.xml` (HomeId attribute)\r\n    - `ctt/project/Config/Saved Items/002_\u003cHOMEID\u003e.xml` (filename)\r\n\r\n### Z-Wave stack binaries frequently crash or trigger the watchdog\r\n\r\nSomething in the storage is likely corrupted. Delete all stack storage files and re-create the CTT network from scratch.\r\nMake sure to update the home ID accordingly, see above.\r\n\r\n## Documentation\r\n\r\n- **[docs/ipc-protocol.md](docs/ipc-protocol.md)** - DUT Runner IPC protocol specification\r\n- **[CTT-Remote/CTT-Remote.md](CTT-Remote/CTT-Remote.md)** - CTT-Remote API documentation\r\n- **[.github/workflows/run-zwave-wsl.yml](.github/workflows/run-zwave-wsl.yml)** - CI workflow configuration\r\n\r\n## License\r\n\r\nMIT\r\n\r\n## Support\r\n\r\nFor issues or questions, please open an issue in the GitHub repository.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzwave-js%2Fremote-ctt-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzwave-js%2Fremote-ctt-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzwave-js%2Fremote-ctt-tests/lists"}