{"id":20857143,"url":"https://github.com/flipperdevices/flipper-github-runner-docker-qa","last_synced_at":"2025-04-10T21:09:18.171Z","repository":{"id":196822135,"uuid":"690954194","full_name":"flipperdevices/flipper-github-runner-docker-qa","owner":"flipperdevices","description":"Self-hosted github actions runners for QA","archived":false,"fork":false,"pushed_at":"2025-04-10T11:41:54.000Z","size":117,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":6,"default_branch":"dev","last_synced_at":"2025-04-10T21:09:15.877Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flipperdevices.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-09-13T08:05:40.000Z","updated_at":"2025-04-10T11:41:58.000Z","dependencies_parsed_at":"2025-03-07T13:31:53.092Z","dependency_job_id":"3c6bff12-3c7f-4666-9250-2fa9c0950119","html_url":"https://github.com/flipperdevices/flipper-github-runner-docker-qa","commit_stats":null,"previous_names":["flipperdevices/flipper-github-runner-docker-qa"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipperdevices%2Fflipper-github-runner-docker-qa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipperdevices%2Fflipper-github-runner-docker-qa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipperdevices%2Fflipper-github-runner-docker-qa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipperdevices%2Fflipper-github-runner-docker-qa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flipperdevices","download_url":"https://codeload.github.com/flipperdevices/flipper-github-runner-docker-qa/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248298346,"owners_count":21080320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-18T04:36:27.513Z","updated_at":"2025-04-10T21:09:18.165Z","avatar_url":"https://github.com/flipperdevices.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flipper Zero GitHub Runner System\n\nThis system provides a complete solution for running self-hosted GitHub runners with Flipper Zero devices for automated testing, flashing, and QA tasks. The system includes a runner management service, monitoring capabilities, and handles the device firmware flashing lifecycle.\n\n## Overview\n\nThe system consists of several components:\n- Docker-based GitHub runner containers\n- Python management scripts\n- Systemd services for automatic device detection and binding\n- Monitoring solution with Prometheus integration\n\n### Flow\n\n1. **Device Detection**: udev rules detect when Flipper devices are connected and trigger binding services\n2. **Binding**: The binding service creates mounts in `/dev/flipper/`\n2. **Initialization**: The systemd service starts a Python script that locates the specified ST-Link and Flipper devices\n3. **Repair Mode**: Docker container runs in 'REPAIR' state first, which flashes the latest release firmware to the Flipper device\n4. **Normal Mode**: After successful firmware flashing, the container restarts in 'NORMAL' state and registers as a GitHub self-hosted runner\n5. **Job Execution**: The runner picks up jobs with matching tags from GitHub and executes them\n6. **Monitoring**: A dedicated monitoring service tracks the status of all runners and provides metrics\n\n## Prerequisites\n\nBefore installation, ensure:\n1. Docker is installed and properly configured\n2. Python 3.6+ is available\n3. The Flipper Zero device and ST-Link device are connected via USB\n4. You have the serial IDs for both devices (can be obtained via `lsusb -v` or `ls -l /dev/serial/by-id/`)\n5. You have appropriate GitHub permissions to register self-hosted runners\n\n## Installation\n\n### Runner Installation\n\n```bash\nsudo ./installer.sh --flipper=FLIPPER_SERIAL --stlink=STLINK_SERIAL [--github-tag=GITHUB_TAG] [--simulate]\n```\n\nExample:\n```bash\nsudo ./installer.sh --flipper=flip_Testii --stlink=002F00000000000000000001 --github-tag=FlipperDeviceTest\n```\n\nThis will:\n1. Create necessary directories in `/opt/flipper-runner/`\n2. Copy scripts and service files to their respective locations\n3. Install udev rules for automatic device detection\n4. Configure systemd services for the specified Flipper and ST-Link devices\n5. Install service binaries to `/usr/local/bin/`\n\nThe `--simulate` flag can be used to preview the changes without actually making them.\n\nAfter that you need just to put firmware repo and build Docker image\n```bash\ncd /var/lib/flipper-docker/\ngit clone --branch 1.2.0 git@github.com:flipperdevices/flipperzero-firmware.git\ndocker build -t flipper-custom-image:FlipperZeroTest .\n```\n\n### Monitor Installation\n\n```bash\nsudo ./monitor-installer.sh\n```\n\nThis will:\n1. Install the monitoring service at `/opt/flipper-monitor/`\n2. Configure Prometheus metrics collection in `/var/lib/node_exporter/textfile_collector/`\n3. Set up log rotation for `/var/log/github-runner-metrics.log`\n4. Enable the monitoring systemd service\n\n## Configuration\n\n### Runner Configuration\n\nCreate a configuration file at `/var/lib/flipper-docker/flipper-docker.cfg`:\n\n```ini\n[github]\naccess_token = GITHUB_ACCESS_TOKEN\norg_name = GITHUB_ORG_NAME\napp_id = GITHUB_APP_ID\napp_private_key = GITHUB_APP_PRIVATE_KEY\n\n[gelf]\nhost = GELF_HTTP_UPLOAD_URL\nport = GELF_HTTP_UPLOAD_PORT\nusername = GELF_HTTP_UPLOAD_BASIC_AUTH_USER\npassword = GELF_HTTP_UPLOAD_BASIC_AUTH_PASS\n```\n\nWhere:\n- `GITHUB_ACCESS_TOKEN` - GitHub Personal Access Token\n- `GITHUB_ORG_NAME` - GitHub organization name\n- `GITHUB_APP_ID` - GitHub App ID (if using GitHub App authentication)\n- `GITHUB_APP_PRIVATE_KEY` - GitHub App private key (if using GitHub App authentication)\n- `GELF_HTTP_UPLOAD_*` - Optional GELF logging configuration\n\n### Docker Configuration\n\nYou'll need to prepare a Docker image with the necessary tools for Flipper Zero development. The Dockerfile should include:\n\n1. Base on GitHub runner image (e.g., `myoung34/github-runner:latest`)\n2. Python 3 and required libraries\n3. Build tools (gcc, make, etc.)\n4. libusb for USB device access\n5. Flipper Zero build toolchain\n\nPlace your Dockerfile and any required assets in a directory that will be copied to `/var/lib/flipper-docker/`.\n\n## Device Management\n\n### Automatic Device Detection\n\nThe system uses udev rules to automatically detect when Flipper devices are connected or disconnected:\n\n1. When a Flipper device is connected, the udev rule triggers the binder service\n2. The binder service creates the necessary device mappings in `/dev/flipper/`\n3. When a Flipper device is disconnected, the unbinder service removes these mappings\n\nThis allows for hot-plugging Flipper devices without manual intervention.\n\n## Monitoring\n\nThe monitoring system collects metrics about runner state, container status, and job execution.\n\n### Metrics\n\nThe following metrics are available in Prometheus format:\n- `github_runner_state` - Current state of GitHub runners (offline, starting, repairing, online, error, flashing)\n- `github_runner_container_status` - Docker container status\n- `github_runner_run_level` - Current run level (REPAIR or NORMAL)\n- `github_runner_service_status` - Systemd service status\n- `github_runner_uptime_seconds` - Runner uptime\n- `github_runner_job_info` - Information about currently running jobs\n- `github_runner_job_runtime_seconds` - Duration of current job\n\n### Metrics Collection\n\nThe `github-runner-monitor.py` script collects data from multiple sources:\n- **systemd Journal**: For runner state information\n- **Docker API**: For container status and job information\n- **systemd Units**: For service status\n\nData is processed and formatted as Prometheus metrics, then written to the Node Exporter textfile directory.\n\n### Integrations\n\nThe monitoring service outputs metrics to `/var/lib/node_exporter/textfile_collector/`, ready to be picked up by Prometheus Node Exporter.\nMake sure to configure Node Exporter to scrape the metrics directory by adding\n`-v /var/lib/node_exporter/textfile_collector:/textfile_collector:ro \\\n--collector.textfile.directory=/textfile_collector` to your docker create script.\n\n## Logs\n\n- Runner logs: `/opt/flipper-runner/FLIPPER_SERIAL/logs/`\n- Container logs: Accessible via `docker logs` or systemd journal\n- Monitoring logs: `/var/log/github-runner-metrics.log`\n\nLog rotation is configured automatically to prevent excessive disk usage.\n\n## Runner States\n\nRunners go through the following states:\n1. **Offline**: Not running\n2. **Starting**: Service is initializing\n3. **Repairing**: In repair mode, preparing to flash firmware\n4. **Flashing**: Actively flashing firmware to Flipper\n5. **Online**: Registered with GitHub and ready for jobs\n6. **Error**: Encountered an error during operation\n\n## Implementation Details\n\n### Run Levels\n\nThe system operates in two primary run levels:\n- **REPAIR**: Focuses on flashing firmware to Flipper Zero\n- **NORMAL**: Operates as a GitHub runner\n\nWhen a new device is added or after a job completes, the system starts in REPAIR mode, flashes the device, then switches to NORMAL mode.\n\n### Container Lifecycle\n\n1. The Docker container is created with device mappings for both Flipper and ST-Link\n2. The container's entrypoint script handles:\n   - Firmware flashing (in REPAIR mode)\n   - GitHub runner registration and startup (in NORMAL mode)\n3. When a job completes, the container exits with code 0, triggering a restart in REPAIR mode\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Device not found**\n   - Check USB connections\n   - Verify ST-Link and Flipper IDs\n   - Check `/run/` directory for environment files\n\n2. **Container fails to start**\n   - Check Docker service status: `systemctl status docker`\n   - Verify configuration file existence and permissions\n   - Check for USB device conflicts: `lsusb -t`\n\n3. **Runner doesn't register with GitHub**\n   - Check GitHub access token permissions\n   - Verify network connectivity\n   - Check the container logs for registration errors\n\n4. **Firmware flashing fails**\n   - Verify ST-Link connection and permissions\n   - Check if the Docker container has access to the device\n   - Review container logs for specific errors\n\n### Diagnostic Commands\n\n```bash\n# Check runner service status\nsystemctl status github-runner-flip-\u003cFLIPPER_SERIAL\u003e\n\n# View runner logs\njournalctl -u github-runner-flip-\u003cFLIPPER_SERIAL\u003e -f\n\n# Check monitor service status\nsystemctl status github-runner-monitor\n\n# View monitor logs\ntail -f /var/log/github-runner-metrics.log\n\n# Check Docker container status\ndocker ps -a | grep \u003cFLIPPER_SERIAL\u003e\n\n# Check udev rules \ncat /etc/udev/rules.d/99-udev-flipper-zero.rules\n\n# Verify device symlinks\nls -la /dev/flipper/\n```\n\n### Restarting Services\n\nIf you need to restart:\n\n```bash\n# Restart a specific runner\nsystemctl restart github-runner-flip-\u003cFLIPPER_SERIAL\u003e\n\n# Restart the monitoring service\nsystemctl restart github-runner-monitor\n\n# Restart all runner services\nsystemctl restart 'github-runner-flip-*'\n```\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflipperdevices%2Fflipper-github-runner-docker-qa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflipperdevices%2Fflipper-github-runner-docker-qa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflipperdevices%2Fflipper-github-runner-docker-qa/lists"}