https://github.com/nicolasarnouts/clipfix
Record a bug, get coding-agent context: annotated keyframes + a brief for Cursor, Claude Code & Codex. Open-source macOS app, powered by Gemini.
https://github.com/nicolasarnouts/clipfix
ai claude-code coding-agents cursor developer-tools devtools gemini llm macos open-source productivity rust screen-recording screencapturekit tauri
Last synced: 7 days ago
JSON representation
Record a bug, get coding-agent context: annotated keyframes + a brief for Cursor, Claude Code & Codex. Open-source macOS app, powered by Gemini.
- Host: GitHub
- URL: https://github.com/nicolasarnouts/clipfix
- Owner: NicolasArnouts
- License: mit
- Created: 2026-05-27T16:03:18.000Z (about 1 month ago)
- Default Branch: develop
- Last Pushed: 2026-06-20T16:28:16.000Z (10 days ago)
- Last Synced: 2026-06-20T18:17:00.906Z (10 days ago)
- Topics: ai, claude-code, coding-agents, cursor, developer-tools, devtools, gemini, llm, macos, open-source, productivity, rust, screen-recording, screencapturekit, tauri
- Language: Rust
- Homepage: https://clipfix.dev
- Size: 23.2 MB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project
README
ClipFix
Record the bug. Paste the fix context.
Website
·
Download
·
Install
·
CLI
·
Report a bug
---
**ClipFix** is an open-source macOS app that turns a short screen recording into context your **coding agent** can actually read. Record a bug — or drop in a clip — and ClipFix watches it once with **Google Gemini**, then hands back annotated keyframes, a written brief, and a paste-ready prompt for Cursor, Claude Code, Codex, or any multimodal agent. Built with **Tauri** and **Rust**; your recordings and API key never leave your Mac except to call Gemini.
What your agent gets — annotated keyframes stitched into one image, paired with a written brief. (a real, unedited run)
## Why
Bug reports decay between brain and keyboard. By the time you've typed "the dropdown closes when I scroll," you've forgotten the three other things you noticed in the recording. Pasting raw video into an agent works, but each agent treats video differently and the signal-to-noise is poor.
ClipFix sits between the recording and the agent. It watches the clip once, on your behalf, and produces something paste-ready: a short written brief, the handful of frames that matter (annotated with bounding boxes around the broken regions), and a single stitched image your agent can glance at to orient itself.
No account. No telemetry. No background daemon. The output is plain files on your disk; the prompt is plain text pointing at them.
## How it works
```
record ─or─ drop a clip
│
▼ one Gemini 3.5 Flash call (structured output, MEDIA_RESOLUTION_MEDIUM)
context pack ──► ffmpeg pulls the chosen keyframes, boxes + captions drawn on
│
▼
~/.clipfix/sessions/-/
│
▼ a smart prompt waiting on your clipboard
paste into Cursor · Claude Code · Codex · anything multimodal
```
1. **Capture.** Hit record (⌃⌘R or the tray), reproduce the bug, stop. ClipFix hides its own window and records the screen with ScreenCaptureKit. Or just drag a `.mov`/`.mp4` onto the window.
2. **Analyze.** One Gemini pass picks the load-bearing frames, writes a brief, and marks the broken regions with bounding boxes. An optional second pass tightens those boxes.
3. **Review.** Walk the keyframes with timestamps and captions, scrub the source recording, read the brief.
4. **Hand off.** Click **Copy prompt** — a structured, paste-ready prompt (pointing at the keyframes and brief) is on your clipboard for your coding agent.
## Screenshots
Drop a clip — or record one
Watch it read the recording
Keyframes, annotated with the fix
One stitched image for your agent
Replay, copy, hand off
Record from the menu bar
## Install
### Desktop app (macOS)
> **Requirements:** macOS 15 (Sequoia) or later · a free [Gemini API key](#set-up-your-gemini-api-key). ffmpeg is **bundled** — nothing else to install.
**Homebrew (recommended)**
```sh
brew install --cask nicolasarnouts/clipfix/clipfix
```
**Direct download**
Grab the latest `.dmg` from the [**Releases** page](https://github.com/NicolasArnouts/ClipFix/releases/latest), open it, and drag **ClipFix** into `/Applications`.
| Your Mac | Download |
|---|---|
| Apple Silicon (M1–M4) | `ClipFix_0.1.0_aarch64.dmg` |
| Intel | `ClipFix_0.1.0_x64.dmg` |
> Not sure which? → Apple menu → **About This Mac**: "Apple M…" is Apple Silicon, "Intel" is x64.
The app is **signed and notarized**, so it opens straight away — no Gatekeeper warning. It also **auto-updates in the background**, so you install once. *(If an older macOS ever blocks it, right-click the app → **Open** → **Open**; only needed once.)*
### Set up your Gemini API key
ClipFix runs one Gemini Flash call per clip, with your own key.
1. Get a **free** key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey).
2. Open ClipFix → **Settings → API key** → paste it → **Save**.
Your key is stored in a private, owner-only file on your Mac and only ever leaves your machine to call the Gemini API directly. The free tier covers typical daily use; see [Google's pricing](https://ai.google.dev/pricing) if you hit a limit. No ClipFix account, ever.
### Command-line (CLI)
Prefer the terminal? The same engine ships as a self-contained `clipfix` binary.
```sh
git clone https://github.com/NicolasArnouts/ClipFix.git
cd ClipFix
cargo install --path crates/clipfix-cli
```
```sh
export GEMINI_API_KEY=your_key_here
clipfix analyze recording.mp4 # smart prompt lands on your clipboard
clipfix analyze recording.mp4 --refine-boxes # second pass to tighten boxes
clipfix verify-key # check your key is valid
```
> The CLI uses the `ffmpeg`/`ffprobe` on your `PATH` (`brew install ffmpeg`). It writes to a project-local `.clipfix/` — add `.clipfix/` to your global gitignore: `echo '.clipfix/' >> ~/.config/git/ignore`.
## Features
| | |
|---|---|
| 🎬 **Record in-app** | Capture the screen with ScreenCaptureKit; ClipFix hides its own window first. Global hotkey **⌃⌘R**, menu-bar tray, live timer + mic meter, auto-stop safety cap. |
| 📥 **Or drag a clip in** | Drop `.mp4` / `.mov` / `.webm` / `.mkv` / `.m4v` / `.avi` — same pipeline. |
| 🧠 **One smart pass** | A single Gemini 3.5 Flash call picks the keyframes, writes the brief, and boxes the broken regions. Optional refine pass sharpens the boxes. |
| ✂️ **Review & trim** | Preview every clip before analysis — trim, re-record, or discard. Long clips are gated automatically. |
| 🗂️ **One library** | Every capture is kept under `~/.clipfix/`, with **Copy prompt** and **Reveal in Finder** on each. |
| 🔁 **Auto-updates** | Signed, notarized, and self-updating via a minisign-verified release feed. |
| 🔒 **Private by default** | No account, no telemetry, no daemon. Key stored locally on your Mac; only the clip you analyze is sent to Gemini. |
| ⌨️ **CLI too** | The same Rust engine as a scriptable `clipfix analyze … --json` binary. |
## What's in a pack
Every run writes a session directory under `~/.clipfix/sessions/` and points `~/.clipfix/latest` at the newest one:
- **`prompt.md`** — the written brief: what's happening, what's likely wrong, what to look at.
- **`keyframes/`** — the frames Gemini chose, each annotated with bounding boxes and a one-line caption (`keyframes-clean/` holds the box-only versions).
- **`context-pack.png`** — all the keyframes stitched into one image, for agents that prefer a single attachment.
- **`manifest.json`** — provenance: video SHA-256, model id, prompt version, latency, cost, validation status.
The **smart prompt** is the glue — it tells the receiving agent where to look:
> Please fix the UI issue described in `.clipfix/latest/prompt.md`. Inspect the keyframes in `.clipfix/latest/keyframes/` for the visual context, and look at `.clipfix/latest/context-pack.png` for a stitched overview. Use that context to identify and implement the smallest correct fix.
## See a real run
[`examples/`](examples/) holds two unedited, committed ClipFix runs — frozen output you can browse without installing anything.
- [**`examples/login-bug/`**](examples/login-bug/) — a login form rejects the correct password; the keyframes catch a hardcoded `value` on the password `` in DevTools. *(`$0.018`, ~105 s, `gemini-3.5-flash`.)*
- [**`examples/connection-bug/`**](examples/connection-bug/) — multiple terminal shells stall on "Connecting…"; the frames show the partial-connection state.
## Privacy
ClipFix is built to be boring about your data, on purpose.
- **No account, no sign-up, no telemetry.** Nothing phones home.
- **Your recordings stay on your Mac** — plain files under `~/.clipfix/`.
- **Only the clip you analyze is sent to Gemini**, over a direct call with *your* key. ClipFix has no server in the middle.
- **Your API key is stored in a private, owner-only file on your Mac** (`~/Library/Application Support/com.clipfix.desktop/`) — never synced, never sent anywhere except Gemini.
## Under the hood
ClipFix is one Rust analysis engine with thin front ends:
- **Tauri v2** desktop app (Rust backend + a vanilla TypeScript/Vite frontend) that calls the engine **in-process** — no subprocess, no PATH lookup.
- **`clipfix-core`** — the engine: transcode → one **Gemini 3.5 Flash** call with structured JSON output → frame extraction → bounding-box annotation (rendered with `image`/`imageproc`, EB Garamond) → bundle.
- **ffmpeg + ffprobe** ship as **signed sidecar binaries** inside the app, resolved by absolute path at runtime — users never install them.
- **API key** in a `0600` file under the app-data dir; recording via **ScreenCaptureKit** (`SCRecordingOutput`, direct-to-MP4).
## Project layout
- **`apps/desktop`** — the macOS app (Tauri). The flagship.
- **`crates/clipfix-core`** — the analysis engine; the single source of truth.
- **`crates/clipfix-cli`** — the `clipfix` CLI, a thin wrapper over the core.
- **`python/clipfix`** — a prompt/quality experimentation lab. *Not* the shipped product.
## Roadmap
- **Windows** — recording + a signed installer are designed and parked behind a build/verify environment ([#87](https://github.com/NicolasArnouts/ClipFix/issues/87)).
- **LGPL ffmpeg** — swap the bundled GPL ffmpeg for an LGPL build ([#132](https://github.com/NicolasArnouts/ClipFix/issues/132)).
- **Eval harness** — v0 ships at gut-feel quality; metrics come next.
## Contributing
Issues and PRs welcome. PRs target the **`develop`** branch. If a pack reads well, the design is working — if it doesn't, please [open an issue](https://github.com/NicolasArnouts/ClipFix/issues).
## License
[MIT](LICENSE) © 2026 Nicolas Arnouts.
ClipFix bundles **ffmpeg/ffprobe** (GPL-2.0-or-later) as separate binaries — see [THIRD-PARTY-LICENSES.md](THIRD-PARTY-LICENSES.md) for the full notices and corresponding-source links.