Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/vim-zz/flight_recall

converting short MP4 videos into a detailed event-by-event text description
https://github.com/vim-zz/flight_recall

Last synced: about 1 month ago
JSON representation

converting short MP4 videos into a detailed event-by-event text description

Awesome Lists containing this project

README 

          
# Flight Recall



`Flight Recall` is a macOS app for going through FPV videos, replaying relevant parts by maneuver type, and reviewing timeline-style visual events. It is visual-first: it works from video frames and OSD text, not audio transcription.



> [!NOTE]

> Personal project. I have a lot of experimental coding experience, but I did not write the code in this repository myself. The code here was generated with AI tools, then reviewed, tested, and iterated on by me.



## Main Value



- Go through FPV videos and quickly replay relevant parts based on maneuver types

- Review timeline-style events for each video

- Build private `Cuts` by collecting snippets from `Video Library` and `Achievements`

- See library-wide timelines and stats across runs

- Track telemetry such as battery levels and reception levels over time

- Use automatic AI analysis of the video footage together with OSD text

- Refine and steer the analysis with user-defined instructions inside the app

- Evaluate prompt changes outside the app against manually labeled maneuver events



| Video Library | Maneuvers |

|---|---|

| Flight Recall Video Library | Flight Recall Maneuvers |

| Cuts | Timeline |

| Flight Recall Cuts | Flight Recall Video Timeline |



## Install



Download the latest macOS zip from the [GitHub Releases page](https://github.com/vim-zz/flight_recall/releases), unzip it, then move `Flight Recall.app` to `/Applications`.



> [!IMPORTANT]

> Because the GitHub-built app is currently not signed or notarized, macOS may show a warning that the app is "corrupted" or say it should be moved to the Trash. If that happens, run:



```bash

xattr -dr com.apple.quarantine "/Applications/Flight Recall.app"

```



## Prerequisites



### To use the desktop app from source



- macOS

- Rust toolchain installed with [Rustup](https://rustup.rs/)

- `ffmpeg` and `ffprobe` available in `PATH`

- Gemini API key



Install `ffmpeg` with Homebrew:



```bash

brew install ffmpeg

```



Set your Gemini key in one of these ways:



- In the app `Settings` tab after launch

- `GEMINI_API_KEY`

- `GOOGLE_API_KEY`



Example:



```bash

export GEMINI_API_KEY="your_key_here"

```



### What you should know before setup



- The desktop app is macOS-only.

- The current desktop flow assumes `ffmpeg` and `ffprobe` are installed.

- The app stores local data under `~/Library/Application Support/fltrcl/`.

- The API key saved in the app is used first. If it is empty, the app falls back to `GEMINI_API_KEY` or `GOOGLE_API_KEY`.

- When a desktop API key is configured, Settings shows only the first 4 and last 4 characters by default, with the middle masked by asterisks; use `Edit` to change it.

- The project is AI-generated and AI-maintained, but changes still need human verification.



## App Usage



### Build the app from source



```bash

# one-time

cargo install cargo-bundle



# build

./scripts/bundle-macos-desktop.sh



# install or update

osascript -e 'tell application id "com.1000ants.fltrcl" to quit' 2>/dev/null || true

rm -rf "/Applications/Flight Recall.app"

cp -fr "target/release/bundle/osx/Flight Recall.app" "/Applications/Flight Recall.app"

open "/Applications/Flight Recall.app"

```



### Use the app



1. Launch `Flight Recall`.

2. Open `Settings`.

3. Add your Gemini API key if it is not already set in the environment.

4. Pick a video or a folder to scan.

5. Wait for analysis to finish, then review the results in `Video Library`, `Achievements`, `Cuts`, and `Tools`. Timeline strips now keep the current visible window stable during manual selection and only shift when replay advances to the next off-screen clip.



To clean noisy drone filter names, open `Tools` and use `Manage Drone Names`. Select or create a drone name, edit the preferred label, and assign detected names into it. Changes save automatically. Existing drone names can be renamed, adjusted by adding or removing detected names, or removed entirely. The Tools drone-name editor includes undo and redo buttons for the current session's assigned-name changes and drone-name removals. `Auto-Group` saves AI-suggested drone groups and marks them with AI confidence.



The app is currently branded as `Flight Recall`, while some internal paths, crate names, commands, env vars, and compatibility identifiers still use `fltrcl`.



### Local Storage



Default desktop paths:



- `~/Library/Application Support/fltrcl/.env`

- `~/Library/Application Support/fltrcl/desktop-settings.json`

- `~/Library/Application Support/fltrcl/db/fltrcl-local.db`

- `~/Library/Application Support/fltrcl/artifacts/`

- `~/Library/Application Support/fltrcl/logs/desktop.log`



Notes:



- Finder-launched scans still try common system and Homebrew locations for `ffmpeg` and `ffprobe`.

- If a scan path came from a mounted macOS network volume, the app can try to remount it when needed.



## Developer



This repository uses [Spec Kit](https://github.com/github/spec-kit) for spec-driven development. For new tasks and features, follow the repository's required spec-first workflow and include the relevant spec files in your PR.



```bash

cargo run -p fltrcl-desktop

cargo build --workspace

cargo test --workspace

```



### Supervised Prompt Refinement



Prompt learning no longer runs inside the desktop app. Use the repo-local skill and evaluator script instead.



From the repository root:



```bash

rust-script scripts/supervised_prompt_refinement.rs --dry-run

rust-script scripts/supervised_prompt_refinement.rs

```



The evaluator selects videos automatically from the manually labeled population until a minimum labeled-event target is reached, runs the real `fltrcl` CLI on that set, scores matches / false positives / false negatives against manual labels, and writes a JSON experiment report under `reports/prompt-refinement/`.



For direct comparison, rerun on the same saved set with `--reuse-set-from `. For a new experiment, reshuffle with `--reshuffle-from `.



Use `--mode labeled-snippets` to trim each labeled event into a shorter snippet before evaluation, for example:



```bash

rust-script scripts/supervised_prompt_refinement.rs \

  --mode labeled-snippets \

  --snippet-before-secs 2 \

  --snippet-after-secs 10

```



The evaluator follows the desktop app scan preprocessing settings from `~/Library/Application Support/fltrcl/desktop-settings.json` when present. If that file is missing, it falls back to the app defaults of `10fps` and `320p`. You can still override either preprocessing value explicitly with `--downsample-fps` and `--downscale-height`.



If a labeled source video is missing from its original path, the evaluator falls back to the repo-local cache manifest at `local-cache/labeled-videos/manifest.csv` when a matching `video_id` entry is available. Use `--local-cache-manifest` to point at a different cache manifest.



The evaluator now checkpoints partial progress into the chosen `--output` report file after each run unit. If some snippets fail because of transient provider errors, rerun the exact same command with the same `--output` path and it will skip completed units and retry only the unfinished or failed ones until the report is complete. New reports also record the effective prompt text used for `gemini_video_event_prompt`, `osd_prompt`, and `user_rules`, plus a unified diff against the repo defaults so prompt changes stay auditable. Older reports created before this metadata existed cannot be backfilled perfectly once the repo defaults have changed.



The evaluator also supports an approval-first workflow:



1. Generate a proposal markdown before the run.

2. Review the proposed prompt diffs and comparison context.

3. Re-run without `--proposal-only` after approval.

4. Inspect the generated results markdown and detail tables.



Example proposal command:



```bash

rust-script scripts/supervised_prompt_refinement.rs \

  --proposal-only \

  --baseline-report reports/prompt-refinement/001-experiment.json \

  --previous-report reports/prompt-refinement/007-prompt-refinement.json \

  --prompt-overrides-json /tmp/candidate-prompts.json \

  --output reports/prompt-refinement/008-prompt-refinement.json

```



That writes:



- `reports/prompt-refinement/008-prompt-refinement-proposal.md`



After approval, rerun the same command without `--proposal-only`. The evaluator will write:



- `reports/prompt-refinement/008-prompt-refinement.json`

- `reports/prompt-refinement/008-prompt-refinement-results.md`

- `reports/prompt-refinement/008-prompt-refinement-details.md`

- `reports/prompt-refinement/008-prompt-refinement-details.csv`



### CLI



Single video:



```bash

cargo run -p fltrcl-cli -- --provider gemini ./sample.mp4

```



Structured JSON:



```bash

cargo run -p fltrcl-cli -- --provider gemini --output-format json ./sample.mp4

```



Directory scan:



```bash

cargo run -p fltrcl-cli -- --provider gemini ./videos

```



Repair legacy manual timeline inserts in an existing app DB:



```bash

cargo run -p fltrcl-cli -- \

  --database-url "$HOME/Library/Application Support/fltrcl/db/fltrcl-local.db" \

  --repair-legacy-manual-timeline-events

```



This repair upgrades pre-fix timeline rows to stable per-instance `manual_event_id` values so later edit/remove actions target only that row. When the stored artifact cleanly proves a legacy manual insert, the repair also backfills a durable replay row for future rebuilds.



Clean historical `scenery` events from local artifacts and rebuild the local DB indexes:



```bash

./scripts/cleanup-scenery-events.sh

```



To target the macOS app data paths explicitly:



```bash

./scripts/cleanup-scenery-events.sh \

  --db "$HOME/Library/Application Support/fltrcl/db/fltrcl-local.db" \

  --artifact-root "$HOME/Library/Application Support/fltrcl/artifacts"

```



Supported input extensions are `.mp4`, `.avi`, and `.ts`.



Recommended Gemini model options for this project:



- `gemini-3.1-flash-lite-preview` is the default app model, paired with the `050` recall-balanced prompt-refinement rules as the current best-performing starting point in this repo.

- `gemini-2.5-flash-lite` and `gemini-2.5-flash` remain available when you want non-preview 2.5 options.

- `gemini-2.5-pro` is the higher-quality option when you want more reasoning budget.

- The desktop reanalyze model pickers suggest only lite variants: `gemini-2.5-flash-lite`, `gemini-3.1-flash-lite-preview`, and `gemini-2.0-flash-lite`.

- Gemini 3 choices are preview models, so availability, behavior, pricing, and retirement timing can change faster than the stable 2.5 line.

- Older `gemini-2.0-flash` and `gemini-2.0-flash-lite` names are legacy/deprecating and should not be the first choice for new setups.



## Limits And Current Assumptions



- The project is centered on visual understanding of short FPV videos.

- Audio is not required and is not the primary source of understanding.

- Gemini is the main cloud provider path documented here.

- There is also local-provider support in the codebase, but the simplest setup path for most users is Gemini.



## Contributing



AI-first contributions are accepted, encouraged, and expected.



The standard is simple:



- Use AI if you want.

- Verify what it changed.

- Test the behavior you touched.

- Keep the change aligned with the project vision.

- The bar for acceptance is value added, not whether the first draft was written by AI or by hand.

- Pull requests are expected to follow the repository's spec-driven process.

- For new tasks and features, include the relevant spec files in the PR.

- In this AI-first workflow, issues should be followed by a PR that addresses them when possible.



See [CONTRIBUTING.md](CONTRIBUTING.md).



### Notes



Workspace layout:



```text

crates/fltrcl-cli        # CLI entrypoint

crates/fltrcl-processor  # processing pipeline, providers, OSD, persistence

crates/fltrcl-core       # shared trends/query service

crates/fltrcl-desktop    # GPUI macOS desktop app

specs/                   # feature specs

```



Useful environment variables:



| Variable | Purpose |

|---|---|

| `GEMINI_API_KEY` | Gemini API key |

| `GOOGLE_API_KEY` | Alternate Gemini key name |