{"id":45535028,"url":"https://github.com/agentralabs/agentic-vision","last_synced_at":"2026-02-25T03:01:07.037Z","repository":{"id":339346654,"uuid":"1161231080","full_name":"agentralabs/agentic-vision","owner":"agentralabs","description":"Persistent visual memory for AI agents — capture screenshots, embed with CLIP ViT-B/32, compare, recall. MCP server + Rust core library.","archived":false,"fork":false,"pushed_at":"2026-02-23T03:38:32.000Z","size":2238,"stargazers_count":2,"open_issues_count":12,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-23T04:38:58.434Z","etag":null,"topics":["agentic","ai-agents","claude","clip","computer-vision","cursor","embeddings","image-similarity","mcp","model-context-protocol","onnx","rust","screenshots","vision","visual-memory"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/agentic-vision","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agentralabs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["xeo-labs"]}},"created_at":"2026-02-18T21:59:16.000Z","updated_at":"2026-02-23T03:38:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"ebcdc633-fb9c-4a09-9a13-a4e2937bd434","html_url":"https://github.com/agentralabs/agentic-vision","commit_stats":null,"previous_names":["agentic-revolution/agentic-vision","xeo-labs/agentic-vision","agentralabs/agentic-vision"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/agentralabs/agentic-vision","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agentralabs%2Fagentic-vision","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agentralabs%2Fagentic-vision/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agentralabs%2Fagentic-vision/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agentralabs%2Fagentic-vision/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agentralabs","download_url":"https://codeload.github.com/agentralabs/agentic-vision/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agentralabs%2Fagentic-vision/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29768304,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-24T01:40:24.820Z","status":"online","status_checked_at":"2026-02-24T02:00:07.497Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic","ai-agents","claude","clip","computer-vision","cursor","embeddings","image-similarity","mcp","model-context-protocol","onnx","rust","screenshots","vision","visual-memory"],"created_at":"2026-02-23T01:37:56.989Z","updated_at":"2026-02-24T02:01:04.705Z","avatar_url":"https://github.com/agentralabs.png","language":"Rust","funding_links":["https://github.com/sponsors/xeo-labs"],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/github-hero-pane.svg\" alt=\"AgenticVision hero pane\" width=\"980\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://crates.io/crates/agentic-vision\"\u003e\u003cimg src=\"https://img.shields.io/badge/cargo_install-agentic--vision-F59E0B?style=for-the-badge\u0026logo=rust\u0026logoColor=white\" alt=\"cargo install\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://crates.io/crates/agentic-vision-mcp\"\u003e\u003cimg src=\"https://img.shields.io/badge/cargo_install-agentic--vision--mcp-3B82F6?style=for-the-badge\u0026logo=rust\u0026logoColor=white\" alt=\"cargo install mcp\"\u003e\u003c/a\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-MIT-8B5CF6?style=for-the-badge\" alt=\"MIT License\"\u003e\u003c/a\u003e\n  \u003ca href=\"publication/paper-ii-agentic-vision-mcp/agentic-vision-mcp-paper.pdf\"\u003e\u003cimg src=\"https://img.shields.io/badge/Research-Paper-EF4444?style=for-the-badge\" alt=\"Research Paper\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#quickstart\"\u003eQuickstart\u003c/a\u003e · \u003ca href=\"#why-agenticvision\"\u003eWhy\u003c/a\u003e · \u003ca href=\"#benchmarks\"\u003eBenchmarks\u003c/a\u003e · \u003ca href=\"#how-it-works\"\u003eHow It Works\u003c/a\u003e · \u003ca href=\"#install\"\u003eInstall\u003c/a\u003e · \u003ca href=\"INSTALL.md\"\u003eFull Install Guide\u003c/a\u003e · \u003ca href=\"publication/paper-ii-agentic-vision-mcp/agentic-vision-mcp-paper.pdf\"\u003ePaper\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## AI agents can't see across sessions.\n\nYour agent takes a screenshot, analyzes it, and forgets. Next session — blank slate. It can't compare what a page looks like now versus yesterday. It can't recall what the error dialog said three conversations ago. It can't search its own visual history.\n\nText-based memory exists. Visual memory doesn't — until now.\n\n**AgenticVision** gives AI agents persistent visual memory. Capture images, embed them with CLIP ViT-B/32, store them in a compact binary format, and query them by similarity, time, or description. Every capture is a first-class MCP resource that any LLM can access.\n\n```bash\ncargo install agentic-vision-mcp\n```\n\nOne binary. 11 MCP tools. Persistent `.avis` files. Works with Claude Desktop, VS Code, Cursor, Windsurf, and any MCP-compatible client.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/github-terminal-pane.svg\" alt=\"AgenticVision terminal pane\" width=\"980\"\u003e\n\u003c/p\u003e\n\n---\n\n\u003ca name=\"benchmarks\"\u003e\u003c/a\u003e\n\n## Benchmarks\n\nRust core. CLIP ViT-B/32 via ONNX Runtime. Binary `.avis` format. Real numbers from `cargo test --release`:\n\n| Operation | Time | Notes |\n|:---|---:|:---|\n| Image capture (file → embed → store) | **47 ms** | CLIP ViT-B/32, 512-dim |\n| Similarity search (top-5) | **1-2 ms** | Brute-force cosine, f64 precision |\n| Visual diff (pixel-level) | **\u003c1 ms** | 8×8 grid region detection |\n| MCP tool round-trip | **7.2 ms** | Including process startup (~6.1 ms) |\n| Storage per capture | **~4.26 KB** | Embedding + JPEG thumbnail |\n| Capacity per GB | **~250K** | Observations |\n\n\u003e All benchmarks on Apple M4, macOS 26.2, Rust 1.90.0 `--release`. ONNX Runtime for CLIP inference. Fallback mode available when ONNX model is not present.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/vision-runtime-flow-agentra.svg\" alt=\"AgenticVision runtime flow from capture to embedding, storage, query, and MCP response\" width=\"980\"\u003e\n\u003c/p\u003e\n\n---\n\n\u003ca name=\"why-agenticvision\"\u003e\u003c/a\u003e\n\n## Why AgenticVision\n\n**Agents need visual continuity.** A debugging agent should remember what the UI looked like before and after a code change. A monitoring agent should detect visual regressions. A research agent should build a visual knowledge base over time.\n\n**Capture once, query forever.** Every image is embedded into a 512-dimensional CLIP vector and stored with its JPEG thumbnail, timestamp, and description. Query by cosine similarity, time range, or text search — in milliseconds.\n\n**Binary format, not a database.** The `.avis` file is a single portable binary — 64-byte header, JSON payload, JPEG thumbnails. Copy it, share it, back it up. No server, no database, no dependencies.\n\n**Works with every MCP client.** AgenticVision-MCP exposes 11 tools, 6 resources, and 4 prompts via the Model Context Protocol. Any LLM that speaks MCP gains visual memory automatically.\n\n**Links to AgenticMemory.** The `vision_link` tool connects visual captures to [AgenticMemory](https://github.com/agentralabs/agentic-memory) cognitive graph nodes — bridging what an agent *sees* with what it *knows*.\n\n---\n\n\u003ca name=\"how-it-works\"\u003e\u003c/a\u003e\n\n## How It Works\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/architecture-agentra-v2.svg\" alt=\"AgenticVision architecture map with MCP clients, transport, tools, resources, prompts, and storage\" width=\"980\"\u003e\n\u003c/p\u003e\n\n1. **Capture** — `vision_capture` accepts images from files, base64, screenshots, or the system clipboard. Each image is resized, embedded via CLIP ViT-B/32 into a 512-dimensional vector, compressed to JPEG thumbnail, and stored in the `.avis` binary file. Screenshots support optional region capture; clipboard reads the current image from the OS clipboard.\n\n2. **Query** — `vision_query` retrieves captures by time range, description, recency, and quality constraints (`min_quality`, `sort_by`). Results include capture metadata, quality scores, thumbnails, and similarity scores.\n\n3. **Compare** — `vision_compare` places two captures side-by-side for LLM analysis. `vision_diff` performs pixel-level differencing with 8×8 grid region detection to identify exactly what changed.\n\n4. **Link** — `vision_link` connects captures to AgenticMemory nodes, bridging visual observations with the agent's cognitive graph. An agent can recall \"what did the UI look like when I made that decision?\"\n\n**The `.avis` binary format** uses a 64-byte fixed header (magic `0x41564953`, version, counts, timestamps) followed by a JSON payload containing captures with embedded JPEG thumbnails and 512-dim float vectors. Single-file, portable, no external dependencies.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eMCP surface area\u003c/strong\u003e\u003c/summary\u003e\n\n\u003cbr\u003e\n\n**11 Tools:**\n\n| Tool | Description |\n|:---|:---|\n| `vision_capture` | Capture and embed an image (file, base64, screenshot, clipboard), with metadata redaction and quality scoring |\n| `vision_compare` | Side-by-side comparison of two captures |\n| `vision_query` | Query captures by time, description, recency |\n| `vision_ocr` | Extract text from a captured image |\n| `vision_similar` | Find visually similar captures (cosine similarity) |\n| `vision_track` | Track visual changes to a target over time |\n| `vision_diff` | Pixel-level diff between two captures |\n| `vision_health` | Quality + staleness + memory-link coverage summary |\n| `vision_link` | Link a capture to an AgenticMemory node |\n| `session_start` | Begin a named observation session |\n| `session_end` | End the current session |\n\n**6 Resources:**\n\n| URI | Description |\n|:---|:---|\n| `avis://capture/{id}` | Single capture with metadata and thumbnail |\n| `avis://session/{id}` | All captures in a session |\n| `avis://timeline/{start}/{end}` | Captures within a time range |\n| `avis://similar/{id}` | Visually similar captures |\n| `avis://stats` | Storage statistics and counts |\n| `avis://recent` | Most recent captures |\n\n**4 Prompts:**\n\n| Prompt | Description |\n|:---|:---|\n| `observe` | Guided visual observation workflow |\n| `compare` | Structured comparison between captures |\n| `track` | Change tracking over time |\n| `describe` | Detailed image description |\n\n\u003c/details\u003e\n\n---\n\n\u003ca name=\"install\"\u003e\u003c/a\u003e\n\n## Install\n\n**One-liner** (desktop profile, backwards-compatible):\n```bash\ncurl -fsSL https://agentralabs.tech/install/vision | bash\n```\n\n**Environment profiles** (one command per environment):\n```bash\n# Desktop MCP clients (auto-merge Claude Desktop + Claude Code when detected)\ncurl -fsSL https://agentralabs.tech/install/vision/desktop | bash\n\n# Terminal-only (no desktop config writes)\ncurl -fsSL https://agentralabs.tech/install/vision/terminal | bash\n\n# Remote/server hosts (no desktop config writes)\ncurl -fsSL https://agentralabs.tech/install/vision/server | bash\n```\n\n| Channel | Command | Result |\n|:---|:---|:---|\n| GitHub installer (official) | `curl -fsSL https://agentralabs.tech/install/vision \\| bash` | Installs release binaries when available, otherwise source fallback; merges MCP config |\n| GitHub installer (desktop profile) | `curl -fsSL https://agentralabs.tech/install/vision/desktop \\| bash` | Explicit desktop profile behavior |\n| GitHub installer (terminal profile) | `curl -fsSL https://agentralabs.tech/install/vision/terminal \\| bash` | Installs binaries only; no desktop config writes |\n| GitHub installer (server profile) | `curl -fsSL https://agentralabs.tech/install/vision/server \\| bash` | Installs binaries only; server-safe behavior |\n| crates.io + Cargo deps (official) | `cargo install agentic-vision-mcp` + `cargo add agentic-vision` | Installs MCP server binary and adds the core library crate to your project |\n\n### Server auth and artifact sync\n\nFor cloud/server runtime:\n\n```bash\nexport AGENTIC_TOKEN=\"$(openssl rand -hex 32)\"\n```\n\nAll MCP clients must send `Authorization: Bearer \u003csame-token\u003e`.\nIf `.avis/.amem/.acb` files are on another machine, sync them to the server first.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/architecture-agentra.svg\" alt=\"AgenticVision architecture in Agentra Labs design system\" width=\"980\"\u003e\n\u003c/p\u003e\n\n**MCP Server** (for Claude Desktop, VS Code, Cursor, Windsurf):\n```bash\ncargo install agentic-vision-mcp\n```\n\n**Core library** (for Rust projects):\n```bash\ncargo add agentic-vision\n```\n\n**Configure Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):\n```json\n{\n  \"mcpServers\": {\n    \"vision\": {\n      \"command\": \"agentic-vision-mcp\",\n      \"args\": [\"--vision\", \"~/.vision.avis\", \"serve\"]\n    }\n  }\n}\n```\n\n\u003e See [INSTALL.md](INSTALL.md) for full installation guide, VS Code / Cursor configuration, build from source, and troubleshooting.\n\n\u003e **Do not use `/tmp` for vision files** — macOS and Linux clear this directory periodically. Use `~/.vision.avis` for persistent storage.\n\n## Deployment Model\n\n- **Standalone by default:** AgenticVision is independently installable and operable. Integration with AgenticMemory or AgenticCodebase is optional, never required.\n- **Autonomic operations by default:** daemon/runtime maintenance uses safe profile-based defaults with cache hygiene, migration safeguards, and health-ledger snapshots.\n\n| Area | Default behavior | Controls |\n|:---|:---|:---|\n| Autonomic profile | Conservative local-first posture | `CORTEX_AUTONOMIC_PROFILE=desktop|cloud|aggressive` |\n| Cache + registry maintenance | Periodic expiry cleanup and registry GC | `CORTEX_MAINTENANCE_TICK_SECS`, `CORTEX_REGISTRY_GC_EVERY_TICKS`, `CORTEX_REGISTRY_GC_KEEP_DELTAS` |\n| Storage migration | Policy-gated with checkpointed auto-safe path | `CORTEX_STORAGE_MIGRATION_POLICY=auto-safe|strict|off` |\n| Storage budget policy | 20-year projection + capture rollup under pressure | `CORTEX_STORAGE_BUDGET_MODE=auto-rollup|warn|off`, `CORTEX_STORAGE_BUDGET_BYTES`, `CORTEX_STORAGE_BUDGET_HORIZON_YEARS`, `CORTEX_STORAGE_BUDGET_TARGET_FRACTION` |\n| Maintenance throttling | SLA-aware under sustained cache pressure | `CORTEX_SLA_MAX_CACHE_ENTRIES_BEFORE_GC_THROTTLE` |\n| Health ledger | Periodic operational snapshots (default: `~/.agentra/health-ledger`) | `CORTEX_HEALTH_LEDGER_DIR`, `AGENTRA_HEALTH_LEDGER_DIR`, `CORTEX_HEALTH_LEDGER_EMIT_SECS` |\n\n---\n\n\u003ca name=\"quickstart\"\u003e\u003c/a\u003e\n\n## Quickstart\n\n### MCP (Claude Desktop, VS Code, Cursor)\n\nAfter configuring the MCP server (see [Install](#install)), ask your agent:\n\n\u003e \"Take a screenshot and remember it.\"\n\nThe LLM calls `vision_capture` automatically. Then later:\n\n\u003e \"What did the screen look like earlier?\"\n\nThe LLM calls `vision_query` to retrieve and display past captures.\n\n### Rust API\n\n```rust\nuse agentic_vision::{VisionStore, CaptureSource};\n\nlet mut store = VisionStore::open(\"observations.avis\")?;\n\n// Capture from file\nlet id = store.capture(\n    CaptureSource::File(\"screenshot.png\"),\n    \"Homepage after deploy\"\n)?;\n\n// Find similar\nlet matches = store.similar(id, 5)?;\nfor m in matches {\n    println!(\"  {} (similarity: {:.3})\", m.description, m.score);\n}\n```\n\n---\n\n## Validation\n\n| Suite | Tests | Notes |\n|:---|---:|:---|\n| Rust core (`agentic-vision`) | **38** | Unit + integration (includes screenshot/clipboard) |\n| Python SDK tests | **47** | Edge cases, format validation |\n| MCP integration suite | **3** | Python → Rust stdio transport |\n| Multi-agent suite | **3** | Shared file, vision-memory linking, rapid handoff |\n| **Total** | **91** | All passing |\n\n**Two research papers:**\n- [Paper I: Cortex — Web Cartography (10 pages, 8 figures, 13 tables)](publication/paper-i-cortex/cortex-paper.pdf)\n- [Paper II: AgenticVision-MCP — Persistent Visual Memory via MCP (8 pages, 4 figures, 7 tables)](publication/paper-ii-agentic-vision-mcp/agentic-vision-mcp-paper.pdf)\n\n---\n\n## Repository Structure\n\nThis is a Cargo workspace monorepo containing the core library and MCP server.\n\n```\nagentic-vision/\n├── Cargo.toml                    # Workspace root\n├── crates/\n│   ├── agentic-vision/           # Core library (crates.io: agentic-vision v0.1.0)\n│   └── agentic-vision-mcp/       # MCP server (crates.io: agentic-vision-mcp v0.1.0)\n├── tests/                        # Integration tests (Python → Rust, multi-agent)\n├── models/                       # ONNX model directory (CLIP ViT-B/32)\n├── publication/                  # Research papers (I, II)\n├── assets/                       # SVG diagrams and visuals\n└── docs/                         # Guides and reference\n```\n\n### Running Tests\n\n```bash\n# All workspace tests (unit + integration)\ncargo test --workspace\n\n# Core library only\ncargo test -p agentic-vision\n\n# MCP server only\ncargo test -p agentic-vision-mcp\n\n# Python integration tests\npython tests/integration/test_mcp_clients.py\npython tests/integration/test_multi_agent.py\n```\n\n### MCP Server Quick Start\n\n```bash\ncargo install agentic-vision-mcp\n```\n\nConfigure Claude Desktop (`~/Library/Application Support/Claude/claude_desktop_config.json`):\n\n```json\n{\n  \"mcpServers\": {\n    \"vision\": {\n      \"command\": \"agentic-vision-mcp\",\n      \"args\": [\"--vision\", \"~/.vision.avis\", \"serve\"]\n    }\n  }\n}\n```\n\n`agentic-vision-mcp` supports both line-delimited JSON-RPC and `Content-Length` framed MCP stdio messages.\n\n---\n\n## Roadmap: v0.2.0 — Remote Server Support\n\nThe next release is planned to add HTTP/SSE transport for remote deployments. Track progress in [#2](https://github.com/agentralabs/agentic-vision/issues/2).\n\n| Feature | Status |\n|:---|:---|\n| `--token` bearer auth | Planned |\n| `--multi-tenant` per-user vision files | Planned |\n| `/health` endpoint | Planned |\n| `--tls-cert` / `--tls-key` native HTTPS | Planned |\n| OCR with Tesseract (`--features ocr`) | Planned |\n| Clipboard TIFF fix | Planned |\n| `delete` / `export` / `compact` CLI commands | Planned |\n| Docker image + compose | Planned |\n| Remote deployment docs | Planned |\n\nPlanned CLI shape (not available in current release):\n\n```text\nagentic-vision-mcp serve-http --port 8081 --token \"\u003ctoken\u003e\"\nagentic-vision-mcp serve-http --multi-tenant --data-dir /data/users --port 8081 --token \"\u003ctoken\u003e\"\n```\n\n---\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md). The fastest ways to help:\n\n1. **Try it** and [file issues](https://github.com/agentralabs/agentic-vision/issues)\n2. **Add an MCP tool** — extend the visual memory surface\n3. **Write an example** — show a real use case\n4. **Improve docs** — every clarification helps someone\n\n---\n\n\u003cp align=\"center\"\u003e\n  \u003csub\u003eBuilt by \u003ca href=\"https://github.com/agentralabs\"\u003e\u003cstrong\u003eAgentra Labs\u003c/strong\u003e\u003c/a\u003e\u003c/sub\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagentralabs%2Fagentic-vision","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagentralabs%2Fagentic-vision","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagentralabs%2Fagentic-vision/lists"}