{"id":44519683,"url":"https://github.com/michaelwoods/aegis-analyst","last_synced_at":"2026-02-13T17:34:41.458Z","repository":{"id":334571453,"uuid":"1141618368","full_name":"michaelwoods/aegis-analyst","owner":"michaelwoods","description":"An edge-optimized AI agent that synthesizes Frigate NVR detections and Home Assistant sensor context into semantic, narrative security digests using local LLMs.","archived":false,"fork":false,"pushed_at":"2026-01-26T02:05:33.000Z","size":406,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-26T09:09:58.136Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/michaelwoods.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-01-25T05:28:00.000Z","updated_at":"2026-01-26T02:05:36.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/michaelwoods/aegis-analyst","commit_stats":null,"previous_names":["michaelwoods/aegis-analyst"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/michaelwoods/aegis-analyst","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelwoods%2Faegis-analyst","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelwoods%2Faegis-analyst/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelwoods%2Faegis-analyst/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelwoods%2Faegis-analyst/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/michaelwoods","download_url":"https://codeload.github.com/michaelwoods/aegis-analyst/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelwoods%2Faegis-analyst/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29413443,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-13T06:24:03.484Z","status":"ssl_error","status_checked_at":"2026-02-13T06:23:12.830Z","response_time":78,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-13T17:34:39.663Z","updated_at":"2026-02-13T17:34:41.453Z","avatar_url":"https://github.com/michaelwoods.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Aegis Analyst for Frigate NVR\n\nA resilient, edge-optimized middleware that bridges the gap between **Frigate NVR**'s object detection and **Local LLMs** (via Ollama) to provide rich, narrative summaries of security events to **Home Assistant**.\n\n## 🚀 Key Features\n\n*   **Edge Optimized:** Designed specifically for consumer hardware (e.g., NVIDIA GTX 1060 6GB).\n*   **Fail-Over Inference:** Automatically switches from a high-quality 7B model (LLaVA) to a lightweight 1.8B model (Moondream) if VRAM is exhausted, ensuring continuous operation.\n*   **Home Assistant Integration:** Automatically discovered via MQTT, pushing a daily \"AI Digest\" and granular event statistics.\n*   **Batch Processing:** Intelligently batches requests to prevent thermal throttling and maintain system stability.\n*   **Frigate Write-back:** (Optional) Automatically updates Frigate events with rich AI-generated descriptions or sub-labels.\n*   **Context Aware:** Correlates visual events with Home Assistant sensor data (e.g., \"Person detected\" + \"Door opened\").\n\n## 🛠️ Architecture\n\n```mermaid\ngraph LR\n    Frigate[Frigate NVR] -- Review Items \u0026 Snapshots --\u003e Agent[Aegis Analyst]\n    Agent -- Tier 1/3 Requests --\u003e Ollama[Ollama LLM]\n    Agent -- Tier 2 Context --\u003e HA[Home Assistant]\n    Agent -- Discovery \u0026 States --\u003e MQTT[MQTT Broker]\n    MQTT --\u003e HA\n```\n\n### 🧠 AI Inference Pipeline\n\n```mermaid\ngraph TD\n    subgraph \"Tier 1: Perception\"\n    F[Frigate Review Item] -- Native GenAI or VLM --\u003e Desc[Visual Description]\n    end\n\n    subgraph \"Tier 2: Correlation\"\n    Desc -- + HA History --\u003e Corr[Semantic Narrative]\n    end\n\n    subgraph \"Tier 3: Synthesis\"\n    Corr -- Text LLM --\u003e Brief[Daily Security Briefing]\n    end\n\n    Brief -- MQTT --\u003e HA_S[Home Assistant Sensors]\n```\n\n1.  **Perception (Tier 1):** [Frigate NVR](https://frigate.video/) detects objects and (ideally) generates a description using its native Generative AI. The agent retrieves these **Review Items**, prioritizing existing descriptions. If missing, it fallbacks to its own VLM logic.\n2.  **Analysis (Tier 2):** The agent performs **Semantic Correlation**. It correlates the visual description with Home Assistant's History API (e.g., \"Person detected\" + \"Door opened\") to produce a cohesive narrative.\n3.  **Reporting (Tier 3):** A Text LLM (e.g., Llama 3) synthesizes the day's enriched narratives into a dense, narrative Daily Report.\n4.  **Presentation:** Results are published via **MQTT** as granular sensors for easy graphing and dashboarding.\n\n## 📋 Prerequisites\n\n*   **Inference Backend (Ollama):**\n    *   NVIDIA GPU (Recommended: 6GB+ VRAM).\n    *   Ollama (v0.1.30+) running on host or in a separate container.\n*   **Frigate NVR (v0.14+):**\n    *   It is highly recommended to enable Frigate's native Generative AI for Tier 1 analysis.\n\n### Configuring Frigate Native GenAI (Recommended)\nTo offload perception to Frigate, add the following to your `config.yml`:\n\n```yaml\ngenai:\n  enabled: True\n  provider: ollama\n  base_url: http://ollama:11434\n  model: llava:7b # Or any supported VLM\n```\n\nAegis Analyst will automatically detect these descriptions and skip its internal VLM step, focusing entirely on Home Assistant correlation.\n\n### 🖥️ Hardware Constraints (Example: GTX 1060 6GB)\nThis project is optimized for edge hardware with limited VRAM.\n*   **Primary Vision Model:** `llava:7b` (Quantized Q4_K_M takes ~4.8GB VRAM).\n*   **Fallback Model:** `moondream` (Takes ~1.6GB VRAM).\n*   **Fail-over Pattern:** If the primary model causes an Out-Of-Memory (OOM) error, the system automatically degrades to the fallback model to ensure reporting continuity.\n\n## ⚙️ Installation \u0026 Setup\n\n### 1. Prepare Ollama\nRun these commands on your host machine to pull the required models:\n\n```bash\n# Vision Model\nollama pull llava:7b\n\n# Analyst Model\nollama pull llama3\n\n# Fallback Model (Edge optimized)\nollama pull moondream\n```\n\n### 2. Deployment options\n\n#### Option A: Docker Compose (Recommended)\nCreate a `docker-compose.yml` file and mount your configuration:\n\n```yaml\nservices:\n  aegis-analyst:\n    image: ghcr.io/michaelwoods/aegis-analyst:latest\n    container_name: aegis-analyst\n    restart: unless-stopped\n    volumes:\n      - ./config.yaml:/app/config.yaml:ro\n      - ./data:/app/data\n    environment:\n      - TZ=America/New_York\n```\n\nStart the agent:\n```bash\ndocker compose up -d aegis-analyst\n```\n\n#### Option B: Manual Installation (Development)\nClone the repository and set up your environment:\n\n```bash\ngit clone https://github.com/your-username/aegis-analyst.git\ncd aegis-analyst\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\ncp config.example.yaml config.yaml\n```\n\nEdit `config.yaml` to match your network environment:\n*   **Frigate URL:** e.g., `http://192.168.1.50:5000`\n*   **Ollama:**\n    *   `vision_model`: e.g., `llava:7b`\n    *   `text_model`: e.g., `llama3`\n    *   `fallback_model`: e.g., `moondream`\n*   **MQTT Broker:** e.g., `192.168.1.50`\n*   **Home Assistant:**\n    *   `context_entities`: List of sensors (e.g., `binary_sensor.front_door`) to query for historical context.\n\n### 3. Automation (Optional)\nTo run the agent on a schedule (e.g., every hour) without Docker, you can use **Cron**:\n\n```bash\n# Example: Run every hour\n0 * * * * /path/to/aegis-analyst/.venv/bin/python /path/to/aegis-analyst/agent.py \u003e\u003e /var/log/aegis_analyst.log 2\u003e\u00261\n```\n\n## 📊 Home Assistant Dashboard\nThe agent uses **MQTT Discovery** to automatically create devices and entities.\n\n\u003cimg src=\"docs/screenshot.png\" alt=\"Aegis Analyst Dashboard\" width=\"600\"\u003e\n\n### Key Sensors\n*   `sensor.aegis_analyst_status`: Current operational state.\n*   `sensor.aegis_analyst_daily_security_report`: Full narrative summary in its `content` attribute.\n*   `sensor.aegis_analyst_[label]_count`: Individual count sensors for each object type.\n\n### Lovelace Card Example\nVisualize the report using a **Vertical Stack** card:\n\n```yaml\ntype: vertical-stack\ncards:\n  - type: entities\n    entities:\n      - entity: sensor.aegis_analyst_status\n        name: Status\n      - entity: sensor.aegis_analyst_bird_count\n        name: Birds Detected\n      - entity: sensor.aegis_analyst_car_count\n        name: Cars Detected\n      - entity: sensor.aegis_analyst_person_count\n        name: People Detected\n  - type: markdown\n    title: Daily Security Briefing\n    content: |\n      {{ state_attr('sensor.aegis_analyst_daily_security_report', 'content') }}\n```\n\n## 🧑‍💻 Development\n\nWe use a `Makefile` to enforce quality standards.\n\n```bash\nmake install   # Setup environment\nmake lint      # Run Ruff linter\nmake type-check# Run MyPy\nmake test      # Run all tests\nmake check     # Run all of the above\n```\n\n## 🛠️ Troubleshooting\n\n*   **`requests.exceptions.ConnectionError`**: Ensure Ollama is running and binding to `0.0.0.0` if the agent is running inside Docker.\n*   **`Ollama error: CUDA out of memory`**: The agent should auto-switch to `moondream`. If it still fails, reduce `batch_limit` in `config.yaml`.\n*   **Home Assistant shows \"Unknown\"**: The sensors are only created after the first successful run.\n\n## 🤝 Contributing\n1.  Fork the repository.\n2.  Create a feature branch (`git checkout -b feature/amazing-feature`).\n3.  Commit your changes.\n4.  Run tests (`make check`).\n5.  Open a Pull Request.\n\n## 📄 License\nMIT License - Copyright (c) 2026 Michael Woods","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelwoods%2Faegis-analyst","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmichaelwoods%2Faegis-analyst","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelwoods%2Faegis-analyst/lists"}