{"id":51463277,"url":"https://github.com/kulkarnishub377/omnisight","last_synced_at":"2026-07-06T08:01:01.331Z","repository":{"id":365896015,"uuid":"1274238258","full_name":"kulkarnishub377/OmniSight","owner":"kulkarnishub377","description":"🔍 OmniSight — Autonomous Multi-Modal Incident Intelligence Fabric. Real-time event correlation, intelligence synthesis, and AI-powered operational dashboards.","archived":false,"fork":false,"pushed_at":"2026-06-19T11:51:40.000Z","size":196,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-19T12:13:25.797Z","etag":null,"topics":["artificial-intelligence","fastapi","intelligence-fabric","multi-modal-ai","omnisight"],"latest_commit_sha":null,"homepage":"https://kulkarnishub377.github.io/OmniSight/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kulkarnishub377.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-19T10:06:24.000Z","updated_at":"2026-06-19T11:51:44.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kulkarnishub377/OmniSight","commit_stats":null,"previous_names":["kulkarnishub377/autonomous-multi-modal-incident-intelligence-fabric"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/kulkarnishub377/OmniSight","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kulkarnishub377%2FOmniSight","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kulkarnishub377%2FOmniSight/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kulkarnishub377%2FOmniSight/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kulkarnishub377%2FOmniSight/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kulkarnishub377","download_url":"https://codeload.github.com/kulkarnishub377/OmniSight/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kulkarnishub377%2FOmniSight/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35182322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-06T02:00:07.184Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","fastapi","intelligence-fabric","multi-modal-ai","omnisight"],"created_at":"2026-07-06T08:00:59.861Z","updated_at":"2026-07-06T08:01:01.311Z","avatar_url":"https://github.com/kulkarnishub377.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# 🌌 OmniSight v1.0\n**Autonomous Multi-Modal Incident Intelligence Fabric**\n\n[![Python Version](https://img.shields.io/badge/python-3.10%2B-blue.svg?style=for-the-badge\u0026logo=python)](https://www.python.org/)\n[![FastAPI](https://img.shields.io/badge/FastAPI-0.103.0-009688.svg?style=for-the-badge\u0026logo=fastapi)](https://fastapi.tiangolo.com/)\n[![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge\u0026logo=docker\u0026logoColor=white)](https://www.docker.com/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge)](https://opensource.org/licenses/MIT)\n\n*A production-grade, event-driven AI architecture project for real-time, multi-modal incident intelligence and agentic investigations.*\n\n[Operator Console](http://localhost:8000/static/console.html) · [Architecture Details](http://localhost:8000/static/architecture.html) · [API Docs](http://localhost:8000/docs) · [Interactive Report](http://localhost:8000/reports/OmniSight_Interactive_Project_Report.html)\n\n\u003c/div\u003e\n\n---\n\n## ⚡ Executive Summary\n\n**OmniSight** (Autonomous Multi-Modal Incident Intelligence Fabric) is an advanced, enterprise-ready AI platform designed to ingest and correlate high-velocity signals from diverse edge sources—including RTSP Cameras, IoT Sensors, Microphones, and standard operating procedure (SOP) Documents. \n\nInstead of relying on human operators to stare at monitors and match anomalies, OmniSight automatically validates, deduplicates, and enriches these events in real time. Once an anomaly is detected, an **Agentic Workflow** kicks in: retrieving semantic evidence from vector memory, reasoning through risk factors, generating safety guardrails, and presenting a holistic view via a high-end, premium Operator Dashboard.\n\n---\n\n## 🎯 Core Use Cases\n\nOmniSight is designed to be highly adaptable to various physical-world monitoring scenarios:\n\n1. **🏭 Smart Manufacturing \u0026 Industrial IoT**\n   - **Scenario**: A critical CNC machine exhibits an IoT temperature spike while an audio sensor detects a grinding anomaly.\n   - **OmniSight Response**: Correlates the two events, queries the machine's maintenance manual using RAG, determines a high risk of catastrophic failure, and issues a severe alert recommending immediate shutdown.\n2. **🏢 Warehouse Safety \u0026 Logistics**\n   - **Scenario**: A camera detects a forklift moving at high speed in a pedestrian-only zone.\n   - **OmniSight Response**: Logs the safety violation, links it to the site safety protocol document, and automatically drafts an incident report for the shift supervisor.\n3. **🏙️ Smart City Operations**\n   - **Scenario**: Traffic cameras detect a collision, and nearby microphones register a loud impact.\n   - **OmniSight Response**: Fuses the multi-modal data, assesses the severity, and prepares a structured payload for emergency dispatch APIs.\n4. **🔐 Secure Facility Monitoring**\n   - **Scenario**: A badge-swipe is registered at a secure door, but the camera detects two faces (tailgating).\n   - **OmniSight Response**: Flags the anomaly, cross-references the access control policy, and alerts the guard station with the exact video frame evidence.\n\n---\n\n## 🧠 Agentic Workflow Engine\n\nOmniSight moves beyond simple rules-based alerts by utilizing a **Deterministic Agentic Runtime** composed of specialized AI personas:\n\n- **👁️ Observer**: Continuously monitors the event stream and identifies raw anomalies (e.g., \"Temperature \u003e 85°C\").\n- **🕵️ Investigator**: Groups related anomalies within specific time-windows and locations into structured **Incidents**.\n- **📚 Retriever**: Queries the Qdrant/Postgres vector database to find relevant SOPs, safety manuals, or past resolutions.\n- **🗺️ Planner**: Analyzes the evidence and formulates a step-by-step resolution plan.\n- **🛡️ Safety Guard**: A deterministic policy engine that reviews the Planner's suggestions to ensure no autonomous actions violate enterprise safety rules.\n- **⚙️ Action Executor**: Dispatches approved actions (e.g., sending emails, creating Jira tickets, triggering webhooks).\n\n---\n\n## 🔄 Platform Architecture \u0026 Data Flow\n\n```mermaid\ngraph LR\n    A[Edge Sensors\u003cbr\u003eCameras/IoT] --\u003e|JSON Payloads| B[FastAPI Gateway]\n    B --\u003e|Schema Validation| C[Event Processor]\n    C --\u003e|Time-Window Match| D[Incident Correlator]\n    \n    D --\u003e E[Memory Fabric]\n    E -.-\u003e|Vector Search| F[Semantic DB]\n    E -.-\u003e|State| G[PostgreSQL]\n    \n    D --\u003e H[Agent Runtime]\n    H --\u003e|RAG| F\n    H --\u003e|Policy Check| I[Safety Guard]\n    I --\u003e J[Action Execution]\n    I --\u003e K[Operator Console]\n    \n    style A fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff\n    style B fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#fff\n    style C fill:#1e293b,stroke:#a78bfa,stroke-width:2px,color:#fff\n    style E fill:#1e293b,stroke:#fbbf24,stroke-width:2px,color:#fff\n    style H fill:#1e293b,stroke:#fb7185,stroke-width:2px,color:#fff\n    style K fill:#022c22,stroke:#34d399,stroke-width:2px,color:#fff\n```\n\n### The Memory Fabric\n- **Durable Layer**: PostgreSQL handles immutable event logs, incident states, and audit trails.\n- **Episodic Layer**: Redis-ready architecture for fast, transient state and rate-limiting.\n- **Semantic Layer**: Qdrant-ready vector store (with a deterministic Postgres fallback) for chunking and retrieving document context via RAG.\n- **Graph Layer**: Neo4j-style endpoints that map relationships between locations, sensors, incidents, and people.\n\n---\n\n## 🛠️ Technology Stack Matrix\n\n| Domain | Technologies Used | Purpose |\n| :--- | :--- | :--- |\n| **Backend API** | FastAPI, Pydantic, Python 3.10+ | High-performance, async REST gateway and data validation. |\n| **Security** | Bcrypt, PyJWT, SlowAPI, OAuth2 | Secure password hashing, stateless tokens, and brute-force rate limiting. |\n| **Database** | SQLAlchemy, PostgreSQL, SQLite | Robust ORM and relational state management. |\n| **Frontend** | Vanilla JS, HTML5, CSS3 (HSL) | Premium glassmorphism UI, zero-dependency Markdown rendering. |\n| **Observability** | Prometheus, Grafana | API metric instrumentation and performance dashboards. |\n| **Infrastructure** | Docker, Docker Compose, K8s | Containerization and cloud-native orchestration readiness. |\n\n---\n\n## 📂 Unified Project Structure\n\nAll application code, connectors, and infrastructure bindings have been cleanly unified into a monolithic backend structure for deployment ease.\n\n```text\namif/\n├── backend/app/                 # Core Application Namespace\n│   ├── connectors/              # Integrations: camera_connector, audio_connector, iot_connector\n│   ├── core/                    # Config, Security (Bcrypt/JWT), RBAC, Rate Limiting\n│   ├── db/                      # SQLAlchemy Engine, Sessions \u0026 Migrations\n│   ├── infra/                   # Kubernetes Manifests \u0026 Prometheus Configs\n│   ├── models/                  # ORM Database Models \u0026 Enums\n│   ├── routers/                 # FastAPI REST Endpoints (Auth, Events, Incidents)\n│   ├── schemas/                 # Pydantic Validation Contracts\n│   ├── scripts/                 # Demo Seed Scripts \u0026 Utilities\n│   ├── services/                # Business Logic (Event Processor, Agents, RAG)\n│   ├── static/                  # Premium Operator Console UI \u0026 Assets\n│   └── workers/                 # Background Task Processors\n├── docs/                        # Architecture \u0026 UX Blueprints (.md)\n├── reports/                     # Generated HTML, PDF, and PPTX reports\n├── index.html                   # Centralized Workspace Portal Hub\n└── docker-compose.yml           # Local multi-container orchestration\n```\n\n---\n\n## 🚀 Quick Start Guide\n\n### Option A: Using Docker (Recommended)\n\n1. Clone the repository and navigate to the project root.\n2. Initialize environment variables:\n   ```bash\n   cp .env.example .env\n   ```\n3. Spin up the infrastructure:\n   ```bash\n   docker compose up --build\n   ```\n\n### Option B: Local Python Execution\n\n1. Navigate to the backend directory and set up a virtual environment:\n   ```bash\n   cd backend\n   python -m venv .venv\n   # Windows: .venv\\Scripts\\activate\n   # Mac/Linux: source .venv/bin/activate\n   ```\n2. Install dependencies:\n   ```bash\n   pip install -r requirements.txt\n   ```\n3. Launch the FastAPI server:\n   ```bash\n   uvicorn app.main:app --reload --host 127.0.0.1 --port 8000\n   ```\n\n---\n\n## 🎮 The Demo Experience\n\nOnce the server is running, the entire ecosystem is mapped and accessible from the root domain!\n\n1. **Launch the Portal**: Open your browser to `http://127.0.0.1:8000/`. You will be greeted by the Central Workspace Hub.\n2. **Access the Console**: Click \"Launch Console\" and log in using the default secure credentials:\n   - **Email**: `admin@example.com`\n   - **Password**: `admin123`\n3. **Run the Scenario**: Click the **Seed Demo** button in the console.\n   - **The Simulation**: AMIF automatically uploads a Machine Safety Manual via the Document Connector, fires a Forklift proximity event via the Camera Connector, and triggers an overheating reading via the IoT Connector.\n   - **The Intelligence**: The Event Processor correlates these into a critical incident. The Agent Workflow wakes up, consults the uploaded manual using RAG, enforces safety guardrails, and publishes an Operator Alert.\n4. **Observe**: Navigate through the console to view the raw events, explore the semantic evidence, review the generated tickets, and inspect the unified incident timeline.\n\n---\n\n## 📖 Developer Documentation \u0026 Observability\n\n- **Centralized Workspace Hub**: `http://localhost:8000/` (Reads directly from the `/docs` directory)\n- **Interactive OpenAPI Specs**: `http://localhost:8000/docs`\n- **Prometheus Metrics**: `http://localhost:9090`\n- **Grafana Dashboards**: `http://localhost:3000` *(admin/admin)*\n\n---\n\n## 🔮 Roadmap \u0026 Future Extensions\n\n- **Microservice Extraction**: Split the monolithic `backend/app` into dedicated Pods (e.g., Ingestion Service, Agent Service).\n- **Real LLM Integration**: Hook the Agent Runtime into OpenAI/Anthropic APIs instead of the deterministic MVP fallback.\n- **Event Streaming**: Replace the synchronous in-memory processor with Redpanda/Kafka topics using FastStream.\n- **Native Vector DB**: Swap the Postgres fallback for a fully dedicated Qdrant instance.\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\u003ci\u003eArchitected with ❤️ for modern AI Safety \u0026 Operations\u003c/i\u003e\u003cbr\u003e\n\u003cb\u003eDistributed under the MIT License.\u003c/b\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkulkarnishub377%2Fomnisight","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkulkarnishub377%2Fomnisight","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkulkarnishub377%2Fomnisight/lists"}