{"id":34100399,"url":"https://github.com/chandantochandan/vidurai","last_synced_at":"2025-12-14T16:58:01.282Z","repository":{"id":321856058,"uuid":"1085986138","full_name":"chandantochandan/vidurai","owner":"chandantochandan","description":"Local-first context infrastructure. Semantic compression (SF-V2) and temporal memory for developer workflows.","archived":false,"fork":false,"pushed_at":"2025-11-30T07:09:29.000Z","size":1883,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-01T17:19:23.975Z","etag":null,"topics":["context-engine","developer-tools","local-first","middleware","python","rag","semantic-compression"],"latest_commit_sha":null,"homepage":"https://vidurai.ai/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chandantochandan.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":"audit_reports/PROMPT_02_GISTING_SYSTEMS.md","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-10-29T19:33:06.000Z","updated_at":"2025-11-30T07:17:36.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/chandantochandan/vidurai","commit_stats":null,"previous_names":["chandantochandan/vidurai"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/chandantochandan/vidurai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandantochandan%2Fvidurai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandantochandan%2Fvidurai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandantochandan%2Fvidurai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandantochandan%2Fvidurai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chandantochandan","download_url":"https://codeload.github.com/chandantochandan/vidurai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chandantochandan%2Fvidurai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27731978,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-14T02:00:11.348Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["context-engine","developer-tools","local-first","middleware","python","rag","semantic-compression"],"created_at":"2025-12-14T16:58:00.761Z","updated_at":"2025-12-14T16:58:01.275Z","avatar_url":"https://github.com/chandantochandan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/chandantochandan/vidurai/main/assets/brand/logo_primary_512.png\" alt=\"Vidurai Logo\" width=\"180\" /\u003e\n  \u003ch1\u003eVidurai\u003c/h1\u003e\n  \u003cp\u003eLocal-First Context Infrastructure \u0026 Semantic Compression Engine\u003c/p\u003e\n\u003c/div\u003e\n\n[![PyPI version](https://img.shields.io/pypi/v/vidurai.svg)](https://pypi.org/project/vidurai/)\n![Python versions](https://img.shields.io/pypi/pyversions/vidurai.svg)\n[![License](https://img.shields.io/pypi/l/vidurai.svg)](https://github.com/chandantochandan/vidurai/blob/main/LICENSE)\n\n---\n\nVidurai is **local-first context middleware** that solves the *context window bottleneck* in LLM-driven workflows.\n\nIt sits between **developer environments** (IDE, terminal, browser) and **AI models**, ingesting high-velocity telemetry (file edits, terminal commands, AI chats), normalizing it via a **Shared Event Schema**, and applying **SF-V2 semantic compression** to maintain a **high-signal temporal graph** of a project.\n\n**Core capability:**\nReduce tokens by 50–70% for AI prompts while preserving 100% of critical technical entities (file paths, stack traces, function names).\n\n---\n\n## 1. Engineering Problem\n\nRaw developer activity is high-entropy:\n\n- **IDE:** rapid file saves, refactors, and diagnostics\n- **Terminal:** command spam, noisy logs\n- **Browser:** ad-hoc AI chats with half-remembered context\n\nFeeding these directly into an LLM causes:\n\n1. **Context saturation** – token limits hit long before the real history is represented.\n2. **State fragmentation** – VS Code, terminal, and browser all have partial views.\n3. **Hallucinations** – the model optimizes on recent noise instead of actual project state.\n4. **Runaway cost** – you pay for the same story over and over again.\n\n**Vidurai's role:**\nAct as a **local context layer** that continuously ingests signals, compresses them into durable \"episodes\", and serves **audience-specific context** back to tools and agents on demand.\n\n---\n\n## 2. Architecture Overview (v2.0.0)\n\n```text\n       [VS Code]      [Browser UI]         [Terminal]\n           │               │                   │\n      (WebSocket)     (WebSocket)           (StdOut)\n           │               │                   │\n           ▼               ▼                   ▼\n    ┌────────────────────────────────────────────────┐\n    │              Vidurai Daemon (Local)            │\n    │  [Normalizer] → [SF-V2 Engine] → [SQLite/WAL]  │\n    └───────────────────────┬────────────────────────┘\n                            │  (HTTP / JSON)\n                ┌───────────┴───────────┐\n                ▼                       ▼\n          [Python SDK / CLI]      [LLM Proxy]\n        (Context Retrieval)     (Prompt Injection)\n```\n\n### Components\n\n#### 1. Vidurai SDK (this package)\n\n- Pydantic models for the Shared Event Schema\n- SF-V2 semantic compression engine\n- Local project client (`VismritiMemory`)\n- CLI tooling (`vidurai …`)\n\n#### 2. Vidurai Daemon (hub service)\n\n- **Stack:** Python 3.9+, FastAPI, AsyncIO\n- Runs on `localhost:7777`\n- Ingests events from VS Code / Browser / CLI\n- Performs entity extraction (paths, symbols, errors, IDs)\n- Applies SF-V2 scoring + consolidation\n- Persists to SQLite (`~/.vidurai/vidurai.db`, WAL mode)\n\n#### 3. VS Code Extension (sensor)\n\n- Watches file edits, terminal commands, and diagnostics\n- Emits structured `ViduraiEvent` messages to the daemon\n- Exposes \"Copy Context for AI\" command for one-click prompt injection\n\n#### 4. Browser Extension (sensor)\n\n- Targets AI chat UIs (ChatGPT, Claude, Gemini, etc.)\n- Captures prompts + responses as events\n- Injects Vidurai context into new chats via hotkey / UI actions\n\n#### 5. LLM Proxy (optional)\n\n- HTTP proxy around LLM APIs\n- Automatically attaches Vidurai gists to outbound prompts\n- Enables context-aware CLI tools, scripts, or agents\n\n---\n\n## 3. Shared Event Schema\n\nAll tools speak the same language: `ViduraiEvent`.\n\n```json\n{\n  \"schema_version\": \"vidurai-events-v1\",\n  \"event_id\": \"d8c8a3f2-9c25-4e2a-9a4d-3f2e91f4b7a1\",\n  \"timestamp\": \"2025-11-26T18:30:00.123Z\",\n  \"source\": \"vscode\",\n  \"channel\": \"human\",\n  \"kind\": \"file_edit\",\n  \"project_root\": \"/home/user/vidurai\",\n  \"project_id\": \"proj_8f21b9a0\",\n  \"session_id\": \"sess_02c9b1d4\",\n  \"payload\": {\n    \"file_path\": \"vidurai/core/sf_v2.py\",\n    \"language\": \"python\",\n    \"change_type\": \"modified\",\n    \"summary\": \"Refactor scoring function\"\n  }\n}\n```\n\nBacked by:\n\n- **Python:** `vidurai.shared.events` (`ViduraiEvent`, `EventKind`, etc.)\n- **TypeScript:** `src/shared/events.ts` in VS Code + browser extensions\n\nThis schema is the contract between all sensors and the daemon.\n\n---\n\n## 4. SF-V2 Semantic Compression\n\nVidurai does not append logs forever.\n\nSF-V2 runs a deterministic pipeline:\n\n1. **Window selection** – select a time/window slice of events for a project.\n2. **Role classification** – label events as `CAUSE`, `ATTEMPT`, `FIX`, `NOISE`, etc.\n3. **Scoring** – compute retention scores using recency, frequency, and structural importance.\n4. **Consolidation** – merge low-value events into summaries while preserving hard entities (paths, line numbers, symbols, IDs).\n5. **Episode creation** – store high-density \"episodes\" that reflect how the project changed.\n\nOutputs are then exposed as **gists**:\n\n| Audience | Purpose |\n|----------|---------|\n| `AI` | token-bounded, machine-optimized context for LLM calls |\n| `DEVELOPER` | richer textual summary for humans |\n| `MANAGER` / `STAKEHOLDER` | higher-level progress narratives |\n\n---\n\n## 5. Security \u0026 Data Governance\n\nVidurai is built for zero-trust environments:\n\n### Local-first storage\n\n- Data lives in `~/.vidurai/` by default (SQLite + JSONL ledgers).\n- No mandatory cloud component.\n\n### Ingestion-time redaction\n\n- Pluggable regex filters to mask API keys, tokens, and secrets before persistence.\n\n### Transparency\n\n- Every compression / deletion decision is logged to a `forgetting_ledger.jsonl` for audit and debugging.\n- Nothing leaves the machine unless you explicitly configure a remote proxy or sync mechanism.\n\n---\n\n## 6. Installation \u0026 Deployment\n\n### 6.1 Python SDK\n\n```bash\npip install vidurai\n```\n\nRequires Python 3.9+.\n\n### 6.2 Daemon (recommended: Docker)\n\n```bash\ndocker run --rm -d \\\n  -p 7777:7777 \\\n  -v ~/.vidurai:/root/.vidurai \\\n  chandantochandan/vidurai-daemon:2.0.0\n```\n\nThis starts the Vidurai Daemon on `http://localhost:7777` with persistent state in `~/.vidurai`.\n\n### 6.3 Optional: LLM Proxy\n\n```bash\ndocker run --rm -d \\\n  -p 9999:9999 \\\n  -v ~/.vidurai:/root/.vidurai \\\n  chandantochandan/vidurai-proxy:2.0.0\n```\n\nUse this if you want Vidurai gists automatically injected into outbound LLM API calls.\n\n---\n\n## 7. SDK Usage Example\n\n```python\nfrom vidurai.vismriti_memory import VismritiMemory\n\n# 1. Initialize memory for a project\nmemory = VismritiMemory(project_path=\".\")\n\n# 2. Store a memory with automatic salience detection\nmemory.remember(\n    gist=\"Refactored authentication module to use JWT tokens\",\n    tags=[\"refactor\", \"auth\", \"jwt\"],\n    metadata={\"file\": \"auth/jwt_handler.py\", \"line\": 42}\n)\n\n# 3. Recall relevant context\nresults = memory.recall(\n    query=\"authentication implementation\",\n    limit=5\n)\n\nfor mem in results:\n    print(f\"[{mem.salience}] {mem.gist}\")\n\n# 4. Get compressed context for AI prompt injection\ncontext = memory.get_context(\n    query=\"current auth flow\",\n    max_tokens=2000\n)\n\nprint(f\"Injecting context into LLM prompt:\")\nprint(context)\n```\n\nTypical uses:\n\n- Pre-pend context to prompts for ChatGPT/Claude/Gemini.\n- Feed specialized gists into autonomous agents / tools.\n- Generate manager-friendly status summaries.\n\n---\n\n## 8. Integrations\n\n### VS Code Extension\n\n- **Marketplace ID:** `vidurai.vidurai`\n- Sends `file_edit`, `terminal_command`, and `diagnostics` events to the daemon.\n- Provides \"Copy Vidurai Context\" command for pasting into AI chats.\n\n### Browser Extension\n\n- Chrome MV3 extension (Universal AI Context).\n- Targets major AI UIs (ChatGPT, Claude, Gemini, etc.).\n- Can inject Vidurai context into new conversations.\n\n### CLI / Scripts\n\n- Use the SDK or HTTP API to pull audience-specific gists and attach them to prompts in custom tools or agents.\n\n---\n\n## 9. Versioning \u0026 Legacy RL Engine\n\n- **Current release:** v2.0.0 – unified architecture around SF-V2 + shared events.\n- **Legacy RL-based v1.x:** preserved for reference in `docs/archive/implementation/`.\n\nThe v2 line focuses on deterministic semantic compression and event-driven context, not on generic RL experimentation.\n\n---\n\n## 10. Repository Layout\n\n```\nvidurai/                     # Python SDK (this package)\nvidurai-daemon/              # Local hub service\nvidurai-vscode-extension/    # VS Code telemetry extension\nvidurai-browser-extension/   # Browser AI context extension\nvidurai-proxy/               # Optional LLM proxy\ntests/                       # SDK tests\ndocs/                        # Documentation\n  └── archive/\n      └── implementation/    # Historical PHASE_* docs\nARCHITECTURE_OVERVIEW.md     # High-level system architecture\nCHANGELOG.md                 # Version history\n```\n\n---\n\n## 11. Links\n\n- **Website:** https://vidurai.ai\n- **Docs:** https://docs.vidurai.ai\n- **Source:** https://github.com/chandantochandan/vidurai\n- **Issues:** https://github.com/chandantochandan/vidurai/issues\n- **Docker Hub:** https://hub.docker.com/u/chandantochandan\n\n---\n\n## 12. License\n\nVidurai is released under the **MIT License**.\nSee the [LICENSE](https://github.com/chandantochandan/vidurai/blob/main/LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchandantochandan%2Fvidurai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchandantochandan%2Fvidurai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchandantochandan%2Fvidurai/lists"}