{"id":51320907,"url":"https://github.com/madmax983/egregore","last_synced_at":"2026-07-01T13:30:40.883Z","repository":{"id":358389500,"uuid":"1241023014","full_name":"madmax983/egregore","owner":"madmax983","description":"Code Intelligence and Agent Memory","archived":false,"fork":false,"pushed_at":"2026-06-26T15:40:57.000Z","size":2598,"stargazers_count":2,"open_issues_count":140,"forks_count":0,"subscribers_count":0,"default_branch":"trunk","last_synced_at":"2026-06-26T17:21:58.112Z","etag":null,"topics":["agent-memory","agentic-ai","code-intelligence","semantic-search"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/madmax983.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-05-16T21:37:55.000Z","updated_at":"2026-06-26T02:39:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/madmax983/egregore","commit_stats":null,"previous_names":["madmax983/aletheia-codegraph","madmax983/egregore"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/madmax983/egregore","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madmax983%2Fegregore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madmax983%2Fegregore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madmax983%2Fegregore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madmax983%2Fegregore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/madmax983","download_url":"https://codeload.github.com/madmax983/egregore/tar.gz/refs/heads/trunk","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madmax983%2Fegregore/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35009266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-01T02:00:05.325Z","response_time":130,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-memory","agentic-ai","code-intelligence","semantic-search"],"created_at":"2026-07-01T13:30:39.971Z","updated_at":"2026-07-01T13:30:40.877Z","avatar_url":"https://github.com/madmax983.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Egregore\n\nEgregore is an `AletheiaDB`-backed knowledge graph substrate for agentic software engineering. It connects deterministic code facts, agent memory, project state, artifacts, and verification evidence in one temporal graph.\n\nThe current implemented slice is the code graph domain: it parses Rust, Python, TypeScript, and Go with Tree-sitter, emits stable JSONL graph records, replays Git history without mutating the working checkout, ingests records into an embedded AletheiaDB store, and exposes structural and semantic query helpers for agent workflows.\n\n## Getting Started\n\nThis section walks you from a fresh clone to a fully queryable semantic search index of your codebase.\n\n### 1. Prerequisites\n\n- **Rust** (stable, edition 2024) — install via [rustup](https://rustup.rs)\n- **Git** — for history replay\n- **Python 3.8+** with **pip** — required once to download the embedding model\n\nOn MSVC (Windows), `.cargo/config.toml` sets `CXXFLAGS_x86_64_pc_windows_msvc=/MT` to align the C++ runtime used by the `embed_anything` / tokenizers native stack. This is already committed; no manual action needed.\n\n### 2. Build\n\n```powershell\ncargo build --release\n```\n\nThe release binary lands at `target/release/egregore.exe` (and `eg.exe` as a short alias). During development, substitute `cargo run --` for `egregore`.\n\n### 3. Scan your codebase\n\nPoint `scan` at any repository. It parses every supported source file — `.rs` (Rust), `.py` (Python), `.ts`/`.tsx` (TypeScript), and `.go` (Go) — with Tree-sitter and emits a deterministic JSONL graph of nodes (files, modules, symbols, imports, diagnostics) and edges (DEFINES, CALLS, IMPORTS, MENTIONS, CONTAINS).\n\n```powershell\negregore scan . --out graph.jsonl\negregore inspect graph.jsonl\n# records: 4386   nodes: 1353   edges: 3033   tombstones: 0   diagnostics: 599\n```\n\n#### Scoping and Ignore Rules\n\nWhen run in a Git repository root, `eg scan` automatically scopes the scan to **Git-tracked files only** (both committed and staged changes in the Git index). It honors `.gitignore` rules (including nested gitignores and negation `!`) by ignoring any untracked or ignored files.\n\nIf there are unsupported, untracked, or ignored files, the scan reports the count of skipped files for each supported language to `stderr`:\n```\nSkipped 3 .rs, 1 .py, 1 .ts/.tsx, 1 .go files by ignore rules\n```\n\nTo list the skipped files in your repository, you can use Git:\n- **Ignored files:** `git ls-files --others --ignored --exclude-standard`\n- **Untracked files:** `git ls-files --others --exclude-standard`\n\nIf the target path is not a Git repository root, `eg scan` falls back to a filesystem walk that walks all files under the directory but hardcodes skipping `.git` and `target` directories.\n\n\n### 4. Ingest into AletheiaDB (structural index)\n\n```powershell\negregore ingest graph.jsonl --adapter embedded --data-dir .egregore\n```\n\nThis writes the graph into a local AletheiaDB store. The store is self-contained in `.egregore/` and does not require a running server.\n\n### 5. Verify setup before semantic work (`eg doctor`)\n\nBefore the semantic embedding step, run the setup preflight to confirm this machine\nhas the tools and permissions needed for the next steps:\n\n```powershell\negregore doctor .\n```\n\nExit 0 means the structural workflow (scan, ingest) is ready. `semantic_ready: true`\nin the JSON output means Python and the embedding model cache are also in place and\nyou can skip to step 7. Consult the `next_command` field in the output for the\nshortest path forward.\n\n```powershell\n# Human-readable version\negregore doctor . --format text\n```\n\n\u003e **Note:** `eg doctor` is a _setup_ preflight — it runs before ingest and checks\n\u003e whether your local tools and permissions are correct. It is distinct from the\n\u003e post-ingest semantic index readiness report (issue #71), which checks whether an\n\u003e already-ingested store has adequate embedding coverage. See\n\u003e [docs/cli/doctor.md](docs/cli/doctor.md) for the full check matrix, exit codes,\n\u003e and the distinction between the two reports.\n\n### 6. Download the embedding model (first time only)\n\nThe semantic search feature uses `sentence-transformers/all-MiniLM-L6-v2` (384 dimensions, ~90 MB). Rust's TLS stack may not share your OS certificate store, so the most reliable way to prime the cache is via Python:\n\n```powershell\npip install -U sentence-transformers\npython -c \"from sentence_transformers import SentenceTransformer; SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')\"\n```\n\nThis downloads the model to `~/.cache/huggingface/hub/`, where Rust's `hf-hub` will find it on subsequent runs. You only need to do this once per machine.\n\n### 7. Ingest with semantic embeddings\n\n```powershell\negregore ingest graph.jsonl --adapter embedded --data-dir .egregore-semantic --embed\n# Generating embeddings for 650 file/symbol nodes…\n# attempted: 4386   succeeded: 4386   failed: 0\n```\n\nThe `--embed` flag generates a 384-dimensional dense vector for every file and symbol node and stores it in AletheiaDB's HNSW vector index. The store at `.egregore-semantic/` supports both structural and semantic queries.\n\n\u003e **Tip:** Use a separate `--data-dir` for the embedded store so you can keep a fast structural-only store alongside the larger semantic one.\n\n### 8. Query\n\n**Structural queries** (exact name, file, or drift — works with or without `--embed`):\n\n```powershell\n# Find a symbol by fully-qualified name\negregore query symbol daemon::handle_query --data-dir .egregore --format text\n\n# List all symbols defined in a file\negregore query file src/daemon.rs --data-dir .egregore --format text\n\n# Show semantic drift records ranked by change score\negregore query drift --data-dir .egregore --limit 10 --format text\n\n# Pin a symbol query to a specific commit\negregore query symbol daemon::handle_query --data-dir .egregore --at \u003ccommit-sha\u003e\n\n# Scope a query to one repository in a shared multi-repo store\negregore query symbol handle_query --data-dir .egregore --repo owner/name\n```\n\n**Semantic queries** (natural language — requires `--embed` store):\n\n```powershell\n# Default output is JSON; pass --format text for human-readable lines\negregore query semantic \"write nodes to database storage\" --data-dir .egregore-semantic\negregore query semantic \"error handling and budget limits\" --data-dir .egregore-semantic --limit 5\negregore query semantic \"temporal history git commit tracking\" --data-dir .egregore-semantic --format text\n```\n\nAgent-citable JSON result (stable fields: `record_id`, `score`, `name`, `repo_relative_path`, `span`):\n\n```json\n{\"record_id\":\"codegraph:v1:abc123\",\"name\":\"EmbeddedAletheiaSink::write_record\",\"repo_relative_path\":\"src/sink/embedded.rs\",\"score\":0.9231,\"span\":{\"start_byte\":4096,\"end_byte\":5200,\"start_line\":142,\"end_line\":168}}\n```\n\nHuman-readable text output (`--format text`, not a stable parsing contract):\n\n```\nhistory::GitCommit score=0.4974 @ src/history.rs:138\nhistory::list_commits score=0.4767 @ src/history.rs:168\nhistory::git_output score=0.4610 @ src/history.rs:312\n```\n\nThe semantic index finds code by **meaning**, not by name. Querying `\"write nodes to database storage\"` surfaces `EmbeddedAletheiaSink::write_record` even though none of those words appear in the function name.\n\nSee [docs/cli/query.md](docs/cli/query.md) for the full JSON output contract, no-result exit codes, and operator actions.\n\n---\n\n## Why not just use ripgrep?\n\nStructural and semantic queries return a typed, citable handle instead of every\nline a text search would make an agent ingest — no false positives from comments\nor string literals, and no bodies to read just to learn what a file defines.\n\nThat saving is **measured, not asserted**. On the pinned token-cost corpus\n(`corpus/token_cost_corpus/`), counting answer tokens with one deterministic\nmethod on both sides, the baseline-to-Egregore ratios are:\n\n| Question | `eg` answer vs. `rg` baseline | Ratio |\n|----------|-------------------------------|-------|\n| Exact symbol lookup | `eg query symbol` vs. `rg -w \u003cname\u003e` | ~14× cheaper |\n| File-defines lookup | `eg query file` vs. reading the file | ~3.6× cheaper |\n| Semantic concept query | `eg query semantic --limit 10` vs. `rg -w 'k1\\|k2\\|…'` (concept keywords) | ~3.3× cheaper |\n| **Aggregate** | | **~4.1× cheaper** |\n\nEach side is measured fairly: the semantic baseline greps the *union of concept\nkeywords* a human would try (not one literal), and the Egregore answer is the\nreal `eg query semantic --limit 10` shape — so both reflect \"explore this\nconcept\", not a top-1 lookup against a broad search.\n\nReproduce it with `eg audit token-cost`; the gate fails if any class regresses\nbelow 3×. The multiplier scales with the corpus — the more comment and\nstring-literal false positives a text search drags in, and the larger the file\nbodies a structural answer summarizes, the larger the saving. See\n[docs/cli/token-cost.md](docs/cli/token-cost.md).\n\n---\n\n## All commands\n\n```powershell\negregore scan . --out graph.jsonl                          # current-tree extraction\negregore scan-history . --out history.graph.jsonl          # commit-by-commit temporal extraction\negregore inspect graph.jsonl                               # graph summary\negregore ingest graph.jsonl --adapter dry-run              # validate without writing\negregore ingest graph.jsonl --adapter embedded \\\n    --data-dir .egregore                                   # structural store\negregore ingest graph.jsonl --adapter embedded \\\n    --data-dir .egregore-semantic --embed                  # structural + semantic store\negregore query symbol \u003cname\u003e  --data-dir .egregore\negregore query file   \u003cpath\u003e  --data-dir .egregore\negregore query drift          --data-dir .egregore\negregore query semantic \u003ctext\u003e --data-dir .egregore-semantic\negregore import-codex session.jsonl --out agent.graph.jsonl  # Codex session/rollout import\negregore bundle export --root-selector \u003cselector\u003e \\\n    --graph \u003cpath\u003e --out \u003cpath\u003e                            # export evidence bundle\negregore bundle inspect \u003cpath\u003e                              # inspect bundle manifest\negregore bundle verify \u003cpath\u003e                               # verify bundle integrity, coverage, and safety\n# eg import github \u003cowner\u003e/\u003crepo\u003e --out github.jsonl          # planned — see docs/schema/import-github.md\n```\n\n---\n\n## Status\n\nImplemented surfaces:\n\n- `scan` — current-tree JSONL extraction (Rust, Python, TypeScript, Go via Tree-sitter)\n- `scan-history` — commit-by-commit temporal extraction\n- `inspect` — graph summary\n- `ingest` — dry-run, embedded AletheiaDB, and daemon adapters\n- `import-codex` — Codex session/rollout JSONL → agent-memory graph records (M3)\n- `query symbol / file / drift / semantic` — structural and semantic agent queries\n- Incremental file-cache planning with tombstones\n- AletheiaDB embedding re-export through the optional `embeddings` feature\n- Semantic drift records and query helpers for symbol-at-commit and largest-drift workflows\n- HNSW vector index via `embeddings` feature (enabled by default)\n- `bundle` — export, verify, and inspect redaction-safe evidence bundles (Issue #68)\n\n`embedded-aletheiadb` is enabled by default and uses the published `aletheiadb` crate with `semantic-search`, `semantic-temporal`, and `semantic-diagnostics`.\n\nOn MSVC, this repo sets `CXXFLAGS_x86_64_pc_windows_msvc=/MT` in `.cargo/config.toml` so AletheiaDB's optional `embeddings` feature can link the transitive `embed_anything`/tokenizers native C++ stack consistently.\n\n## Usage\n\n```powershell\ncargo run -- scan . --out graph.jsonl\ncargo run -- scan-history . --out history.graph.jsonl\ncargo run -- inspect graph.jsonl\ncargo run -- ingest graph.jsonl --adapter dry-run\ncargo run -- ingest history.graph.jsonl --adapter embedded --data-dir .egregore\n\n# Import a rust-swe-agent / Maxwell's Daemon .traj file as agent-memory records\ncargo run -- import-traj run.traj --out agent-memory.jsonl\ncargo run -- ingest agent-memory.jsonl --adapter embedded --data-dir .egregore\n\n# Query from a JSONL graph (where-is-this-symbol, what-does-this-file-define, when-did-this-drift)\neg query symbol scan_repository --graph graph.jsonl\neg query file src/lib.rs --graph graph.jsonl\neg query drift --graph history.graph.jsonl --limit 5\neg query symbol scan_repository --graph history.graph.jsonl --at \u003ccommit-sha\u003e\neg query symbol scan_repository --graph history.graph.jsonl --as-of 2026-05-01T00:00:00Z\n\n# Query directly from an ingested AletheiaDB store\neg query symbol scan_repository --data-dir .egregore\neg query file src/lib.rs --data-dir .egregore\neg query drift --data-dir .egregore --limit 5\n\n# Flag agent notes whose cited code has drifted since recording (freshness lead, not a truth claim)\neg query evidence-freshness --graph history.graph.jsonl --stale-only\n\n# Export and verify evidence bundles (Issue #68)\neg bundle export --root-selector symbol:scan_repository --graph graph.jsonl --out bundle.json\neg bundle inspect bundle.json\neg bundle verify bundle.json --format text\n```\n\nThe primary binary is `egregore`; `eg` is also built as a short CLI alias.\n\nQuery output is newline-delimited JSON by default (`--format json`). Pass `--format text` for human-readable terminal output. See [docs/cli/query.md](docs/cli/query.md) for the stable output schema.\n\n## Primary Documents\n\n- Product vision: [docs/prd/0000-egregore-vision.md](docs/prd/0000-egregore-vision.md)\n- Code graph requirements: [docs/prd/0001-codebase-knowledge-graph.md](docs/prd/0001-codebase-knowledge-graph.md)\n- **Agent-memory domain schema (v1):** [docs/schema/agent-memory.md](docs/schema/agent-memory.md)\n- **Agent-actions and PatchArtifact schema (v1):** [docs/schema/agent-actions.md](docs/schema/agent-actions.md)\n- **Verification-evidence domain schema (v1):** [docs/schema/verification.md](docs/schema/verification.md)\n- **Project-graph domain schema (v1):** [docs/schema/project-graph.md](docs/schema/project-graph.md)\n- **User-context domain schema (v1):** [docs/schema/user-context.md](docs/schema/user-context.md)\n- **Local project/task JSONL file format (v1):** [docs/schema/local-project-jsonl.md](docs/schema/local-project-jsonl.md)\n- **Semantic drift domain schema (v1):** [docs/schema/semantic-drift.md](docs/schema/semantic-drift.md)\n- **Record schema-version policy:** [docs/schema/schema-versioning.md](docs/schema/schema-versioning.md)\n- **Producer identity envelope (v1):** [docs/schema/producer-version.md](docs/schema/producer-version.md)\n- **Redaction schema (v1):** [docs/schema/redaction.md](docs/schema/redaction.md)\n- **Bi-temporal selector grammar (v2):** [docs/schema/temporal-selectors.md](docs/schema/temporal-selectors.md)\n- **Symbol identity ADR:** [docs/adr/0004-symbol-identity.md](docs/adr/0004-symbol-identity.md)\n- Architecture decisions: [docs/adr/README.md](docs/adr/README.md)\n- **Daemon runtime discovery contract (v1):** [docs/schema/daemon-runtime.md](docs/schema/daemon-runtime.md)\n- **Daemon HTTP wire contract (v1):** [docs/schema/daemon-api.md](docs/schema/daemon-api.md)\n- **Daemon query verb set (v1):** [docs/schema/daemon-query.md](docs/schema/daemon-query.md)\n- **Repository node identity (v2):** [docs/schema/repository-identity.md](docs/schema/repository-identity.md)\n- **Evidence bundle CLI reference:** [docs/cli/bundle.md](docs/cli/bundle.md)\n- Implementation plans: [docs/plans/](docs/plans/)\n\n## Development\n\n```powershell\ncargo fmt --all\ncargo test --all-targets\ncargo test --all-targets --no-default-features\ncargo test --all-targets --features nova\ncargo test --all-targets --all-features\ncargo clippy --all-targets --all-features -- -D warnings\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadmax983%2Fegregore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmadmax983%2Fegregore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadmax983%2Fegregore/lists"}