https://github.com/evermind-ai/everalgo
EverAlgo is the algorithm library behind EverMind's memory system — stateless, storage-free, focused on extraction and ranking only. Orchestration, persistence, and routing live upstream in evermem.
https://github.com/evermind-ai/everalgo
Last synced: 1 day ago
JSON representation
EverAlgo is the algorithm library behind EverMind's memory system — stateless, storage-free, focused on extraction and ranking only. Orchestration, persistence, and routing live upstream in evermem.
- Host: GitHub
- URL: https://github.com/evermind-ai/everalgo
- Owner: EverMind-AI
- License: apache-2.0
- Created: 2026-05-11T03:09:00.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2026-06-24T03:50:33.000Z (15 days ago)
- Last Synced: 2026-06-24T04:24:41.179Z (15 days ago)
- Language: Python
- Homepage:
- Size: 2.43 MB
- Stars: 11
- Watchers: 0
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Security: SECURITY.md
- Agents: AGENTS.md
Awesome Lists containing this project
README
# EverAlgo
[](LICENSE)
[](https://www.python.org/)
[](https://pypi.org/project/everalgo-core/)
EverAlgo is the **algorithm library** behind EverMind's memory system — stateless, storage-free, focused on extraction and ranking only. Orchestration, persistence, and routing live upstream in EverOS.
## Why split EverAlgo from EverOS?
- **Algorithm engineers iterate fast.** EverAlgo is the algorithm team's home base — every change to extraction strategies, prompts, fusion math, and ranker weights happens here without going through service-layer ceremony.
- **Pure functions, easy to reason about.** No DB, no filesystem, no business state. All operators are plain in-memory transforms with explicit input / output types.
- **One codebase serves both the open-source and the commercial cloud builds.** The same `everalgo.*` packages are consumed by both editions.
The full architecture lives in [`docs/concepts/architecture.md`](docs/concepts/architecture.md).
## Repository layout
This repo is a **monorepo** of 8 distributions sharing the `everalgo.*` namespace via [PEP 420](https://peps.python.org/pep-0420/), managed with [uv workspace](https://docs.astral.sh/uv/concepts/projects/workspaces/) (all published to PyPI):
| Distribution | What it provides |
|---|---|
| [`everalgo-core`](packages/everalgo-core/) | Types, LLM client + providers, prompt helpers, testing utilities |
| [`everalgo-boundary`](packages/everalgo-boundary/) | `detect_boundaries` + `DetectionResult` + token helpers |
| [`everalgo-clustering`](packages/everalgo-clustering/) | `Cluster` value object + `cluster_by_geometry` / `cluster_by_llm` operators |
| [`everalgo-rank`](packages/everalgo-rank/) | 4 rankers (episodic / profile / case / skill) over fusion / weight / rerank toolkit |
| [`everalgo-parser`](packages/everalgo-parser/) | Multimodal raw-file → `ParsedContent` (image, audio, PDF, HTML, email, office, URL; video deferred) |
| [`everalgo-user-memory`](packages/everalgo-user-memory/) | `BoundaryDetector` + `Episode` / `Foresight` / `AtomicFact` / `Profile` extractors |
| [`everalgo-agent-memory`](packages/everalgo-agent-memory/) | `AgentBoundaryDetector` + `AgentCase` / `AgentSkill` extractors |
| [`everalgo-knowledge`](packages/everalgo-knowledge/) | `KnowledgeExtractor` pipeline + `aclassify_category` (file-based knowledge extraction) |
## 60-second quickstart
Install all packages into a shared editable venv, then run the offline examples — no API key required:
```bash
git clone git@github.com:EverMind-AI/EverAlgo.git
cd everalgo
uv sync --all-packages # editable-install all 8 packages into a shared venv
uv run pre-commit install # register the git hook (ruff + standard sanitisers)
ls .git/hooks/pre-commit # MUST exist — see AGENTS.md §3 "Common pitfall"
uv run python examples/01_boundary_chat.py # Chat → MemCell
uv run python examples/03_user_memory_episode.py # MemCell → Episode
uv run python examples/04_agent_memory_case.py # Agent trajectory → AgentCase
uv run python examples/06_full_user_memory_pipeline.py # Full pipeline
uv run pytest # workspace-wide test suite
```
> [!NOTE]
> If `.git/hooks/pre-commit` does not exist after the `install` step, hooks won't fire at `git commit` time. Verify the file every clone. Details in [`AGENTS.md`](AGENTS.md) §3.
## Code example — full user-memory pipeline
```python
import asyncio
import json
import numpy as np
from everalgo.clustering import Cluster, cluster_by_geometry
from everalgo.llm.types import ChatResponse
from everalgo.testing.fake_llm import FakeLLMClient
from everalgo.types import ChatMessage
from everalgo.user_memory import (
BoundaryDetector, EpisodeExtractor, ForesightExtractor,
AtomicFactExtractor, ProfileExtractor,
)
_BOUNDARY_JSON = json.dumps({"reasoning": "single topic", "boundaries": [], "should_wait": False})
_EPISODE_JSON = json.dumps({"title": "Alice asks about async", "content": "Alice explored async patterns."})
_FORE_JSON = json.dumps([{"content": "Alice will read the follow-up doc", "evidence": "assistant promised a doc", "start_time": "2023-11-14", "end_time": "2023-11-21", "duration_days": 7}])
_FACT_JSON = json.dumps({"atomic_facts": {"time": "Nov 14 2023", "atomic_fact": ["Alice is learning Python async."]}})
_PROFILE_JSON = json.dumps({"explicit_info": [], "implicit_traits": [{"category": "Technical", "description": "Python developer."}]})
async def main() -> None:
fake = FakeLLMClient(responses=[
ChatResponse(content=_BOUNDARY_JSON, model="fake"),
ChatResponse(content=_EPISODE_JSON, model="fake"),
ChatResponse(content=_FORE_JSON, model="fake"),
ChatResponse(content=_FACT_JSON, model="fake"),
ChatResponse(content=_PROFILE_JSON, model="fake"),
])
messages = [
ChatMessage(id="m1", role="user", content="I want to learn Python async retry patterns.", timestamp=1_700_000_000_000, sender_id="u_alice", sender_name="Alice"),
ChatMessage(id="m2", role="assistant", content="Sure — I'll send a follow-up doc next week.", timestamp=1_700_000_001_000, sender_id="assistant"),
]
# Step 1 — boundary detection
result = await BoundaryDetector(llm=fake).adetect(messages, is_final=True)
mc = result.cells[0]
# Steps 2–4 — extract user-memory products (sender_id is required, not inferred)
episode = await EpisodeExtractor(llm=fake).aextract(mc, sender_id="u_alice")
foresights = await ForesightExtractor(llm=fake).aextract(mc, sender_id="u_alice")
facts = await AtomicFactExtractor(llm=fake).aextract(mc, sender_id="u_alice")
# Step 5 — cluster the cell (caller wraps as size-1 Cluster, no LLM), then extract profile
vector = np.random.rand(2560).astype(np.float32)
existing: list[Cluster] = []
new_cluster = Cluster(centroid=vector, last_ts=mc.timestamp)
merged = cluster_by_geometry(new_cluster, existing)
if merged is None:
existing.append(new_cluster.model_copy(update={"id": "cid_001"}))
profile = await ProfileExtractor(llm=fake).aextract([mc], sender_id="u_alice")
print(f"episode: {episode.subject!r}")
print(f"foresights: {len(foresights)}, facts: {len(facts)}")
print(f"profile summary: {profile.summary!r}")
asyncio.run(main())
```
## Install — consumers
Install only the distributions you need; transitive deps are pulled automatically:
```bash
pip install everalgo-user-memory # pulls core + boundary
pip install everalgo-agent-memory # pulls core + boundary + clustering
pip install everalgo-rank # pulls core
pip install everalgo-clustering # pulls core
```
## Status & known limitations
All eight distributions are published on PyPI. Two operators are **unimplemented placeholders that raise `NotImplementedError`** — they are deliberately kept out of the public `__all__` (import the submodule directly if you want the reserved stub) and must not be relied on yet:
| Placeholder | Reachable at | Status |
|---|---|---|
| `WorkspaceMemCellExtractor` | `everalgo.boundary.workspace` | Jira / Email / Confluence slicing — not implemented |
| video parsing | `everalgo.parser.video` | deferred pending an ADR (Gemini Video vs Whisper + frame sampling) |
Everything else is fully implemented and tested: boundary detection, both clustering operators, all four rankers, the user-memory extractors (Episode / Foresight / AtomicFact / Profile), the agent-memory extractors (Case / Skill), and the knowledge extractor (KnowledgeExtractor + document classification).
## Releasing
Every distribution is released **independently**: each `packages/everalgo-*/pyproject.toml` carries its own `version = "..."` and follows its own SemVer cadence. There is no umbrella version — bumping `everalgo-rank` does not require bumping anything else.
The repo uses a **two-tier CHANGELOG**:
- [`CHANGELOG.md`](CHANGELOG.md) at the root — current-version overview table.
- `packages/everalgo-/CHANGELOG.md` per distribution — full history; ships inside the wheel.
Per-distribution changelogs are generated by [git-cliff](https://git-cliff.org/) from Conventional Commit messages on `main` (see [`cliff.toml`](cliff.toml) for the parser config; see [`AGENTS.md`](AGENTS.md) §6 for the commit-message contract).
### Cutting a release
> **The full, verified step-by-step procedure is in [`docs/releasing.md`](docs/releasing.md)** — including the canary-first / one-tag-at-a-time lessons learned in the 0.2.0 release. The steps below are a quick reference.
Steps for releasing `everalgo-clustering v0.2.0`:
1. **Bump the version.** Open an MR editing `packages/everalgo-clustering/pyproject.toml`; merge to `main` via squash.
2. **Generate the changelog fragment.**
```bash
git checkout main && git pull
uv run git-cliff \
--tag everalgo-clustering/v0.2.0 \
--include-path 'packages/everalgo-clustering/**' \
--unreleased \
--prepend packages/everalgo-clustering/CHANGELOG.md
```
3. **Review and polish.** git-cliff gives a draft, not a final — edit for clarity and add Breaking-Changes call-outs.
4. **Update the root CHANGELOG** table row for this distribution.
5. **Commit + tag + push.**
```bash
git add packages/everalgo-clustering/CHANGELOG.md CHANGELOG.md
git commit -m "📝 docs(clustering): release notes for v0.2.0"
git push origin main
git tag everalgo-clustering/v0.2.0
git push origin everalgo-clustering/v0.2.0
```
6. **Publish to PyPI.** The `.gitlab-ci.yml` `publish` stage triggers automatically on `/v` tag push using [PyPI Trusted Publishers](https://docs.pypi.org/trusted-publishers/) (GitLab OIDC, no long-lived tokens). See the CI file for prerequisites and mechanism details.
Tag format: `/v` — e.g. `everalgo-core/v0.1.0`, `everalgo-rank/v0.2.0`.
### Pre-flight checklist
- [ ] `uv run pytest` is green on `main`.
- [ ] `uv run ruff check . && uv run ruff format --check .` pass.
- [ ] `uv run mypy . && uv run pyright` pass.
- [ ] The bumped `version` honours SemVer relative to the previous tag.
- [ ] The `packages/everalgo-/CHANGELOG.md` section has been reviewed and edited.
- [ ] The root `CHANGELOG.md` table row has been updated.
## For AI coding assistants
Read [`AGENTS.md`](AGENTS.md) — the single source of truth for assistant context. `CLAUDE.md` and `.cursorrules` are symlinks to it.
## Documentation
- [`docs/index.md`](docs/index.md) — project landing page
- [`docs/installation.md`](docs/installation.md) — install + prerequisites
- [`docs/getting-started.md`](docs/getting-started.md) — 5-minute quickstart
- [`docs/concepts/`](docs/concepts/) — architecture, stateless-design, async/sync bridge
- [`docs/api/`](docs/api/) — per-package API reference
- [`docs/version-policy.md`](docs/version-policy.md) — SemVer + supported Python versions
- [`docs/contributing.md`](docs/contributing.md) — how to contribute
- [`examples/`](examples/) — runnable quickstart scripts (01 through 07)
- [`AGENTS.md`](AGENTS.md) — onboarding for AI assistants + contributors
## License
[Apache License 2.0](LICENSE).