https://github.com/robbiebusinessacc/justllm
Production LLM calls. Just the three lines. Cross-provider fallback, native caching, and reversible context compression on by default.
https://github.com/robbiebusinessacc/justllm
ai context-compression litellm llm llm-orchestration prompt-caching
Last synced: 29 days ago
JSON representation
Production LLM calls. Just the three lines. Cross-provider fallback, native caching, and reversible context compression on by default.
- Host: GitHub
- URL: https://github.com/robbiebusinessacc/justllm
- Owner: robbiebusinessacc
- License: mit
- Created: 2026-06-16T02:18:01.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2026-06-16T04:08:59.000Z (about 1 month ago)
- Last Synced: 2026-06-16T04:23:12.074Z (about 1 month ago)
- Topics: ai, context-compression, litellm, llm, llm-orchestration, prompt-caching
- Language: Python
- Homepage: https://pypi.org/project/justllm/
- Size: 77.1 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
- Roadmap: ROADMAP.md
Awesome Lists containing this project
README
# justllm
[](https://pypi.org/project/justllm/)
[](https://github.com/robbiebusinessacc/justllm/actions/workflows/ci.yml)
[](https://pypi.org/project/justllm/)
[](LICENSE)
**Production LLM calls. Just the three lines.**
```python
from justllm import LLM
llm = LLM("anthropic/claude-opus-4-8")
llm("Summarize this contract.")
```
That call already does the work you'd normally wire up yourself, on by default:
- **Context compression.** [Headroom](https://github.com/chopratejas/headroom) shrinks tool output by 50–95% before it reaches the model.
- **Prompt-cache optimization.** Cache breakpoints go where each provider wants them (Anthropic, OpenAI, Google).
- **Reliability.** Calls retry with backoff, then fail over to the next provider.
You don't call any of these yourself; they run inside `llm(...)`. To turn them off
per client: `LLM(model, compress=False, cache="off")`.
```bash
pip install 'justllm[all]'
```
## More, when you need it
You set up `llm` once (those three lines). After that, each of these is a single
call on it. Reach for the ones you need and ignore the rest:
```python
llm.stream("...") # token streaming
await llm.acall("...") # async
llm.map(prompts, concurrency=8) # many prompts at once, in order
llm.extract(Invoice, text) # structured output (validated Pydantic)
llm.chat() # multi-turn, keeps history
llm.agent(system="...").run("...") # tool-calling loop
llm.judge(output, criteria="...") # LLM-as-judge score
llm.evaluate(cases) # run + grade a test set
```
Also there, all opt-in: `llm.embed(...)`, routing (`Router` and `Cascade`),
OpenTelemetry traces with the per-call dollar cost, Langfuse-backed prompts, and
exact-match caching. Runnable versions of everything are in the
[cookbook](examples/).
Runnable recipes: **[cookbook](examples/)**
## Why
The ecosystem splits two ways. You can have powerful but heavy (LiteLLM,
LangChain), or simple but thin (aisuite, any-llm). justllm sits in the middle:
every optimization is on, and the surface stays at three lines. Keeping it that
small was most of the work.
| | justllm | LiteLLM | aisuite |
|---|---|---|---|
| three-line call | yes | yes | yes |
| cross-provider fallback | on by default | config | no |
| context compression | on by default (Headroom) | manual trim | no |
| prompt-cache optimization | on by default | passthrough | no |
| structured output | yes (instructor) | passthrough | no |
| tool-calling agent | yes (minimal) | no | no |
| surface area | tiny | large | tiny |
It runs on LiteLLM underneath, so think of it as the opinionated layer on top
rather than a replacement.
---
*Alpha. The wiring is tested on CI (Python 3.10–3.13) and the call paths are
checked against live models.*
[Cookbook](examples/) · [Roadmap](ROADMAP.md) · [Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md) · [MIT](LICENSE)