{"id":51189105,"url":"https://github.com/robbiebusinessacc/justllm","last_synced_at":"2026-06-27T13:03:25.124Z","repository":{"id":365140863,"uuid":"1270737419","full_name":"robbiebusinessacc/justllm","owner":"robbiebusinessacc","description":"Production LLM calls. Just the three lines. Cross-provider fallback, native caching, and reversible context compression on by default.","archived":false,"fork":false,"pushed_at":"2026-06-16T04:08:59.000Z","size":79,"stargazers_count":0,"open_issues_count":6,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-16T04:23:12.074Z","etag":null,"topics":["ai","context-compression","litellm","llm","llm-orchestration","prompt-caching"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/justllm/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robbiebusinessacc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-16T02:18:01.000Z","updated_at":"2026-06-16T04:09:03.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/robbiebusinessacc/justllm","commit_stats":null,"previous_names":["robbiebusinessacc/justllm"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/robbiebusinessacc/justllm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robbiebusinessacc%2Fjustllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robbiebusinessacc%2Fjustllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robbiebusinessacc%2Fjustllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robbiebusinessacc%2Fjustllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robbiebusinessacc","download_url":"https://codeload.github.com/robbiebusinessacc/justllm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robbiebusinessacc%2Fjustllm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34854196,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-27T02:00:06.362Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","context-compression","litellm","llm","llm-orchestration","prompt-caching"],"created_at":"2026-06-27T13:03:23.861Z","updated_at":"2026-06-27T13:03:25.119Z","avatar_url":"https://github.com/robbiebusinessacc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# justllm\n\n[![PyPI](https://img.shields.io/pypi/v/justllm)](https://pypi.org/project/justllm/)\n[![CI](https://github.com/robbiebusinessacc/justllm/actions/workflows/ci.yml/badge.svg)](https://github.com/robbiebusinessacc/justllm/actions/workflows/ci.yml)\n[![Python](https://img.shields.io/pypi/pyversions/justllm)](https://pypi.org/project/justllm/)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n\n**Production LLM calls. Just the three lines.**\n\n![justllm demo](assets/demo.gif)\n\n```python\nfrom justllm import LLM\n\nllm = LLM(\"anthropic/claude-opus-4-8\")\nllm(\"Summarize this contract.\")\n```\n\nThat call already does the work you'd normally wire up yourself, on by default:\n\n- **Context compression.** [Headroom](https://github.com/chopratejas/headroom) shrinks tool output by 50–95% before it reaches the model.\n- **Prompt-cache optimization.** Cache breakpoints go where each provider wants them (Anthropic, OpenAI, Google).\n- **Reliability.** Calls retry with backoff, then fail over to the next provider.\n\nYou don't call any of these yourself; they run inside `llm(...)`. To turn them off\nper client: `LLM(model, compress=False, cache=\"off\")`.\n\n```bash\npip install 'justllm[all]'\n```\n\n## More, when you need it\n\nYou set up `llm` once (those three lines). After that, each of these is a single\ncall on it. Reach for the ones you need and ignore the rest:\n\n```python\nllm.stream(\"...\")                    # token streaming\nawait llm.acall(\"...\")               # async\nllm.map(prompts, concurrency=8)      # many prompts at once, in order\nllm.extract(Invoice, text)           # structured output (validated Pydantic)\nllm.chat()                           # multi-turn, keeps history\nllm.agent(system=\"...\").run(\"...\")   # tool-calling loop\nllm.judge(output, criteria=\"...\")    # LLM-as-judge score\nllm.evaluate(cases)                  # run + grade a test set\n```\n\nAlso there, all opt-in: `llm.embed(...)`, routing (`Router` and `Cascade`),\nOpenTelemetry traces with the per-call dollar cost, Langfuse-backed prompts, and\nexact-match caching. Runnable versions of everything are in the\n[cookbook](examples/).\n\nRunnable recipes: **[cookbook](examples/)**\n\n## Why\n\nThe ecosystem splits two ways. You can have powerful but heavy (LiteLLM,\nLangChain), or simple but thin (aisuite, any-llm). justllm sits in the middle:\nevery optimization is on, and the surface stays at three lines. Keeping it that\nsmall was most of the work.\n\n| | justllm | LiteLLM | aisuite |\n|---|---|---|---|\n| three-line call | yes | yes | yes |\n| cross-provider fallback | on by default | config | no |\n| context compression | on by default (Headroom) | manual trim | no |\n| prompt-cache optimization | on by default | passthrough | no |\n| structured output | yes (instructor) | passthrough | no |\n| tool-calling agent | yes (minimal) | no | no |\n| surface area | tiny | large | tiny |\n\nIt runs on LiteLLM underneath, so think of it as the opinionated layer on top\nrather than a replacement.\n\n---\n\n*Alpha. The wiring is tested on CI (Python 3.10–3.13) and the call paths are\nchecked against live models.*\n\n[Cookbook](examples/) · [Roadmap](ROADMAP.md) · [Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md) · [MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobbiebusinessacc%2Fjustllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobbiebusinessacc%2Fjustllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobbiebusinessacc%2Fjustllm/lists"}