{"id":30657063,"url":"https://github.com/just-llms/justllms","last_synced_at":"2025-08-31T11:03:41.774Z","repository":{"id":308818169,"uuid":"1034232586","full_name":"just-llms/justllms","owner":"just-llms","description":"Production-ready Python library for multi-provider LLM orchestration","archived":false,"fork":false,"pushed_at":"2025-08-24T09:16:02.000Z","size":367,"stargazers_count":14,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-24T15:31:53.333Z","etag":null,"topics":["ai","api-wrapper","chat-completion","conversation-management","cost-optimization","deepseek","google-gemini","grok","intelligent-routing","justllms","langchain","llm","machine-learning","multi-provider","openai"],"latest_commit_sha":null,"homepage":"https://www.just-llms.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/just-llms.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-08T04:18:48.000Z","updated_at":"2025-08-24T13:25:12.000Z","dependencies_parsed_at":"2025-08-08T05:36:51.687Z","dependency_job_id":null,"html_url":"https://github.com/just-llms/justllms","commit_stats":null,"previous_names":["just-llms/justllms"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/just-llms/justllms","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/just-llms%2Fjustllms","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/just-llms%2Fjustllms/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/just-llms%2Fjustllms/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/just-llms%2Fjustllms/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/just-llms","download_url":"https://codeload.github.com/just-llms/justllms/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/just-llms%2Fjustllms/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272971549,"owners_count":25024099,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-31T02:00:09.071Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","api-wrapper","chat-completion","conversation-management","cost-optimization","deepseek","google-gemini","grok","intelligent-routing","justllms","langchain","llm","machine-learning","multi-provider","openai"],"created_at":"2025-08-31T11:03:41.150Z","updated_at":"2025-08-31T11:03:41.731Z","avatar_url":"https://github.com/just-llms.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JustLLMs\n\nA production-ready Python library that simplifies working with multiple Large Language Model providers through intelligent routing, comprehensive analytics, and enterprise-grade features.\n\n[![PyPI version](https://badge.fury.io/py/justllms.svg)](https://pypi.org/project/justllms/) [![Downloads](https://pepy.tech/badge/justllms)](https://pepy.tech/project/justllms)\n\n## Why JustLLMs?\n\nManaging multiple LLM providers is complex. You need to handle different APIs, optimize costs, monitor usage, and ensure reliability. JustLLMs solves these challenges by providing a unified interface that automatically routes requests to the best provider based on your criteria—whether that's cost, speed, or quality.\n\n## Installation\n\n```bash\n# Basic installation\npip install justllms\n\n# With PDF export capabilities\npip install \"justllms[pdf]\"\n\n# All optional dependencies (PDF export, Redis caching, advanced analytics)\npip install \"justllms[all]\"\n```\n\n**Package size**: 1.1MB | **Lines of code**: ~11K | **Dependencies**: Minimal production requirements\n\n## Quick Start\n\n```python\nfrom justllms import JustLLM\n\n# Initialize with your API keys\nclient = JustLLM({\n    \"providers\": {\n        \"openai\": {\"api_key\": \"your-openai-key\"},\n        \"google\": {\"api_key\": \"your-google-key\"},\n        \"anthropic\": {\"api_key\": \"your-anthropic-key\"}\n    }\n})\n\n# Simple completion - automatically routes to best provider\nresponse = client.completion.create(\n    messages=[{\"role\": \"user\", \"content\": \"Explain quantum computing briefly\"}]\n)\nprint(response.content)\n```\n\n## Core Features\n\n### Multi-Provider Support\nConnect to all major LLM providers with a single, consistent interface:\n- **OpenAI** (GPT-5, GPT-4, etc.) \u003cyes, you can use GPT 5 :)\u003e\n- **Google** (Gemini 2.5, Gemini 1.5 models)  \n- **Anthropic** (Claude 3.5, Claude 3 models)\n- **Azure OpenAI** (with deployment mapping)\n- **xAI Grok**, **DeepSeek**, and more\n\n```python\n# Switch between providers seamlessly\nclient = JustLLM({\n    \"providers\": {\n        \"openai\": {\"api_key\": \"your-key\"},\n        \"google\": {\"api_key\": \"your-key\"},\n        \"anthropic\": {\"api_key\": \"your-key\"}\n    }\n})\n\n# Same interface, different providers automatically chosen\nresponse1 = client.completion.create(\n    messages=[{\"role\": \"user\", \"content\": \"Explain AI\"}],\n    provider=\"openai\"  # Force specific provider\n)\n\nresponse2 = client.completion.create(\n    messages=[{\"role\": \"user\", \"content\": \"Explain AI\"}]\n    # Auto-routes to best provider based on your strategy\n)\n```\n\n### Intelligent Routing\n**The game-changing feature that sets JustLLMs apart.** Instead of manually choosing models, let our intelligent routing engine automatically select the optimal provider and model for each request based on your priorities.\n\n#### Available Strategies\n\n**🆕 Cluster-Based Routing** - *AI-Powered Query Analysis*\nOur most advanced routing strategy uses machine learning to analyze query semantics and route to the optimal model based on similarity to training data. Achieves **+7% accuracy improvement** and **-27% cost reduction** compared to single-model approaches.\n\n```python\n# Cluster-based routing (recommended for production)\nclient = JustLLM({\n    \"providers\": {...},\n    \"routing\": {\"strategy\": \"cluster\"}\n})\n```\n\n*Based on research from [Beyond GPT-5: Making LLMs Cheaper and Better via Performance–Efficiency Optimized Routing](https://arxiv.org/pdf/2508.12631) - AvengersPro framework*\n\n**Traditional Routing Strategies**\n\n```python\n# Cost-optimized: Always picks the cheapest option\nclient = JustLLM({\n    \"providers\": {...},\n    \"routing\": {\"strategy\": \"cost\"}\n})\n\n# Speed-optimized: Prioritizes fastest response times\nclient = JustLLM({\n    \"providers\": {...},\n    \"routing\": {\"strategy\": \"latency\"}\n})\n\n# Quality-optimized: Uses the best models for complex tasks\nclient = JustLLM({\n    \"providers\": {...},\n    \"routing\": {\"strategy\": \"quality\"}\n})\n\n# Task-based: Automatically detects query type and routes accordingly\nclient = JustLLM({\n    \"providers\": {...},\n    \"routing\": {\"strategy\": \"task\"}\n})\n```\n\n#### How Cluster Routing Works\n1. **Query Analysis**: Your request is embedded using Qwen3-Embedding-0.6B\n2. **Cluster Matching**: Finds the most similar cluster from pre-trained data\n3. **Model Selection**: Routes to the best-performing model for that cluster\n4. **Fallback**: Falls back to quality-based routing if needed\n\n**Result**: Up to 60% cost reduction while improving accuracy, with automatic failover to backup providers.\n\n### Real-time Streaming\nFull streaming support with proper token handling across all providers:\n\n```python\nstream = client.completion.create(\n    messages=[{\"role\": \"user\", \"content\": \"Write a short story\"}],\n    stream=True\n)\n\nfor chunk in stream:\n    print(chunk.content, end=\"\", flush=True)\n```\n\n### Conversation Management\nBuilt-in conversation state management with context preservation:\n\n```python\n# Create client\nconversation = Conversation(client=client)\n\n# Set system message\nconversation.add_system_message(\"You are a helpful math tutor. Keep answers concise.\")\n\n# Turn 1\nresponse = conversation.send(\"What is 15 + 25?\")\n\n# Turn 2 - Context is automatically preserved\nresponse = conversation.send(\"Now divide that by 8\")\n\n# Get conversation stats\nhistory = conversation.get_history()\n```\n**Conversation Features:**\n- **Auto-save**: Persist conversations automatically\n- **Context management**: Smart context window handling\n- **Export/Import**: JSON, Markdown, and TXT formats\n- **Analytics**: Track usage, costs, and performance per conversation\n- **Search**: Find conversations by content or metadata\n\n### Smart Caching\nIntelligent response caching that dramatically reduces costs and improves response times:\n\n```python\nclient = JustLLM({\n    \"providers\": {...},\n    \"caching\": {\n        \"enabled\": True,\n        \"ttl\": 3600,  # 1 hour\n        \"max_size\": 1000\n    }\n})\n\n# First call - cache miss\nresponse1 = client.completion.create(\n    messages=[{\"role\": \"user\", \"content\": \"What is AI?\"}]\n)  # ~2 seconds, full cost\n\n# Second call - cache hit\nresponse2 = client.completion.create(\n    messages=[{\"role\": \"user\", \"content\": \"What is AI?\"}]\n)  # ~50ms, no cost\n```\n\n### Enterprise Analytics\n**Comprehensive usage tracking and cost analysis** that gives you complete visibility into your LLM operations. Unlike other solutions that require external tools, JustLLMs provides built-in analytics that finance and engineering teams actually need.\n\n#### What You Get\n- **Cross-provider metrics**: Compare performance across providers\n- **Cost tracking**: Detailed cost analysis per model/provider\n- **Performance insights**: Latency, throughput, success rates\n- **Export capabilities**: CSV, PDF with charts\n- **Time series analysis**: Usage patterns over time\n- **Top models/providers**: Usage and cost rankings\n\n```python\n# Generate detailed reports\nreport = client.analytics.generate_report()\nprint(f\"Total requests: {report.cross_provider_metrics.total_requests}\")\nprint(f\"Total cost: ${report.cross_provider_metrics.total_cost:.2f}\")\nprint(f\"Fastest provider: {report.cross_provider_metrics.fastest_provider}\")\nprint(f\"Cost per request: ${report.cross_provider_metrics.avg_cost_per_request:.4f}\")\n\n# Get granular insights\nprint(f\"Cache hit rate: {report.performance_metrics.cache_hit_rate:.1f}%\")\nprint(f\"Token efficiency: {report.optimization_suggestions.token_savings:.1f}%\")\n\n# Export reports for finance teams\nfrom justllms.analytics.reports import CSVExporter, PDFExporter\ncsv_exporter = CSVExporter()\ncsv_exporter.export(report, \"monthly_llm_costs.csv\")\n\npdf_exporter = PDFExporter(include_charts=True)\npdf_exporter.export(report, \"executive_summary.pdf\")\n```\n\n**Business Impact**: Teams typically save 40-70% on LLM costs within the first month by identifying usage patterns and optimizing model selection.\n\n### Unified LLM Interface\n**Streamlined access to multiple LLM providers** with intelligent routing, comprehensive analytics, and enterprise-grade features for production deployments.\n\n### Business Rule Validation\n**Enterprise-grade content filtering and compliance** built for regulated industries. Ensure your LLM applications meet security, privacy, and business requirements without custom development.\n\n#### Compliance Features\n- **PII Detection** - Automatically detect and handle social security numbers, credit cards, phone numbers\n- **Content Filtering** - Block inappropriate content, profanity, or sensitive topics\n- **Custom Business Rules** - Define your own validation logic with regex patterns or custom functions\n- **Audit Trail** - Complete logging of all validation actions for compliance reporting\n\n```python\nfrom justllms.validation import ValidationConfig, BusinessRule, RuleType, ValidationAction\n\nclient = JustLLM({\n    \"providers\": {...},\n    \"validation\": ValidationConfig(\n        enabled=True,\n        business_rules=[\n            # Block sensitive data patterns\n            BusinessRule(\n                name=\"no_ssn\",\n                type=RuleType.PATTERNS,\n                pattern=r\"\\\\b\\\\d{3}-\\\\d{2}-\\\\d{4}\\\\b\",\n                action=ValidationAction.BLOCK,\n                message=\"SSN detected - request blocked for privacy\"\n            ),\n            # Content filtering\n            BusinessRule(\n                name=\"professional_content\",\n                type=RuleType.CONTENT_FILTER,\n                categories=[\"hate\", \"violence\", \"adult\"],\n                action=ValidationAction.SANITIZE\n            ),\n            # Custom business logic\n            BusinessRule(\n                name=\"company_policy\",\n                type=RuleType.CUSTOM,\n                validator=lambda content: \"competitor\" not in content.lower(),\n                action=ValidationAction.WARN\n            )\n        ],\n        # Compliance presets\n        compliance_mode=\"GDPR\",  # or \"HIPAA\", \"PCI_DSS\"\n        audit_logging=True\n    )\n})\n\n# All requests are automatically validated\nresponse = client.completion.create(\n    messages=[{\"role\": \"user\", \"content\": \"My SSN is 123-45-6789\"}]\n)\n# This request would be blocked and logged for compliance\n```\n\n**Regulatory Compliance**: Built-in support for major compliance frameworks saves months of custom security development.\n\n## Advanced Usage\n\n### Async Operations\nFull async/await support for high-performance applications:\n\n```python\nimport asyncio\n\nasync def process_batch():\n    tasks = []\n    for prompt in prompts:\n        task = client.completion.acreate(\n            messages=[{\"role\": \"user\", \"content\": prompt}]\n        )\n        tasks.append(task)\n    \n    responses = await asyncio.gather(*tasks)\n    return responses\n```\n\n### Error Handling \u0026 Reliability\nAutomatic retries and fallback providers ensure high availability:\n\n```python\nclient = JustLLM({\n    \"providers\": {...},\n    \"retry\": {\n        \"max_attempts\": 3,\n        \"backoff_factor\": 2,\n        \"retry_on\": [\"timeout\", \"rate_limit\", \"server_error\"]\n    }\n})\n\n# Automatically retries on failures\ntry:\n    response = client.completion.create(\n        messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n        provider=\"invalid-provider\"  # Will fail and retry\n    )\nexcept Exception as e:\n    print(f\"All retries failed: {e}\")\n```\n\n### Configuration Management\nFlexible configuration with environment variable support:\n\n```python\n# Environment-based config\nimport os\nclient = JustLLM({\n    \"providers\": {\n        \"openai\": {\"api_key\": os.getenv(\"OPENAI_API_KEY\")},\n        \"azure_openai\": {\n            \"api_key\": os.getenv(\"AZURE_OPENAI_KEY\"),\n            \"resource_name\": os.getenv(\"AZURE_RESOURCE_NAME\"),\n            \"api_version\": \"2024-12-01-preview\"\n        }\n    }\n})\n\n# File-based config\nimport yaml\nwith open(\"config.yaml\") as f:\n    config = yaml.safe_load(f)\nclient = JustLLM(config)\n```\n\n## 🏆 Comparison with Alternatives\n\n| Feature | JustLLMs | LangChain | LiteLLM | OpenAI SDK | Haystack |\n|---------|----------|-----------|---------|------------|----------|\n| **Package Size** | 1.1MB | ~50MB | ~5MB | ~1MB | ~20MB |\n| **Setup Complexity** | Simple config | Complex chains | Medium | Simple | Complex |\n| **Multi-Provider** | ✅ 6+ providers | ✅ Many integrations | ✅ 100+ providers | ❌ OpenAI only | ✅ Limited LLMs |\n| **Intelligent Routing** | ✅ Cost/speed/quality | ❌ Manual only | ⚠️ Basic routing | ❌ None | ❌ Pipeline-based |\n| **Built-in Analytics** | ✅ Enterprise-grade | ❌ External tools needed | ⚠️ Basic metrics | ❌ None | ⚠️ Pipeline metrics |\n| **Conversation Management** | ✅ Full lifecycle | ⚠️ Memory components | ❌ None | ❌ Manual handling | ✅ Dialog systems |\n| **Business Rules** | ✅ Content validation | ❌ Custom implementation | ❌ None | ❌ None | ⚠️ Custom filters |\n| **Cost Optimization** | ✅ Automatic routing | ❌ Manual optimization | ⚠️ Basic cost tracking | ❌ None | ❌ None |\n| **Streaming Support** | ✅ All providers | ✅ Provider-dependent | ✅ Most providers | ✅ OpenAI only | ⚠️ Limited |\n| **Production Ready** | ✅ Out of the box | ⚠️ Requires setup | ✅ Minimal setup | ⚠️ Basic features | ✅ Complex setup |\n| **Caching** | ✅ Multi-backend | ⚠️ Custom implementation | ✅ Basic caching | ❌ None | ✅ Document stores |\n\n## Enterprise Configuration\n\nFor production deployments with advanced features:\n\n```python\nenterprise_config = {\n    \"providers\": {\n        \"azure_openai\": {\n            \"api_key\": os.getenv(\"AZURE_OPENAI_KEY\"),\n            \"resource_name\": \"my-enterprise-resource\",\n            \"deployment_mapping\": {\n                \"gpt-4\": \"my-gpt4-deployment\",\n                \"gpt-3.5-turbo\": \"my-gpt35-deployment\"\n            }\n        },\n        \"anthropic\": {\"api_key\": os.getenv(\"ANTHROPIC_KEY\")},\n        \"google\": {\"api_key\": os.getenv(\"GOOGLE_KEY\")}\n    },\n    \"routing\": {\n        \"strategy\": \"cost\",\n        \"fallback_provider\": \"azure_openai\",\n        \"fallback_model\": \"gpt-3.5-turbo\"\n    },\n    \"validation\": {\n        \"enabled\": True,\n        \"business_rules\": [\n            # PII detection, content filtering, compliance rules\n        ]\n    },\n    \"analytics\": {\n        \"enabled\": True,\n        \"track_usage\": True,\n        \"track_performance\": True\n    },\n    \"caching\": {\n        \"enabled\": True,\n        \"backend\": \"redis\",\n        \"ttl\": 3600\n    },\n    \"conversations\": {\n        \"backend\": \"disk\",\n        \"auto_save\": True,\n        \"auto_title\": True,\n        \"max_context_tokens\": 8000\n    }\n}\n\nclient = JustLLM(enterprise_config)\n```\n## License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n[![Star History Chart](https://api.star-history.com/svg?repos=just-llms/justllms\u0026type=Date)](https://www.star-history.com/#just-llms/justllms\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjust-llms%2Fjustllms","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjust-llms%2Fjustllms","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjust-llms%2Fjustllms/lists"}