{"id":44998274,"url":"https://github.com/zakariaf/rag-cache","last_synced_at":"2026-02-18T22:02:43.291Z","repository":{"id":323633415,"uuid":"1094054064","full_name":"zakariaf/RAG-Cache","owner":"zakariaf","description":null,"archived":false,"fork":false,"pushed_at":"2025-11-28T10:40:47.000Z","size":795,"stargazers_count":4,"open_issues_count":17,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-30T18:10:18.460Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zakariaf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-11T07:33:18.000Z","updated_at":"2025-11-30T15:20:19.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/zakariaf/RAG-Cache","commit_stats":null,"previous_names":["zakariaf/rag-cache"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zakariaf/RAG-Cache","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakariaf%2FRAG-Cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakariaf%2FRAG-Cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakariaf%2FRAG-Cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakariaf%2FRAG-Cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zakariaf","download_url":"https://codeload.github.com/zakariaf/RAG-Cache/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakariaf%2FRAG-Cache/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29596332,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T20:59:56.587Z","status":"ssl_error","status_checked_at":"2026-02-18T20:58:41.434Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-18T22:02:42.129Z","updated_at":"2026-02-18T22:02:43.272Z","avatar_url":"https://github.com/zakariaf.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RAG Cache\n\nA high-performance caching layer for Large Language Model (LLM) queries with exact and semantic matching capabilities.\n\n## 🎯 What is RAG Cache?\n\nRAG Cache sits between your application and LLM providers (OpenAI, Anthropic) to:\n\n- **💰 Save Money** - Avoid paying for duplicate queries\n- **⚡ Reduce Latency** - Return cached responses in ~2ms instead of ~10 seconds\n- **🛡️ Increase Reliability** - Reduce dependency on external APIs\n\n### How It Works\n\n```\nUser Query → Exact Match (Redis) → Semantic Match (Qdrant) → LLM (if needed)\n                   ↓                       ↓                      ↓\n              ~1ms hit              ~450ms hit              ~8,500ms\n```\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n- Python 3.11+\n- Docker \u0026 Docker Compose\n- OpenAI API key\n\n### Installation\n\n```bash\n# Clone the repository\ngit clone \u003cyour-repo\u003e\ncd \"Rag cache\"\n\n# Create virtual environment\npython3.11 -m venv venv\nsource venv/bin/activate\n\n# Install dependencies\npip install -r requirements.txt\n\n# Configure environment\ncp .env.example .env\n# Edit .env and add your OPENAI_API_KEY\n\n# Start Redis \u0026 Qdrant\ndocker-compose up -d redis qdrant\n\n# Run the application\nuvicorn app.main:app --host 0.0.0.0 --port 8000\n```\n\n### Verify Installation\n\n```bash\n# Health check\ncurl http://localhost:8000/health\n```\n\nExpected response:\n```json\n{\n  \"status\": \"healthy\",\n  \"environment\": \"development\",\n  \"version\": \"0.1.0\"\n}\n```\n\n## 📡 API Usage\n\n### Query Endpoint\n\n```bash\ncurl -X POST http://localhost:8000/api/v1/query \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"query\": \"What is machine learning?\",\n    \"use_cache\": true\n  }'\n```\n\n**First Request (Cache Miss):**\n```json\n{\n  \"response\": \"Machine learning is a subset of AI...\",\n  \"provider\": \"openai\",\n  \"model\": \"gpt-4o-mini-2024-07-18\",\n  \"usage\": {\n    \"prompt_tokens\": 12,\n    \"completion_tokens\": 418,\n    \"total_tokens\": 430\n  },\n  \"cache_info\": {\n    \"cache_hit\": false,\n    \"cache_type\": null\n  },\n  \"latency_ms\": 10090.93\n}\n```\n\n**Second Request (Exact Cache Hit - Redis):**\n```json\n{\n  \"response\": \"Machine learning is a subset of AI...\",\n  \"cache_info\": {\n    \"cache_hit\": true,\n    \"cache_type\": \"exact\"\n  },\n  \"latency_ms\": 1.11\n}\n```\n\n**Similar Query (Semantic Cache Hit - Qdrant):**\n```bash\ncurl -X POST http://localhost:8000/api/v1/query \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"query\": \"Can you explain machine learning?\", \"use_cache\": true}'\n```\n```json\n{\n  \"response\": \"Machine learning is a subset of AI...\",\n  \"cache_info\": {\n    \"cache_hit\": true,\n    \"cache_type\": \"semantic\",\n    \"similarity_score\": 0.8563\n  },\n  \"latency_ms\": 457.89\n}\n```\n\n### Other Endpoints\n\n| Endpoint | Method | Description |\n|----------|--------|-------------|\n| `/health` | GET | Health check |\n| `/docs` | GET | Swagger UI documentation |\n| `/api/v1/query` | POST | Process query with caching |\n| `/api/v1/metrics` | GET | Application metrics |\n| `/api/v1/metrics/prometheus` | GET | Prometheus format metrics |\n\n## 📊 Real Performance Tests\n\nTests performed on November 27, 2025:\n\n### Cache Hit Performance\n\n| Test | Cache Type | Latency | Speedup |\n|------|------------|---------|---------|\n| First Query (Cache Miss) | LLM Call | 8,573ms | baseline |\n| Same Query (Exact Match) | **Redis** | 1.11ms | **7,723x faster** |\n| Similar Query (Semantic Match) | **Qdrant** | 457ms | **18.7x faster** |\n| Different Query (Cache Miss) | LLM Call | 8,154ms | - |\n\n### Semantic Cache Example\n\n```bash\n# Original cached query\n\"What is deep learning?\"\n\n# Similar query (semantic match)\n\"Can you explain deep learning to me?\"\n# → Similarity Score: 0.856\n# → Latency: 457ms (instead of ~8,500ms)\n# → Saved: 811 tokens\n```\n\n### Cache Types Explained\n\n| Cache | Storage | Latency | Use Case |\n|-------|---------|---------|----------|\n| **Exact** | Redis | ~1ms | Identical queries |\n| **Semantic** | Qdrant | ~450ms | Similar questions |\n| **Miss** | LLM API | ~8,500ms | New unique queries |\n\n## 📈 Metrics\n\n```bash\ncurl http://localhost:8000/api/v1/metrics\n```\n\n```json\n{\n  \"application\": {\n    \"name\": \"RAGCache\",\n    \"environment\": \"development\",\n    \"version\": \"0.1.0\"\n  },\n  \"pipeline\": {\n    \"total_requests\": 0,\n    \"cache_hits\": 0,\n    \"cache_hit_rate\": 0.0,\n    \"avg_latency_ms\": 0.0\n  },\n  \"cache\": {\n    \"total_keys\": 286,\n    \"memory_used_bytes\": 1585248,\n    \"hits\": 2,\n    \"misses\": 2032,\n    \"hit_rate\": 0.1\n  },\n  \"config\": {\n    \"semantic_cache_enabled\": true,\n    \"exact_cache_enabled\": true,\n    \"similarity_threshold\": 0.85,\n    \"cache_ttl_seconds\": 3600\n  }\n}\n```\n\n### Prometheus Metrics\n\n```bash\ncurl http://localhost:8000/api/v1/metrics/prometheus\n```\n\n```\n# HELP ragcache_info Application information\n# TYPE ragcache_info gauge\nragcache_info{version=\"0.1.0\",environment=\"development\"} 1\n\n# HELP ragcache_requests_total Total requests processed\n# TYPE ragcache_requests_total counter\nragcache_requests_total 0\n\n# HELP ragcache_cache_hits_total Total cache hits\n# TYPE ragcache_cache_hits_total counter\nragcache_cache_hits_total 0\n\n# HELP ragcache_redis_keys Total Redis keys\n# TYPE ragcache_redis_keys gauge\nragcache_redis_keys 283\n```\n\n## 🏗️ Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│                    Your Application                              │\n└────────────────────────────────┬────────────────────────────────┘\n                                 │ HTTP\n                                 ▼\n┌─────────────────────────────────────────────────────────────────┐\n│                      RAG Cache (FastAPI)                         │\n│                                                                  │\n│  ┌──────────────────────────────────────────────────────────┐   │\n│  │                    Query Service                          │   │\n│  │  1. Check Redis (exact match)                             │   │\n│  │  2. Check Qdrant (semantic match)                         │   │\n│  │  3. Call LLM if no cache hit                              │   │\n│  │  4. Store response in cache                               │   │\n│  └──────────────────────────────────────────────────────────┘   │\n│                                                                  │\n│  ┌────────────┐  ┌────────────┐  ┌──────────────────────────┐   │\n│  │   Redis    │  │   Qdrant   │  │    LLM Providers         │   │\n│  │  (Exact)   │  │ (Semantic) │  │  OpenAI / Anthropic      │   │\n│  └────────────┘  └────────────┘  └──────────────────────────┘   │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n## ⚙️ Configuration\n\n### Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `OPENAI_API_KEY` | - | OpenAI API key (required) |\n| `ANTHROPIC_API_KEY` | - | Anthropic API key (optional) |\n| `REDIS_HOST` | `localhost` | Redis host |\n| `QDRANT_HOST` | `localhost` | Qdrant host |\n| `SEMANTIC_SIMILARITY_THRESHOLD` | `0.85` | Similarity threshold (0.0-1.0) |\n| `CACHE_TTL_SECONDS` | `3600` | Cache TTL in seconds |\n| `DEFAULT_MODEL` | `gpt-3.5-turbo` | Default LLM model |\n\n## 📁 Project Structure\n\n```\napp/\n├── api/              # FastAPI routes \u0026 middleware\n│   ├── routes/       # API endpoints\n│   └── middleware/   # Auth, rate limiting, logging\n├── cache/            # Redis \u0026 Qdrant clients\n├── llm/              # LLM provider integrations\n│   ├── openai_provider.py\n│   └── anthropic_provider.py\n├── embeddings/       # Text embedding generation\n├── pipeline/         # Query processing pipeline\n├── monitoring/       # Prometheus metrics \u0026 alerts\n├── optimization/     # Performance optimization\n├── models/           # Pydantic data models\n├── services/         # Business logic\n└── utils/            # Utilities (logger, hasher)\n```\n\n## 🧪 Running Tests\n\n```bash\n# Run all tests\npytest\n\n# Run with coverage\npytest --cov=app\n\n# Run specific test file\npytest tests/unit/test_config.py -v\n```\n\n## 🐳 Docker\n\n### Start Services\n\n```bash\n# Start all services\ndocker-compose up -d\n\n# Start only Redis and Qdrant\ndocker-compose up -d redis qdrant\n\n# View logs\ndocker-compose logs -f\n\n# Stop services\ndocker-compose down\n```\n\n### Production Deployment\n\n```bash\n# Use production compose file\ndocker-compose -f docker-compose.prod.yml up -d\n\n# Or use deployment script\n./scripts/deploy.sh production\n```\n\n## 💰 Cost Savings Example\n\n| Scenario | Without Cache | With Cache (50% hit rate) |\n|----------|---------------|---------------------------|\n| 10,000 queries/day | ~$50/day | ~$25/day |\n| Average latency | ~10,000ms | ~5,000ms |\n| P95 latency | ~15,000ms | ~2ms (cache hit) |\n\n## 📚 Documentation\n\n- [API Documentation](docs/API.md)\n- [Architecture](docs/ARCHITECTURE.md)\n- [Deployment Guide](docs/DEPLOYMENT.md)\n- [Configuration Guide](docs/CONFIGURATION.md)\n- [Testing Guide](docs/TESTING.md)\n- [Troubleshooting](docs/TROUBLESHOOTING.md)\n- [Security Best Practices](docs/SECURITY.md)\n- [Performance Tuning](docs/PERFORMANCE.md)\n\n## 🛠️ Tech Stack\n\n| Component | Technology |\n|-----------|------------|\n| Language | Python 3.11 |\n| Framework | FastAPI |\n| Exact Cache | Redis 7.2 |\n| Semantic Cache | Qdrant 1.6 |\n| LLM Providers | OpenAI, Anthropic |\n| Embeddings | sentence-transformers |\n| Testing | pytest |\n| Containerization | Docker |\n\n## 🤝 Contributing\n\nSee [CONTRIBUTING.md](docs/CONTRIBUTING.md) for guidelines.\n\n## 📄 License\n\nMIT License - See [LICENSE](LICENSE) file.\n\n---\n\n**Built with ❤️**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakariaf%2Frag-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzakariaf%2Frag-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakariaf%2Frag-cache/lists"}