{"id":18126990,"url":"https://github.com/hoduy511/rag-langchain","last_synced_at":"2026-05-06T20:31:53.061Z","repository":{"id":260500548,"uuid":"881478550","full_name":"hoduy511/rag-langchain","owner":"hoduy511","description":"A production-ready RAG (Retrieval Augmented Generation) system built with FastAPI, LangChain, LangServe, LangSmith, Hugging Face, and Qdrant for document processing and intelligent querying.","archived":false,"fork":false,"pushed_at":"2024-12-08T12:21:21.000Z","size":1586,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-06T13:47:37.869Z","etag":null,"topics":["docker","huggingface-models","langchain","langchain-huggingface","langserve","langsmith","python3","qdrant"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hoduy511.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-31T16:47:51.000Z","updated_at":"2024-12-10T09:11:46.000Z","dependencies_parsed_at":"2025-02-12T19:49:09.298Z","dependency_job_id":"8bd3aff2-298b-4aaa-b684-43b5c39977fd","html_url":"https://github.com/hoduy511/rag-langchain","commit_stats":null,"previous_names":["hoduy511/rag-langchain"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hoduy511/rag-langchain","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoduy511%2Frag-langchain","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoduy511%2Frag-langchain/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoduy511%2Frag-langchain/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoduy511%2Frag-langchain/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hoduy511","download_url":"https://codeload.github.com/hoduy511/rag-langchain/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoduy511%2Frag-langchain/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32711027,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-06T19:35:05.142Z","status":"ssl_error","status_checked_at":"2026-05-06T19:35:03.996Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","huggingface-models","langchain","langchain-huggingface","langserve","langsmith","python3","qdrant"],"created_at":"2024-11-01T09:13:45.289Z","updated_at":"2026-05-06T20:31:53.031Z","avatar_url":"https://github.com/hoduy511.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RAG LangChain\nA production-ready RAG (Retrieval Augmented Generation) system built with FastAPI, LangChain, LangServe, LangSmith, Hugging Face, and Qdrant for document processing and intelligent querying.\n\n\n## Table of Contents\n- [Core Features](#core-features)\n- [Tech Stack](#tech-stack)\n- [Quick Start](#quick-start)\n- [API Endpoints](#api-endpoints)\n- [Components](#Components)\n- [Development](#development)\n- [Testing](#testing)\n- [Configuration](#configuration)\n\n\n## Core Features\n- PDF Document Processing with automatic chunking and metadata enrichment\n- Vector Search using Qdrant with sentence-transformers/all-MiniLM-L6-v2 embedding model for efficient document retrieval\n- Integration with google/flan-t5-base for question answering\n- RESTful API with streaming support\n- Docker containerization with multi-service architecture\n\n\n## Tech Stack\n- FastAPI + Uvicorn\n- LangChain + LangServe\n- Qdrant Vector Database\n- Hugging Face Models ([google/flan-t5-base](https://huggingface.co/google/flan-t5-base), [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2))\n- LangSmith for monitoring and debugging\n- PyPDF for document processing\n\n\n## Quick Start\n1. Clone the repository:\n```bash\ngit clone https://github.com/hoduy511/rag-langchain.git\ncd rag-langchain\n```\n\n2. Create and configure your .env file with required variables:\n```bash\ncp .env-dev .env\n```\n\n3. Start the services using `make up` command:\n```bash\nmake up\n```\n\n\n## API Endpoints\n### Base URLs\n- API Documentation: http://localhost:8000/docs\n- Interactive Playground: http://localhost:8000/rag/playground\n\n### Core Endpoints\n- `GET /health` - Health check endpoint\n- `POST /api/v1/upload` - Upload and process PDF documents\n- `POST /api/v1/query` - Query the knowledge base  \n- `POST /api/v1/search` - Perform similarity search\n\n\n## Components\n### API Layer (`src/api/`)\n- FastAPI application handling HTTP requests\n- Route definitions for document upload, querying, and search\n- Input validation and response formatting\n- CORS and middleware configuration\n\n### Core Services (`src/services/`)\n#### Document Formatter\n- Text chunking with configurable size and overlap\n- Metadata enrichment\n- UTF-8 encoding handling\n- Content cleaning and normalization\n\n#### LLM Service\n- google/flan-t5-base model integration\n- Text generation pipeline configuration\n- Token length management\n- Model parameter optimization\n\n#### PDF Service\n- PDF document processing\n- Text extraction and cleaning\n- Temporary file management\n- Chunk generation and storage\n\n#### Vector Store\n- Qdrant vector database integration\n- Document embedding using sentence-transformers\n- Similarity search functionality\n- Collection management and indexing\n\n### RAG Chain (`src/chains/`)\n- LangChain implementation for question answering\n- Integration with google/flan-t5-base model\n- Prompt management and chain composition\n- Context retrieval and response generation\n\n### Data Models (`src/models/`)\n- Pydantic schemas for request/response validation\n- Data transfer object definitions\n- Type hints and validation rules\n\n\n## Development\nAvailable make commands for development:\n- `up`: Start all services with docker-compose\n- `down`: Stop all services and remove containers\n- `logs`: View container logs in follow mode\n- `shell`: Open interactive shell in app container\n- `clean`: Remove all containers, volumes and prune system\n- `test`: Run pytest test suite\n- `format`: Format Python code with autopep8 and isort\n- `lint`: Run flake8 linter checks\n\n\n## Testing\nThe project includes comprehensive tests for all components including API endpoints, RAG chain implementation, and various services.\n\n\n## Configuration\n### Environment Variables\n- API settings\n- Qdrant vector database configuration\n- Model settings (google/flan-t5-base and sentence-transformers/all-MiniLM-L6-v2)\n- LangChain integration parameters\n- LangSmith API keys and project settings\n- Hugging Face API tokens and model configurations\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoduy511%2Frag-langchain","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhoduy511%2Frag-langchain","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoduy511%2Frag-langchain/lists"}