{"id":29403502,"url":"https://github.com/mmonire/librechat","last_synced_at":"2026-05-07T13:41:02.101Z","repository":{"id":303951916,"uuid":"1017266665","full_name":"Mmonire/LibreChat","owner":"Mmonire","description":"Simple RAG chatbot designed to answer questions about Linux, free software, and influential figures like Linus Torvalds and Richard Stallman.","archived":false,"fork":false,"pushed_at":"2025-07-10T09:38:44.000Z","size":21,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-10T17:40:55.886Z","etag":null,"topics":["chatbot","cohere","rag"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mmonire.png","metadata":{"files":{"readme":"README.markdown","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-10T09:18:56.000Z","updated_at":"2025-07-10T09:42:06.000Z","dependencies_parsed_at":"2025-07-10T17:56:16.593Z","dependency_job_id":null,"html_url":"https://github.com/Mmonire/LibreChat","commit_stats":null,"previous_names":["mmonire/librechat"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Mmonire/LibreChat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mmonire%2FLibreChat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mmonire%2FLibreChat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mmonire%2FLibreChat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mmonire%2FLibreChat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mmonire","download_url":"https://codeload.github.com/Mmonire/LibreChat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mmonire%2FLibreChat/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264637823,"owners_count":23642056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","cohere","rag"],"created_at":"2025-07-10T19:00:20.980Z","updated_at":"2026-05-07T13:41:02.016Z","avatar_url":"https://github.com/Mmonire.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🤖 LibreChat: Linux Philosophy Chatbot\n\n## 📖 Overview\n**LibreChat** is a Retrieval-Augmented Generation (RAG) chatbot designed to answer questions about Linux, free software, and influential figures like Linus Torvalds and Richard Stallman. It leverages diverse data sources, including a PDF book, web content, Wikipedia pages, and HTML files from stallman.org, to provide concise, accurate responses in Persian (up to 4 words). Built using **LangChain** and **Cohere**, LibreChat processes documents, embeds them in a vector store, and retrieves relevant information to answer 16 predefined questions for automated evaluation.\n\n## 🎯 Objectives\n- **Build a RAG Chatbot**: Implement a retrieval-augmented system to answer questions based on provided data sources.\n- **Process Diverse Data**: Load and preprocess PDF, web, Wikipedia, and HTML content into unified documents.\n- **Ensure Concise Responses**: Generate short (≤4 words) Persian answers in a structured JSON format.\n- **Achieve High Accuracy**: Correctly answer at least 12 out of 16 evaluation questions to meet project requirements.\n\n## ✨ Features\n- **Multi-Source Data Handling**:\n  - 📄 **PDF**: Extracts text from \"Just for Fun\" (Persian translation) using `PyPDFium2Loader`.\n  - 🌐 **Web**: Scrapes \"Linux and Life\" from linuxbook.ir using `WebBaseLoader`.\n  - 📚 **Wikipedia**: Loads Persian pages on Linux, free software, and key figures with `WikipediaLoader`.\n  - 🖥️ **HTML**: Processes stallman.org pages using `DirectoryLoader` and `BSHTMLLoader`.\n- **RAG Architecture**:\n  - Splits documents into chunks (`RecursiveCharacterTextSplitter`).\n  - Embeds using Cohere’s multilingual model (`embed-multilingual-light-v3.0`).\n  - Stores embeddings in a Chroma vector store for semantic search.\n  - Uses Cohere reranking (`rerank-multilingual-v3.0`) for improved retrieval.\n- **Structured Output**: Returns JSON dictionaries with `question_number` (int) and `answer` (str) for evaluation.\n- **Few-Shot Prompting**: Guides the model with example questions and answers to ensure format compliance.\n\n## 🛠 Prerequisites\nTo run this project, you need:\n- **Python** 3.9 or higher.\n- **Libraries**:\n  - `langchain`, `langchain-community`, `langchain-cohere`, `langchain-chroma`\n  - `pypdfium2`, `pandas`, `beautifulsoup4`\n- **Cohere API Key**: Obtain from [Cohere](https://cohere.ai/) and set as `COHERE_API_KEY`.\n- **Dataset Files** (in `data` folder):\n  - `justforfun_persian.pdf`\n  - `html/` (containing stallman.org HTML files)\n- **Environment**: Google Colab with GPU recommended for Hugging Face models (optional).\n\n## 📦 Installation\n1. Clone or download the repository:\n   ```bash\n   git clone \u003crepository-url\u003e\n   ```\n2. Navigate to the project directory:\n   ```bash\n   cd \u003cproject-directory\u003e\n   ```\n3. Install required libraries:\n   ```bash\n   pip install langchain langchain-community langchain-cohere langchain-chroma pypdfium2 pandas beautifulsoup4\n   ```\n4. Set up the Cohere API key:\n   ```python\n   import os\n   os.environ[\"COHERE_API_KEY\"] = \"your-api-key\"\n   ```\n5. Ensure dataset files are in the `data` folder.\n\n## 🚀 Usage\n1. Open `librechat.ipynb` in Jupyter Notebook or Google Colab.\n2. Run cells sequentially to:\n   - Load and preprocess data (PDF, web, Wikipedia, HTML).\n   - Split documents into chunks and embed them in a Chroma vector store.\n   - Set up the RAG chain with Cohere LLM and reranker.\n   - Answer 16 evaluation questions and save responses.\n3. Generate the submission file:\n   ```bash\n   python -m notebook librechat.ipynb\n   ```\n4. Output:\n   - `result.zip` containing:\n     - `librechat.ipynb`\n     - `answers.json` (16 JSON dictionaries with question numbers and answers)\n\n## 📊 Code Structure\n- **Data Loading**:\n  - Loads PDF (`PyPDFium2Loader`), web (`WebBaseLoader`), Wikipedia (`WikipediaLoader`), and HTML (`DirectoryLoader` with `BSHTMLLoader`).\n  - Saves preprocessed documents to `processed_docs.json`.\n- **Document Splitting**:\n  - Uses `RecursiveCharacterTextSplitter` (chunk size: 500, overlap: 50).\n  - Saves chunks to `splitted_docs.json`.\n- **Vector Store**:\n  - Embeds documents with `CohereEmbeddings` (`embed-multilingual-light-v3.0`).\n  - Stores in Chroma (`persist_directory=\"./vec_store_chroma\"`).\n- **RAG Chain**:\n  - Retrieves documents with `vector_store.as_retriever` (k=50).\n  - Reranks with `CohereRerank` (top_n=20).\n  - Generates answers using `ChatCohere` with a custom prompt template.\n  - Outputs JSON dictionaries via `rag_chain` function.\n- **Submission**:\n  - Combines answers (`answer1` to `answer16`) into `answers.json`.\n  - Zips with notebook into `result.zip`.\n\n## 🔍 Evaluation\n- **Questions**: 16 predefined questions (e.g., \"پرسش ۴: چه کسی بنیاد نرم‌افزارهای آزاد را بنا نهاد؟\").\n- **Output Format**: Each answer is a JSON dictionary:\n  ```json\n  {\"question_number\": 4, \"answer\": \"ریچارد استالمن\"}\n  ```\n- **Constraints**:\n  - Answers must be ≤4 words in Persian.\n  - At least 12 correct answers required for success.\n\n## 🎓 Credits\nDeveloped as part of a Quera academic project, leveraging LangChain and Cohere for RAG-based chatbot implementation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmonire%2Flibrechat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmmonire%2Flibrechat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmonire%2Flibrechat/lists"}