{"id":15175510,"url":"https://github.com/anasaber/rag_in_cpu","last_synced_at":"2025-10-06T10:31:23.779Z","repository":{"id":248772139,"uuid":"829650843","full_name":"AnasAber/RAG_in_CPU","owner":"AnasAber","description":"This repo is for advanced RAG systems, each branch will represent a project based on RAG.","archived":false,"fork":false,"pushed_at":"2024-10-15T23:12:19.000Z","size":12204,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-24T23:37:58.576Z","etag":null,"topics":["advanced-rag","chat-application","chat-with-pdf","chromadb","cohere","cohere-api","faiss","faiss-vector-database","generative-ai","groq","groq-api","langchain","llms","rag","rag-implementation","rag-pipeline","redis","redis-cache"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AnasAber.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-16T21:38:28.000Z","updated_at":"2024-10-16T19:10:51.000Z","dependencies_parsed_at":"2024-10-17T12:03:34.390Z","dependency_job_id":null,"html_url":"https://github.com/AnasAber/RAG_in_CPU","commit_stats":{"total_commits":15,"total_committers":1,"mean_commits":15.0,"dds":0.0,"last_synced_commit":"4cc084fcc72dd1413b36d0fa2f2301eb6ac74348"},"previous_names":["anasaber/rag_in_cpu"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnasAber%2FRAG_in_CPU","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnasAber%2FRAG_in_CPU/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnasAber%2FRAG_in_CPU/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnasAber%2FRAG_in_CPU/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AnasAber","download_url":"https://codeload.github.com/AnasAber/RAG_in_CPU/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235519926,"owners_count":19003201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["advanced-rag","chat-application","chat-with-pdf","chromadb","cohere","cohere-api","faiss","faiss-vector-database","generative-ai","groq","groq-api","langchain","llms","rag","rag-implementation","rag-pipeline","redis","redis-cache"],"created_at":"2024-09-27T12:39:20.310Z","updated_at":"2025-10-06T10:31:18.210Z","avatar_url":"https://github.com/AnasAber.png","language":"Python","readme":"# Chat with your Resume, an advanced RAG system\nA tool for chatting with your resume, as simple as that.\n\n### FrontEnd technologies\n- Used React for a chat interface\n\n## Models Used\n- Language Model (LLM): Leveraged Groq API with `llama3-70b-8192`.\n- Embedding Model: Utilized `nomic-embed-text-v1` using HuggingFace API.\n- Reranker: Utilized `rerank-english-v2.0` using Cohere API.\n\n## System Overview\n\n![System Architecture Diagram](src/images/new.png)\n\nThe RAG system consists of the following components:\n\n#### Chunking and Embedding:\n\nText data is chunked into manageable pieces, with an additional summary of the resume.\nEach chunk is embedded using a model from HuggingFace.\nEmbeddings are stored in a vector database (ChromaDB).\n\n### Caching\nThe system is able to cache previous questions and their answers, to retrieve the content automatically when the question is repeated. \n\n#### Retrieval and Reranking:\nRelevant chunks are retrieved from ChromaDB based on the query.\nRetrieved chunks are reranked using the Cohere API to ensure the most relevant chunks are prioritized.\n\n#### Response Generation:\nThe top-ranked chunks are passed to the Llama model (via Groq API) to generate a coherent and relevant response.\n\n### How to start\n\n1. Clone the repository\n```bash\ngit clone https://github.com/AnasAber/RAG_in_CPU.git\n```\n\n2. Install the dependencies\n```bash\npip install -r requirements.txt\n```\n\n3. Rename .env.example to .env, and Set up the environment variables:\n```bash\nexport GROQ_API_KEY=\"your_groq_key\"\nexport COHERE_API_KEY=\"your_cohere_key\"\nexport HUGGINGFACE_API_KEY=\"your_huggingFace_key\"\n```\n\n4. Run the `app.py` file\n```bash\npython app.py\n```\n\n\n5. To install and use Redis, Create a new terminal and follow this tutorial:\n\nYou'll first need to enable WSL2 (Windows Subsystem for Linux): https://learn.microsoft.com/en-us/windows/wsl/install\n\nNext, type this command:\n```bash\nsudo service redis-server start\n```\n\n7. Move to `frontend` folder:\n```bash\nnpm install\nnpm start\n```\n\nThis project is not deployed yet as I'm still working on the frontend, and the cache mechanism needs to get optimized.\n\nThis project's RAG uses semantic search using ChromaDB and FAISS, I'll work on doing a combination of Hybrid Search and a HyDE following the best practices of RAG mentioned in the following paper: [link](https://arxiv.org/html/2407.01219v1#:~:text=A%20typical%20RAG%20workflow%20usually,based%20on%20their%20relevance%20to)\n\n![System Architecture Diagram](src/images/x1.png)\n\nIf you encounter an error just hit me up, make a pull request, or report an issue, and I'll happily respond.\n\n### Disadvantages\n- For cohere API, it's free for testing and unlimited, but not for production use as it's paid\n\n### Next goals\n\n- Optimize the caching mechanism\n- Create a better frontend.\n- See if there's a fast and good alternative to cohere api\n- Evaluating the performance of this RAG pipeline\n- Implement a combination of Hybrid Search and HyDE\n- Add Repacking after Reranking, and before giving the prompt back to the model\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanasaber%2Frag_in_cpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanasaber%2Frag_in_cpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanasaber%2Frag_in_cpu/lists"}