{"id":22446046,"url":"https://github.com/axioma-ai-labs/rag","last_synced_at":"2025-08-01T20:31:34.029Z","repository":{"id":252432790,"uuid":"840410989","full_name":"axioma-ai-labs/RAG","owner":"axioma-ai-labs","description":"Educational toolkit for all things RAG (Retrieval Augmented Generation)","archived":false,"fork":false,"pushed_at":"2024-08-09T17:41:52.000Z","size":44028,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2024-11-19T05:27:27.670Z","etag":null,"topics":["azure","data-processing","rag-agents","rag-implementation","retrieval-augmented-generation"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/axioma-ai-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-09T16:36:54.000Z","updated_at":"2024-10-25T09:52:01.000Z","dependencies_parsed_at":"2024-08-09T19:15:37.108Z","dependency_job_id":null,"html_url":"https://github.com/axioma-ai-labs/RAG","commit_stats":null,"previous_names":["cybernetica-labs/rag","axioma-ai-labs/rag"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axioma-ai-labs%2FRAG","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axioma-ai-labs%2FRAG/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axioma-ai-labs%2FRAG/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axioma-ai-labs%2FRAG/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/axioma-ai-labs","download_url":"https://codeload.github.com/axioma-ai-labs/RAG/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228402441,"owners_count":17914267,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","data-processing","rag-agents","rag-implementation","retrieval-augmented-generation"],"created_at":"2024-12-06T03:18:33.953Z","updated_at":"2024-12-06T03:18:34.489Z","avatar_url":"https://github.com/axioma-ai-labs.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Retrieval Augmented Generation (RAG)\n\nThis repository contains detailed documentation, example notebooks (as well as tips \u0026 tricks) for implementing Retrieval Augmented Generation (RAG) pipelines using Python.\n\n![\"Self-querying Retriever\"](/images/readme_img.jpeg)\n\n## Notebooks Overview\n\n### 0. [Tips \u0026 Tricks](0-tips-n-tricks)\nTips \u0026 tricks on data processing, cleaning, chunking, and augmentation. It also highlights various approaches, accompanied by code snippets, for evaluating RAG pipelines. Explore all the effective techniques and methods designed to enhance the accuracy of a RAG system.\n\n**What you can find inside?**\n- **Table Processing**: Techniques for processing of complex tables from the PDF documents.\n- **RAG Evaluation**: Approaches for evaluation of precision, recall, faithfulness, etc. of your RAG system.\n- **Query Extension**: Methodologies for enhanced retrieval (e.g., re-reanking, query extensions, etc.)\n- **Azure Indexing**: Two different approaches for data ingestion in Azure AI Search\n\n### 1. [RAG Contextual Compression](1-rag-contextual-compression)\nMethods to compress context for RAG systems to improve performance and accuracy and minimize costs.\n\n**What you can find inside?**\n- **Context Summarization**: Techniques to provide only relevant information to retrievers\n- **Embedding Compression**: Reducing the size of embeddings while maintaining information.\n- **Cost Optimization**: How to make more with LLMs for less money?\n\n### 2. [RAG for Semi-Structured Data](2-rag-semi-structured)\nImplementation of a RAG system for semi-structured data.\n\n**What you can find inside?**\n- **Data Retrieval and Processing**: Fetching and processing data from MongoDB.\n- **Querying and Chain Integration**: Building end-to-end query-response systems with enhanced retrieval capabilities for semi-structured data (e.g. JSON)\n\n### 3. [RAG for Structured Data](3-rag-structured)\nExploration \u0026 implementation of a RAG system for structured data with SQL databases.\n\n**What you can find inside?**\n- **SQL Database Interaction**: Configuring connections and interacting with SQL databases using pyodbc and SQLAlchemy.\n- **Agent Creation**: Setting up SQL agents for intelligent database interaction \u0026 querying.\n- **Query Enhancements**: Some ideas (not implemented yet) for informaiton filtering \u0026 retrieval.\n\n### 4. [RAG Agent](4-rag-agent)\nDetailed implementation and usage of autonomous RAG agents for different retrieval tasks.\n\n**What you can find inside?**\n- **Autonomous Agents**: Developing agents that perform retrieval tasks autonomously \u0026 efficiently.\n- **Integration**: Combining agents with other system components for seamless operation.\n\n### 5. [RAG Routing](5-rag-routing)\nTechniques and examples on how to route queries effectively in RAG systems to ensure accurate and relevant information retrieval depending on user's intent.\n\n**What you can find inside?**\n- **Intent Detection**: Methods to accurately determine user intent from queries.\n- **Query Routing**: Techniques for directing queries to the appropriate retrieval systems.\n- **Response Optimization**: Enhancing the relevance and accuracy of generated responses.\n\n## Data\nContains sample datasets and scripts for data preprocessing used in the example notebooks.\n\n## Images\nIncludes images and visual aids used in the documentation and notebooks.\n\n## Setup and Configuration\nBefore running the notebooks, ensure that you have the necessary environment variables configured as described in the notebooks. Don't forget to install packages from `requirements.txt`.\n\n## Work with us\n\nIf you are interested in:\n\n- Developing custom, production-ready RAG pipelines\n- Creating unique chatbot \u0026 assistant solutions\n- Integrating AI into various aspects of your business operations\n- Building prototypes or proofs of concepts to validate your next groundbreaking idea\n\nFeel free to reach out and share your story with us! We're just an email away.\n\n## Contributions and Issues\n\nFeel free to fork this repository, contribute changes, or submit issues.\n\nTruly yours!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxioma-ai-labs%2Frag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faxioma-ai-labs%2Frag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxioma-ai-labs%2Frag/lists"}