{"id":21441720,"url":"https://github.com/dmis-lab/rag2","last_synced_at":"2025-04-19T19:23:28.348Z","repository":{"id":261017403,"uuid":"877200851","full_name":"dmis-lab/RAG2","owner":"dmis-lab","description":"[NAACL 2025] Rationale-Guided Retrieval Augmented Generation for Medical Question Answering","archived":false,"fork":false,"pushed_at":"2025-01-22T23:33:39.000Z","size":32,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-23T00:26:26.436Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmis-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-23T09:04:16.000Z","updated_at":"2025-01-22T23:33:43.000Z","dependencies_parsed_at":"2025-01-04T13:45:05.559Z","dependency_job_id":"79ff45fe-9159-4e27-a1dc-b2ff1ee7c238","html_url":"https://github.com/dmis-lab/RAG2","commit_stats":null,"previous_names":["dmis-lab/rag2"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FRAG2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FRAG2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FRAG2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FRAG2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmis-lab","download_url":"https://codeload.github.com/dmis-lab/RAG2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235263529,"owners_count":18962165,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T01:41:16.004Z","updated_at":"2025-04-19T19:23:28.340Z","avatar_url":"https://github.com/dmis-lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [NAACL 2025] Rationale-Guided Retrieval Augmented Generation for Medical Question Answering\n\n**Paper** | [Rationale-Guided Retrieval Augmented Generation for Medical Question Answering](https://arxiv.org/abs/2411.00300)\n\n**Authors**: Jiwoong Sohn, Yein Park, Chanwoong Yoon, Sihyeon Park, Hyeon Hwang, Mujeen Sung, Hyunjae Kim, Jaewoo Kang\n\n**Abstract**: Large language models (LLM) hold significant potential for applications in biomedicine, but they struggle with hallucinations and outdated knowledge. While retrieval-augmented generation (RAG) is generally employed to address these issues, it also has its own set of challenges: (1) LLMs are vulnerable to irrelevant or incorrect context, (2) medical queries are often not well-targeted for helpful information, and (3) retrievers are prone to bias toward the specific source corpus they were trained on. In this study, we present RAG² (RAtionale-Guided RAG), a new framework for enhancing the reliability of RAG in biomedical contexts. RAG² incorporates three key innovations: a small filtering model trained on perplexity-based labels of rationales, which selectively augments informative snippets of documents while filtering out distractors; LLM-generated rationales as queries to improve the utility of retrieved snippets; a structure designed to retrieve snippets evenly from a comprehensive set of four biomedical corpora, effectively mitigating retriever bias. Our experiments demonstrate that RAG² improves the state-of-the-art LLMs of varying sizes, with improvements of up to 6.1%, and it outperforms the previous best medical RAG model by up to 5.6% across three medical question-answering benchmarks.\n\n**Repository Overview**\n\nThis repository contains the implementation of **Rationale-Guided Retrieval-Augmented Generation (RAG²)**. It includes code for training the filtering model, setting up the retriever, and running inference. The repository is organized as follows:\n\n## Getting Started\n\n### 1. Training Dataset Preparation\n- Generate Chain-of-Thought (CoT) rationales using LLMs\n- Calculate perplexity scores for each rationale\n- Create training labels based on perplexity thresholds\n- Process and format the training data\n\n### 2. Retriever Setup\n- Index setup for multiple biomedical corpora\n- Configuration for balanced retrieval across corpora\n- Embedding model initialization\n- Retrieval parameter settings\n\n### 3. Filtering Model Training\nThe filtering model training code is based on [Adaptive-RAG](https://github.com/starsuzi/Adaptive-RAG).\n- Model architecture and configuration\n- Training with perplexity-based labels\n- Validation and model selection\n- Checkpoint saving\n\n### 4. Inference Pipeline\n- Initial CoT generation for query enhancement\n- Multi-corpus retrieval\n- Filtering retrieved passages\n- Final response generation\n\n## Usage\nDetailed instructions for each component will be provided soon.\n\n### Citation\nIf you use this work, please cite our paper:\n\n```\n@article{sohn2024rag,\n  title={Rationale-Guided Retrieval Augmented Generation for Medical Question Answering},\n  author={Jiwoong Sohn and Yein Park and Chanwoong Yoon and Sihyeon Park and Hyeon Hwang and Mujeen Sung and Hyunjae Kim and Jaewoo Kang},\n  journal={arXiv preprint arXiv:2411.00300},\n  year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Frag2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmis-lab%2Frag2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Frag2/lists"}