{"id":51561773,"url":"https://github.com/dmis-lab/mrpo","last_synced_at":"2026-07-10T11:01:56.924Z","repository":{"id":368409180,"uuid":"1283634686","full_name":"dmis-lab/MRPO","owner":"dmis-lab","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-30T11:41:51.000Z","size":6612,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-30T13:22:16.267Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmis-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-29T05:21:41.000Z","updated_at":"2026-06-30T11:41:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dmis-lab/MRPO","commit_stats":null,"previous_names":["dmis-lab/mrpo"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/dmis-lab/MRPO","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FMRPO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FMRPO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FMRPO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FMRPO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmis-lab","download_url":"https://codeload.github.com/dmis-lab/MRPO/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FMRPO/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35329609,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-10T02:00:06.465Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-10T11:01:51.955Z","updated_at":"2026-07-10T11:01:56.914Z","avatar_url":"https://github.com/dmis-lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning\n\n\u003cp align=\"left\"\u003e\n  \u003ca href=\"https://arxiv.org/abs/2606.31825\" target=\"_blank\"\u003e\u003cimg src=\"https://img.shields.io/badge/arXiv-2606.31825-red\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://jungjunha.github.io/MRPO-page/\" target=\"_blank\"\u003e\u003cimg src=\"https://img.shields.io/badge/Project-Page-blue\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/collections/dmis-lab/mrpo\" target=\"_blank\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face%20-Model-orange\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\n### News\n* [June/30/2026] 🎉 We release our paper \"Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning\" on arXiv.\n\n\n\n## 📖 Overview\n- MRPO is a novel reinforcement learning framework that improves medical multimodal reasoning by directly addressing failures in the reasoning process. It reshapes GRPO-style advantages using both answer-level and step-wise process rewards, assigning exponentially larger penalties to earlier invalid steps when the final answer is incorrect, thereby correcting early-stage failures before they cascade while preserving successful trajectories. By redistributing the learning signal according to where reasoning first fails, MRPO induces transferable reasoning that improves both reasoning quality and final answer accuracy across diverse medical VQA benchmarks.\n\n![alt text](assets/MRPO_overview.png)\n\n## Key Results\n- On out-of-distribution medical VQA benchmarks, MRPO consistently outperforms standard GRPO and a recent RL baseline across all three backbones, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs like HuatuoGPT-Vision-34B by 2.79 points using only 13K training samples. On step-wise reasoning analysis, MRPO breaks failure cascades by reducing early-stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of early failures improves both reasoning quality and final answer accuracy.\n\n\n### Answer Accuracy Comparison\n\n**Performance comparison of MRPO against existing MLLMs**\n\n![alt text](assets/performance.png)\n\n**Cross-backbone ablation of training methods**\n\n![alt text](assets/performance_2.png)\n\n\n### Reasoning Quality Comparison\n\n![alt text](assets/reasoning_quality.png)\n\n\n\n## 🚀 Reproducibility\nTo enable reproduction under the same settings as our experiments, we release our full reinforcement learning recipe, including the complete code, datasets, and infrastructure.\n\n\n### Environment Setup\nWe recommend using conda to set up the environment:\n\n```bash\nbash install_env.sh\nconda activate mrpo\n```\n\n---\n\n### Datasets\nTo download and preprocess all datasets used in our experiments, run:\n\n```bash\npython Download_dataset.py \\\n  --data-dir \u003cdirectory-to-download-data\u003e\n```\n\nEach training instance follows the format below, with `image`, `problem`, and `solution` keys:\n\n```json\n{\n  \"image\": \"/Data_RAW/vqa-rad/images/synpic54610.jpg\",\n  \"problem\": \"Where is the pathology in this image?\",\n  \"solution\": \"vasculature\"\n}\n```\n\nIf you want to download the datasets individually, the three medical VQA benchmarks can be obtained from their original sources: VQA-RAD from [flaviagiammarino/vqa-rad](https://huggingface.co/datasets/flaviagiammarino/vqa-rad), SLAKE from [BoKelvin/SLAKE](https://huggingface.co/datasets/BoKelvin/SLAKE), and PathVQA from [flaviagiammarino/path-vqa](https://huggingface.co/datasets/flaviagiammarino/path-vqa). The gold reasoning annotations are provided by MedThink, which can be downloaded from [Tang-xiaoxiao/Medthink](https://github.com/Tang-xiaoxiao/Medthink). After downloading, align each MedThink rationale to its corresponding VQA instance to reproduce our training set.\n\n---\n\n### Training\nRun the training script:\n\n```bash\nbash Train_MRPO.sh\n```\n\n**Note**: Please update the following environment variables in the script:\n- `\u003cDATA_DIR\u003e`: Dataset root path (use the same directory specified in `--data-dir` during dataset download)\n- `\u003cMODEL_DIR\u003e`: Backbone model path\n- `\u003cBIOMEDBERT_PATH\u003e`: BiomedBERT path for BERTScore in the answer reward\n- `\u003cOUTPUT_DIR\u003e`: Output directory path\n- `\u003cWANDB_API_KEY\u003e`: Weights \u0026 Biases API key\n- `\u003cWANDB_PROJECT\u003e`: Weights \u0026 Biases project name\n\nWe currently support training on Qwen2.5-VL, Qwen3-VL, and InternVL3. The `\u003cMODEL_DIR\u003e` path must contain the corresponding model name (i.e., `qwen2.5-vl`, `qwen3-vl`, or `internvl3`).\n\n---\n\n### Inference\nRun the inference script:\n\n```bash\npython inference.py \\\n  --model_path \u003cpath-to-trained-model\u003e \\\n  --test_json \u003cpath-to-test-json\u003e \\\n  --output_dir \u003cpath-to-output-dir\u003e \\\n  --cuda_visible_devices \u003cgpu-ids\u003e\n```\nAfter running `Download_dataset.py`, the test set is available at `\u003cDATA_DIR\u003e/Data_Preprocessed/test_open_ended.json`, which can be passed directly to `--test_json`. The test data must follow the same format as the training data, with each instance containing the `image`, `problem`, and `solution` keys.\n\n---\n\n### Evaluation\nWe provide two evaluation protocols, both using GPT-5-mini as an LLM judge. **Answer correctness** assesses whether the generated answer matches the ground-truth answer, reported as the proportion of examples judged correct. **Reasoning evaluation** assesses step-wise reasoning quality by scoring each reasoning step as valid or invalid against the gold reasoning trajectory.\n\n**Answer correctness:**\n\n```bash\npython Eval_code/Answer_Evaluation.py \\\n  --input_json \u003cpath-to-inference-output-json\u003e \\\n  --output_json \u003cpath-to-answer-eval-output-json\u003e\n```\n\n**Reasoning evaluation:**\n\n```bash\npython Eval_code/Reasoning_Evaluation.py \\\n  --input_json \u003cpath-to-inference-output-json\u003e \\\n  --output_json \u003cpath-to-reasoning-eval-output-json\u003e \\\n  --gold_reasoning_file \u003cpath-to-gold-reasoning-json\u003e\n```\n\n\n## Contact\nFeel free to reach out to goodjungjun@korea.ac.kr or wjdalsquf@gmail.com\n\nBibTeX Citation: If you use MRPO in your research, please cite it using the following BibTeX entry:\n\n   ```bibtex\n        @misc{jung2026breakingfailurecascadesstepaware,\n        title={Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning}, \n        author={Junha Jung and Minbyul Jeong and Suhyeon Lim and Sungwook Jung and Jaehoon Yun and Taeyun Roh and Mujeen Sung and Jaewoo Kang},\n        year={2026},\n        eprint={2606.31825},\n        archivePrefix={arXiv},\n        primaryClass={cs.CV},\n        url={https://arxiv.org/abs/2606.31825}, \n        }\n   ```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Fmrpo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmis-lab%2Fmrpo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Fmrpo/lists"}