{"id":23872469,"url":"https://github.com/FreedomIntelligence/HuatuoGPT-o1","last_synced_at":"2025-09-08T22:32:07.702Z","repository":{"id":270055905,"uuid":"906833146","full_name":"FreedomIntelligence/HuatuoGPT-o1","owner":"FreedomIntelligence","description":"Medical o1, Towards medical complex reasoning with LLMs","archived":false,"fork":false,"pushed_at":"2025-01-20T02:31:13.000Z","size":3912,"stargazers_count":685,"open_issues_count":10,"forks_count":69,"subscribers_count":22,"default_branch":"main","last_synced_at":"2025-01-20T03:25:53.216Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FreedomIntelligence.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-22T03:25:58.000Z","updated_at":"2025-01-20T03:04:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"107f877b-1eb7-4a67-b0d0-01073a8bc97c","html_url":"https://github.com/FreedomIntelligence/HuatuoGPT-o1","commit_stats":null,"previous_names":["freedomintelligence/huatuogpt-o1"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/FreedomIntelligence/HuatuoGPT-o1","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FHuatuoGPT-o1","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FHuatuoGPT-o1/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FHuatuoGPT-o1/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FHuatuoGPT-o1/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FreedomIntelligence","download_url":"https://codeload.github.com/FreedomIntelligence/HuatuoGPT-o1/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FHuatuoGPT-o1/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274231448,"owners_count":25245659,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-08T02:00:09.813Z","response_time":121,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-03T16:00:53.807Z","updated_at":"2025-09-08T22:32:07.693Z","avatar_url":"https://github.com/FreedomIntelligence.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话","Open-source Projects","Open-source","Model","🤖 Scientific Models","中文医疗大模型","🤖 Foundation Models for Science","Medical LLMs \u0026 Foundation Models"],"sub_categories":["大语言对话模型及数据","Step-wise and Process-based Optimization","Models","🧬 Life Sciences","Domain-Specific Models"],"readme":"# HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs\n\u003cdiv align=\"center\"\u003e\n\u003ch3\u003e\n  HuatuoGPT-o1\n\u003c/h3\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n📃 \u003ca href=\"https://arxiv.org/pdf/2412.18925\" target=\"_blank\"\u003ePaper\u003c/a\u003e ｜🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-7B\" target=\"_blank\"\u003eHuatuoGPT-o1-7B\u003c/a\u003e ｜🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-8B\" target=\"_blank\"\u003eHuatuoGPT-o1-8B\u003c/a\u003e ｜ 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-70B\" target=\"_blank\"\u003eHuatuoGPT-o1-70B\u003c/a\u003e  | 📚 \u003ca href=\"https://huggingface.co/datasets/FreedomIntelligence/medical-o1-reasoning-SFT\" target=\"_blank\"\u003eData\u003c/a\u003e\n\u003c/p\u003e\n\n\n## ⚡ Introduction\nHello! Welcome to the repository for [HuatuoGPT-o1](https://arxiv.org/pdf/2412.18925)!\n\n\u003cdiv align=center\u003e\n\u003cimg src=\"assets/pic1.jpg\"  width = \"90%\" alt=\"HuatuoGPT-o1\" align=center/\u003e\n\u003c/div\u003e\n\n\n**HuatuoGPT-o1** is a medical LLM designed for advanced medical reasoning. It can identify mistakes, explore alternative strategies, and refine its answers.  By leveraging verifiable medical problems and a specialized medical verifier, it advances reasoning through:\n\n- Using the verifier to guide the search for a complex reasoning trajectory for fine-tuning LLMs.\n- Applying reinforcement learning (PPO) with verifier-based rewards to enhance complex reasoning further.\n\nWe open-sourced our models, data, and code here.\n\n## 👨‍⚕️ Model\n- **Model Access**\n\n|                      | Backbone     | Supported Languages | Link                                                                  |\n| -------------------- | ------------ | ----- | --------------------------------------------------------------------- |\n| **HuatuoGPT-o1-8B**  | LLaMA-3.1-8B  | English    | [HF Link](https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-8B) |\n| **HuatuoGPT-o1-70B** | LLaMA-3.1-70B | English    | [HF Link](https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-70B) |\n| **HuatuoGPT-o1-7B**  | Qwen2.5-7B   | English \u0026 Chinese | [HF Link](https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-7B) |\n| **HuatuoGPT-o1-72B** | Qwen2.5-72B  | English \u0026 Chinese | [HF Link](https://huggingface.co/FreedomIntelligence/HuatuoGPT-o1-72B) |\n\n- **Deploy**\n\nHuatuoGPT-o1 can be used just like `Llama-3.1-8B-Instruct`. You can deploy it with tools like [vllm](https://github.com/vllm-project/vllm) or [Sglang](https://github.com/sgl-project/sglang),  or perform direct inference:\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel = AutoModelForCausalLM.from_pretrained(\"FreedomIntelligence/HuatuoGPT-o1-8B\",torch_dtype=\"auto\",device_map=\"auto\")\ntokenizer = AutoTokenizer.from_pretrained(\"FreedomIntelligence/HuatuoGPT-o1-8B\")\n\ninput_text = \"How to stop a cough?\"\nmessages = [{\"role\": \"user\", \"content\": input_text}]\n\ninputs = tokenizer(tokenizer.apply_chat_template(messages, tokenize=False,add_generation_prompt=True\n), return_tensors=\"pt\").to(model.device)\noutputs = model.generate(**inputs, max_new_tokens=2048)\nprint(tokenizer.decode(outputs[0], skip_special_tokens=True))\n```\n\nHuatuoGPT-o1 adopts a *thinks-before-it-answers* approach, with outputs formatted as:\n\n```\n## Thinking\n[Reasoning process]\n\n## Final Response\n[Output]\n```\n\n## 📚 Data\n- **Data Access**\n\n| Data                  | Description                                                                                   | Link                                                                                           |\n| -------------------------- | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |\n| Medical Verifiable Problems | Open-ended medical problems sourced from challenging medical exams,  paired with ground-truth answers. | [Link](https://huggingface.co/datasets/FreedomIntelligence/medical-o1-verifiable-problem)  |\n| SFT Data in Stage 1        | Fine-tuning data generated using GPT-4o, including complex chains of thought (**Complex CoT**) and output (**Response**). | [Link](https://huggingface.co/datasets/FreedomIntelligence/medical-o1-reasoning-SFT)       |\n\n- **Data Construction**\n\nWe provide scripts to construct verifiable problems and searching reasoning paths.\n\n**1. Constructing Verifiable Problems from Multi-choice Questions.** \n```bash\npython construct_verifiable_medical_problems.py --data_path  data/demo_data.json --filter_data --model_name gpt-4o --api_key [your api key]\n```\n**2. Searching Complex Reasoning Paths for SFT**\n\n```bash\npython search_for_complex_reasoning_path.py --data_path  data/demo_data.json --efficient_search True  --max_search_attempts 1 --max_search_depth 2 --model_name gpt-4o --api_key [your api key]\n```\n\n\n## 🚀 Training\n\n- **Stage 1: Supervised Fine-Tuning (SFT)**\n\nFine-tune the model on an 8-GPU setup:\n```bash\naccelerate launch --config_file ./configs/deepspeed_zero3.yaml \\\n    --num_processes 8  \\\n    --num_machines 1 \\\n    --machine_rank 0 \\\n    --deepspeed_multinode_launcher standard SFT_stage1.py \\\n    --model_path [meta-llama/Llama-3.1-8B-Instruct] \\\n    --data_path [FreedomIntelligence/medical-o1-reasoning-SFT] \n```\n\n- **Stage 2: Reinforcement Learning (RL)**\n\nWe provide a simple PPO script using the [trl](https://github.com/huggingface/trl) library. Below is an example for training an 8B model with PPO on an 8-GPU A100 machine. Ensure you first download our [medical verifier](https://huggingface.co/FreedomIntelligence/medical_o1_verifier_3B) as the reward model.\n\n```bash\naccelerate launch \\\n\t--num_processes 8 \\\n\t--num_machines 1 \\\n\t--machine_rank 0 \\\n    --config_file ./configs/deepspeed_zero3.yaml \\\n\t--deepspeed_multinode_launcher standard RL_stage2.py \\\n    --model_name_or_path [FreedomIntelligence/HuatuoGPT-o1-8B] \\\n    --reward_model_path [FreedomIntelligence/medical_o1_verifier_3B] \\\n    --value_model_path [meta-llama/Llama-3.2-3B-Instruct] \\\n    --dataset_name  [FreedomIntelligence/medical-o1-verifiable-problem]\\\n    --response_length 1300 \\\n    --temperature 0.5 \\\n    --local_rollout_forward_batch_size 8 \\\n    --num_ppo_epochs 3 \\\n    --num_mini_batches 1 \\\n    --total_episodes 20000 \\\n    --per_device_train_batch_size 1 \\\n    --gradient_accumulation_steps 16 \\\n    --bf16 True \\\n    --output_dir ./ckpts \\\n    --save_strategy steps \\\n    --save_step 20 \\\n    --save_total_limit 1 \\\n    --eval_strategy steps \\\n    --eval_steps 20 \\\n    --kl_coef 0.03 \\\n    --learning_rate 5e-7 \\\n    --warmup_ratio 0.05 \\\n    --gradient_checkpointing True \\\n    --dataloader_num_workers 4 \\\n    --run_name ppo_medical_o1_8B \\\n    --num_sample_generations -1 \\\n    --report_to wandb\n```\n\n## 🧐 Evaluation\n1. You first need to install [Sglang](https://github.com/sgl-project/sglang). After installation, deploy the model you want to test using Sglang with the following command:\n```bash\nlog_num=0\nmodel_name=\"FreedomIntelligence/HuatuoGPT-o1-8B\" # Path to the model you are deploying\nport=28${log_num}35\nCUDA_VISIBLE_DEVICES=0  python -m sglang.launch_server --model-path $model_name --port $port --mem-fraction-static 0.8 --dp 1 --tp 1  \u003e sglang${log_num}.log 2\u003e\u00261 \u0026\n```\n2. Wait for the model to be deployed. After deployment, you can run the following code for evaluation. We use prompts that allow the model to respond freely. We find that the extracted results are consistently reliable and broadly cover the intended scope. You can also set the `--strict_prompt` option to use stricter prompts for more precise answer extraction.\n```bash\npython evaluation/eval.py --model_name $model_name  --eval_file evaluation/data/eval_data.json --port $port \n```\n3. After completing the evaluation, run the following code to stop the Sglang service and release GPU memory.\n```bash\nbash evaluation/kill_sglang_server.sh\n```\nThe evaluation code above can be used to test most models supported by Sglang.\n\n## 🩺 HuatuoGPT Series \n\nExplore our HuatuoGPT series:\n- [**HuatuoGPT**](https://github.com/FreedomIntelligence/HuatuoGPT): Taming Language Models to Be a Doctor\n- [**HuatuoGPT-II**](https://github.com/FreedomIntelligence/HuatuoGPT-II): One-stage Training for Medical Adaptation of LLMs\n- [**HuatuoGPT-Vision**](https://github.com/FreedomIntelligence/HuatuoGPT-Vision): Injecting Medical Visual Knowledge into Multimodal LLMs at Scale\n- [**CoD (Chain-of-Diagnosis)**](https://github.com/FreedomIntelligence/Chain-of-Diagnosis): Towards an Interpretable Medical Agent using Chain of Diagnosis\n- [**HuatuoGPT-o1**](https://github.com/FreedomIntelligence/HuatuoGPT-o1): Towards Medical Complex Reasoning with LLMs\n\n\n## 📖 Citation\n```\n@misc{chen2024huatuogpto1medicalcomplexreasoning,\n      title={HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs}, \n      author={Junying Chen and Zhenyang Cai and Ke Ji and Xidong Wang and Wanlong Liu and Rongsheng Wang and Jianye Hou and Benyou Wang},\n      year={2024},\n      eprint={2412.18925},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https://arxiv.org/abs/2412.18925}, \n}\n```\n\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=FreedomIntelligence/HuatuoGPT-o1\u0026type=Date)](https://star-history.com/#FreedomIntelligence/HuatuoGPT-o1\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFreedomIntelligence%2FHuatuoGPT-o1","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FFreedomIntelligence%2FHuatuoGPT-o1","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFreedomIntelligence%2FHuatuoGPT-o1/lists"}