{"id":20779435,"url":"https://github.com/freedomintelligence/apollo","last_synced_at":"2025-04-04T16:12:36.348Z","repository":{"id":218701602,"uuid":"746621917","full_name":"FreedomIntelligence/Apollo","owner":"FreedomIntelligence","description":"Multilingual Medicine: Model, Dataset, Benchmark, Code","archived":false,"fork":false,"pushed_at":"2024-10-15T03:37:06.000Z","size":13027,"stargazers_count":185,"open_issues_count":4,"forks_count":9,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-03-28T15:06:57.768Z","etag":null,"topics":["llm","medical","open-source"],"latest_commit_sha":null,"homepage":"https://apollo.llmzoo.com/#/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FreedomIntelligence.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-22T11:31:15.000Z","updated_at":"2025-03-13T08:08:03.000Z","dependencies_parsed_at":"2024-01-29T12:58:00.288Z","dependency_job_id":"db89093c-a09a-401b-afdf-343fe6b1304c","html_url":"https://github.com/FreedomIntelligence/Apollo","commit_stats":null,"previous_names":["freedomintelligence/medbase","freedomintelligence/apollo"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FApollo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FApollo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FApollo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FApollo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FreedomIntelligence","download_url":"https://codeload.github.com/FreedomIntelligence/Apollo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247208141,"owners_count":20901570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","medical","open-source"],"created_at":"2024-11-17T13:27:55.125Z","updated_at":"2025-04-04T16:12:36.334Z","avatar_url":"https://github.com/FreedomIntelligence.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multilingual Medicine: Model, Dataset, Benchmark, Code\n\nCovering English, Chinese, French, Hindi, Spanish, Hindi, Arabic So far\n\n\n\u003cp align=\"center\"\u003e\n   📃 \u003ca href=\"https://arxiv.org/abs/2403.03640\" target=\"_blank\"\u003ePaper\u003c/a\u003e • 🌐 \u003ca href=\"https://apollo.llmzoo.com/\" target=\"_blank\"\u003eDemo\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/datasets/FreedomIntelligence/ApolloCorpus\" target=\"_blank\"\u003eApolloCorpus\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/datasets/FreedomIntelligence/XMedbench\" target=\"_blank\"\u003eXMedBench\u003c/a\u003e  • 🌐 \u003ca href=\"https://github.com/FreedomIntelligence/ApolloMoE\" target=\"_blank\"\u003eApolloMoE\u003c/a\u003e\n   \u003cbr\u003e  \u003ca href=\"./README_zh.md\"\u003e 中文 \u003c/a\u003e | \u003ca href=\"./README.md\"\u003e English\n\u003c/p\u003e\n\n![Apollo](assets/apollo_medium_final.png)\n\n## 🌈 Update\n\n* **[2024.10.15]** [ApolloMoE](https://github.com/FreedomIntelligence/ApolloMoE) repo released, covering 50 Languages.\n* **[2024.04.25]** [MedJamba](https://huggingface.co/FreedomIntelligence/Apollo-MedJamba) released, train and evaluation code refer to [repo](https://github.com/FreedomIntelligence/MedJamba).\n* **[2024.03.07]** [Paper](https://arxiv.org/abs/2403.03640) released.\n* **[2024.02.12]** \u003ca href=\"https://huggingface.co/datasets/FreedomIntelligence/ApolloCorpus\" target=\"_blank\"\u003eApolloCorpus\u003c/a\u003e and  \u003ca href=\"https://huggingface.co/datasets/FreedomIntelligence/XMedbench\" target=\"_blank\"\u003eXMedBench\u003c/a\u003e  is published！🎉\n* **[2024.01.23]** Apollo repo is published！🎉\n\n\n## Results\n   🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-0.5B\" target=\"_blank\"\u003eApollo-0.5B\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-1.8B\" target=\"_blank\"\u003eApollo-1.8B\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-2B\" target=\"_blank\"\u003eApollo-2B\u003c/a\u003e  • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-6B\" target=\"_blank\"\u003eApollo-6B\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-7B\" target=\"_blank\"\u003eApollo-7B\u003c/a\u003e  • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-34B\" target=\"_blank\"\u003eApollo-34B\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-72B\" target=\"_blank\"\u003eApollo-72B\u003c/a\u003e  \n   \n   🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-MedJamba\" target=\"_blank\"\u003eMedJamba\u003c/a\u003e\n\n   🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-0.5B-GGUF\" target=\"_blank\"\u003eApollo-0.5B-GGUF\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-2B-GGUF\" target=\"_blank\"\u003eApollo-2B-GGUF\u003c/a\u003e  • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-6B-GGUF\" target=\"_blank\"\u003eApollo-6B-GGUF\u003c/a\u003e • 🤗 \u003ca href=\"https://huggingface.co/FreedomIntelligence/Apollo-7B-GGUF\" target=\"_blank\"\u003eApollo-7B-GGUF\u003c/a\u003e \n   \n   \n   \n   ![Apollo](assets/result.png)\n      \n   \n\n## Usage Format\n\n- 0.5B, 1.8B, 2B, 6B, 7B: User:{query}\\nAssistant:{response}\u003c|endoftext|\u003e\n- 34B, 72B: \u003c|User|\u003e:{query}\\n\u003c|Assistant|\u003e:{response}\u003c|endoftext|\u003e\n\n## Dataset \u0026 Evaluation\n\n- Dataset\n  🤗 \u003ca href=\"https://huggingface.co/datasets/FreedomIntelligence/ApolloCorpus\" target=\"_blank\"\u003eApolloCorpus\n\n   \u003cdetails\u003e\u003csummary\u003eClick to expand\u003c/summary\u003e\n\n    ![Apollo](assets/dataset.png)\n\n    - [Zip File](https://huggingface.co/datasets/FreedomIntelligence/ApolloCorpus/blob/main/ApolloCorpus.zip)\n    - [Data category](https://huggingface.co/datasets/FreedomIntelligence/ApolloCorpus/tree/main/train)\n       - Pretrain:\n         - data item:\n            - json_name: {data_source}_{language}_{data_type}.json\n            - data_type: medicalBook, medicalGuideline, medicalPaper, medicalWeb(from online forum), medicalWiki\n            - language: en(English), zh(chinese), es(spanish), fr(french), hi(Hindi)\n            - data_type: qa(generated qa from text)\n            - data_type==text: list of string\n              ```\n              [\n                \"string1\",\n                \"string2\",\n                ...\n              ]\n              ```\n            - data_type==qa: list of qa pairs(list of string)\n              ```\n              [\n                [\n                  \"q1\",\n                  \"a1\",\n                  \"q2\",\n                  \"a2\",\n                  ...\n                ],\n                ...\n              ]\n              ```\n      - SFT:\n          - json_name: {data_source}_{language}.json\n          - data_type: code, general, math, medicalExam, medicalPatient\n          - data item: list of qa pairs(list of string)\n            ```\n              [\n                [\n                  \"q1\",\n                  \"a1\",\n                  \"q2\",\n                  \"a2\",\n                  ...\n                ],\n                ...\n              ]\n              ```\n\n\n   \u003c/details\u003e\n   \n- Evaluation\n  🤗 \u003ca href=\"https://huggingface.co/datasets/FreedomIntelligence/XMedbench\" target=\"_blank\"\u003eXMedBench\u003c/a\u003e \n\n   \u003cdetails\u003e\u003csummary\u003eClick to expand\u003c/summary\u003e\n      \n     - EN:\n       - [MedQA-USMLE](https://huggingface.co/datasets/GBaker/MedQA-USMLE-4-options) \n       - [MedMCQA](https://huggingface.co/datasets/medmcqa/viewer/default/test)\n       - [PubMedQA](https://huggingface.co/datasets/pubmed_qa): Because the results fluctuated too much, they were not used in the paper.\n       - [MMLU-Medical](https://huggingface.co/datasets/cais/mmlu)\n         - Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine\n     - ZH:\n       - [MedQA-MCMLE](https://huggingface.co/datasets/bigbio/med_qa/viewer/med_qa_zh_4options_bigbio_qa/test)\n       - [CMB-single](https://huggingface.co/datasets/FreedomIntelligence/CMB): Not used in the paper\n         - Randomly sample 2,000 multiple-choice questions with single answer.\n       - [CMMLU-Medical](https://huggingface.co/datasets/haonan-li/cmmlu)\n         - Anatomy, Clinical_knowledge, College_medicine, Genetics, Nutrition, Traditional_chinese_medicine, Virology\n       - [CExam](https://github.com/williamliujl/CMExam): Not used in the paper\n         - Randomly sample 2,000 multiple-choice questions\n\n\n     - ES: [Head_qa](https://huggingface.co/datasets/head_qa)\n     - FR: [Frenchmedmcqa](https://github.com/qanastek/FrenchMedMCQA)\n     - HI: [MMLU_HI](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Arabic)\n        - Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine\n     - AR: [MMLU_Ara](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Hindi)\n        - Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine\n\n\n   \u003c/details\u003e\n\n   \n## Results reproduction\n   \u003cdetails\u003e\u003csummary\u003eClick to expand\u003c/summary\u003e\n\n   \n   We take Gemma-2b as example\n   1. Download Dataset for project:\n\n      ```\n      bash 0.download_data.sh\n      ```\n    \n   2. Prepare test and dev for specific model:\n\n      \n      - Create test data for with special token, you can use ./util/check.ipynb to check models' special tokens\n        \n       ```\n       bash 1.data_process_test\u0026dev.sh\n       ```\n    \n   3. Prepare train data for specific model (Create tokenized data in advance):\n\n    \n      - You can adjust data Training order and Training Epoch in this step\n\n       ```\n       bash 2.data_process_train.sh\n       ```\n    \n   4. Train the model\n\n    \n      - If you want to train in Multi Nodes please refer to ./scripts/multi_node_train_*.sh\n\n\n\n\n       ```\n       bash 3.single_node_train_gemma.sh\n       ```\n\n   5. (Optional) Proxy-Tuning: Directly improve model capabilities without fine-tuning\n\n       ```\n         bash src/proxy-tuning/scripts/eval/proxy_tuning.sh\n       ```\n   6. Evaluate your model: Generate score for benchmark\n      \n         ```\n         bash 4.eval.sh\n         ```\n\n   7. Evaluate your model: Play with your ckpts in bash\n    \n         ```\n         python ./src/evaluate/cli_demo.py --model_name='./ckpts/your/path/tfmr'\n         ```\n   \n   \u003c/details\u003e\n\n\n##  Acknowledgment\n\n- [HuatuoGPT-II](https://github.com/FreedomIntelligence/HuatuoGPT-II)\n- [proxy-tuning](https://github.com/alisawuffles/proxy-tuning)\n\n##  Citation\nPlease use the following citation if you intend to use our dataset for training or evaluation:\n\n```\n@misc{wang2024apollo,\n   title={Apollo: Lightweight Multilingual Medical LLMs towards Democratizing Medical AI to 6B People},\n   author={Xidong Wang and Nuo Chen and Junyin Chen and Yan Hu and Yidong Wang and Xiangbo Wu and Anningzhe Gao and Xiang Wan and Haizhou Li and Benyou Wang},\n   year={2024},\n   eprint={2403.03640},\n   archivePrefix={arXiv},\n   primaryClass={cs.CL}\n}\n```\n\n## Star History\n\n\u003ca href=\"https://star-history.com/#FreedomIntelligence/Apollo\u0026Date\"\u003e\n  \u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://api.star-history.com/svg?repos=FreedomIntelligence/Apollo\u0026type=Date\u0026theme=dark\" /\u003e\n    \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://api.star-history.com/svg?repos=FreedomIntelligence/Apollo\u0026type=Date\" /\u003e\n    \u003cimg alt=\"Star History Chart\" src=\"https://api.star-history.com/svg?repos=FreedomIntelligence/Apollo\u0026type=Date\" /\u003e\n  \u003c/picture\u003e\n\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fapollo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreedomintelligence%2Fapollo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fapollo/lists"}