{"id":20779428,"url":"https://github.com/freedomintelligence/multilingualsift","last_synced_at":"2025-04-30T19:42:02.021Z","repository":{"id":180708009,"uuid":"660456277","full_name":"FreedomIntelligence/MultilingualSIFT","owner":"FreedomIntelligence","description":"MultilingualSIFT: Multilingual Supervised Instruction Fine-tuning","archived":false,"fork":false,"pushed_at":"2023-08-15T17:58:46.000Z","size":219,"stargazers_count":89,"open_issues_count":1,"forks_count":5,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-30T19:22:13.248Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FreedomIntelligence.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-30T03:49:55.000Z","updated_at":"2025-01-13T18:33:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"6a0386d9-60a4-4cad-8b2c-0cd92a78fef4","html_url":"https://github.com/FreedomIntelligence/MultilingualSIFT","commit_stats":null,"previous_names":["freedomintelligence/multilingualsift"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FMultilingualSIFT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FMultilingualSIFT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FMultilingualSIFT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FMultilingualSIFT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FreedomIntelligence","download_url":"https://codeload.github.com/FreedomIntelligence/MultilingualSIFT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251771387,"owners_count":21641252,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T13:27:53.543Z","updated_at":"2025-04-30T19:42:02.000Z","avatar_url":"https://github.com/FreedomIntelligence.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multilingual Supervised Instruction Fine-tuning\n\nThis repo aims to provide the data, models, evaluation benchmark for multilingual instruction fine-tuning.\n\n## 📚 Data\nWe translate Alpaca-GPT4 and Evol-Instruct from English to languages using GPT-3.5 Turbo, where\n\n* For Alpaca-GPT4, we directly translate the instructions and responses.\n* For Evol-Instruct, we translate the instructions and use to generate the responses using the translated instructions.\n* For ShareGPT, we translate the English data from ShareGPT to other languages (Note: Due to the large scale of ShareGPT, we have yet to translate all the data).\n\n| Language   | Alpaca-GPT4                                                                                 | Evol-instruct                                                                                 |ShareGPT                                                                                 |\n|------------|---------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|\n| Chinese    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-chinese)    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-chinese)    |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-chinese)|\n| Japanese   | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-japanese)   | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-japanese)   |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-japanese)|\n| Korean     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-korean)     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-korean)     |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-korean)|\n| German     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-deutsch)    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-deutsch)    |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-deutsch)|\n| French     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-french)     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-french)     |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-french)|\n| Italian    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-italian)    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-italian)    |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-italian)| \n| Arabic     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-arabic)     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-arabic)     |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-arabic)|\n| Portuguese | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-portuguese) | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-portuguese) |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-portuguese)|\n| Spanish    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-spanish)    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-spanish)    |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-spanish)|\n| Hindi      | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-hindi)      | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-hindi)      |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-hindi)|\n| Indonesian | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/alpaca-gpt4-indonesian) | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/evol-instruct-indonesian) |[[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/sharegpt-indonesian)|\n\n## 🤖 Models\n### CLI Interation\n```bash\npython -m src.deploy.cli --model-path /path/to/weights/\n```\nFor example, you can use [FreedomIntelligence/phoenix-multiple-langs-v1](https://huggingface.co/FreedomIntelligence/phoenix-multiple-langs-v1) fine-tuned on eight languages (`English`, `Chinese`, `French`, `Spanish`, `Portuguese`, `Arabic`, `Indonesian`, `Hindi`):\n```bash\npython -m src.deploy.cli --model-path FreedomIntelligence/phoenix-multiple-langs-v1\n```\n\n### Deployment\n1. Launch a controller\n```shell\npython -m src.deploy.webapp.controller\n```\n\n2. Launch a model worker\n```shell\npython -m src.deploy.webapp.model_worker --model-path /path/to/weights/\n```\n\n3. Launch a gradio web server\n```shell\npython -m src.deploy.webapp.gradio_web_server\n```\nNow, you can open your browser and chat with a model.\n\n### Training\nSpecify the `train_data_path` and `val_data_path` and then run\n```shell\nbash scripts/train.sh\n```\n\n## 💯 Evaluation Benchmark\n\n### Evaluation Data\n\n* We translate [MMLU](https://github.com/hendrycks/test) and [Vicuna-80](https://github.com/lm-sys/FastChat/blob/main/fastchat/llm_judge/data/vicuna_bench/question.jsonl) to languages above for evaluation.\n\n| Language   | MMLU                                                                                 |\n|------------|--------------------------------------------------------------------------------------|\n| Chinese    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Chinese)    |\n| Japanese   | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Japanese)   |\n| Korean     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Korean)     |\n| German     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Deutsch)    |\n| French     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_French)     |\n| Italian    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Italian)    |\n| Arabic     | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Arabic)     |\n| Portuguese | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Portuguese) |\n| Spanish    | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Spanish)    |\n| Hindi      | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Hindi)      |\n| Indonesian | [[huggingface]](https://huggingface.co/datasets/FreedomIntelligence/MMLU_Indonesian) |\n\n### Evaluation\n\n* For MMLU\n```shell\nbash scripts/eval_mmlu.sh ${LANGUAGE} ${MODEL_PATH} ${MODEL_ID}\n```\n\n* For Vicuna-80\n```shell\nbash scripts/eval_vicuna-80.sh ${LANGUAGE} ${MODEL_PATH} ${MODEL_ID}\n```\n\n## Citation\n\nIf you find this repository helpful, please cite the repository below.\n\n```angular2\n@software{Chen_MultilingualSIFT_Multilingual_Supervised_2023,\nauthor = {Chen, Zhihong and Yan, Shuo and Liang, Juhao and Jiang, Feng and Wu, Xiangbo and Yu, Fei and Chen, Guiming Hardy and Chen, Junying and Zhang, Hongbo and Li Jianquan and Wan Xiang and Wang, Benyou},\nmonth = jul,\ntitle = {{MultilingualSIFT: Multilingual Supervised Instruction Fine-tuning}},\nurl = {https://github.com/FreedomIntelligence/MultilingualSIFT.git},\nversion = {0.1},\nyear = {2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fmultilingualsift","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreedomintelligence%2Fmultilingualsift","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fmultilingualsift/lists"}