{"id":27943626,"url":"https://github.com/alpha-vllm/wemix-llm","last_synced_at":"2025-05-07T12:18:13.564Z","repository":{"id":191774942,"uuid":"685192121","full_name":"Alpha-VLLM/WeMix-LLM","owner":"Alpha-VLLM","description":null,"archived":false,"fork":false,"pushed_at":"2023-10-15T16:29:56.000Z","size":79,"stargazers_count":17,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-07T12:18:05.675Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Alpha-VLLM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-08-30T17:50:13.000Z","updated_at":"2024-02-23T05:13:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"3c5fcd20-6b21-4186-82c9-959c1974e003","html_url":"https://github.com/Alpha-VLLM/WeMix-LLM","commit_stats":null,"previous_names":["alpha-vllm/wemix-llm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alpha-VLLM%2FWeMix-LLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alpha-VLLM%2FWeMix-LLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alpha-VLLM%2FWeMix-LLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alpha-VLLM%2FWeMix-LLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Alpha-VLLM","download_url":"https://codeload.github.com/Alpha-VLLM/WeMix-LLM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252873889,"owners_count":21817715,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-07T12:18:12.529Z","updated_at":"2025-05-07T12:18:13.407Z","avatar_url":"https://github.com/Alpha-VLLM.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"## WeMix-LLM\n\nWeMix-LLM includes a series of LLMs and multimodal LLMs following the same paradigm. WeMix-LLM is built on [LLaMA2-Accessory](https://github.com/Alpha-VLLM/LLaMA2-Accessory).\n\n\u003cimg src=\"docs/opencompass_all.png\" width=\"90%\" /\u003e\n\n### Changelog\n* **[2023-10-16]** WeMix-LLM-V2 is now avaliable at [WeMix-LLaMA2-V2-70B](https://huggingface.co/Alpha-VLLM/WeMix-LLaMA2-V2-70B).\n* **[2023-8-31]** Release WeMix-LLM!\n\n### Setup\n\nPlease follow the [Environment Setup](https://llama2-accessory.readthedocs.io/en/latest/install.html) of LLaMA2-Accessory.\n\n### Models\n\n#### WeMix-LLaMA2: An Instruction-Following LLM\n* Weight: [WeMix-LLaMA2-7B](https://huggingface.co/Alpha-VLLM/WeMix-LLaMA2-7B), [WeMix-LLaMA2-70B](https://huggingface.co/Alpha-VLLM/WeMix-LLaMA2-70B), [WeMix-LLaMA2-V2-70B](https://huggingface.co/Alpha-VLLM/WeMix-LLaMA2-V2-70B).\n* Demo:\n    ```bash\n    wemix_weight=path/to/WeMix-LLaMA2-[7B/70B]/\n\n    python demos/multi_turn.py \\\n    --llama_config ${wemix_weight}/params.json --tokenizer_path ${wemix_weight}/tokenizer.model \\\n    --pretrained_path ${wemix_weight} --n_gpus [1/4]\n    ```\n* Benchmark (OpenCompass):\n\n| Model         | WeMix-LLaMA2-70B | LLaMA2-70B | Vicuna-33B | WeMix-LLaMA2-7B | LLaMA-2-7B-Chat | Vicuna-7B | LLaMA-2-7B |\n|---------------|------------------|------------|------------|-----------------|-----------------|-----------|------------|\n| OVERALL       | 58.6             | 57.4       | 50         | 49.6            | 44.8            | 43.4      | 41.6       |\n| EXAM          | 62.3             | 57.3       | 49.2       | 45.5            | 40.1            | 40.5      | 35.5       |\n| LANGUAGE      | 52.6             | 51.6       | 44.9       | 45.1            | 44              | 39.6      | 44.1       |\n| KNOWLEDGE     | 69               | 67.7       | 61.3       | 59.4            | 54.3            | 51.7      | 53.3       |\n| UNDERSTANDING | 62.9             | 60.8       | 58.5       | 55.5            | 50.9            | 50.5      | 42.4       |\n| REASONING     | 54.1             | 55         | 44.7       | 47.4            | 41.4            | 39.9      | 40.1       |\n\n\u003e Please refer to [benchmark.md](./benchmark.md) for more details.\n\n#### WeMix-LLaMA2-13B-MM: A Multimodal LLM\n\n* Weight: [Alpha-VLLM/WeMix-LLaMA2-13B-MM](https://huggingface.co/Alpha-VLLM/WeMix-LLaMA2-13B-MM)\n* Demo:\n```bash\nwemix_weight=path/to/WeMix-LLaMA2-13B-MM\n\ntorchrun --nproc-per-node=2 demos/single_turn_mm.py \\\n--llama_config ${wemix_weight}/params.json --tokenizer_path ${wemix_weight}/tokenizer.model \\\n--pretrained_path ${wemix_weight}\n```\n* Multimodal Benchmark:\n\n| Model                     | NoCaps               | Flickr30K |\n|---------------------------|----------------------|-----------|\n| Flamingo-9B               | -                    | 61.5      |\n| Flamingo-80B              | -                    | 67.2      |\n| Unified-IO-XL             | 100.0                | -         |\n| Kosmos-1                  | -                    | 67.1      |\n| Kosmos-2                  | -                    | 66.7      |\n| BLIP-2 (Vicuna-13B)       | 103.9                | 71.6      |\n| InstructBLIP (Vicuna-13B) | 121.9                | 82.8      |\n| Shikra (Vicuna-13B)       | -                    | 73.9      |\n| Qwen-VL (Qwen-7B)         | 121.4                | 85.8      |\n| Qwen-VL-Chat              | 120.2                | 81.0      |\n| WeMix-LLaMA2-13B-MM        | 114.7                | 86.0      |\n\n\u003e The multimodal benchmark is still in progress. Stay tuned!🎉\n\n\u003c!-- ### Contributors --\u003e\n\n### Acknowledgement\n\n[LLaMA2-Accessory](https://github.com/Alpha-VLLM/LLaMA2-Accessory), [LLaMA-Adapter](https://github.com/OpenGVLab/LLaMA-Adapter), [LLaMA](https://github.com/facebookresearch/llama).\n\n### License\n\nLlama 2 is licensed under the [LLAMA 2 Community License](https://github.com/facebookresearch/llama/blob/main/LICENSE), Copyright (c) Meta Platforms, Inc. All Rights Reserved.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falpha-vllm%2Fwemix-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falpha-vllm%2Fwemix-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falpha-vllm%2Fwemix-llm/lists"}