{"id":26154485,"url":"https://github.com/datacanvasio/mmalaya","last_synced_at":"2025-04-14T07:11:27.415Z","repository":{"id":218710866,"uuid":"747148586","full_name":"DataCanvasIO/MMAlaya","owner":"DataCanvasIO","description":null,"archived":false,"fork":false,"pushed_at":"2024-12-27T08:08:51.000Z","size":196,"stargazers_count":8,"open_issues_count":1,"forks_count":1,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-27T20:51:14.404Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataCanvasIO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-23T11:11:35.000Z","updated_at":"2024-12-27T08:08:55.000Z","dependencies_parsed_at":"2024-02-01T07:24:54.106Z","dependency_job_id":"35b45ee5-6397-4c7f-ab3f-d4755fba695b","html_url":"https://github.com/DataCanvasIO/MMAlaya","commit_stats":null,"previous_names":["datacanvasio/mmalaya"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCanvasIO%2FMMAlaya","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCanvasIO%2FMMAlaya/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCanvasIO%2FMMAlaya/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCanvasIO%2FMMAlaya/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataCanvasIO","download_url":"https://codeload.github.com/DataCanvasIO/MMAlaya/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248837287,"owners_count":21169374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-11T08:29:58.280Z","updated_at":"2025-04-14T07:11:27.385Z","avatar_url":"https://github.com/DataCanvasIO.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MMAlaya\nMMAlaya是基于大语言模型[Alaya](https://github.com/DataCanvasIO/Alaya)的多模态模型，模型权重文件在[DataCanvas/MMAlaya](https://huggingface.co/DataCanvas/MMAlaya/tree/main)\n\nMMAlaya包含以下三个模块：\n\u003cbr\u003e1，大语言模型[Alaya-7B-Chat](https://huggingface.co/DataCanvas/Alaya-7B-Chat)。\n\u003cbr\u003e2，图像文本特征编码器来自[blip2-opt-2.7b](https://huggingface.co/Salesforce/blip2-opt-2.7b)的EVA-G。\n\u003cbr\u003e3，图像文本特征到大预言模型的连接器,使用的是来自[blip2-opt-2.7b](https://huggingface.co/Salesforce/blip2-opt-2.7b)的Qformer和线性投影器。\n\n模型的训练主要基于[LLaVA](https://github.com/haotian-liu/LLaVA)架构\n\nOpenCompass 评测榜单，均分41.1，排名25名。\n![opencompass-leaderboard-multimodal](https://github.com/DataCanvasIO/MMAlaya/blob/main/data/opencompass-leaderboard-multimodal.png)\nMMBench 评测榜单，开源开放的模型，中文测试集，均分58.6，排名25名。\n![opencompass-leaderboard-multimodal-cn](https://github.com/DataCanvasIO/MMAlaya/blob/main/data/opencompass-leaderboard-multimodal-cn.png)\n\n推理可以参考 [inference.py](https://github.com/DataCanvasIO/MMAlaya/blob/main/inference.py)\n请使用transformers==4.33.0，否则会报错ImportError: cannot import name '_expand_mask' from 'transformers.models.bloom.modeling_bloom'。\n\n# Citation\nMMAlaya使用\u003ca href=\"https://github.com/DataCanvasIO/Alaya/blob/main/LICENSE\"\u003eApache 2.0 Lisense\u003c/a\u003e，开放模型权重，允许商业用途。如果您的项目引用了我们的MMAlaya，请标明出处，并点个小星星 :star: ：\n```\n@misc{datacanvas2024mmalaya,\n    author = {DataCanvas Ltd.},\n    title = {mmalaya},\n    year = {2024},\n    howpublished = {\\url{https://github.com/DataCanvasIO/MMAlaya}},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacanvasio%2Fmmalaya","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatacanvasio%2Fmmalaya","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacanvasio%2Fmmalaya/lists"}