https://github.com/LinkSoul-AI/Chinese-LLaVA

支持中英文双语视觉-文本对话的开源可商用多模态模型。
https://github.com/LinkSoul-AI/Chinese-LLaVA

Last synced: 3 months ago
JSON representation

支持中英文双语视觉-文本对话的开源可商用多模态模型。

• Host: GitHub
• URL: https://github.com/LinkSoul-AI/Chinese-LLaVA
• Owner: LinkSoul-AI
• License: apache-2.0
• Created: 2023-07-30T13:17:00.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-09-23T14:17:04.000Z (over 1 year ago)
• Last Synced: 2024-10-18T21:17:03.351Z (8 months ago)
• Language: Python
• Homepage: https://github.com/LinkSoul-AI/Chinese-LLaVA
• Size: 4.74 MB
• Stars: 352
• Watchers: 5
• Forks: 31
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-llama-resources - Chinese-LLaVA

README

        
# Chinese LLaVA

[![](https://img.shields.io/badge/Chinese-LLaVA-blue)](https://huggingface.co/spaces/LinkSoul/Chinese-LLaVA) [![](https://img.shields.io/badge/Commercial-Support-blue)](https://huggingface.co/spaces/LinkSoul/Chinese-LLaVA) [![](https://img.shields.io/badge/License-Apache_v2-blue)](https://github.com/LinkSoul-AI/Chinese-LLaVA/blob/main/LICENSE) [![](https://img.shields.io/badge/HuggingFace-Live_Demo-green)](https://huggingface.co/spaces/LinkSoul/Chinese-LLaVA) [![](https://img.shields.io/badge/Datasets-Chinese_Vision_Instructions-blue)](https://huggingface.co/spaces/LinkSoul/Chinese-LLaVA)

开源，可商用的**中英文双语视觉-语言助手 Chinese-LLaVA 以及中英文视觉 SFT 数据集 Chinese-LLaVA-Vision-Instructions**，支持中英文视觉-文本多模态对话的开源可商用对话模型。



    



## 模型框架

![Framework](.github/framework.png)

## 基础演示

![Base Demo](.github/demo.gif)

## 在线试玩

> Talk is cheap, Show you the Demo.

- [Demo 地址 / HuggingFace Spaces](https://huggingface.co/spaces/LinkSoul/Chinese-LLaVA) 

## 资源下载

- HuggingFace模型下载：

  - [Chinese-LLaVA-Chinese-Llama-2-7B](https://huggingface.co/LinkSoul/Chinese-LLaVA-Cllama2)

  - [Chinese-LLaVA-Baichuan-7B](https://huggingface.co/LinkSoul/Chinese-LLaVA-Baichuan)

- 百度网盘下载:

  - [Chinese-LLaVA-Chinese-Llama-2-7B](https://pan.baidu.com/s/16e_LEacMy2bqOYanIFWy8Q?pwd=9j61)

  - [Chinese-LLaVA-Baichuan-7B](https://pan.baidu.com/s/1WuYPrIaul0i6KA-to98cHw?pwd=6jwz)

- 语言模型:

  - [Chinese-Llama-2-7b](https://github.com/LinkSoul-AI/Chinese-Llama-2-7b)

  - [Baichuan-7B](https://huggingface.co/baichuan-inc/Baichuan-7B)

- 数据集：

  - HuggingFace Dataset: [Chinese-LLaVA-Vision-Instructions](https://huggingface.co/datasets/LinkSoul/Chinese-LLaVA-Vision-Instructions)

  - 百度网盘链接：https://pan.baidu.com/s/1-jgINIkW0MxusmJuSif85w?pwd=q62v

## 环境安装

```shell

# clone the repository

git clone https://github.com/LinkSoul-AI/Chinese-LLaVA

cd Chinese-LLaVA

# install package

conda create -n Cllava python=3.10 -y

conda activate Cllava

pip install --upgrade pip

pip install -e .

```

## 快速测试

```shell

python infer.py \

    --model-name PATH/TO/THE/CHINESE_LLAVA_MODEL \

    --llm-type "Chinese_llama2" or "baichuan" \

    --image-file PATH/TO/THE/INPUT/IMAGE \

    --query QUERY/PROMPT

```

## TODO

- 如何训练

- int4 量化

- docker 部署

## 相关项目

- [LLaVA](https://llava-vl.github.io/)

- [Chinese-Llama-2-7B](https://huggingface.co/LinkSoul/Chinese-Llama-2-7b)

- [baichuan-inc/Baichuan-7B](https://huggingface.co/baichuan-inc/Baichuan-7B)

## 项目协议

[Apache-2.0 license](https://github.com/LinkSoul-AI/Chinese-LLaVA/blob/main/LICENSE)

## 微信交流群