https://github.com/awkrail/videollm_suite
[WIP] A collection of inference-only video LLMs.
https://github.com/awkrail/videollm_suite
video-llms video-processing
Last synced: 3 months ago
JSON representation
[WIP] A collection of inference-only video LLMs.
- Host: GitHub
- URL: https://github.com/awkrail/videollm_suite
- Owner: awkrail
- License: apache-2.0
- Created: 2024-10-22T05:39:17.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-12-03T10:15:24.000Z (6 months ago)
- Last Synced: 2024-12-03T11:23:14.839Z (6 months ago)
- Topics: video-llms, video-processing
- Language: Python
- Homepage:
- Size: 36.4 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# VideoLLM_suite
VideoLLM_suite is a collection of inference-only open-source video LLMs.
# Usage
Excepted.
```python
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model = VideoLLaMA(model_path)
model.to(device)
input_path = "videos/input.mp4"
prompt = "Describe the video for details."
model.encode_video(input_path)
sentence = model.generate(prompt)
```
# Models
- [ ] : video LLaMA
- [ ] : video LLaMA 2
- [ ] : video Chat
- [ ] : video Chat 2
- [ ] : LITA
- [ ] : Video LLaVA
- [ ] : Chat-UniVi
- [ ] : LaViLa
- [ ] : MA-LMM
- [ ] : MovieChat
- [ ] : MovieChat+
- [ ] : Koala
- [ ] : LongVLM
- [ ] : MiniGPT-video
- [ ] : PLLaVA
- [ ] : ST-LLM
- [ ] : InternVideo2
- [ ] : ViLA
- [ ] : Video-ChatGPT
- [ ] : TimeChat
# Test
```
pytest tests
```
# Mypy + Ruff
```
mypy firefly
ruff check firefly
```