https://github.com/fminference/flexgen

Running large language models on a single GPU for throughput-oriented scenarios.
https://github.com/fminference/flexgen

deep-learning gpt-3 high-throughput large-language-models machine-learning offloading opt

Last synced: 4 months ago
JSON representation

Running large language models on a single GPU for throughput-oriented scenarios.

• Host: GitHub
• URL: https://github.com/fminference/flexgen
• Owner: FMInference
• License: apache-2.0
• Archived: true
• Created: 2023-02-15T21:18:53.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-10-28T03:05:41.000Z (6 months ago)
• Last Synced: 2024-12-10T03:00:28.340Z (4 months ago)
• Topics: deep-learning, gpt-3, high-throughput, large-language-models, machine-learning, offloading, opt
• Language: Python
• Homepage:
• Size: 37.1 MB
• Stars: 9,233
• Watchers: 112
• Forks: 552
• Open Issues: 58
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-ChatGPT-repositories - FlexGen - Running large language models on a single GPU for throughput-oriented scenarios. (NLP)