https://github.com/lyogavin/airllm

AirLLM 70B inference with single 4GB GPU
https://github.com/lyogavin/airllm

chinese-llm chinese-nlp finetune generative-ai instruct-gpt instruction-set llama llm lora open-models open-source open-source-models qlora

Last synced: about 1 month ago
JSON representation

AirLLM 70B inference with single 4GB GPU

• Host: GitHub
• URL: https://github.com/lyogavin/airllm
• Owner: lyogavin
• License: apache-2.0
• Created: 2023-06-12T16:28:41.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-07-29T21:22:55.000Z (about 1 month ago)
• Last Synced: 2024-07-30T22:13:08.120Z (about 1 month ago)
• Topics: chinese-llm, chinese-nlp, finetune, generative-ai, instruct-gpt, instruction-set, llama, llm, lora, open-models, open-source, open-source-models, qlora
• Language: Jupyter Notebook
• Homepage:
• Size: 3.24 MB
• Stars: 3,516
• Watchers: 98
• Forks: 293
• Open Issues: 68
• Metadata Files:
  • Readme: README.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE

Awesome Lists containing this project

• awesome-production-machine-learning - AirLLM - AirLLM optimizes inference memory usage, allowing 70B large language models to run inference on a single 4GB GPU card without quantization, distillation and pruning. (Model Serving and Monitoring)
• awesome-LLM-resourses - AirLLM