https://github.com/princeton-nlp/LLM-Shearing

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
https://github.com/princeton-nlp/LLM-Shearing

efficiency llama llama2 llm nlp pre-training pruning

Last synced: about 1 month ago
JSON representation

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

• Host: GitHub
• URL: https://github.com/princeton-nlp/LLM-Shearing
• Owner: princeton-nlp
• License: mit
• Created: 2023-10-16T12:26:08.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-03-04T22:14:40.000Z (10 months ago)
• Last Synced: 2024-10-29T08:03:48.862Z (about 2 months ago)
• Topics: efficiency, llama, llama2, llm, nlp, pre-training, pruning
• Language: Python
• Homepage: https://arxiv.org/abs/2310.06694
• Size: 19 MB
• Stars: 551
• Watchers: 23
• Forks: 44
• Open Issues: 20
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• StarryDivineSky - princeton-nlp/LLM-Shearing - 2-7B 模型（使用 2T 令牌预训练）的存在，修剪它会产生一个与 OpenLLaMA 模型一样强大的模型，其预训练成本仅为 3%。 (A01_文本生成_文本对话 / 大语言对话模型及数据)