Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/SafeAILab/EAGLE
Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://github.com/SafeAILab/EAGLE
large-language-models llm-inference speculative-decoding
Last synced: 4 months ago
JSON representation
Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
- Host: GitHub
- URL: https://github.com/SafeAILab/EAGLE
- Owner: SafeAILab
- License: apache-2.0
- Created: 2023-12-07T19:08:39.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-09-27T22:30:59.000Z (5 months ago)
- Last Synced: 2024-09-28T16:47:34.015Z (5 months ago)
- Topics: large-language-models, llm-inference, speculative-decoding
- Language: Python
- Homepage: https://arxiv.org/pdf/2406.16858
- Size: 55.2 MB
- Stars: 776
- Watchers: 11
- Forks: 78
- Open Issues: 29
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-adaptive-computation - PyTorch code
- StarryDivineSky - SafeAILab/EAGLE - 1是该方法的第一个版本，其速度比传统解码快3倍，比Lookahead快2倍，比Medusa快1.6倍。EAGLE-2则利用草稿模型的置信度分数来动态调整草稿树结构，进一步提升了性能，速度比传统解码快4倍，比EAGLE-1快1.4倍。EAGLE-2还支持与vLLM、DeepSpeed、Mamba、FlashAttention等并行技术结合使用，并可以在8个RTX 3090 GPU上进行训练和测试。 (A01_文本生成_文本对话 / 大语言对话模型及数据)