https://github.com/fasterdecoding/medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

llm llm-inference

Last synced: 14 May 2025

https://github.com/FasterDecoding/Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

llm llm-inference

Last synced: 09 May 2025

https://github.com/fasterdecoding/snapkv

long-context-modeling

Last synced: 03 Aug 2025

https://github.com/fasterdecoding/rest

REST: Retrieval-Based Speculative Decoding, NAACL 2024

llm-inference retrieval speculative-decoding

Last synced: 16 May 2025

https://github.com/FasterDecoding/REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024

llm-inference retrieval speculative-decoding

Last synced: 07 May 2025

https://github.com/fasterdecoding/bitdelta

llm quantization serving

Last synced: 04 Apr 2025

https://github.com/fasterdecoding/teal

llm llm-inference sparsity

Last synced: 06 Apr 2025