{"id":85863,"url":"https://github.com/geralt-targaryen/awesome-speculative-decoding","name":"awesome-speculative-decoding","description":"Reading notes on Speculative Decoding papers","projects_count":161,"last_synced_at":"2026-10-15T19:00:17.320Z","repository":{"id":282002779,"uuid":"945924955","full_name":"Geralt-Targaryen/Awesome-Speculative-Decoding","owner":"Geralt-Targaryen","description":"Reading notes on Speculative Decoding papers","archived":false,"fork":false,"pushed_at":"2026-06-02T11:11:54.000Z","size":4944,"stargazers_count":49,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-10-14T08:10:54.333Z","etag":null,"topics":["awesome","llm","nlp","papers","speculative-decoding"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Geralt-Targaryen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-10T10:43:15.000Z","updated_at":"2026-10-06T11:27:22.000Z","dependencies_parsed_at":"2025-04-27T14:01:08.632Z","dependency_job_id":"4f19219e-7c23-433a-9767-23dbf9f55cc6","html_url":"https://github.com/Geralt-Targaryen/Awesome-Speculative-Decoding","commit_stats":null,"previous_names":["geralt-targaryen/awesome-speculative-decoding"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Geralt-Targaryen/Awesome-Speculative-Decoding","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geralt-Targaryen%2FAwesome-Speculative-Decoding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geralt-Targaryen%2FAwesome-Speculative-Decoding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geralt-Targaryen%2FAwesome-Speculative-Decoding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geralt-Targaryen%2FAwesome-Speculative-Decoding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Geralt-Targaryen","download_url":"https://codeload.github.com/Geralt-Targaryen/Awesome-Speculative-Decoding/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geralt-Targaryen%2FAwesome-Speculative-Decoding/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":344794859,"owners_count":38307885,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-10-03T21:59:58.778Z","status":"online","status_checked_at":"2026-10-14T02:00:06.392Z","response_time":82,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2025-03-14T06:16:16.679Z","updated_at":"2026-10-15T19:00:17.320Z","primary_language":null,"list_of_lists":false,"displayable":true,"categories":["History \u0026 Origin","Bibliography by Venues","Draft Models","Speculative Decoding + Other Technologies","Draft Length Control","Other Awesome Lists","Retrieval-based Speculative Decoding","Draft Tree Construction","Verification Strategies"],"sub_categories":["NAACL 2024","ICLR 2024","Others","Draft Model Based on Target Hidden States","ICLR 2025","ACL 2024","EMNLP 2024","NAACL 2025","ACL 2025","NeurIPS 2024","Layer Skipping \u0026 Early Exiting","EMNLP 2025","ICML 2025","Lookahead Padding","ICML 2024","NeurIPS 2025","ICLR 2026"],"readme":"# Awesome-Speculative-Decoding\n\nReading list on speculative decoding.\n\n**Table of Contents**\n\n- [Bibliography by Venues](#bibliography-by-venues)\n  - [ICLR 2026](#iclr-2026)\n  - [NeurIPS 2025](#neurips-2025)\n  - [EMNLP 2025](#emnlp-2025)\n  - [ACL 2025](#acl-2025)\n  - [ICML 2025](#icml-2025)\n  - [ICLR 2025](#iclr-2025)\n  - [NAACL 2025](#naacl-2025)\n  - [NeurIPS 2024](#neurips-2024)\n  - [EMNLP 2024](#emnlp-2024)\n  - [ACL 2024](#acl-2024)\n  - [ICML 2024](#icml-2024)\n  - [ICLR 2024](#iclr-2024)\n  - [NAACL 2024](#naacl-2024)\n- [History \u0026 Origin](#history--origin)\n- [Draft Models](#draft-models)\n  - [Lookahead Padding](#lookahead-padding)\n  - [Layer Skipping \u0026 Early Exiting](#layer-skipping--early-exiting)\n  - [Draft Model Based on Target Hidden States](#draft-model-based-on-target-hidden-states)\n  - [Others](#others)\n- [Retrieval-based Speculative Decoding](#retrieval-based-speculative-decoding)\n- [Draft Tree Construction](#draft-tree-construction)\n- [Verification Strategies](#verification-strategies)\n- [Draft Length Control](#draft-length-control)\n- [Speculative Decoding + Other Technologies](#speculative-decoding--other-technologies)\n- [Citation](#citation)\n- [Other Awesome Lists](#other-awesome-lists)\n\n## Bibliography by Venues\n\n### ICLR 2026\n\n- \"SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism\" [2025-05] [[paper](https://arxiv.org/abs/2506.01979)]\n\n- \"Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling\" [2025-08] [[paper](https://arxiv.org/abs/2509.04474)]\n\n- \"FastGRPO: Accelerating Policy Optimization via Concurrency-aware Speculative Decoding and Online Draft Learning\" [2025-09] [[paper](https://arxiv.org/abs/2509.21792)]\n\n- \"Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding\" [2025-09] [[paper](https://arxiv.org/abs/2509.22134)]\n\n- \"Speculative Actions: A Lossless Framework for Faster Agentic Systems\" [2025-10] [[paper](https://arxiv.org/abs/2510.04371)]\n\n- \"Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs\" [2025-10] [[paper](https://arxiv.org/abs/2510.20064)]\n\n- \"Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Minimization\" [2025-11] [[paper](https://arxiv.org/abs/2511.15898)]\n\n- \"Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match\" [2025-11] [[paper](https://arxiv.org/abs/2511.22972)]\n\n- \"Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding\" [2026-01] [[paper](https://arxiv.org/abs/2601.05724)]\n\n- \"Flatter Tokens are More Valuable for Speculative Draft Model Training\" [2026-01] [[paper](https://arxiv.org/abs/2601.18902)]\n\n- \"Improving the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models\" [2026-02] [[paper](https://arxiv.org/abs/2602.01428)]\n\n- \"Learning to Draft: Adaptive Speculative Decoding with Reinforcement Learning\" [2026-03] [[paper](https://arxiv.org/abs/2603.01639)]\n\n- \"Speculative Speculative Decoding\" [2026-03] [[paper](https://arxiv.org/abs/2603.03251)]\n\n- \"Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling\" [2026-04] [[paper](https://arxiv.org/abs/2604.04987)]\n\n- \"RepSpec: Structural Re-parameterized Draft Model Training for Speculative Decoding\" [2026-04] [[paper](https://openreview.net/forum?id=bqEi97qzzzF)]\n\n### NeurIPS 2025\n\n- \"SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications\" [2024-11] [[paper](https://arxiv.org/abs/2411.04975)]\n\n- \"EasySpec: Layer-Parallel Speculative Decoding for Efficient Multi-GPU Utilization\" [2025-02] [[paper](https://arxiv.org/abs/2502.02493)]\n\n- \"GRIFFIN: Effective Token Alignment for Faster Speculative Decoding\" [2025-02] [[paper](https://arxiv.org/abs/2502.11018)]\n\n- \"Traversal Verification for Speculative Tree Decoding\" [2025-05] [[paper](https://arxiv.org/abs/2505.12398)]\n\n- \"STree: Speculative Tree Decoding for Hybrid State-Space Models\" [2025-05] [[paper](https://arxiv.org/abs/2505.14969)]\n\n- \"DREAM: Drafting with Refined Target Features and Entropy-Adaptive Cross-Attention Fusion for Multimodal Speculative Decoding\" [2025-05] [[paper](https://arxiv.org/abs/2505.19201)]\n\n- \"MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE\" [2025-05] [[paper](https://arxiv.org/abs/2505.19645)]\n\n- \"List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compression\" [2025-06] [[paper](https://arxiv.org/abs/2506.05632)]\n\n- \"Scaling Speculative Decoding with Lookahead Reasoning\" [2025-06] [[paper](https://arxiv.org/abs/2506.19830)]\n\n- \"OmniDraft: A Cross-vocabulary, Online Adaptive Drafter for On-device Speculative Decoding\" [2025-07] [[paper](https://arxiv.org/abs/2507.02659)]\n\n- \"ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding\" [2025-09] [[paper](https://arxiv.org/abs/2509.15235)]\n\n- \"Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding\" [2025-09] [[paper](https://arxiv.org/abs/2509.18344)]\n\n- \"AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders\" [2025-10] [[paper](https://arxiv.org/abs/2510.19779)]\n\n- \"CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs\" [2025-10] [[paper](https://arxiv.org/abs/2510.26843)]\n\n- \"Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding\" [2025-12] [[paper](https://arxiv.org/abs/2512.23858)]\n\n### EMNLP 2025\n\n**Main**\n\n- \"QSpec: Speculative Decoding with Complementary Quantization Schemes\" [2024-10] [[paper](https://arxiv.org/abs/2410.11305)]\n\n- \"Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation\" [2024-11] [[paper](https://arxiv.org/abs/2411.18462)]\n\n- \"Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference\" [2024-12] [[paper](https://arxiv.org/abs/2412.18934)]\n\n- \"Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification\" [2025-05] [[paper](https://arxiv.org/abs/2505.13204)]\n\n- \"Accelerated Test-Time Scaling with Model-Free Speculative Sampling\" [2025-06] [[paper](https://arxiv.org/abs/2506.04708)]\n\n- \"Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance\" [2025-07] [[paper](https://arxiv.org/abs/2507.22424)]\n\n- \"Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner\" [2025-08] [[paper](https://arxiv.org/abs/2508.15044)]\n\n- \"SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning\" [2025-08] [[paper](https://arxiv.org/abs/2508.16201)]\n\n- \"Speculative Safety-Aware Decoding\" [2025-08] [[paper](https://arxiv.org/abs/2508.17739)]\n\n- \"Faster In-Context Learning for LLMs via N-Gram Trie Speculative Decoding\" [2025-11] [[paper](https://aclanthology.org/2025.emnlp-main.911/)]\n\n- \"Speculative Streaming: Efficient and Scalable Speculative Decoding with Multi-Stream Attention\" [2025-11] [[paper](https://aclanthology.org/2025.emnlp-main.986/)]\n\n- \"Cacheback: Speculative Decoding With Nothing But Cache\" [2025-11] [[paper](https://www.arxiv.org/abs/2511.21699)]\n\n**Findings**\n\n- \"Speculative Decoding for Multi-Sample Inference\" [2025-03] [[paper](https://arxiv.org/abs/2503.05330)]\n\n- \"MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models\" [2025-05] [[paper](https://arxiv.org/abs/2505.10526)]\n\n- \"Mamba Drafters for Speculative Decoding\" [2025-06] [[paper](https://arxiv.org/abs/2506.01206)]\n\n- \"FractalLLM: Lossless Self-Speculative Decoding with Layer Embedded Self-Compression\" [2025-11] [[paper](https://aclanthology.org/2025.findings-emnlp.1286/)]\n\n- \"SpecCoT: Accelerating Chain-of-Thought Reasoning through Speculative Exploration\" [2025-11] [[paper](https://aclanthology.org/2025.findings-emnlp.1326/)]\n\n### ACL 2025\n\n**Main**\n\n- \"Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling\" [2024-08] [[paper](https://arxiv.org/abs/2408.08696)]\n\n- \"SAM Decoding: Speculative Decoding via Suffix Automaton\" [2024-11] [[paper](https://arxiv.org/abs/2411.10666)]\n\n- \"FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling\" [2025-02] [[paper](https://arxiv.org/abs/2502.14856)]\n\n- \"TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding\" [2025-02] [[paper](https://arxiv.org/abs/2502.15197)]\n\n- \"CORAL: Learning Consistent Representations across Multi-step Training with Lighter Speculative Drafter\" [2025-02] [[paper](https://arxiv.org/abs/2502.16880)]\n\n- \"CLaSp: In-Context Layer Skip for Self-Speculative Decoding\" [2025-05] [[paper](https://arxiv.org/abs/2505.24196)]\n\n- \"A Drop-In Solution for On-the-Fly Adaptation of Speculative Decoding in Large Language Models\" [2025-07] [[paper](https://aclanthology.org/2025.acl-long.482/)]\n\n- \"Faster Speculative Decoding via Effective Draft Decoder with Pruned Candidate Tree\" [2025-07] [[paper](https://aclanthology.org/2025.acl-long.486/)]\n\n- \"SPECTRA: Faster Large Language Model Inference with Optimized Internal and External Speculation\" [2025-07] [[paper](https://aclanthology.org/2025.acl-long.685/)]\n\n**Findings**\n\n- \"DReSD: Dense Retrieval for Speculative Decoding\" [2025-02] [[paper](https://arxiv.org/abs/2502.15572)]\n\n- \"Fuzzy Speculative Decoding for a Tunable Accuracy-Runtime Tradeoff\" [2025-02] [[paper](https://arxiv.org/abs/2502.20704)]\n\n- \"RASD: Retrieval-Augmented Speculative Decoding\" [2025-03] [[paper](https://arxiv.org/abs/2503.03434)]\n\n- \"Speculative Sampling via Exponential Races\" [2025-04] [[paper](https://arxiv.org/abs/2504.15475)]\n\n### ICML 2025\n\n- \"Accelerated Diffusion Models via Speculative Sampling\" [2025-01] [[paper](https://arxiv.org/abs/2501.05370)]\n\n- \"Reward-Guided Speculative Decoding for Efficient LLM Reasoning\" [2025-01] [[paper](https://arxiv.org/abs/2501.19324)]\n\n- \"Fast Large Language Model Collaborative Decoding via Speculation\" [2025-02] [[paper](https://arxiv.org/abs/2502.01662)]\n\n- \"Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation\" [2025-02] [[paper](https://arxiv.org/abs/2502.02789)]\n\n- \"Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies\" [2025-02] [[paper](https://arxiv.org/abs/2502.05202)]\n\n- \"QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache\" [2025-02] [[paper](https://arxiv.org/abs/2502.10424)]\n\n- \"RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding\" [2025-02] [[paper](https://arxiv.org/abs/2502.20330)]\n\n- \"Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding\" [2025-03] [[paper](https://arxiv.org/abs/2503.10135)]\n\n- \"SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs\" [2025-03] [[paper](https://arxiv.org/abs/2503.16163)]\n\n- \"Accelerating Large Language Model Reasoning via Speculative Search\" [2025-05] [[paper](https://arxiv.org/abs/2505.02865)]\n\n- \"Diffusion Models are Secretly Exchangeable: Parallelizing DDPMs via Auto Speculation\" [2025-05] [[paper](https://arxiv.org/abs/2505.03983)]\n\n- \"BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms\" [2025-05] [[paper](https://arxiv.org/abs/2505.15141)]\n\n- \"Attention-Level Speculation\" [2025-07] [[paper](https://proceedings.mlr.press/v267/cai25g.html)]\n\n- \"polybasic Speculative Decoding Through a Theoretical Perspective\" [2025-07] [[paper](https://arxiv.org/abs/2510.26527)]\n\n### ICLR 2025\n\n- \"Block Verification Accelerates Speculative Decoding\" [2024-03] [[paper](https://arxiv.org/abs/2403.10444)]\n\n- \"Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference\" [2024-05] [[paper](https://arxiv.org/abs/2405.14105)]\n\n- \"Faster Cascades via Speculative Decoding\" [2024-05] [[paper](https://arxiv.org/abs/2405.19261)]\n\n- \"Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting\" [2024-07] [[paper](https://arxiv.org/abs/2407.08223)]\n\n- \"MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding\" [2024-08] [[paper](https://arxiv.org/abs/2408.11049)]\n\n- \"PEARL: Parallel Speculative Decoding with Adaptive Draft Length\" [2024-08] [[paper](https://arxiv.org/abs/2408.11850)]\n\n- \"Learning Harmonized Representations for Speculative Sampling\" [2024-08] [[paper](https://arxiv.org/abs/2408.15766)]\n\n- \"Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding\" [2024-10] [[paper](https://arxiv.org/abs/2410.01699)]\n\n- \"LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding\" [2024-10] [[paper](https://arxiv.org/abs/2410.03355)]\n\n- \"Mixture of Attentions For Speculative Decoding\" [2024-10] [[paper](https://arxiv.org/abs/2410.03804)]\n\n- \"SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration\" [2024-10] [[paper](https://arxiv.org/abs/2410.06916)]\n\n- \"Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits\" [2024-10] [[paper](https://arxiv.org/abs/2410.18234)]\n\n- \"Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment\" [2025-01] [[paper](https://arxiv.org/abs/2501.19309)]\n\n- \"Towards Optimal Multi-draft Speculative Decoding\" [2025-02] [[paper](https://arxiv.org/abs/2502.18779)]\n\n### NAACL 2025\n\n**Main**\n\n- \"Decoding Speculative Decoding\" [2024-02] [[paper](https://arxiv.org/abs/2402.01528)]\n\n- \"EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models\" [2024-05] [[paper](https://arxiv.org/abs/2405.07542)]\n\n- \"Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion\" [2024-08] [[paper](https://arxiv.org/abs/2408.05636)]\n\n- \"Constrained Decoding with Speculative Lookaheads\" [2024-12] [[paper](https://arxiv.org/abs/2412.10418)]\n\n**Findings**\n\n- \"Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding\" [2025-02] [[paper](https://arxiv.org/abs/2502.05609)]\n\n- \"Hierarchical Speculative Decoding with Dynamic Window\" [2025-04] [[paper](https://aclanthology.org/2025.findings-naacl.462/)]\n\n### NeurIPS 2024\n\n- \"Cascade Speculative Drafting for Even Faster LLM Inference\" [2023-12] [[paper](https://arxiv.org/abs/2312.11462)]\n\n- \"Sequoia: Scalable and Robust Speculative Decoding\" [2024-02] [[paper](https://arxiv.org/abs/2402.12374)]\n\n- \"Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exiting\" [2024-04] [[paper](https://arxiv.org/abs/2404.18911)]\n\n- \"Nearest Neighbor Speculative Decoding for LLM Generation and Attribution\" [2024-05] [[paper](https://arxiv.org/abs/2405.19325)]\n\n- \"SpecExec: Massively Parallel Speculative Decoding For Interactive LLM Inference on Consumer Devices\" [2024-06] [[paper](https://arxiv.org/abs/2406.02532)]\n\n- \"A Theoretical Perspective for Speculative Decoding Algorithm\" [2024-10] [[paper](https://arxiv.org/abs/2411.00841)]\n\n- \"Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models\" [2024-10] [[paper](https://arxiv.org/abs/2410.20418)]\n\n- \"Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration\" [2024-12] [[paper](https://arxiv.org/abs/2412.00061)]\n\n### EMNLP 2024\n\n**Main**\n\n- \"Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding\" [2024-02] [[paper](https://arxiv.org/abs/2402.13720)]\n\n- \"Optimized Speculative Sampling for GPU Hardware Accelerators\" [2024-06] [[paper](https://arxiv.org/abs/2406.11016)]\n\n- \"Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters\" [2024-06] [[paper](https://arxiv.org/abs/2406.16758)]\n\n- \"EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees\" [2024-06] [[paper](https://arxiv.org/abs/2406.16858)]\n\n- \"SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding\" [2024-11] [[paper](https://arxiv.org/abs/2411.05289)]\n\n**Findings**\n\n- \"Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity\" [2024-10] [[paper](https://arxiv.org/abs/2410.01028)]\n\n- \"Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation\" [2024-10] [[paper](https://arxiv.org/abs/2410.10141)]\n\n### ACL 2024\n\n**Main**\n\n- \"Draft \u0026 Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding\" [2023-09] [[paper](https://arxiv.org/abs/2309.08168)]\n\n- \"Speculative Contrastive Decoding\" [2023-11] [[paper](https://arxiv.org/abs/2311.08981)]\n\n- \"LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding\" [2024-04] [[paper](https://arxiv.org/abs/2404.16710)]\n\n**Findings**\n\n- \"Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding\" [2024-01] [[paper](https://arxiv.org/abs/2401.07851)]\n\n- \"BASS: Batched Attention-optimized Speculative Sampling\" [2024-04] [[paper](https://arxiv.org/abs/2404.15778)]\n\n- \"Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism\" [2024-06] [[paper](https://arxiv.org/abs/2406.03853)]\n\n- \"Graph-Structured Speculative Decoding\" [2024-07] [[paper](https://arxiv.org/abs/2407.16207)]\n\n### ICML 2024\n\n- \"Online Speculative Decoding\" [2023-10] [[paper](https://arxiv.org/abs/2310.07177)]\n\n- \"Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads\" [2024-01] [[paper](https://arxiv.org/abs/2401.10774)]\n\n- \"EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty\" [2024-01] [[paper](https://arxiv.org/abs/2401.15077)]\n\n- \"Break the Sequential Dependency of LLM Inference Using Lookahead Decoding\" [2024-02] [[paper](https://arxiv.org/abs/2402.02057)]\n\n- \"GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding\" [2024-02] [[paper](https://arxiv.org/abs/2402.02082)]\n\n- \"Accelerated Speculative Sampling Based on Tree Monte Carlo\" [2024-07] [[paper](https://proceedings.mlr.press/v235/hu24f.html)]\n\n### ICLR 2024\n\n- \"DistillSpec: Improving Speculative Decoding via Knowledge Distillation\" [2023-10] [[paper](https://arxiv.org/abs/2310.08461)]\n\n### NAACL 2024\n\n**Main**\n\n- \"REST: Retrieval-Based Speculative Decoding\" [2023-11] [[paper](https://arxiv.org/abs/2311.08252)]\n\n**Findings**\n\n- \"SLiM: Speculative Decoding with Hypothesis Reduction\" [2024-06] [[paper](https://aclanthology.org/2024.findings-naacl.63/)]\n\n## History \u0026 Origin\n\n- \"Fast Inference from Transformers via Speculative Decoding\" [2022-11] [ICML 2023] [[paper](https://arxiv.org/abs/2211.17192)]\n\n  \u003e Experiments on: T5-11B, LaMDA-137B | WMT En-De, CNN/DM\n\n- \"Accelerating Large Language Model Decoding with Speculative Sampling\" [2023-02] [[paper](https://arxiv.org/abs/2302.01318)]\n\n  \u003e Experiemnts on: Chinchilla-70B | XSum, HumanEval\n\n## Draft Models\n\n### Lookahead Padding\n\n- \"Accelerating Transformer Inference for Translation via Parallel Decoding\" [2023-05] [ACL 2023] [[paper](https://arxiv.org/abs/2305.10427)]\n\n  \u003e A block of [PAD] tokens are iteratively refined until no token changes in the block. Applies off-the-shelf to any autoregressive model.\n  \u003e\n  \u003e Experiments on: machine translation\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2305.10427-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"PaSS: Parallel Speculative Sampling\" [2023-11] [[paper](https://arxiv.org/abs/2311.13581)]\n\n  \u003e Learn special tokens ($L$ lookahead embeddings $[LA]_1, \\cdots, [LA]_L$) on a small training set\n  \u003e\n  \u003e Experiments on: LLaMA-7B | Wikipedia, Stack\n\n### Layer Skipping \u0026 Early Exiting\n\n- \"Draft \u0026 Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding\" [2023-09] [ACL 2024] [[paper](https://arxiv.org/abs/2309.08168)]\n\n  \u003e The target LLM selectively skips some of its intermediate layers to generate draft tokens\n  \u003e\n  \u003e Experiments on: LLaMA-2-13B/70B, LLaMA-2-13B-Chat, CodeLLaMA-13B | CNN/DM, XSum, HumanEval\n\n- \"Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism\" [2024-06] [ACL 2024 Findings] [[paper](https://arxiv.org/abs/2406.03853)]\n\n  \u003e Introduces a trainable early-exiting layer on top of the target model's N-th layer hidden states to generate draft tokens\n  \u003e\n  \u003e Experiments on: LLaMA-2-13B/70B, LLaMA-2-Chat-70B, Vicuna-13B, CodeLLaMA-13B | GSM8K, XSum, HumanEval, MT-Bench\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2406.03853-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n### Draft Model Based on Target Hidden States\n\n- \"Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads\" [2024-01] [ICML 2024] [[paper](https://arxiv.org/abs/2401.10774)]\n\n  \u003e Train one head (a two-layer FFN with residual connection) for each draft token position\n  \u003e\n  \u003e Experiemnts on: Vicuna-7B/13B/33B, Zephyr-7B | MT-Bench\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2401.10774-1.png\" width=\"300\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty\" [2024-01] [ICML 2024] [[paper](https://arxiv.org/abs/2401.15077)]\n\n  \u003e The draft model autoregressively processes at the feature (hidden states before LM head) level and then derives tokens using the LM head of the target model\n  \u003e\n  \u003e Experiments on: Vicuna-7B/13B/33B, LLaMA2-Chat-7B/13B/70B, Mixtral-8x7B-Instruct | MT-Bench, HumanEval, GSM8K, Alpaca\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2401.15077-1.png\" width=\"300\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding\" [2024-02] [ICML 2024] [[paper](https://arxiv.org/abs/2402.02082)]\n\n  \u003e Each layer in the draft model attends to a corresponding layer in the target, counting from the top\n  \u003e\n  \u003e Experiments on: Vicuna-7B/13B/33B, Mistral-7B-Instruct-v0.1 | GSM8K, Finance-Alpaca, Spider, CodeSearchNet-Python\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2402.02082-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding\" [2024-02] [[paper](https://arxiv.org/abs/2402.05109)]\n\n  \u003e A variant of Medusa where each draft head takes output from the previous head as input\n  \u003e\n  \u003e Experiments on: Vicuna-7B/13B/33B | MT-Bench\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2402.05109-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding\" [2025-03] [[paper](https://arxiv.org/abs/2503.10135)]\n\n  \u003e EAGLE for the first two draft tokens, Medusa for the next 5.\n  \u003e\n  \u003e Experiments on: Vicuna7B/13B, LLaMA-2-Chat-7B/13B/70B, LLaMA-3-Instruct-8B/70B | MT-Bench, HumanEval, GSM8K, Alpaca, CNN/DM, Natural Questions.\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2503.10135-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n### Others\n\n- \"Online Speculative Decoding\" [2023-11] [ICML 2024] [[paper](https://arxiv.org/abs/2310.07177)]\n\n  \u003e Continuously update the draft model on observed user query data\n  \u003e\n  \u003e Experiments on: Vicuna-7B, Flan-T5-3B | Spider, GSM8K, CodeSearchNet-Python, Alpaca-finance\n\n- \"Cascade Speculative Drafting for Even Faster LLM Inference\" [2023-12] [NeurIPS 2024] [[paper](https://arxiv.org/abs/2312.11462)]\n\n  \u003e - Vertical cascade: in a series of draft models, each model reviews drafts from a smaller one, with the smallest model being a statistical model\n  \u003e - Horizontal cascade: assigns the largest draft model to generate the first draft token, and uses progressively smaller draft models to generate the following tokens (which are less likely to be accepted)\n  \u003e\n  \u003e Experiemnts on: Flan-T5, LLaMA-2-Chat-7B | GSM8K, MMLU\n\n- \"Training Domain Draft Models for Speculative Decoding: Best Practices and Insights\" [2025-03] [[paper](https://arxiv.org/abs/2503.07807)]\n\n  \u003e domain-specific draft models (function calling, biology, Chinese)\n  \u003e\n  \u003e Experiments on: LLaMA-3.1-8B\n\n- \"ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts\" [2025-03] [[paper](https://arxiv.org/abs/2503.13565)]\n\n  \u003e 4-bit quantization in MXFP4 datatype as draft model\n  \u003e\n  \u003e Experiments on: LLaMA2 7B, Qwen2.5-Coder 7B | HAGRID, MBPP\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2503.13565-1.png\" width=\"300\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"Automatic Task Detection and Heterogeneous LLM Speculative Decoding\" [2025-05] [[paper](https://arxiv.org/abs/2505.08600)]\n\n  \u003e Automatically categorizes downstream tasks into different sub-tasks and assigns them to a set of heterogeneous (LoRA-trained) draft models\n  \u003e\n  \u003e Experiments on: LLaMA2 13B | Wanjuan 1.0, ChemData700K\n\n- \"Mamba Drafters for Speculative Decoding\" [2025-06] [[paper](https://arxiv.org/abs/2506.01206)]\n\n  \u003e Experiments on: Pythia 6.9B, Mistral 7B | XSum, CNN/DM, GSM8K, MT-Bench, Alpaca, HumanEval, LongBench\n\n## Retrieval-based Speculative Decoding\n\n- \"Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding\" [2024-02] [EMNLP 2024] [[paper](https://arxiv.org/abs/2402.13720)]\n\n  \u003e Extends the last draft token to phrases. Does not require additional training.\n  \u003e\n  \u003e Experiments on: Yi-Base-34B, DeepSeek-Coder-Instruct-33B, CodeLLaMA-Instruct-34B, LLaMA-2-Chat-70B | HumanEval, MBPP, GSM8K, CNN/DM, WMT16\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2402.13720-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"SuffixDecoding: A Model-Free Approach to Speeding Up Large Language Model Inference\" [2024-11] [[paper](https://arxiv.org/abs/2411.04975)]\n\n  \u003e N-gram draft model that's built on-the-fly from a suffix tree\n  \u003e\n  \u003e Experiments on: LLaMA-3-70B-Instruct | WildChat, Magicoder, Spider, AgenticSQL\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2411.04975-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"SAM Decoding: Speculative Decoding via Suffix Automaton\" [2024-11] [[paper](https://arxiv.org/abs/2411.10666)]\n\n  \u003e Another n-gram draft model that's built on-the-fly from a suffix tree\n  \u003e\n  \u003e Experiments on: Vicuna-7B-v1.3 | MT-Bench, WMT14 De-En, CNN/DM, Natural Question, GSM8K, DPR\n\n- \"Speculative Decoding for Multi-Sample Inference\" [2025-03] [[paper](https://arxiv.org/abs/2503.05330)]\n\n  \u003e SD method tailored for multi-sample reasoning scenarios, such as self-consistency and Best-of-N sampling.\n  \u003e\n  \u003e \"for any partial sequence on the path $i$, we use its $k$-token suffix as a query to search for matching prefixes in other paths\"\n  \u003e\n  \u003e Experiments on: Qwen2.5-7B-Instruct, LLaMA-3-8B-Instruct | GSM8K, MATH\n\n## Draft Tree Construction\n\n- \"GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding\" [2024-02] [ICML 2024] [[paper](https://arxiv.org/abs/2402.02082)]\n\n  \u003e Expand each draft token to top-$k$ candidates where $k$ is a piecewise linear function of the top-1 confidence score $p$, set to 7, 5, 3, 1 for $p$ in (0, 0.3], (0.3, 0.6], (0.6, 0.8], (0.8, 1] respectively.\n  \u003e\n  \u003e Experiments on: Vicuna-7B/13B/33B | MT-Bench\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2402.02082-2.png\" width=\"300\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees\" [2024-06] [EMNLP 2024] [[paper](https://arxiv.org/abs/2406.16858)]\n\n  \u003e In the draft tree, some shallow nodes that are not expanded may have higher values than the deeper expanded nodes. Thus, EAGLE-2 reranks all draft tokens and select the top $m$ tokens with the highest values.\n  \u003e\n  \u003e Experiments on: Vicuna-7B/13B, LLaMA2-Chat-7B/13B/70B, LLaMA3-Instruct-8B/70B | MT-Bench, HumanEval, GSM8K, CNN/DM, Natural Questions\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2406.16858-2.png\" width=\"290\"\u003e\u003c/img\u003e\n    \u003cimg src=\"imgs/2406.16858-1.png\" width=\"290\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding\" [2025-05] [[paper](https://arxiv.org/abs/2505.13254)]\n\n  \u003e HeteroSpec dynamically optimizes computational resource allocation based on linguistic context complexity\n  \u003e\n  \u003e - The depth of low-entropy draft paths is increased\n  \u003e - More branches are pruned in low-entropy draft paths to focus on high-likelihood branches.\n  \u003e - Just-in-time graph tracing and compilation are employed to optimize computation graphs.\n  \u003e\n  \u003e Experiments on: Vicuna 13B, LLaMA-3.1-Instruct 8B, LLaMA-3.3-Instruct 70B, R1 8B | MT-Bench, HumanEval, GSM8K, Alpaca, CNN/DM.\n\n## Verification Strategies\n\n- \"SpecTr: Fast Speculative Decoding via Optimal Transport\" [2023-10] [NeurIPS 2023] [[paper](https://arxiv.org/abs/2310.15141)]\n\n  \u003e Introduced OTM - Optimal Transport with Membership cost - and an approximation that's linear in vocabulary size and logarithmic in candidate set size to tackle draft selection when there are multiple drafts\n  \u003e\n  \u003e Experiments on: PALM-2-Bison | LM1B\n\n- \"TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding\" [2025-02] [[paper](https://arxiv.org/abs/2502.15197)]\n\n  \u003e Selects draft tokens for verification based on draft model's output probability\n  \u003e\n  \u003e Experiments on: Vicuna-33B-v1.3, LLaMA-3.1-70B/405B-Instruct | ShareGPT, Chatbot Arena, Domain Tough Questions\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2502.15197-1.png\" width=\"300\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n## Draft Length Control\n\n- \"Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models\" [2024-05] [[paper](https://arxiv.org/abs/2405.04304)]\n\n  \u003e After generating a draft token, an FFN decides whether or not to continue drafting (input of FFN: top-10 draft model confidence, draft model entropy, token position)\n  \u003e\n  \u003e Experiments on: Vicuna-13B-v1.3, Starcoder-15B | CNN/DM, Alpaca, HumanEval, MBPP\n\n- \"Dynamic Depth Decoding: Faster Speculative Decoding for LLMs\" [2024-08] [[paper](https://arxiv.org/abs/2409.00142)]\n\n  \u003e In the EAGLE framework, use the sum of the probabilities of all the sequences in the beam as a heuristic for whether or not to continue draft generation\n  \u003e\n  \u003e Experiments on: Vicuna-7B/13B, LLaMA2-Chat-7B/13B | MT-Bench\n\n- \"Draft Model Knows When to Stop: A Self-Verification Length Policy for Speculative Decoding\" [2024-11] [[paper](https://arxiv.org/abs/2411.18462)]\n\n  \u003e After generating a draft token, the model decides whether or not to continue draft based on draft model entropy.\n  \u003e\n  \u003e Applies off-the-shelf to any autoregressive speculative decoding system without training.\n  \u003e\n  \u003e Experiments on: Pythia-6.9B, Vicuna-7B/13B-v1.3, LLaMA-3-70B, Qwen2.5-14B/32B, QwQ | MT-Bench, HumanEval, GSM8K, Alpaca, CNN/DM, Natural Questions, MATH, GPQA, AIME\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2411.18462-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures\" [2024-12] [[paper](https://arxiv.org/abs/2412.18910)]\n\n  \u003e In the EAGLE framework, train a 3-layer MLP on the penultimate prefix token's input embedding and last hidden states to predict next round's draft length.\n  \u003e\n  \u003e Experiments on: Vicuna-7B-v1.3 | MT-Bench, Alpaca, HumanEval, GSM8K, CNN/DM, Natural Questions\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2412.18910-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n- \"SpecServe: Efficient and SLO-Aware Large Language Model Serving with Adaptive Speculative Decoding\" [2025-03] [[paper](https://arxiv.org/abs/2503.05096)]\n\n  \u003e 1. Adaptive drafter: incorporates efficiency estimation (based on historical data) into the drafting phase, achieving step-level speculative length control\n  \u003e 2. Confidence prior verifier: prioritizes the verification of tokens with high acceptance rates, achieving fine-grained request-level speculative length control\n  \u003e 3. SLO-aware Efficiency estimator: evaluates the efficiency of speculative decoding and achieves SLO (service level objective) awareness\n  \u003e\n  \u003e Experiments on: Vicuna-7B-v1.5, Vicuna-33B-v1.3, LLaMA-3.1-70B | MT-Bench, WMT14 De-En, CNN/DM, Natural Questions, GSM8K, DPR\n\n- \"Utility-Driven Speculative Decoding for Mixture-of-Experts\" [2025-06] [[paper](https://arxiv.org/abs/2506.20675)]\n\n  \u003e Speculative decoding for MoEs. (MoEs break the key assumption in SD that the verification overhead compared to decoding a single token is negligible.)\n  \u003e\n  \u003e Experiments on: Mixtral, Phi-3.5, OLMoE, DeepSeek-V1, Qwen1.5 | GSM8K, HumanEval, MT-Bench\n\n## Speculative Decoding + Other Technologies\n\n- \"Speculative Contrastive Decoding\" [2023-11] [ACL 2024 Short] [[paper](https://arxiv.org/abs/2311.08981)]\n\n  \u003e Speculative decoding + contrastive decoding to achieve both decoding acceleration and quality improvement\n  \u003e\n  \u003e Experiments on: LLaMA-2-70B | WikiText, HumanEval, AlpacaEval, GSM8K\n\n- \"LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding\" [2024-04] [ACL 2024] [[paper](https://arxiv.org/abs/2404.16710)]\n\n  \u003e - Apply layer dropout (higher dropout rates for later layers) and early exit loss during training\n  \u003e - At inference time, exit at early layers to generate draft tokens, and verify the draft tokens with the ramaining layers.\n  \u003e - Note: _this changes target model!_\n  \u003e\n  \u003e Experiments on: LLaMA-2-7B/13B, LLaMA-3-8B, LLaMA-3.2-1B\n\n- \"DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decoding\" [2025-04] [[paper](https://arxiv.org/abs/2504.05598)]\n\n  \u003e DEL, a plug-and-play method that adaptively selects the exit layer and speculation length during inference on top of LayerSkip (2404.16710).\n  \u003e\n  \u003e Experiments on: LLaMA-2 7B/13B/70B, LLaMA-3.2 1B, CodeLLaMA 7B/34B | AQuA-RAT, CNN/DM, XSUM, HumanEval\n\n    \u003cp align=\"center\"\u003e\n    \u003cimg src=\"imgs/2504.05598-1.png\" width=\"600\"\u003e\u003c/img\u003e\n    \u003c/p\u003e\n\n## Citation\n\nIf you refer to this repo, please cite the following paper:\n\n```\n@inproceedings{Zhang2025SVIP,\n  author       = {Ziyin Zhang and\n                  Jiahao Xu and\n                  Tian Liang and\n                  Xingyu Chen and\n                  Zhiwei He and\n                  Rui Wang and\n                  Zhaopeng Tu},\n  editor       = {Christos Christodoulopoulos and\n                  Tanmoy Chakraborty and\n                  Carolyn Rose and\n                  Violet Peng},\n  title        = {Draft Model Knows When to Stop: Self-Verification Speculative Decoding\n                  for Long-Form Generation},\n  booktitle    = {Proceedings of the 2025 Conference on Empirical Methods in Natural\n                  Language Processing, {EMNLP} 2025, Suzhou, China, November 4-9, 2025},\n  pages        = {16685--16697},\n  publisher    = {Association for Computational Linguistics},\n  year         = {2025},\n  url          = {https://doi.org/10.18653/v1/2025.emnlp-main.844},\n  doi          = {10.18653/V1/2025.EMNLP-MAIN.844}\n}\n```\n\n## Other Awesome Lists\n\n[**Awesome-Code-LLM**](https://github.com/codefuse-ai/Awesome-Code-LLM), [**Awesome-Education-LLM**](https://github.com/Geralt-Targaryen/Awesome-Education-LLM)\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/geralt-targaryen%2Fawesome-speculative-decoding/projects"}