https://github.com/Blueyee/Efficient-CoT-LRMs
Chain of Thoughts (CoT) is so hot! so long! We need short reasoning process!
https://github.com/Blueyee/Efficient-CoT-LRMs
Last synced: 18 days ago
JSON representation
Chain of Thoughts (CoT) is so hot! so long! We need short reasoning process!
- Host: GitHub
- URL: https://github.com/Blueyee/Efficient-CoT-LRMs
- Owner: Blueyee
- Created: 2025-03-12T12:42:14.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2025-04-01T10:07:25.000Z (24 days ago)
- Last Synced: 2025-04-01T11:23:46.656Z (24 days ago)
- Size: 6.84 KB
- Stars: 45
- Watchers: 1
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome_Efficient_LRM_Reasoning - Blueyee/Efficient-CoT-LRMs
- Awesome-Efficient-Reasoning - Blueyee/Efficient-CoT-LRMs
README
# Efficient-CoT-LRMs
Chain of Thoughts (CoT) is hot now. But we can observe that CoT is also getting longer and sometimes overthinks.
Here, we track the latest news of how to keep CoT/LRMs more efficient.
## Topics included:
- Prompting-based
- Budget Control
- Compress in Latent Space
- Summarization
- Skip Something
- KV Cache Management
- Reinforment Learning
- General Topic
Here we go.
## Prompting-based
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|07/03/2025| [Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching ](https://arxiv.org/abs/2503.05179) | [link](https://github.com/SimonAytes/SoT) | / |
|25/02/2025| [Chain of Draft: Thinking Faster by Writing Less](https://arxiv.org/abs/2502.18600) | [link](https://github.com/sileix/chain-of-draft) | / |
|17/02/2025| [SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs](https://arxiv.org/abs/2502.12134) | / | /|
|04/06/2024| [Break the Chain: Large Language Models Can be Shortcut Reasoners ](https://arxiv.org/abs/2406.06580) | / | / |
|28/07/2023| [Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation](https://arxiv.org/abs/2307.15337) | [link](https://github.com/imagination-research/sot) | ICLR2024 |
## Budget Control
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|06/03/2025| [DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models](https://arxiv.org/abs/2503.04472) | / | / |
|24/12/2024| [Token-budget-aware llm reasoning ](https://arxiv.org/abs/2412.18547) | [link](https://github.com/GeniusHTX/TALE) | / |
|29/07/2024| [Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost](https://arxiv.org/abs/2407.19825) | / | / |
## Compress in Latent Space
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|24/02/2025| [Reasoning with Latent Thoughts: On the Power of Looped Transformers](https://arxiv.org/abs/2502.17416) | / | ICLR2025 |
|13/02/2025| [CoT-Valve: Length-Compressible Chain-of-Thought Tuning](https://arxiv.org/abs/2502.09601) | [link](https://github.com/horseee/CoT-Valve) | / |
|31/01/2025| [Efficient Reasoning with Hidden Thinking](https://arxiv.org/abs/2501.19201) | [link](https://github.com/shawnricecake/Heima) | / |
|17/12/2024| [Compressed Chain of Thought: Efficient Reasoning Through Dense Representations](https://arxiv.org/abs/2412.13171) | / | / |
|09/12/2024| [Training Large Language Models to Reason in a Continuous Latent Space](https://arxiv.org/abs/2412.06769) | [link](https://github.com/facebookresearch/coconut) | / |
|23/03/2024| [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://arxiv.org/abs/2405.14838) | / | / |
## Summarization
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|09/03/2025| [InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models](https://arxiv.org/abs/2503.06692) | / | / |
|21/02/2025| [LightThinker: Thinking Step-by-Step Compression](https://arxiv.org/abs/2502.15589)) | / | / |
|16/12/2024| [C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness](https://arxiv.org/abs/2412.11664) | / | AAAI2025 |
|12/02/2024| [Anchor-based Large Language Models](https://arxiv.org/abs/2402.07616) (related work) | [link](https://github.com/pangjh3/AnLLM) | ACL2024 |
## Skip Something
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|18/02/2025| [Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models](https://arxiv.org/abs/2502.13260) | / | / |
|17/02/2025| [TokenSkip: Controllable Chain-of-Thought Compression in LLMs](https://arxiv.org/abs/2502.12067) | [link](https://github.com/hemingkx/TokenSkip) | / |
|04/11/2024| [Can Language Models Learn to Skip Steps?](https://arxiv.org/abs/2411.01855) | / | NeurIPS2024) |
## KV Cache Management
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|21/02/2025| [LightThinker: Thinking Step-by-Step Compression](https://arxiv.org/abs/2502.15589)) | / | / |
|16/02/2025| [Efficient Long-Decoding Inference with Reasoning-Aware Attention Sparsity](https://arxiv.org/abs/2502.11147) | / | / |
## Reinforment Learning
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|06/03/2025| [L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning](https://www.arxiv.org/abs/2503.04697) | [link](https://github.com/cmu-l3/l1) | / |
|22/01/2025| [O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning](https://arxiv.org/abs/2501.12570) | [link](https://github.com/StarDewXXX/O1-Pruner) | / |
|22/01/2025| [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://arxiv.org/abs/2501.12599) | / | / |
## General Topic
| Time | Title | code| published |
|:----:|:----:|:----:|:----:|
|19/02/2025| [AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence](https://arxiv.org/abs/2502.13943) | / | / |
|30/12/2024| [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://arxiv.org/abs/2412.21187) | / | / |
|10/01/2024| [The Impact of Reasoning Step Length on Large Language Models](https://arxiv.org/abs/2401.04925) | [link](https://github.com/MingyuJ666/The-Impact-of-Reasoning-Step-Length-on-Large-Language-Models) | ACL2024findings |
# About us:
[PEILab](https://peilab.netlify.app/), Hong Kong University of Science and Technology (HKUST)