Codebases

• **open-r1** - r1?style=flat-square&logo=github)](https://github.com/huggingface/open-r1/stargazers) | Fully open reproduction of the DeepSeek-R1 pipeline (SFT, distillation, GRPO, evaluation) |
• **OpenRLHF** - square&logo=github)](https://github.com/OpenRLHF/OpenRLHF/stargazers) | An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & vLLM & Ray & Dynamic Sampling & Async Agentic RL)* |
• **verl** - square&logo=github)](https://github.com/volcengine/verl/stargazers) | a flexible, efficient and production-ready RL training library for large language models |
• **TinyZero** - Pan/TinyZero?style=flat-square&logo=github)](https://github.com/Jiayi-Pan/TinyZero/stargazers) | Minimal reproduction of DeepSeek R1-Zero |
• **PRIME** - RL/PRIME?style=flat-square&logo=github)](https://github.com/PRIME-RL/PRIME/stargazers) | PRIME (Process Reinforcement through IMplicit REwards), an open-source solution for online RL with process rewards |
• **rllm** - project/rllm?style=flat-square&logo=github)](https://github.com/agentica-project/rllm/stargazers) | an open-source framework for post-training language agents via reinforcement learning |
• **Open-Reasoner-Zero** - Reasoner-Zero/Open-Reasoner-Zero?style=flat-square&logo=github)](https://github.com/Open-Reasoner-Zero/Open-Reasoner-Zero/stargazers) | one open source implementation of large-scale reasoning-oriented RL training focusing on scalability, simplicity and accessibility |
• **ROLL** - square&logo=github)](https://github.com/alibaba/ROLL/stargazers) | an Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models |
• **AReaL** - square&logo=github)](https://github.com/inclusionAI/AReaL/stargazers) | Ant Reasoning Reinforcement Learning for LLMs |
• **RAGEN** - AI/RAGEN?style=flat-square&logo=github)](https://github.com/RAGEN-AI/RAGEN/stargazers) | RAGEN (Reasoning AGENt, pronounced like "region") leverages reinforcement learning (RL) to train LLM reasoning agents in interactive, stochastic environments. |
• **slime** - square&logo=github)](https://github.com/THUDM/slime/stargazers) | an LLM post-training framework for RL scaling with high-performance training and flexible data generation|
• **Nemo-Aligner** - Aligner?style=flat-square&logo=github)](https://github.com/NVIDIA/NeMo-Aligner/stargazers) | Scalable toolkit for efficient model alignment |
• **Trinity-RFT** - RFT?style=flat-square&logo=github)](https://github.com/modelscope/Trinity-RFT/stargazers) | A unified RFT framework with plug-and-play modules (for algorithms, data pipelines, and synchronization) |

Other Awesome Lists

2024 & Earlier

• Awesome-LLM
• Awesome-LLM-Reasoning
• Awesome-RL-Based-LLM-Reasoning
• Awesome-LLM-RLVR

Papers

2024 & Earlier

• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Math-Shepherd: Verify and Reinforce LLMs Step-by-Step without Human Annotations
• Let’s Verify Step by Step
• Training Language Models to Follow Instructions with Human Feedback
• Solving Olympiad Geometry without Human Demonstrations

2025

• Brain Bandit: A Biologically Grounded Neural Network for Efficient Control of Exploration
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Demystifying Long Chain-of-Thought Reasoning in LLMs
• Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning
• Kimi K 1.5: Scaling Reinforcement Learning with LLMs
• S²R: Teaching LLMs to Self-Verify and Self-Correct via Reinforcement Learning
• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
• QLASS: Boosting Language Agent Inference via Q-Guided Stepwise Search
• Process Reward Models That Think
• Learning to Reason under Off-Policy Guidance
• THINKPRUNE: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning
• GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning
• When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning
• SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks
• L1: Controlling How Long a Reasoning Model Thinks With Reinforcement Learning
• Scaling Test-Time Compute Without Verification or RL is Suboptimal
• DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models
• Reasoning with Reinforced Functional Token Tuning
• Provably Optimal Distributional RL for LLM Post-Training
• On the Emergence of Thinking in LLMs I: Searching for the Right Intuition
• STP: Self-Play LLM Theorem Provers with Iterative Conjecturing and Proving
• A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
• Recitation over Reasoning: How Cutting-Edge LMs Fail on Elementary Reasoning Problems
• Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad
• (REINFORCE++) A Simple and Efficient Approach for Aligning Large Language Models
• ReFT v3: Reasoning with Reinforced Fine-Tuning
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• SimPO: Simple Preference Optimization with a Reference-Free Reward
• DeepSeek-Prover v1.5: Harnessing Proof Assistant Feedback for RL and MCTS
• Tülu 3: Pushing Frontiers in Open Language Model Post-Training
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
• Model Alignment as Prospect Theoretic Optimization
• UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning
• GUI-R1: A Generalist R1-Style Vision-Language Action Model for GUI Agents
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
• VinePPO: Refining Credit Assignment in RL Training of LLMs
• Controlling Large Language Model with Latent Action
• Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs
• Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation
• Demystifying Long Chain-of-Thought Reasoning
• MA-LoT: Model-Collaboration Lean-based Long Chain-of-Thought Reasoning enhances Formal Theorem Proving
• SHIELDAGENT: Shielding Agents via Verifiable Safety Policy Reasoning
• TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference
• AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress
• Towards Agentic Self-Learning LLMs in Search Environment
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
• Language Models that Think, Chat Better
• RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
• Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning
• MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning
• Smart-Searcher: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning
• ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
• Absolute Zero: Reinforced Self-play Reasoning with Zero Data
• CURE: Co-Evolving Coders and Unit Testers via Reinforcement Learning
• Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles
• Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards
• Beyond Verifiable Rewards: Scaling Reinforcement Learning in Language Models to Unverifiable Data
• RLVR-World: Training World Models with Reinforcement Learning
• SeRL: Self-play Reinforcement Learning for Large Language Models with Limited Data
• SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution
• rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset
• SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond
• SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
• AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
• Agentic RL Scaling Law: Spontaneous Code Execution for Mathematical Problem Solving
• Rethinking Verification for LLM Code Generation: From Generation to Testing
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
• Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling
• ATLAS: Autoformalizing Theorems through Lifting, Augmentation, and Synthesis of Data
• QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation
• QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
• miniF2F-Lean Revisited: Reviewing Limitations and Charting a Path Forward
• SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications
• To Think or Not To Think: A Study of Thinking in Rule-Based Visual Reinforcement Fine-Tuning
• RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents
• Tencent/DigitalHuman/RLVER
• Rectifying LLM Thought from Lens of Optimization
• EEPO: Exploration-Enhanced Policy Optimization via Sample-Then-Forget
• ChanLiang/EEPO
• HiPO: Self-Hint Policy Optimization for RLVR
• Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
• Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning
• RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards
• The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
• Generalization of RLVR Using Causal Reasoning as a Testbed
• Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
• References Improve LLM Alignment in Non-Verifiable Domains
• Parameter-Efficient Reinforcement Learning using Prefix Optimization
• HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithmic Coding
• Agnostics: Learning to Synthesize Code in Any Programming Language with a Universal Reinforcement Learning Environment
• BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping
• WooooDyy/BAPO
• Learning to Reason as Action Abstractions with Scalable Mid-Training RL
• Diversity-Enhanced Reasoning for Subjective Questions
• PROS: Towards Compute-Efficient RLVR via Rollout Prefix Reuse
• LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
• real-absolute-AI/LongRLVR
• Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
• Lookahead Tree-Based Rollouts for Enhanced Trajectory-Level Exploration in Reinforcement Learning with Verifiable Rewards
• Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning
• Group Verification-based Policy Optimization for Interactive Coding Agents
• Beyond Magnitude: Leveraging Direction of RLVR Updates for LLM Reasoning
• Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning
• ReVeal: Self-Evolving Code Agents via Reliable Self-Verification
• Process-Verified Reinforcement Learning for Theorem Proving via Lean
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
• Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
• Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
• Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?
• Tina: Tiny Reasoning Models via LoRA
• Learning to Reason without External Rewards
• sunblaze-ucb/Intuitor
• $ extbf{Re}^{2}$: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
• Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models
• MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
• Diversity-Incentivized Exploration for Versatile Reasoning
• CORE: Concept-Oriented Reinforcement for Bridging the Definition–Application Gap in Mathematical Reasoning
• Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards
• ExGRPO: Learning to Reason from Experience
• ElliottYan/LUFFY
• MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model
• CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models
• LaSeR: Reinforcement Learning with Last-Token Self-Rewarding
• RUCBM/LaSeR
• Learn More with Less: Uncertainty Consistency Guided Query Selection for RLVR
• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
• Spotlight on Token Perception for Multimodal Reinforcement Learning
• SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Start
• Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
• DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Tree-based Search
• smiles724/DeepSearch
• No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping
• R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?
• meituan-longcat/R-HORIZON
• DuPO: Enabling Reliable Self-Verification via Dual Preference Optimization
• FAPO: Flawed-Aware Policy Optimization for Efficient and Reliable Reasoning
• Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Models
• Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning
• Agentic Reinforced Policy Optimization
• RUC-NLPIR/ARPO
• Beyond Pass@ 1: Self-Play with Variational Problem Synthesis Sustains RLVR
• MasterVito/SvS
• AnesSuite: A Comprehensive Benchmark and Dataset Suite for Anesthesiology Reasoning in LLMs
• Native Reasoning Models: Training Language Models to Reason on Unverifiable Data
• The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward
• Overthinking Reduction with Decoupled Rewards and Curriculum Data Scheduling
• Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
• Buffer Matters: Unleashing the Power of Off-Policy Reinforcement Learning in Large Language Model Reasoning
• QuRL: Low-Precision Reinforcement Learning for Efficient Reasoning
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward
• Perception-Aware Policy Optimization for Multimodal Reasoning
• Sample Lottery: Unsupervised Discovery of Critical Instances for LLM Reasoning
• Scheduling Your LLM Reinforcement Learning with Reasoning Trees
• Risk-Sensitive Reinforcement Learning for Alleviating Exploration Dilemmas in Large Language Models
• QuRL: Rubrics As Judge For Open-Ended Question Answering
• Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions
• Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners
• Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward
• How Far Can Unsupervised RLVR Scale LLM Training?
• TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoning
• Search Self-Play: Pushing the Frontier of Agent Capability without Supervision
• Qwen-Applications/SSP
• SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning
• spiral-rl/spiral
• From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation
• A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Models
• volcengine/verl/recipe/fapo

2026

• 1229095296/ResRL
• icip-cas/DCPO
• Spurious Rewards: Rethinking Training Signals in RLVR
• Discounted Beta-Bernoulli Reward Estimation for Sample-Efficient Reinforcement Learning with Verifiable Rewards
• Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewards
• The Unlearnability Phenomenon in RLVR for Language Models
• Evaluating Parameter Efficient Methods for RLVR