Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

awesomeopd

Awesome List for On-Policy Distillation
https://github.com/thinkwee/awesomeopd

Last synced: 4 days ago
JSON representation

  • 🤖 Agent & Embodied OPD (by application)

    • 🔁 Iterative Self-Bootstrapping

      • LLM4Teach - AMMI/LLM4Teach?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2023.11 (updated 2025) | ZJ Lab AMMI | [arXiv 2311.13373](https://arxiv.org/abs/2311.13373) | LLM4Teach — small-RL agent guided by LLM |
      • RPD - Policy-Distillation/RPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.03 | TUM / Freiburg | [arXiv 2503.05833](https://arxiv.org/abs/2503.05833) · [project](https://refined-policy-distillation.github.io/) | Refined Policy Distillation, VLA (IROS 2026) |
      • easydistill - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.09 | Alibaba ModelScope | [SCoRe arXiv 2509.14257](https://arxiv.org/abs/2509.14257) | `/projects/SCoRe` |
      • VLA-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2603.26666) | 2026.03 | HKUST (Guangzhou) — IRPN Lab | [arXiv 2603.26666](https://arxiv.org/abs/2603.26666) · [project](https://irpn-lab.github.io/VLA-OPD/) | **VLA-OPD** — bridging offline SFT & online RL for VLA via OPD (code coming soon) |
      • Skill-SD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2604.10674) | 2026.04 | Vivo | [arXiv 2604.10674](https://arxiv.org/abs/2604.10674) | Skill-SD — skill-conditioned self-distillation for multi-turn LLM agents|
      • TCOD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2604.24005) | 2026.04 | Tongyi Lab, Alibaba / CUHK | [arXiv 2604.24005](https://arxiv.org/abs/2604.24005) | TCOD — temporal curriculum OPD for multi-turn agents; F2B & B2F schedules |
      • Healthcare AI GYM - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | Upstage AI / Korea University | [arXiv 2605.02943](https://arxiv.org/abs/2605.02943) | Healthcare AI GYM — medical agent RL environment + turn-level truncated OPD |
      • HyperEyes - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | Xiaohongshu / Cambridge | [arXiv 2605.07177](https://arxiv.org/abs/2605.07177) | HyperEyes — parallel multimodal search agent with dual-grained efficiency-aware RL (TRACE + OPD) |
      • SDAR - REAL/SDAR?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | ZJU-REAL / Meituan | [arXiv 2605.15155](https://arxiv.org/abs/2605.15155) | **SDAR — Self-Distilled Agentic RL**; multi-turn agent rolls out, a privileged-context self-teacher gives gated token-level OPSD as an auxiliary objective while RL stays the primary backbone (ALFWorld, WebShop, Search-QA) |
      • TCOD - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.04 | Tongyi Lab, Alibaba / CUHK | [arXiv 2604.24005](https://arxiv.org/abs/2604.24005) | TCOD — temporal curriculum OPD for multi-turn agents; F2B & B2F schedules (COLM 2026) |
      • KbSD - RAG-R1?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | ZJU / PKU | [arXiv 2606.29863](https://arxiv.org/abs/2606.29863) | **KbSD — Knowledge Boundary aware Self-Distillation**; a structured boundary hint (parametric certainty, semantic stability, retrieval quality) is privileged context, and the *divergence direction itself* is chosen per behavioural quadrant. Unreliability rate −17 points at both 3B and 7B |
      • Two-Phase Distillation - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.30044) | 2026.06 | Tsinghua / Ant Group | [arXiv 2606.30044](https://arxiv.org/abs/2606.30044) | **Building Multi-Task Agentic LLMs via Two-Phase Distillation**; shows off-policy distillation degrades in *multi-task* settings not from exposure bias but from forward KL's mode-covering, then fixes it with an on-policy refinement phase |
      • SEED - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | Tsinghua / ZJU / CUHK / NTU / Tongji | [arXiv 2607.14777](https://arxiv.org/abs/2607.14777) | **SEED — Self-Evolving OPD for Agentic RL**; the policy becomes its own trajectory analyzer, converting completed rollouts into hindsight skills that condition a self-teacher. ALFWorld 91.8 vs 75.0 GRPO and 84.4 SDAR; matches full-data GRPO with 60% of the data |
      • ReOPD - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | Microsoft Research / Univ. of Amsterdam | [arXiv 2607.04763](https://arxiv.org/abs/2607.04763) | **ReOPD — Multi-Turn OPD with Prefix Replay**; reuses pre-collected teacher prefixes so the student acts at one step per sample and **never queries the environment** (zero tool calls, ≥4× faster). Beats fully online OPD on math (57.2 vs 55.1) ⚠️ see strictness note |
      • UI-MOPD - MOPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | Tsinghua SIGS / Xiaomi / HIT-SZ / ZJU / Peng Cheng Lab | [arXiv 2607.04425](https://arxiv.org/abs/2607.04425) | **UI-MOPD — Multi-Platform OPD for Continual GUI Agents**; replaces GRPO's KL term with a platform-routed teacher penalty, masked off wherever task reward is already strong. Fixes the cross-platform forgetting that makes single-platform SFT collapse the other platform to 0% |
      • TurnOPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.05804) | 2026.07 | Fudan / Tencent Hunyuan | [arXiv 2607.05804](https://arxiv.org/abs/2607.05804) | **TurnOPD — Making OPD Turn-Aware**; shows per-turn KL is front-loaded and its success/failure discriminability *inverts* with depth, while token-level normalisation leaves the deepest third of turns only 3.6–4.5% of the loss mass. 2.29× faster at higher accuracy |
      • FA-SD Failure - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.17558) | 2026.07 | Chapman Univ. / Lawrence Berkeley National Lab | [arXiv 2607.17558](https://arxiv.org/abs/2607.17558) | **Why Does Feedback-Augmented Self-Distillation Fail for Search Agents?** — a clean **negative result**: every FA-SD variant collapses within ~50 steps and loses to plain GRPO, with the mechanism named **decoding collapse** |
      • Tool-Call Boundary Drift - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.07050) | 2026.07 | Ant Group | [arXiv 2607.07050](https://arxiv.org/abs/2607.07050) | Diagnoses **"behavior leverage imbalance"** in multi-teacher tool-use OPD: mode-entry tokens like `<tool_call>` control the whole trajectory (top 1% of tokens = 41.2% of total JSD), so GKD roughly doubles over-calling (7.2%→14.2%); fixed by a batch-adaptive soft clamp |
      • ROAD-VLA - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.25800) | 2026.06 | UNSW Sydney / Deakin / AFRL | [arXiv 2606.25800](https://arxiv.org/abs/2606.25800) | **ROAD-VLA — Robust Online Adaptation via Self-Distillation for VLA**; reports that *text*-based privileged teachers fail for VLA adaptation (a modality gap), and builds an action-space proximal teacher instead ⚠️ see strictness note |
      • GUI-SD - ucas/GUI-SD-code?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | IIE CAS / Nankai | [arXiv 2605.00642](https://arxiv.org/abs/2605.00642) | **Learn where to Click from Yourself** — first OPSD for **GUI grounding**; teacher gets *visually* privileged context (ground-truth bbox + Gaussian soft-mask zoom) and supervision is positionally-weighted reverse KL **per coordinate digit token**. ScreenSpot-Pro 60.7% at 4× faster training than GRPO |
      • SOD - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | Tencent / ZJU / USTC / NUS | [arXiv 2605.07725](https://arxiv.org/abs/2605.07725) | **Step-wise OPD for Small Language Model Agents**; step weights from the ratio of consecutive per-step divergences attenuate supervision after tool-call errors cause drift. **A 0.6B student reaches 26.13% avg@32 on AIME 2025** |
      • LiteGUI - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.07505) | 2026.05 | Moore Threads AI | [arXiv 2605.07505](https://arxiv.org/abs/2605.07505) | **Distilling Compact GUI Agents**; "Guided-OPD" teacher-forces the teacher with a **Most-Matched-GT reference action dynamically retrieved to match the student’s own behaviour**. Lite-GUI-2B 46.86% on ScreenSpot-Pro, beating UI-TARS-72B |
      • Search-E1 - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.22511) | 2026.05 | Academic | [arXiv 2605.22511](https://arxiv.org/abs/2605.22511) | **Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning**; alternates GRPO with OPSD rounds whose privileged context is **the best sibling trajectory mined from the policy’s own rollout pool** |
      • GAPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.29584) | 2026.05 | USTC / CASIA / ShanghaiTech / Ant Group | [arXiv 2605.29584](https://arxiv.org/abs/2605.29584) | **Gold-Action Policy Distillation for Agentic RL in KBQA**; the policy conditioned on an entity-anchor-matched **gold next action** forms the self-teacher, fused additively with GRPO. GrailQA F1 86.1→89.5 |
      • MGSD - l/MGSD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Tsinghua / HKUST | [arXiv 2606.06076](https://arxiv.org/abs/2606.06076) | **Modality-Gap-Aware Self-Distillation**; a frozen **text-only** teacher conditioned on privileged symbolic state supervises an *image*-conditioned student, with symbolic context discarded at inference. Macro avg 17.2→35.6 |
      • HERO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.11559) | 2026.06 | UC San Diego / UC Berkeley | [arXiv 2606.11559](https://arxiv.org/abs/2606.11559) | **Hindsight-Enhanced Reflection from Environment Observations**; a compact turn-level hint conditions the self-teacher, explicitly beating naive full-demo privileged teachers. **Still learns from all-failed rollout groups where GRPO’s advantage collapses to zero** |
      • Guided-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.15912) | 2026.06 | CASIA / Tencent / USTC / ZJU / NUS | [arXiv 2606.15912](https://arxiv.org/abs/2606.15912) | **OPD with Curriculum Turn-level Guidance**; mixes teacher and student turns within a rollout on a cosine-decaying schedule, using **reverse KL on student turns and forward KL on teacher turns**. A 4B student surpasses its 30B teacher |
      • OPD-Evolver - evolver?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | LV-NUS Lab / Fudan / PKU / ByteDance | [arXiv 2606.17628](https://arxiv.org/abs/2606.17628) | **Cultivating Holistic Agent Evolver via OPD**; outcome-calibrated memory-value attribution turns delayed reward into hindsight context for a privileged teacher over a 4-tier agent memory. **9B beats Step-3.5-Flash (196B) on 9/10 subsets** |
      • Trust the Right Teacher - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.18101) | 2026.06 | Univ. of Georgia / INFLY Tech / Tencent AI Lab / HK PolyU | [arXiv 2606.18101](https://arxiv.org/abs/2606.18101) | **Quality-Aware Self-Distillation for GUI Grounding**; teacher sees layout-masked highlighted input, with gating that down-weights coordinate-token signals falling **outside the ground-truth box**. Macro-avg 72.23 vs naive OPSD 68.91 |
      • SAGE-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.19659) | 2026.06 | Meta AI | [arXiv 2606.19659](https://arxiv.org/abs/2606.19659) | **Selective Agent-Guided Intervention for Multi-Turn OPD**; a teacher-judged Skip/Weak/Strong gate decides *which turns* receive KL at all, weighted by teacher confidence. Up to +13.3% relative on ALFWorld unseen |
      • PBSD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.09348) | 2026.06 | SJTU / XYZ AI Lab | [arXiv 2606.09348](https://arxiv.org/abs/2606.09348) | **Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignment**; a turn-level Bayesian evidence score between a gold-answer-conditioned teacher and the student reweights the GRPO advantage. BrowseComp 46.21 vs 40.05 |
      • Gigapixel self-play driving - AD](https://arxiv.org/abs/2605.16241) are classic DAgger/behaviour-cloning — the student visits its own states but is regressed onto teacher-generated trajectories, so C2 fails. [SocraticPO](https://arxiv.org/abs/2606.09887) and [ZPPO](https://arxiv.org/abs/2606.18216) insert teacher text into the *prompt* only, training with plain RLVR. [TAPO](https://arxiv.org/abs/2606.18844) trains on self-synthesized corrective trajectories with outcome reward and explicitly reports that adding a distillation KL *underperforms*.
      • 2607.19395 - prefix-replay device as ReOPD, but the continuations are optimised by ordinary GRPO against *environment reward*; there is no teacher distribution, log-prob target, or KL anywhere. C1 ✓ / C2 ✗ — the non-distillation counterpart to ReOPD, and a useful contrast.

  • 🌟 Curator's Picks — where to start

  • 🛠️ Frameworks & Toolkits

    • 🔁 Iterative Self-Bootstrapping

      • trl - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2019.11 | Hugging Face | `trl/experimental/{gkd,gold,minillm,sdft,self_distillation,sdpo,nash_md,xpo,online_dpo}/` | TRL — **the most diverse OPD trainer collection** |
      • LLaMA-Factory - Factory?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2023.05 | hiyouga | — | LLaMA-Factory — OPD only via TRL integration; not native |
      • ms-swift - swift?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2024 | Alibaba ModelScope | `examples/train/rlhf/gkd/`, multimodal/megatron variants | ms-swift — wraps TRL `GKDTrainer` |
      • verl - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2024.10 | ByteDance Seed | `recipe/on_policy_distill/`; [Async OPD doc](https://verl.readthedocs.io/en/latest/advance/async-on-policy-distill.html) | verl |
      • rllm - org/rllm?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.01 | UC Berkeley Sky | `examples/math_distill/` (incl. `opsd/` self-distill); `rllm/trainer/distill/` | rllm |
      • SkyRL - AI/SkyRL?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.04 | UC Berkeley NovaSky | `skyrl-train/examples/on_policy_distillation/`; [blog](https://novasky-ai.notion.site/on-policy-distillation) | SkyRL |
      • ROLL - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.06 | Alibaba | `roll/pipeline/distill/` | ROLL — with VLM support and various-divergence library |
      • AReaL - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.06 | AntGroup / Tsinghua | `examples/distillation/gsm8k_grpo_distill.yaml` | AReaL |
      • slime - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.06 | Tsinghua THUDM | `examples/on_policy_distillation/` | slime — RL framework behind GLM-4.5/4.6/4.7 |
      • RL - NeMo/RL?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.01 | NVIDIA | `nemo_rl/algorithms/distillation.py` | NeMo-RL — native OPD with student rollouts |
      • KDFlow - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.03 | BJTU (Songming Zhang et al.) | `examples/on_policy_kd/` (LLM + Qwen3-VL); [arXiv 2603.01875](https://arxiv.org/abs/2603.01875) | KDFlow — **KD-first framework**; SGLang teacher + FSDP2 student decoupled; cross-tokenizer & VLM native |
      • spider - ai/spider?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.10 | Collinear AI | `server/on_policy.py`, `server/on_policy_utils.py`; recipes `scripts/train_on_policy_{precise_if,tool_search_nemo,swe}.py` | **Spider** — on/off-policy distillation engine; OPD in sandboxed multi-turn / tool / SWE-agent environments, with **cross-tokenizer** chat-template realignment (e.g. Qwen3-8B ← Kimi-K2-Thinking) |
      • async-opd - ai/async-opd?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | FuriosaAI (+ Ajou / Berkeley / MSR / KRAFTON) | [arXiv 2606.24143](https://arxiv.org/abs/2606.24143) | **AsyncOPD — How Stale Can OPD Be?**; fully asynchronous rollout ∥ teacher-scoring ∥ learner pipeline. Finds **forward KL is staleness-robust while reverse KL is not**, and that recomputing the RKL signal under the current student beats generic async-RL fixes. 1.6–3.8× throughput |
      • EasyOPD - ustc/EasyOPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | USTC / Tencent / Shanghai Innovation Institute / NUS | [arXiv 2607.11012](https://arxiv.org/abs/2607.11012) | **EasyOPD** — verl-based unified OPD framework; factors the method landscape into five extension points (loss, rollout metadata, reward, tokenizer alignment, **teacher sidecar**) so cross-tokenizer OPD, OPSD and step-wise OPD share one backend, selected by YAML |
      • EffOPD - ustc/EffOPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | USTC / Tencent / SHU / NUS / HKUST(GZ) / IIE-UCAS | [arXiv 2605.11739](https://arxiv.org/abs/2605.11739) | **Learning to Foresee** — shows OPD updates are low-rank and early-stabilized ("foresight") unlike RL, then extrapolates along the early update direction to cut training steps **~3×** |
      • SAfactory - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | Shanghai AI Laboratory / DeepLink | [arXiv 2605.06230](https://arxiv.org/abs/2605.06230) | **SAfactory** — agent-safety infrastructure whose Autonomous Evolution Platform ships an optional OPD plugin (teacher token log-probs + reverse-KL regularization inside agentic RL) ⚠️ OPD is one module, with no standalone ablation |

  • 🏭 Industrial / Production Model Reports

    • 🔁 Iterative Self-Bootstrapping

      • gemma - deepmind/gemma?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2024.07 | Google DeepMind | [arXiv 2408.00118](https://arxiv.org/abs/2408.00118) | **Gemma 2** (explicit OPD) |
      • Qwen3 - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.05 | Alibaba Qwen | [arXiv 2505.09388](https://arxiv.org/abs/2505.09388) | **Qwen3** (canonical OPD recipe) |
      • GLM-4.5 - org/GLM-4.5?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.08 | Zhipu / Z.ai | [arXiv 2508.06471](https://arxiv.org/abs/2508.06471) | **GLM-4.5 / 4.6** |
      • HY-MT - Hunyuan/HY-MT?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.12 | Tencent Hunyuan | [arXiv 2512.24092](https://arxiv.org/abs/2512.24092) · [HF 1.8B](https://huggingface.co/tencent/HY-MT1.5-1.8B) · [HF 7B](https://huggingface.co/tencent/HY-MT1.5-7B) | strong-to-weak distillation for MT |
      • MiMo-V2-Flash - V2-Flash?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.01 | Xiaomi | [arXiv 2601.02780](https://arxiv.org/abs/2601.02780) | **MiMo-V2-Flash** (MOPD) |
      • Typhoon-S - 10x/typhoon-s?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.01 | Typhoon AI & SCB 10X | [arXiv 2601.18129](https://arxiv.org/pdf/2601.18129) | GAD style OPD: Full logits greatly outperforms Top-K in Thai.|
      • Baichuan-M3-235B - inc/Baichuan-M3-235B?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.02 | Baichuan | [arXiv 2602.06570](https://arxiv.org/abs/2602.06570) · [HF Collection](https://huggingface.co/collections/baichuan-inc/baichuan-m3) | Baichuan-M3 (learn critically from multi-teacher OPD) |
      • GLM-5 - org/GLM-5?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.02 | Zhipu / Z.ai | [arXiv 2602.15763](https://arxiv.org/abs/2602.15763) | **GLM-5** (cross-stage OPD) |
      • Nemotron Cascade 2 - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2603.19220) | 2026.03 | NVIDIA | [arXiv 2603.19220](https://arxiv.org/abs/2603.19220) · [HF Collection](https://huggingface.co/collections/nvidia/nemotron-cascade-2) · [project](https://research.nvidia.com/labs/nemotron/nemotron-cascade-2/) | **Nemotron Cascade 2** (multi-domain OPD; "we sample y∼π_inf(·\|x)"); HF-only release |
      • Qwen3-Coder - Coder?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.03 | Alibaba Qwen | Tech report | Qwen3-Coder |
      • KAT-Coder-V2 - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2603.27703) | 2026.03 | Kuaishou KwaiKAT | [arXiv 2603.27703](https://arxiv.org/abs/2603.27703) | step-level OPD for agentic coding|
      • HY-Embodied - Hunyuan/HY-Embodied?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.04 | Tencent Hunyuan | [arXiv 2604.07430](https://arxiv.org/abs/2604.07430) | **HY-Embodied-0.5** (FKL embodied distillation) |
      • DeepSeek-V4 - paper-845C40?style=for-the-badge)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf) | 2026.04 | DeepSeek-AI | [Tech Report](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf) · [V4-Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) · [V4-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) | **DeepSeek-V4** (multi-teacher OPD replaces unified mixed-RL stage) |
      • ![Paper - studio/qwen-omni) | **Qwen3.5-Omni** (cross-modal OPD for audio reasoning) |
      • RLSD
      • Qwen-Image-2.0-RL - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.27608) | 2026.06 | Alibaba Qwen | [arXiv 2606.27608](https://arxiv.org/abs/2606.27608) | **Qwen-Image-2.0-RL** — MOPD for **image generation**: two GRPO reward-specialist teachers consolidated via trajectory-level velocity matching along the student's own reverse-ODE path (report-only) |
      • Agents-A1 - A1?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Shanghai AI Laboratory | [arXiv 2606.30616](https://arxiv.org/abs/2606.30616) · [HF Collection](https://huggingface.co/collections/InternScience/agents-a1) | **Agents-A1** (35B MoE) — Domain-Routed OPD with **Salient Vocabulary Alignment**: truncated reverse KL over a *teacher*-selected top-k support, with domain-normalised aggregation. Beats 1T-class models on SEAL-0 / IFBench / HiPhO |
      • Mach-Mind-4-Flash - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.09375) | 2026.07 | Li Auto | [arXiv 2607.09375](https://arxiv.org/abs/2607.09375) | **Mach-Mind-4-Flash** (35B-A3B) — MOPD across Reasoning / General / Agent expert tracks; ships the clearest **"RL and OPD as one system"** design, switching between pure RL, pure distillation and joint modes through a single weighted loss (report-only) |
      • OvisOCR2 - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.13639) | 2026.07 | ATH-MaaS, Alibaba | [arXiv 2607.13639](https://arxiv.org/abs/2607.13639) · [HF](https://huggingface.co/ATH-MaaS/OvisOCR2) | **OvisOCR2** (0.8B) — the counter-example to the MOPD trend: OPD for classic **large→small compression** (4B RL teacher → 0.8B student) with student-selected top-k reverse KL. SOTA 96.58 on OmniDocBench v1.6 |
      • Solar Open 2 - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.20062) | 2026.07 | Upstage | [arXiv 2607.20062](https://arxiv.org/abs/2607.20062) · [HF](https://huggingface.co/upstage/Solar-Open2-250B) | **Solar Open 2** (250B-A15B, 1M ctx) — the most detailed MOPD writeup to date: **twelve** routed specialist teachers, **exact full-vocabulary** reverse KL, and **no outcome reward at all**, which lets consolidation run at 1 rollout per prompt |
      • Audex - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.05196) | 2026.07 | NVIDIA | [arXiv 2607.05196](https://arxiv.org/abs/2607.05196) · [HF Collection](https://huggingface.co/collections/nvidia/nemotron-labs-audex) | Nemotron-Labs-Audex-30B-A3B — MOPD as an inherited stage of the [Nemotron Cascade 2](https://arxiv.org/abs/2603.19220) recipe; reports the honest observation that **MOPD degrades ASR/MMSU before two later RL stages recover it** ⚠️ see strictness note |
      • Kwai Keye-VL-2.0 - Keye/Keye?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Kuaishou (Keye Team) | [arXiv 2606.10651](https://arxiv.org/abs/2606.10651) | **Kwai Keye-VL-2.0** (30B-A3B MoE) — Cross-Modal MOPD over **13 RL-trained domain experts**, the largest teacher pool of any report here; top-k-overlap advantage on student-visited states. SOTA long-video understanding at its scale |
      • Nemotron 3 Ultra - NeMo/Nemotron?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | NVIDIA | [arXiv 2606.15007](https://arxiv.org/abs/2606.15007) | **Nemotron 3 Ultra** — SFT → RLVR → **2 iterations of MOPD over 10+ domain teachers**, run asynchronously with PPO-style clipping around a proximal policy; the report frames MOPD explicitly as the dense-supervision answer to sparse RLVR reward. SWE-Bench Verified 70.7 |

  • 🖼️ Multimodal OPD (VLM, Video, Audio, Image)

    • 🔁 Iterative Self-Bootstrapping

      • piFlow - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.10 | Multi-org | [arXiv 2510.14974](https://arxiv.org/abs/2510.14974) | π-Flow — image / flow OPD (ICLR 2026) |
      • VOLD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2510.23497) | 2025.10 | INRIA / Goethe Univ. | [arXiv 2510.23497](https://arxiv.org/abs/2510.23497) · [project page](https://walidbousselham.com/VOLD/) | VOLD (LLM→VLM OPD) — repo placeholder; ICLR 2026 |
      • Step-Audio-R1 - ai/Step-Audio-R1?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.11 | StepFun | [arXiv 2511.15848](https://arxiv.org/abs/2511.15848) | Step-Audio-R1 |
      • CORD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2601.16547) | 2026.01 | Baidu Ernie | [arXiv 2601.16547](https://arxiv.org/abs/2601.16547) | Reasoning: Text ➡️ Audio |
      • Video-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2602.02994) | 2026.02 | Industrial | [arXiv 2602.02994](https://arxiv.org/abs/2602.02994) | Video-OPD |
      • X-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2603.24596) | 2026.03 | Tencent Hunyuan / ZJU | [arXiv 2603.24596](https://arxiv.org/abs/2603.24596) | X-OPD (Speech LLM) |
      • Uni-OPD - OPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | Multi-org | [arXiv 2605.03677](https://arxiv.org/abs/2605.03677) | **Uni-OPD** — unified OPD across LLMs & MLLMs via dual-perspective recipe |
      • Flow-OPD - OPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | Multi-org | [arXiv 2605.08063](https://arxiv.org/abs/2605.08063) | **Flow-OPD** — first to integrate OPD into Flow-Matching text-to-image models; consolidates multiple single-reward GRPO expert teachers into one student via on-policy sampling + reverse-KL (SD-3.5-Medium) |
      • Decomposed-OPD - suk-yoon/Decomposed_OPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | KAIST / Microsoft Research Asia | [arXiv 2606.00564](https://arxiv.org/abs/2606.00564) | **Decomposed-OPD / VGS** — decomposes the OPD gradient into (near-orthogonal) language-prior vs. visual-grounding components; *Visual Gradient Steering* reorients updates toward the visual subspace for VLM reasoning |
      • DiffusionOPD - vilab/DiffusionOPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | Fudan / Wan Team, Alibaba | [arXiv 2605.15055](https://arxiv.org/abs/2605.15055) | **DiffusionOPD — A Unified Perspective of OPD in Diffusion Models**; lifts token-level KL to a continuous-state Markov chain where identical kernel covariance collapses reverse KL into closed-form L2 transition matching, consolidating multiple single-reward RL experts. Avg 0.929 vs 0.851 Cascade NFT at ~⅔ the GPU-hours |
      • V-Zero - group-SCU/V-Zero?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Sichuan Univ. / XJTU / TeleAI / PKU | [arXiv 2606.25319](https://arxiv.org/abs/2606.25319) | **V-Zero — Answer-Label-Free OPD with Contrastive Evidence Gating**; the teacher replays the student's tokens under a *positive* question-relevant crop and a *negative* off-region crop, and the log-likelihood gap gates the OPD loss. No answer labels, verifiable rewards, or inference-time tools |
      • H-OPD - OPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | BUPT / ByteDance / USTC / Zhongguancun Academy | [arXiv 2607.02592](https://arxiv.org/abs/2607.02592) | **H-OPD — Confidence-Aware Heterogeneous Multi-Teacher Multimodal OPD**; replaces sample-level teacher routing with **token-level arbitration** between a vision-language teacher and a text-only teacher fed a precomputed image description |
      • OPSD-V - AI/OPSD-V?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | Meituan (MeiGen) / HKUST / CityU HK | [arXiv 2607.08766](https://arxiv.org/abs/2607.08766) | **OPSD-V** — OPSD for few-step autoregressive **video** generators; real long videos become privileged *temporal context* in the teacher's KV cache rather than reconstruction targets, fixing long-horizon drift at unchanged NFE |
      • Med-OPD - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | NUS / USTC | [arXiv 2607.16303](https://arxiv.org/abs/2607.16303) | **Med-OPD — Evidence-Aware OPD for Medical VLMs**; counterfactually degrades the image to isolate which tokens actually depend on imaging evidence (top 10% carry ~88% of the signal), then reweights the KL accordingly |
      • OPD-IAD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.18850) | 2026.07 | Tsinghua / Guangdong Lab of AI & Digital Economy / SZU | [arXiv 2607.18850](https://arxiv.org/abs/2607.18850) | **OPD-IAD — From Language Judgment to Industrial Anomaly Detection**; evidence-privileged dense OPSD (EMA self-teacher sees a training-only defect-analysis reference) plus language-guided visual anchoring for pixel-level maps |
      • VISD - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | HUST / WHU / PKU / Tsinghua | [arXiv 2605.06094](https://arxiv.org/abs/2605.06094) | **Structured Self-Distillation for Video Reasoning**; an EMA self-teacher conditioned on a **video-aware judge’s** correctness + temporal/spatial grounding modulates advantage magnitude while the rule reward sets direction. Matches VisionCoach with ~50% of its steps |
      • AnyFlow - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | NVIDIA / Show Lab NUS / MIT | [arXiv 2605.13724](https://arxiv.org/abs/2605.13724) | **Any-Step Video Diffusion via On-Policy Flow Map Distillation**; the student rolls out its own flow-map backward simulation and a frozen teacher supplies DMD-style reverse-divergence supervision. **VBench 84.05 at 4 NFEs** |
      • DeltaPrompts - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.15532) | 2026.05 | NVIDIA Research | [arXiv 2605.15532](https://arxiv.org/abs/2605.15532) | **Escaping the Zero-Delta Trap in Multimodal Distillation**; finds **up to 69% of off-the-shelf chart/document prompts are "zero-delta"** — teacher and student already agree, so they carry no signal — and synthesizes answer-divergent prompts instead |
      • Visual-Advantage OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.21924) | 2026.05 | CASIA / UCAS / Hello Group / SYSU | [arXiv 2605.21924](https://arxiv.org/abs/2605.21924) | **Visual-Advantage OPD for VLMs**; reweights teacher supervision by the teacher’s log-prob gap **with vs without fine-grained visual detail**, at both rollout and token granularity |
      • EchoDistill - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.23954) | 2026.05 | NTU SG / SHU / ICT-CAS / HDU / BUPT / USTC | [arXiv 2605.23954](https://arxiv.org/abs/2605.23954) | **Noisy-to-Clean Self-Distillation for Robust Audio LLMs**; the privileged context is **clean audio** while the student hears a corrupted signal. Avg GSR 79.28% at SNR −10 |
      • ASR-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.28139) | 2026.05 | AutoArk-AI | [arXiv 2605.28139](https://arxiv.org/abs/2605.28139) | **Data-Efficient OPD for Automatic Speech Recognition**; teacher re-scores the student’s own transcript in teacher-forcing mode with KL over the union of top-k supports. **1.95% CER on AISHELL-1 using 100k hours vs Qwen3-Omni’s 20M** |
      • ViCuR - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Shanghai AI Lab / Fudan / Nanjing Univ. | [arXiv 2606.05718](https://arxiv.org/abs/2606.05718) | **Visual Cues as Recoverable Privilege**; privileges only cues **derivable from the same input**, with a sink-token module letting the student internally recover them — the direct answer to privilege the student can never reproduce |
      • Imagine-OPD - OPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | PKU / CSU / USTC / Huawei | [arXiv 2606.08719](https://arxiv.org/abs/2606.08719) | **Thinking Without Images**; teacher sees privileged zoomed-in ground-truth crops, student reasons from the full image only. Beats explicit tool-call "thinking with images" baselines at **1.5–2.7× faster inference** |
      • GNDPO - Mente-Lab/GNDPO?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | OPPO AI Center | [arXiv 2606.09091](https://arxiv.org/abs/2606.09091) | **Stabilizing OPD for MLLM Reasoning with Global Normalization**; fixes gradient explosions from OOD low-probability teacher tokens by batch-normalizing KL rewards into relative advantages under a GSPO-style clip |
      • ParaBridge - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.10581) | 2026.06 | CUHK-SZ / Tencent Hunyuan / Amphion / Tsinghua | [arXiv 2606.10581](https://arxiv.org/abs/2606.10581) | **Bridging Paralinguistic Perception and Dialogue Behavior in Speech LMs**; internalizes a **paralinguistic instruction scaffold** by symmetric JSD. Scaffold-free VoxSafeBench SAR 14.6%→40.3% |
      • OmniOPSD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.15920) | 2026.06 | Shenzhen Univ. / Guangdong Lab / Tsinghua / SJTU / Huawei | [arXiv 2606.15920](https://arxiv.org/abs/2606.15920) | **Rationale-Privileged OPSD for Affective Computing**; a local self-teacher conditioned on a frontier-model evidence rationale, **3.2× faster than external-teacher OPD** |
      • ViGOS - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | SIA CAS / UCAS | [arXiv 2606.19120](https://arxiv.org/abs/2606.19120) | **Seeing Before Reasoning**; *two different teachers supervise different segments of one rollout* — an image-only perception teacher for the description, an answer-privileged teacher for the reasoning. Mean Pass@5 68.13→75.60 |
      • WMSD - stapf/world-model-self-distillation?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | University of Bern | [arXiv 2606.12072](https://arxiv.org/abs/2606.12072) | **World Model Self-Distillation**; a video-diffusion Executor samples its own rollouts and a Demonstrator teacher conditioned on a **privileged detailed execution description** gives per-timestep velocity supervision. The Executor **surpasses its own teacher** |
      • GeoStream - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.15162) | 2026.06 | CMU / Northeastern / UIUC / Rice / Snap / KAUST | [arXiv 2606.15162](https://arxiv.org/abs/2606.15162) | **Precise Camera Controlled Streaming Video Generation**; a causal student is distilled on-policy from a **bidirectional** (privileged full-context) teacher via DMD, closing both exposure bias and a second-order geometric feedback loop |
      • MaineCoon - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.17800) | 2026.06 | Catnip AI | [arXiv 2606.17800](https://arxiv.org/abs/2606.17800) | **Real-Time Audio-Visual Social World Model**; its ROPD stage blends a domain-expert teacher’s per-step velocity target with the student’s own via a **reward-adaptive weight**. 0.934 vs 0.895 best baseline at 47.5 FPS |
      • PF-OPSD - OPSD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Univ. of Macau / LIGHTSPEED | [arXiv 2606.03603](https://arxiv.org/abs/2606.03603) | **World Models Meet Language Models**; a privileged evaluator that **sees the ground-truth future video** scores the student’s own trajectories, teaching the MLLM when to invoke and trust a video world model. +10.6/+10.9 over SFT |
      • InternVideo3 - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Shanghai AI Lab / Shanghai Innovation Institute / Nanjing Univ. | [arXiv 2606.12195](https://arxiv.org/abs/2606.12195) | **InternVideo3**; OPD is the final stage of a 4-stage recipe, applied to examples filtered for a **teacher>student gap**. Best open-weight Video-MME 73.8 |
      • 2607.19850 - policy sampling, but the paper's own architecture figure shows ground-truth tokens fed to *both* teacher and student with a label mask, which is the signature of a teacher-forced offline KD pipeline over the fixed SNEI/MUSON label sets; no rollout infrastructure is described and no code is released to arbitrate.
      • piFlow - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.10 | Multi-org | [arXiv 2510.14974](https://arxiv.org/abs/2510.14974) | π-Flow — image / flow OPD (ICLR 2026) |

  • 🤝 OPD-RL Hybrids — Inside-RL OPD

    • 🔁 Iterative Self-Bootstrapping

      • BOND - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2407.14622) | 2024.07 | Google DeepMind | [arXiv 2407.14622](https://arxiv.org/abs/2407.14622) | BOND (Best-of-N Distillation) |
      • Faster WIND - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2410.20727) | 2024.10 | CMU / Google | [arXiv 2410.20727](https://arxiv.org/abs/2410.20727) | Faster WIND (iterative BoN) — AISTATS 2025 |
      • AlignDistil - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.03 | BJTU / Tencent | [arXiv 2503.02832](https://arxiv.org/abs/2503.02832) | AlignDistil — RLHF-equivalent KD (ACL 2025) |
      • LUFFY - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.04 | Westlake U. | [arXiv 2504.14945](https://arxiv.org/abs/2504.14945) | LUFFY — mixed-policy GRPO |
      • KETCHUP - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2504.19024) | 2025.04 | U. Alberta | [arXiv 2504.19024](https://arxiv.org/abs/2504.19024) | KETCHUP (k-step RL-KD) |
      • KDRL - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2506.02208) | 2025.06 | HIT / Huawei | [arXiv 2506.02208](https://arxiv.org/abs/2506.02208) | KDRL (Joint KD + RL) |
      • SDPO - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.01 | ETH / MIT | [arXiv 2601.20802](https://arxiv.org/abs/2601.20802) · [project](https://self-distillation.github.io/SDPO) | SDPO — RL via Self-Distillation |
      • KEPO - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.01 | Industrial | [arXiv 2602.00400](https://arxiv.org/abs/2602.00400) | KEPO |
      • Open-AgentRL - Verse/Open-AgentRL?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.02 | Gen-Verse | — | Open-AgentRL — RLAnything / DemyAgent multi-domain |
      • Towards-On-Policy-SFT - On-Policy-SFT?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.02 | MSRA / Shopee | [arXiv 2602.12222](https://arxiv.org/abs/2602.12222) | DDT — on-policy SFT theory |
      • 𝒳-KD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2602.12674) | 2026.02 | BUPT | [arXiv 2602.12674](https://arxiv.org/abs/2602.12674) | 𝒳-KD (IRL-style) |
      • RLAD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2602.22495) | 2026.02 | AWS | [arXiv 2602.22495](https://arxiv.org/abs/2602.22495) | RLAD (Reinforcement-aware KD) |
      • OpenClaw-RL - Verse/OpenClaw-RL?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.03 | Gen-Verse | [arXiv 2603.10165](https://arxiv.org/abs/2603.10165) | OpenClaw-RL — combines GRPO + OPD |
      • ExGRPO - Tan-dmml/ExGRPO?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.03 | UNC / ASU | [arXiv 2603.19266](https://arxiv.org/abs/2603.19266) | Probing-to-Refine / EI / EXGRPO |
      • HDPO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2603.23871) | 2026.03 | NVIDIA | [arXiv 2603.23871](https://arxiv.org/abs/2603.23871) | HDPO (Hybrid Distillation PO) |
      • RLSD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2604.03128) | 2026.04 | Multi-org | [arXiv 2604.03128](https://arxiv.org/abs/2604.03128) | Self-Distilled RLVR (RLSD) |
      • NPO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2604.20733) | 2026.04 | IIE CAS / UCAS / JD.COM | [arXiv 2604.20733](https://arxiv.org/abs/2604.20733) | NPO / AutoNPO — mixed-policy GRPO with **near-future self** as teacher |
      • ROSD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.28014) | 2026.05 | PolyU / Baidu | [arXiv 2605.28014](https://arxiv.org/abs/2605.28014) | ROSD — reflective error-localized self-distillation |
      • TGPO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.13230) | 2026.05 | NEU NLP / Meituan | [arXiv 2605.13230](https://arxiv.org/abs/2605.13230) | **TGPO — Teacher-Guided Policy Optimization**; teacher directly guides token-level generation conditioned on student contexts, fused with RLVR trajectory rewards; targets *large* teacher–student divergence |
      • OPD+ - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.01039) | 2026.06 | Columbia / Capital One | [arXiv 2606.01039](https://arxiv.org/abs/2606.01039) | **OPD+ — Rethinking the Advantage Design**; formulates OPD as RL with an f-divergence reward, proves the common stop-gradient advantage estimator is biased & gives a corrected estimator |
      • OPID - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Tsinghua / ZJU / CUHK / NTU / Tongji | [arXiv 2606.26790](https://arxiv.org/abs/2606.26790) | **OPID — On-Policy Skill Distillation**; turns completed agent trajectories into hierarchical hindsight skills, then re-scores the *same* sampled response under skill-augmented context; needs no external skill memory or retrieval at inference |
      • ATOD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.27814) | 2026.06 | Tencent / Tsinghua | [arXiv 2606.27814](https://arxiv.org/abs/2606.27814) | **ATOD — Annealed Turn-aware OPD**; fuses an external-teacher OPD advantage with GRPO under an annealed schedule (OPD-dominant early, RL-dominant late) plus a turn-level disagreement×entropy weight. **Surpasses its own teacher by +2.16** |
      • PASS - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.29296) | 2026.06 | Tsinghua / WeChat Tencent | [arXiv 2606.29296](https://arxiv.org/abs/2606.29296) | **Process Advantage Signal Shaping** — paradigm-agnostic middleware between any step-level process signal and GRPO; its OPD/G-OPD instantiation lifts multi-hop QA pass@1 by +5.50 |
      • CRAFT - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.29476) | 2026.06 | HKUST | [arXiv 2606.29476](https://arxiv.org/abs/2606.29476) | **CRAFT — Counterfactual Credit from Free Sibling Rollouts**; reuses the G−1 sibling GRPO rollouts to turn the teacher–student gap into a **signed** bounded per-token credit at ~zero extra compute; a credit-assignment upgrade to [SDAR](#-agent--embodied-opd-by-application) |
      • UCOB - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | CASIA / Pengcheng Lab / Memorax AI | [arXiv 2606.29502](https://arxiv.org/abs/2606.29502) | **UCOB — Credit-Aware Bidirectional Self-Distillation**; shows the usual privileged-teacher assumption (skill-conditioned ⇒ better) is **empirically false**, so it distills whichever branch actually won — in either direction |
      • DRIFT - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.30345) | 2026.06 | Beike (Ke.com) / Tsinghua / ENS Paris-Saclay | [arXiv 2606.30345](https://arxiv.org/abs/2606.30345) | **DRIFT — Difficulty Routing Self-Distillation**; routes *incorrect* samples to SDPO (a wrong trajectory gets a token-by-token correct path to compare against) and *correct* ones to gated GRPO, with a cross-batch success buffer keeping supervision alive when positives are scarce |
      • RG-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.04037) | 2026.07 | Univ. of Cologne / AWS AI / UT Austin / MIT | [arXiv 2607.04037](https://arxiv.org/abs/2607.04037) | **Reward-Gated OPD**; keeps a trajectory only when verifier reward and teacher–student likelihood gap are *directionally aligned*, then applies top-k reverse KL with an explicit tail-correction bucket. The cleanest verifier-gated entry |
      • CPO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.14614) | 2026.07 | CUHK / SCUT / NTU | [arXiv 2607.14614](https://arxiv.org/abs/2607.14614) | **CPO — Beyond Entropy**; shapes the GRPO advantage by a token-level posterior-vs-prior log-ratio and **proves OPD's reverse KL is a special case** of it. +7.7% / +8.5% over GRPO on pass@16 |
      • Distilled RL - RL?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.07 | Nankai / Zhongguancun Academy / BIT / ZJU / CASIA / HIT | [arXiv 2607.17247](https://arxiv.org/abs/2607.17247) | **Distilled RL**; drops the KL loss entirely and injects the teacher as a *reverse importance ratio* on the GRPO advantage, with a negative-sample reset and sequence-level geometric normalisation so the teacher only redistributes credit |
      • CriPO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.18082) | 2026.07 | ZJU / ByteDance | [arXiv 2607.18082](https://arxiv.org/abs/2607.18082) | **CriPO — Enhancing Rubric-based RL via Self-Distillation**; names *Suppressed Criteria* alongside Unexplored Criteria and fixes both with criterion-injected forward-KL behaviour injection + counterfactual advantage flipping |
      • H²SD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.18955) | 2026.07 | Shanghai AI Lab / HIT / Fudan / CUHK | [arXiv 2607.18955](https://arxiv.org/abs/2607.18955) | **H²SD — Hybrid Hindsight Self-Distillation**; routes by correctness — successes get magnitude modulation, failures get hint-conditioned reverse KL. Ships a rare head-to-head ranking of *privileged-context types* (hint ≫ sibling solution ≫ ground truth ≫ verifier feedback) |
      • CADENCE - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2607.16955) | 2026.07 | Independent (Kumar & Jha) | [arXiv 2607.16955](https://arxiv.org/abs/2607.16955) | CADENCE — coverage-adaptive FKL↔RKL scheduling + dense partial-credit reward; notable for being trained end-to-end on a single Apple Mac Studio |
      • CoPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2604.27083) | 2026.04 | JD.COM / IIE CAS / UCAS | [arXiv 2604.27083](https://arxiv.org/abs/2604.27083) | **Co-Evolving Policy Distillation**; parallel expert branches alternate branch-specific GRPO with **mutual OPD** — each branch distills from a co-evolving *sibling* checkpoint on the other’s data |
      • RLRT - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.10781) | 2026.05 | Microsoft Research / KAIST | [arXiv 2605.10781](https://arxiv.org/abs/2605.10781) | **Rebellious Student — Reversing Teacher Signals**; *amplifies* rather than suppresses tokens where the student diverged from the privileged teacher **and still succeeded**. avg@16 +18.0% on Qwen3-4B-Base |
      • GEAR - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.11853) | 2026.05 | HKUST / MSRA | [arXiv 2605.11853](https://arxiv.org/abs/2605.11853) | **Granularity-Adaptive Advantage Reweighting**; entropy-bounded segmentation of reverse-KL spikes down-weights high-divergence tokens in successful trajectories and up-weights them in failed ones |
      • Sparse-to-Dense - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.12483) | 2026.05 | Academic | [arXiv 2605.12483](https://arxiv.org/abs/2605.12483) | **Beyond GRPO and OPD: An Empirical Sparse-to-Dense Reward Principle**; RL-train a large teacher, bridge to the student with dense OPD, then optionally resume sparse RL. 79.3% MATH avg@16 vs 75.9% direct GRPO |
      • AMR-SD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.18529) | 2026.05 | Meituan / UCAS | [arXiv 2605.18529](https://arxiv.org/abs/2605.18529) | **Asymmetric Meta-Reflective Self-Distillation**; the self-teacher is conditioned on a **self-generated Socratic critique** rather than ground truth, and its ReLU-thresholded log-ratio ("causal information gain") shapes the GRPO advantage |
      • SD-Search - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.18299) | 2026.05 | Kuaishou | [arXiv 2605.18299](https://arxiv.org/abs/2605.18299) | **On-Policy Hindsight Self-Distillation for Search-Augmented Reasoning**; the teacher additionally sees sibling rollouts’ queries and outcome labels, with JSD applied **only at search-query positions**. Matches a 72B-teacher process-supervision baseline with no external teacher |
      • CEPO - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | MBZUAI / Linköping / ANU | [arXiv 2605.19436](https://arxiv.org/abs/2605.19436) | **Contrastive Evidence Policy Optimization**; replaces RLSD’s single privileged-answer ratio with a **contrastive** one (correct-answer teacher vs rejected-peer teacher), provably leakage-free |
      • OPPO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.21851) | 2026.05 | George Washington Univ. / UT Dallas | [arXiv 2605.21851](https://arxiv.org/abs/2605.21851) | **Bayesian Value Recursion for Token-Level Credit Assignment**; derives a closed-form Bayesian token advantage from the OPD log-ratio and adds it to the GRPO/DAPO advantage. AMC’23 +6.0 |
      • DASD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.22263) | 2026.05 | HIT Shenzhen / Peng Cheng Lab / Keeta AI | [arXiv 2605.22263](https://arxiv.org/abs/2605.22263) | **Direction-Adaptive Self-Distillation**; an entropy-routed **signed** coefficient makes the KL term attractive on low-entropy scaffolding tokens and *repulsive* on high-entropy forking tokens. Avg@16 52.5 vs GRPO 46.5 |
      • StepOPSD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.27140) | 2026.05 | Independent / Tencent / DeepWisdom | [arXiv 2605.27140](https://arxiv.org/abs/2605.27140) | **Step-Aware Online Preference Distillation**; parses agent trajectories into action-centered step segments rescored by a hindsight-privileged self-teacher, folded in as a clipped multiplicative advantage reweighting |
      • LGR - jiang/LGR?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | UCAS / ISCAS / Xiaohongshu | [arXiv 2605.30833](https://arxiv.org/abs/2605.30833) | **Combating Supervision Fidelity Decay in OPD**; adds a **Lookahead Group Reward** over top-K candidate tokens using teacher confidence *one step ahead*, with entropy-triggered tree attention |
      • CAST - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.00172) | 2026.06 | PKU | [arXiv 2606.00172](https://arxiv.org/abs/2606.00172) | **Non-Privileged Clipped Asymmetric Self-Teaching**; the self-teacher sees **no privileged information at all**, and its gap drives bidirectional advantage-sign reversal — including for zero-variance groups. AIME24 41.25 vs GRPO 21.04 |
      • SDPG - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | UCLA / Princeton AI Lab | [arXiv 2606.04036](https://arxiv.org/abs/2606.04036) | **Self-Distilled Policy Gradient**; GRPO plus an *exact full-vocabulary* reverse-KL self-distillation term, positive-advantage-masked and warmup-decayed |
      • SG-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.09304) | 2026.06 | ZJU / Hunan / Tianjin / SJTU / Jilin | [arXiv 2606.09304](https://arxiv.org/abs/2606.09304) | **Sign-Gated OPD**; teacher advantages are gated against the verifier advantage — consensus tokens extrapolated, conflict tokens softened — plus phased teacher-rollout injection at cold start. pass@32 59.17 vs 51.67 |
      • Tokenizer Barrier - Policy-Distill?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | HKUST(GZ) / Tencent | [arXiv 2606.09456](https://arxiv.org/abs/2606.09456) | **Breaking the Tokenizer Barrier: OPD across Model Families**; dual-pointer chunk alignment projects a different-family teacher’s chunk log-probability onto mismatched student tokens. Matches same-tokenizer OPD gains at **~4% of the FLOPs** of equivalent SFT scaling |
      • Sibling-Guided Credit - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.12634) | 2026.06 | AWS | [arXiv 2606.12634](https://arxiv.org/abs/2606.12634) | **Keep Policy Gradient in Charge**; an external LLM turns sibling-rollout contrast into a *training-only* credit reference, and the top-K reverse KL between clean and credit-conditioned scorings reweights token advantages |
      • SC-GRPO - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.18810) | 2026.06 | BIT / Beihang | [arXiv 2606.18810](https://arxiv.org/abs/2606.18810) | **Self-Conditioned Credit Assignment for RLVR**; conditions on one of the model’s own verifier-approved trajectories, using the resulting KL **purely as a multiplicative advantage weight** — additive-KL variants underperform. +8.1% over GRPO |
      • Long-Context Recipe - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2605.12227) | 2026.05 | IST Lisboa / IT / TransPerfect / ELLIS Lisbon | [arXiv 2605.12227](https://arxiv.org/abs/2605.12227) | **A Recipe for Long-Context Reasoning via On-Policy Optimization and Distillation**; replaces GRPO’s reference-KL anchor with a stronger external teacher. **128K-token RULER 11.0→44.5** |

  • 🎭 OPD with Black-Box / Outcome-Based Teachers

    • ORPO-Distill - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2509.25100) | 2025.09 | Industrial | [arXiv 2509.25100](https://arxiv.org/abs/2509.25100) | ORPO-Distill |
    • LMOps `/gad` - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.11 | Microsoft Research | [arXiv 2511.10643](https://arxiv.org/abs/2511.10643) · [project](https://ytianzhu.github.io/Generative-Adversarial-Distillation/) | GAD — Black-Box OPD |
    • OVD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2601.21968) | 2026.01 | HKU / Huawei | [arXiv 2601.21968](https://arxiv.org/abs/2601.21968) | OVD (On-policy Verbal Distillation) — project page `OVD.github.io` 404s |
    • SODA - paper-845C40?style=for-the-badge)](https://arxiv.org/pdf/2604.03873) | 2026.04 | Academic | [arXiv 2604.03873](https://arxiv.org/pdf/2604.03873) | SODA — Semi On-Policy Black-Box Distillation |
    • SPoT - AI/SPoT?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.03 | Visual-AI | [arXiv 2603.01683](https://arxiv.org/abs/2603.01683) | **SPOT: Surgical Post-Training** — black-box oracle edits student failures into proximal rollouts |
    • ROPD - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | NUS / USTC / Tencent | [arXiv 2605.07396](https://arxiv.org/abs/2605.07396) | **ROPD — Rubric-based On-Policy Distillation**; induces prompt-specific rubrics from teacher–student contrasts, then scores student rollouts by those rubrics (logit-free / black-box); up to 10× sample efficiency |
    • PRISM - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.04 | HKUST(GZ) / Tsinghua / NTU / RUC / USTC / UCAS | [arXiv 2604.28123](https://arxiv.org/abs/2604.28123) | **PRISM — Pre-alignment via Black-Box OPD for Multimodal RL**; an adversarial OPD stage inserted *between SFT and RLVR*, scoring student rollouts with a **Mixture-of-Experts discriminator** (separate perception and reasoning experts, Bradley–Terry loss). +4.4 / +6.0 avg over SFT→RLVR at 4B / 8B |
    • OmniOPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.01476) | 2026.06 | Meta AI | [arXiv 2606.01476](https://arxiv.org/abs/2606.01476) | **OmniOPD — Logit-Free OPD via Speculative Verification**; an entropy-driven scheduler picks uncertain chunks of the student’s rollout, a black-box teacher generates Monte-Carlo continuations, and they are scored by **semantic similarity** rather than logits. **Beats white-box OPD with the same teacher family (69.08 vs 64.16)** |
    • ExpRL - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Stanford / CMU | [arXiv 2606.17024](https://arxiv.org/abs/2606.17024) | **ExpRL — Exploratory RL for LLM Mid-Training**; an LLM judge scores the student’s own rollouts and prefixes against a *hidden* reference solution under a fixed rubric, giving dense process-level reward before sparse RL ⚠️ see strictness note |
    • ZPPO

  • 🔬 OPD with Larger External Teachers — White-Box

    • LMOps `/minillm` - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2023.06 | Microsoft / Tsinghua | [arXiv 2306.08543](https://arxiv.org/abs/2306.08543) | MiniLLM (ICLR 2024) |
    • distillm - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2024.02 | KAIST / Microsoft | [arXiv 2402.03898](https://arxiv.org/abs/2402.03898) | DistiLLM (ICML 2024) |
    • google-research `/speculative_kd` - research/google-research?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2024.10 | UCSB / Google | [arXiv 2410.11325](https://arxiv.org/abs/2410.11325) | Speculative KD (ICLR 2025) |
    • distillm-2 - 2?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.03 | KAIST / Microsoft | [arXiv 2503.07067](https://arxiv.org/abs/2503.07067) | DistiLLM-2 (ICML 2025 Oral) |
    • DSKDv2 - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2025.04 | BJTU | [arXiv 2504.11426](https://arxiv.org/abs/2504.11426) | DSKDv2 — cross-tokenizer; supports on-policy mode |
    • Constrained OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2509.22921) | 2025.09 | Huawei Noah's Ark | [arXiv 2509.22921](https://arxiv.org/abs/2509.22921) | Constrained OPD (CMDP) |
    • AdaSwitch - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2510.07842) | 2025.10 | RUC / Baidu | [arXiv 2510.07842](https://arxiv.org/abs/2510.07842) | AdaSwitch (on-/off-policy switching) |
    • Veto - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2601.07155) | 2026.01 | SNU | [arXiv 2601.07155](https://arxiv.org/abs/2601.07155) | Veto (Stable OPD) — ACL 2026 Findings |
    • G-OPD - OPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.02 | RUC / Tencent | [arXiv 2602.12125](https://arxiv.org/abs/2602.12125) | G-OPD |
    • Fast OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2602.15260) | 2026.02 | Industrial | [arXiv 2602.15260](https://arxiv.org/abs/2602.15260) | Fast OPD (prefix-truncated) |
    • Entropy-Aware OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2603.07079) | 2026.03 | KAIST / IBM | [arXiv 2603.07079](https://arxiv.org/abs/2603.07079) | Entropy-Aware OPD |
    • REOPOLD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2603.11137) | 2026.03 | KAIST / Microsoft | [arXiv 2603.11137](https://arxiv.org/abs/2603.11137) | REOPOLD (Relaxed OPD) — code soon |
    • OPSD_OnPolicyDistillation - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.03 | LinkedIn | [arXiv 2603.11178](https://arxiv.org/abs/2603.11178) | PACED — frontier curriculum self-distill |
    • TSD-KD - KD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.03 | Korea Univ. | [arXiv 2603.13260](https://arxiv.org/abs/2603.13260) | TSD-KD — token-selective dual KD (ICLR 2026) |
    • SCOPE - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.04 | USTC / Meituan / Fudan | [arXiv 2604.10688](https://arxiv.org/abs/2604.10688) | SCOPE — signal-calibrated dual-path |
    • Hybrid-Policy-Distillation - Policy-Distillation?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.04 | zwhong714 | [arXiv 2604.20244](https://arxiv.org/abs/2604.20244) | HPD — Hybrid Policy Distillation; LlamaFactory + verl backends |
    • trd - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | McGill / Mila / UT Austin (Jiang et al.) | [arXiv 2606.08432](https://arxiv.org/abs/2606.08432) | **TRD — Trajectory-Refined Distillation**; diagnoses *prefix failure* of dense per-token OPD, refines student rollouts at trajectory level before distilling; verl-based, also applies to OPSD |
    • BRTS - keke/BRTS?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.05 | JHU (Patel group) | [arXiv 2605.09725](https://arxiv.org/abs/2605.09725) | **BRTS — Best-of-N Teacher Rollout Selection**; augments student-context OPD with a curated teacher-context branch (correctness-first, then student-alignment) to cut single-rollout teacher variance |
    • FiRe-OPD - OPD?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | THU / HKUST / Meituan (Li et al.) | [arXiv 2606.02684](https://arxiv.org/abs/2606.02684) | **FiRe-OPD — Filter, then Reweight**; decouples *optimization granularity* — **hard** trajectory-level filtering (drop bottom-p% rollouts by teacher log-prob) + **soft** token-level reweighting (teacher-confidence × student-confusion), arguing soft weighting beats hard token selection (cf. TIP); verl-based, with a multi-teacher math+code variant |
    • OPRD - the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | ZJU / Ant Group | [arXiv 2606.06021](https://arxiv.org/abs/2606.06021) | **OPRD — On-Policy Representation Distillation**; first OPD to supervise in *hidden-state space* (aligns teacher/student representations across layers on student rollouts, bypassing the LM head) rather than logits; built on the THUNLP OPD stack |
    • IW-OPD - Weighted-On-Policy-Distillation?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | Xidian / Georgia Tech / Amazon AGI SF Lab | [arXiv 2606.22600](https://arxiv.org/abs/2606.22600) · [project](https://yannx1e.github.io/IW-OPD/) | **IW-OPD — On the Position Bias of OPD**; supervising only the *prefix* 30% of tokens matches full-token OPD while suffix-30% barely learns; reweights the OPD advantage by a prefix-importance term derived from a trust-region argument |
    • DEAR - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.22830) | 2026.06 | Meituan LongCat / Nanjing Univ. / TJUNLP | [arXiv 2606.22830](https://arxiv.org/abs/2606.22830) | **DEAR — Finding the Evidence**; argues entropy-selective OPD (cf. TIP) captures only *decision* tokens and structurally misses low-entropy, high-divergence **evidence** tokens where the student is confident yet wrong |
    • ReNIO - lab/ReNIO?style=for-the-badge&logo=github&logoColor=white&labelColor=181717&color=ffd700" alt="Stars"> | 2026.06 | ECNU / Shanghai Innovation Institute | [arXiv 2606.23104](https://arxiv.org/abs/2606.23104) | **ReNIO — Reweighting Negative Trajectory Importance**; finds training on *incorrect* student outputs beats correct-only under both OPD and OPSD, then approximates correctness with a prefix-computable pivotal-token proxy |
    • PG-OPD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.21994) | 2026.06 | TeleAI / SJTU | [arXiv 2606.21994](https://arxiv.org/abs/2606.21994) | **Prefix-Guided OPD — Mining Golden Trajectories from Rollouts**; probes candidates at a short prefix by teacher–student top-k overlap and only continues the promising ones to full length (up to +4.80 avg, 2.46× wall-clock) |
    • SEAD - paper-845C40?style=for-the-badge)](https://arxiv.org/abs/2606.28562) | 2026.06 | Capital One | [arXiv 2606.28562](https://arxiv.org/abs/2606.28562) | **SEAD — Competence-Aware OPD**; entropy zones the tokens (skip / RKL / FKL), cosine-anneals FKL→RKL, and adds the **first prompt-level curriculum for OPD** |
Programming Languages
Python 97 Jupyter Notebook 3 Shell 2 HTML 1
Categories
♻️ Self-Distillation with Privileged Context — OPSD 70 🔬 OPD with Larger External Teachers — White-Box 56 🤝 OPD-RL Hybrids — Inside-RL OPD 50 🖼️ Multimodal OPD (VLM, Video, Audio, Image) 34 🤖 Agent & Embodied OPD (by application) 33 🏭 Industrial / Production Model Reports 23 📚 Surveys, Foundations & Position Papers 22 ⚡ Speculative-Decoding Distillation 16 🛠️ Frameworks & Toolkits 16 🎭 OPD with Black-Box / Outcome-Based Teachers 10 🌟 Curator's Picks — where to start 9 Star History 2 🧠 Reasoning OPD (by application) 1 Updates 1
Sub Categories
🔁 Iterative Self-Bootstrapping 187
Keywords
llm 8 distillation 5 large-language-models 4 on-policy-distillation 4 rlhf 3 reinforcement-learning 3 instruction-tuning 2 llama 2 pytorch 2 lora 2 agentic-ai 2 moe 2 peft 2 coding 2 sglang 2 rl 2 multimodal 2 fine-tuning 2 gpt 2 language-model 2 grpo 2 opsd 2 internvl 1 open-set-recognition 1 self-supervised 1 spatio-temporal-action-localization 1 temporal-action-localization 1 video-clip 1 embedding 1 video-data 1 video-dataset 1 deepseek-r1 1 self-play 1 deep-learning 1 zero-shot-retrieval 1 zero-shot-classification 1 vision-transformer 1 video-question-answering 1 video-understanding 1 video-retrieval 1 agi 1 lm 1 lmops 1 nlp 1 pretraining 1 prompt 1 promptist 1 x-prompt 1 agent 1 ai 1