LLM推理蒸馏最弱链约束强化学习

1. 12:00
   arXiv · Machine Learning Theory

   蒸馏 LLM 推理能力的最弱环节方法：基于最坏情况约束强化学习的推理蒸馏

   论文提出将 LLM 推理蒸馏建模为约束强化学习问题，以轨迹前缀上教师似然的最坏情况为硬约束，避免纯 RL 的奖励欺骗与 KL 蒸馏的逻辑稀释。该方法在数学推理和代码生成任务上扩展了准确率-保真度帕累托前沿，以与纯 RL 相当的最终答案正确率大幅降低教师约束违反，实现最高的严谨推理成功率。

Current heat 9·Comparable peak 10（Oct 2）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.