arXiv · Software Engineering· Tan Yu, Alexander Bukharin, Khushi Bhardwaj, Jennifer Williams, Zirui Liu, Jonathan Lingjie Li, Soumye Singhal, Joseph Jennings, Sanjeev Satheesh, Yash Jain, Ashish Vaswani, Venkat Krishna Srinivasan, Matthew Papakipos, Hyunwoo Kim, Jian Zhang, Oleksii Kuchaiev, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jonathan Cohen, Jiantao Jiao·· 3 小时前AI 评分45
预测基座模型的后训练编码智能体表现
Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models
AI 导读
研究提出在昂贵的后训练之前预测基座模型潜力的方法，把后训练智能体的成功轨迹当作前瞻信号，并定位让代码库由失败转为通过的决定性步骤。方法包含 Decisive-Action BPB、Patch MCQ 与前缀条件 pass@K 三种筛选指标，在十对公开基座与后训练模型上，其排序与后训练 SWE-bench Verified pass@1 高度一致。
来源：arXiv · Software Engineering · arxiv.org