论文称最佳优化器取决于批大小

1. 12:00
   arXiv · Machine Learning Theory

   最佳优化器取决于批大小

   一项研究挑战了“最优优化器不随批大小改变”的假设，指出不存在能在所有训练设置下稳定生效的 Muon 缩放规则，且语言模型预训练的最佳优化器即使经过充分超参数调优仍会随批大小变化。该结论来自对自适应优化器仅在单一 batch size 下评测做法的反思。论文已提交至 arXiv（cs.LG）。

Current heat 9·Comparable peak 10（Oct 8）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.