行归一化优化器在高维多分类中可优于 Adam 与 Muon

1. 12:00
   arXiv · Statistics Machine Learning

   从几何到泛化：为何行归一化能胜过 Adam 与 Muon

   论文证明在高维多分类中，行归一化可获得严格高于 full-batch Adam、random-reshuffling Adam 代理与 exact-SVD Muon 的总体准确率。优势源于其逐类欧氏几何渐近保留总体决策边界方向，而 Adam 的逐坐标几何与 Muon 的谱几何引入非零畸变；该结论在幂律谱与独立取向的类均值、测试噪声协方差下仍成立，合成实验与最后一层语言模型实验支持预测。

Current heat 9·Comparable peak 10（Oct 9）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.