Spectrally Targeted Muon 优化器论文

1. 12:00
   arXiv · Machine Learning Theory

   Spectrally Targeted Muon 优化器：只对部分奇异值正交化

   研究者提出 Spectrally Targeted Muon 优化器，仅对高于或低于阈值 τ 的奇异值做正交化，通过在平移 Gram 矩阵上运行 Newton-Schulz 迭代隔离相关奇异子空间，无需 SVD，并可在 normalized SGD 与 Muon 之间插值。

Current heat 9·Comparable peak 10（Oct 9）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.