arXiv · Statistics Machine Learning· Richard Bergna (University of Cambridge, Spotify), Fernando Ruiz Mazo (University of Cambridge), Nicol\`o Felicioni (Spotify), Jos\'e Miguel Hern\'andez-Lobato (University of Cambridge), Kamil Ciosek (Spotify)·· 3 小时前AI 评分37
σTransfer：在 μP 下将不确定性从小网络迁移到大网络
$\sigma$Transfer: Uncertainty Transfer from Small to Large Networks under $\mu\mathrm{P}$
AI 导读
论文提出 σTransfer，在 Maximal Update Parametrization（μP）下推导出先验协方差的缩放规则，使选定先验精度随模型宽度增大保持稳定，从而可在小模型上选定精度并零样本迁移到大模型，无需在大模型上重新搜索。
来源：arXiv · Statistics Machine Learning · arxiv.org