Skip to content
Hot eventLive

Adam与自然梯度下降的几何偏差研究

1 reports1 sources7 hr ago updated

Get the story

AI overview

2026-10-02，arXiv Neural and Evolutionary Computing 发表研究，将 Adam 完整更新规则视为对角经验 Fisher 近似，在四种损失曲面上测量其与自然梯度下降的几何偏差。结果显示：在良态设定下偏差较低，在 ill-conditioned 和小型神经网络中偏差可达约 10³，但 Adam 仍能收敛到低损失。

Generated from reports · updated 6 hr ago

LatestOct 2
研究量化 Adam 与自然梯度下降的几何偏差，ill-conditioned 与小型网络中偏差约 10³，但 Adam 仍收敛到低损失。

Timeline

Follow the coverage from different angles.

Oct 2, 2026
  1. arXiv · Neural and Evolutionary Computing
    Adam 优化器距离自然梯度下降有多远？

    summary_zh: 研究将 Adam 完整更新规则视为对角经验 Fisher 近似，在四种损失曲面上测量其与自然梯度下降的几何偏差。在良态设定下偏差较低， ill-conditioned 和小型神经网络中偏差可达约 10³，但 Adam 仍能收敛到低损失。

Heat trend

Current heat 8·Comparable peak 10（Oct 2）·Comparable change over 24 hours –

02.557.510Oct2Oct2Oct2Oct2

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.