注意力头消融因果推断的混淆因素与地板效应

1. 12:00
   arXiv · Machine Learning Theory

   注意力头消融何时支持因果推断：投影层混淆、地板效应与匹配对照

   论文以 GPT-2 small 为例，发现后投影置零与前投影消融相关性极低（Pearson r=0.057）且 top-5 头集完全不交；二值准确率在行为地板和天花板处会掩盖效应，gold-token log-probability 更稳定。

Current heat 9·Comparable peak 10（Oct 2）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.