arXiv · Machine Learning Theory· Juli Huang·2026-10-02 12:00· 5 小时前AI 评分57

注意力头消融何时支持因果推断：投影层混淆、地板效应与匹配对照

When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls

AI 导读

论文以 GPT-2 small 为例，发现后投影置零与前投影消融相关性极低（Pearson r=0.057）且 top-5 头集完全不交；二值准确率在行为地板和天花板处会掩盖效应，gold-token log-probability 更稳定。