CIG-SFT：用条件信息增益改进监督微调

1. 12:00
   arXiv · Computation and Language

   SFT 应该学什么？面向可泛化后训练的条件信息增益

   论文提出 CIG-SFT，一个信息论 SFT 框架，用于估计每个示范 token 在任务条件下携带的信息量。该方法在教师强制下比较冻结参考模型在有无任务输入时的 token 似然，把条件信息增益转化为 KL 正则化投影，并推导出等价的标量加权 SFT 目标。

Current heat 9·Comparable peak 10（Oct 12）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.