更精简Transformer更易学会聚类

1. 12:00
   arXiv · Statistics Machine Learning

   更精简的 Transformer 更易学会聚类

   论文提出一种表达力相同但更小的 Transformer，以嵌入维度 d_emb = (d + ⌈log₂ k⌉) 执行 Lloyd 算法的 k-means 聚类，低于此前的 d_emb = d+k。研究在聚类任务分布上训练这些 Transformer，并从理论与实验两方面刻画影响随机梯度学习算法收敛与分布内泛化的因素。论文还探测所学算法的通用聚类能力，分析其成功与失败的情形。

Current heat 9·Comparable peak 10（Oct 8）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.