Tiny-Scale中文BERT预训练策略对比研究

1. 12:00
   arXiv · Computation and Language

   Tiny-Scale Chinese BERT 预训练：MLM、WWM 与 MacBERT 策略的对照比较

   该研究在 4 层、256 维、8.7M 参数的 tiny-scale Chinese BERT 上，用同一语料（1.29M 句中文 Wikipedia）和超参数对照 MLM、WWM 与 MacBERT 三种预训练策略。

Current heat 9·Comparable peak 10（Oct 8）·Comparable change over 24 hours –

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.