CFM框架用于视觉引导声学高亮

1. 12:00
   arXiv · Audio and Speech

   用于视觉引导音频高亮的条件流匹配方法

   研究者提出 Conditional Flow Matching（CFM）框架，把视觉引导音频高亮从判别式建模重构为生成式建模，以解决音频重混中不存在一一对应映射的歧义问题。方法引入 rollout loss 惩罚末步漂移，并采用在向量场回归前融合音视频线索的条件模块，实现显式跨模态声源选择。定量与定性评估显示其持续超越此前 SOTA 判别式方法。

Current heat 5·Comparable peak 10（Oct 9）·Comparable change over 24 hours -50%

The trend compares only the same participants observed continuously; its range may be smaller than the current heat count. Move or click on the chart to inspect hourly heat; use the left and right arrow keys to switch.