تقرير
SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models
| العنوان: | SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models |
|---|---|
| المؤلفون: | Tang, Yuxun, Wu, Yuning, Shi, Jiatong, Jin, Qin |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing |
| الوصف: | Discrete representation has shown advantages in speech generation tasks, wherein discrete tokens are derived by discretizing hidden features from self-supervised learning (SSL) pre-trained models. However, the direct application of speech SSL models to singing generation encounters domain gaps between speech and singing. Furthermore, singing generation necessitates a more refined representation than typical speech. To address these challenges, we introduce SingOMD, a novel method to extract singing-oriented multi-resolution discrete representations from speech SSL models. Specifically, we first adapt the features from speech SSL through a resynthesis task and incorporate multi-resolution modules based on resampling to better serve singing generation. These adapted multi-resolution features are then discretized via clustering. Extensive experiments demonstrate the robustness, efficiency, and effectiveness of these representations in singing vocoders and singing voice synthesis. Comment: Accepted by Interspeech 2024 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2406.08905 |
| رقم الأكسشن: | edsarx.2406.08905 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |