تقرير
Towards Improving NAM-to-Speech Synthesis Intelligibility using Self-Supervised Speech Models
| العنوان: | Towards Improving NAM-to-Speech Synthesis Intelligibility using Self-Supervised Speech Models |
|---|---|
| المؤلفون: | Shah, Neil, Karande, Shirish, Gandhi, Vineet |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Sound, Computer Science - Artificial Intelligence, Electrical Engineering and Systems Science - Audio and Speech Processing |
| الوصف: | We propose a novel approach to significantly improve the intelligibility in the Non-Audible Murmur (NAM)-to-speech conversion task, leveraging self-supervision and sequence-to-sequence (Seq2Seq) learning techniques. Unlike conventional methods that explicitly record ground-truth speech, our methodology relies on self-supervision and speech-to-speech synthesis to simulate ground-truth speech. Despite utilizing simulated speech, our method surpasses the current state-of-the-art (SOTA) by 29.08% improvement in the Mel-Cepstral Distortion (MCD) metric. Additionally, we present error rates and demonstrate our model's proficiency to synthesize speech in novel voices of interest. Moreover, we present a methodology for augmenting the existing CSTR NAM TIMIT Plus corpus, setting a benchmark with a Word Error Rate (WER) of 42.57% to gauge the intelligibility of the synthesized speech. Speech samples can be found at https://nam2speech.github.io/NAM2Speech/ Comment: Accepted at Interspeech 2024 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2407.18541 |
| رقم الأكسشن: | edsarx.2407.18541 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |