دورية أكاديمية
Impact Sound Generation for Audiovisual Interaction with Real-World Movable Objects in Building-Scale Virtual Reality
| العنوان: | Impact Sound Generation for Audiovisual Interaction with Real-World Movable Objects in Building-Scale Virtual Reality |
|---|---|
| المؤلفون: | Katashi Nagao, Kaho Kumon, Kodai Hattori |
| المصدر: | Applied Sciences, Vol 11, Iss 16, p 7546 (2021) |
| بيانات النشر: | MDPI AG, 2021. |
| سنة النشر: | 2021 |
| المجموعة: | LCC:Technology LCC:Engineering (General). Civil engineering (General) LCC:Biology (General) LCC:Physics LCC:Chemistry |
| مصطلحات موضوعية: | building-scale VR, impact sound generation, GAN, Technology, Engineering (General). Civil engineering (General), TA1-2040, Biology (General), QH301-705.5, Physics, QC1-999, Chemistry, QD1-999 |
| الوصف: | In building-scale VR, where the entire interior of a large-scale building is a virtual space that users can walk around in, it is very important to handle movable objects that actually exist in the real world and not in the virtual space. We propose a mechanism to dynamically detect such objects (that are not embedded in the virtual space) in advance, and then generate a sound when one is hit with a virtual stick. Moreover, in a large indoor virtual environment, there may be multiple users at the same time, and their presence may be perceived by hearing, as well as by sight, e.g., by hearing sounds such as footsteps. We, therefore, use a GAN deep learning generation system to generate the impact sound from any object. First, in order to visually display a real-world object in virtual space, its 3D data is generated using an RGB-D camera and saved, along with its position information. At the same time, we take the image of the object and break it down into parts, estimate its material, generate the sound, and associate the sound with that part. When a VR user hits the object virtually (e.g., hits it with a virtual stick), a sound is generated. We demonstrate that users can judge the material from the sound, thus confirming the effectiveness of the proposed method. |
| نوع الوثيقة: | article |
| وصف الملف: | electronic resource |
| اللغة: | English |
| تدمد: | 2076-3417 |
| Relation: | https://www.mdpi.com/2076-3417/11/16/7546; https://doaj.org/toc/2076-3417 |
| DOI: | 10.3390/app11167546 |
| URL الوصول: | https://doaj.org/article/71c872941a70402581154c4f9e898730 |
| رقم الأكسشن: | edsdoj.71c872941a70402581154c4f9e898730 |
| قاعدة البيانات: | Directory of Open Access Journals |
Full Text Finder | تدمد: | 20763417 |
|---|---|
| DOI: | 10.3390/app11167546 |