دورية أكاديمية
Image classification and reconstruction from low-density EEG.
| العنوان: | Image classification and reconstruction from low-density EEG. |
|---|---|
| المؤلفون: | Guenther S; School of Computation, Information and Technology, Technical University of Munich, Munich, Germany. sven.guenther@tum.de., Kosmyna N; Media Lab, Massachusetts Institute of Technology, Cambridge, USA., Maes P; Media Lab, Massachusetts Institute of Technology, Cambridge, USA. |
| المصدر: | Scientific reports [Sci Rep] 2024 Jul 16; Vol. 14 (1), pp. 16436. Date of Electronic Publication: 2024 Jul 16. |
| نوع المنشور: | Journal Article |
| اللغة: | English |
| بيانات الدورية: | Publisher: Nature Publishing Group Country of Publication: England NLM ID: 101563288 Publication Model: Electronic Cited Medium: Internet ISSN: 2045-2322 (Electronic) Linking ISSN: 20452322 NLM ISO Abbreviation: Sci Rep Subsets: MEDLINE |
| أسماء مطبوعة: | Original Publication: London : Nature Publishing Group, copyright 2011- |
| مواضيع طبية MeSH: | Electroencephalography*/methods , Brain*/physiology , Brain*/diagnostic imaging , Image Processing, Computer-Assisted*/methods, Humans ; Adult ; Female ; Male ; Brain Mapping/methods ; Young Adult ; Photic Stimulation ; Magnetic Resonance Imaging/methods ; Algorithms |
| مستخلص: | Recent advances in visual decoding have enabled the classification and reconstruction of perceived images from the brain. However, previous approaches have predominantly relied on stationary, costly equipment like fMRI or high-density EEG, limiting the real-world availability and applicability of such projects. Additionally, several EEG-based paradigms have utilized artifactual, rather than stimulus-related information yielding flawed classification and reconstruction results. Our goal was to reduce the cost of the decoding paradigm, while increasing its flexibility. Therefore, we investigated whether the classification of an image category and the reconstruction of the image itself is possible from the visually evoked brain activity measured by a portable, 8-channel EEG. To compensate for the low electrode count and to avoid flawed predictions, we designed a theory-guided EEG setup and created a new experiment to obtain a dataset from 9 subjects. We compared five contemporary classification models with our setup reaching an average accuracy of 34.4% for 20 image classes on hold-out test recordings. For the reconstruction, the top-performing model was used as an EEG-encoder which was combined with a pretrained latent diffusion model via double-conditioning. After fine-tuning, we reconstructed images from the test set with a 1000 trial 50-class top-1 accuracy of 35.3%. While not reaching the same performance as MRI-based paradigms on unseen stimuli, our approach greatly improved the affordability and mobility of the visual decoding technology. (© 2024. The Author(s).) |
| References: | Behav Res Methods. 2019 Feb;51(1):195-203. (PMID: 30734206) Biomed Phys Eng Express. 2019 Aug 02;5(5):. (PMID: 32983573) Commun Biol. 2022 Nov 14;5(1):1247. (PMID: 36376446) Neuron. 2008 Dec 26;60(6):1126-41. (PMID: 19109916) J Neurosci Methods. 2021 Mar 15;352:109080. (PMID: 33508412) Front Neurosci. 2023 Feb 13;17:1122661. (PMID: 36860620) J Neurosci. 2020 Aug 26;40(35):6779-6789. (PMID: 32703903) Electroencephalogr Clin Neurophysiol Suppl. 1999;52:3-6. (PMID: 10590970) Twin Res Hum Genet. 2012 Jun;15(3):384-92. (PMID: 22856372) IEEE Trans Pattern Anal Mach Intell. 2021 Nov;43(11):3833-3849. (PMID: 32750768) PLoS One. 2015 Aug 21;10(8):e0135697. (PMID: 26295970) Front Neuroinform. 2015 Jun 18;9:16. (PMID: 26150785) Curr Opin Neurobiol. 2019 Oct;58:199-208. (PMID: 31586749) eNeuro. 2018 Jan 27;5(1):. (PMID: 29379880) PLoS One. 2022 Sep 21;17(9):e0274847. (PMID: 36129927) Front Hum Neurosci. 2020 Sep 18;14:365. (PMID: 33061900) Sci Data. 2022 Jan 10;9(1):3. (PMID: 35013331) J Vis. 2013 Aug 01;13(10):. (PMID: 23908380) IEEE Trans Neural Syst Rehabil Eng. 2022 Dec 16;PP:. (PMID: 37015413) J Neural Eng. 2018 Oct;15(5):056013. (PMID: 29932424) Front Psychol. 2010 Jul 08;1:28. (PMID: 21833198) Neuron. 2012 Apr 12;74(1):12-29. (PMID: 22500626) Hum Brain Mapp. 2017 Nov;38(11):5391-5420. (PMID: 28782865) Trends Cogn Sci. 2002 Apr 1;6(4):176-184. (PMID: 11912041) PLoS One. 2010 Dec 30;5(12):e14465. (PMID: 21209937) Proc Natl Acad Sci U S A. 2014 Mar 25;111(12):4590-5. (PMID: 24591602) Neuropsychologia. 2017 Oct;105:165-176. (PMID: 28215698) IEEE Trans Pattern Anal Mach Intell. 2020 Nov 19;PP:. (PMID: 33211652) Psychophysiology. 2023 Sep;60(9):e14320. (PMID: 37171024) |
| تواريخ الأحداث: | Date Created: 20240716 Date Completed: 20240716 Latest Revision: 20240801 |
| رمز التحديث: | 20240801 |
| مُعرف محوري في PubMed: | PMC11252274 |
| DOI: | 10.1038/s41598-024-66228-1 |
| PMID: | 39013929 |
| قاعدة البيانات: | MEDLINE |
Find this article in full text from Springer Verlag. 
Full Text Finder | تدمد: | 2045-2322 |
|---|---|
| DOI: | 10.1038/s41598-024-66228-1 |