ObjectiveThis study aimed to evaluate the effects of multiple virtual reality (VR) interaction modalities based on force-haptic feedback combined with visual or auditory feedback in different ways on cerebral cortical activation by functional near-infrared spectroscopy (fNIRS). Methods: A modular multi-sensory VR interaction system based on a planar upper-limb rehabilitation robot was developed. Twenty healthy participants completed active elbow flexion and extension training in four VR interaction patterns, including haptic (H), haptic + auditory (HA), haptic + visual (HV), and haptic + visual + auditory (HVA). Cortical activation changes in the sensorimotor cortex (SMC), premotor cortex (PMC), and prefrontal cortex (PFC) were measured.ResultsFour interaction patterns all had significant activation effects on the motor and cognitive regions of the cerebral cortex (p < 0.05). Among them, in the HVA interaction mode, the cortical activation of each ROI was the strongest, followed by HV, HA, and H. The connectivity between channels of SMC and bilateral PFC, as well as the connectivity between channels in PMC, was the strongest under HVA and HV conditions. Besides, the two-way ANOVA of visual and auditory feedback showed that it was difficult for auditory feedback to have a strong impact on activation without visual feedback. In addition, under the condition of visual feedback, the effect of fusion auditory feedback on the activation degree was significantly higher than that of no auditory feedback.ConclusionsThe interaction mode of visual, auditory, and haptic multi-sensory integration is conducive to stronger cortical activation and cognitive control. Besides, there is an interaction effect between visual and auditory feedback, thus improving the cortical activation level. This research enriches the research on activation and connectivity of cognitive and motor cortex in the process of modular multi-sensory interaction training of rehabilitation robots. These conclusions provide a theoretical basis for the optimal design of the interaction mode of the rehabilitation robot and the possible scheme of clinical VR rehabilitation.
Full Text Finder