Larg-scale multimodal fMRI dataset of visual and haptic 3D object category learning.

How do humans recognize objects in their environment using either vision or touch, even when only one sense is available? Humans routinely integrate visual and haptic sensory systems for object recognition, yet the mechanisms behind cross-modal transfer of category information are not fully understood. In this study, we collected behavioral and fMRI data from 134 participants. Fifty participants (25 per group; 13 females each) performed a similarity rating task with novel 3D objects ("digital embryos") using either visual or haptic modalities. The remaining 84 participants completed a 3D object category learning experiment in unimodal (Visual-Visual: n = 19, 11 females; Haptic-Haptic: n = 22, 10 females) or cross-modal (Visual-Haptic: n = 22, 11 females; Haptic-Visual: n = 21, 10 females) conditions. Alongside task-based fMRI, we acquired resting-state fMRI data at four time points: pre-training, post-training, pre-test, and post-test. High-resolution T1-weighted images were also obtained for each MRI session. These data support investigations into visual and haptic representations of the environment, representational similarity across modalities, cross-modal transfer of category information for object recognition, and neural plasticity in brain networks.