Multisensory Experience and the Mistaken Body Problem
Further, “one’s awareness of tactile (as well as visual and auditory) events should always be considered within an interpretational framework that is ‘multisensory’” (89). This extends from a consideration in the text of a debate (as yet unresolved) regarding the flow of sensory information. On the one hand, “the awareness of tactile information might also require (and be influenced by) stages of information processing mediated by higher-order, likely multisensory, areas of the brain, where spatial information is represented” (88). That is, signals from multisensory brain areas such as the posterior parietal cortex “project (down) to the somatosensory areas,” thereby explaining the relationship between visual and tactile information processing (88). On the other hand, different sensory modalities may “enhance the awareness of a signal presented in another sensory modality…within the very unisensory area that is involved in the initial processing of such a signal in the first place (e.g., the somatosensory cortex)” (88).
Virtual reality requires the mistaken body—the product of multisensory s(t)imulation—to transfer from problem to solution. If the visual illusion is strong enough, it can overcompensate for what is bereft on the haptic front. This is evident in certain accounts—such as the sensation of vertigo in response to what is an almost exclusively visual virtual reality game or movie—but not yet in the more complex, comprehensive senses that even mild haptic engagement could unlock.