Technosense and Sensibilia: Haptic Interfaces, Selfhood, and Obstruction in Virtual Reality

Multisensory Experience and the Mistaken Body Problem

The mistaken body problem proceeds from the connection between the tactile modality and the related senses. Numerous scientists have studied the connection especially between the tactile and visual modalities, observing in separate instances that people are unable to locate where they have been touched when provided with distracting visual cues. For example, in one study by Pavani, Spence, and Driver, rubber hands were placed above the hands of the participant, and when the position of the rubber hands most closely resembled (was compatible with) the position of the participant’s actual, hidden hands, the participant was less able to distinguish where they were actually touched from where they saw the rubber hands touched (86-7).
 
Further, “one’s awareness of tactile (as well as visual and auditory) events should always be considered within an interpretational framework that is ‘multisensory’” (89). This extends from a consideration in the text of a debate (as yet unresolved) regarding the flow of sensory information. On the one hand, “the awareness of tactile information might also require (and be influenced by) stages of information processing mediated by higher-order, likely multisensory, areas of the brain, where spatial information is represented” (88). That is, signals from multisensory brain areas such as the posterior parietal cortex “project (down) to the somatosensory areas,” thereby explaining the relationship between visual and tactile information processing (88). On the other hand, different sensory modalities may “enhance the awareness of a signal presented in another sensory modality…within the very unisensory area that is involved in the initial processing of such a signal in the first place (e.g., the somatosensory cortex)” (88).
 
Virtual reality requires the mistaken body—the product of multisensory s(t)imulation—to transfer from problem to solution. If the visual illusion is strong enough, it can overcompensate for what is bereft on the haptic front. This is evident in certain accounts—such as the sensation of vertigo in response to what is an almost exclusively visual virtual reality game or movie—but not yet in the more complex, comprehensive senses that even mild haptic engagement could unlock. 

 

This page has paths: