In the multimodal attention laboratory, we will focus on a variety of research questions. The goal is to expand our knowledge about the neural basis of selective attention to auditory, visual, and tactile stimuli. Using realistic experimental paradigms, we will also investigate the electrophysiological correlates of perception, formation, and disintegration of moving objects in free-field.
Background
I have been using behavioral, psychophysical, and electrophysiological (high-density electro-encephalogram, EEG) measures in order to investigate the neural basis of selective attention in ecologically valid free-field settings. These studies include stimuli in three different modalities, audition, vision, and touch. I record “event-related potentials” (ERPs) trying to identify neuronal correlates of selective uni/multimodal attention. ERPs are tiny changes in voltage buried in the EEG and are viewed as highly specific signatures of brain activity related to cognitive operations.
A new field in Cognitive Neuroscience looks at cross-modal integration and interaction phenomena, trying to identify “links” between different stimulus modalities during the time course of information processing in the brain. A typical observation in cross-modal integration studies is that a stimulus is perceived to be more intense if it is accompanied in space and time by a stimulus in another modality. The system responds to bimodal stimuli in a “super-additive” mode, i.e., the response is larger than the sum of the responses to the uni-modal stimuli presented alone. We all know what a cross-modal interaction looks like: While driving in a city, people turn down the volume of their car radios when looking for a street. In this particular case, auditory input appears to recruit processing resources necessary to perform an effective visual search task. These observations will guide our future understanding of how we extract meaningful information in the presence of potential distractors, be that the design of radar screens or human interfaces in general; in essence, any application where a missed signal comes with a high cost.
In order to achieve these goals, sophisticated equipment is of paramount importance. We gather EEG non-invasively with a 168-channel system from BioSemi Inc. with surface electrodes, and digitize the shape of a subject’s head with the 3-D Patriot Polhemus Digitizer. The raw data including behavioral responses will be analyzed on a 64-bit Opteron Sledgehammer series system processor with 8G RAM using the EEGLAB toolbox 4.512 (Delorme & Makeig, 2004), ERPSS (Hillyard laboratory, UCSD), and BESA.
I have designed a 55-channel free-field stimulator to investigate brain responses to audio-visual signals with the help of Dr. Jacob Glower (Electrical Engineering Department, NDSU). Loudspeakers and light-emitting diodes (LEDs) are arranged in an array of five rows and eleven columns in a cylindrical section. Each of these 55 locations has its own microprocessor with memory, and can deliver up to six sounds with 255 different light patterns in 255 intensities via banks of multi-color LEDs. This unique computer-controlled device has millisecond accuracy, and is an essential prerequisite for simulating moving “objects” in free-field. Experiments using a computer monitor equipped with speakers are sub-optimal, because sounds and lights do not emerge from exactly the same spatial location. Furthermore, conventional displays have screen refresh rates, the image is “drawn” as a series of dots starting at the top left corner. This takes time (< 20 ms), poses obvious constraints on the experimental paradigm, and obscures the identification of very early ERP components.
I have designed these computer-controlled tactile stimulators that rely on gravity to return to their default positions. The driving elements are coils with a few hundred turns of magnet wire each. We have a variety of mechanoreceptors with which we detect stimuli delivered to our skin. Heat, shear forces, vibrations, electric shocks, and air puffs produce different tactile sensations involving different sensory pathways. However, a brief mechanical touch is free from artifacts and seems ideal to study very early signatures of tactile processing in the brain. With these stimulators, it is possible to investigate electrophysiological correlates of tactile perception and attention, such as the distribution of spatial attention across the fingers of the hand, for example.
These miniature computer-controlled tactile stimulators have the advantage that they do not rely on gravity to return to their default position. With these devices, I will be able to investigate how the brain maps our extra-personal space, as well as how proprioceptive information is used (with or without vision) to encode the relative coordinates of our extremities in free-field.