A novel approach to study audiovisual integration in speech perception: Localizer fMRI and sparse sampling

Gregor Rafael Szycik, Peggy Tausche, Thomas F. Münte*

*Corresponding author for this work
30 Citations (Scopus)


Speech comprehension in noisy environments is greatly improved by the availability of visual information, i.e. lip and facial movements. This suggests that the brain is in possession of an audiovisual integration mechanism supplementing speech comprehension. To reveal the respective brain areas, we used a step-wise functional neuroimaging approach in healthy German speaking subjects. First, a functional localizer session using a block design with auditory speech, visual speech (lip movements, no audio), and audiovisual speech (audio and video-signal in sync) was used to identify areas activated by auditory and visual speech, which showed in addition further increased activity in the audiovisual condition. This procedure revealed two clusters of brain activity in the posterior part of the superior temporal sulcus bilaterally. In a second session using slow-event-related imaging in conjunction with sparse sampling, these functionally defined volumes were further examined in a design crossing audiovisual congruity and intelligibility (with/without added noise). Within these areas, regions showing an interaction of audiovisual congruity and intelligibility were found with the greatest activity for incongruent speech stimuli with added noise. This underscores their role in audiovisual integration. We propose that the stepwise approach introduced here allows a finer analysis of audiovisual speech integration than previous methods.

Original languageEnglish
JournalBrain Research
Pages (from-to)142-149
Number of pages8
Publication statusPublished - 18.07.2008

Research Areas and Centers

  • Academic Focus: Center for Brain, Behavior and Metabolism (CBBM)


Dive into the research topics of 'A novel approach to study audiovisual integration in speech perception: Localizer fMRI and sparse sampling'. Together they form a unique fingerprint.

Cite this