Combined perception of emotion in pictures and musical sounds

Katja N. Spreckelmeyer, Marta Kutas, Thomas P. Urbach, Eckart Altenmüller, Thomas F. Münte*

*Korrespondierende/r Autor/-in für diese Arbeit
50 Zitate (Scopus)


Evaluation of emotional scenes requires integration of information from different modality channels, most frequently from audition and vision. Neither the psychological nor neural basis of auditory-visual interactions during the processing of affect is well understood. In this study, possible interactions in affective processing were investigated via event-related potential (ERP) recordings during simultaneous presentation of affective pictures (from IAPS) and affectively sung notes that either matched or mismatched each other in valence. To examine the role of attention in multisensory affect-integration ERPs were recorded in two different rating tasks (voice affect rating, picture affect rating) as participants evaluated the affect communicated in one of the modalities, while that in the other modality was ignored. Both the behavioral and ERP data revealed some, although non-identical, patterns of cross-modal influences; modulation of the ERP-component P2 suggested a relatively early integration of affective information in the attended picture condition, though only for happy picture-voice pairs. In addition, congruent pairing of sad pictures and sad voice stimuli affected the late positive potential (LPP). Responses in the voice affect rating task were overall more likely to be modulated by the concomitant picture's affective valence than vice versa.

ZeitschriftBrain Research
Seiten (von - bis)160-170
PublikationsstatusVeröffentlicht - 27.01.2006

Strategische Forschungsbereiche und Zentren

  • Forschungsschwerpunkt: Gehirn, Hormone, Verhalten - Center for Brain, Behavior and Metabolism (CBBM)


Untersuchen Sie die Forschungsthemen von „Combined perception of emotion in pictures and musical sounds“. Zusammen bilden sie einen einzigartigen Fingerprint.