View Item 

Show simple item record

Audiovisual interactions in speeded discrimination of a visual event
dc.contributor.authorKaya, Utku
dc.contributor.authorKafaligonul, Hulusi
dc.date.accessioned2021-04-06T02:12:15Z
dc.date.available2022-05-05 22:12:14en
dc.date.available2021-04-06T02:12:15Z
dc.date.issued2021-04
dc.identifier.citationKaya, Utku; Kafaligonul, Hulusi (2021). "Audiovisual interactions in speeded discrimination of a visual event." Psychophysiology (4): n/a-n/a.
dc.identifier.issn0048-5772
dc.identifier.issn1469-8986
dc.identifier.urihttps://hdl.handle.net/2027.42/167075
dc.description.abstractThe integration of information from different senses is central to our perception of the external world. Audiovisual interactions have been particularly well studied in this context and various illusions have been developed to demonstrate strong influences of these interactions on the final percept. Using audiovisual paradigms, previous studies have shown that even task‐irrelevant information provided by a secondary modality can change the detection and discrimination of a primary target. These modulations have been found to be significantly dependent on the relative timing between auditory and visual stimuli. Although these interactions in time have been commonly reported, we have still limited understanding of the relationship between the modulations of event‐related potentials (ERPs) and final behavioral performance. Here, we aimed to shed light on this important issue by using a speeded discrimination paradigm combined with electroencephalogram (EEG). During the experimental sessions, the timing between an auditory click and a visual flash was varied over a wide range of stimulus onset asynchronies and observers were engaged in speeded discrimination of flash location. Behavioral reaction times were significantly changed by click timing. Furthermore, the modulations of evoked activities over medial parietal/parieto‐occipital electrodes were associated with this effect. These modulations were within the 126–176 ms time range and more importantly, they were also correlated with the changes in reaction times. These results provide an important functional link between audiovisual interactions at early stages of sensory processing and reaction times. Together with previous research, they further suggest that early crossmodal interactions play a critical role in perceptual performance.Crossmodal interactions in the temporal domain are a crucial aspect of multisensory integration. Using onset timing (SOA) as a critical factor, we found audiovisual interactions in early ERPs correlated with the changes in behavioral reaction times. These findings provide an important functional link between audiovisual interactions at early stages of sensory processing and perceptual performance.
dc.publisherCRC Press
dc.publisherWiley Periodicals, Inc.
dc.subject.othermultisensory
dc.subject.otherreaction time
dc.subject.othervisual timing
dc.subject.otherEEG
dc.subject.otheraudiovisual interactions
dc.titleAudiovisual interactions in speeded discrimination of a visual event
dc.typeArticle
dc.rights.robotsIndexNoFollow
dc.subject.hlbsecondlevelPhysiology
dc.subject.hlbsecondlevelPsychology
dc.subject.hlbtoplevelHealth Sciences
dc.description.peerreviewedPeer Reviewed
dc.description.bitstreamurlhttp://deepblue.lib.umich.edu/bitstream/2027.42/167075/1/psyp13777_am.pdf
dc.description.bitstreamurlhttp://deepblue.lib.umich.edu/bitstream/2027.42/167075/2/psyp13777.pdf
dc.identifier.doi10.1111/psyp.13777
dc.identifier.sourcePsychophysiology
dc.identifier.citedreferenceSpence, C., & Squire, S. B. ( 2003 ). Multisensory integration: Maintaining the perception of synchrony. Current Biology, 13, R519 – R521. https://doi.org/10.1016/s0960‐9822(03)00445‐7
dc.identifier.citedreferenceOostenveld, R., Fries, P., Maris, E., & Schoffelen, J. M. ( 2011 ). FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience, 2011, 156869. https://doi.org/10.1155/2011/156869
dc.identifier.citedreferencePelli, D. ( 1997 ). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437 – 442. https://doi.org/10.1163/156856897X00366
dc.identifier.citedreferencePerrin, F., Pernier, J., Bertrand, O., & Echallier, J. F. ( 1989 ). Spherical splines for scalp potential and current density mapping. Electroencephalography and Clinical Neurophysiology, 72, 184 – 187. https://doi.org/10.1016/0013‐4694(89)90180‐6
dc.identifier.citedreferenceRaij, T., Ahveninen, J., Lin, F.‐H., Witzel, T., Jääskeläinen, I. P., Letham, B., Israeli, E., Sahyoun, C., Vasios, C., Stufflebeam, S., Hämäläinen, M., & Belliveau, J. W. ( 2010 ). Onset timing of cross‐sensory activations and multisensory interactions in auditory and visual sensory cortices. European Journal of Neuroscience, 31, 1772 – 1782. https://doi.org/10.1111/j.1460‐9568.2010.07213.x
dc.identifier.citedreferenceRammsayer, T. H., Borter, N., & Troche, S. J. ( 2015 ). Visual‐auditory differences in duration discrimination of intervals in the subsecond and second range. Frontiers in Psychology, 6, 1626. https://doi.org/10.3389/fpsyg.2015.01626
dc.identifier.citedreferenceRibeiro, M. J., & Castelo‐Branco, M. ( 2019 ). Neural correlates of anticipatory cardiac deceleration and its association with the speed of perceptual decision‐making, in young and older adults. NeuroImage, 199, 521 – 533. https://doi.org/10.1016/j.neuroimage.2019.06.004
dc.identifier.citedreferenceSenkowski, D., Talsma, D., Grigutsch, M., Herrmann, C. S., & Woldorff, M. G. ( 2007 ). Good times for multisensory integration: Effects of the precision of temporal synchrony as revealed by gamma‐band oscillations. Neuropsychologia, 45, 561 – 571. https://doi.org/10.1016/j.neuropsychologia.2006.01.013
dc.identifier.citedreferenceSpence, C. ( 2018 ). Multisensory perception. In I. J. Wixted (Ed.), Stevens’ handbook of experimental psychology and cognitive neuroscience (Vol. 2, pp. 1 – 56 ). John Wiley and Sons. https://doi.org/10.1002/9781119170174.epcn214
dc.identifier.citedreferenceStefanics, G., Hangya, B., Hernadi, I., Winkler, I., Lakatos, P., & Ulbert, I. ( 2010 ). Phase entrainment of human delta oscillations can mediate the effects of expectation on reaction speed. Journal of Neuroscience, 30, 13578 – 13585. https://doi.org/10.1523/jneurosci.0703‐10.2010
dc.identifier.citedreferenceStekelenburg, J. J., & Vroomen, J. ( 2005 ). An event‐related potential investigation of the time‐course of temporal ventriloquism. NeuroReport, 16, 641 – 644. https://doi.org/10.1097/00001756‐200504250‐00025
dc.identifier.citedreferenceStevenson, R. A., Ghose, D., Fister, J. K., Sarko, D. K., Altieri, N. A., Nidiffer, A. R., Kurela, L. A. R., Siemann, J. K., James, T. W., & Wallace, M. T. ( 2014 ). Identifying and quantifying multisensory integration: A tutorial review. Brain Topography, 27, 707 – 730. https://doi.org/10.1007/s10548‐014‐0365‐7
dc.identifier.citedreferenceTalsma, D., Senkowski, D., Soto‐Faraco, S., & Woldorff, M. G. ( 2010 ). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14, 400 – 410. https://doi.org/10.1016/j.tics.2010.06.008
dc.identifier.citedreferenceTalsma, D., Senkowski, D., & Woldorff, M. G. ( 2009 ). Intermodal attention affects the processing of the temporal alignment of audiovisual stimuli. Experimental Brain Research, 198, 313 – 328. https://doi.org/10.1007/s00221‐009‐1858‐6
dc.identifier.citedreferenceTeder‐Sälejärvi, W. A., Di Russo, F., McDonald, J. J., & Hillyard, S. A. ( 2005 ). Effects of spatial congruity on audio‐visual multimodal integration. Journal of Cognitive Neuroscience, 17 ( 9 ), 1396 – 1409. https://doi.org/10.1162/0898929054985383
dc.identifier.citedreferenceTeder‐Sälejärvi, W. A., McDonald, J. J., Di Russo, F., & Hillyard, S. A. ( 2002 ). An analysis of audio‐visual crossmodal integration by means of event‐related potential (ERP) recordings. Cognitive Brain Research, 14, 106 – 114. https://doi.org/10.1016/s0926‐6410(02)00065‐4
dc.identifier.citedreferenceThorne, J. D., De Vos, M., Viola, F. C., & Debener, S. ( 2011 ). Cross‐modal phase reset predicts auditory task performance in humans. Journal of Neuroscience, 31, 3853 – 3861. https://doi.org/10.1523/jneurosci.6176‐10.2011
dc.identifier.citedreferenceThorne, J. D., & Debener, S. ( 2014 ). Look now and hear what’s coming: On the functional role of cross‐modal phase reset. Hearing Research, 307, 144 – 152. https://doi.org/10.1016/j.heares.2013.07.002
dc.identifier.citedreferencevan Atteveldt, N., Murray, M. M., Thut, G., & Schroeder, C. E. ( 2014 ). Multisensory integration: Flexible use of general operations. Neuron, 81, 1240 – 1253. https://doi.org/10.1016/j.neuron.2014.02.044
dc.identifier.citedreferencevan den Brink, R. L., Cohen, M. X., van der Burg, E., Talsma, D., Vissers, M. E., & Slagter, H. A. ( 2014 ). Subcortical, modality‐specific pathways contribute to multisensory processing in humans. Cerebral Cortex, 24 ( 8 ), 2169 – 2177. https://doi.org/10.1093/cercor/bht069
dc.identifier.citedreferenceVroomen, J., & Keetels, M. ( 2010 ). Perception of intersensory synchrony: A tutorial review. Attention Perception and Psychophysics, 72, 871 – 884. https://doi.org/10.3758/app.72.4.871
dc.identifier.citedreferenceWorld Medical Association. ( 2013 ). Declaration of Helsinki: Ethical principles for medical research involving human subjects. Journal of the American Medical Association, 310 ( 20 ), 2191 – 2194. https://doi.org/10.1001/jama.2013.281053
dc.identifier.citedreferenceZhou, H. Y., Cheung, E. F. C., & Chan, R. C. K. ( 2020 ). Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions. Neuropsychologia, 140, 107396. https://doi.org/10.1016/j.neuropsychologia.2020.107396
dc.identifier.citedreferenceFreeman, E., & Driver, J. ( 2008 ). Direction of visual apparent motion driven solely by timing of a static sound. Current Biology, 18, 1262 – 1266. https://doi.org/10.1016/j.cub.2008.07.066
dc.identifier.citedreferenceGetzmann, S. ( 2007 ). The effect of brief auditory stimuli on visual apparent motion. Perception, 36, 1089 – 1103. https://doi.org/10.1068/p5741
dc.identifier.citedreferenceAllen, P. J., Polizzi, G., Krakow, K., Fish, D. R., & Lemieux, L. ( 1998 ). Identification of EEG events in the MR scanner: The problem of pulse artifact and a method for its subtraction. NeuroImage, 8, 229 – 239. https://doi.org/10.1006/nimg.1998.0361
dc.identifier.citedreferenceBenjamini, Y., & Hochberg, Y. ( 1995 ). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society B, 57, 289 – 300. https://doi.org/10.1111/j.2517‐6161.1995.tb02031.x
dc.identifier.citedreferenceBenjamini, Y., & Yekutieli, D. ( 2001 ). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29, 1165 – 1188. https://doi.org/10.1214/aos/1013699998
dc.identifier.citedreferenceBesle, J., Fort, A., & Giard, M. ( 2004 ). Interest and validity of the additive model in electrophysiological studies of multisensory interactions. Cognitive Processing, 5, 189 – 192. https://doi.org/10.1007/s10339‐004‐0026‐y
dc.identifier.citedreferenceBrainard, D. ( 1997 ). The psychophysics toolbox. Spatial Vision, 10, 433 – 436. https://doi.org/10.1163/156856897X00357
dc.identifier.citedreferenceBurr, D., Banks, M., & Morrone, M. ( 2009 ). Auditory dominance over vision in the perception of interval duration. Experimental Brain Research, 198, 49 – 57. https://doi.org/10.1007/s00221‐009‐1933‐z
dc.identifier.citedreferenceCappe, C., & Barone, P. ( 2005 ). Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. European Journal of Neuroscience, 22, 2886 – 2902. https://doi.org/10.1111/j.1460‐9568.2005.04462.x
dc.identifier.citedreferenceCappe, C., Thut, G., Romei, V., & Murray, M. M. ( 2010 ). Auditory‐visual multisensory interactions in humans: Timing, topography, directionality, and sources. Journal of Neuroscience, 30, 12572 – 12580. https://doi.org/10.1523/jneurosci.1099‐10.2010
dc.identifier.citedreferenceCardoso‐Leite, P., Gorea, A., & Mamassian, P. ( 2007 ). Temporal order judgment and simple reaction times: Evidence for a common processing system. Journal of Vision, 7 ( 6 ), 11. https://doi.org/10.1167/7.6.11
dc.identifier.citedreferenceCecere, R., Gross, J., Willis, A., & Thut, G. ( 2017 ). Being first matters: Topographical representational similarity analysis of ERP signals reveals separate networks for audiovisual temporal binding depending on the leading sense. Journal of Neuroscience, 37, 5274 – 5287. https://doi.org/10.1523/jneurosci.2926‐16.2017
dc.identifier.citedreferenceChen, L., & Vroomen, J. ( 2013 ). Intersensory binding across space and time: A tutorial review. Attention Perception and Psychophysics, 75, 790 – 811. https://doi.org/10.3758/s13414‐013‐0475‐4
dc.identifier.citedreferenceClavagnier, S., Falchier, A., & Kennedy, H. ( 2004 ). Long‐distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective and Behavioral Neuroscience, 4, 117 – 126. https://doi.org/10.3758/cabn.4.2.117
dc.identifier.citedreferenceColosio, M., Shestakova, A., Nikulin, V. V., Blagovechtchenski, E., & Klucharev, V. ( 2017 ). Neural mechanisms of cognitive dissonance (revised): An EEG study. Journal of Neuroscience, 37 ( 20 ), 5074 – 5083. https://doi.org/10.1523/jneurosci.3209‐16.2017
dc.identifier.citedreferenceDiederich, A., Schomburg, A., & Colonius, H. ( 2012 ). Saccadic reaction times to audiovisual stimuli show effects of oscillatory phase reset. PLoS One, 7 ( 10 ), e44910. https://doi.org/10.1371/journal.pone.0044910
dc.identifier.citedreferenceDonchin, E., & Lindsley, D. B. ( 1966 ). Average evoked potentials and reaction times to visual stimuli. Electroencephalography and Clinical Neurophysiology, 20, 217 – 223. https://doi.org/10.1016/0013‐4694(66)90086‐1
dc.identifier.citedreferenceFalchier, A., Clavagnier, S., Barone, P., & Kennedy, H. ( 2002 ). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22, 5749 – 5759. https://doi.org/10.1523/jneurosci.22‐13‐05749.2002
dc.identifier.citedreferenceFort, A., Delpuech, C., Pernier, J., & Giard, M. H. ( 2002 ). Dynamics of cortico‐subcortical cross‐modal operations involved in audio‐visual object detection in humans. Cerebral Cortex, 12, 1031 – 1039. https://doi.org/10.1093/cercor/12.10.1031
dc.identifier.citedreferenceGiard, M. H., & Peronnet, F. ( 1999 ). Auditory–visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11, 473 – 490. https://doi.org/10.1162/089892999563544
dc.identifier.citedreferenceGomez‐Ramirez, M., Kelly, S. P., Molholm, S., Sehatpour, P., Schwartz, T. H., & Foxe, J. J. ( 2011 ). Oscillatory sensory selection mechanisms during intersensory attention to rhythmic auditory and visual inputs: A human electrocorticographic investigation. Journal of Neuroscience, 31, 18556 – 18567. https://doi.org/10.1523/jneurosci.2164‐11.2011
dc.identifier.citedreferenceHan, C. E., Yoo, S. W., Seo, S. W., Na, D. L., & Seong, J.‐K. ( 2013 ). Cluster‐based statistics for brain connectivity in correlation with behavioral measures. PLoS One, 8, e72332. https://doi.org/10.1371/journal.pone.0072332
dc.identifier.citedreferenceJung, T.‐P., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., & Sejnowski, T. J. ( 2000 ). Removal of eye activity artifacts from visual event‐related potentials in normal and clinical subjects. Clinical Neurophysiology, 111, 1745 – 1758. https://doi.org/10.1016/s1388‐2457(00)00386‐2
dc.identifier.citedreferenceKafaligonul, H., & Stoner, G. R. ( 2010 ). Auditory modulation of visual apparent motion with short spatial and temporal intervals. Journal of Vision, 10 ( 12 ), 31. https://doi.org/10.1167/10.12.31
dc.identifier.citedreferenceKaya, U., & Kafaligonul, H. ( 2019 ). Cortical processes underlying the effects of static sound timing on perceived visual speed. NeuroImage, 199, 194 – 205. https://doi.org/10.1016/j.neuroimage.2019.05.062
dc.identifier.citedreferenceKaya, U., Yildirim, F. Z., & Kafaligonul, H. ( 2017 ). The involvement of centralized and distributed processes in sub‐second time interval adaptation: An ERP investigation of apparent motion. European Journal of Neuroscience, 46, 2325 – 2338. https://doi.org/10.1111/ejn.13691
dc.identifier.citedreferenceKeil, J., Pomper, U., Feuerbach, N., & Senkowski, D. ( 2017 ). Temporal orienting precedes intersensory attention and has opposing effects on early evoked brain activity. NeuroImage, 148, 230 – 239. https://doi.org/10.1016/j.neuroimage.2017.01.039
dc.identifier.citedreferenceMaris, E., & Oostenveld, R. ( 2007 ). Nonparametric statistical testing of EEG‐ and MEG‐data. Journal of Neuroscience Methods, 164 ( 1 ), 177 – 190. https://doi.org/10.1016/j.jneumeth.2007.03.024
dc.identifier.citedreferenceMercier, M. R., Foxe, J. J., Fiebelkorn, I. C., Butler, J. S., Schwartz, T. H., & Molholm, S. ( 2013 ). Auditory‐driven phase reset in visual cortex: Human electrocorticography reveals mechanisms of early multisensory integration. NeuroImage, 79, 19 – 29. https://doi.org/10.1016/j.neuroimage.2013.04.060
dc.identifier.citedreferenceMolholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. ( 2002 ). Multisensory auditory‐visual interactions during early sensory processing in humans: A high‐density electrical mapping study. Cognitive Brain Research, 14, 115 – 128. https://doi.org/10.1016/s0926‐6410(02)00066‐6
dc.identifier.citedreferenceMorein‐Zamir, S., Soto‐Faraco, S., & Kingstone, A. ( 2003 ). Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research, 17, 154 – 163. https://doi.org/10.1016/s0926‐6410(03)00089‐2
dc.identifier.citedreferenceMurray, M. M., Lewkowicz, D. J., Amedi, A., & Wallace, M. T. ( 2016 ). Multisensory processes: A balancing act across the lifespan. Trends in Neurosciences, 39, 567 – 579. https://doi.org/10.1016/j.tins.2016.05.003
dc.identifier.citedreferenceMurray, M. M., & Wallace, M. T. ( 2012 ). The neural bases of multisensory processes. Boca Raton, FL: CRC Press.
dc.identifier.citedreferenceNaue, N., Rach, S., Strüber, D., Huster, R. J., Zaehle, T., Körner, U., & Herrmann, C. S. ( 2011 ). Auditory event‐related response in visual cortex modulates subsequent visual responses in humans. Journal of Neuroscience, 31, 7729 – 7736. https://doi.org/10.1523/JNEUROSCI.1076‐11.2011
dc.identifier.citedreferenceNavarra, J., Hartcher‐O’Brien, J., Piazza, E., & Spence, C. ( 2009 ). Adaptation to audiovisual asynchrony modulates the speeded detection of sound. Proceedings of the National Academy of Sciences USA, 106, 9169 – 9173. https://doi.org/10.1073/pnas.0810486106
dc.identifier.citedreferenceOgulmus, C., Karacaoglu, M., & Kafaligonul, H. ( 2018 ). Temporal ventriloquism along the path of apparent motion: Speed perception under different spatial grouping principles. Experimental Brain Research, 236, 629 – 643. https://doi.org/10.1007/s00221‐017‐5159‐1
dc.working.doiNOen
dc.owningcollnameInterdisciplinary and Peer-Reviewed


Files in this item

Name:
psyp13777_am.pdf
Size:
1.121MB
Format:
PDF
View/Open
Name:
psyp13777.pdf
Size:
1.259MB
Format:
PDF
View/Open

Show simple item record

Remediation of Harmful Language

The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.

Accessibility

If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.