Audiovisual interactions in speeded discrimination of a visual event
dc.contributor.author | Kaya, Utku | |
dc.contributor.author | Kafaligonul, Hulusi | |
dc.date.accessioned | 2021-04-06T02:12:15Z | |
dc.date.available | 2022-05-05 22:12:14 | en |
dc.date.available | 2021-04-06T02:12:15Z | |
dc.date.issued | 2021-04 | |
dc.identifier.citation | Kaya, Utku; Kafaligonul, Hulusi (2021). "Audiovisual interactions in speeded discrimination of a visual event." Psychophysiology (4): n/a-n/a. | |
dc.identifier.issn | 0048-5772 | |
dc.identifier.issn | 1469-8986 | |
dc.identifier.uri | https://hdl.handle.net/2027.42/167075 | |
dc.description.abstract | The integration of information from different senses is central to our perception of the external world. Audiovisual interactions have been particularly well studied in this context and various illusions have been developed to demonstrate strong influences of these interactions on the final percept. Using audiovisual paradigms, previous studies have shown that even task‐irrelevant information provided by a secondary modality can change the detection and discrimination of a primary target. These modulations have been found to be significantly dependent on the relative timing between auditory and visual stimuli. Although these interactions in time have been commonly reported, we have still limited understanding of the relationship between the modulations of event‐related potentials (ERPs) and final behavioral performance. Here, we aimed to shed light on this important issue by using a speeded discrimination paradigm combined with electroencephalogram (EEG). During the experimental sessions, the timing between an auditory click and a visual flash was varied over a wide range of stimulus onset asynchronies and observers were engaged in speeded discrimination of flash location. Behavioral reaction times were significantly changed by click timing. Furthermore, the modulations of evoked activities over medial parietal/parieto‐occipital electrodes were associated with this effect. These modulations were within the 126–176 ms time range and more importantly, they were also correlated with the changes in reaction times. These results provide an important functional link between audiovisual interactions at early stages of sensory processing and reaction times. Together with previous research, they further suggest that early crossmodal interactions play a critical role in perceptual performance.Crossmodal interactions in the temporal domain are a crucial aspect of multisensory integration. Using onset timing (SOA) as a critical factor, we found audiovisual interactions in early ERPs correlated with the changes in behavioral reaction times. These findings provide an important functional link between audiovisual interactions at early stages of sensory processing and perceptual performance. | |
dc.publisher | CRC Press | |
dc.publisher | Wiley Periodicals, Inc. | |
dc.subject.other | multisensory | |
dc.subject.other | reaction time | |
dc.subject.other | visual timing | |
dc.subject.other | EEG | |
dc.subject.other | audiovisual interactions | |
dc.title | Audiovisual interactions in speeded discrimination of a visual event | |
dc.type | Article | |
dc.rights.robots | IndexNoFollow | |
dc.subject.hlbsecondlevel | Physiology | |
dc.subject.hlbsecondlevel | Psychology | |
dc.subject.hlbtoplevel | Health Sciences | |
dc.description.peerreviewed | Peer Reviewed | |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/167075/1/psyp13777_am.pdf | |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/167075/2/psyp13777.pdf | |
dc.identifier.doi | 10.1111/psyp.13777 | |
dc.identifier.source | Psychophysiology | |
dc.identifier.citedreference | Spence, C., & Squire, S. B. ( 2003 ). Multisensory integration: Maintaining the perception of synchrony. Current Biology, 13, R519 – R521. https://doi.org/10.1016/s0960‐9822(03)00445‐7 | |
dc.identifier.citedreference | Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J. M. ( 2011 ). FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience, 2011, 156869. https://doi.org/10.1155/2011/156869 | |
dc.identifier.citedreference | Pelli, D. ( 1997 ). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437 – 442. https://doi.org/10.1163/156856897X00366 | |
dc.identifier.citedreference | Perrin, F., Pernier, J., Bertrand, O., & Echallier, J. F. ( 1989 ). Spherical splines for scalp potential and current density mapping. Electroencephalography and Clinical Neurophysiology, 72, 184 – 187. https://doi.org/10.1016/0013‐4694(89)90180‐6 | |
dc.identifier.citedreference | Raij, T., Ahveninen, J., Lin, F.‐H., Witzel, T., Jääskeläinen, I. P., Letham, B., Israeli, E., Sahyoun, C., Vasios, C., Stufflebeam, S., Hämäläinen, M., & Belliveau, J. W. ( 2010 ). Onset timing of cross‐sensory activations and multisensory interactions in auditory and visual sensory cortices. European Journal of Neuroscience, 31, 1772 – 1782. https://doi.org/10.1111/j.1460‐9568.2010.07213.x | |
dc.identifier.citedreference | Rammsayer, T. H., Borter, N., & Troche, S. J. ( 2015 ). Visual‐auditory differences in duration discrimination of intervals in the subsecond and second range. Frontiers in Psychology, 6, 1626. https://doi.org/10.3389/fpsyg.2015.01626 | |
dc.identifier.citedreference | Ribeiro, M. J., & Castelo‐Branco, M. ( 2019 ). Neural correlates of anticipatory cardiac deceleration and its association with the speed of perceptual decision‐making, in young and older adults. NeuroImage, 199, 521 – 533. https://doi.org/10.1016/j.neuroimage.2019.06.004 | |
dc.identifier.citedreference | Senkowski, D., Talsma, D., Grigutsch, M., Herrmann, C. S., & Woldorff, M. G. ( 2007 ). Good times for multisensory integration: Effects of the precision of temporal synchrony as revealed by gamma‐band oscillations. Neuropsychologia, 45, 561 – 571. https://doi.org/10.1016/j.neuropsychologia.2006.01.013 | |
dc.identifier.citedreference | Spence, C. ( 2018 ). Multisensory perception. In I. J. Wixted (Ed.), Stevens’ handbook of experimental psychology and cognitive neuroscience (Vol. 2, pp. 1 – 56 ). John Wiley and Sons. https://doi.org/10.1002/9781119170174.epcn214 | |
dc.identifier.citedreference | Stefanics, G., Hangya, B., Hernadi, I., Winkler, I., Lakatos, P., & Ulbert, I. ( 2010 ). Phase entrainment of human delta oscillations can mediate the effects of expectation on reaction speed. Journal of Neuroscience, 30, 13578 – 13585. https://doi.org/10.1523/jneurosci.0703‐10.2010 | |
dc.identifier.citedreference | Stekelenburg, J. J., & Vroomen, J. ( 2005 ). An event‐related potential investigation of the time‐course of temporal ventriloquism. NeuroReport, 16, 641 – 644. https://doi.org/10.1097/00001756‐200504250‐00025 | |
dc.identifier.citedreference | Stevenson, R. A., Ghose, D., Fister, J. K., Sarko, D. K., Altieri, N. A., Nidiffer, A. R., Kurela, L. A. R., Siemann, J. K., James, T. W., & Wallace, M. T. ( 2014 ). Identifying and quantifying multisensory integration: A tutorial review. Brain Topography, 27, 707 – 730. https://doi.org/10.1007/s10548‐014‐0365‐7 | |
dc.identifier.citedreference | Talsma, D., Senkowski, D., Soto‐Faraco, S., & Woldorff, M. G. ( 2010 ). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14, 400 – 410. https://doi.org/10.1016/j.tics.2010.06.008 | |
dc.identifier.citedreference | Talsma, D., Senkowski, D., & Woldorff, M. G. ( 2009 ). Intermodal attention affects the processing of the temporal alignment of audiovisual stimuli. Experimental Brain Research, 198, 313 – 328. https://doi.org/10.1007/s00221‐009‐1858‐6 | |
dc.identifier.citedreference | Teder‐Sälejärvi, W. A., Di Russo, F., McDonald, J. J., & Hillyard, S. A. ( 2005 ). Effects of spatial congruity on audio‐visual multimodal integration. Journal of Cognitive Neuroscience, 17 ( 9 ), 1396 – 1409. https://doi.org/10.1162/0898929054985383 | |
dc.identifier.citedreference | Teder‐Sälejärvi, W. A., McDonald, J. J., Di Russo, F., & Hillyard, S. A. ( 2002 ). An analysis of audio‐visual crossmodal integration by means of event‐related potential (ERP) recordings. Cognitive Brain Research, 14, 106 – 114. https://doi.org/10.1016/s0926‐6410(02)00065‐4 | |
dc.identifier.citedreference | Thorne, J. D., De Vos, M., Viola, F. C., & Debener, S. ( 2011 ). Cross‐modal phase reset predicts auditory task performance in humans. Journal of Neuroscience, 31, 3853 – 3861. https://doi.org/10.1523/jneurosci.6176‐10.2011 | |
dc.identifier.citedreference | Thorne, J. D., & Debener, S. ( 2014 ). Look now and hear what’s coming: On the functional role of cross‐modal phase reset. Hearing Research, 307, 144 – 152. https://doi.org/10.1016/j.heares.2013.07.002 | |
dc.identifier.citedreference | van Atteveldt, N., Murray, M. M., Thut, G., & Schroeder, C. E. ( 2014 ). Multisensory integration: Flexible use of general operations. Neuron, 81, 1240 – 1253. https://doi.org/10.1016/j.neuron.2014.02.044 | |
dc.identifier.citedreference | van den Brink, R. L., Cohen, M. X., van der Burg, E., Talsma, D., Vissers, M. E., & Slagter, H. A. ( 2014 ). Subcortical, modality‐specific pathways contribute to multisensory processing in humans. Cerebral Cortex, 24 ( 8 ), 2169 – 2177. https://doi.org/10.1093/cercor/bht069 | |
dc.identifier.citedreference | Vroomen, J., & Keetels, M. ( 2010 ). Perception of intersensory synchrony: A tutorial review. Attention Perception and Psychophysics, 72, 871 – 884. https://doi.org/10.3758/app.72.4.871 | |
dc.identifier.citedreference | World Medical Association. ( 2013 ). Declaration of Helsinki: Ethical principles for medical research involving human subjects. Journal of the American Medical Association, 310 ( 20 ), 2191 – 2194. https://doi.org/10.1001/jama.2013.281053 | |
dc.identifier.citedreference | Zhou, H. Y., Cheung, E. F. C., & Chan, R. C. K. ( 2020 ). Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions. Neuropsychologia, 140, 107396. https://doi.org/10.1016/j.neuropsychologia.2020.107396 | |
dc.identifier.citedreference | Freeman, E., & Driver, J. ( 2008 ). Direction of visual apparent motion driven solely by timing of a static sound. Current Biology, 18, 1262 – 1266. https://doi.org/10.1016/j.cub.2008.07.066 | |
dc.identifier.citedreference | Getzmann, S. ( 2007 ). The effect of brief auditory stimuli on visual apparent motion. Perception, 36, 1089 – 1103. https://doi.org/10.1068/p5741 | |
dc.identifier.citedreference | Allen, P. J., Polizzi, G., Krakow, K., Fish, D. R., & Lemieux, L. ( 1998 ). Identification of EEG events in the MR scanner: The problem of pulse artifact and a method for its subtraction. NeuroImage, 8, 229 – 239. https://doi.org/10.1006/nimg.1998.0361 | |
dc.identifier.citedreference | Benjamini, Y., & Hochberg, Y. ( 1995 ). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society B, 57, 289 – 300. https://doi.org/10.1111/j.2517‐6161.1995.tb02031.x | |
dc.identifier.citedreference | Benjamini, Y., & Yekutieli, D. ( 2001 ). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29, 1165 – 1188. https://doi.org/10.1214/aos/1013699998 | |
dc.identifier.citedreference | Besle, J., Fort, A., & Giard, M. ( 2004 ). Interest and validity of the additive model in electrophysiological studies of multisensory interactions. Cognitive Processing, 5, 189 – 192. https://doi.org/10.1007/s10339‐004‐0026‐y | |
dc.identifier.citedreference | Brainard, D. ( 1997 ). The psychophysics toolbox. Spatial Vision, 10, 433 – 436. https://doi.org/10.1163/156856897X00357 | |
dc.identifier.citedreference | Burr, D., Banks, M., & Morrone, M. ( 2009 ). Auditory dominance over vision in the perception of interval duration. Experimental Brain Research, 198, 49 – 57. https://doi.org/10.1007/s00221‐009‐1933‐z | |
dc.identifier.citedreference | Cappe, C., & Barone, P. ( 2005 ). Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. European Journal of Neuroscience, 22, 2886 – 2902. https://doi.org/10.1111/j.1460‐9568.2005.04462.x | |
dc.identifier.citedreference | Cappe, C., Thut, G., Romei, V., & Murray, M. M. ( 2010 ). Auditory‐visual multisensory interactions in humans: Timing, topography, directionality, and sources. Journal of Neuroscience, 30, 12572 – 12580. https://doi.org/10.1523/jneurosci.1099‐10.2010 | |
dc.identifier.citedreference | Cardoso‐Leite, P., Gorea, A., & Mamassian, P. ( 2007 ). Temporal order judgment and simple reaction times: Evidence for a common processing system. Journal of Vision, 7 ( 6 ), 11. https://doi.org/10.1167/7.6.11 | |
dc.identifier.citedreference | Cecere, R., Gross, J., Willis, A., & Thut, G. ( 2017 ). Being first matters: Topographical representational similarity analysis of ERP signals reveals separate networks for audiovisual temporal binding depending on the leading sense. Journal of Neuroscience, 37, 5274 – 5287. https://doi.org/10.1523/jneurosci.2926‐16.2017 | |
dc.identifier.citedreference | Chen, L., & Vroomen, J. ( 2013 ). Intersensory binding across space and time: A tutorial review. Attention Perception and Psychophysics, 75, 790 – 811. https://doi.org/10.3758/s13414‐013‐0475‐4 | |
dc.identifier.citedreference | Clavagnier, S., Falchier, A., & Kennedy, H. ( 2004 ). Long‐distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective and Behavioral Neuroscience, 4, 117 – 126. https://doi.org/10.3758/cabn.4.2.117 | |
dc.identifier.citedreference | Colosio, M., Shestakova, A., Nikulin, V. V., Blagovechtchenski, E., & Klucharev, V. ( 2017 ). Neural mechanisms of cognitive dissonance (revised): An EEG study. Journal of Neuroscience, 37 ( 20 ), 5074 – 5083. https://doi.org/10.1523/jneurosci.3209‐16.2017 | |
dc.identifier.citedreference | Diederich, A., Schomburg, A., & Colonius, H. ( 2012 ). Saccadic reaction times to audiovisual stimuli show effects of oscillatory phase reset. PLoS One, 7 ( 10 ), e44910. https://doi.org/10.1371/journal.pone.0044910 | |
dc.identifier.citedreference | Donchin, E., & Lindsley, D. B. ( 1966 ). Average evoked potentials and reaction times to visual stimuli. Electroencephalography and Clinical Neurophysiology, 20, 217 – 223. https://doi.org/10.1016/0013‐4694(66)90086‐1 | |
dc.identifier.citedreference | Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. ( 2002 ). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22, 5749 – 5759. https://doi.org/10.1523/jneurosci.22‐13‐05749.2002 | |
dc.identifier.citedreference | Fort, A., Delpuech, C., Pernier, J., & Giard, M. H. ( 2002 ). Dynamics of cortico‐subcortical cross‐modal operations involved in audio‐visual object detection in humans. Cerebral Cortex, 12, 1031 – 1039. https://doi.org/10.1093/cercor/12.10.1031 | |
dc.identifier.citedreference | Giard, M. H., & Peronnet, F. ( 1999 ). Auditory–visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11, 473 – 490. https://doi.org/10.1162/089892999563544 | |
dc.identifier.citedreference | Gomez‐Ramirez, M., Kelly, S. P., Molholm, S., Sehatpour, P., Schwartz, T. H., & Foxe, J. J. ( 2011 ). Oscillatory sensory selection mechanisms during intersensory attention to rhythmic auditory and visual inputs: A human electrocorticographic investigation. Journal of Neuroscience, 31, 18556 – 18567. https://doi.org/10.1523/jneurosci.2164‐11.2011 | |
dc.identifier.citedreference | Han, C. E., Yoo, S. W., Seo, S. W., Na, D. L., & Seong, J.‐K. ( 2013 ). Cluster‐based statistics for brain connectivity in correlation with behavioral measures. PLoS One, 8, e72332. https://doi.org/10.1371/journal.pone.0072332 | |
dc.identifier.citedreference | Jung, T.‐P., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., & Sejnowski, T. J. ( 2000 ). Removal of eye activity artifacts from visual event‐related potentials in normal and clinical subjects. Clinical Neurophysiology, 111, 1745 – 1758. https://doi.org/10.1016/s1388‐2457(00)00386‐2 | |
dc.identifier.citedreference | Kafaligonul, H., & Stoner, G. R. ( 2010 ). Auditory modulation of visual apparent motion with short spatial and temporal intervals. Journal of Vision, 10 ( 12 ), 31. https://doi.org/10.1167/10.12.31 | |
dc.identifier.citedreference | Kaya, U., & Kafaligonul, H. ( 2019 ). Cortical processes underlying the effects of static sound timing on perceived visual speed. NeuroImage, 199, 194 – 205. https://doi.org/10.1016/j.neuroimage.2019.05.062 | |
dc.identifier.citedreference | Kaya, U., Yildirim, F. Z., & Kafaligonul, H. ( 2017 ). The involvement of centralized and distributed processes in sub‐second time interval adaptation: An ERP investigation of apparent motion. European Journal of Neuroscience, 46, 2325 – 2338. https://doi.org/10.1111/ejn.13691 | |
dc.identifier.citedreference | Keil, J., Pomper, U., Feuerbach, N., & Senkowski, D. ( 2017 ). Temporal orienting precedes intersensory attention and has opposing effects on early evoked brain activity. NeuroImage, 148, 230 – 239. https://doi.org/10.1016/j.neuroimage.2017.01.039 | |
dc.identifier.citedreference | Maris, E., & Oostenveld, R. ( 2007 ). Nonparametric statistical testing of EEG‐ and MEG‐data. Journal of Neuroscience Methods, 164 ( 1 ), 177 – 190. https://doi.org/10.1016/j.jneumeth.2007.03.024 | |
dc.identifier.citedreference | Mercier, M. R., Foxe, J. J., Fiebelkorn, I. C., Butler, J. S., Schwartz, T. H., & Molholm, S. ( 2013 ). Auditory‐driven phase reset in visual cortex: Human electrocorticography reveals mechanisms of early multisensory integration. NeuroImage, 79, 19 – 29. https://doi.org/10.1016/j.neuroimage.2013.04.060 | |
dc.identifier.citedreference | Molholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. ( 2002 ). Multisensory auditory‐visual interactions during early sensory processing in humans: A high‐density electrical mapping study. Cognitive Brain Research, 14, 115 – 128. https://doi.org/10.1016/s0926‐6410(02)00066‐6 | |
dc.identifier.citedreference | Morein‐Zamir, S., Soto‐Faraco, S., & Kingstone, A. ( 2003 ). Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research, 17, 154 – 163. https://doi.org/10.1016/s0926‐6410(03)00089‐2 | |
dc.identifier.citedreference | Murray, M. M., Lewkowicz, D. J., Amedi, A., & Wallace, M. T. ( 2016 ). Multisensory processes: A balancing act across the lifespan. Trends in Neurosciences, 39, 567 – 579. https://doi.org/10.1016/j.tins.2016.05.003 | |
dc.identifier.citedreference | Murray, M. M., & Wallace, M. T. ( 2012 ). The neural bases of multisensory processes. Boca Raton, FL: CRC Press. | |
dc.identifier.citedreference | Naue, N., Rach, S., Strüber, D., Huster, R. J., Zaehle, T., Körner, U., & Herrmann, C. S. ( 2011 ). Auditory event‐related response in visual cortex modulates subsequent visual responses in humans. Journal of Neuroscience, 31, 7729 – 7736. https://doi.org/10.1523/JNEUROSCI.1076‐11.2011 | |
dc.identifier.citedreference | Navarra, J., Hartcher‐O’Brien, J., Piazza, E., & Spence, C. ( 2009 ). Adaptation to audiovisual asynchrony modulates the speeded detection of sound. Proceedings of the National Academy of Sciences USA, 106, 9169 – 9173. https://doi.org/10.1073/pnas.0810486106 | |
dc.identifier.citedreference | Ogulmus, C., Karacaoglu, M., & Kafaligonul, H. ( 2018 ). Temporal ventriloquism along the path of apparent motion: Speed perception under different spatial grouping principles. Experimental Brain Research, 236, 629 – 643. https://doi.org/10.1007/s00221‐017‐5159‐1 | |
dc.working.doi | NO | en |
dc.owningcollname | Interdisciplinary and Peer-Reviewed |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.