Subject: Re: [AUDITORY] Visual references in sound localisation From: "Engel Alonso-Martinez, Isaac" <isaac.engel@xxxxxxxx> Date: Wed, 28 Feb 2018 15:23:06 +0000 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>--_000_DB6PR0601MB21679593DED30ADD6A2651ABD3C70DB6PR0601MB2167_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Dear all, Thank you all very much for your responses. It seems that there is plenty of literature on the effect of visual stimuli= in auditory localisation. If anyone is interested, a summary of relevant k= eywords for this topic could be: 'visual capture', 'visual dominance', 'vis= ual bias' and 'cross-modal bias'. Also, one may find relevant papers under:= 'multimodal integration', 'multisensory integration' and 'cross-modal plas= ticity'. I have found that a common practice is to use only one visual cue and one a= uditory cue at the same time. If the two stimuli are close to be spatially = congruent, the subject will probably bind the two of them together unconsci= ously, thus causing this 'visual capture' effect in which the visual stimul= us dominates the auditory one. This may not happen if the two stimuli are n= ot spatially congruent in a noticeable way [1, 2]. However, in the scenario that I proposed originally there are two auditory = stimuli: one of them is explicitly associated to the visual cue and would a= ct as an 'anchor', while the other one has to be located. Intuitively, one = might think that if the two auditory cues are perceived as different source= s, the risk of visual dominance should be small. As it has been pointed out, another part of the question is on 'relative lo= calisation' and comparative judgements, particularly in multimodal scenario= s. How good are we at estimating the location of two sound sources with res= pect to each other? And what happens if we introduce visual cues? All suggestions are welcome! Thank you all again for your contributions. Kind regards, Isaac Engel References: [1] Bosen, Adam K. et al. 2016. =93Comparison of Congruence Judgment and Au= ditory Localization Tasks for Assessing the Spatial Limits of Visual Captur= e.=94 Biological Cybernetics 110(6): 455=9671 [2] Berger, Christopher C., et al. "Generic HRTFs may be good enough in Vir= tual Reality. Improving source localization through cross-modal plasticity.= " Frontiers in Neuroscience 12 (2018): 21. -- Isaac Engel PhD student at Dyson School of Design Engineering Imperial College London 10 Princes Gardens South Kensington, SW7 1NA, London E-mail: isaac.engel@xxxxxxxx <http://www.imperial.ac.uk/design-engineering-school> www.imperial.ac.uk/design-engineering/research/human-performance-and-experi= ence/sound-and-audio-systems<http://www.imperial.ac.uk/design-engineering/r= esearch/human-performance-and-experience/sound-and-audio-systems> ________________________________ From: Engel Alonso-Martinez, Isaac Sent: 24 February 2018 19:08 To: auditory@xxxxxxxx Subject: Visual references in sound localisation Dear all, I am interested in the impact of audible visual references in sound localis= ation tasks. For instance, let's say that you are presented two different continuous sou= nds (e.g., speech) coming from sources A and B, which are in different loca= tions. While source A is clearly visible to you, B is invisible and you are= asked to estimate its location. Will source A act as a spatial reference, = helping you in doing a more accurate estimation, or will it be distracting = and make the task more difficult? If anyone can point to some literature on this, it would be greatly appreci= ated. Kind regards, Isaac Engel --_000_DB6PR0601MB21679593DED30ADD6A2651ABD3C70DB6PR0601MB2167_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1= 252"> <style type=3D"text/css" style=3D"display:none;"><!-- P {margin-top:0;margi= n-bottom:0;} --></style> </head> <body dir=3D"ltr"> <div id=3D"divtagdefaultwrapper" style=3D"font-size:12pt;color:#000000;font= -family:Calibri,Helvetica,sans-serif;" dir=3D"ltr"> <div id=3D"divtagdefaultwrapper" style=3D"font-size: 12pt; color: rgb(0, 0,= 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Co= lor Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI= Symbol", "Android Emoji", EmojiSymbols;" dir=3D"ltr"> <p style=3D"margin-top:0;margin-bottom:0"></p> <div>Dear all,</div> <div><br> </div> <div>Thank you all very much for your responses. </div> <div><br> </div> <div>It seems that there is plenty of literature on the effect of visual st= imuli in auditory localisation. If anyone is interested, a summary of relev= ant keywords for this topic could be: 'visual capture', 'visual dominance',= 'visual bias' and 'cross-modal bias'. Also, one may find relevant papers under: 'multimodal integration',= 'multisensory integration' and 'cross-modal plasticity'.</div> <div><br> </div> <div>I have found that a common practice is to use only one visual cue and = one auditory cue at the same time. If the two stimuli are close to be spati= ally congruent, the subject will probably bind the two of them together unc= onsciously, thus causing this 'visual capture' effect in which the visual stimulus dominates the auditory one. T= his may not happen if the two stimuli are not spatially congruent = ;in a noticeable way [1, 2].</div> <div><br> </div> <div>However, in the scenario that I proposed originally there are two audi= tory stimuli: one of them is explicitly associated to the visual cue and wo= uld act as an 'anchor', while the other one has to be located. Intuitively,= one might think that if the two auditory cues are perceived as different sources, the risk of visual domin= ance should be small.</div> <div><br> </div> <div>As it has been pointed out, another part of the question is on 'r= elative localisation' and comparative judgements, particularly in multimoda= l scenarios. How good are we at estimating the location of two sound s= ources with respect to each other? And what happens if we introduce visual cues?</div> <div><br> </div> <div>All suggestions are welcome! Thank you all again for your contrib= utions.</div> <div><br> </div> <div>Kind regards,</div> <div>Isaac Engel</div> <div><br> </div> <div>References:</div> <div>[1] Bosen, Adam K. et al. 2016. =93Comparison of Congruence Judgment a= nd Auditory Localization Tasks for Assessing the Spatial Limits of Visual C= apture.=94 Biological Cybernetics 110(6): 455=9671</div> <div>[2] Berger, Christopher C., et al. "Generic HRTFs may be good eno= ugh in Virtual Reality. Improving source localization through cross-modal p= lasticity." Frontiers in Neuroscience 12 (2018): 21.</div> <p style=3D"margin-top:0;margin-bottom:0"><br> </p> <div id=3D"Signature"> <div id=3D"divtagdefaultwrapper" dir=3D"ltr" style=3D"font-size: 12pt; colo= r: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif, EmojiF= ont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEm= oji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;"= > <p><font size=3D"2" style=3D"font-family:Tahoma"><span style=3D"font-size:1= 0pt">-- </span></font><font size=3D"2" style=3D"font-family:Tahoma"><s= pan style=3D"font-size:10pt"><br> Isaac Engel</span></font><font size=3D"2" style=3D"font-family:Tahoma"><spa= n style=3D"font-size:10pt"><br> </span></font><font size=3D"2" style=3D"font-family:Tahoma"><span style=3D"= font-size:10pt">PhD student at Dyson School of Design Engineering</spa= n></font><font size=3D"2" style=3D"font-family:Tahoma"><span style=3D"font-= size:10pt"><br> Imperial College London</span></font><font size=3D"2" style=3D"font-family:= Tahoma"><span style=3D"font-size:10pt"><br> 10 Princes Gardens</span></font><font size=3D"2" style=3D"font-family:Tahom= a"><span style=3D"font-size:10pt"><br> South Kensington, SW7 1NA, London </span></font><font size=3D"2" style= =3D"font-family:Tahoma"><span style=3D"font-size:10pt"><br> E-mail: isaac.engel@xxxxxxxx</span></font><font size=3D"2" style=3D"f= ont-family:Tahoma"><span style=3D"font-size:10pt"><br> </span></font><font size=3D"2" style=3D"font-family:Tahoma"><span style=3D"= font-size:10pt"><br> </span></font><a href=3D"http://www.imperial.ac.uk/design-engineering-schoo= l" target=3D"_blank" id=3D"LPNoLP" style=3D"font-family:Tahoma; font-size:1= 6px" title=3D"http://www.imperial.ac.uk/design-engineering-school=0A= Ctrl+Click or tap to follow the link"><font size=3D"2"><span id=3D"LPNo= LP" style=3D"font-size:10pt"></span></font></a></p> <p><a href=3D"http://www.imperial.ac.uk/design-engineering/research/human-p= erformance-and-experience/sound-and-audio-systems" class=3D"OWAAutoLink" id= =3D"LPlnk65443" previewremoved=3D"true">www.imperial.ac.uk/design-engineeri= ng/research/human-performance-and-experience/sound-and-audio-systems</a><br= > </p> </div> </div> <p style=3D"margin-top:0;margin-bottom:0"><br> </p> <div id=3D"Signature"> <div id=3D"divtagdefaultwrapper" dir=3D"ltr" style=3D"font-size:12pt; color= :#000000; font-family:Calibri,Arial,Helvetica,sans-serif"> <p><br> </p> </div> </div> <div style=3D"color: rgb(0, 0, 0);"> <hr style=3D"display:inline-block;width:98%" tabindex=3D"-1"> <div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st= yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Engel Alonso-Martinez= , Isaac<br> <b>Sent:</b> 24 February 2018 19:08<br> <b>To:</b> auditory@xxxxxxxx<br> <b>Subject:</b> Visual references in sound localisation</font> <div> </div> </div> <div dir=3D"ltr"> <div id=3D"x_divtagdefaultwrapper" dir=3D"ltr" style=3D"font-size:12pt; col= or:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"App= le Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe = UI Symbol","Android Emoji",EmojiSymbols"> <p style=3D"margin-top:0; margin-bottom:0"></p> <div></div> <div style=3D"font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Appl= e Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe U= I Symbol","Android Emoji",EmojiSymbols; font-size:16px"> <font face=3D"Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Sego= e UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" size= =3D"3" color=3D"black"><span style=3D"font-size:12pt"> <div>Dear all,</div> <div><br> </div> </span></font></div> <div style=3D"font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Appl= e Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe U= I Symbol","Android Emoji",EmojiSymbols; font-size:16px"> <font face=3D"Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Sego= e UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" size= =3D"3" color=3D"black"><span style=3D"font-size:12pt"> <div>I am interested in the impact of audible visual references in sound lo= calisation tasks.</div> <div><br> </div> </span></font></div> <div style=3D"font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Appl= e Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe U= I Symbol","Android Emoji",EmojiSymbols; font-size:16px"> <font face=3D"Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Sego= e UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" size= =3D"3" color=3D"black"><span style=3D"font-size:12pt"> <div>For instance, let's say that you are presented two different continuou= s sounds (e.g., speech) coming from sources A and B, which are in different= locations. While source A is clearly visible to you, B is invisible and yo= u are asked to estimate its location. Will source A act as a spatial reference, helping you in doing a more accu= rate estimation, or will it be distracting and make the task more difficult= ?</div> <div><br> </div> </span></font></div> <div style=3D"font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Appl= e Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe U= I Symbol","Android Emoji",EmojiSymbols; font-size:16px"> <font face=3D"Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Sego= e UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" size= =3D"3" color=3D"black"><span style=3D"font-size:12pt"> <div>If anyone can point to some literature on this, it would be greatly ap= preciated.</div> <div><br> </div> </span></font></div> <div style=3D"font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Appl= e Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe U= I Symbol","Android Emoji",EmojiSymbols; font-size:16px"> <font face=3D"Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Sego= e UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" size= =3D"3" color=3D"black"><span style=3D"font-size:12pt"> <div>Kind regards,</div> </span></font></div> <div style=3D"font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Appl= e Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe U= I Symbol","Android Emoji",EmojiSymbols; font-size:16px"> <font face=3D"Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Sego= e UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" size= =3D"3" color=3D"black"><span style=3D"font-size:12pt"> <div>Isaac Engel</div> </span></font></div> <div></div> <p></p> </div> </div> </div> </div> <style type=3D"text/css" style=3D"display:none"> <!-- p {margin-top:0; ma= rgin-bottom:0} --> </style></div> </body> </html> --_000_DB6PR0601MB21679593DED30ADD6A2651ABD3C70DB6PR0601MB2167_--