Subject: Re: [AUDITORY] Semantic McGurk Effect From: Bob Carlyon <Bob.Carlyon@xxxxxxxx> Date: Fri, 7 Aug 2020 08:49:05 +0000--_000_47fb66049d4843b2a7858a32e6e28988mrccbucamacuk_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Malcolm Nice video. Kind of you to shave your legs for our benefit. I think this is an example of the general finding that prior information af= fects the perception of degraded speech, which has been extensively investi= gated with vocoded speech. When vocoded with few channels this can sound in= telligible, but sounds clear and obvious when preceded by either written or= spoken (clear speech) versions of the original. It's been found that this = kind of clear-then-distorted exposure speeds up learning so that it is eas= ier to recognise new sentences vocoded in the same way: Davis MH, Johnsrude IS, Hervais-Adelman A, Taylor K, McGettigan C (2005) Le= xical information drives perceptual learning of distorted speech: Evidence = from the comprehension of noise-vocoded sentences. Journal of Experimental = Psychology-General 134:222-241. The difference between the video and most published studies is that the dis= torted speech could plausibly 'map onto' one of two sentences. My colleague= Matt Davis, in a public science lecture, had the audience play "vocoder bi= ngo", in which Matt played several vocoded words, and the audience were all= given cards with some words written on, and had to tick off each word when= they heard it. Much excitement ensued (I'm not sure what the prize was), b= ut the catch was that none o fthe words were actually presented - they were= just sufficiently similar/ambiguous to be convincing when paired with the = written text. Matt tells me that they have published an imaging study, look= ing at brain responses to ambiguous vocoded words when cued to hear them on= e way or another (e.g. 'pit' or 'kitsch'). This may be the closest publishe= d work to what you ask for: Blank, H., Spangenberg, M., Davis, M.H. (2018) Neural prediction errors dis= tinguish perception and misperception of speech. Journal of Neuroscience, 3= 8 (27) 6076-6089 https://www.jneurosci.org/content/38/27/6076 Finally, I suspect that this is not a semantic effect as I expect it would = work with non-words All the best, Bob From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxx>= On Behalf Of Malcolm Slaney Sent: 07 August 2020 00:59 To: AUDITORY@xxxxxxxx Subject: Semantic McGurk Effect Has there been anything formal published on this effect? https://www.iflscience.com/brain/what-the-hell-is-going-on-in-this-tikto= k-audio-illusion It sounds to me like a semantic version of the McGurk effect. Nice demo. - Malcolm --_000_47fb66049d4843b2a7858a32e6e28988mrccbucamacuk_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= > <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style><!-- /* Font Definitions */ @xxxxxxxx {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @xxxxxxxx {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @xxxxxxxx {font-family:"Segoe UI"; panose-1:2 11 5 2 4 2 4 2 2 3;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman",serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} p.msonormal0, li.msonormal0, div.msonormal0 {mso-style-name:msonormal; mso-margin-top-alt:auto; margin-right:0cm; mso-margin-bottom-alt:auto; margin-left:0cm; font-size:12.0pt; font-family:"Times New Roman",serif;} span.EmailStyle18 {mso-style-type:personal-reply; font-family:"Calibri",sans-serif; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @xxxxxxxx WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--> </head> <body lang=3D"EN-GB" link=3D"blue" vlink=3D"purple"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Hi Malcolm= <o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Nice video= . Kind of you to shave your legs for our benefit.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">I think th= is is an example of the general finding that prior information affects the = perception of degraded speech, which has been extensively investigated with vocoded speech. When vocoded with few channe= ls this can sound intelligible, but sounds clear and obvious when preceded = by either written or spoken (clear speech) versions of the original. ItR= 17;s been found that this kind of clear-then-distorted exposure speeds up learning so that it is easier to recognise new se= ntences vocoded in the same way:<o:p></o:p></span></p> <p class=3D"MsoNormal" style=3D"margin-left:36.0pt;text-indent:-36.0pt;text= -autospace:none"> <span style=3D"font-size:9.0pt;font-family:"Segoe UI",sans-serif"= >Davis MH, Johnsrude IS, Hervais-Adelman A, Taylor K, McGettigan C (2005) L= exical information drives perceptual learning of distorted speech: Evidence= from the comprehension of noise-vocoded sentences. Journal of Experimental Psychology-General 134:222-241.<o:p></o:p></span><= /p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">The differ= ence between the video and most published studies is that the distorted spe= ech could plausibly ‘map onto’ one of two sentences. My colleague Matt Davis, in a public science lecture, had the audience pla= y “vocoder bingo”, in which Matt played several vocoded words, = and the audience were all given cards with some words written on, and had t= o tick off each word when they heard it. Much excitement ensued (I’m not sure what the prize was), but the catch w= as that none o fthe words were actually presented – they were just su= fficiently similar/ambiguous to be convincing when paired with the written = text. Matt tells me that they have published an imaging study, looking at brain responses to ambiguous vocoded words wh= en cued to hear them one way or another (e.g. ‘pit’ or ‘k= itsch’). This may be the closest published work to what you ask for:<= o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">Blank, H., Spangenberg, M., Davis, M.H. (2018) Neur= al prediction errors distinguish perception and misperception of speech. Jo= urnal of Neuroscience, 38 (27) 6076-6089<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"><a href=3D"https://www.jneurosci.org/content/38/27/= 6076">https://www.jneurosci.org/content/38/27/6076</a><o:p></o:p></span></p= > <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Finally, I= suspect that this is not a semantic effect as I expect it would work with = non-words<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">All the be= st,<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Bob<o:p></= o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> = ;</o:p></span></p> <div> <div style=3D"border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm = 0cm 0cm"> <p class=3D"MsoNormal"><b><span lang=3D"EN-US" style=3D"font-size:11.0pt;fo= nt-family:"Calibri",sans-serif">From:</span></b><span lang=3D"EN-= US" style=3D"font-size:11.0pt;font-family:"Calibri",sans-serif"> = AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxx> <b>On Behalf Of </b>Malcolm Slaney<br> <b>Sent:</b> 07 August 2020 00:59<br> <b>To:</b> AUDITORY@xxxxxxxx<br> <b>Subject:</b> Semantic McGurk Effect<o:p></o:p></span></p> </div> </div> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">Has there been anything formal published on this eff= ect?<o:p></o:p></p> <div> <p class=3D"MsoNormal"> <a href=3D"https://www.iflscience.com/b= rain/what-the-hell-is-going-on-in-this-tiktok-audio-illusion" target=3D"_bl= ank"><span style=3D"font-family:"Arial",sans-serif;color:#1155CC;= background:white">https://www.iflscience.com/brain/what-the-hell-is-going-o= n-in-this-tiktok-audio-illusion</span></a><o:p></o:p></p> </div> <div> <p class=3D"MsoNormal"><o:p> </o:p></p> </div> <div> <p class=3D"MsoNormal">It sounds to me like a semantic version of the McGur= k effect.<o:p></o:p></p> </div> <div> <p class=3D"MsoNormal"><o:p> </o:p></p> </div> <div> <p class=3D"MsoNormal">Nice demo.<o:p></o:p></p> </div> <div> <p class=3D"MsoNormal"><o:p> </o:p></p> </div> <div> <p class=3D"MsoNormal">- Malcolm<o:p></o:p></p> </div> <div> <p class=3D"MsoNormal"><o:p> </o:p></p> </div> </div> </body> </html> --_000_47fb66049d4843b2a7858a32e6e28988mrccbucamacuk_--