|
From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> on behalf of Bob Carlyon <Bob.Carlyon@xxxxxxxxxxxxxxxxx>
Sent: Friday, August 7, 2020 4:49 AM To: AUDITORY@xxxxxxxxxxxxxxx Subject: [External] Re: Semantic McGurk Effect
This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources.
Hi Malcolm
Nice video. Kind of you to shave your legs for our benefit.
I think this is an example of the general finding that prior information affects the perception of degraded speech, which has been extensively investigated with vocoded speech. When vocoded with few channels this can sound intelligible, but sounds clear and obvious when preceded by either written or spoken (clear speech) versions of the original. It’s been found that this kind of clear-then-distorted exposure speeds up learning so that it is easier to recognise new sentences vocoded in the same way: Davis MH, Johnsrude IS, Hervais-Adelman A, Taylor K, McGettigan C (2005) Lexical information drives perceptual learning of distorted speech: Evidence from the comprehension of noise-vocoded sentences. Journal of Experimental Psychology-General 134:222-241.
The difference between the video and most published studies is that the distorted speech could plausibly ‘map onto’ one of two sentences. My colleague Matt Davis, in a public science lecture, had the audience play “vocoder bingo”, in which Matt played several vocoded words, and the audience were all given cards with some words written on, and had to tick off each word when they heard it. Much excitement ensued (I’m not sure what the prize was), but the catch was that none o fthe words were actually presented – they were just sufficiently similar/ambiguous to be convincing when paired with the written text. Matt tells me that they have published an imaging study, looking at brain responses to ambiguous vocoded words when cued to hear them one way or another (e.g. ‘pit’ or ‘kitsch’). This may be the closest published work to what you ask for:
Blank, H., Spangenberg, M., Davis, M.H. (2018) Neural prediction errors distinguish perception and misperception of speech. Journal of Neuroscience, 38 (27) 6076-6089 https://www.jneurosci.org/content/38/27/6076
Finally, I suspect that this is not a semantic effect as I expect it would work with non-words
All the best,
Bob
From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx>
On Behalf Of Malcolm Slaney
Has there been anything formal published on this effect?
It sounds to me like a semantic version of the McGurk effect.
Nice demo.
- Malcolm
|
Attachment:
Goldiamond&Hawkins.1958.pdf
Description: Goldiamond&Hawkins.1958.pdf