[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [AUDITORY] Semantic McGurk Effect



I think you will find that the mondegreen is something different.  As in the Beatles song, "When I'm six feet four"
Elaine


From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> on behalf of Nathaniel Zuk <ZUKN@xxxxxx>
Sent: Saturday, 8 August 2020 7:57 PM
To: AUDITORY@xxxxxxxxxxxxxxx <AUDITORY@xxxxxxxxxxxxxxx>
Subject: Re: [AUDITORY] Semantic McGurk Effect
 
(Today I learned) It's called a mondegreen.

Nate

From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> on behalf of Prof. Roger K. Moore <0000011559506d60-dmarc-request@xxxxxxxxxxxxxxx>
Sent: Friday, August 7, 2020 3:40 PM
To: AUDITORY@xxxxxxxxxxxxxxx <AUDITORY@xxxxxxxxxxxxxxx>
Subject: Re: [AUDITORY] Semantic McGurk Effect
 
Great example Julia - and, of course, comedians such as Bec Hill (https://youtu.be/nEdR44Iftb4) and Peter Kay (https://youtu.be/7my5baoCVv8) trade on this effect - R



On Fri, 7 Aug 2020 at 14:42, Julia Strand <jstrand@xxxxxxxxxxxx> wrote:
I'm always delighted when auditory phenomena spark the public's interest! 

I wouldn't call this a semantic McGurk, given that it doesn't have to be driven by simultaneous bottom-up input from two modalities. That is, even if nothing is written on the screen but you're just thinking "green needle" to yourself, that's what you're likely to hear (whereas thinking "ga" while hearing "ba" won't get you to "da" - you need the simultaneous input from face and voice). So I'd agree with Roger that it's more akin to the phoneme restoration effect or work like Cynthia Connine's "she ran hot water for the p/bath," showing how expectations influence interpretation of bottom-up input.

I think most of US wouldn't be surprised that the same stimulus can be perceived in different ways, but my impression is that the general public tends to believe "what you see is what you get" and underestimates the power of top-down influences. Same reason #TheDress was such a hit. 

When I include this in my class on speech perception, I also include this video which shows Grover from Sesame street saying EITHER "Yes, yes, that sounds like an excellent idea!" OR "Yes, yes, that's a f*%#g excellent idea!"

Like I'm always telling my students - Speech is hard! Context helps!

Best,
Julia

On Fri, Aug 7, 2020 at 4:28 AM Prof. Roger K. Moore <0000011559506d60-dmarc-request@xxxxxxxxxxxxxxx> wrote:
I must admit to being surprised by the surprise engendered by this video.  Anyone who was around during the early days of text-to-speech synthesis is very aware of the danger of presenting the text in advance of or simultaneous with the generated speech.  The intelligibility of the resulting synthesis could be zero without the 'prior' and 100% with the visual cue.

So, given that we know that perception involves the integration of top-down expectations with bottom-up evidence (going right back to Richard Warren's work on the 'phoneme restoration effect'), why is this TikTok demo surprising?  Or maybe I'm missing something?

Best wishes
Roger

--------------------------------------------------------------------------------------------
Prof ROGER K MOORE* BA(Hons) MSc PhD FIOA FISCA MIET

Chair of Spoken Language Processing
Vocal Interactivity Lab (VILab), Sheffield Robotics
Speech & Hearing Research Group (SPandH)
Department of Computer Science, UNIVERSITY OF SHEFFIELD
Regent Court, 211 Portobello, Sheffield, S1 4DP, UK

* Winner of the 2016 Antonio Zampolli Prize for "Outstanding Contributions 
to the Advancement of Language Resources & Language Technology 
Evaluation within Human Language Technologies"

e-mail:  r.k.moore@xxxxxxxxxxxxxxx
web: http://staffwww.dcs.shef.ac.uk/people/R.K.Moore/
twitter: @rogerkmoore
Tel: +44 (0) 11422 21807
Fax: +44 (0) 11422 21810
Mob: +44 (0) 7910 073631

Editor-in-Chief: COMPUTER SPEECH AND LANGUAGE
(http://www.journals.elsevier.com/computer-speech-and-language/)
--------------------------------------------------------------------------------------------


On Fri, 7 Aug 2020 at 05:12, Malcolm Slaney <malcolm@xxxxxxxx> wrote:
Has there been anything formal published on this effect?

It sounds to me like a semantic version of the McGurk effect.

Nice demo.

- Malcolm



--
Julia Strand, PhD
Assistant Professor of Psychology
Carleton College
One North College Street
Northfield, MN 55057
507-222-5637