[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [AUDITORY] Semantic McGurk Effect



It's ever so interesting, and I think we had a round on a similar illusory conjunction before, with the Laurel-Yanny conundrum.

Incidentally, that same year was the "BrainStorm" / "Green Needle" conjunction : https://www.youtube.com/watch?v=1okD66RmktA
In this case you heard what you want to hear! (Don't bet on it as some folk hear neither of it, or are biased to one or the other)

  • I replicated (with due attribution) and extended upon the Current Biology paper on Laurel_yanny by Pressnitzer & Jackson Graves (et al) if anyone is interested in this non-peer reviewed pre-print on SSRN, with interrupted speech perception and correlation with hearing loss. 

Ramakrishnan, Sharath Chandra, Illusory Conjunctions in Speech Perception as a Lens into the Characteristics of Hearing Loss (August 24, 2019). Available at SSRN: https://ssrn.com/abstract=3442202 or http://dx.doi.org/10.2139/ssrn.3442202

It's no where close to a full study and I lost the impetus/encouragement to do a rigorous experiment, but perhaps it is time that I re-gained some interest in this topic and approach again. Happy to hear from interested and prospective collaborators.

Regards
Chandra




On Sat, Aug 8, 2020 at 11:23 PM Nathaniel Zuk <ZUKN@xxxxxx> wrote:
(Today I learned) It's called a mondegreen.

Nate

From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> on behalf of Prof. Roger K. Moore <0000011559506d60-dmarc-request@xxxxxxxxxxxxxxx>
Sent: Friday, August 7, 2020 3:40 PM
To: AUDITORY@xxxxxxxxxxxxxxx <AUDITORY@xxxxxxxxxxxxxxx>
Subject: Re: [AUDITORY] Semantic McGurk Effect
 
Great example Julia - and, of course, comedians such as Bec Hill (https://youtu.be/nEdR44Iftb4) and Peter Kay (https://youtu.be/7my5baoCVv8) trade on this effect - R



On Fri, 7 Aug 2020 at 14:42, Julia Strand <jstrand@xxxxxxxxxxxx> wrote:
I'm always delighted when auditory phenomena spark the public's interest! 

I wouldn't call this a semantic McGurk, given that it doesn't have to be driven by simultaneous bottom-up input from two modalities. That is, even if nothing is written on the screen but you're just thinking "green needle" to yourself, that's what you're likely to hear (whereas thinking "ga" while hearing "ba" won't get you to "da" - you need the simultaneous input from face and voice). So I'd agree with Roger that it's more akin to the phoneme restoration effect or work like Cynthia Connine's "she ran hot water for the p/bath," showing how expectations influence interpretation of bottom-up input.

I think most of US wouldn't be surprised that the same stimulus can be perceived in different ways, but my impression is that the general public tends to believe "what you see is what you get" and underestimates the power of top-down influences. Same reason #TheDress was such a hit. 

When I include this in my class on speech perception, I also include this video which shows Grover from Sesame street saying EITHER "Yes, yes, that sounds like an excellent idea!" OR "Yes, yes, that's a f*%#g excellent idea!"

Like I'm always telling my students - Speech is hard! Context helps!

Best,
Julia

On Fri, Aug 7, 2020 at 4:28 AM Prof. Roger K. Moore <0000011559506d60-dmarc-request@xxxxxxxxxxxxxxx> wrote:
I must admit to being surprised by the surprise engendered by this video.  Anyone who was around during the early days of text-to-speech synthesis is very aware of the danger of presenting the text in advance of or simultaneous with the generated speech.  The intelligibility of the resulting synthesis could be zero without the 'prior' and 100% with the visual cue.

So, given that we know that perception involves the integration of top-down expectations with bottom-up evidence (going right back to Richard Warren's work on the 'phoneme restoration effect'), why is this TikTok demo surprising?  Or maybe I'm missing something?

Best wishes
Roger

--------------------------------------------------------------------------------------------
Prof ROGER K MOORE* BA(Hons) MSc PhD FIOA FISCA MIET

Chair of Spoken Language Processing
Vocal Interactivity Lab (VILab), Sheffield Robotics
Speech & Hearing Research Group (SPandH)
Department of Computer Science, UNIVERSITY OF SHEFFIELD
Regent Court, 211 Portobello, Sheffield, S1 4DP, UK

* Winner of the 2016 Antonio Zampolli Prize for "Outstanding Contributions 
to the Advancement of Language Resources & Language Technology 
Evaluation within Human Language Technologies"

e-mail:  r.k.moore@xxxxxxxxxxxxxxx
web: http://staffwww.dcs.shef.ac.uk/people/R.K.Moore/
twitter: @rogerkmoore
Tel: +44 (0) 11422 21807
Fax: +44 (0) 11422 21810
Mob: +44 (0) 7910 073631

Editor-in-Chief: COMPUTER SPEECH AND LANGUAGE
(http://www.journals.elsevier.com/computer-speech-and-language/)
--------------------------------------------------------------------------------------------


On Fri, 7 Aug 2020 at 05:12, Malcolm Slaney <malcolm@xxxxxxxx> wrote:
Has there been anything formal published on this effect?

It sounds to me like a semantic version of the McGurk effect.

Nice demo.

- Malcolm



--
Julia Strand, PhD
Assistant Professor of Psychology
Carleton College
One North College Street
Northfield, MN 55057
507-222-5637


--
Sharath Chandra Ram

PhD Fellow @https://atec.utdallas.edu/

Machine Listening & Computational Media Art

Applied Cognition in HCI & Virtual Environments

Information, Media & Technology Policy

Alumni@Edinburgh School of Informatics
Signal Lab : radiospock.tumblr.com
Twitter @AgentSpock