Subject: Re: [AUDITORY] Visual references in sound localisation From: Brian FG Katz <katz@xxxxxxxx> Date: Mon, 26 Feb 2018 10:00:00 +0100 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>Regarding audio-visual integration, I would be wary of the measure of = acceptance/annoyance in film production as a negation of our perception = of audio-visual spatial coherence. If you have the opportunity to = compare accurate audio-visual spatial coherence of the actor's voice, or = musical instrument, etc. the difference is flagrant. As a measure of the = "acceptable" spatial disparity, I invite you to look at a study of ours = a few years ago comparing stereo and WFS reproduction and audience = position: Subjective evaluation of the audiovisual spatial congruence in the case = of stereoscopic-3D video and Wave Field Synthesis; = doi:10.1016/j.ijhcs.2013.09.004 For localization tests, personally, I would avoid visual cues/anchors, = unless you present a visual reporting grid that is *finer* than = perceptual blur.=20 Regarding audio influence visual tasks, and vice versa, I point you to a = study we carried out regarding distance perception: Audio, visual, and audio-visual egocentric distance perception by = moving subjects in virtual environments doi:10.1145/2355598.2355602 A second study examined conditions in which comparable saliency = degradations were applied to both audio and visual stimuli, allowing for = an investigation of the audio/visual decision weighting in a search = task.=20 Cueing multimedia search with audio-visual blur; = doi:10.1145/2465780.2465781 Best regards, -- Brian FG Katz, Ph.D, HDR Research Director, CNRS Groupe Lutheries - Acoustique - Musique=20 Sorbonne Universit=C3=A9, CNRS, UMR 7190, Institut Jean Le Rond = =E2=88=82'Alembert bureau 510, 5=C3=A8me, all=C3=A9 des tours 55-65 4, Place Jussieu=20 75005 Paris=20 From: AUDITORY - Research in Auditory Perception = [mailto:AUDITORY@xxxxxxxx On Behalf Of Les Bernstein Sent: Sunday, February 25, 2018 6:52 PM To: AUDITORY@xxxxxxxx Subject: Re: [AUDITORY] Visual references in sound localisation I believe the question was about what would occur under the = circumstances described and the nature of visual influences. It's also = important to recognize that the visual modality is not dominant, per se. = Rather, it depends upon the reliability (indexed by variance) of the = transduced information within each modality. One can manipulate those = variances and humans will generally weight the information within each = modality in a near-optimal fashion (inverse to the variance). Think = Alais and Burr. On 2/25/2018 12:50 AM, Kent Walker wrote: When doing localization tests best practice is to use visually-opaque = acoustically-transparent curtains. However, it's also best practice to = provided respondents with visual references which they can use to = respond. =20 Depending on the perceptual task, providing a reference stimulus with = known location (visual & acoustic) can be extremely useful.=20 In audio engineering, things get more interesting when visual and = auditory cues are in different spatial locations.=20 For example, in film sound mixing dialogue is pretty much always mixed = to the centre channel only, even when actors are visible at the left and = right of the projected image. There are technical limitations that = prevent using phantom sources to match the sound to the viewed location = of the actors. The visual-auditory mismatch is generally not annoying or = troublesome and we perceive the dialogue as eminating from the visual = location on the screen - not the physical location of the loudspeaker. = In large theatres the physical mismatch between the stimuli can be quite = large, routinely 30 feet. This is because in multimodal perception vision generally dominates = (think McGurk effect). On Feb 24, 2018 22:11, "Engel Alonso-Martinez, Isaac" = mailto:isaac.engel@xxxxxxxx wrote: Dear all, I am interested in the impact of audible visual references in sound = localisation tasks. For instance, let's say that you are presented two different continuous = sounds (e.g., speech) coming from sources A and B, which are in = different locations. While source A is clearly visible to you, B is = invisible and you are asked to estimate its location. Will source A act = as a spatial reference, helping you in doing a more accurate estimation, = or will it be distracting and make the task more difficult? If anyone can point to some literature on this, it would be greatly = appreciated. Kind regards, Isaac Engel --=20 Leslie R. Bernstein, Ph.D. | Professor Depts. of Neuroscience and Surgery (Otolaryngology)| UConn School of = Medicine=20 263 Farmington Avenue, Farmington, CT 06030-3401 Office: 860.679.4622 | Fax: 860.679.2495