Re: [AUDITORY] AV discourse material recommendations

Subject: Re: [AUDITORY] AV discourse material recommendations

From: "Reichenbach, Tobias" <tobias.j.reichenbach@xxxxxx>

Date: Fri, 27 Jun 2025 07:53:15 +0000

Accept-language: de-DE, en-US

Approved-by: tobias.j.reichenbach@xxxxxx

Arc-authentication-results: i=1; mx.google.com; dkim=pass header.i=@LISTS.MCGILL.CA header.s=SELECTOR1 header.b=P+97SPgH; spf=pass (google.com: domain of owner-auditory@xxxxxxxxxxxxxxx designates 132.206.27.103 as permitted sender) smtp.mailfrom=owner-auditory@xxxxxxxxxxxxxxx; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=fau.de

Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-archive:list-owner:list-subscribe:list-unsubscribe:list-help :precedence:in-reply-to:to:subject:from:sender:reply-to:date :message-id:mime-version:content-language:accept-language:references :thread-index:thread-topic:approved-by:dkim-signature; bh=CTgA7B/4XwJH4HOpRwizs5LbUQanBFFHQyO+mqbu5Z0=; fh=5/42mu9FVmfuMp6n0xGXVcDar2H3ENcHt8Uv11Om8gY=; b=I6/Alevcfb+ASGiJ1Dk+pq6DHit/ZSPpdRBckH1r7QpPKOU47t3a90S4B3QgjURHbL i/26eAhrhrp0fVTXDpt0wO30iGNeXT1Wl0Y1Bf61/UtTZUZ6uawVcH+1KmhrGbc6Uv3k LGFSETmqOcyHgPgRVflgK8ZQYJrEMrANVrRnKaiBvmEAYRcR/45AmiXyhIueA0W0XOQ4 Bcva94wfx9eEODF85nZg0H6b7RKmJScNkbAzhNRjA1YZNjMxkKXvmfNuxgUcwquSe0Po TeZ6WxL4OWioA/8/TcJFBT7Hby3jUCmbV1OwrHl1oW5P/qUkA66xSHvBoUG8NCKAmKlF QWDA==; dara=google.com

Arc-seal: i=1; a=rsa-sha256; t=1751011502; cv=none; d=google.com; s=arc-20240605; b=KQnnkDRVAhMxVkaPfe2yIegBCsB5YNxB4KQSCzSP8gYf8bJth0JumJunbaQZodrQHI ioz3pCmOt/bCDklsMxcxWZ3mNPMJTf6BPJf8ySCBJC0K66Cxrl9BuIp3thl7oyYKFF7h F5dsmHwyiVSN0Wq65JImNW2O3epm2mz9Ril6UBY0uu6oksAjFCxi/zkVcEQdhaBwYiDx crF34uv16nMeRvEkQnhhPjI14QJ9CfNppSz/3lLpqcKU3bFrgfyIZiAbrqARVFTcsi7m xZ/Ip0yglrKaZ0QIQ2glLK2hDmMfcQTfYmCTUJgqGohIzKDRxJTFM64iqB3j9mEKbRpU 2o7A==

Authentication-results: mx.google.com; dkim=pass header.i=@LISTS.MCGILL.CA header.s=SELECTOR1 header.b=P+97SPgH; spf=pass (google.com: domain of owner-auditory@xxxxxxxxxxxxxxx designates 132.206.27.103 as permitted sender) smtp.mailfrom=owner-auditory@xxxxxxxxxxxxxxx; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=fau.de

Delivered-to: dan.ellis@xxxxxxxxx

Dkim-signature: v=1; a=rsa-sha256; d=LISTS.MCGILL.CA; s=SELECTOR1; c=relaxed/relaxed; bh=CTgA7B/4XwJH4HOpRwizs5LbUQanBFFHQyO+mqbu5Z0=; i=@LISTS.MCGILL.CA; h=Approved-By:Thread-Topic:Thread-Index:References:Accept-Language:Content-Language:Content-Type:MIME-Version:Message-ID:Date:Reply-To:Sender:From:Subject:To:In-Reply-To:List-Help:List-Unsubscribe:List-Subscribe:List-Owner:List-Archive; b=P+97SPgHB/nFiobNhWt1YKnTohIjYpLe1wFPWLV5JFAkGJS25u4J1DyEr+XKPGaj5ub47Uce/NqmxwMPiuIreg9aINV9Hs9StThizmG5vLSB7Ir2DppaK4eimkTOo8SyCmgsFMpMjs7dQzmnCfHjVl4s9e/hLQRypkRwWcOhT6UK3SbLCChbIJE6zrtSYaTXMABUtI2pONlH/ynD1nJFjimmbVqj+j5cvnhmgf38eNprzd+Wl0vfbvdvUccm5XpPp1I/ZvtS4gBbQFNGLdp76zGUjjV0o0/9mujHN5ypJhwGvsGIczEdIUtArzQUwNi48+rqQu4pAVeZbA9gBcPoZw==

In-reply-to: <CH2PR02MB6262D3E42EFC9148A91B0935DC7AA@CH2PR02MB6262.namprd02.prod.outlook.com>

List-archive: <https://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>

List-help: <https://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>, <mailto:LISTSERV@LISTS.MCGILL.CA?body=INFO%20AUDITORY>

List-owner: <mailto:AUDITORY-request@LISTS.MCGILL.CA>

List-subscribe: <mailto:AUDITORY-subscribe-request@LISTS.MCGILL.CA>

List-unsubscribe: <mailto:AUDITORY-unsubscribe-request@LISTS.MCGILL.CA>

References: <CADQXohPjSwM2MyHLunQeKS6ysre-=0t5JGUMWvE84mgka9-h8g@mail.gmail.com> <CH2PR02MB6262D3E42EFC9148A91B0935DC7AA@CH2PR02MB6262.namprd02.prod.outlook.com>

Reply-to: "Reichenbach, Tobias" <tobias.j.reichenbach@xxxxxx>

Sender: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx>

Thread-index: AQHb5lI+mpewVJtUBkaCSoISoIRpcLQVkDewgADyrQA=

Thread-topic: AV discourse material recommendations

Dear Jing,

we recently recorded a corpus of AV speech, called AVBook. The corpus consists of 3.6 hours of audiovisual recordings of two speakers who narrate stories. it is divided into different passages, each of which has a duration of about two minutes. We also provide multiple-choice questions for each passage. Perhaps this might be useful for you! The corpus is available on Zenodo:

https://zenodo.org/records/7387047

We published a description of the corpus in JASA:

3130_1_10.0019460.figures.online.highlight_f1.jpeg

AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception

doi.org

With best regards,

Tobias Reichenbach

Am 26.06.2025 um 17:27 schrieb Sommers, Mitchell <000003f8138a08e4-dmarc-request@xxxxxxxxxxxxxxx>:

Jing…We have materials from an old project on AV discourse comprehension that may be of use. They are 3-5 minutes AV recordings of narratives (taken from the Rutgers oral history project), interviews, and lectures. If these would be useful to you, let me know. Might take a bit to resurrect them, but I should be able to share a google drive with you if that would work.

Best,

Mitch Sommers

Mitchell S. Sommers

Professor of Psychological and Brain Sciences

Washington University of St. Louis

St. Louis, MO 63130

Email: Msommers@xxxxxxxxx

From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> On Behalf Of Jing Shen
Sent: Wednesday, June 25, 2025 1:54 PM
To: AUDITORY@xxxxxxxxxxxxxxx
Subject: AV discourse material recommendations

Dear List,

I am looking for audio-visual speech materials for a new project this year. We need videos of continuous discourse/narrative/story, with the speaker's face centered on the scene. They should be a few minutes long and ideally come with more than 30 videos in the stimuli set. I would greatly appreciate your recommendations!

Best,

Jing

----------------------

Jing Shen, Ph.D.

(she/her pronouns)

Assistant Professor, Department of Communication Sciences and Disorders

Director, Speech Perception and Cognition Lab

College of Public Health, Temple University

Weiss Hall 145, 1701 N. 13^th Street, Philadelphia, PA 19122

215-204-1478

-------------------------------------------

Prof. Dr. Tobias Reichenbach

Chair of Sensory Neurotechnology
Department Artificial Intelligence in Biomedical Engineering (AIBE)
Friedrich-Alexander-University (FAU) Erlangen-Nürnberg
Nürnberger Strasse 74
91052 Erlangen

https://www.neurotech.tf.fau.eu/