Re: [AUDITORY] How is the signal of a cochlear implant? [Sound art Project in honor to my deaf sister]

Dear Hugo,

one thing you must appreciate is that, although there are a number of vocoders out there to simulate cochlear implants, aone Alan recommended is perfectly fine, it is nevertheless important to appreciate it is fundamentally impossible to give a true, veridical impression of the sensation cochlear implants create through acoustic stimulation of the normal cochlea. The main reason for this is that the mechanics of the cochlea links temporal stimulation patterns to places of stimulation, and CIs don't do anything like that. Many established CI designs do not pay much attention to the precise temporal patterning of stimuus pulses, so CI users lose important cues for the pitch of complex sounds, for binaural scene analysis and for spatial fearing. What exactly that means cannot be simulated with sound, although "vocoding techniques" give an impression. You may have seen the demo here which I like to use of a Beethoven sonata: http://auditoryneuroscience.com/prosthetics/music If you listen to the original it is very clearly two instruments playing two distinct melodies. The vocoded version sounds much more like a single stream and the melody is much harder to appreciate, but the rhythm is unimpaired.

That demo I made with a bit of simple Matlab code, a bank of bandpass followed by envelope extraction, and then I use the envelope to modulate narrow band noise. Happy to share the code but it is pretty trivial.

Good luck with your public engagement artwork, and all the best to your sister.

Jan

---------------------------------------

Prof Jan Schnupp
City University of Hong Kong
Dept. of Neuroscience

31 To Yuen Street,

Kowloon Tong

Hong Kong

https://auditoryneuroscience.com

http://jan.schnupp.net

On Tue, 9 Mar 2021 at 13:15, Alan Kan <alan.kan@xxxxxxxxx> wrote:

Hi Hugo,

Check out https://cihackathon.com/docs/presentations. It’s a hackathon that just finished but they provide Python code for a vocoder that follows the Advanced Bionics cochlear implant signal processing. All you would need to do is just run your sound files through it.

Cheers

Alan

---
Alan Kan, PhD
Research Fellow

School of Engineering | Macquarie University
Level 1, 50 Waterloo Road, Macquarie Park, NSW 2113, Australia

Australian Hearing Hub

Level 1, 16 University Avenue, Macquarie University, NSW 2109, Australia

T: +61 (2) 9850 2247
E: alan.kan@xxxxxxxxx
W: https://researchers.mq.edu.au/en/persons/alan-kan | mq.edu.au

L: www.linkedin.com/in/alan-kan

CRICOS Provider 00002J. ABN: 90 952 801 237.

This message is intended for the addressee named and may
contain confidential information. If you are not the intended
recipient, please delete the message and notify the sender.
Views expressed in this message are those of the individual
sender and are not necessarily the views of Macquarie
University and its controlled entities.

From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxxxxxxxxx> On Behalf Of Hugo Solís
Sent: Sunday, 7 March 2021 11:55 PM
To: AUDITORY@xxxxxxxxxxxxxxx
Subject: [AUDITORY] How is the signal of a cochlear implant? [Sound art Project in honor to my deaf sister]

Hi everybody,

My names is Hugo and I am a sound artist with some background in
computer science. I have a sister that was born fully deaf and she got a
cochlear implant when she was 40 years old. She is now 48. The cochlear
changed too little in my sister's live and she doesn't describe music as
a pleasant experience.

I want to create a piece of art where hearing people could hear the real
signal that the cochlear implant sends to the brain. I know that the
signal is processed and that pulses are generated on each one of the
electrodes. However I do not nothing about the details of the
transformation.

I am capable of write code in Python (ussing the Essentia Library
(https://essentia.upf.edu/) in order the emulate the transformation to a
signal but I don´t know what is the typical process. I could also write
the code in SuperCollider (https://supercollider.github.io/) but
although it has tons of unit generators it does not have as many
extractors of audio descriptors and common phsyacoustic process as Python.

I am not an audiologist and I have a lack of the signal processing
transformation that happens in a cochlear implant. I do know a lot about
digital signal processing though.

So I need some basics:

1. Code or libraries in any programming code but ideally in Python that
does the emulation. I could write the process but I imagine that many
people has already done this and that there is opensource code already
written.

2. Basic reference about the process that happens in the cochlear device
that could help me to either write the code or tuning the opensource
code in order to make my piece. The work will be shown in a exhibition
and I am running out of time. So any help would be more than appreciate it.

I will be forever thankfull with your support.

Warm regards

Hugo Solís