Re: [AUDITORY] AW: How is the signal of a cochlear implant? [Sound art Project in honor to my deaf sister] (Jeremy Marozeau )

Subject: Re: [AUDITORY] AW: How is the signal of a cochlear implant? [Sound art Project in honor to my deaf sister] From: Jeremy Marozeau <000000aae0ca5ce5-dmarc-request@xxxxxxxx> Date: Sat, 20 Mar 2021 19:13:18 +0000 --_004_7b5e25c2626147d096361672ff87ca67emailandroidcom_ Content-Type: multipart/alternative; boundary="_000_7b5e25c2626147d096361672ff87ca67emailandroidcom_" --_000_7b5e25c2626147d096361672ff87ca67emailandroidcom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Dear Hugo, Niki, and everyone, I agree with Niki that music's perception in CI users is a complex topic, a= nd listening to music through a vocoder might be misleading. First, the vocoder should not be considered a tool to simulate the sound pe= rceived through a CI but to simulate speech score. In other words, when a n= ormal hearing person (NH) listens to a vocoded speech, we cannot assume tha= t he/she will have the same percept as a CI user, but just that she/he will= understand the same amount of words. For a given situation, if an NH under= stands 100% of a sentence and a CI user 50%, the same NH will also understa= nd about 50% of a vocoded speech. It is very difficult for a CI user to describe how they perceive sounds, as= we lack vocabulary. Just for NH, for CI users, the sound of a bird sounds = like a "bird singing." The only way is to ask CI users, who have enough residual hearing on one ea= r, to compare the same sound presented into the two ears. You can find some= studies that will do just that: Lazard et al., (2012), https://journals.plos.org/plosone/article?id=3D10.13= 71/journal.pone.0038687 Adel et al. (2019) https://www.frontiersin.org/articles/10.3389/fnins.2019.= 01119/full Marozeau et al. (2020) https://journals.plos.org/plosone/article?id=3D10.13= 71/journal.pone.0235504 Dorman et al. (2020) https://journals.sagepub.com/doi/full/10.1177/23312165= 20920079 <https://journals.sagepub.com/doi/full/10.1177/2331216520920079> Based on those studies' results, it seems that there are as many answers as= CI users in the world. Some of them will claim that a sound through a CI i= s exactly as in the normal ear, some as white noise, and some as inharmonic= sounds. Now to answer your question, we do not know how CI users will perceive musi= c because they will all perceive it differently. However, we know that the= sound processor will not send enough information to convey pitch cues prop= erly (see here<http://www.acoustics.asn.au/journal/Vol42No2-HiRes.pdf>). Al= though there are some star performers (see Maarefvand, 2013<https://www.res= earchgate.net/publication/236062277_A_cochlear_implant_user_with_exceptiona= l_musical_hearing_ability>), it is pretty safe to assume that most CI users= will not perceive the melody. Nevertheless, as Niki mentioned, many CI use= rs appreciate music and are engaged in musical activities. They can probabl= y focus on different musical cues such as rhythm and dynamic. Similarly, ma= ny NH people can appreciate music without a clear tonal structure and defin= ed melodies. To be provocative, I will propose that as the vocoder is a good model for s= peech understanding, some contemporary music (like Boulez) can be a good mo= del of how CI users can experience (not perceive !) music. And as for the m= usic composed by Boulez, some people love it, and many people hate it. To support my point, we have made a study in which we have asked NH and CIL= to rate the musical tension of a piano piece of Mozart (Spangmose, 2019, w= ww.frontiersin.org/articles/10.3389/fnins.2019.00987/full<http://www.fronti= ersin.org/articles/10.3389/fnins.2019.00987/full>.) Surprisingly, CIL and N= HL rated overall musical tension in a very similar way. Then, we have repea= ted the task but on a modified version of the piano, in which all the notes= were shuffled. Removing the melody had an important effect on NH's musical= judgment, but none for the CIL. Furthermore, CIL reports appreciating the = piece of music with original notes or with the random one similarly. In summary, for your project, you should look into atonal or purely rhythmi= cal music. Good luck, Jeremy Jeremy Marozeau Associate Professor DTU Health Tech Technical University of Denmark Department of Health Technology =D8rsteds Plads Building 352, Room 124 2800 Kgs. Lyngby Direct +45 45254790 jemaroz@xxxxxxxx<mailto:jemaroz@xxxxxxxx> On Mar 20, 2021 5:15 AM, "Vavatzanidis, Niki" <Niki.Vavatzanidis@xxxxxxxx= M-DRESDEN.DE> wrote: Dear Hugo and all, as a cognitive neuroscientist I=92d like to add: don=92t forget the brain! = Jan made some very valid points about how much we can infer from vocoders f= rom a technical point of view. Complicating things further is the fact that= no brain will interpret the sound in the same fashion. How much hearing ex= perience you have had before you get a CI is crucial for what you can extra= ct from the CI signal. A person whose brain experienced decades of hearing = and only a relatively short period of deafness before getting the implant w= ill extract much more from the CI signal than someone whose brain has never= learned to decode (audio) speech and gets an implant late in life like you= r sister. Speech discrimination may come almost effortlessly for some in th= e first case, while it is out of reach for almost everybody in the latter c= ase. The CI might still be useful because it informs you about environmenta= l sounds (your child is crying in the next room, someone is addressing you = from behind etc) but understanding speech without lip-reading is no hope on= e should pose on cochlear implantation if born deaf and not getting the imp= lant early in life. With music, it will be similar in certain aspects. The interviews of CI use= rs Kathy and Angela made for the CI hackathon that Alan send around (https:= //cihackathon.com/docs/CI_interviews) describe very nicely, I think, how th= ey are able to fill in missing information for songs they know from before = their hearing loss (and which they are able to enjoy) and how this does not= work for new pieces of music for which they do not have a =93pre-CI=94 mem= ory. On the other hand, music enjoyment has so much to do with your own exp= ectations that in one of our studies we found that those who have never exp= erienced music bore they got the CI actually tend to enjoy it much more tha= n those who can compare it to =93how it used to sound=94 before their heari= ng loss and who are disappointed by how different the music sounds with the= CI ( Hahne et al., 2020, doi: 10.1055/s-0040-1711102). This is just to give you an idea of how diverse the experience of one and t= he same CI output may be depending on your individual history and how it ha= s shaped your brain. Of course, there are more factors that shape what you = hear with the CI (many related to the individual brain, others linked to th= e technology itself), but one=92s hearing history is a very fundamental one= . I for one would be very interested to hear your CI art project, maybe you c= ould point me/us towards it when the time comes? That would be great! All the best Niki *********************************** Dr. rer. nat. Niki K. Vavatzanidis (she/her) Saxonian Cochlear Implant Center Dresden University Hospital Dresden Fetscherstr. 74 01307 Dresden Germany niki.vavatzanidis(at)ukdd.de https://www.uniklinikum-dresden.de/scic/research Von: Jan Schnupp <jan.schnupp@xxxxxxxx> Gesendet: Dienstag, 9. M=E4rz 2021 11:22 Betreff: Re: How is the signal of a cochlear implant? [Sound art Project in= honor to my deaf sister] Dear Hugo, one thing you must appreciate is that, although there are a number of vocod= ers out there to simulate cochlear implants, aone Alan recommended is perfe= ctly fine, it is nevertheless important to appreciate it is fundamentally i= mpossible to give a true, veridical impression of the sensation cochlear im= plants create through acoustic stimulation of the normal cochlea. The main = reason for this is that the mechanics of the cochlea links temporal stimula= tion patterns to places of stimulation, and CIs don't do anything like that= . Many established CI designs do not pay much attention to the precise temp= oral patterning of stimuus pulses, so CI users lose important cues for the = pitch of complex sounds, for binaural scene analysis and for spatial fearin= g. What exactly that means cannot be simulated with sound, although "vocodi= ng techniques" give an impression. You may have seen the demo here which I = like to use of a Beethoven sonata: http://auditoryneuroscience.com/prosthet= ics/music If you listen to the original it is very clearly two instrument= s playing two distinct melodies. The vocoded version sounds much more like = a single stream and the melody is much harder to appreciate, but the rhythm= is unimpaired. That demo I made with a bit of simple Matlab code, a bank of bandpass follo= wed by envelope extraction, and then I use the envelope to modulate narrow = band noise. Happy to share the code but it is pretty trivial. Good luck with your public engagement artwork, and all the best to your si= ster. Jan --------------------------------------- Prof Jan Schnupp City University of Hong Kong Dept. of Neuroscience 31 To Yuen Street, Kowloon Tong Hong Kong https://auditoryneuroscience.com http://jan.schnupp.net On Tue, 9 Mar 2021 at 13:15, Alan Kan <alan.kan@xxxxxxxx<mailto:alan.kan@xxxxxxxx= q.edu.au>> wrote: Hi Hugo, Check out https://cihackathon.com/docs/presentations. It=92s a hackathon th= at just finished but they provide Python code for a vocoder that follows th= e Advanced Bionics cochlear implant signal processing. All you would need t= o do is just run your sound files through it. Cheers Alan --- Alan Kan, PhD Research Fellow School of Engineering | Macquarie University Level 1, 50 Waterloo Road, Macquarie Park, NSW 2113, Australia Australian Hearing Hub Level 1, 16 University Avenue, Macquarie University, NSW 2109, Australia T: +61 (2) 9850 2247 E: alan.kan@xxxxxxxx<mailto:alan.kan@xxxxxxxx> W: https://researchers.mq.edu.au/en/persons/alan-kan | mq.edu.au<http://mq= .edu.au/> L: www.linkedin.com/in/alan-kan<http://www.linkedin.com/in/alan-kan> [Macquarie University]<http://mq.edu.au/> CRICOS Provider 00002J. ABN: 90 952 801 237. This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete the message and notify the sender. Views expressed in this message are those of the individual sender and are not necessarily the views of Macquarie University and its controlled entities. From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxx<= mailto:AUDITORY@xxxxxxxx>> On Behalf Of Hugo Sol=EDs Sent: Sunday, 7 March 2021 11:55 PM To: AUDITORY@xxxxxxxx<mailto:AUDITORY@xxxxxxxx> Subject: [AUDITORY] How is the signal of a cochlear implant? [Sound art Pro= ject in honor to my deaf sister] Hi everybody, My names is Hugo and I am a sound artist with some background in computer science. I have a sister that was born fully deaf and she got a cochlear implant when she was 40 years old. She is now 48. The cochlear changed too little in my sister's live and she doesn't describe music as a pleasant experience. I want to create a piece of art where hearing people could hear the real signal that the cochlear implant sends to the brain. I know that the signal is processed and that pulses are generated on each one of the electrodes. However I do not nothing about the details of the transformation. I am capable of write code in Python (ussing the Essentia Library (https://essentia.upf.edu/<https://essentia.upf.edu>) in order the emulate = the transformation to a signal but I don=B4t know what is the typical process. I could also write the code in SuperCollider (https://supercollider.github.io/<https://superco= llider.github.io>) but although it has tons of unit generators it does not have as many extractors of audio descriptors and common phsyacoustic process as Python. I am not an audiologist and I have a lack of the signal processing transformation that happens in a cochlear implant. I do know a lot about digital signal processing though. So I need some basics: 1. Code or libraries in any programming code but ideally in Python that does the emulation. I could write the process but I imagine that many people has already done this and that there is opensource code already written. 2. Basic reference about the process that happens in the cochlear device that could help me to either write the code or tuning the opensource code in order to make my piece. The work will be shown in a exhibition and I am running out of time. So any help would be more than appreciate it. I will be forever thankfull with your support. Warm regards Hugo Sol=EDs --_000_7b5e25c2626147d096361672ff87ca67emailandroidcom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1= 252"> <meta content=3D"text/html; charset=3Dutf-8"> <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style>  </style> </head> <body lang=3D"DE" link=3D"blue" vlink=3D"purple"> <div dir=3D"auto"> <div dir=3D"ltr" style=3D"font-size:12pt; font-family:calibri,helvetica,san= s-serif"> <div> Dear Hugo, Niki, and everyone, I agree with Niki that music's perception in CI users is a complex topic= , and listening to music through a vocoder might be misleading. First, the vocoder should not be considered a tool to simulate the sound= perceived through a CI but to simulate speech score. In other words, when = a normal hearing person (NH) listens to a vocoded speech, we cannot assume = that he/she will have the same percept as a CI user, but just that she/he will understand the same amount of word= s. For a given situation, if an NH understands 100% of a sentence and a CI = user 50%, the same NH will also understand about 50% of a vocoded speech.<b= r> It is very difficult for a CI user to describe how they perceive sounds,= as we lack vocabulary. Just for NH, for CI users, the sound of a bird soun= ds like a "bird singing." The only way is to ask CI users, who have enough residual hearing on one= ear, to compare the same sound presented into the two ears. You can find s= ome studies that will do just that: Lazard et al., (2012), <a href=3D"https://journals.plos.org/plosone= /article?id=3D10.1371/journal.pone.0038687" style=3D"color:rgb(66,133,244)"= >https://journals.plos.org/plosone/article?id=3D10.1371/journal.pone.003868= 7</a> Adel et al. (2019) https://www.frontiersin.org/articles/10.3389/fnins.20= 19.01119/full Marozeau et al. (2020) <a href=3D"https://journals.plos.org/plosone= /article?id=3D10.1371/journal.pone.0235504" style=3D"color:rgb(66,133,244)"= >https://journals.plos.org/plosone/article?id=3D10.1371/journal.pone.023550= 4</a> Dorman et al. (2020) <a href=3D"https://journals.sagepub.com/doi/fu= ll/10.1177/2331216520920079" style=3D"color:rgb(66,133,244)">https://journa= ls.sagepub.com/doi/full/10.1177/2331216520920079</a> <a href=3D"https://journals.sagepub.com/doi/full/10.1177/233121652092007= 9" style=3D"color:rgb(66,133,244)"> </a> Based on those studies' results, it seems that there are as many answers= as CI users in the world. Some of them will claim that a sound through a C= I is exactly as in the normal ear, some as white noise, and some as inharmo= nic sounds. Now to answer your question, we do not know how CI users will perceive m= usic because they will all perceive it differently.  However, we know = that the sound processor will not send enough information to convey pitch c= ues properly (see <a href=3D"http://www.acoustics.asn.au/journal/Vol42= No2-HiRes.pdf" style=3D"color:rgb(66,133,244)">here</a>). Although there are some star performers (see <a href=3D"https://www.r= esearchgate.net/publication/236062277_A_cochlear_implant_user_with_exceptio= nal_musical_hearing_ability" style=3D"color:rgb(66,133,244)">Maarefvand, 20= 13</a>), it is pretty safe to assume that most CI users will not perceive the melody. Nevertheless, as Niki mentione= d, many CI users appreciate music and are engaged in musical activities. Th= ey can probably focus on different musical cues such as rhythm and dynamic.= Similarly, many NH people can appreciate music without a clear tonal structure and defined melodies. To be provocative, I will propose that as the vocoder is a good model fo= r speech understanding, some contemporary music (like Boulez) can be a good= model of how CI users can experience (not perceive !) music. And as for th= e music composed by Boulez, some people love it, and many people hate it. To support my point, we have made a study in which we have asked NH and = CIL to rate the musical tension of a piano piece of Mozart (Spangmose, 2019= , <a href=3D"http://www.frontiersin.org/articles/10.3389/fnins.2019.00= 987/full" style=3D"color:rgb(66,133,244)">www.frontiersin.org/articles/10.3= 389/fnins.2019.00987/full</a>.) Surprisingly, CIL and NHL rated overall musical tension in a very similar = way. Then, we have repeated the task but on a modified version of the piano= , in which all the notes were shuffled. Removing the melody had an importan= t effect on NH's musical judgment, but none for the CIL. Furthermore, CIL reports appreciating the piece of m= usic with original notes or with the random one similarly. In summary, for your project, you should look into atonal or purely rhyt= hmical music. Good luck, Jeremy </div> <table cellspacing=3D"0" cellpadding=3D"0" width=3D"400px" border=3D"0" sty= le=3D"width:auto!important; table-layout:auto!important; font-family:'arial= '; font-size:8pt; border-collapse:collapse"> <tbody> <tr> <td colspan=3D"2" width=3D"400px" valign=3D"top" style=3D"font-weight:bold;= font-size:8pt; margin-bottom:0px"> Jeremy Marozeau</td> </tr> <tr> <td colspan=3D"2" width=3D"400px" valign=3D"top" style=3D"color:rgb(155 ,15= 5 ,155); font-size:8pt; margin-bottom:0px"> Associate Professor</td> </tr> <tr> <td colspan=3D"2" width=3D"400px" valign=3D"top" style=3D"font-size:8pt; ma= rgin-bottom:0px"> </td> </tr> <tr> <td colspan=3D"2" width=3D"400px" valign=3D"top" style=3D"font-size:8pt; ma= rgin-bottom:0px"> DTU Health Tech</td> </tr> <tr> <td colspan=3D"2" width=3D"400px" valign=3D"top" style=3D"font-size:8pt; li= ne-height:0px; border-bottom:1px solid rgb(174 ,174 ,174); padding-bottom:0= px; padding-top:0px; margin-top:0px">  </td> </tr> <tr> <td width=3D"350px" valign=3D"top" style=3D"padding-top:8px; vertical-align= :top"> <table cellspacing=3D"0" cellpadding=3D"0" width=3D"350px" border=3D"0" sty= le=3D"width:auto!important; table-layout:auto!important; font-size:8pt; bor= der-collapse:collapse"> <tbody> <tr> <td style=3D"width:350px; font-size:8pt; margin:0px; font-weight:bold">Tech= nical University of Denmark</td> </tr> <tr> <td style=3D"font-size:8pt; margin:0px">Department of Health Technology</td= > </tr> <tr> <td style=3D"font-size:8pt; margin:0px">=D8rsteds Plads</td> </tr> <tr> <td style=3D"font-size:8pt; margin:0px">Building 352, Room 124</td> </tr> <tr> <td style=3D"font-size:8pt; margin:0px">2800 Kgs. Lyngby</td> </tr> <tr> <td style=3D"font-size:8pt; margin:0px">Direct +45 45254790</td> </tr> <tr style=3D"font-size:8pt; margin:0px"> <td><a href=3D"mailto:jemaroz@xxxxxxxx" style=3D"color:rgb(66,133,244)">jemar= oz@xxxxxxxx</a></td> </tr> </tbody> </table> </td> </tr> </tbody> </table> </div> </div> <div class=3D"gmail_extra"> <div class=3D"gmail_quote">On Mar 20, 2021 5:15 AM, "Vavatzanidis, Nik= i" <Niki.Vavatzanidis@xxxxxxxx> wrote: </div> </div> <div> <div class=3D"WordSection1"> Dear Hugo and all,</= span>   as a cognitive neuro= scientist I=92d like to add: don=92t forget the brain! Jan made some very v= alid points about how much we can infer from vocoders from a technical point of view. Complicating things further is the fact th= at no brain will interpret the sound in the same fashion. How much hearing = experience you have had before you get a CI is crucial for what you can ext= ract from the CI signal. A person whose brain experienced decades of hearing and only a relatively short per= iod of deafness before getting the implant will extract much more from the = CI signal than someone whose brain has never learned to decode (audio) spee= ch and gets an implant late in life like your sister. Speech discrimination may come almost effortlessly for s= ome in the first case, while it is out of reach for almost everybody in the= latter case. The CI might still be useful because it informs you about env= ironmental sounds (your child is crying in the next room, someone is addressing you from behind etc) but un= derstanding speech without lip-reading is no hope one should pose on cochle= ar implantation if born deaf and not getting the implant early in life. &nb= sp; With music, it will = be similar in certain aspects. The interviews of CI users Kathy and Angela = made for the CI hackathon that Alan send around (<a href=3D"https://cihackathon.com/docs/CI_interviews">https://cihackatho= n.com/docs/CI_interviews</a>) describe very nicely, I think, how they are a= ble to fill in missing information for songs they know from before their he= aring loss (and which they are able to enjoy) and how this does not work for new pieces of music for which the= y do not have a =93pre-CI=94 memory. On the other hand, music enjoyment has= so much to do with your own expectations that in one of our studies we fou= nd that those who have never experienced music bore they got the CI actually tend to enjoy it much more than those = who can compare it to =93how it used to sound=94 before their hearing loss = and who are disappointed by how different the music sounds with the CI ( Ha= hne et al., 2020, doi: 10.1055/s-0040-1711102).   This is just to give= you an idea of how diverse the experience of one and the same CI output ma= y be depending on your individual history and how it has shaped your brain. Of course, there are more factors that shape wha= t you hear with the CI (many related to the individual brain, others linked= to the technology itself), but one=92s hearing history is a very fundament= al one. I for one would be v= ery interested to hear your CI art project, maybe you could point me/us tow= ards it when the time comes? That would be great!   All the best<= /p> Niki     ********************= *************** Dr. rer. nat. Niki K= . Vavatzanidis (she/her) Saxonian Cochlear Im= plant Center Dresden University Hospital = Dresden Fetscherstr. 74 01307 Dresden Germany   niki.vavatzanidis(at)ukdd.de= <a href=3D"https://www.uniklinikum-= dresden.de/scic/research">https://www.uniklinikum-dresden.de/scic/research<= /a>       Von: Jan Schnupp <jan.schnupp@xxxxxxxx= OOGLEMAIL.COM> Gesendet: Dienstag, 9. M=E4rz 2021 11:22 Betreff: Re: How is the signal of a cochlear implant? [Sound art Pro= ject in honor to my deaf sister]   <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> Dear Hugo,</p= > </div> <div>   </div> <div> one thing you must a= ppreciate is that, although there are a number of vocoders out there to sim= ulate cochlear implants, aone Alan recommended is perfectly fine, it is nev= ertheless important to appreciate it is fundamentally impossible to give a true, veridical impression of the se= nsation cochlear implants create through acoustic stimulation of the normal= cochlea. The main reason for this is that the mechanics of the cochlea lin= ks temporal stimulation patterns to places of stimulation, and CIs don't do anything like that. Many establ= ished CI designs do not pay much attention to the precise temporal patterni= ng of stimuus pulses, so CI users lose important cues for the pitch of comp= lex sounds, for binaural scene analysis and for spatial fearing. What exactly that means cannot be simulated with = sound, although "vocoding techniques" give an impression. You may= have seen the demo here which I like to use of a Beethoven sonata: <a= href=3D"http://auditoryneuroscience.com/prosthetics/music">http://auditory= neuroscience.com/prosthetics/music</a>   If you listen to the original it is very clearly two instruments pla= ying two distinct melodies. The vocoded version sounds much more like a sin= gle stream and the melody is much harder to appreciate, but the rhythm is u= nimpaired.  </div> <div> That demo I made wit= h a bit of simple Matlab code, a bank of bandpass followed by envelope extr= action, and then I use the envelope to modulate narrow band noise. Happy to= share the code but it is pretty trivial. </div> <div>   </div> <div> Good luck with your = public engagement  artwork, and all the best to your sister.</p= > </div> <div>   </div> <div> Jan </div> <div> --------------------= ------------------- </div> <div> Prof Jan Schnupp City University of Hong Kong Dept. of Neuroscience </div> <div> <div> 31 To Yuen Street,&n= bsp; </div> <div> Kowloon Tong<= /p> </div> <div> Hong Kong </div> <div>   </div> <a href=3D"https://auditoryneuroscience.com" target= =3D"_blank">https://auditoryneuroscience.com</a> </div> <div> <a href=3D"http://jan.schnupp.net" target=3D"_blank"= >http://jan.schnupp.net</a> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div>   </div>   <div> <div> On Tue, 9 Mar 2021 at 13:15, Alan Kan <<a href=3D= "mailto:alan.kan@xxxxxxxx">alan.kan@xxxxxxxx</a>> wrote: </div> <blockquote style=3D"border:none; border-left:solid #CCCCCC 1.0pt; padding:= 0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-right:0cm"> <div> <div> Hi Hugo,   Check out <a href=3D= "https://cihackathon.com/docs/presentations" target=3D"_blank"> https://cihackathon.com/docs/presentations</a>. It=92s a hackathon that jus= t finished but they provide Python code for a vocoder that follows the Adva= nced Bionics cochlear implant signal processing. All you would need to do i= s just run your sound files through it.   Cheers   Alan   --- <br= > Alan Kan, PhD Research Fellow  = School of Engineering | Macquarie University</b= > Level 1, 50 Waterloo Road, Macquarie Park, NSW&= nbsp;2113, Australia A= ustralian Hearing Hub Level 1, 16 University Avenue, Macquarie University, NS= W 2109, Australia   T:=  +61 (2) 9850 2247 <b= r> E:  <a href=3D"mailto:alan.kan@xxxxxxxx" target=3D"_blank">alan.kan@xxxxxxxx<= /a>  W: = <a href=3D"https://researchers.mq.edu.au/en/persons/alan-kan" target=3D"_bl= ank">https://researchers.mq.edu.au/en/persons/alan-kan</a> <= span lang=3D"EN-AU" style=3D"color:#DD1911; border:none windowtext 1.0pt; p= adding:0cm">|  <a href=3D"http://mq.edu.au/" target=3D"_blank= " title=3D"Macquarie University">mq.edu.au</a> L= : <a href=3D"http://www.linkedin.com/in/alan-kan" target=3D"_blank">www.link= edin.com/in/alan-kan</a> &nbs= p; <a href=3D"http://mq= .edu.au/" target=3D"_blank"><img border=3D"0" wi= dth=3D"260" height=3D"58" id=3D"gmail-m_-212532390942490478Picture_x0020_1"= src=3D"cid:image001.png@xxxxxxxx" alt=3D"Macquarie University" st= yle=3D"width:2.7083in; height:.6041in"></a> CRI= COS Provider 00002J. ABN: 90 952 801 237.</= span> This message is intende= d for the addressee named and may contain confidential information. If you are not the intended recipient, please delete the message and notify the sender. Views expressed in this message are those of the individual sender and are not necessarily the views of Macquarie University and its controlled entities.</sp= an>         <div> <div style=3D"border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0c= m 0cm 0cm"> From:<= span lang=3D"EN-US"> AUDITORY - Research in Auditory Perception <<a href= =3D"mailto:AUDITORY@xxxxxxxx" target=3D"_blank">AUDITORY@xxxxxxxx= L.CA</a>> On Behalf Of Hugo Sol=EDs Sent: Sunday, 7 March 2021 11:55 PM To: <a href=3D"mailto:AUDITORY@xxxxxxxx" target=3D"_blank">AU= DITORY@xxxxxxxx</a> Subject: [AUDITORY] How is the signal of a cochlear implant? [Sound = art Project in honor to my deaf sister]<= /p> </div> </div>   Hi everybody, My names is Hugo and I am a sound artist with some background in computer science. I have a sister that was born fully deaf and she got a <b= r> cochlear implant when she was 40 years old. She is now 48. The cochlear <br= > changed too little in my sister's live and she doesn't describe music as <b= r> a pleasant experience. I want to create a piece of art where hearing people could hear the real <b= r> signal that the cochlear implant sends to the brain. I know that the signal is processed and that pulses are generated on each one of the electrodes. However I do not nothing about the details of the transformation. I am capable of write code in Python (ussing the Essentia Library (<a href=3D"https://essentia.upf.edu" target=3D"_blank">https://essentia.up= f.edu/</a>) in order the emulate the transformation to a signal but I don=B4t know what is the typical process. I could also write <= br> the code in SuperCollider (<a href=3D"https://supercollider.github.io" targ= et=3D"_blank">https://supercollider.github.io/</a>) but although it has tons of unit generators it does not have as many extractors of audio descriptors and common phsyacoustic process as Python.<= br> I am not an audiologist and I have a lack of the signal processing transformation that happens in a cochlear implant. I do know a lot about <b= r> digital signal processing though. So I need some basics: 1. Code or libraries in any programming code but ideally in Python that <br= > does the emulation. I could write the process but I imagine that many people has already done this and that there is opensource code already written. 2. Basic reference about the process that happens in the cochlear device <b= r> that could help me to either write the code or tuning the opensource code in order to make my piece. The work will be shown in a exhibition and I am running out of time. So any help would be more than appreciate it.= I will be forever thankfull with your support. Warm regards Hugo Sol=EDs </div> </div> </blockquote> </div> </div> </div> </body> </html> --_000_7b5e25c2626147d096361672ff87ca67emailandroidcom_-- --_004_7b5e25c2626147d096361672ff87ca67emailandroidcom_ Content-Type: image/png; name="image001.png" Content-Description: image001.png Content-Disposition: inline; filename="image001.png"; size=4605; creation-date="Fri, 19 Mar 2021 11:20:22 GMT"; modification-date="Fri, 19 Mar 2021 11:20:22 GMT" Content-ID: <image001.png@xxxxxxxx> Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAQQAAAA6CAMAAAEW1ufHAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJ bWFnZVJlYWR5ccllPAAAAyhpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdp bj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6 eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuNS1jMDIxIDc5LjE1 NTc3MiwgMjAxNC8wMS8xMy0xOTo0NDowMCAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJo dHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlw dGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAv IiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RS ZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtcDpD cmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENDIDIwMTQgKE1hY2ludG9zaCkiIHhtcE1NOklu c3RhbmNlSUQ9InhtcC5paWQ6NkQwQTU5ODM3OEVCMTFFNEJBMzFFQzBCODNGMzdEOTIiIHhtcE1N OkRvY3VtZW50SUQ9InhtcC5kaWQ6NkQwQTU5ODQ3OEVCMTFFNEJBMzFFQzBCODNGMzdEOTIiPiA8 eG1wTU06RGVyaXZlZEZyb20gc3RSZWY6aW5zdGFuY2VJRD0ieG1wLmlpZDo2RDBBNTk4MTc4RUIx MUU0QkEzMUVDMEI4M0YzN0Q5MiIgc3RSZWY6ZG9jdW1lbnRJRD0ieG1wLmRpZDo2RDBBNTk4Mjc4 RUIxMUU0QkEzMUVDMEI4M0YzN0Q5MiIvPiA8L3JkZjpEZXNjcmlwdGlvbj4gPC9yZGY6UkRGPiA8 L3g6eG1wbWV0YT4gPD94cGFja2V0IGVuZD0iciI/PhxRxJUAAAMAUExURYtGTbKtoc3HuWxpYtqs rPPHy+7l5Kmkmf///+3s6ui2tnVyatNGVeNVaIc5Q/PW19aCin56cs8rPvb188+Sl765rNLNvq+r n0xKRdXDvOOZl9QlOomFfONwe/Ho6YElMs7Ju8nEtsbBtNhXZIB9dOnczQ0MC+LczcO+saailtQC HcK8vMqprc2wskRCPXYiLoWCeY6Lgc7OzPzz88gVLNeztd/ZyrWwpKyHibq2qXkmMql2eraJjZBQ WDs5NZZBSdnUxXh1bbwqO+ammeTfzxwcGuTj4nwrNtUMJdUdM9rZ1ZdcYrU4ReLTx9TPwNLS0rOF i4I0PlRSTBQUEiYlIpEsOeDayysqJzMyLpKOhJiUitbQwqRBS+3p3cY1RNUXLriLibWcnOPU1D49 OHFuZs8HIqyonNo4SUhHQldVT5aTiFZUTrizprMmNPz8+51han4vOdJibn8wO8K9r/r6+gQDA/Lu 5NUxRcnIxlpYUXkhLrM2ROvg3y4tKr6vrtzXyL87SUE/O8LCwbuRl+ygplNQS5WRh2RhWzY1MWBe V8PCwl9dVjAuK7GAh7k4RhoZF6c1Qjk3NIOAd3EPIL6Xk8HBwaGckebXyeHGu+jLvN4cFN4eFuac j+ni0t0aEmZjXOPYyr43P8NbYOnWyOji0r9QV+nf0Ofh0tDLvenj0+HbzF1aVNjSw+bg0QcHB9nT xOXg0GdkXaOflBAPDqKdkq87R09NRyMiIB8eHKKek/39/evl1puXjXckMBgXFmJgWdK2ud/LzunS 1efHuJ5KUa6Ym3p3b79KUejMwdQuQurg0Pj39efc2/K0vPGvt/Pv7p8yP8enpOvq6OiHkd7c2dF0 fdvGxs+Ij82JkLuVkOLg3b03RujRxcKanMAkN7tZYEZEP+F/gtjW08BPWcFXXenPv/Pg4aEzP9XT 0O+fquKin9e+wc6yqcXFxf77+r6Yk6qMjygnJMLBwYdASOTazM/Fw2hlXszGudW8voyIf9UAHN4a EnANH60QIQAAAOrk1MotrZkAAAtfSURBVHjaYvhPCDAQo0L/l/4vrHJ/ESrcC/5PmvTPsPTfhn/h GRDZ30AFYBV/QCr0a379n4Sq/fdvoAKQij8gFb9+cej//2+26v9/RZfVMAW/YQr+gF1a9h/dBIQC rH5BUfAfIIAYKA4OfKEBU+ChBVSwcY3g//8RYpP+TUMEBjws6n79wnTof6SgmKqv/19zyf//Yv+j 0TwCDYlYXB4lFBDgcAAIIMKxTlRA6evrV2n9+vWLfBM8gLr1OfQL/v+fJL7uP/v/7LWu/9cBk0/a //DajGmoHoCkuL9wgT8wN3w4gOqGf2v/r8QaAGD9cBNgaUo/ToBT2CIGaAIkeRoms//vAdKCGPoh 6RVmwp8/MBOAEfYIlHbxArD+3+j6//wnNi5w6f//HyCAqBSbg8AEBmBsNgAjM514bXPRTej71aC/ lQQTFk2HMyeDTQDa3+Cdrw80YZ3C/NXrmP/nsCuJ//+Xrcb0X1w8E1mrM8SAv3ATJoPTA0PVr181 +vogN3BLzP/3T+L//zXLzf+X/vvHtOE/ignOvyEGwE2YDElRwHDoK9qKFA4K//9P+8+E6XhncKJ+ 8hduwuQ/cBPk9VMhJoSD9YMA0z8sBgBNWPoXbsLkP3ATfvGfPwP2xf+Prv/Ugv/Pj/yv9++fuh2G Ab8hBvxVhRsANuHVr19T89z09d8ALf7PxAR0PiOIXoXqgjngdA0xYAbCAGiK8pz6luMz/giEGLAQ wwB4muT4T6YBxKZq3AYQaQIeA/4DBOC2/l0aBqI4gEO6RDup1A4q1ApHBTGLUlqHWoqL4lKxsf+A uIu4unRSkbZUx4AUig4uCrp0cXIS8Q9wFtx0EFJ9z3d3MUlja38O4ishpct9cpf3fe29s/5Ac/8n Ag8prTpGMZMwzc7CCrtJpYaEdbH0nhafiWtzklCg5mYGnsE03QFiFQDIBGmYApYhSL0PgFvBY35r Uik3wAklVxU/bMKwEJi3WvWQ9iMhCX7YGTdoQcCbMJ/E/JIE/lHYrEXQmwBqNTegAaHozFxxELQP r6qq0R5YB1Hw02VsROmZK5Bm+xEw6OHBd5nUY1gBRUf2ToS1fL4JwCacyll78hNQT6BaeZwwnXdB GUWlbKW2VT7+czvnnpJjXnyXmeglSICHMHg/MKK+zNuEAk99FrhAdhXCTT0UyOaG0gFxCAd43gZA EGxAHeEb4CI80Lq7JV5vhHkShMkoHuESLcgwl8RrzGTpz30kTIRthMVWWVhzjZVPJxc9AIeAOEWG 0rPKXwZtoeU+G+V2AFYk/wbwRNNdaZkIqz31fqeAvqdj54A+E7oBIH4JwI7dhDYRRHEAp0uFlEou ElMtaFshWwPNUJCKH+AHEREkYOJB8eBHUWrRWotB0gqCiLV60IQ0EEushyKhBxVTsZRaBKHgwUOF nlTwYsGbNyXJzvPNzCa7WybNGpOekiWbZHKY387O7Pvv1stUnVAnyAiiSpEvb0SVytaks1R5AlZr UaprQ0hpsta4iTDMHzgQctRbG0JKkxHieYMgYhMfjQM6ge6BM5iUjuB7J6UZCFLay6KTB2ABPzCw NbPKTWmiVLc/LAJNJigSjutzgBBeKkcEgcIL1lWmh3UcZDs6LQjUo/KffCv5mkhaBJpMUCSI8Jpt dZKu90jYJQgt3acozEde82MH10mA7zOc0P9w3ge0b0EfhVKCXNIi0GQC04ng47BxiZ2IkcKJ4FuY 9YG9qGoYIPMbv4ULbcHRtUZhIlckpDQZIZ63EthsuNnk2CduJHSCYMAjVzTUguHpvi8MvrejvC3U STHCsv+j0YYSggJBF2gygYVAhq8PDnV55YvSzx4/3psNALxK21oATKATnmsyQjwvIZDzv37eNS5N PQoEeFo2kuqlQ7hTbQsEoSjQZIJVhG+DJxpNV0ec+S4rgb/UBrsCTjAEmkxgITz+tPLZse3PYQsh FllUEzgBm2P+MHiax6AP3G6Ay9BrQ8AIJoEmE5gI271nN6w0bbp1YWjvRYPQGQNFScCi/zSdnMRA n6YuTshM+W0IkGAWXJUJDMIxPPQ77DbiAY6Hs7Ai2qYEAUIwdhuvkxDBRveVgx3ts2sInhQTZLK8 4FyB0MpG/yOG+DknOyd688xLUAKKoqShHaBj8zR7lAVtCuwG1lBekPtQXmDkha+4GHeM423ENULe PSs32/q7bQme2hCYI8vW7PL4jTnHfrL0XwXxXwXW1NTIRmELrK9gdXAbGID1FlQ7O1YgqDKhEkF1 CRUJqkqoTAB/BWjXfmObKMMAgGdtXBs2jCW0HUkNY6k3Jso6NENURoerTqNTtrm2oIC2jmszFbfU rBrFghOEwfT6BzTO+Wc6yGYFFgsEHZL4J3Ef/KARNVExfvCbhvhnYYTn8X3fdu1de+1aaEbBPR92 d+192P363Hv3Ps9bAPOI2cnULMIswixCQSP8/ZY07ntXGnv/DwixhkQ0Xtkw+UL1mtWToth02VxY /zt7Lxph4fpYm4x27B567HJD6JdvwslGwC+LQFIgFgyh+lYyr1qdhKAV3GxbJejZVFswDbHjJqPX a2yie/p/m08cO4C4XzhMDw1soYxKMNEmyhaBRM1isndYIPMjgcWifBJkixDwi/tBMYSF668XpT9F 2H1SffIBSkESIoEgAPjI5hDAANkUgQ16yLacgxKdohksQwKMVZmLWmE76oGtb+NoqaadnEcXdCjg iHaFEUCHPOgxBD49iZEcr/UOUREthSA7hIBf2hRj8ZtYIIqweVuxunhbZzQlfhEh8GDDYYhQhAZu 9GAz2BFtoIp+rQOOtc2s0CFCaONahiKwliJYyVELTMQQysiMWZcrQaKGJkOQDULAn9wZnLodfr87 YbDhH3rxa6oZwdU/T0oyoWSRxwPLBYrAgzL0sQf24xnQRr9eDBa2dYBKhOADZ2gteJoIgocLw1Iz xhBaBwYGNLkSyCAkCKZHCPhT26OiMWHqljj1o1pd/Il4bJQgoFkoR4pggLJQKCRAK7rhKEnqib56 PA0vk7M05BcnSTFCbwQLGiFCztPAc9FMWEGhogiu3LNABkFMMB1CwC/XI056RJLB8fvguJooTA2K UgQDp4ltlQqula0pdHJOrN0VBhgrIYOk1gEAvXTU3OIin43WD1Zw0aUNSq6+nNtBdoa5L7CHK8Iq jkVVjgRJCFKCzAhiggwItPoafOSv7/548YLfEwabATzmfD/9EgQShPffO38+WwQpwTQIny+58aar 5BG0vvBZRHvLrngFOsRl+M9Jzme32iMXAhFCKkF6hGSCDAjP7Gmcv+T5BY9+fcPmj+QyQWDXbNW2 88NFDqsdu/ieNp4/hOt4vtbkq3sJO/jh9h2GEe/SurPYw3eV+8ClqeH5DjTxNfkhiCPIEaRDSCVI h/DgvMlv5s6dc/+yO9XqT3c/eXv1qpvTIaDQh1gBCpoJQxYnVjrw6HaFwmJC+xgZKVxnKgbbWCbo 6XuCPfwhGtx5IpjqbsgSyCPIEcgi3EsbpKU7t7IqPIutK8nHnc9KENxwkPxt0aFgoY99hoDrPG5H A47WszPs7CF5QNXLmeMIqLQc0+SLINrcSEPw+L5sCVIQPpsXz/xT7PK7u3cGb4nfI6tKE6c6+4ym XhV2vMqZKr3g7TodLkFcTn5prPWN8Vahy0GfHxGbps6LSijTNRyJ2CbIq8KJCyJ44/VzMvFaPgg2 ptQTvvq1MYZQGgwuU7NYGSPo/KH/Ioc1V0WNMn8E5978Mh8E8kWVOZsowlNBtrKMxG1sud8CvFSR juDt/BCkrSxdd23jNcHgPePBJwjCT3dt/ACvXIKM5bXK7m9pHow/fCkLAzNAMF2N8fjTx/GKJyjs QusMERQywowRFC7CDBIUKkIeCP7MmgDxP44XujxBWD0mAAAAAElFTkSuQmCC --_004_7b5e25c2626147d096361672ff87ca67emailandroidcom_--

This message came from the mail archive
src/postings/2021/
maintained by:

DAn Ellis <dpwe@ee.columbia.edu>
Electrical Engineering Dept., Columbia University