Subject: [AUDITORY] New tools & data for soundscape synthesis and online audio annotation From: Justin Salamon <justin.salamon@xxxxxxxx> Date: Tue, 10 Oct 2017 11:56:46 -0400 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>--001a113ecd46eb0fd3055b335bc5 Content-Type: text/plain; charset="UTF-8" *** apologies for cross-posting *** Dear list, We're glad to announce the release of two open-source tools and a new dataset developed as part of the SONYC <http://wp.nyu.edu/sonyc> project we hope will be of use to the community: *Scaper <https://github.com/justinsalamon/scaper>: a library for soundscape synthesis and augmentation* - Automatically synthesize soundscapes with corresponding ground truth annotations - Useful for running controlled ML experiments (ASR, sound event detection, bioacoustic species recognition, etc.) - Useful for running controlled experiments to assess human annotation performance - Potentially useful for generating data for source separation experiments (might require some extra code) - Potentially useful for generating ambisonic soundscapes (definitely requires some extra code) *AudioAnnotator <https://github.com/CrowdCurio/audio-annotator>: a javascript web interface for annotating audio data* - Developed in collaboration with Edith Law and her students <http://edithlaw.ca/people.html> at the University of Waterloo's HCI Lab <https://hci.cs.uwaterloo.ca/> - A web interface that allows users to annotate audio recordings - Supports 3 types of visualization (waveform, spectrogram, invisible) - Useful for crowdsourcing audio labels - Useful for running controlled experiments on crowdsourcing audio labels - Supports feedback mechanisms for providing real-time feedback to the user based on their annotations *URBAN-SED dataset <http://urbansed.weebly.com/>: a new dataset for sound event detection* - Includes 10,000 soundscapes with strongly labeled sound events generated using scaper - Totals almost 30 hours and includes close to 50,000 annotated sound events - Baseline convnet results on URBAN-SED are included in the scaper-paper <http://www.justinsalamon.com/uploads/4/3/9/4/4394963/salamon_scaper_waspaa_2017.pdf> . Further information about scaper, the AudioAnnotator and the URBAN-SED dataset, including controlled experiments on the quality of crowdsourced human annotations as a function of visualization and soundscape complexity, are provided in the following papers: Seeing sound: Investigating the effects of visualizations and complexity on crowdsourced audio annotations <https://wp.nyu.edu/sonyc/seeing-sound-investigating-the-effects-of-visualizations-and-complexity-on-crowdsourced-audio-annotations/> M. Cartwright, A. Seals, J. Salamon, A. Williams, S. Mikloska, D. MacConnell, E. Law, J. Bello, and O. Nov. Proceedings of the ACM on Human-Computer Interaction, 1(2), 2017. Scaper: A Library for Soundscape Synthesis and Augmentation <http://www.justinsalamon.com/uploads/4/3/9/4/4394963/salamon_scaper_waspaa_2017.pdf> J. Salamon, D. MacConnell, M. Cartwright, P. Li, and J. P. Bello. In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, Oct. 2017. We hope you find these tools and data useful and look forward to receiving your feedback (and pull requests!). Cheers, on behalf of the entire team, Justin Salamon & Mark Cartwright. -- Justin Salamon, PhD Senior Research Scientist Music and Audio Research Laboratory (MARL) & Center for Urban Science and Progress (CUSP) New York University, New York, NY www.justinsalamon.com --001a113ecd46eb0fd3055b335bc5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><span class=3D"gmail-m_8218605140721400829gmail-im" style= =3D"font-size:12.8px"><div style=3D"font-size:12.8px">*** apologies for cro= ss-posting ***</div><div style=3D"font-size:12.8px"><br></div><div style=3D= "font-size:12.8px">Dear list,</div><div style=3D"font-size:12.8px"><br></di= v></span><div style=3D"font-size:12.8px">We're glad to announce the rel= ease of two open-source tools and a new dataset developed as part of the=C2= =A0<a href=3D"http://wp.nyu.edu/sonyc" target=3D"_blank">SONYC</a>=C2=A0pro= ject we hope will be of use to the community:=C2=A0</div><span class=3D"gma= il-m_8218605140721400829gmail-im" style=3D"font-size:12.8px"><div style=3D"= font-size:12.8px"><br></div><div style=3D"font-size:12.8px"><b><a href=3D"h= ttps://github.com/justinsalamon/scaper" target=3D"_blank">Scaper</a>: a lib= rary for soundscape synthesis and augmentation</b></div><div style=3D"font-= size:12.8px">- Automatically synthesize soundscapes with corresponding grou= nd truth annotations=C2=A0<br></div><div style=3D"font-size:12.8px">- Usefu= l for running controlled ML experiments (ASR, sound event detection, bioaco= ustic species recognition, etc.)</div><div style=3D"font-size:12.8px">- Use= ful for running controlled experiments to assess human annotation performan= ce</div><div style=3D"font-size:12.8px">- Potentially useful for generating= data for source separation experiments (might require some extra code)</di= v><div style=3D"font-size:12.8px">- Potentially useful for generating ambis= onic soundscapes (definitely requires some extra code)</div><div style=3D"f= ont-size:12.8px"><br></div><div style=3D"font-size:12.8px"><b><a href=3D"ht= tps://github.com/CrowdCurio/audio-annotator" target=3D"_blank">AudioAnnotat= or</a>: a javascript web interface for annotating audio data</b><br></div><= div style=3D"font-size:12.8px">- Developed in collaboration with <a href=3D= "http://edithlaw.ca/people.html">Edith Law and her students</a> at the Univ= ersity of Waterloo's <a href=3D"https://hci.cs.uwaterloo.ca/">HCI Lab</= a></div><div style=3D"font-size:12.8px">-=C2=A0A web interface that allows = users to annotate audio recordings</div><div style=3D"font-size:12.8px">- S= upports 3 types of visualization (waveform, spectrogram, invisible)</div><d= iv style=3D"font-size:12.8px">- Useful for crowdsourcing audio labels</div>= <div style=3D"font-size:12.8px">- Useful for running controlled experiments= on crowdsourcing audio labels</div><div style=3D"font-size:12.8px">- Suppo= rts feedback mechanisms for providing real-time feedback to the user based = on their annotations</div></span><div style=3D"font-size:12.8px"><br></div>= <div style=3D"font-size:12.8px"><b><a href=3D"http://urbansed.weebly.com/" = target=3D"_blank">URBAN-SED dataset</a>:=C2=A0<span style=3D"font-size:12.8= px">a new dataset for sound event detection</span></b></div><div style=3D"f= ont-size:12.8px">- Includes 10,000 soundscapes with strongly labeled sound = events generated using scaper</div><div style=3D"font-size:12.8px">- Totals= almost 30 hours and includes close to 50,000 annotated sound events</div><= div style=3D"font-size:12.8px">- Baseline convnet results on URBAN-SED are = included in the=C2=A0<a href=3D"http://www.justinsalamon.com/uploads/4/3/9/= 4/4394963/salamon_scaper_waspaa_2017.pdf" target=3D"_blank">scaper-paper</a= >.</div><div style=3D"font-size:12.8px"><br></div><div style=3D"font-size:1= 2.8px">Further information about scaper, the AudioAnnotator and the URBAN-S= ED dataset, including controlled experiments on the quality of crowdsourced= human annotations as a function of visualization and soundscape complexity= , are provided in the following papers:</div><div style=3D"font-size:12.8px= "><br></div><div style=3D"font-size:12.8px"><span class=3D"gmail-m_82186051= 40721400829gmail-im"><div><a href=3D"https://wp.nyu.edu/sonyc/seeing-sound-= investigating-the-effects-of-visualizations-and-complexity-on-crowdsourced-= audio-annotations/" target=3D"_blank">Seeing sound: Investigating the effec= ts of visualizations and complexity on crowdsourced audio annotations</a><b= r></div><div><div>M. Cartwright, A. Seals, J. Salamon, A. Williams, S. Mikl= oska, D. MacConnell, E. Law, J. Bello, and O. Nov.</div><div>Proceedings of= the ACM on Human-Computer Interaction, 1(2), 2017.</div></div><div><br></d= iv><div><div style=3D"font-size:12.8px"><a href=3D"http://www.justinsalamon= .com/uploads/4/3/9/4/4394963/salamon_scaper_waspaa_2017.pdf" target=3D"_bla= nk">Scaper: A Library for Soundscape Synthesis and Augmentation</a></div><d= iv style=3D"font-size:12.8px">J. Salamon, D. MacConnell, M. Cartwright, P. = Li, and J. P. Bello.</div><div style=3D"font-size:12.8px">In IEEE Workshop = on Applications of Signal Processing to Audio and Acoustics (WASPAA), New P= altz, NY, USA, Oct. 2017.</div></div><div><br></div></span><div>We hope you= find these tools and data useful and look forward to receiving your feedba= ck (and pull requests!).</div></div><div class=3D"gmail-m_82186051407214008= 29gmail-yj6qo gmail-m_8218605140721400829gmail-ajU" style=3D"font-size:12.8= px;margin:2px 0px 0px"><div id=3D"gmail-m_8218605140721400829gmail-:9lr" cl= ass=3D"gmail-m_8218605140721400829gmail-ajR"><img class=3D"gmail-m_82186051= 40721400829gmail-ajT gmail-CToWUd" src=3D"https://ssl.gstatic.com/ui/v1/ico= ns/mail/images/cleardot.gif"></div></div><div class=3D"gmail-m_821860514072= 1400829gmail-adL" style=3D"font-size:12.8px"><span class=3D"gmail-m_8218605= 140721400829gmail-im"><div style=3D"font-size:12.8px">Cheers, on behalf of = the entire team,</div><div style=3D"font-size:12.8px">Justin Salamon & = Mark Cartwright.</div></span></div><br clear=3D"all"><div><br></div>-- <br>= <div class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div>= <div dir=3D"ltr"><div>Justin Salamon, PhD</div><div>Senior Research Scienti= st</div><div>Music and Audio Research Laboratory (MARL)</div><div>& Cen= ter for Urban Science and Progress (CUSP)</div><div>New York University, Ne= w York, NY</div><div><a href=3D"http://www.justinsalamon.com/" style=3D"col= or:rgb(17,85,204)" target=3D"_blank">www.justinsalamon.com</a></div></div><= /div></div></div></div></div></div> --001a113ecd46eb0fd3055b335bc5--