Subject: Re: [AUDITORY] Software for internet-based auditory testing From: "Richard F. Lyon" <dicklyon@xxxxxxxx> Date: Mon, 2 Oct 2017 12:57:03 -0700 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>--f403045c0c965486ff055a95c7ed Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Five years on, are there any updates on experience using Mechanical Turk and such for sound perception experiments? I've never conducted psychoacoustic experiments myself (other than informal ones on myself), but now I think I have some modeling ideas that need to be tuned and tested with corresponding experimental data. Is MTurk the way to go? If it is, are IRB approvals still needed? I don't even know if that applies to me; probably my company has corresponding approval requirements. I'm interested in things like SNR thresholds for binaural detection and localization of different types of signals and noises -- 2AFC tests whose relative results across conditions would hopefully not be strongly dependent on level or headphone quality. Are there good MTurk task structures that motivate people to do a good job on these, e.g. by making their space quieter, paying attention, getting more pay as the task gets harder, or just getting to do more similar tasks, etc.? Can the pay depend on performance? Or just cut them off when the SNR has been lowered to threshold, so that people with lower thresholds stay on and get paid longer= ? If anyone in academia has a good setup for human experiments and an interest in collaborating on binaural model improvements, I'd love to discuss that, too, either privately or on the list. Dick On Tue, Nov 13, 2012 at 3:36 AM, Kevin Tang <kevin.tang.10@xxxxxxxx> wrote= : > Dear all, > > I would recommend "Experigen" - A framework for creating phonology > experiments. The platform can be used for all kinds of experiments, but a= t > the moment, it doesn't support reaction time. > > https://github.com/tlozoot/experigen > > All the best, > Kevin > > ------------------------------ > Kevin Tang > Doctoral candidate > UCL (University College London) > Division of Psychology and Language Sciences > Department of Linguistics > Rm 102, Chandler House, 2 Wakefield St > London WC1N 1PF > > mailto: kevin.tang.10@xxxxxxxx, kevin.tang@xxxxxxxx > http://www.ucl.ac.uk/psychlangsci/research/linguistics/People/research- > students/k_tang > https://sites.google.com/site/kevintangcantab/home > ------------------------------ > > > > On 12 November 2012 22:35, Jieun Oh <jieun5@xxxxxxxx> wrote: > >> Hello, >> >> I recently presented (at this year's ICMPC conference) a paper on using >> online crowdsourcing (e.g. Amazon Mechanical Turk) for conducting music >> perception experiments. I've attached the paper for your information, wh= ich >> includes, in Section IV, examples of how the MIR community has been usin= g >> this approach in recent years. >> >> Online crowdsourcing as a research methodology definitely has pros and >> cons, but I feel that there is a growing potential for it, especially if >> the study requires mass participation and can be designed as a short/sim= ple >> task. >> >> Best, Jieun >> >> >> >> On Sat, Nov 10, 2012 at 6:32 AM, Sam Mathias <smathias@xxxxxxxx> wrote= : >> >>> During my PhD I created this online test using adobe flash: >>> www.auditorytest.zxq.net >>> >>> I used it to search for potential participants who were poor at pitch >>> discrimination. [Mathias, Micheyl, & Bailey (2010) JASA, 127, 3026-3037= ]. I >>> found flash to be highly versatile, have good functionality for auditor= y >>> playback, and not too difficult to learn. However, I think support for = this >>> software is really dwindling these days (being replaced by HTML 5?) so = it >>> may be obsolete soon. It was also horrendously expense: I coded it with= the >>> 30-day trial version so I didn't have to pay for it. >>> >>> Sam >>> >>> >>> On 9 November 2012 14:34, Robert Zatorre <robert.zatorre@xxxxxxxx> >>> wrote: >>> > >>> > Dear list >>> > >>> > Several times the list has received requests for participation in >>> web-based experiments. We would like to implement something along these >>> lines, and I am wondering if any of you who have experience with it hav= e >>> recommendations (for or against) software to use. We are looking for >>> something reasonably inexpensive and simple to program that would allow= us >>> to present audio stimuli and collect behavioral responses, ideally with >>> response times although that may not be so simple I realize. >>> > >>> > Any advice would be welcome. Thank you in advance >>> > >>> > Robert Zatorre >>> > >>> > >>> > >>> > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>> > >>> > Robert J. Zatorre, Ph.D. >>> > Montreal Neurological Institute >>> > 3801 University St. >>> > Montreal, QC Canada H3A 2B4 >>> > phone: 1-514-398-8903 >>> > fax: 1-514-398-1338 >>> > e-mail: robert.zatorre@xxxxxxxx >>> > web site: www.zlab.mcgill.ca >>> >>> >>> >>> >>> -- >>> Dr. Samuel R. Mathias >>> Neural Mechanisms of Human Communication >>> Max Planck Institute for Human Cognitive and Brain Sciences >>> Stephanstra=C3=9Fe 1 >>> 04103 Leipzig, Germany >>> Tel: +49 341 9940 2479 >>> >> >> > --f403045c0c965486ff055a95c7ed Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div><div><div>Five years on, are there any updates on exp= erience using Mechanical Turk and such for sound perception experiments?<br= ><br></div>I've never conducted psychoacoustic experiments myself (othe= r than informal ones on myself), but now I think I have some modeling ideas= that need to be tuned and tested with corresponding experimental data.=C2= =A0 Is MTurk the way to go?=C2=A0 If it is, are IRB approvals still needed?= I don't even know if that applies to me; probably my company has corre= sponding approval requirements.<br><br></div>I'm interested in things l= ike SNR thresholds for binaural detection and localization of different typ= es of signals and noises -- 2AFC tests whose relative results across condit= ions would hopefully not be strongly dependent on level or headphone qualit= y.=C2=A0 Are there good MTurk task structures that motivate people to do a = good job on these, e.g. by making their space quieter, paying attention, ge= tting more pay as the task gets harder, or just getting to do more similar = tasks, etc.?=C2=A0 Can the pay depend on performance?=C2=A0 Or just cut the= m off when the SNR has been lowered to threshold, so that people with lower= thresholds stay on and get paid longer?</div><div><br></div><div>If anyone= in academia has a good setup for human experiments and an interest in coll= aborating on binaural model improvements, I'd love to discuss that, too= , either privately or on the list.<br></div><div><br></div>Dick<br><div><br= ></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On T= ue, Nov 13, 2012 at 3:36 AM, Kevin Tang <span dir=3D"ltr"><<a href=3D"ma= ilto:kevin.tang.10@xxxxxxxx" target=3D"_blank">kevin.tang.10@xxxxxxxx</a>= ></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0= 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font><font face=3D"ar= ial,helvetica,sans-serif">Dear all,<br><br><font>I would recommend</font> &= quot;</font></font>Experigen" - A framework for creating phonology exp= eriments.<font><font face=3D"arial,helvetica,sans-serif"> <font>The platfor= m <font>can be use<font>d for all kinds of <font>experiments, but <font>at = the <font>moment, it doesn't support reaction time.</font></font></font= ></font></font></font></font></font><br> <br><a href=3D"https://github.com/tlozoot/experigen" target=3D"_blank">http= s://github.com/tlozoot/<wbr>experigen</a><br><br><font>All the best,</font>= <br clear=3D"all"><div style=3D"font-family:arial,helvetica,sans-serif"><fo= nt>Kevin</font><font><br> </font></div><br style=3D"font-family:arial,helvetica,sans-serif"><span sty= le=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-= style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;l= ine-height:normal;text-align:auto;text-indent:0px;text-transform:none;white= -space:normal;word-spacing:0px;font-size:medium"><span style=3D"border-coll= apse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font= -variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal= ;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;fo= nt-size:medium"><span style=3D"border-collapse:separate;color:rgb(0,0,0);fo= nt-family:Helvetica;font-size:medium;font-style:normal;font-variant:normal;= font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px= ;text-transform:none;white-space:normal;word-spacing:0px"><span style=3D"bo= rder-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-size:med= ium;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing= :normal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(= 0,0,0);font-family:Helvetica;font-size:medium;font-style:normal;font-varian= t:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-i= ndent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span st= yle=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font= -size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-= spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white= -space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;col= or:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-v= ariant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;t= ext-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><sp= an style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica= ;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;le= tter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;= white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separat= e;color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;f= ont-variant:normal;font-weight:normal;letter-spacing:normal;line-height:nor= mal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px= "><span style=3D"font-family:arial,helvetica,sans-serif"> </span><div> <div style=3D"word-wrap:break-word"> <div> <div style=3D"word-wrap:break-word"> <div style=3D"font-family:arial,helvetica,sans-serif"><font size=3D"1">----= --------------------------</font></div> <div style=3D"font-family:arial,helvetica,sans-serif"><font size=3D"1">Kevi= n Tang<br></font></div> <div style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><fon= t size=3D"1">Doctoral candidate<br><span style=3D"border-collapse:separate;= color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant:norma= l;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0= px;text-transform:none;white-space:normal;word-spacing:0px"><span style=3D"= border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style:= normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-he= ight:normal;text-indent:0px;text-transform:none;white-space:normal;word-spa= cing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-fam= ily:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;lett= er-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;wh= ite-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;= color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant:norma= l;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0= px;text-transform:none;white-space:normal;word-spacing:0px"><span style=3D"= border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style:= normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-he= ight:normal;text-indent:0px;text-transform:none;white-space:normal;word-spa= cing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-fam= ily:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;lett= er-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;wh= ite-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;= color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant:norma= l;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0= px;text-transform:none;white-space:normal;word-spacing:0px"><span style=3D"= border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style:= normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-he= ight:normal;text-indent:0px;text-transform:none;white-space:normal;word-spa= cing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-fam= ily:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;lett= er-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;wh= ite-space:normal;word-spacing:0px">UCL (University College London</span></s= pan></span></span></span></span></span></span></span>)<br> </font><font size=3D"1">Division of Psychology and Language Sciences<br></f= ont></div> <div style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><fon= t size=3D"1">Department of Linguistics</font></div> <div><font size=3D"1"><font color=3D"#000000"><span style=3D"background-col= or:transparent"></span></font></font> <div style=3D"margin:0px"> <font size=3D"1">Rm 102, Chandler House, 2 Wakefield St<br>London WC1N 1PF<= br><br></font> </div><font size=3D"1">mailto: <a href=3D"mailto:kevin.tang.10@xxxxxxxx" t= arget=3D"_blank">kevin.tang.10@xxxxxxxx</a>, <a href=3D"mailto:kevin.tang@xxxxxxxx= cantab.net" target=3D"_blank">kevin.tang@xxxxxxxx</a><br><a href=3D"http:= //www.ucl.ac.uk/psychlangsci/research/linguistics/People/research-students/= k_tang" target=3D"_blank">http://www.ucl.ac.uk/<wbr>psychlangsci/research/<= wbr>linguistics/People/research-<wbr>students/k_tang</a><br> <a href=3D"https://sites.google.com/site/kevintangcantab/home" target=3D"_b= lank">https://sites.google.com/site/<wbr>kevintangcantab/home</a><br></font= ></div></div></div></div></div></span></span></span></span></span></span></= span></span></span><font size=3D"1"><span style=3D"border-collapse:separate= ;color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant:norm= al;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:= 0px;text-transform:none;white-space:normal;word-spacing:0px"><span style=3D= "border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style= :normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-h= eight:normal;text-indent:0px;text-transform:none;white-space:normal;word-sp= acing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-fa= mily:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;let= ter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;w= hite-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate= ;color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant:norm= al;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:= 0px;text-transform:none;white-space:normal;word-spacing:0px"><span style=3D= "border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style= :normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-h= eight:normal;text-indent:0px;text-transform:none;white-space:normal;word-sp= acing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-fa= mily:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;let= ter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;w= hite-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate= ;color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant:norm= al;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:= 0px;text-transform:none;white-space:normal;word-spacing:0px"><span style=3D= "border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style= :normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-h= eight:normal;text-indent:0px;text-transform:none;white-space:normal;word-sp= acing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-fa= mily:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;let= ter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;w= hite-space:normal;word-spacing:0px">------------------------------</span></= span></span></span></span></span></span></span></span></font><div class=3D"= HOEnZb"><div class=3D"h5"><div> </div><div></div><div></div><div></div><br> <br><br><div class=3D"gmail_quote">On 12 November 2012 22:35, Jieun Oh <spa= n dir=3D"ltr"><<a href=3D"mailto:jieun5@xxxxxxxx" target=3D"_b= lank">jieun5@xxxxxxxx</a>></span> wrote:<br><blockquote class= =3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd= ing-left:1ex"> <div> <div>Hello,</div> <div><br> </div> <div>I recently presented (at this year's ICMPC conference) a paper on = using online crowdsourcing (e.g. Amazon Mechanical Turk) for conducting mus= ic perception experiments. I've attached the paper for your information= , which includes, in Section IV, examples of how the MIR community has been using this approach in recent years.=C2= =A0</div> <div><br> </div> <div>Online crowdsourcing as a research methodology definitely has pros and= cons, but I feel that there is a growing potential for it, especially if t= he study requires mass participation and can be designed as a short/simple = task.</div> <div><br> </div> <div>Best, Jieun</div><div><div class=3D"m_-3386482373432126768h5"> <div><br> </div> <div><br> </div> <div><br> <div class=3D"gmail_quote">On Sat, Nov 10, 2012 at 6:32 AM, Sam Mathias <sp= an dir=3D"ltr"> <<a href=3D"mailto:smathias@xxxxxxxx" target=3D"_blank">smathias@xxxxxxxx= pg.de</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> During my PhD I created this online test using adobe flash:=C2=A0<a href=3D= "http://www.auditorytest.zxq.net" target=3D"_blank">www.auditorytest.zxq.<w= br>net</a><br> <br> I used it to search for potential participants who were poor at pitch discr= imination. [Mathias, Micheyl, & Bailey (2010) JASA, 127, 3026-3037]. I = found flash to be highly versatile, have good functionality for auditory pl= ayback, and not too difficult to learn. However, I think support for this software is really dwindling these days = (being replaced by HTML 5?) so it may be obsolete soon. It was also horrend= ously expense: I coded it with the 30-day trial version so I didn't hav= e to pay for it. <div><br> Sam <div> <div><br> <br> On 9 November 2012 14:34, Robert Zatorre <<a href=3D"mailto:robert.zator= re@xxxxxxxx" target=3D"_blank">robert.zatorre@xxxxxxxx</a>> wrote:<br> ><br> > Dear list<br> ><br> > Several times the list has received requests for participation in web-= based experiments. We would like to implement something along these lines, = and I am wondering if any of you who have experience with it have recommend= ations (for or against) software to use. We are looking for something reasonably inexpensive and simple to pro= gram that would allow us to present audio stimuli and collect behavioral re= sponses, ideally with response times although that may not be so simple I r= ealize.<br> ><br> > Any advice would be welcome. Thank you in advance<br> ><br> > Robert Zatorre<br> ><br> ><br> ><br> > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<wbr>-+-+<br> ><br> > Robert J. Zatorre, Ph.D.<br> > Montreal Neurological Institute<br> > 3801 University St.<br> > Montreal, QC Canada H3A 2B4<br> > phone: <a href=3D"tel:1-514-398-8903" value=3D"+15143988903" target=3D= "_blank">1-514-398-8903</a><br> > fax: <a href=3D"tel:1-514-398-1338" value=3D"+15143981338" target=3D"_= blank">1-514-398-1338</a><br> > e-mail: <a href=3D"mailto:robert.zatorre@xxxxxxxx" target=3D"_blank">= robert.zatorre@xxxxxxxx</a><br> > web site: <a href=3D"http://www.zlab.mcgill.ca" target=3D"_blank">www.= zlab.mcgill.ca</a><br> <br> <br> <br> <br> </div> </div> --<br> Dr. Samuel R. Mathias<br> Neural Mechanisms of Human Communication<br> Max Planck Institute for Human Cognitive and Brain Sciences<br> Stephanstra=C3=9Fe 1<br> 04103 Leipzig, Germany<br> Tel: <a href=3D"tel:%2B49%20341%209940%202479" value=3D"+4934199402479" tar= get=3D"_blank"> +49 341 9940 2479</a></div> </blockquote> </div> <br> </div> </div></div></div> </blockquote></div><br> </div></div></blockquote></div><br></div> --f403045c0c965486ff055a95c7ed--