Subject: Re: [AUDITORY] Software for internet-based auditory testing From: Pierre Divenyi <pdivenyi@xxxxxxxx> Date: Tue, 7 Nov 2017 18:34:33 -0800 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>This is a multi-part message in MIME format. --------------3840FFF6F980408C1E44C176 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by edgeum2.it.mcgill.ca id vA82Yc1v032271 Dick, The only IRB approval needed is from Ankara. Good luck. Pierre On 10/2/17 12:57 PM, Richard F. Lyon wrote: > Five years on, are there any updates on experience using Mechanical=20 > Turk and such for sound perception experiments? > > I've never conducted psychoacoustic experiments myself (other than=20 > informal ones on myself), but now I think I have some modeling ideas=20 > that need to be tuned and tested with corresponding experimental=20 > data.=C2=A0 Is MTurk the way to go? If it is, are IRB approvals still=20 > needed? I don't even know if that applies to me; probably my company=20 > has corresponding approval requirements. > > I'm interested in things like SNR thresholds for binaural detection=20 > and localization of different types of signals and noises -- 2AFC=20 > tests whose relative results across conditions would hopefully not be=20 > strongly dependent on level or headphone quality.=C2=A0 Are there good=20 > MTurk task structures that motivate people to do a good job on these,=20 > e.g. by making their space quieter, paying attention, getting more pay=20 > as the task gets harder, or just getting to do more similar tasks,=20 > etc.?=C2=A0 Can the pay depend on performance?=C2=A0 Or just cut them o= ff when=20 > the SNR has been lowered to threshold, so that people with lower=20 > thresholds stay on and get paid longer? > > If anyone in academia has a good setup for human experiments and an=20 > interest in collaborating on binaural model improvements, I'd love to=20 > discuss that, too, either privately or on the list. > > Dick > > > On Tue, Nov 13, 2012 at 3:36 AM, Kevin Tang <kevin.tang.10@xxxxxxxx=20 > <mailto:kevin.tang.10@xxxxxxxx>> wrote: > > Dear all, > > I would recommend "Experigen" - A framework for creating phonology > experiments.The platform can be used for all kinds of experiments, > but at the moment, it doesn't support reaction time. > > https://github.com/tlozoot/experigen > <https://github.com/tlozoot/experigen> > > All the best, > Kevin > > ------------------------------ > Kevin Tang > Doctoral candidate > UCL (University College London) > Division of Psychology and Language Sciences > Department of Linguistics > Rm 102, Chandler House, 2 Wakefield St > London WC1N 1PF > > mailto: kevin.tang.10@xxxxxxxx <mailto:kevin.tang.10@xxxxxxxx>, > kevin.tang@xxxxxxxx <mailto:kevin.tang@xxxxxxxx> > http://www.ucl.ac.uk/psychlangsci/research/linguistics/People/resea= rch-students/k_tang > <http://www.ucl.ac.uk/psychlangsci/research/linguistics/People/rese= arch-students/k_tang> > https://sites.google.com/site/kevintangcantab/home > <https://sites.google.com/site/kevintangcantab/home> > ------------------------------ > > > > On 12 November 2012 22:35, Jieun Oh <jieun5@xxxxxxxx > <mailto:jieun5@xxxxxxxx>> wrote: > > Hello, > > I recently presented (at this year's ICMPC conference) a paper > on using online crowdsourcing (e.g. Amazon Mechanical Turk) > for conducting music perception experiments. I've attached the > paper for your information, which includes, in Section IV, > examples of how the MIR community has been using this approach > in recent years. > > Online crowdsourcing as a research methodology definitely has > pros and cons, but I feel that there is a growing potential > for it, especially if the study requires mass participation > and can be designed as a short/simple task. > > Best, Jieun > > > > On Sat, Nov 10, 2012 at 6:32 AM, Sam Mathias > <smathias@xxxxxxxx <mailto:smathias@xxxxxxxx>> wrote: > > During my PhD I created this online test using adobe > flash: www.auditorytest.zxq.net > <http://www.auditorytest.zxq.net> > > I used it to search for potential participants who were > poor at pitch discrimination. [Mathias, Micheyl, & Bailey > (2010) JASA, 127, 3026-3037]. I found flash to be highly > versatile, have good functionality for auditory playback, > and not too difficult to learn. However, I think support > for this software is really dwindling these days (being > replaced by HTML 5?) so it may be obsolete soon. It was > also horrendously expense: I coded it with the 30-day > trial version so I didn't have to pay for it. > > Sam > > > On 9 November 2012 14:34, Robert Zatorre > <robert.zatorre@xxxxxxxx > <mailto:robert.zatorre@xxxxxxxx>> wrote: > > > > Dear list > > > > Several times the list has received requests for > participation in web-based experiments. We would like to > implement something along these lines, and I am wondering > if any of you who have experience with it have > recommendations (for or against) software to use. We are > looking for something reasonably inexpensive and simple to > program that would allow us to present audio stimuli and > collect behavioral responses, ideally with response times > although that may not be so simple I realize. > > > > Any advice would be welcome. Thank you in advance > > > > Robert Zatorre > > > > > > > > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > > Robert J. Zatorre, Ph.D. > > Montreal Neurological Institute > > 3801 University St. > > Montreal, QC Canada H3A 2B4 > > phone: 1-514-398-8903 <tel:1-514-398-8903> > > fax: 1-514-398-1338 <tel:1-514-398-1338> > > e-mail: robert.zatorre@xxxxxxxx > <mailto:robert.zatorre@xxxxxxxx> > > web site: www.zlab.mcgill.ca <http://www.zlab.mcgill.ca> > > > > > -- > Dr. Samuel R. Mathias > Neural Mechanisms of Human Communication > Max Planck Institute for Human Cognitive and Brain Sciences > Stephanstra=C3=9Fe 1 > 04103 Leipzig, Germany > Tel: +49 341 9940 2479 <tel:%2B49%20341%209940%202479> > > > > --------------3840FFF6F980408C1E44C176 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by edgeum2.it.mcgill.ca id vA82Yc1v032271 <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf= -8"> </head> <body text=3D"#000000" bgcolor=3D"#FFFFFF"> <p><font face=3D"Calibri">Dick,</font></p> <p><font face=3D"Calibri">The only IRB approval needed is from Ankara. Good luck.</font></p> <p><font face=3D"Calibri">Pierre</font><br> </p> <br> <div class=3D"moz-cite-prefix">On 10/2/17 12:57 PM, Richard F. Lyon wrote:<br> </div> <blockquote type=3D"cite" cite=3D"mid:19169_1507004599_59D310B7_19169_402_1_CAA=3DYKqiDKDfZoyN6b=3D= DpRnqC3z0vbXG+1ig8Vg9tgkytw_gkZg@xxxxxxxx"> <div dir=3D"ltr"> <div> <div> <div>Five years on, are there any updates on experience using Mechanical Turk and such for sound perception experiments?<br> <br> </div> I've never conducted psychoacoustic experiments myself (other than informal ones on myself), but now I think I have some modeling ideas that need to be tuned and tested with corresponding experimental data.=C2=A0 Is MTurk the way to go= ?=C2=A0 If it is, are IRB approvals still needed? I don't even know if that applies to me; probably my company has corresponding approval requirements.<br> <br> </div> I'm interested in things like SNR thresholds for binaural detection and localization of different types of signals and noises -- 2AFC tests whose relative results across conditions would hopefully not be strongly dependent on level or headphone quality.=C2=A0 Are there good MTurk task structures t= hat motivate people to do a good job on these, e.g. by making their space quieter, paying attention, getting more pay as the task gets harder, or just getting to do more similar tasks, etc.?=C2=A0 Can the pay depend on performance?=C2=A0 Or just cu= t them off when the SNR has been lowered to threshold, so that people with lower thresholds stay on and get paid longer?</div> <div><br> </div> <div>If anyone in academia has a good setup for human experiments and an interest in collaborating on binaural model improvements, I'd love to discuss that, too, either privately or on the list.<br> </div> <div><br> </div> Dick<br> <div><br> </div> </div> <div class=3D"gmail_extra"><br> <div class=3D"gmail_quote">On Tue, Nov 13, 2012 at 3:36 AM, Kevin Tang <span dir=3D"ltr"><<a href=3D"mailto:kevin.tang.10@xxxxxxxx" target=3D"_blank" moz-do-not-send=3D"true">kevin.tang.10@xxxxxxxx</a>></s= pan> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font><font face=3D"arial,helvetica,sans-serif">Dear all,<br> <br> <font>I would recommend</font> "</font></font>Experigen" - A framework for creating phonology experiments.<font><font face=3D"arial,helvetica,sans-serif"> <font>The platform <= font>can be use<font>d for all kinds of <font>experiments, but <font>at the <font>moment, it doesn't support reaction time.</font></font></font></= font></font></font></font></font><br> <br> <a href=3D"https://github.com/tlozoot/experigen" target=3D"_blank" moz-do-not-send=3D"true">https://github.c= om/tlozoot/<wbr>experigen</a><br> <br> <font>All the best,</font><br clear=3D"all"> <div style=3D"font-family:arial,helvetica,sans-serif"><font>K= evin</font><font><br> </font></div> <br style=3D"font-family:arial,helvetica,sans-serif"> <span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-align:auto;text-indent:0px;text-transform:n= one;white-space:normal;word-spacing:0px;font-size:medium"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px;font-size:medium"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-size:medium;font-style:normal;font-variant:normal;font-weight:normal= ;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:= none;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-size:medium;font-style:normal;font-variant:normal;font-weight:normal= ;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:= none;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-size:medium;font-style:normal;font-variant:normal;font-weight:normal= ;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:= none;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;l= etter-spacing:normal;line-height:normal;text-indent:0px;text-transform:no= ne;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;l= etter-spacing:normal;line-height:normal;text-indent:0px;text-transform:no= ne;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;l= etter-spacing:normal;line-height:normal;text-indent:0px;text-transform:no= ne;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;l= etter-spacing:normal;line-height:normal;text-indent:0px;text-transform:no= ne;white-space:normal;word-spacing:0px"><span style=3D"font-family:arial,helvetica,sans-serif"> </span> <div> <div style=3D"word-wrap:break-word"> <div> <div style=3D"word-wrap:break-word"> <div style=3D"font-family:arial,helvet= ica,sans-serif"><font size=3D"1">--------------------= ----------</font></div> <div style=3D"font-family:arial,helvet= ica,sans-serif"><font size=3D"1">Kevin Tang<br> </font></div> <div style=3D"font-family:arial,helvet= ica,sans-serif;color:rgb(0,0,0)"><font size=3D"1">Doctoral candidate<b= r> <span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px">UCL (University College London<= /span></span></span></span></span></span></span></span></span>)<br> </font><font size=3D"1">Division of Psychology and Language Sciences<br> </font></div> <div style=3D"font-family:arial,helvet= ica,sans-serif;color:rgb(0,0,0)"><font size=3D"1">Department of Linguistics</font></div> <div><font size=3D"1"><font color=3D"#000000"><span style=3D"background-color:t= ransparent"></span></font></font> <div style=3D"margin:0px"> <font size=3D"1">Rm 102, Chandler House, 2 Wakefield St<br> London WC1N 1PF<br> <br> </font> </div> <font size=3D"1">mailto: <a href=3D"mailto:kevin.tang.10@xxxxxxxx= ucl.ac.uk" target=3D"_blank" moz-do-not-send=3D"true">kevi= n.tang.10@xxxxxxxx</a>, <a href=3D"mailto:kevin.tang@xxxxxxxx= tab.net" target=3D"_blank" moz-do-not-send=3D"true">kevi= n.tang@xxxxxxxx</a><br> <a href=3D"http://www.ucl.ac.uk/psychlangsci/research/linguistics/People/res= earch-students/k_tang" target=3D"_blank" moz-do-not-send=3D"true">http= ://www.ucl.ac.uk/<wbr>psychlangsci/research/<wbr>linguistics/People/resea= rch-<wbr>students/k_tang</a><br> <a href=3D"https://sites.google.= com/site/kevintangcantab/home" target=3D"_blank" moz-do-not-send=3D"true">http= s://sites.google.com/site/<wbr>kevintangcantab/home</a><br> </font></div> </div> </div> </div> </div> </span></span></span></span></span></span></s= pan></span></span><font size=3D"1"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;= font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:n= ormal;line-height:normal;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px">------------------------------</span></span></sp= an></span></span></span></span></span></span></font> <div class=3D"HOEnZb"> <div class=3D"h5"> <div> </div> <br> <br> <br> <div class=3D"gmail_quote">On 12 November 2012 22:35, Jieun Oh <span dir=3D"ltr"><<a href=3D"mailto:jieun5@xxxxxxxx" target=3D"_blank" moz-do-not-send=3D"true">jieun5@xxxxxxxx= crma.stanford.edu</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div> <div>Hello,</div> <div><br> </div> <div>I recently presented (at this year's ICMPC conference) a paper on using online crowdsourcing (e.g. Amazon Mechanical Turk) for conducting music perception experiments. I've attached the paper for your information, which includes, in Section IV, examples of how the MIR community has been using this approach in recent years.=C2=A0</div> <div><br> </div> <div>Online crowdsourcing as a research methodology definitely has pros and cons, but I feel that there is a growing potential for it, especially if the study requires mass participation and can be designed as a short/simple task.</div> <div><br> </div> <div>Best, Jieun</div> <div> <div class=3D"m_-3386482373432126768h5"> <div><br> </div> <div><br> </div> <div><br> <div class=3D"gmail_quote">On Sat, Nov 10, 2012 at 6:32 AM, Sam Mathias <span dir=3D"ltr"> <<a href=3D"mailto:smathias@xxxxxxxx= " target=3D"_blank" moz-do-not-send=3D"tr= ue">smathias@xxxxxxxx</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> During my PhD I created this online test using adobe flash:=C2=A0<a href=3D"http://www.auditorytest.zxq.net= " target=3D"_blank" moz-do-not-send=3D"tr= ue">www.auditorytest.zxq.<wbr>net</a><br> <br> I used it to search for potential participants who were poor at pitch discrimination. [Mathias, Micheyl, & Bailey (2010) JASA, 127, 3026-3037]. I found flash to be highly versatile, have good functionality for auditory playback, and not too difficult to learn. However, I think support for this software is really dwindling these days (being replaced by HTML 5?) so it may be obsolete soon. It was also horrendously expense: I coded it with the 30-day trial version so I didn't have to pay for it. <div><br> Sam <div> <div><br> <br> On 9 November 2012 14:34, Robert Zatorre <<a href=3D"mailto:robert.zatorre@xxxxxxxx= ill.ca" target=3D"_blank" moz-do-not-send=3D"true">robert.z= atorre@xxxxxxxx</a>> wrote:<br> ><br> > Dear list<br> ><br> > Several times the list has received requests for participation in web-based experiments. We would like to implement something along these lines, and I am wondering if any of you who have experience with it have recommendations (for or against) software to use. We are looking for something reasonably inexpensive and simple to program that would allow us to present audio stimuli and collect behavioral responses, ideally with response times although that may not be so simple I realize.<br> ><br> > Any advice would be welcome. Thank you in advance<br> ><br> > Robert Zatorre<br> ><br> ><br> ><br> > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<wbr>= -+-+<br> ><br> > Robert J. Zatorre, Ph.D.<br> > Montreal Neurological Institute<br> > 3801 University St.<br> > Montreal, QC Canada H3A 2B4<br= > > phone: <a href=3D"tel:1-514-398-8903" value=3D"+15143988903" target=3D"_blank" moz-do-not-send=3D"true">1-514-39= 8-8903</a><br> > fax: <a href=3D"tel:1-514-398-1338" value=3D"+15143981338" target=3D"_blank" moz-do-not-send=3D"true">1-514-39= 8-1338</a><br> > e-mail: <a href=3D"mailto:robert.zatorre@xxxxxxxx= ill.ca" target=3D"_blank" moz-do-not-send=3D"true">robert.z= atorre@xxxxxxxx</a><br> > web site: <a href=3D"http://www.zlab.mcgill.ca= " target=3D"_blank" moz-do-not-send=3D"true">www.zlab= .mcgill.ca</a><br> <br> <br> <br> <br> </div> </div> --<br> Dr. Samuel R. Mathias<br> Neural Mechanisms of Human Communication<br> Max Planck Institute for Human Cognitive and Brain Sciences<br> Stephanstra=C3=9Fe 1<br> 04103 Leipzig, Germany<br> Tel: <a href=3D"tel:%2B49%20341%209940%202479= " value=3D"+4934199402479" target=3D"_blank" moz-do-not-send=3D"true"> +49 341 9940 2479</a></div> </blockquote> </div> <br> </div> </div> </div> </div> </blockquote> </div> <br> </div> </div> </blockquote> </div> <br> </div> </blockquote> <br> </body> </html> --------------3840FFF6F980408C1E44C176--