Subject: Re: [AUDITORY] Software for internet-based auditory testing From: Samuel Mehr <sam@xxxxxxxx> Date: Tue, 3 Oct 2017 17:59:37 +1300 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>--089e08244930084d21055a9d5e1a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Dear Dick, Lots of folks do successful audio-based experiments on Turk and I generally find it to be a good platform for the sort of work you're describing (which is not really what I do, but experimentally is similar enough for the purposes of your question). I've done a few simple listening experiments of the form "listen to this thing, answer some questions about it", and the results directly replicate parallel in-person experiments in my lab, even when Turkers geolocate to lots of far-flung countries. I require subjects to wear headphones and validate that requirement with this great task from Josh McDermott's lab: Woods, K. J. P., Siegel, M. H., Traer, J., & McDermott, J. H. (2017). Headphone screening to facilitate web-based auditory experiments. *Attentio= n, Perception, & Psychophysics*, 1=E2=80=939. https://doi.org/10.3758/s13414-0= 17-1361-2 In a bunch of piloting, passing the headphone screener correlates with a bunch of other checks on Turker compliance, positively. Things like "What color is the sky? Please answer incorrectly, on purpose" and "Tell us honestly how carefully you completed this HIT". Basically, if you have a few metrics in an experiment that capture variance on some dimension related to participant quality, you should be able to easily tell which Turkers are actually doing good work and which aren't. Depending on how your ethics approval is set up, you can either pay everyone and filter out bad subjects, or require them to pass some level of quality control to receive payment. best Sam --=20 Samuel Mehr Department of Psychology Harvard University themusiclab.org naturalhistoryofsong.org On Tue, Oct 3, 2017 at 8:57 AM, Richard F. Lyon <dicklyon@xxxxxxxx> wrote: > Five years on, are there any updates on experience using Mechanical Turk > and such for sound perception experiments? > > I've never conducted psychoacoustic experiments myself (other than > informal ones on myself), but now I think I have some modeling ideas that > need to be tuned and tested with corresponding experimental data. Is MTu= rk > the way to go? If it is, are IRB approvals still needed? I don't even kn= ow > if that applies to me; probably my company has corresponding approval > requirements. > > I'm interested in things like SNR thresholds for binaural detection and > localization of different types of signals and noises -- 2AFC tests whose > relative results across conditions would hopefully not be strongly > dependent on level or headphone quality. Are there good MTurk task > structures that motivate people to do a good job on these, e.g. by making > their space quieter, paying attention, getting more pay as the task gets > harder, or just getting to do more similar tasks, etc.? Can the pay depe= nd > on performance? Or just cut them off when the SNR has been lowered to > threshold, so that people with lower thresholds stay on and get paid long= er? > > If anyone in academia has a good setup for human experiments and an > interest in collaborating on binaural model improvements, I'd love to > discuss that, too, either privately or on the list. > > Dick > > > On Tue, Nov 13, 2012 at 3:36 AM, Kevin Tang <kevin.tang.10@xxxxxxxx> > wrote: > >> Dear all, >> >> I would recommend "Experigen" - A framework for creating phonology >> experiments. The platform can be used for all kinds of experiments, but = at >> the moment, it doesn't support reaction time. >> >> https://github.com/tlozoot/experigen >> <https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__github.com_tlozo= ot_experigen&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL85hQ&r=3DJU36mKiQOsv42HCBbLwT= O302m5Y3d6JVmlBV92I0soM&m=3D4MUnqICVVpK7XrN3Be2VIFPKinSSytURTO3leZ9OPwA&s= =3DeI2DGG0-IDm1SRMvIl3WMSKxQdG8obVegCsDMX8NyDc&e=3D> >> >> All the best, >> Kevin >> >> ------------------------------ >> Kevin Tang >> Doctoral candidate >> UCL (University College London) >> Division of Psychology and Language Sciences >> Department of Linguistics >> Rm 102, Chandler House, 2 Wakefield St >> <https://maps.google.com/?q=3D2+Wakefield+St+London+WC1N+1PF&entry=3Dgma= il&source=3Dg> >> London WC1N 1PF >> <https://maps.google.com/?q=3D2+Wakefield+St+London+WC1N+1PF&entry=3Dgma= il&source=3Dg> >> >> mailto: kevin.tang.10@xxxxxxxx, kevin.tang@xxxxxxxx >> http://www.ucl.ac.uk/psychlangsci/research/linguistics/Peopl >> e/research-students/k_tang >> <https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.ucl.ac.uk_psy= chlangsci_research_linguistics_People_research-2Dstudents_k-5Ftang&d=3DDwMF= aQ&c=3DWO-RGvefibhHBZq3fL85hQ&r=3DJU36mKiQOsv42HCBbLwTO302m5Y3d6JVmlBV92I0s= oM&m=3D4MUnqICVVpK7XrN3Be2VIFPKinSSytURTO3leZ9OPwA&s=3DKsTUlaFCItm3eXJcXxv0= lJhbaWkuWGkEcblrD-R9k4I&e=3D> >> https://sites.google.com/site/kevintangcantab/home >> <https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__sites.google.com= _site_kevintangcantab_home&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL85hQ&r=3DJU36mK= iQOsv42HCBbLwTO302m5Y3d6JVmlBV92I0soM&m=3D4MUnqICVVpK7XrN3Be2VIFPKinSSytURT= O3leZ9OPwA&s=3DTEkzqd18yqtZQaTqACm9hWeGEKhWmryzcQh5FttO5hQ&e=3D> >> ------------------------------ >> >> >> >> On 12 November 2012 22:35, Jieun Oh <jieun5@xxxxxxxx> wrote: >> >>> Hello, >>> >>> I recently presented (at this year's ICMPC conference) a paper on using >>> online crowdsourcing (e.g. Amazon Mechanical Turk) for conducting music >>> perception experiments. I've attached the paper for your information, w= hich >>> includes, in Section IV, examples of how the MIR community has been usi= ng >>> this approach in recent years. >>> >>> Online crowdsourcing as a research methodology definitely has pros and >>> cons, but I feel that there is a growing potential for it, especially i= f >>> the study requires mass participation and can be designed as a short/si= mple >>> task. >>> >>> Best, Jieun >>> >>> >>> >>> On Sat, Nov 10, 2012 at 6:32 AM, Sam Mathias <smathias@xxxxxxxx> >>> wrote: >>> >>>> During my PhD I created this online test using adobe flash: >>>> www.auditorytest.zxq.net >>>> <https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.auditorytes= t.zxq.net&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL85hQ&r=3DJU36mKiQOsv42HCBbLwTO30= 2m5Y3d6JVmlBV92I0soM&m=3D4MUnqICVVpK7XrN3Be2VIFPKinSSytURTO3leZ9OPwA&s=3DQK= BXmLLfxza96D7Ovoumzbs1K0O8wR_qGgHBy3yvGOM&e=3D> >>>> >>>> I used it to search for potential participants who were poor at pitch >>>> discrimination. [Mathias, Micheyl, & Bailey (2010) JASA, 127, 3026-303= 7]. I >>>> found flash to be highly versatile, have good functionality for audito= ry >>>> playback, and not too difficult to learn. However, I think support for= this >>>> software is really dwindling these days (being replaced by HTML 5?) so= it >>>> may be obsolete soon. It was also horrendously expense: I coded it wit= h the >>>> 30-day trial version so I didn't have to pay for it. >>>> >>>> Sam >>>> >>>> >>>> On 9 November 2012 14:34, Robert Zatorre <robert.zatorre@xxxxxxxx> >>>> wrote: >>>> > >>>> > Dear list >>>> > >>>> > Several times the list has received requests for participation in >>>> web-based experiments. We would like to implement something along thes= e >>>> lines, and I am wondering if any of you who have experience with it ha= ve >>>> recommendations (for or against) software to use. We are looking for >>>> something reasonably inexpensive and simple to program that would allo= w us >>>> to present audio stimuli and collect behavioral responses, ideally wit= h >>>> response times although that may not be so simple I realize. >>>> > >>>> > Any advice would be welcome. Thank you in advance >>>> > >>>> > Robert Zatorre >>>> > >>>> > >>>> > >>>> > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >>>> > >>>> > Robert J. Zatorre, Ph.D. >>>> > Montreal Neurological Institute >>>> > 3801 University St. >>>> <https://maps.google.com/?q=3D3801+University+St.+%0D+%3E+Montreal,+QC= +Canada+H3A+2B4&entry=3Dgmail&source=3Dg> >>>> > Montreal, QC Canada H3A 2B4 >>>> <https://maps.google.com/?q=3D3801+University+St.+%0D+%3E+Montreal,+QC= +Canada+H3A+2B4&entry=3Dgmail&source=3Dg> >>>> > phone: 1-514-398-8903 >>>> > fax: 1-514-398-1338 >>>> > e-mail: robert.zatorre@xxxxxxxx >>>> > web site: www.zlab.mcgill.ca >>>> <https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.zlab.mcgill= .ca&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL85hQ&r=3DJU36mKiQOsv42HCBbLwTO302m5Y3d= 6JVmlBV92I0soM&m=3D4MUnqICVVpK7XrN3Be2VIFPKinSSytURTO3leZ9OPwA&s=3Dg5Vi4QE9= Nl27c_ni3SaaSKkwQikMQAyKbL6l76ypxKg&e=3D> >>>> >>>> >>>> >>>> >>>> -- >>>> Dr. Samuel R. Mathias >>>> Neural Mechanisms of Human Communication >>>> Max Planck Institute for Human Cognitive and Brain Sciences >>>> Stephanstra=C3=9Fe 1 >>>> <https://maps.google.com/?q=3DStephanstra%C3%9Fe+1+%0D+04103+Leipzig,+= Germany&entry=3Dgmail&source=3Dg> >>>> 04103 Leipzig, Germany >>>> Tel: +49 341 9940 2479 >>>> >>> >>> >> > --089e08244930084d21055a9d5e1a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Dear Dick,<div><br></div><div>Lots of folks do successful = audio-based experiments on Turk and I generally find it to be a good platfo= rm for the sort of work you're describing (which is not really what I d= o, but experimentally is similar enough for the purposes of your question).= I've done a few simple listening experiments of the form "listen = to this thing, answer some questions about it", and the results direct= ly replicate parallel in-person experiments in my lab, even when Turkers ge= olocate to lots of far-flung countries. I require subjects to wear headphon= es and validate that requirement with this great task from Josh McDermott&#= 39;s lab:<div><span><br></span></div><div><span>Woods, K. J. P., Siegel, M.= H., Traer, J., & McDermott, J. H. (2017). Headphone screening to facil= itate web-based auditory experiments. </span><i>Attention, Perception, &= ; Psychophysics</i><span>, 1=E2=80=939. <a href=3D"https://doi.org/10.3758/= s13414-017-1361-2" target=3D"_blank">https://doi.org/10.3758/s13414<wbr>-01= 7-1361-2</a></span><div class=3D"m_-6318531832382596260gmail-m_-39418137628= 89735983gmail-csl-bib-body" style=3D"line-height:2;margin-left:2em"> =20 <span class=3D"m_-6318531832382596260gmail-m_-3941813762889735983gmail-Z3= 988" title=3D"url_ver=3DZ39.88-2004&ctx_ver=3DZ39.88-2004&rfr_id=3D= info%3Asid%2Fzotero.org%3A2&rft_id=3Dinfo%3Adoi%2F10.3758%2Fs13414-017-= 1361-2&rft_val_fmt=3Dinfo%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.gen= re=3Darticle&rft.atitle=3DHeadphone%20screening%20to%20facilitate%20web= -based%20auditory%20experiments&rft.jtitle=3DAttention%2C%20Perception%= 2C%20%26%20Psychophysics&rft.stitle=3DAtten%20Percept%20Psychophys&= rft.aufirst=3DKevin%20J.%20P.&rft.aulast=3DWoods&rft.au=3DKevin%20J= .%20P.%20Woods&rft.au=3DMax%20H.%20Siegel&rft.au=3DJames%20Traer&am= p;rft.au=3DJosh%20H.%20McDermott&rft.date=3D2017-07-10&rft.pages=3D= 1-9&rft.spage=3D1&rft.epage=3D9&rft.issn=3D1943-3921%2C%201943-= 393X&rft.language=3Den"></span></div></div><div class=3D"gmail_extra"><= br></div><div class=3D"gmail_extra">In a bunch of piloting, passing the hea= dphone screener correlates with a bunch of other checks on Turker complianc= e, positively. Things like "What color is the sky? Please answer incor= rectly, on purpose" and "Tell us honestly how carefully you compl= eted this HIT". Basically, if you have a few metrics in an experiment = that capture variance on some dimension related to participant quality, you= should be able to easily tell which Turkers are actually doing good work a= nd which aren't. Depending on how your ethics approval is set up, you c= an either pay everyone and filter out bad subjects, or require them to pass= some level of quality control to receive payment.</div><div class=3D"gmail= _extra"><br></div><div class=3D"gmail_extra">best</div><div class=3D"gmail_= extra">Sam</div><div class=3D"gmail_extra"><div><br class=3D"m_-63185318323= 82596260gmail-Apple-interchange-newline"><br></div>--=C2=A0<br><div class= =3D"m_-6318531832382596260gmail-m_-3941813762889735983gmail_signature"><div= dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D= "ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><= div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir= =3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr= "><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div style=3D"font-siz= e:12.8px"><div style=3D"font-size:12.8px"><div style=3D"font-size:12.8px"><= div style=3D"font-size:12.8px"><div style=3D"font-size:12.8px"><div style= =3D"font-size:12.8px"><div style=3D"font-size:12.8px"><div style=3D"color:r= gb(136,136,136);font-size:12.8px"><div dir=3D"ltr" style=3D"color:rgb(34,34= ,34);font-size:12.8px"><div dir=3D"ltr" style=3D"font-size:12.8px"><div dir= =3D"ltr" style=3D"font-size:12.8px"><div dir=3D"ltr" style=3D"font-size:12.= 8px"><div dir=3D"ltr" style=3D"font-size:12.8px"><div dir=3D"ltr" style=3D"= font-size:12.8px"><font color=3D"#666666">Samuel Mehr</font></div><div styl= e=3D"font-size:12.8px"><font color=3D"#666666">Department of Psychology</fo= nt></div><div dir=3D"ltr" style=3D"font-size:12.8px"><span style=3D"color:r= gb(102,102,102);font-size:12.8px">Harvard University</span><br></div><div d= ir=3D"ltr" style=3D"font-size:12.8px"><font color=3D"#666666"><a href=3D"ht= tp://themusiclab.org/" target=3D"_blank">themusiclab.org</a></font></div><d= iv dir=3D"ltr" style=3D"font-size:12.8px"><font color=3D"#666666"><a href= =3D"http://naturalhistoryofsong.org/" style=3D"font-size:12.8px" target=3D"= _blank">naturalhistoryofsong.org</a></font></div></div></div></div></div></= div></div></div></div></div></div></div></div></div></div></div></div></div= ></div></div></div></div></div></div></div></div></div></div></div></div></= div></div></div></div></div></div></div><div class=3D"gmail_extra"><br></di= v><div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra"><br><div = class=3D"gmail_quote">On Tue, Oct 3, 2017 at 8:57 AM, Richard F. Lyon <span= dir=3D"ltr"><<a href=3D"mailto:dicklyon@xxxxxxxx" target=3D"_blank">dick= lyon@xxxxxxxx</a>></span> wrote:<br><blockquote class=3D"gmail_quote" sty= le=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddi= ng-left:1ex"><div dir=3D"ltr"><div><div><div>Five years on, are there any u= pdates on experience using Mechanical Turk and such for sound perception ex= periments?<br><br></div>I've never conducted psychoacoustic experiments= myself (other than informal ones on myself), but now I think I have some m= odeling ideas that need to be tuned and tested with corresponding experimen= tal data.=C2=A0 Is MTurk the way to go?=C2=A0 If it is, are IRB approvals s= till needed? I don't even know if that applies to me; probably my compa= ny has corresponding approval requirements.<br><br></div>I'm interested= in things like SNR thresholds for binaural detection and localization of d= ifferent types of signals and noises -- 2AFC tests whose relative results a= cross conditions would hopefully not be strongly dependent on level or head= phone quality.=C2=A0 Are there good MTurk task structures that motivate peo= ple to do a good job on these, e.g. by making their space quieter, paying a= ttention, getting more pay as the task gets harder, or just getting to do m= ore similar tasks, etc.?=C2=A0 Can the pay depend on performance?=C2=A0 Or = just cut them off when the SNR has been lowered to threshold, so that peopl= e with lower thresholds stay on and get paid longer?</div><div><br></div><d= iv>If anyone in academia has a good setup for human experiments and an inte= rest in collaborating on binaural model improvements, I'd love to discu= ss that, too, either privately or on the list.<br></div><div><br></div>Dick= <br><div><br></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail= _quote">On Tue, Nov 13, 2012 at 3:36 AM, Kevin Tang <span dir=3D"ltr"><<= a href=3D"mailto:kevin.tang.10@xxxxxxxx" target=3D"_blank">kevin.tang.10@xxxxxxxx= cl.ac.uk</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"><font><font face=3D"arial,helvetica,sans-serif">Dear all,<br><br= ><font>I would recommend</font> "</font></font>Experigen" - A fra= mework for creating phonology experiments.<font><font face=3D"arial,helveti= ca,sans-serif"> <font>The platform <font>can be use<font>d for all kinds of= <font>experiments, but <font>at the <font>moment, it doesn't support r= eaction time.</font></font></font></font></font></font></font></font><br> <br><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__githu= b.com_tlozoot_experigen&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL85hQ&r= =3DJU36mKiQOsv42HCBbLwTO302m5Y3d6JVmlBV92I0soM&m=3D4MUnqICVVpK7XrN3Be2V= IFPKinSSytURTO3leZ9OPwA&s=3DeI2DGG0-IDm1SRMvIl3WMSKxQdG8obVegCsDMX8NyDc= &e=3D" target=3D"_blank">https://github.com/tlozoot/exp<wbr>erigen</a><= br><br><font>All the best,</font><br clear=3D"all"><div style=3D"font-famil= y:arial,helvetica,sans-serif"><font>Kevin</font><font><br> </font></div><br style=3D"font-family:arial,helvetica,sans-serif"><span sty= le=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-= style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-we= ight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-t= ransform:none;white-space:normal;word-spacing:0px;font-size:medium"><span s= tyle=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;fon= t-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-= weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text= -transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span= style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;f= ont-size:medium;font-style:normal;font-variant-ligatures:normal;font-varian= t-caps:normal;font-weight:normal;letter-spacing:normal;line-height:normal;t= ext-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><sp= an style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica= ;font-size:medium;font-style:normal;font-variant-ligatures:normal;font-vari= ant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:normal= ;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><= span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helveti= ca;font-size:medium;font-style:normal;font-variant-ligatures:normal;font-va= riant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:norm= al;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"= ><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helve= tica;font-size:12px;font-style:normal;font-variant-ligatures:normal;font-va= riant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:norm= al;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"= ><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helve= tica;font-size:12px;font-style:normal;font-variant-ligatures:normal;font-va= riant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:norm= al;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"= ><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helve= tica;font-size:12px;font-style:normal;font-variant-ligatures:normal;font-va= riant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:norm= al;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"= ><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helve= tica;font-size:12px;font-style:normal;font-variant-ligatures:normal;font-va= riant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:norm= al;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"= ><span style=3D"font-family:arial,helvetica,sans-serif"> </span><div> <div style=3D"word-wrap:break-word"> <div> <div style=3D"word-wrap:break-word"> <div style=3D"font-family:arial,helvetica,sans-serif"><font size=3D"1">----= --------------------------</font></div> <div style=3D"font-family:arial,helvetica,sans-serif"><font size=3D"1">Kevi= n Tang<br></font></div> <div style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><fon= t size=3D"1">Doctoral candidate<br><span style=3D"border-collapse:separate;= color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant-ligat= ures:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:norm= al;line-height:normal;text-indent:0px;text-transform:none;white-space:norma= l;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0= );font-family:Helvetica;font-style:normal;font-variant-ligatures:normal;fon= t-variant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:= normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:= 0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:H= elvetica;font-style:normal;font-variant-ligatures:normal;font-variant-caps:= normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-ind= ent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span styl= e=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-s= tyle:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-wei= ght:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-tr= ansform:none;white-space:normal;word-spacing:0px"><span style=3D"border-col= lapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style:normal;fon= t-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;lett= er-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;wh= ite-space:normal;word-spacing:0px"><span style=3D"border-collapse:separate;= color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant-ligat= ures:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:norm= al;line-height:normal;text-indent:0px;text-transform:none;white-space:norma= l;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0= );font-family:Helvetica;font-style:normal;font-variant-ligatures:normal;fon= t-variant-caps:normal;font-weight:normal;letter-spacing:normal;line-height:= normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:= 0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:H= elvetica;font-style:normal;font-variant-ligatures:normal;font-variant-caps:= normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-ind= ent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span styl= e=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-s= tyle:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-wei= ght:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-tr= ansform:none;white-space:normal;word-spacing:0px">UCL (University College L= ondon</span></span></span></span></span></span></span></span></span>)<br> </font><font size=3D"1">Division of Psychology and Language Sciences<br></f= ont></div> <div style=3D"font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><fon= t size=3D"1">Department of Linguistics</font></div> <div><font size=3D"1"><font color=3D"#000000"><span style=3D"background-col= or:transparent"></span></font></font> <div style=3D"margin:0px"> <font size=3D"1">Rm 102, Chandler House, <a href=3D"https://maps.google.com= /?q=3D2+Wakefield+St+London+WC1N+1PF&entry=3Dgmail&source=3Dg" targ= et=3D"_blank">2 Wakefield St</a><br><a href=3D"https://maps.google.com/?q= =3D2+Wakefield+St+London+WC1N+1PF&entry=3Dgmail&source=3Dg" target= =3D"_blank">London WC1N 1PF</a><br><br></font> </div><font size=3D"1">mailto: <a href=3D"mailto:kevin.tang.10@xxxxxxxx" t= arget=3D"_blank">kevin.tang.10@xxxxxxxx</a>, <a href=3D"mailto:kevin.tang@xxxxxxxx= cantab.net" target=3D"_blank">kevin.tang@xxxxxxxx</a><br><a href=3D"https= ://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.ucl.ac.uk_psychlangsci= _research_linguistics_People_research-2Dstudents_k-5Ftang&d=3DDwMFaQ&am= p;c=3DWO-RGvefibhHBZq3fL85hQ&r=3DJU36mKiQOsv42HCBbLwTO302m5Y3d6JVmlBV92= I0soM&m=3D4MUnqICVVpK7XrN3Be2VIFPKinSSytURTO3leZ9OPwA&s=3DKsTUlaFCI= tm3eXJcXxv0lJhbaWkuWGkEcblrD-R9k4I&e=3D" target=3D"_blank">http://www.u= cl.ac.uk/psychlang<wbr>sci/research/linguistics/Peopl<wbr>e/research-studen= ts/k_tang</a><br> <a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__sites.goo= gle.com_site_kevintangcantab_home&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL= 85hQ&r=3DJU36mKiQOsv42HCBbLwTO302m5Y3d6JVmlBV92I0soM&m=3D4MUnqICVVp= K7XrN3Be2VIFPKinSSytURTO3leZ9OPwA&s=3DTEkzqd18yqtZQaTqACm9hWeGEKhWmryzc= Qh5FttO5hQ&e=3D" target=3D"_blank">https://sites.google.com/site/<wbr>k= evintangcantab/home</a><br></font></div></div></div></div></div></span></sp= an></span></span></span></span></span></span></span><font size=3D"1"><span = style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;fo= nt-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font= -weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;tex= t-transform:none;white-space:normal;word-spacing:0px"><span style=3D"border= -collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style:normal= ;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;= letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:non= e;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separ= ate;color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant-l= igatures:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:= normal;line-height:normal;text-indent:0px;text-transform:none;white-space:n= ormal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0= ,0,0);font-family:Helvetica;font-style:normal;font-variant-ligatures:normal= ;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;line-hei= ght:normal;text-indent:0px;text-transform:none;white-space:normal;word-spac= ing:0px"><span style=3D"border-collapse:separate;color:rgb(0,0,0);font-fami= ly:Helvetica;font-style:normal;font-variant-ligatures:normal;font-variant-c= aps:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text= -indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span = style=3D"border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;fo= nt-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font= -weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;tex= t-transform:none;white-space:normal;word-spacing:0px"><span style=3D"border= -collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-style:normal= ;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;= letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:non= e;white-space:normal;word-spacing:0px"><span style=3D"border-collapse:separ= ate;color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-variant-l= igatures:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:= normal;line-height:normal;text-indent:0px;text-transform:none;white-space:n= ormal;word-spacing:0px"><span style=3D"border-collapse:separate;color:rgb(0= ,0,0);font-family:Helvetica;font-style:normal;font-variant-ligatures:normal= ;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;line-hei= ght:normal;text-indent:0px;text-transform:none;white-space:normal;word-spac= ing:0px">------------------------------</span></span></span></span></span><= /span></span></span></span></font><div class=3D"m_-6318531832382596260gmail= -m_-3941813762889735983m_2962783548217721872HOEnZb"><div class=3D"m_-631853= 1832382596260gmail-m_-3941813762889735983m_2962783548217721872h5"><div> </div><div></div><div></div><div></div><br> <br><br><div class=3D"gmail_quote">On 12 November 2012 22:35, Jieun Oh <spa= n dir=3D"ltr"><<a href=3D"mailto:jieun5@xxxxxxxx" target=3D"_b= lank">jieun5@xxxxxxxx</a>></span> wrote:<br><blockquote class= =3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg= b(204,204,204);padding-left:1ex"> <div> <div>Hello,</div> <div><br> </div> <div>I recently presented (at this year's ICMPC conference) a paper on = using online crowdsourcing (e.g. Amazon Mechanical Turk) for conducting mus= ic perception experiments. I've attached the paper for your information= , which includes, in Section IV, examples of how the MIR community has been using this approach in recent years.=C2= =A0</div> <div><br> </div> <div>Online crowdsourcing as a research methodology definitely has pros and= cons, but I feel that there is a growing potential for it, especially if t= he study requires mass participation and can be designed as a short/simple = task.</div> <div><br> </div> <div>Best, Jieun</div><div><div class=3D"m_-6318531832382596260gmail-m_-394= 1813762889735983m_2962783548217721872m_-3386482373432126768h5"> <div><br> </div> <div><br> </div> <div><br> <div class=3D"gmail_quote">On Sat, Nov 10, 2012 at 6:32 AM, Sam Mathias <sp= an dir=3D"ltr"> <<a href=3D"mailto:smathias@xxxxxxxx" target=3D"_blank">smathias@xxxxxxxx= pg.de</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"> During my PhD I created this online test using adobe flash:=C2=A0<a href=3D= "https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__www.auditorytest.zxq= .net&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL85hQ&r=3DJU36mKiQOsv42HCB= bLwTO302m5Y3d6JVmlBV92I0soM&m=3D4MUnqICVVpK7XrN3Be2VIFPKinSSytURTO3leZ9= OPwA&s=3DQKBXmLLfxza96D7Ovoumzbs1K0O8wR_qGgHBy3yvGOM&e=3D" target= =3D"_blank">www.auditorytest.zxq.ne<wbr>t</a><br> <br> I used it to search for potential participants who were poor at pitch discr= imination. [Mathias, Micheyl, & Bailey (2010) JASA, 127, 3026-3037]. I = found flash to be highly versatile, have good functionality for auditory pl= ayback, and not too difficult to learn. However, I think support for this software is really dwindling these days = (being replaced by HTML 5?) so it may be obsolete soon. It was also horrend= ously expense: I coded it with the 30-day trial version so I didn't hav= e to pay for it. <div><br> Sam <div> <div><br> <br> On 9 November 2012 14:34, Robert Zatorre <<a href=3D"mailto:robert.zator= re@xxxxxxxx" target=3D"_blank">robert.zatorre@xxxxxxxx</a>> wrote:<br> ><br> > Dear list<br> ><br> > Several times the list has received requests for participation in web-= based experiments. We would like to implement something along these lines, = and I am wondering if any of you who have experience with it have recommend= ations (for or against) software to use. We are looking for something reasonably inexpensive and simple to pro= gram that would allow us to present audio stimuli and collect behavioral re= sponses, ideally with response times although that may not be so simple I r= ealize.<br> ><br> > Any advice would be welcome. Thank you in advance<br> ><br> > Robert Zatorre<br> ><br> ><br> ><br> > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<wbr>-+-+<br> ><br> > Robert J. Zatorre, Ph.D.<br> > Montreal Neurological Institute<br> > <a href=3D"https://maps.google.com/?q=3D3801+University+St.+%0D+%3E+Mo= ntreal,+QC+Canada+H3A+2B4&entry=3Dgmail&source=3Dg" target=3D"_blan= k">3801 University St.</a><br> > <a href=3D"https://maps.google.com/?q=3D3801+University+St.+%0D+%3E+Mo= ntreal,+QC+Canada+H3A+2B4&entry=3Dgmail&source=3Dg" target=3D"_blan= k">Montreal, QC Canada H3A 2B4</a><br> > phone: <a href=3D"tel:1-514-398-8903" value=3D"+15143988903" target=3D= "_blank">1-514-398-8903</a><br> > fax: <a href=3D"tel:1-514-398-1338" value=3D"+15143981338" target=3D"_= blank">1-514-398-1338</a><br> > e-mail: <a href=3D"mailto:robert.zatorre@xxxxxxxx" target=3D"_blank">= robert.zatorre@xxxxxxxx</a><br> > web site: <a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp= -3A__www.zlab.mcgill.ca&d=3DDwMFaQ&c=3DWO-RGvefibhHBZq3fL85hQ&r= =3DJU36mKiQOsv42HCBbLwTO302m5Y3d6JVmlBV92I0soM&m=3D4MUnqICVVpK7XrN3Be2V= IFPKinSSytURTO3leZ9OPwA&s=3Dg5Vi4QE9Nl27c_ni3SaaSKkwQikMQAyKbL6l76ypxKg= &e=3D" target=3D"_blank">www.zlab.mcgill.ca</a><br> <br> <br> <br><span class=3D"m_-6318531832382596260gmail-m_-3941813762889735983HOEnZb= "><font color=3D"#888888"> <br> </font></span></div><span class=3D"m_-6318531832382596260gmail-m_-394181376= 2889735983HOEnZb"><font color=3D"#888888"> </font></span></div><span class=3D"m_-6318531832382596260gmail-m_-394181376= 2889735983HOEnZb"><font color=3D"#888888"> --<br> Dr. Samuel R. Mathias<br> Neural Mechanisms of Human Communication<br> Max Planck Institute for Human Cognitive and Brain Sciences<br> <a href=3D"https://maps.google.com/?q=3DStephanstra%C3%9Fe+1+%0D+04103+Leip= zig,+Germany&entry=3Dgmail&source=3Dg" target=3D"_blank">Stephanstr= a=C3=9Fe 1</a><br> 04103 Leipzig, Germany<br> Tel: <a href=3D"tel:%2B49%20341%209940%202479" value=3D"+4934199402479" tar= get=3D"_blank"> +49 341 9940 2479</a></font></span></div><span class=3D"m_-6318531832382596= 260gmail-m_-3941813762889735983HOEnZb"><font color=3D"#888888"> </font></span></blockquote><span class=3D"m_-6318531832382596260gmail-m_-39= 41813762889735983HOEnZb"><font color=3D"#888888"> </font></span></div><span class=3D"m_-6318531832382596260gmail-m_-394181376= 2889735983HOEnZb"><font color=3D"#888888"> <br> </font></span></div><span class=3D"m_-6318531832382596260gmail-m_-394181376= 2889735983HOEnZb"><font color=3D"#888888"> </font></span></div></div></div><span class=3D"m_-6318531832382596260gmail-= m_-3941813762889735983HOEnZb"><font color=3D"#888888"> </font></span></blockquote></div><span class=3D"m_-6318531832382596260gmail= -m_-3941813762889735983HOEnZb"><font color=3D"#888888"><br> </font></span></div></div></blockquote></div><br></div> </blockquote></div><br><br> </div></div></div> --089e08244930084d21055a9d5e1a--