Subject: Re: Experiments with large N From: "J. Devin McAuley" <mcauley@xxxxxxxx> Date: Mon, 3 Dec 2007 13:58:55 -0500 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>This issue nicely highlights the need to report effect size measures. With a large enough sample, even the smallest of effects will show up as reliable! :) Best regards, Devin > -----Original Message----- > From: AUDITORY - Research in Auditory Perception > [mailto:AUDITORY@xxxxxxxx On Behalf Of Robert Zatorre > Sent: Monday, December 03, 2007 1:42 PM > To: AUDITORY@xxxxxxxx > Subject: Re: [AUDITORY] Experiments with large N > > Huge samples are very nice if you can get 'em, though such is not always > the case, alas. > > So one thing that I would like to see from people who do have gigantic N > is to do some analyses to determine at what point the data reach some > asymptote. In other words, if you've collected 1,000,000 people, at what > earlier point in your sampling could you have stopped, and come to the > identical conclusions with valid statistics? > > Obviously, the answer to this question will be different for different > types of studies with different types of variance and so forth. But > having the large N allows one to perform this calculation, so that next > time one does a similar study, one could reasonably stop after reaching > a smaller and more manageable sample size. > > Has anybody already done this for those large samples that were recently > discussed? It would be really helpful for those who cannot always > collect such samples. > > Best > > Robert > -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Robert J. Zatorre, Ph.D. > Montreal Neurological Institute > 3801 University St. > Montreal, QC Canada H3A 2B4 > phone: 1-514-398-8903 > fax: 1-514-398-1338 > e-mail: robert.zatorre@xxxxxxxx > web site: www.zlab.mcgill.ca > > > Malcolm Slaney wrote: > > This music paper has 380k subjects :-) > > > http://cobweb.ecn.purdue.edu/~malcolm/yahoo/Slaney2007(SimilarityByUserRat > ingISMIR).pdf > > > > While Ben Marlin collected another 30k subjects for this > > music-recommendation study. > > > http://cobweb.ecn.purdue.edu/~malcolm/yahoo/Marlin2007(UserBiasUncertainty > ).pdf > > > > The underlying data for both papers is available for academic > > researchers (fully anonymized, both by song and by user). Send me email > > if you want more information. > > > > - Malcolm > > > > On Dec 1, 2007, at 5:43 PM, Matt Wright wrote: > > > >> Trevor Cox recently published the results of an online experiment > >> about listeners' ratings of sound files on a six-point scale ("not > >> horrible", "bad", "really bad", "awful", "really awful", and > >> "horrible"). To date he has 130,000 subjects (!) and about 1.5 > >> million data points: > >> > >> http://www.sea-acustica.es/WEB_ICA_07/fchrs/papers/ppa-09-003.pdf > >> > >> Here's the website for his experiment: http://www.sound101.org > >> > >> Clearly this is related to the "effect of visual stimuli on the > >> horribleness of awful sounds" that Kelly Fitz pointed out. > >> > >> -Matt > >> > >> > >> On Jun 29, 2007, at 12:32 AM, Massimo Grassi wrote: > >>> So far it looks that the experiment with the largest N (513!) is "The > >>> role of contrasting temporal amplitude patterns in the perception of > >>> speech" Healy and Warren JASA but I didn't check yet the methodology > >>> to see whether is a between or a within subject design. > >