Re: AUDITORY Digest - 26 Oct 2000 to 27 Oct 2000 (#2000-155)

[Michael Kubovy <mk9y@xxxxxxxxxxxx>]

> Date:    Fri, 27 Oct 2000 16:30:39 +1100
> From:    Chris Chambers <Chris.Chambers@SCI.MONASH.EDU.AU>
> Subject: within-subject comparisons in psychoacoustics

> one problem with
> determining statistical significance of within-subject comparisons is
> that ANOVA and t-tests require that data-points within each condition
> are statistically independent. If all the data in a particular analysis
> are provided by one subject then it can be argued that individual data
> points within each treatment are no longer independent, and hence not
> amenable to ANOVA.

The problem can't be answered unless we know which correlations between
observations you are worried about. The most likely correlation between
conditions may come from correlations between successive observations.
Standard experimental techniques are designed to mitigate the problem.
Suppose we are studying n conditions (C[1], ... C[n]), which we present
to a listener in random order (or better yet in a long series of blocks
of n trials presented in random order within blocks, or some more
sophisticated counter-balancing scheme), then there should be no
correlation between the responses to two of these conditions. Of course,
if there are large effects of learning, then the effect of C[i] on
C[i+1] in the first block is confounded with relatively poor
performance. Finding a remedy to this requires a bit further thought.

If you are really worried about the effect of certain correlations on
whether you are falsely rejecting H[0], you could, after collecting your
data, run your ANOVA by performing Randomization Tests as described by
E. S. Edgington in his book, published by M. Dekker.

All in all, don't we always looking for patterns in our data, and hope
that different observers will show the same pattern? We never do
psychophysical studies in which we are uninterested in generalizing
(unless we are focused on the individual rather than on theory).

Finally, excessive concern with null-hypothesis statistical tests can be
detrimental to the health of our enterprise. I recommend a diet of
Exploratory Data Analysis. In particular, since your concern is ANOVA,
take a look at Hoaglin, Mosteller & Tukey, Fundamentals of Exploratory
Analysis of Variance (Wiley).
--
|\  /|  / Michael Kubovy, Dept. of Psychology, Univ. of Virginia
| \/ | /  P.O.Box 400400, Charlottesville, VA 22904-4400
|    |/\  office (B011): 804-982-4729, lab (B019): -4751
|    |  \ Dept. fax: -4766; personal fax: 240-218-2334
|    |   \www.virginia.edu/~mklab/; FTP: ftp.virginia.edu/pub/mk9y