Re: within subject comparisons

[Chris Chambers <Chris.Chambers@xxxxxxxxxxxxxxxxx>]

Ward Drennan wrote:
>
> I encountered  these types of difficulties in my dissertation work a few
> years ago. We (Chuck Watson and I) had used Levitt staircase techniques
> to measure spectral-shape discrimination thresholds for a large group of
> listeners and four different spectral profiles. We used long tracking
> histories normally about 2,000 trials. Certainly, if one listener's
> tracking history was completely higher or lower than another listeners'
> tracking history for 2,000 trials, these listeners had different
> thresholds. The same applies within a single listener on two types of
> spectral profiles. But, we only had one estimate of threshold from this
> long history-- the mean of the last 140 reversals. What Chris says is
> true, these are not independent observations since the level visited on
> any trial is dependent on the level visited on the previous trial. Have
> we lost all the statistical power in 2,000 observations?

2000 observations is a lot of observations...
I think you could get an accurate estimate of the variance two ways from
this.
Presumably the 2000 trials were collected over separate sessions, so one
way would have been to collect, for example, 20 staircase tracks of 100
trials and use the SD of the 20 thresholds in the calculation of the SE
(with a test for autocorrelation at lag_1 thrown in for good measure).
Or you could calculate it by using Kollmeier, Gilkey and Sieben's (1988)
method of including the covariance *and* the variance in the SE
estimate.

> To get 'quasi-independent estimates, we invented a method called the
> 'mean of multiple samples' (MMS method). We calculated the mean of 10
> reversals (this is one estimate) then skipped over 20 reversals and
> calculated the mean of the next 10 (a second estimate), etc. From the
> last 1/3 of the history we could get 5 thresholds estimates and have a
> reasonable estimate of within-listener variance and have some
> independent basis for an ANOVA analysis. This is not ideal as we had to
> discard some data, but the threshold estimates were nearly the same as
> the mean of 140 reversals (r=0.96). Note that we didn't selectively
> discard data, we only took an estimate of thresholds for all the
> listeners and spectral profiles at the 5 pre-selected times during
> training.