Re: Usual settings for the transformed up-down procedure

[Athanassios Protopapas <protopap@xxxxxxx>]

An additional issue concerns the experience of the participants.
Concentration/attention and adaptation/learning can both affect
threshold estimates very severely when participants are not both
trained and motivated to perform the task. This is something that I
have not seen discussed in the literature, which typically compares
the efficiency of different adaptive procedures assuming almost
perfect participants. Our experience with maximum-likelihood-type
procedures was that they can be utterly useless with naive
participants, because a single inattentive response early on throws
off the entire procedure completely. Constant-stimuli (fixed-level)
methods were a waste of time, both in requiring many repetitions and
also in preventing the participants from learning and concentrating on
the fine discriminations, resulting in much higher and less reliable
threshold estimates. Adaptive staircases seem best to "train"
participants to listen to whatever it is you want listened to.  After
trying out several different approaches, we ended up using a
modification of the "accelerated stochastic approximation" (see
Treutwein, B., Adaptive psychophysical procedures, Vision Research,
35, 2503-2522, 2008), which is similar in conception to the 3-up
1-down except that the up step is smaller, thus making convergence
smoother and easier for the participant. The step becomes arbitrarily
small and the procedure can be terminated after a set number of
reversals or when the step becomes smaller than the anticipated
variance. Our modification was to allow the step-size parameter to
fall back to a larger value (by removing one reversal) when there were
too many responses of the same kind ("same kind" being as either all
correct or all incorrect; "too many" means twice as many as expected
by the procedure parameters, ie, for a 3:1 up/down step ratio, 6 up or
2 down steps are "too many"). This allows the procedure to recover
quickly from lucky streaks or inattentive responses that might throw
off tracking of the threshold and delay convergence.  Taking into
account our special population (including children and adults with no
relevant experience, poor auditory skills and poor concentration),
this modification served remarkably well. I'd say, if you throw out a
quick first run (serving to familiarize the participant with the
sounds and task) and then take three threshold runs with this
procedure, you should be able to get stable and reliable estimates. On
the other hand, if your participants are of the standard type seen in
psychophysical experiments, well-trained and fully focused, then more
efficient methods may work for you.


On Fri, May 29, 2009 at 11:36 PM, Peter Marvit <pmarvit@xxxxxxxxx> wrote:
> Tooting my own aging horn:
>
> Marvit, P., Florentine, M., and Buus, S. (2003). "A comparison of
> psychophysical procedures for level-discrimination thresholds," Journal of
> the Acoustical Society of America 113, 3348-3361.
>
> My bottom line interpretation from that study, apart from the specific
> procedures: More samples/estimates may provide better accuracy for threshold
> estimates, rather than trying to be more precise in a single session.
> Session-to-session (day-to-day) variance is much larger than within session
> (i.e., during any particular track). Thus, taking many "imprecise" but short
> estimates gives better overall results than a few very long (e.g.,
> interleaved) but supposedly precise measurements.
>
> Personally, I also like the algorithms developed by Chris Kaernbach, who
> graciously outlined them in a prior message. Of course, there are now large
> numbers of adaptive procedures from which to choose. The old 2-down, 1-up
> has the greatest virtue in simplicity of understanding and implementation,
> but is far from optimal in practicaly application.
>
> Chris' message had an important point: Making the procedure easy and
> comfortable for your subjects will do more for your data than optimizing
> theoretical efficiency. His "I don't know" response works well in that
> regard, in addition to providing computational data. I've found many
> subjects who really liked working at P(correct) 85% (or more) and got quite
> anxious at 71%. Likewise, my subjects generally liked short runs MUCH more
> than long ones.
>
> Good luck!
> Peter
>
>
> : Peter Marvit, PhD
> :<pmarvit at gmail dot com OR pmarvit at psyc dot umd dot edu>
> : Psychology Department, University of Maryland, College Park
> : College Park, MD 20742
> : (lab) 301-405-5940    (fax) 301-314-9566
>
>
> On Fri, May 29, 2009 at 10:08 AM, Sylvain BUSSON <Sylvain.BUSSON@xxxxxxxxxx>
> wrote:
>>
>> The adaptive up-dow procedure offers many benefits over methode of
>> constants.
>> Does anyone know if one of theses benefits is related to the number of
>> repetition for each condition ?
>> In other words, can we get a reliable threshold estimate with only 3
>> repetitions (3 tracks for the same condition)  ?
>
>