Re: identification task...negative d' values (Richard Pastore )

Subject: Re: identification task...negative d' values From: Richard Pastore <pastore@xxxxxxxx> Date: Thu, 4 May 2006 13:41:16 -0400 --=====================_5385578==_.ALT Content-Type: text/plain; charset="us-ascii"; format=flowed Dear List: silvadmr first wrote: "I have some few small negative values of d' obtained from an identification task in which subjects label vowel sounds from a continuum with three phoneme labels. " silvadmr then wrote: "I have imagined two solutions: "1. take only the d's obtained from the probabilities of responding with the label corresponding to the category in the middle of the continuum - "o". "2. simply split the results in two at an arbitrary point in the middle of the '"o" category' Looking over the many responses illustrates the problems in trying to do research through internet discussion. First look at what is being asked: silvadmr has a few small negative values. Note the words "few" and "small negative." Keep in mind that there is not a rigid threshold at d' = 0. If subjects cannot do the task, the average value of d' may be 0, but some subjects will exhibit small positive and some small negative values of d' . Thus, the extensive discussion about the negative values seems to miss one of many greater concerns. Doug Creelman is right on in pointing out some considerations, then concluding, "This is only the beginning of what could be an extensive discussion. Is your continuum really unidimensional?" silvadmr, and some of the others responding, seem to want to simply force fit analyses of the superficially described research with impoverished notions from the detection version of the Gaussian Equal Variance Model of SDT. I've seen, cringed at, and done some modeling with data from others who have tried using silvadmr's two solutions. As Doug implies, more than a simple, force-fit is needed. What follows simply points out a few of the problems. Let's assume (not necessarily valid) that the stimulus set maps into a unidimensional decision continuum. Our goal, then, is to compute a z-score based descriptive statistic (d') that reflects the statistical separation of perceptual categories for each subject. Think about what one is doing when looking up a z-score value. One can look up the z-score distance for a probability that represents the logical bisection of a distribution; the z-score is the distance from the mean to the logical partition point of the distribution. Now apply this idea. Let's assume that we use 4 ordinal response categories, where 1 and 4 are used to indicate the decision reflecting strong judgment of the two extremes of the unidimensional continuum, with 2 & 3 representing the partitioning of those intermediate values in terms of ordered relationship to the end points of the continuum. This is essentially the Rating Task that sometimes is used to generate ROC curves. One thus has 3 ordered criteria: 1 vs (2+3+4), (1+2) vs (3+4), and (1+2+3) vs 4. One can compute the z score distances for any given stimulus (i.e., repeated presentations of stimulus n), and thus the z-score distances between the means for statistical distributions for any given pair of stimuli (stimuli n and n+j), and this latter gives us d'. Does it make sense, however, to try to look up the "z-score" value for the area under a Normal distribution that is between two arbitrary points located somewhere within the distribution (e.g., those that represent response 2, or the grouping of responses 2 and 3)? Note that I puts quotes around "z-score", since what I have just described has no logical meaning. Yes, the probability of response "1" specifies a meaningful z-score, as does the probability of response "4", but the probability of responses "2 or 3" can have any value between minus and plus infinity. If what one computes as a nominal z-score is invalid, then so is any nominal descriptive statistic based upon that computation. What I have just described is equivalent to using the middle of three ordered categories, and this is what silvadmr has described as the first solution (see above). Does it make any more sense to arbitrarily partition the data in the middle of this middle category - silvadmr's alternative solution. I have provided an explanation of a logical problem with what silvadmr has proposed. However, as Doug noted, this is only the beginning of what would need to be an extensive discussion - one that is far too complex for a few simple statements on a web chat site, when an understanding is needed of the material in one of the books by Macmillan & Creelman, Wicken, McNichol, or Green & Swets. Dick Pastore Richard E. Pastore Professor of Psychology Psychology Department Binghamton University (SUNY) Binghamton, NY 13902-6000 http://bingweb.binghamton.edu/~pastore --=====================_5385578==_.ALT Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by drizzle.cc.mcgill.ca id k44HfKsL015520 <html> Dear List: silvadmr first wrote:  =93I have some few small neg= ative values of d' obtained from an identification task in which subjects label vowel sounds from a continuum with three phoneme labels. =93 silvadmr<fon= t face=3D"Verdana" size=3D1> then wrote: =93I have imagined = two solutions: =931. take only the d's obtained from the probabilities of responding with the label corresponding to the category in the middle of the continuum - "o". =932. simply split the results in two at an arbitrary point in the middle of the '"o" category' Looking over the many responses il= lustrates the problems in trying to do research through internet discussi= on. First look at what is being asked:     silvadmr has a few small negative values.  Note t= he words =93few=94 and =93small negative.=94 = Keep in mind that there is not a rigid threshold at d=92 =3D 0.  If = subjects cannot do the task, the average value of d=92 may be 0, but some= subjects will exhibit small positive and some small negative values of d= =92 .   Thus, the extensive discussion about the negative value= s seems to miss one of many greater concerns.  Doug Creelman is right on in pointing out some considerations, then concl= uding, =93This is only the= beginning of what could be an extensive discussion. Is your continuum re= ally unidimensional?=94 silvadmr, and some of the others r= esponding, seem to want to simply  force fit analyses of the superfi= cially described research with impoverished notions from the detection ve= rsion of the Gaussian Equal Variance Model of SDT.  I=92ve seen, cri= nged at, and done some modeling with data from others who have tried usin= g silvadmr=92s two solutions.  As Doug implies, more than a simple, = force-fit is needed.  What follows simply points out a few of the pr= oblems. Let=92s assume (not necessarily valid) that the stimulus set maps into a = unidimensional decision continuum.   Our goal, then, is to comp= ute a z-score based descriptive statistic (d') that reflects the statisti= cal separation of perceptual categories for each subject.  Think abo= ut what one is doing when looking up a z-score value.  One can look = up the z-score distance for a probability that represents the logical bis= ection of a distribution; the z-score is the distance from the mean to th= e logical partition point of the distribution.  Now apply this idea.  Let=92s assume that we use 4 ordinal response = categories, where 1 and 4 are used to indicate the decision reflecting st= rong judgment of the two extremes of the unidimensional continuum, with 2= & 3 representing the partitioning of those intermediate values in te= rms of ordered relationship to the end points of the continuum. &nbs= p; This is essentially the Rating Task that sometimes is used to generate= ROC curves.  One thus has 3 ordered criteria:  1  vs (2+3= +4), (1+2) vs (3+4), and (1+2+3) vs 4.  One can compute the z score = distances for any given stimulus (i.e., repeated presentations of stimulu= s n), and thus the z-score distances between the means for statistical di= stributions for any given pair of stimuli (stimuli n and n+j), and this l= atter gives us d=92.  Does it make sense, however, to try to look up the =93z-score=94 value fo= r the area under a Normal distribution that is between two arbitrary poin= ts located somewhere within the distribution (e.g., those that represent = response 2, or the grouping of responses 2 and 3)?   Note that = I puts quotes around =93z-score=94, since what I have just described has = no logical meaning.  Yes, the probability of response =931=94 specif= ies a meaningful z-score, as does the probability of response =934=94, bu= t the probability of responses =932 or 3=94 can have any value between mi= nus and plus infinity.  If what one computes as a nominal z-score is= invalid, then so is any nominal descriptive statistic based upon that co= mputation.  What I have just described is equivalent to using the mi= ddle of three ordered categories, and this is what silvadmr has described= as the first solution (see above). Does it make any more sense to arbitrarily partition the data in the midd= le of this middle category  - silvadmr=92s alternative solution.&nbs= p; I have provided an explanation of a logical problem with what silvadmr ha= s proposed.  However, as Doug noted, this is only the beginning of w= hat would need to be an extensive discussion - one that is far too comple= x for a few simple statements on a web chat site, when an understanding i= s needed of the material in one of the books by Macmillan & Creelman,= Wicken, McNichol, or Green & Swets. Dick Pastore <x-sigsep></x-sigsep> Richard E. Pastore   Professor of Psychology   Psychology Department   Binghamton University (SUNY)   Binghamton, NY 13902-6000    <a href=3D"http://bingweb.binghamton.edu/~pastore" eudora=3D= "autourl">http://bingweb.binghamton.edu/~pastore</a> </html> --=====================_5385578==_.ALT--

This message came from the mail archive
http://www.auditory.org/postings/2006/
maintained by:

DAn Ellis <dpwe@ee.columbia.edu>
Electrical Engineering Dept., Columbia University