Re: Confidence ratings in identification

["Bruno L. Giordano" <bruno.giordano@xxxxxxxxxxxxxxx>]

Dear Daniel,

Macmillan & Creelman (2005) give only one reference to a Choice Theory 
computation of parameters from identification proportions, but not from 
confidence ratings; confidence ratings are treated extensively only with 
respect to the binary case.

Also, one reference is given to a jackknife procedure for the pooling of 
rating data, but the effects of pooling are not discussed as it is 
instead done for the pooling of proportions or sensitivity measures.

My stimuli are multidimensional, as multiple phonemes might be. Maybe 
somebody in the speech area read of recent ROC investigations that suit 
my needs.

Thanks for pointing out the medical literature, and for the reference.

Regards,

	Bruno



Daniel Oberfeld wrote:
> Dear Bruno,
>
> I'm sure you can find the answers to your questions in
>
> Macmillan, N.A., & Creelman, C. D. (2005). Detection Theory. A User's 
> Guide. 2nd edition. Mawah: Erlbaum.
>
> Could you give some more details about the stimuli you want to use, then 
> it will be easier to point out studies related to your questions.
>
> Just a few general notes:
>
> I'd suggest considering the area under the ROC curve as a measure of 
> sensitivity because it relies on fewer assumptions than d' (Macmillan & 
> Creelman discuss this in detail). The area under the ROC curve is widely 
> used in medical decision making (e.g., a radiologist looks at mammograms 
> and classifies them as normal, benign, probably benign, suspicious, or 
> malignant), so it might be interesting to consider this body of 
> literature. Specifically, models have been formulated which take into 
> account the data from several observers (for a recent review see 
> Obuchovski, N. A. (2007). New methodological tools for multiple-reader 
> ROC studies. Radiology, 243, 10-12). Alternatively, Macmillan and 
> Creelman discuss the pooling of observations.
>
> Kind regards,
>
> Daniel