| Hi Bruno, Sabine, Daniel, and all,
Thanks for the clarification. I was indeed wondering how the MUSHRA approach would deal with multidimensionality, but since several people had independently suggested it I figured I just needed to read more to see how it was done…
I think for now we will indeed probably stick with pairwise dissimilarity ratings to start with with using a small sample of <=20 stimuli, while also thinking about other possible options…
Cheers, Pat
Dear Pat,
in addition to what pointed out by Daniel, I personally favour pairwise dissimilarity ratings over sorting unless the number of stimuli is so large that it is not possible to acquire a full dissimilarity matrix in one humane experimental session (<= ~40 stimuli). As you might have read, dissimilarity ratings produces estimates of the distances that are much more reliable (because of the larger number of stimulus playback involved, I suppose), much less distorted (cf. binarization of dissimilarities in free sorting and skewed distribution of hierarchical sorting dissimilarities), and much more indicative of stimulus features than the more efficient sorting methods.
Alternative methods come to mind that rely on the placement of stimuli on a visual space and consider the inter-stimulus distances as estimates of the perceptual dissimilarities (e.g., Harbke, 2003; Kriegeskorte and Mur, 2012). Importantly and unsurprisingly, these "direct MDS" methods bias the perceptual space towards a 2D representation (see Harbke, 2003) and for this reason are a suboptimal choice for the discovery of perceptually relevant stimulus features.
In short, there is no free meal in the behavioural estimation of distances: if your goal is accuracy, methods that are less efficient from the time allocation point of view are, in my opinion, still the best option.
Best,
Bruno
@MastersThesis{harbke2003evaluation, author = {C. R. Harbke}, title = {Evaluation of data collection techniques for multidimensional scaling with large stimulus sets}, school = {Washington State University, Department of Psychology}, year = {2003}, }
@article{kriegeskorte2012inverse, title={Inverse MDS: Inferring dissimilarity structure from multiple item arrangements}, author={Kriegeskorte, Nikolaus and Mur, Marieke}, journal={Frontiers in psychology}, volume={3}, pages={245}, year={2012}, publisher={Frontiers} }
|