Subject: Re: [AUDITORY] Measuring perceptual similarity From: Pat Savage <patsavagenz@xxxxxxxx> Date: Tue, 20 Mar 2018 14:19:08 +0900 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>--Apple-Mail=_84096475-33D5-452A-9A99-97FD16FD3A71 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Dear list, Thanks very much for all of your responses. I=E2=80=99m summarizing = below all the reference recommendations I received.=20 I still want to more fully read some of these, but so far my impression = is that the Giordano et al. (2011) paper gives a good review of the = benefits and drawbacks of previous methods, but since that was published = MUSHRA seems to have become the standard method for these types of = subjective perceptual similarity ratings. Please let me know if I seem to be misunderstanding anything here. Cheers, Pat=20 -- Flexer, A., & Grill, T. (2016). The Problem of Limited Inter-rater = Agreement in Modelling Music Similarity. Journal of New Music Research, = 45(3), 1=E2=80=9313. P. Susini, S. McAdams, S. Winsberg: A multidimensional technique for = sound quality assessment. Acta Acustica united with Acustica 85 (1999), = 650=E2=80=93656. Novello, A., McKinney, M. F., & Kohlrausch, A. (2006). Perceptual = evaluation of music similarity. In Proceedings of the 7th International = Conference on Music Information Retrieval. Retrieved from = http://ismir2006.ismir.net/PAPERS/ISMIR06148_Paper.pdf = <http://ismir2006.ismir.net/PAPERS/ISMIR06148_Paper.pdf> Michaud, P. Y., Meunier, S., Herzog, P., Lavandier, M., & D=E2=80=99Aubign= y, G. D. (2013). Perceptual evaluation of dissimilarity between auditory = stimuli: An alternative to the paired comparison. Acta Acustica United = with Acustica, 99(5), 806=E2=80=93815. Wolff, D., & Weyde, T. (2011). Adapting Metrics for Music Similarity = Using Comparative Ratings. 12th International Society for Music = Information Retrieval Conference (ISMIR=E2=80=9911), Proc., (Ismir), = 73=E2=80=9378.=20 B. L. Giordano, C. Guastavino, E. Murphy, M. Ogg, B. K. Smith, S. = McAdams: Comparison of methods for collecting and modeling dissimilarity = data: Applications to complex sound stimuli. Multivariate Behavioral = Research 46 (2011), 779=E2=80=93811. P. Y. Michaud, S. Meunier, P. Herzog, M. Lavandier, G. d=E2=80=99Aubigny: = Perceptual evaluation of dissimilarity between auditory stimuli: an = alternative to the paired comparison. Acta Acustica united with = Acustica 99 (2013), 806=E2=80=93815. Collett, E., Marx, M., Gaillard, P., Roby, B., Fraysse, B., & Deguine, = O. (2016). Categorization of common sounds by cochlear implanted and = normal hearing adults. Hearing Research, 335, 207=E2=80=93219. International Telecommunication Union. (2015). ITU-R BS.1534-3, Method = for the subjective assessment of intermediate quality level of audio = systems. ITU-R Recommendation, 1534=E2=80=933, 1534=E2=80=933. Retrieved = from = https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-3-201510-I!!PDF-= E.pdf = <https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-3-201510-I!!PDF= -E.pdf> Lavandier, M., Meunier, S., & Herzog, P. (2008). Identification of some = perceptual dimensions underlying loudspeaker dissimilarities. Journal of = the Acoustical Society of America, 123(6), 4186=E2=80=934198.=20 DZHAFAROV, E. N., & OLDENBURG, H. C. (2006). RECONSTRUCTING DISTANCES = AMONG OBJECTS FROM THEIR DISCRIMINABILITY. Psychometrika, 71(2), = 365=E2=80=93386. Software:=20 Various: = http://www.ak.tu-berlin.de/menue/digitale_ressourcen/research_tools/whispe= r_listening_test_toolbox_for_matlab/ = <http://www.ak.tu-berlin.de/menue/digitale_ressourcen/research_tools/whisp= er_listening_test_toolbox_for_matlab/> MUSHRA: https://github.com/audiolabs/webMUSHRA = <https://github.com/audiolabs/webMUSHRA> Free-sorting: http://petra.univ-tlse2.fr/spip.php?rubrique49&lang=3Den = <http://petra.univ-tlse2.fr/spip.php?rubrique49&lang=3Den> --- Dr. Patrick Savage Project Associate Professor Faculty of Environment and Information Studies Keio University SFC (Shonan Fujisawa Campus) http://PatrickESavage.com <http://patrickesavage.com/>= --Apple-Mail=_84096475-33D5-452A-9A99-97FD16FD3A71 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = -webkit-line-break: after-white-space;" class=3D"">Dear list,<div = class=3D""><br class=3D""></div><div class=3D"">Thanks very much for all = of your responses. I=E2=80=99m summarizing below all the reference = recommendations I received. </div><div class=3D""><br = class=3D""></div><div class=3D"">I still want to more fully read some of = these, but so far my impression is that the Giordano et al. (2011) paper = gives a good review of the benefits and drawbacks of previous methods, = but since that was published MUSHRA seems to have become the standard = method for these types of subjective perceptual similarity = ratings.</div><div class=3D""><br class=3D""></div><div class=3D"">Please = let me know if I seem to be misunderstanding anything here.</div><div = class=3D""><br class=3D""></div><div class=3D"">Cheers,</div><div = class=3D"">Pat </div><div class=3D"">--</div><div class=3D""><p = class=3D"" style=3D"margin-left: 24pt; text-indent: -24pt;"><span = class=3D"" style=3D"text-indent: -24pt;">Flexer, A., & Grill, T. = (2016). The Problem of Limited Inter-rater Agreement in Modelling Music = Similarity. </span><i class=3D"" style=3D"text-indent: = -24pt;">Journal of New Music Research</i><span class=3D"" = style=3D"text-indent: -24pt;">, </span><i class=3D"" = style=3D"text-indent: -24pt;">45</i><span class=3D"" style=3D"text-indent:= -24pt;">(3), 1=E2=80=9313.</span></p><p class=3D"" style=3D"margin-left: = 24pt; text-indent: -24pt;"><span style=3D"background-color: rgb(255, = 255, 255);" class=3D""><br class=3D""></span></p><p class=3D"" = style=3D"margin-left: 24pt; text-indent: -24pt;"><span = style=3D"background-color: rgb(255, 255, 255);" class=3D"">P. Susini, S. = McAdams, S. Winsberg: A multidimensional technique for sound quality = assessment. Acta Acustica united with Acustica 85 (1999), = 650=E2=80=93656.</span></p><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span class=3D"" style=3D"text-indent: -24pt;"><br = class=3D""></span></p><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span class=3D"" style=3D"text-indent: = -24pt;">Novello, A., McKinney, M. F., & Kohlrausch, A. (2006). = Perceptual evaluation of music similarity. In </span><i class=3D"" = style=3D"text-indent: -24pt;">Proceedings of the 7th International = Conference on Music Information Retrieval</i><span class=3D"" = style=3D"text-indent: -24pt;">. Retrieved from <a = href=3D"http://ismir2006.ismir.net/PAPERS/ISMIR06148_Paper.pdf" = class=3D"">http://ismir2006.ismir.net/PAPERS/ISMIR06148_Paper.pdf</a></spa= n></p></div><div class=3D""><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span class=3D"" style=3D"text-indent: = -24pt;">Michaud, P. Y., Meunier, S., Herzog, P., Lavandier, M., & = D=E2=80=99Aubigny, G. D. (2013). Perceptual evaluation of dissimilarity = between auditory stimuli: An alternative to the paired = comparison. </span><i class=3D"" style=3D"text-indent: -24pt;">Acta = Acustica United with Acustica</i><span class=3D"" style=3D"text-indent: = -24pt;">, </span><i class=3D"" style=3D"text-indent: = -24pt;">99</i><span class=3D"" style=3D"text-indent: -24pt;">(5), = 806=E2=80=93815.</span></p><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span style=3D"text-indent: -24pt;" class=3D""><br = class=3D""></span></p><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span style=3D"text-indent: -24pt;" class=3D"">Wolff,= D., & Weyde, T. (2011). Adapting Metrics for Music Similarity Using = Comparative Ratings. </span><i class=3D"" style=3D"text-indent: = -24pt;">12th International Society for Music Information Retrieval = Conference (ISMIR=E2=80=9911), Proc.</i><span style=3D"text-indent: = -24pt;" class=3D"">, (Ismir), 73=E2=80=9378. </span></p><p class=3D""= style=3D"margin-left: 24pt; text-indent: -24pt;"><span = style=3D"background-color: rgb(255, 255, 255);" class=3D""><br = class=3D""></span></p><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span style=3D"background-color: rgb(255, 255, = 255);" class=3D"">B. L. Giordano, C. Guastavino, E. Murphy, M. Ogg, B. = K. Smith, S. McAdams: Comparison of methods for collecting and modeling = dissimilarity data: Applications to complex sound stimuli. Multivariate = Behavioral Research 46 (2011), 779=E2=80=93811.</span></p><p class=3D"" = style=3D"margin-left: 24pt; text-indent: -24pt;"><span = style=3D"text-indent: 0px; background-color: rgb(255, 255, 255);" = class=3D""><br class=3D""></span></p><p class=3D"" style=3D"margin-left: = 24pt; text-indent: -24pt;"><span style=3D"text-indent: 0px; = background-color: rgb(255, 255, 255);" class=3D"">P. Y. Michaud, S. = Meunier, P. Herzog, M. Lavandier, G. d=E2=80=99Aubigny: Perceptual = evaluation of dissimilarity between auditory stimuli: an alternative to = the paired comparison. Acta Acustica united with Acustica 99 = (2013), 806=E2=80=93815.</span></p><p class=3D"" style=3D"margin-left: = 24pt; text-indent: -24pt;"><span class=3D"" style=3D"text-indent: = -24pt;"><br class=3D""></span></p><p class=3D"" style=3D"margin-left: = 24pt; text-indent: -24pt;"><span class=3D"" style=3D"text-indent: = -24pt;">Collett, E., Marx, M., Gaillard, P., Roby, B., Fraysse, B., = & Deguine, O. (2016). Categorization of common sounds by cochlear = implanted and normal hearing adults. </span><i class=3D"" = style=3D"text-indent: -24pt;">Hearing Research</i><span class=3D"" = style=3D"text-indent: -24pt;">, </span><i class=3D"" = style=3D"text-indent: -24pt;">335</i><span class=3D"" = style=3D"text-indent: -24pt;">, 207=E2=80=93219.</span></p><p class=3D"" = style=3D"margin-left: 24pt; text-indent: -24pt;"><span = style=3D"text-indent: -24pt;" class=3D""><br class=3D""></span></p><p = class=3D"" style=3D"margin-left: 24pt; text-indent: -24pt;"><span = style=3D"text-indent: -24pt;" class=3D"">International Telecommunication = Union. (2015). ITU-R BS.1534-3, Method for the subjective assessment of = intermediate quality level of audio systems. </span><i class=3D"" = style=3D"text-indent: -24pt;">ITU-R Recommendation</i><span = style=3D"text-indent: -24pt;" class=3D"">, </span><i class=3D"" = style=3D"text-indent: -24pt;">1534</i><span style=3D"text-indent: = -24pt;" class=3D"">=E2=80=93</span><i class=3D"" style=3D"text-indent: = -24pt;">3</i><span style=3D"text-indent: -24pt;" class=3D"">, 1534=E2=80=93= 3. Retrieved from </span><a = href=3D"https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-3-201510= -I!!PDF-E.pdf" class=3D"" style=3D"text-indent: = -24pt;">https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-3-201510= -I!!PDF-E.pdf</a></p><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span style=3D"text-indent: -24pt;" class=3D""><br = class=3D""></span></p><p class=3D"" style=3D"margin-left: 24pt; = text-indent: -24pt;"><span style=3D"text-indent: -24pt;" = class=3D"">Lavandier, M., Meunier, S., & Herzog, P. (2008). = Identification of some perceptual dimensions underlying loudspeaker = dissimilarities. </span><i class=3D"" style=3D"text-indent: = -24pt;">Journal of the Acoustical Society of America</i><span = style=3D"text-indent: -24pt;" class=3D"">, </span><i class=3D"" = style=3D"text-indent: -24pt;">123</i><span style=3D"text-indent: -24pt;" = class=3D"">(6), 4186=E2=80=934198. </span></p><p class=3D"" = style=3D"margin-left: 24pt; text-indent: -24pt;"><span = style=3D"text-indent: -24pt;" class=3D""><br class=3D""></span></p><p = class=3D"" style=3D"margin-left: 24pt; text-indent: -24pt;"><span = style=3D"text-indent: -24pt;" class=3D"">DZHAFAROV, E. N., & = OLDENBURG, H. C. (2006). RECONSTRUCTING DISTANCES AMONG OBJECTS FROM = THEIR DISCRIMINABILITY. </span><i class=3D"" style=3D"text-indent: = -24pt;">Psychometrika</i><span style=3D"text-indent: -24pt;" = class=3D"">, </span><i class=3D"" style=3D"text-indent: = -24pt;">71</i><span style=3D"text-indent: -24pt;" class=3D"">(2), = 365=E2=80=93386.</span></p></div><div class=3D""><br class=3D""></div><div= class=3D"">Software: </div><div class=3D""><br class=3D""></div><div= class=3D"">Various:</div><div class=3D""><a = href=3D"http://www.ak.tu-berlin.de/menue/digitale_ressourcen/research_tool= s/whisper_listening_test_toolbox_for_matlab/" class=3D"" style=3D"color: = purple; font-family: Calibri, sans-serif; font-size: = 14.666666984558105px;">http://www.ak.tu-berlin.de/menue/digitale_ressource= n/research_tools/whisper_listening_test_toolbox_for_matlab/</a></div><div = class=3D""><br class=3D""></div><div class=3D"">MUSHRA:</div><div = class=3D""><a href=3D"https://github.com/audiolabs/webMUSHRA" = class=3D"">https://github.com/audiolabs/webMUSHRA</a></div><div = class=3D""><br class=3D""></div><div class=3D"">Free-sorting:</div><div = class=3D""><a class=3D"moz-txt-link-freetext" = href=3D"http://petra.univ-tlse2.fr/spip.php?rubrique49&lang=3Den">http= ://petra.univ-tlse2.fr/spip.php?rubrique49&lang=3Den</a></div><div = class=3D""><div class=3D"">---<br class=3D"">Dr. Patrick Savage<br = class=3D"">Project Associate Professor</div><div class=3D"">Faculty of = Environment and Information Studies</div><div class=3D"">Keio University = SFC (Shonan Fujisawa Campus)<br class=3D""><a = href=3D"http://PatrickESavage.com" = class=3D"">http://PatrickESavage.com</a></div></div></body></html>= --Apple-Mail=_84096475-33D5-452A-9A99-97FD16FD3A71--