Subject: Re: perceptual audio evaluation tests From: Emmanuel Vincent <emmanuel.vincent@xxxxxxxx> Date: Tue, 16 Dec 2014 09:30:37 +0100 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>Dear Mark, As far as I know, there is no standard test that is perfectly suitable for source separation, as these tests were designed with other applications in mind (coding, denoising). We proposed a new subjective test derived from MUSHRA along with a corresponding objective measure and some code in Valentin Emiya, Emmanuel Vincent, Niklas Harlander, Volker Hohmann, "Subjective and objective quality assessment of audio source separation", IEEE Transactions on Audio, Speech and Language Processing, 19(7):2046-2057, 2011. https://hal.inria.fr/inria-00567152 We will be happy if this is used as a starting point towards a standardized test in the future. Best, Emmanuel -- Emmanuel Vincent Multispeech Project-Team Inria Nancy - Grand Est 615 rue du Jardin Botanique, 54600 Villers-lès-Nancy, France Phone: +33 3 8359 3083 - Fax: +33 3 8327 8319 Web: http://www.loria.fr/~evincent/ On 12/15/2014 07:20 PM, Mark Cartwright wrote: > Hello, > > I'm looking for the list's opinions on perceptual audio evaluation > listening tests for signals that have large impairments. In particular, > I'm primarily interested in the evaluation of the output of source > separation algorithms. What standardized tests do people recommend (e.g. > ITU-R BS.1534-2 / MUSRHA, ITU-T P.800, etc.) and what are their pros > and cons? Also, are there other tests that are preferred over these but > have not yet been standardized? > > Thanks! > > Mark > > -- > Mark Cartwright <mcartwright@xxxxxxxx > <mailto:mcartwright@xxxxxxxx>> > PhD Candidate in Computer Science > Interactive Audio Lab <http://music.cs.northwestern.edu/>, Northwestern > University > www.markcartwright.com <http://www.markcartwright.com/> >