Subject: Re: perceptual audio evaluation tests From: "James W. Beauchamp" <jwbeauch@xxxxxxxx> Date: Tue, 16 Dec 2014 18:23:01 -0600 List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>For source-separation algorithms, most researchers have started with mixtures where the separate tracks are completely known -- i.e., the "ground truth". Then it's just a matter of comparing the separated signal to original signal. The rms difference between the two can be computed, for example. Judging the quality without knowing the original tracks would perhaps be best done by human juries, although I know there are programs such as PSNR, PAQM and PEAQ available. I think human perceived quality would be the best in the long run, although if a program could match this perception, it would certaily be a lot more efficient. Jim Mark Carwright wrote: >From: Mark Cartwright <mcartwright@xxxxxxxx> >Date: Mon, 15 Dec 2014 12:20:13 -0600 >To: AUDITORY@xxxxxxxx >Subject: perceptual audio evaluation tests > >Hello, > >I'm looking for the list's opinions on perceptual audio evaluation >listening tests for signals that have large impairments. In particular, >I'm primarily interested in the evaluation of the output of source >separation algorithms. What standardized tests do people recommend (e.g. >ITU-R BS.1534-2 / MUSRHA, ITU-T P.800, etc.) and what are their pros and >cons? Also, are there other tests that are preferred over these but have >not yet been standardized? > >Thanks! > >Mark > >-- >Mark Cartwright <mcartwright@xxxxxxxx> >PhD Candidate in Computer Science >Interactive Audio Lab <http://music.cs.northwestern.edu/>, Northwestern >University >www.markcartwright.com