Re: perceptual audio evaluation tests

["James W. Beauchamp" <jwbeauch@xxxxxxxxxxxxxxxxxxxxxx>]

For source-separation algorithms, most researchers have started with 
mixtures where the separate tracks are completely known -- i.e., the 
"ground truth". Then it's just a matter of comparing the separated
signal to original signal. The rms difference between the two can be
computed, for example.

Judging the quality without knowing the original tracks would perhaps
be best done by human juries, although I know there are programs such
as PSNR, PAQM and PEAQ available. I think human perceived quality
would be the best in the long run, although if a program could match
this perception, it would certaily be a lot more efficient.

Jim

Mark Carwright wrote:
>From: Mark Cartwright <mcartwright@xxxxxxxxx>
>Date: Mon, 15 Dec 2014 12:20:13 -0600
>To: AUDITORY@xxxxxxxxxxxxxxx
>Subject: perceptual audio evaluation tests
>
>Hello,
>
>I'm looking for the list's opinions on perceptual audio evaluation
>listening tests for signals that have large impairments. In particular,
>I'm primarily interested in the evaluation of the output of source
>separation algorithms. What standardized tests do people recommend (e.g.
>ITU-R BS.1534-2 / MUSRHA, ITU-T P.800, etc.) and what are their pros and
>cons? Also, are there other tests that are preferred over these but have
>not yet been standardized?
>
>Thanks!
>
>Mark
>
>-- 
>Mark Cartwright <mcartwright@xxxxxxxxxxxxxxxxxx>
>PhD Candidate in Computer Science
>Interactive Audio Lab <http://music.cs.northwestern.edu/>, Northwestern
>University
>www.markcartwright.com