Re: Gaussian vs uniform noise audibility

[John Hershey <jhershey@xxxxxxxxxxxxxxx>]

> With Gaussian random variables, uncorrelated implies independent.
That is only true for jointly Gaussian random variables.
Even if each marginal variate is Gaussian, the joint distribution need not
be Gaussian, in which case uncorrelated does not imply independence.

Here is a good way to visualize what's going on:   you have some
non-Gaussian distribution of i.i.d. random variables  Now you rotate it
arbitrarily, and look at the marginal distribution on each axis:  it looks
more Gaussian.  The shape of the joint distribution hasn't changed, however
it is no longer aligned with the axes in such a way that it factorizes on
the axes.  The only i.i.d. distribution that factorizes on the axes no
matter how you rotate it is a Gaussian.

Try this in just two dimensions.   Choose say two uniform independent random
variables -- say U(-1,1) ---.  The distribution looks like a square, in this
case.   Now you rotate it so that it looks like a diamond.  The new marginal
distributions will look like a triangle :  the distribution of the sum of
two uniform random variables.  The distribution is uncorrelated no matter
how you rotate it.  However, the joint distribution is not the product of
two triangular distributions.   That would be a kind of pyramid shaped
distribution with a peak at (0,0).  Ours is a diamond-shaped platau. There
are higher-order statistical dependencies at work!



----- Original Message -----
From: "Julius Smith" <jos@CCRMA.STANFORD.EDU>
To: <AUDITORY@LISTS.MCGILL.CA>
Sent: Friday, January 23, 2004 11:04 PM
Subject: Re: Gaussian vs uniform noise audibility


> With Gaussian random variables, uncorrelated implies independent.
>
> At 12:49 PM 1/23/2004, John Hershey wrote:
> >So, according to the central limit theorem, each frequency component,
being
> >a weighted sum of a large number of independent random variables
approaches
> >a Gaussian distribution. However the sums are all over the same
independent
> >random variables, so in general the sums are not independent.  It seems
> >clear, though, that the frequency components are uncorrelated, because
the
> >Fourier transform is orthogonal, and they were assumed to be uncorrelated
in
> >the time domain.  However, unless I'm missing something, if the time
domain
> >distributions are not Gaussian, then the frequency components are in
general
> >not jointly Gaussian, despite being individually Gaussian and being
> >uncorrelated.  Lack of correlation is necessary but not sufficient for
> >independence, so in general there still may be higher-order statistical
> >dependencies between the frequency components.
> >
> >
> >----- Original Message -----
> >From: "Julius Smith" <jos@CCRMA.STANFORD.EDU>
> >To: <AUDITORY@LISTS.MCGILL.CA>
> >Sent: Friday, January 23, 2004 11:11 AM
> >Subject: Re: Gaussian vs uniform noise audibility
> >
> >
> > > I am surprised nobody seems to have mentioned the central limit
theorem
> > > which shows that the sum of random variables from most any
distribution
> > > (including uniform) converges to a Gaussian random variable.  As a
result,
> > > the Fourier transform of almost any type of stationary random process
> > > yields a set of iid complex Gaussian random variables.  On a more
> >practical
> > > level, two spectral samples from a (finite-length) FFT can be regarded
as
> > > independent as long as they are separated by at least one "resolution
> >cell"
> > > --- i.e., the "band slices" they represent do not overlap
> > > significantly.  For a rectangular window, the width of a resolution
cell
> > > can be defined conservatively as twice the sampling rate divided by
the
> > > window length.  For Hamming and Hann windows, it's double that of the
> > > rectangular window, Blackman three times, and so on.
> > >
> > > In summary, any time a noise process has been heavily filtered, it can
be
> > > regarded as approximately Gaussian, by the central limit theorem, and
> > > disjoint spectral regions are statistically independent.
> > >
> > > -- Julius
> > >
> > > Reference:
> > >
>
> _____________________________
> Julius O. Smith III <jos@ccrma.stanford.edu>
> Assoc. Prof. of Music and (by courtesy) Electrical Engineering
> CCRMA, Stanford University
> http://www-ccrma.stanford.edu/~jos/
>