Subject: Re: Gaussian vs uniform noise audibility From: John Hershey <jhershey(at)COGSCI.UCSD.EDU> Date: Sat, 24 Jan 2004 09:42:22 -0800> With Gaussian random variables, uncorrelated implies independent. That is only true for jointly Gaussian random variables. Even if each marginal variate is Gaussian, the joint distribution need not be Gaussian, in which case uncorrelated does not imply independence. Here is a good way to visualize what's going on: you have some non-Gaussian distribution of i.i.d. random variables Now you rotate it arbitrarily, and look at the marginal distribution on each axis: it looks more Gaussian. The shape of the joint distribution hasn't changed, however it is no longer aligned with the axes in such a way that it factorizes on the axes. The only i.i.d. distribution that factorizes on the axes no matter how you rotate it is a Gaussian. Try this in just two dimensions. Choose say two uniform independent random variables -- say U(-1,1) ---. The distribution looks like a square, in this case. Now you rotate it so that it looks like a diamond. The new marginal distributions will look like a triangle : the distribution of the sum of two uniform random variables. The distribution is uncorrelated no matter how you rotate it. However, the joint distribution is not the product of two triangular distributions. That would be a kind of pyramid shaped distribution with a peak at (0,0). Ours is a diamond-shaped platau. There are higher-order statistical dependencies at work! ----- Original Message ----- From: "Julius Smith" <jos(at)CCRMA.STANFORD.EDU> To: <AUDITORY(at)LISTS.MCGILL.CA> Sent: Friday, January 23, 2004 11:04 PM Subject: Re: Gaussian vs uniform noise audibility > With Gaussian random variables, uncorrelated implies independent. > > At 12:49 PM 1/23/2004, John Hershey wrote: > >So, according to the central limit theorem, each frequency component, being > >a weighted sum of a large number of independent random variables approaches > >a Gaussian distribution. However the sums are all over the same independent > >random variables, so in general the sums are not independent. It seems > >clear, though, that the frequency components are uncorrelated, because the > >Fourier transform is orthogonal, and they were assumed to be uncorrelated in > >the time domain. However, unless I'm missing something, if the time domain > >distributions are not Gaussian, then the frequency components are in general > >not jointly Gaussian, despite being individually Gaussian and being > >uncorrelated. Lack of correlation is necessary but not sufficient for > >independence, so in general there still may be higher-order statistical > >dependencies between the frequency components. > > > > > >----- Original Message ----- > >From: "Julius Smith" <jos(at)CCRMA.STANFORD.EDU> > >To: <AUDITORY(at)LISTS.MCGILL.CA> > >Sent: Friday, January 23, 2004 11:11 AM > >Subject: Re: Gaussian vs uniform noise audibility > > > > > > > I am surprised nobody seems to have mentioned the central limit theorem > > > which shows that the sum of random variables from most any distribution > > > (including uniform) converges to a Gaussian random variable. As a result, > > > the Fourier transform of almost any type of stationary random process > > > yields a set of iid complex Gaussian random variables. On a more > >practical > > > level, two spectral samples from a (finite-length) FFT can be regarded as > > > independent as long as they are separated by at least one "resolution > >cell" > > > --- i.e., the "band slices" they represent do not overlap > > > significantly. For a rectangular window, the width of a resolution cell > > > can be defined conservatively as twice the sampling rate divided by the > > > window length. For Hamming and Hann windows, it's double that of the > > > rectangular window, Blackman three times, and so on. > > > > > > In summary, any time a noise process has been heavily filtered, it can be > > > regarded as approximately Gaussian, by the central limit theorem, and > > > disjoint spectral regions are statistically independent. > > > > > > -- Julius > > > > > > Reference: > > > > > _____________________________ > Julius O. Smith III <jos(at)ccrma.stanford.edu> > Assoc. Prof. of Music and (by courtesy) Electrical Engineering > CCRMA, Stanford University > http://www-ccrma.stanford.edu/~jos/ >