Subject: Re: An Auditory Illusion From: DeLiang Wang <dwang(at)CIS.OHIO-STATE.EDU> Date: Tue, 20 May 1997 10:21:12 -0400Dear Auditory list, Al writes: > Since one table is red, the firing of RED and TABLE must be synchronized. > Since the other table is blue, BLUE and TABLE must also be synchronized. > Following the argument to completion implies that all four concepts must > be synchronized. What we need is two separate instantiations of RED, one > bound to TABLE, the other bound to BALL. This isn't really a problem for the oscillatory framework. The idea is the following: to have an oscillator assembly corresponding to TABLE oscillate with double frequency so that this assembly synchronizes with both the BLUE assembly and the RED assembly. At the same time the synchronized assembly for BLUE TABLE and that for RED TABLE are desynchronized. The above idea was used in an early paper on oscillatory associative memory (Wang, Buhmann, and von der Malsburg, 1990, refs. below) to handle the overlap problem, essentially the same as the above problem. But the oscillator model used there proves to be too clumpsy. A recent model by Brown and Wang (1996) explicitly addressed the problem of "duplex perception" in audition using the same idea. But the Brown/Wang model is based on relaxation oscillator networks, which have an elegant theory and computational properties behind (see below). Christian Kaernbach writes > It woul be really better to ask people of the von der Malsburg group > how they deal with this problem, but as far as I know their theory it > works well even for a male voice at the left ear and a female voice > at the right ear saying the same word ("happy"). The first ideas > about synchronisation were associated with oscillations (e.g. 40 Hz > in cats visual cortex), but this is no longer up to date. If people > still model synchrony with oscillations, they use at least chaotic > oscillators which show a multitude of limit cycles depending on their > stimulation. My group has been working on oscillatory correlation for years. Oscillatory correlation provides a rich representation, which appears sufficient for a variety of perceptual tasks. But representing a solution is one thing, computing the solution is another. So far the discussion is focussed on the former, but the latter is often proven to be more difficult. We have made quite a progress in computations using locally excitatory global inhibitory relaxation oscillators, which admit an elegant computational theory. See Wang (1996) for an auditory account, and Brown/Wang for modeling double vowel separation. For vision, a paper just came out of the last issue of Neural Computation (Wang and Terman, 1997). The use of chaotic oscillators has not proven to be productive presumably because of the computational difficulty associated. Perhaps Dr. Kaernbach can elaborate on the last statement in the above quote? > Many models don't use osscilations at all. If you think > about non-periodic synchrony, you can imagine many different ways how > to implement perfect synchrony, partial synchrony, no synchrony, > antisynchrony, and even net-like correlations that could very well > code the above situations. Node MALE could well be in synchrony with > node LEFT and with node HAPPY, and node FEMALE with node RIGHT and > node HAPPY, without nodes MALE/LEFT and nodes RIGHT/FEMALE being in > synchrony. Node HAPPY could, e.g. fire with a 60-Hz cycle, nodes > RIGHT/FEMALE firing with approx. 30-Hz bursts and taking every even > burst, and nodes MALE/LEFT taking every odd burst. And this > explanation was on the basis of periodic osscillations. Nonperiodic > synchrony could probably do even better. I am well aware that the > correlation theory is not apt to solve all binding problems today. > The upoint I wanted to make is that one can imagine several > mechanisms how node based systems could deal with complicated binding > problems. It is another job to prove that this is the way it is done. The above point is similar to what we actually modeled. Again, imagining is one thing, computing is quite another. Perhaps a little history on oscillatory modeling could help here. Much is drawn on visual modeling, where experimental evidence is most comprehensive (some from the auditory system too). The idea of using oscillators for binding was first proposed by P. Milner in 1974 (one could find close speculations in Hebb's well-known book of 1949), and systematically proposed and advocated by C. von der Malsburg since 1981. Since the discovery of coherent oscillations in the visual cortex in late 80s (Eckhorn's group published in 1988, and Singer's group published in 1989), there are perhaps more than 100 papers published on modeling oscillations and on building systems for image analysis. A persistent stumbling block has been the lack of a reliable computational mechanism for achieving rapid synchronization based on local coupling (important for encoding topology), and rapid desynchronization in the presence of multiple organizations (streams in audition). The problem was recently resolved, and thus there is ground for optimism for making substantial progress in this direction. This again illustrates the importance of computational mechanism. The references listed below contain more comprehensive accounts. DeLiang Wang ------------ References: Brown G.J. and Wang D.L. (1996): Modelling the perceptual segregation of double vowels with a network of neural oscillators. Technical report CS-96-06, Sheffield Univ. Computer Sci. Dept. (available by anonymous FTP from the site "ftp.dcs.shef.ac.uk", followed by the commands: "cd /share/spandh/pubs/brown" and "get bw-report96.ps.Z"). A revised version to appear in Neural Networks. Wang D.L. (1996): Primitive auditory segregation based on oscillatory correlatoin. Cognitive Science, vol. 20, 409-456. (available on my web "http://www.cis.ohio-state.edu/~dwang") Wang D.L. and Terman D. (1997): Image segmentation based on oscillatory correlation. Neural Computation, vol. 9, 805-836.