[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
source separation
This just in from the connectionists mailing list -
- lonce
------------------------------------------------------------------------
Blind Signal Processing is an emerging area in adaptive signal processing
and neural networks.
It was originated in France in the late 80's .
Below please find an advanced program of a special invited session devoted
to blind separation of sources and their applications at 1995 INTERNATIONAL
SYMPOSIUM ON NONLINEAR THEORY AND ITS APPLICATIONS , NOLTA'95 in Las Vegas.
Any comments will be highly appreciated, especially association of this approach
to brain information processing and image and speech enhancement, filtering and
noise reduction.
Andrzej Cichocki,
Head of Laboratory for Artificial Brain Systems,
Frontier Research Program RIKEN,
Institute of Physical and Chemical Research,
Hirosawa 2-1, Saitama 351-01,
WAKO-Schi,
JAPAN
E-mail: cia@kamo.riken.go.jp,
FAX (+81) 048 462 4633.
URL: http://zoo.riken.go.jp/bip.html
===========================================================================
NOLTA'95,
1995 INTERNATIONAL SYMPOSIUM ON
NONLINEAR THEORY AND ITS APPLICATIONS
Caesars Palace, LAS VEGAS
Dec. 10 -14, 1995
Program for Special Invited Session on
"BLIND SEPARATION OF SOURCES
-Brain Information Processing"
Organizer and chair
Dr. A. Cichocki
Frontier Research Program RIKEN,
Institute of Physical and Chemical Research,
Hirosawa 2-1, Saitama 351-01,
WAKO-Schi,
JAPAN
Advanced Program:
1. Prof.Christian JUTTEN , Laboratory TIRF, INPG, Grenoble, FRANCE,
"Separation of Sources: Blind or Unsupervised? "
Abstract:
Basically, separation of sources are referred as BLIND methods. However,
adaptive algorithms for source separation only emphasize on UNSUPERVISED
aspects of the learning. In this talk, we propose a selected review of
recent works to show how A PRIORI KNOWLEDGES on the sources or on the
mixtures can simplify the algorithms and improve performance.
2. Prof. Jean-Francois CARDOSO , Ecole Nationale Superieure des
Telecommunications, Telecom Paris, FRANCE
"The Invariant Approach to Source Separation"
Abstract:
The `invariant approach' to source separation is based on the recognition
that the unknown parameter in a source mixture is the mixing matrix, hence
it belongs to a multiplicative group. In this contribution, we show that
this simple fact can be exploited to build source separation algorithms
behaving uniformly well in the mixing matrix. This is achieved if two
sufficient conditions are met.
+ First, contrast functions (or estimating equations) used to identify the
mixture should be designed in such a way that source separation is achieved
when they are optimized (or solved) **without constraints** (such as
normalization, etc). Examples of such contrast functions will be given,
some of them being simple variants of classic contrast functions. This
requirement is sufficient to guarantee uniform performance of the resulting
batch algorithms.
+ Second, in the case of adaptive algorithm, uniform performance has a
more extensive meaning: not only the residual error but also the
convergence are important. Again, the multiplicative nature of the
parameter calls for a special form of the learning rule, namely it suggests
a `multiplicative update'. This approach results in adaptive source
separation algorithms and enjoying uniform performance: convergence speed,
residual error, stable points, etc... do not depend on the mixing matrix.
In addition these algorithms show a very simple (and parallelizable)
structure.
The paper includes analytical results based on asymptotic performance
analysis that quantify the behavior of both batch and adaptive
equivariant source separators. In particular, these results allow to
determine, given the source distribution, the optimal nonlinearities to
be used in the learning rule.
3. Dr. Jie ZHU, Prof. Xi-Ren CAO, and prof. Ruey-Wen LIU,
The Hong Kong University of Science and Technology Kowloon, Hong Kong,
The University of Notre Dame, Notre Dame, NI 46556, U.S.A.
"Blind Source Separation Based on Output Independence - Theory and
Implementation"
Abstract:
The paper presents some recent results on the theory and implementation
techniques of blind source separation. the approach is based on
independence property of the outputs of a filter.
In the theory part, we identify and study two major issues in the blind
source separation problem: separability and separation principles. We
show that separability is an intrinsic property of the measured signals
and can be described by the concept of $m$-row decomposability introduced
in this paper, and that the separation principles can be developed by
using the structure characterization theory of random variables. In
particular, we show that these principles can be derived concisely and
intuitively by applying the Darmois-Skitovich theorem, which is
well-known in statistical inference theory and psychology.
In the implementation part, we show that if at most one of the source
signals has a zero third (or fourth) order cumulant, then these signals
can be separated by a filter whose parameters can be determined by a
system of nonlinear equations using only third (or fourth) order
cumulants of the measured signals. This results covers some previous
results as special cases.
4. Dr. Jie HUANG, Prof. Noboru OHNISHI and Dr. Noboru Sugie ;
Bio-Mimetic Control Research Center , The Institute of Physical and
Chemical Research (RIKEN), Nagoya, JAPAN
"Sound Separation Based on Perceptual Grouping of Sound Segments"
Abstract :
We would like to propose a sound separation method,
which combines spatial cues (source direction)
and structural cues (continuity and harmony).
Sound separation is important in various scientific fields.
There are mainly two different approaches to achieve this goal.
One is based on blind estimation of inverse transfer functions
from multiple sources to multiple receivers (microphones).
The other is based on grouping sound segments in time-frequency domain.
Our approach is based on the sound segments grouping.
However, we use multiple microphones to obtain the spatial information.
This approach is strongly inspired by the precedence effect
and the cocktail party effect of human auditory system.
The precedence effect suggests to us the way of coping with
echoes in reverberant environment.
The cocktail party effect suggests the use of spatial cues
for sound separation.
Psychological factors of auditory stream integration and segregation,
such as continuity and harmony, are used as structural cues.
It is realized by a continuity enhancement filter and a harmonic
histogram to supplement the spatial segments grouping.
The use of this method with real human speeches
recorded in an anechoic chamber and a normal room was demonstrated.
The experiments have shown that the method was effective to
separate sounds in reverberant environments.
5. Dr. Kiyotoshi MATSUOKA and Dr. Mitsuru KAWAMOTO, Department of Control
Engineering, Kyushu Institute of Technology,
1-1,Tobata, Kitakyushu, 804 Japan
"Blind Signal Separation Based on a Mutual Information Criterion"
Abstract:
This paper deals with the problem of the so-called blind separation
of sources. The problem is to recover a set of source signals from their
linear mixtures observed by the same number of sensors, in the absence of
any particular information about the transfer function that couples the
sources and the sensors. The only a priori knowledge is, basically, the
fact that the source signals are statistically mutually independent. Such
a task arises in noise canceling of sound signals, image enhancement,
medical measurement, etc.
If the observed signals are stationary, Gaussian, white ones, then
blind separation is essentially impossible. Conversely, blind separation
can be realized by exploiting some information on nonstationary,
non-Gaussian, or nonwhite characteristics of the observed signals, if any.
Most of the conventional methods stipulate that the source signals are
non-Gaussian, and use some high-order moments or cumulants. However, it is
sometimes difficult to accurately estimate non-Gaussian statistics because
random signals in practice are usually not so far from Gaussian.
In this paper we propose an approach that utilizes only
second-order moments of the observed signals. We consider two cases:
(i) the source signals are nonstationary;
(ii) the source signals have some temporal correlations, i.e., they are
nonwhite signals.
To realize signal separation we consider a recovering filter which
takes in the observed signals as input and provides an estimate of the
source signals as output. The parameters of the filter are determined
such that all the outputs of the filter be mutually statistically
independent. As a criterion of the statistical independence we adopt the
well-known mutual information between the outputs. The adaptation rule for
the filter's parameters is derived from the steepest descent minimization
of the criterion function.
A remarkable feature of our approach is that it is able to treat
time-convolutive mixtures of a general number of (stationary) source
signals. In contrast, most of the conventional studies on blind separation
only consider the static mixing of the source signals. Namely, any delay
in the mixing process is not taken into account. So, those methods are
useless for many of the important applications of blind separation, e.g.,
separation of sound signals. Although there are some studies that deal
with convolutive mixtures involving some delay, all of them consider only
the case of two sources (2 x 2 channels) and do not seem extendible to the
case of more than two sources. In our approach, also for convolutive
mixtures, the adaptation rule is easily obtained by defining the
information criterion in the frequency domain.
6. Dr. Eric MOREAU, and Prof. Odile MACCHI Laboratoire des Signaux
et Systems, CNRS-ESE, FRANCE
"Adaptive Unsupervised Separation of Discrete Sources"
ABSTRACT: We consider the unsupervised source separation problem where
observations are captured at the output of an unknown linear mixture of
random signals called sources.
The sources are assumed discrete, zero-mean and statistically independent.
In this paper we consider the problem with a prewhitening stage. The a
priori knowledge that sources are discrete with known level, is used in
order to improve performances.
A novel contrast which combines two parts, is proved. The first part forces
statistical independence of the outputs while the second one forces the
outputs to have the known distribution. Then a stochastic gradient adaptive
algorithm is proposed.
Its performance is illustrated thanks to computer simulations that clearly
show that the novel contrast achieves much better performance.
7. Dr. Adel BELOUCHRANI, and Prof. Jean-Francois CARDOSO ,
Telecom Paris, CNRS URA 820, GdR TdSI 46 rue Barrault, 75634 Paris Cedex
13, FRANCE
"Maximum Likelihood Source Separation by the Expectation-Maximization
Technique: Deterministic and Stochastic Implementation"
Abstract:
This paper deals with the source separation problem which consists
in the separation of a mixture of independent sources without
a priori knowledge about the mixing matrix. When the source distributions
are known in advance, this problem can be solved via the maximum
likelihood (ML) approach by maximizing the data likelihood function using
(i) the Expectation-Maximization (EM) algorithm and (ii) a stochastic
version of it, the SEM, wich is efficiently implemented by resorting to
Metropolis sampler. Two important features of our algorithm are that
(a) the covariance of the additive noise can be estimated as a regular
parameter,
(b) in the case of discrete sources, it is possible to separate more
sources than sensors.
The effectiveness of this method is illustrated by numerical simulations.
8. Prof. L. TONG and Dr. X. CHEN, University of Connecticut, USA.
"Blind Separation of Dynamically Mixed Multiple Sources
and its Applications in CDMA Systems"
Abstract:
In this paper, we consider the problem of separating dynamically mixed
multiple sources. Specifically, we address the problem of recovering the
sources of an multiple-input multiple-output system. Two issues will be
addressed:
(i) Source Blind Separability;
(ii) Blind Signal Separation Algorithms.
Applications of the proposed approach to code-division multiple-access
schemes in wireless communication are presented.
9. Prof. Shun-ichi AMARI, Prof. Andrzej CICHOCKI and Dr. Howard Hua YANG,
Frontier Research Program RIKEN (Institute of Physical and Chemical
Research), Wako-shi, JAPAN
"Multi-layer Neural Networks with Local Learning Rules for Blind
Separation of Sources"
Abstract:
In this paper we will propose multi-layer neural network models
(feedforward and recurrent) with novel, local, adaptive, unsupervised
learning rules which enable not only to separate on-line independent
sources but also determine the number of active sources.
In other words, we assume that the number of sources and their waveforms
are completely unknown. Moreover, the separation problem can be very
ill-conditioned and/or badly scaled. In fact the performance of the learning
algorithm is independent of scaling factors and a condition number of the
mixing matrix.
Universal (flexible) computer simulation program will be presented which
enable comparison of validity and performance of various recently
developed adaptive on-line learning algorithms.
10.. Dr.L. De Lathauwer and Dr.P. Comon;
E.E. Dept. - ESAT - SISTA, K.U.Leuven, BELGIUM, CNRS - I3S, Sophia
Antipolis,
Valbonne, FRANCE
"Higher - Order Power Method"
Abstract
The scientific boom in the field of higher-order statistics
involves an increasing need for numerical tools in multi-linear
algebra: higher-order moments and cumulants of multivariate
stochastic processes are higher-order tensors.
We consider the problem of generalizing the computation of the
best rank-R approximation of a given matrix to the computation
of the best rank-(R1,R2,...,RN) approximation of an Nth-order
tensor. We mainly focus on the best rank-1 approximation of
third-order tensors.
It is shown that this problem leads in a very natural way to
a higher-order equivalent of the well-known power method for
the computation of the eigendecomposition of matrices.
It can be proved that each power iteration step decreases the
least-squares error between the initial tensor and the lower-rank
estimate. In the tensor case several stationary points might
exist, each with a different domain of attraction.
Surprisingly, the power iteration for a super-symmetric tensor
can produce intermediate results that are unsymmetric.
Imposing symmetry on the algorithm does not necessarily
improve the convergence speed; the symmetric power iteration
can even fail to converge.
In the matrix case truncation of the singular value decomposition (SVD)
yields the best rank-R approximation; in the tensor case it can
only be proved that truncation of the higher-order singular
value decomposition (HOSVD) yields a fairly good approximation.
All our simulations show that the HOSVD-guess belongs to the
attraction region corresponding to the optimal fit.
---------------------------------------------------------------------------