Re: auditory memory and sound classification

["f.maintenant" <f.maintenant@xxxxxxxxxxxx>]

 Dear Hadi

>was wondering if we can suppose from a scientific point of view that human
>subjects when classifying a stimulus into high level concepts (rain,
>explosion, speech) base their judgments on the auditory memory state (the
>integration and correlation of the past spectral activities for instance)
>which is updated continuously by new acoustic activities. Also I would like
>to know if such auditory memory models exist.

The faculty of naming concepts implies probably one of the most complex
connection between brain and mind, and it is probably one of the most
important characteristic of human behaviour. However the recognition of
physically induced concept such as rain is not particularly characteristic
of human behaviour, and it can be assumed that a primitive form of memory
associates the sound of rain with a water induced phenomenon (although it
can be confused with fire). Dennis Smalley has stressed in some of his
articles the importance of the mechanical origin of sound sources; even when
a sound source is virtual, we try to associate it with an imaginery source.
This reflex probably relies on auditory memory.
http://www.city.ac.uk/music/denis1.htm
Typically a sound is divided in 3 parts, attack, steady-state and decay.
Attack is very important in sound synthesis, but it can be argued that the
steady-state (from which the spectrum maybe extracted) contains most of the
information on sound, not an information that we are necessary capable to
exploite fully: a often given example being that the sound of a piano
without attack can be mistaken for a flute.
In several experiments with Kohonen maps, I have used only spectra of the
steady-state of the sounds of music intruments as analysed by Gregory
Sandell (SHARC database) and was surprised to find that it was enough
information in order to classify most music instruments providing that
enough (10) notes per instruments were considered. Furthermore the positions
on the map of the various sounds were, relativelly speaking, in accordance
with musical knowledge . It seems justified to believe that the
self-organizing artificial neural network algorithms model to a certain
extend an auditory memory. After all Kohonen was inspired by his
neurological investigations, in particular of the tonotopic maps.

>(we suppose for instance a simple
> model of the ear consisting of estimating and transferring the energy
> information into different frequency bands to be processed by the cortex)

Have you try to collect your information by amplitude bands; i.e. build a
frequency vector such that its coordonates have frequency related values,
and that the underlying vectorial space is partitioned in sub-spaces Aj
according to amplitude values [v=(f1, f2,.., fk,..fn) such that for k in
{1,..,n}, v/Aj=(fk,..,fk+p) where the energy values corresponding to fk,
...fk+p lie in the Amplitude space Aj (j<K)]. That's what I did in my study
in order to minimise the influence of pitch when comparing sounds of music
instruments. I suppose that we can remember the sound of an instrument
without knowing at which pitch it was played. Obviously there is always
musical trick  that can confuse the listener.

Do you know the GRAME in Lyon, they work a lot with sound, in a musical way,
but have also an interest in sound recognition.
http://www.grame.fr/Research/fr/


Frédéric Maintenant
http://www.spnm.org.uk/shortlist/maintenant03.html

Springdown house
Spring Hill
Kingsdown
Bristol BS2 8HX - UK
tel&fax. +44 (0)117 924 0944





----- Original Message -----
From: "Hadi Harb" <Hadi.Harb@EC-LYON.FR>
To: <AUDITORY@LISTS.MCGILL.CA>
Sent: Tuesday, April 20, 2004 7:32 PM
Subject: auditory memory and sound classification


> Dear list,
>
> During my PhD thesis I was working on developing a general audio
classifier
> for multimedia applications. I started from the supposition that if we
model
> some aspects of the way humans classify audio semantic classes (in our
> context semantic class means: speech, music, jazz, man, action.) while
> building an audio classifier will be advantageous. In two experiments I
made
> for the human discrimination capability between speech/music and
> man(speech)/woman(speech) the context and the duration of the stimuli
> affected seriously the discrimination judgments by the subjects. For
> example, a short speech stimulus (70ms) presented after a 900ms music
> stimulus was not noticed by the subjects in the majority of cases
(similarly
> to the spectral contrast in speech recognition).
> From an engineering perspective I supposed that some kind of integration
of
> the spectral activity over time exists (we suppose for instance a simple
> model of the ear consisting of estimating and transferring the energy
> information into different frequency bands to be processed by the cortex).
I
> supposed then an auditory memory model consisting of some kind of a mean
> (accumulation) and a variance (surprise) of the past spectral energy in
> different frequency bands. This model, although very simplistic and based
> only on intuition, when used as the basis for audio signal feature
> extraction was shown to have interesting properties for general audio
> classification in multimedia indexing applications.
>
> I was wondering if we can suppose from a scientific point of view that
human
> subjects when classifying a stimulus into high level concepts (rain,
> explosion, speech) base their judgments on the auditory memory state (the
> integration and correlation of the past spectral activities for instance)
> which is updated continuously by new acoustic activities. Also I would
like
> to know if such auditory memory models exist.
>
> I am seeking clarification, directions, and references about the effect of
> the auditory memory models on the human perception of general sound if
> possible.
>
> Thank you,
>
> Hadi Harb
>  Dept. Mathematiques - Informatique,
>  ECOLE CENTRALE DE LYON
>  36, av Guy de Collongue,
>  69134 Ecully France EUROPE
>  tel: +33 (0)4.72.18.64.59
>  fax: +33 (0)4.78.33.16.15
>  e-mail: hadi.harb@ec-lyon.fr
>  web: http://perso.ec-lyon.fr/hadi.harb