Subject: Re: auditory memory and sound classification From: "f.maintenant" <f.maintenant(at)NTLWORLD.COM> Date: Wed, 21 Apr 2004 16:50:17 +0100Dear Hadi >was wondering if we can suppose from a scientific point of view that human >subjects when classifying a stimulus into high level concepts (rain, >explosion, speech) base their judgments on the auditory memory state (the >integration and correlation of the past spectral activities for instance) >which is updated continuously by new acoustic activities. Also I would like >to know if such auditory memory models exist. The faculty of naming concepts implies probably one of the most complex connection between brain and mind, and it is probably one of the most important characteristic of human behaviour. However the recognition of physically induced concept such as rain is not particularly characteristic of human behaviour, and it can be assumed that a primitive form of memory associates the sound of rain with a water induced phenomenon (although it can be confused with fire). Dennis Smalley has stressed in some of his articles the importance of the mechanical origin of sound sources; even when a sound source is virtual, we try to associate it with an imaginery source. This reflex probably relies on auditory memory. http://www.city.ac.uk/music/denis1.htm Typically a sound is divided in 3 parts, attack, steady-state and decay. Attack is very important in sound synthesis, but it can be argued that the steady-state (from which the spectrum maybe extracted) contains most of the information on sound, not an information that we are necessary capable to exploite fully: a often given example being that the sound of a piano without attack can be mistaken for a flute. In several experiments with Kohonen maps, I have used only spectra of the steady-state of the sounds of music intruments as analysed by Gregory Sandell (SHARC database) and was surprised to find that it was enough information in order to classify most music instruments providing that enough (10) notes per instruments were considered. Furthermore the positions on the map of the various sounds were, relativelly speaking, in accordance with musical knowledge . It seems justified to believe that the self-organizing artificial neural network algorithms model to a certain extend an auditory memory. After all Kohonen was inspired by his neurological investigations, in particular of the tonotopic maps. >(we suppose for instance a simple > model of the ear consisting of estimating and transferring the energy > information into different frequency bands to be processed by the cortex) Have you try to collect your information by amplitude bands; i.e. build a frequency vector such that its coordonates have frequency related values, and that the underlying vectorial space is partitioned in sub-spaces Aj according to amplitude values [v=(f1, f2,.., fk,..fn) such that for k in {1,..,n}, v/Aj=(fk,..,fk+p) where the energy values corresponding to fk, ...fk+p lie in the Amplitude space Aj (j<K)]. That's what I did in my study in order to minimise the influence of pitch when comparing sounds of music instruments. I suppose that we can remember the sound of an instrument without knowing at which pitch it was played. Obviously there is always musical trick that can confuse the listener. Do you know the GRAME in Lyon, they work a lot with sound, in a musical way, but have also an interest in sound recognition. http://www.grame.fr/Research/fr/ Frédéric Maintenant http://www.spnm.org.uk/shortlist/maintenant03.html Springdown house Spring Hill Kingsdown Bristol BS2 8HX - UK tel&fax. +44 (0)117 924 0944 ----- Original Message ----- From: "Hadi Harb" <Hadi.Harb(at)EC-LYON.FR> To: <AUDITORY(at)LISTS.MCGILL.CA> Sent: Tuesday, April 20, 2004 7:32 PM Subject: auditory memory and sound classification > Dear list, > > During my PhD thesis I was working on developing a general audio classifier > for multimedia applications. I started from the supposition that if we model > some aspects of the way humans classify audio semantic classes (in our > context semantic class means: speech, music, jazz, man, action.) while > building an audio classifier will be advantageous. In two experiments I made > for the human discrimination capability between speech/music and > man(speech)/woman(speech) the context and the duration of the stimuli > affected seriously the discrimination judgments by the subjects. For > example, a short speech stimulus (70ms) presented after a 900ms music > stimulus was not noticed by the subjects in the majority of cases (similarly > to the spectral contrast in speech recognition). > From an engineering perspective I supposed that some kind of integration of > the spectral activity over time exists (we suppose for instance a simple > model of the ear consisting of estimating and transferring the energy > information into different frequency bands to be processed by the cortex). I > supposed then an auditory memory model consisting of some kind of a mean > (accumulation) and a variance (surprise) of the past spectral energy in > different frequency bands. This model, although very simplistic and based > only on intuition, when used as the basis for audio signal feature > extraction was shown to have interesting properties for general audio > classification in multimedia indexing applications. > > I was wondering if we can suppose from a scientific point of view that human > subjects when classifying a stimulus into high level concepts (rain, > explosion, speech) base their judgments on the auditory memory state (the > integration and correlation of the past spectral activities for instance) > which is updated continuously by new acoustic activities. Also I would like > to know if such auditory memory models exist. > > I am seeking clarification, directions, and references about the effect of > the auditory memory models on the human perception of general sound if > possible. > > Thank you, > > Hadi Harb > Dept. Mathematiques - Informatique, > ECOLE CENTRALE DE LYON > 36, av Guy de Collongue, > 69134 Ecully France EUROPE > tel: +33 (0)4.72.18.64.59 > fax: +33 (0)4.78.33.16.15 > e-mail: hadi.harb(at)ec-lyon.fr > web: http://perso.ec-lyon.fr/hadi.harb