Re: Cariani's question: "What is the visual analogue of pitch?" (Pierre Divenyi )

Subject: Re: Cariani's question: "What is the visual analogue of pitch?"
From:    Pierre Divenyi  <pdivenyi(at)EBIRE.ORG>
Date:    Tue, 20 Jan 2004 23:18:33 -0800

I must congratulate Peter for his thoughtful, and erudite, discussion on
pitch and form. I might want to add another book on visual art/music
analogy: Pierre Boulez's book "The fertile land -- Paul Klee" (Le pays
fertile). That guy has ideas...

There is only one reason for which I keep finding music/art analogies
contrived: music implies a temporal rigor (meter, rhythm) which images
don't require. We can imagine that forms juxtaposed will demand instances
of focusing that cannot be simultaneous and whose rate of succession is
determined by higher-order cortical time constants, but this assumption is
also a hand-waving exercise. That is, until some astute neuroscientist
finds a way to measure observation of visual art. At least they are trying
to break into music...

Pierre Divenyi
At 11:10 PM 1/20/2004 -0500, Peter Cariani wrote:
>The debate between Kubovy and Neuhoff is interesting,
>although it will take some time to digest.
>I found that the URL for Kubovy's papers that works is:
>http://www.people.virginia.edu/~mk9y/mySite/papers.html
>
>There are a number of provocative interchanges between music and film
>that always
>come to mind in these discussions: the abstract films of Dadaist
>Hans Richter and Eggeling's Symphonie Diagonale. This I think is the
>closest visual
>art comes to music, where repetition and rhythm of form and movement
>play
>strong roles. On the music as visual form front,
>I taught Psychology of Music last fall and used Stephen Malinkowski's
>Music Animation Machine piano roll music animations to help visualize
>melodic structure.
>http://www.well.com/user/smalin/mam.html
>It's worth having a look at it (and his tapes) if you're interested in
>these issues.
>
>The Gestalists certainly included melody and rhythm as examples of
>coherent, relational organizations.  Melodic and rhythmic grouping
>mechanisms
>arguably form the "chunks" that cause us to parse music in particular
>ways that
>are then described by the cognitivists in terms of nested hierarchical
>organizations.
>
>Along with Handel's Listening (1989),  I've found Snyder's book,
>Music and Memory very useful in developing these notions in
>musical contexts.
>
>I agree that the relation between audition and vision is not simple.
>We understand neither system well. Pitch is not frequency per se, and
>visual
>form is not simply a spatial pattern of activation on the retina, but
>there are
>nevertheless parallels between the kinds of correlational invariances
>and
>transformations  that underlie say magnification invariance of form in
>vision
>and transpositional invariance of chords and melodies in music. One
>looks
>at various binocular spatial-disparity effects (stereodiagrams) and
>there are
>temporal analogues in the binaural system (Huggins pitch). Time delays
>in the binocular
>system map to depth (Pulfrich), while they map to azimuthal location in
>audition. The correspondences are not those that would be predicted by
>simple analogies, but neither do they seem arbitrary.
>
>I tend to think of timbre as the auditory analogue of visual texture
>and color,
>and melody as an auditory analogue of visual figure or contour. Because
>of
>eye movements, a figure is constantly being presented to different
>retinal
>locations, such that the spatiotemporal (spike) volley pattern
>associated with the
>spatial form is re-presented to the system over and over again. We can
>imagine circuits
>that build up this invariant volley pattern as a stable object.  A
>series of notes repeated likewise creates
>an auditory volley pattern that is repeated, and the same kind of
>mechanism
>would create an auditory image of the whole repeated sequence.
>
>When the melody is transposed, we hear the similarity of the patterns,
>but also the shift in pitch
>(upward or downward) of the pattern as a whole: i.e. apparent movement
>of an object.
>Music theory is rife with all sorts of metaphors of movement (rhythmic,
>melodic,
>tonal, thematic, etc.), which involves this combination of an invariant
>pattern (object) being transformed in a manner that preserves its
>essential organization (that made it a stable object in the first
>place).
>The paper by Pitts & McCulloch (1947) on How We Know Universals
>had the right spirit in trying to conceive of a mechanism, but their
>neural coding assumptions -- re: the nature of the representations -- I
>think
>were flawed.The pattern invariants could be volley patterns of spikes,
>rather than
>channel patterns (rate-place profiles in auditory and visual areas).
>This
>might explain why our sensory systems so effortlessly recognize the
>similarity of
>the patterns even when they are transposed or translated onto completely
>different sets of neural channels (different retinotopic and
>cochleotopic positions in neural maps).
>It's easy to move temporal patterns around in neural
>systems, but much harder to move spatial patterns.
>In the 1930's Lashley recognized the problems these channel-translations
>pose for "switchboard models" of vision.
>But today our thinking is are so enamored of features and rate-channel
>codes
>that it becomes nearly impossible to conceive of anything else.
>
>--Peter Cariani