Eric Vatikiotis-Bateson
ATR Human Information Process. Res. Labs., 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-02 Japan
Kevin Munhall
Queens Univ., Kingston, ON, Canada
David Ostry
McGill Univ., Montreal, PQ, Canada
Optoelectronic position sensing devices have been important in the study of speech articulation. While limited to the kinematics of externally visible structures such as the lips and the jaw, such systems can be used easily with a wide range of subjects. This video shows how an innovative marker-tracking system, OPTOTRAK (Northern Digital, Inc.), is currently being used in three areas of experimentation: (1) rigid body analysis in which 3-D motion of the jaw is decomposed into its three orientation and three translation components; (2) modeling of the relation between muscle activity (EMG) and 3-D motion of the lips and jaw using artificial neural networks; and (3) mapping facial motions for possible use in orofacial modeling and audiovisual recognition. The first experiment shows how technical developments, resulting in accurate measurement of 3-D motion, now make it possible to calculate all orientation angles and positions necessary to fully describe structures such as the jaw. The second area demonstrates a more traditional paradigm in which several types of data must be acquired simultaneously. The third area illustrates the growing interest in mapping speech events to facial motion as well as to changes in vocal tract shape.