Abstract:

Three measures of auditory-visual (AV) integration using both congruent and discrepant speech stimuli were evaluated in 30 hearing-impaired subjects. The first measure was based on Braida's model of pre-labeling integration [Q. J. Exp. Psych. 43, 647--677 (1991)] which makes predictions of AV consonant recognition from separate auditory (A) and visual (V) consonant confusion matrices. The ratio of observed-to-predicted AV score was used as an index of integration, with unity representing optimal integration. The second measure used discrepant A and V consonant--vowel stimuli to determine susceptibility to ``McGurk'' effects. Susceptibility was defined as the difference between A and AV consonant recognition, with greater differences reflecting greater susceptibility. The third measure used temporally misaligned AV speech stimuli in a sentence recognition task to determine individual sensitivity to AV asynchrony. Auditory onset delays between 0-320 ms were studied and individual sensitivity to AV asynchrony was defined in terms of decrements in sentence intelligibility relative to the synchronous condition (0-ms delay). Relations among these three potential measures of AV integration, and between integration measures and a separate measure of relative AV sentence benefit, will be discussed. [Work supported by NIH Grant No. DC00792 and the Department of Clinical Investigation.]