Abstract:
It is now well established that visible speech is an important source of information in face-to-face communication. Given the valuable role of audible speech synthesis in experimental, theoretical, and applied arenas, visible speech synthesis has been developed. Research has shown that the talking head closely resembles real heads in the quality of its speech and its realism (when texture mapping is used). The talking head can be heard, communicates paralinguistic as well as linguistic information, and is controlled by a text-to-speech system. Several sources of evidence are presented which show that visible speech perception (speechreading) is fairly robust across various forms of degradation. Speechreading remains fairly accurate even when the mouth is viewed in noncentral vision; eliminating and distorting high-spatial frequency information does not completely disrupt speechreading; and speechreading is possible when additional visual information is simultaneously being used to recognize the speech input. The results are consistent with the fuzzy logical model of perception in which multiple sources of information are used to recognize patterns. Various visible feature sets are tested within the framework of the model to determine which visible features are functional in speechreading. Demonstrations of the talking head and various psychological phenomena will be provided. [Work supported by NIDCD.]