Abstract:
This study extends previous evaluations of phonetic labeling agreement between multiple transcribers (ICSLP94, ICPhS95). In the current experiment five labelers transcribe a total of 2 min of conversational speech in English, French, German, Spanish, Vietnamese, and Japanese, with four female and four male talkers for each language. The paper presents three sets of results: labeler agreement, the effect on labeler agreement of using an orthographic reference as an aid, and analysis of the types of ``errors'' contributing to labeler (dis)agreement. Interlabeler agreement is measured at the phonetic, phonemic, and broad category levels of transcription. Broad category includes classes of sounds such as nasals, vowels, fricatives, and plosives. For half of the data in each language, transcribers could refer to an orthographic transcription; the paper describes the effect of this additional information on labeler agreement. The paper also presents detailed analyses of the kinds of insertions, deletions, and substitutions leading to labeler disagreement. Focus is on whether transcribers converge on a pattern of favored transcription conventions, such that what seems to be disagreement in identification of sound segments may be a matter of transcriber convention. [The authors would like to thank the National Science Foundation for their support of this research.]