Speechreading by humans
Speechreading is necessary for hearing impaired persons to (partially) understand speech by using what is recoverable from visual speech.
In case of normal hearers, auditory modality is the most important, but the visual modality may allow better understanding of speech.
Vision may almost entirely replace spectral information if the laryngeal timing is provided, but only movements of lip, jaw, teeth, etc. cannot provide normal speech intelligibility.
In the presence of background noise or competing speakers (e.g. party), even normal hearers use lip reading to some extent.
The presence of visual speech signal is equivalent to a 12 dB gain in acoustic SNR.
In case of sentences from a foreign language or spoken by an accented speaker, even without noise, speech is easier to recognize when visual speech signal is present.
Department of Informatics
Aristotle University of Thessaloniki