Examples of audio-visual speech recognition systems

• Goldschen:

  • extend Petajan’s system by using HMM as classifier in the acoustic & visual recognizer
  • use delta visual features, i.e. time derivatives of:
    • area; perimeter; H; W of mouth
    ? Lips movement provides more information than lips position !!!

• Stork:

  • use TDNN for speech recognition (recognition based on time variation of mouth parameters)
  • late integration strategy for audiovisual recognition gives good results

• Bregler: similar to Stork:

  • use TDNN for speech recognition and outer lip contour as visual feature

Department of Informatics

Aristotle University of Thessaloniki

Previous slide

Next slide

Back to first slide

View graphic version