Ultrax: Vocal Tract Imaging for Speech Therapy and Research

Ultrax: Vocal Tract Imaging for Speech Therapy and Research

Korin Richmond1 , Steve Renals1 , Joanne Cleland2 , Jim Scobbie2 , Alan Wrench3 

1Centre for Speech Technology Research, Informatics, University of Edinburgh, UK

2Clinical Audiology, Speech and Language Research Centre, Queen Margaret University, UK

3Articulate Instruments Ltd, Edinburgh, UK

Background and motivation

Speech sound disorders are the most common communication impairment in childhood, affecting 1 in 15 children in the UK, for example. Sufferers have difficulty producing one or more speech sounds of their native language. This can affect a child’s confidence, hindering communication with peers and teachers, and so harming learning as well as social interaction and development. Current speech therapy interventions mainly rely on the child’s auditory skills, as little technology is available to help diagnosis and therapy. The Ultrax project (http://www.ultrax-speech.org) is a three-year project, funded by the EPSRC Healthcare Partnerships programme, which aims to use ultrasound to provide real-time visual feedback of lingual articulation. The ultimate aim is to exploit ultrasound to track tongue movement, while a simplified, diagrammatic vocal tract is displayed to the user as a realtime animation (see Fig. 1).

Approach and data

A machine learning approach is employed in conjunction with MRI and ultrasound imaging data. MRI is used to obtain a wide variety of vocal tract (VT) shapes (hand-labelled contours) for multiple subjects. This corpus is used to train a dimensionality-reducing VT shape model. The dimensionality reduction methods evaluated for this purpose include principal component analysis (PCA) and Laplacian eigenmaps [1].

Siemens Verio 3T scanner was used (1.7x1.7mm voxels, 52ms echo time, 400ms repetition time(=2.5fps), with an FOMRI-III optical mic for audio). Matching ultrasound data has been collected from the same subjects using an Ultrasonix RP system (B mode, 10mm microconvex probe at 5MHz), and similarly from around 90 typically-developing children performing a range of speech tasks. The control parameters of the above VT shape model are then used as the latent space representation of a Kalman filter variant [2], which in turn tracks the tongue in the ultrasound image sequence and animates the simplified VT display.

MRI data has been obtained from a total of 12 adult phoneticians, each producing 52 sound shapes. A Beyond speech therapy alone, the methods developed may also facilitate the collection of human articulography imaging data for other purposes. These include research in the fields of speech production and technology, for example silent speech interfaces (e.g. for laryngectomy patients), foreign language training, automatic speech recognition, text-to-speech synthesis and visual speech animation.

References

[1] M. A. Carreira-Perpina´n and Z. Lu, “The Laplacian eigenmaps latent variable model,” in Proc.11th International Conference on Artificial Intelli- gence and Statistics (AISTATS), 2007, pp. 59–66.

[2] D. Simon, Optimal state estimation: Kalman, H [infinity] and nonlinear approaches. John Wiley and Sons, 2006.