Section: New Software and Platforms

VisArtico

Visualization of multimodal speech data

Keywords: Data visualization - 3D movement - Speech processing - Videos

Scientific Description: VisArtico is a multimodal data visualization software acquired by several systems: articulograph, motion capture, depth camera. This software makes it possible to visualize the positions of real or virtual sensors and to animate them simultaneously with acoustics. Regarding the articulatory data, the user has the possibility to visualise the contour of the tongue and the lips. It also makes it possible to find the midsagittal plane of the speaker, and to deduce the position of the palate, if this information is absent during the acquisition. The software makes it possible to display the segmentation at the level of sentences, words or phonemes. The goal is to provide an effective multimodal data visualization tool that can be useful to anyone studying speech production, audio-visual synthesis, or animation in a more general way.

Functional Description: VisArtico is a user-friendly software which allows visualizing multimodal data acquired by several systems : an articulograph (AG500, AG501 or NDI Wave), motion capture system, depth camera. This visualization software has been designed so that it can directly use the data provided by the different systems to display the spatial and temporal positions of the sensors (real and virtual), synchronized with the corresponding acoustic recordings. Moreover, for articulatory data, VisArtico not only allows viewing the sensors but also enriches the visual information by indicating clearly and graphically the data for the tongue, lips and jaw. Finally, it is possible to generate a movie for any articulatory-acoustic sequence. This software can be useful for researchers in speech production, audiovisual speech synthesis or articulatory speech analysis.

Release Functional Description: The main improvement in this version is the ability to view a video that was recorded along with the articulatory or motion capture data. The software also allows for automatic speech segmentation.

News Of The Year: This year, we have added the possibility to visualize a video simultaneously with the multimodal data and the acoustic data. Several bugs have been fixed.

• Participants: Ilef Ben Farhat, Loïc Mangeonjean and Slim Ouni

• Partners: CNRS - Université de Lorraine

• Contact: Slim Ouni

• Publication: VisArtico: a visualization tool for articulatory data

• URL: http://visartico.loria.fr