Analysis of Latent Representations of Neural Text-To-Speech Models for Expressive Audio-Visual Synthesis - TEL - Thèses en ligne Accéder directement au contenu
Thèse Année : 2023

Analysis of Latent Representations of Neural Text-To-Speech Models for Expressive Audio-Visual Synthesis

Analyse des Représentations Latentes des Modèles de Text-To-Speech Neuronaux pour le Contrôle de la Synthèse Audio-Visuelle Expressive

Résumé

In recent years, deep neural architectures display groundbreaking performances in various speech processing area, including Text-To-Speech (TTS). Models have grown bigger, including more layers and millions of trainable parameters to achieve almost natural synthesis, at the expense of interpretability of computed intermediate representations, called embeddings. However, statistical learning performed by these neural models constitutes a valuable source of information about language. This presentation aims at openning this "black box" to explore intermediate embeddings computed by state-of-the-art TTS models. By identifying phonetic and acoustic features in model representations, the proposed methods help understanding how neural TTS are able to organize speech information on an unsupervised manner and provide new insights on phonetic regularities captured by statistical learning on massive data that are beyond human expertise. This work open the route toward designing more careful control architectures for neural TTS, without the need for additional data nor training process. These results led us to propose an auxiliary module for expressive synthesis called Local Style Tokens (LST), which models local variations in prosody with respect to the type of embeddings to bias.
In recent years, deep neural architectures display groundbreaking performances in various speech processing area, including Text-To-Speech (TTS). Models have grown bigger, including more layers and millions of trainable parameters to achieve almost natural synthesis, at the expense of interpretability of computed intermediate representations, called embeddings. However, statistical learning performed by these neural models constitutes a valuable source of information about language. This presentation aims at openning this "black box" to explore intermediate embeddings computed by state-of-the-art TTS models. By identifying phonetic and acoustic features in model representations, the proposed methods help understanding how neural TTS are able to organize speech information on an unsupervised manner and provide new insights on phonetic regularities captured by statistical learning on massive data that are beyond human expertise. This work open the route toward designing more careful control architectures for neural TTS, without the need for additional data nor training process. These results led us to propose an auxiliary module for expressive synthesis called Local Style Tokens (LST), which models local variations in prosody with respect to the type of embeddings to bias.
Fichier principal
Vignette du fichier
LENGLET_2023_diffusion.pdf (29.3 Mo) Télécharger le fichier
Origine : Version validée par le jury (STAR)

Dates et versions

tel-04541736 , version 1 (11-04-2024)

Identifiants

  • HAL Id : tel-04541736 , version 1

Citer

Martin Lenglet. Analysis of Latent Representations of Neural Text-To-Speech Models for Expressive Audio-Visual Synthesis. Signal and Image processing. Université Grenoble Alpes [2020-..], 2023. English. ⟨NNT : 2023GRALT091⟩. ⟨tel-04541736⟩
0 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More