WaveTransfer: A Flexible End-to-end Multi-instrument Timbre Transfer with Diffusion - Equipe Signal, Statistique et Apprentissage
Conference Papers Year : 2024

WaveTransfer: A Flexible End-to-end Multi-instrument Timbre Transfer with Diffusion

Abstract

As diffusion-based deep generative models gain prevalence, researchers are actively investigating their potential applications across various domains, including music synthesis and style alteration. Within this work, we are interested in timbre transfer, a process that involves seamlessly altering the instrumental characteristics of musical pieces while preserving essential musical elements. This paper introduces WaveTransfer, an end-to-end diffusion model designed for timbre transfer. We specifically employ the bilateral denoising diffusion model (BDDM) for noise scheduling search. Our model is capable of conducting timbre transfer between audio mixtures as well as individual instruments. Notably, it exhibits versatility in that it accommodates multiple types of timbre transfer between unique instrument pairs in a single model, eliminating the need for separate model training for each pairing. Furthermore, unlike recent works limited to 16 kHz, WaveTransfer can be trained at various sampling rates, including the industry-standard 44.1 kHz, a feature of particular interest to the music community.
Fichier principal
Vignette du fichier
MLSP_2024 _WaveTransfer.pdf (2.47 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04685184 , version 1 (05-09-2024)

Identifiers

  • HAL Id : hal-04685184 , version 1

Cite

Teysir Baoueb, Xiaoyu Bie, Hicham Janati, Gael Richard. WaveTransfer: A Flexible End-to-end Multi-instrument Timbre Transfer with Diffusion. 2024 IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2024), Sep 2024, London (UK), United Kingdom. ⟨hal-04685184⟩
99 View
74 Download

Share

More