Training a new voice for Piper utilizing a sole phrase: A guide on single-word voice training for Piper's audio system.
In a groundbreaking development, a team is utilizing cutting-edge AI to give people the ability to speak once more. Cal Bryant, an innovative tech enthusiast, embarked on a mission to fine-tune the Piper TTS AI voice model, a technology that promises more natural-sounding speech compared to existing free-to-use TTS systems.
The process of fine-tuning Piper TTS involves a unique approach. Starting with a single phrase clone from a commercial TTS voice, a powerful AI voice cloning system like ChatterBox is employed. This system generates a vast array of synthetic audio data, around 1,300 phrases, in the cloned voice style, creating a diverse and sufficiently large training dataset.
A representative corpus of everyday English text phrases is then used to generate the cloned audio. This audio is produced by running the text phrases through the ChatterBox engine, which clones the voice from the single phrase.
With this dataset in hand, the Piper TTS AI voice model is fine-tuned by continuing training from an existing checkpoint. Fine-tuning typically requires fewer epochs than training from scratch, about 1,000 extra epochs instead of 2,000. A good-sized dataset for fine-tuning usually has about 1,300 phrases, which matches the synthetic dataset generated from the single phrase clone.
Training considerations include the need for paired text and audio data, the benefit of a recent GPU for efficient processing, and the use of techniques like multiple generation attempts and transcription verification to ensure high-quality training data.
Post fine-tuning, the voice output can be refined by adjusting parameters in Piper’s configuration files, such as `phoneme_duration_scale`, `length_scale`, `noise_scale`, and `noise_w`.
Cal Bryant, in his quest to fine-tune the Piper TTS AI voice model, faced the challenge of generating a large volume of training phrases. He used OpenAI's Whisper software to transcribe the fine-tuned Piper TTS audio back to text, which served as the training data for further fine-tuning the Piper AI model on a GPU rig.
Piper TTS does not require massive resources to run, making it an accessible solution for many. This was achieved using a heavyweight AI model, ChatterBox, capable of zero-shot training. The problem of generating a large volume of training phrases was thus effectively solved.
Cal Bryant has utilised Piper TTS in a home automation system for various undisclosed purposes. While traditional methods of making things talk may seem outdated, the advancements in AI speech synthesis are revolutionizing the way we interact with technology.
During the fine-tuning process of Piper TTS, Cal Bryant employed the ChatterBox AI voice cloning system, a technology that relies on artificial-intelligence, to generate a synthetic dataset containing over 1,300 phrases for training. Furthermore, after fine-tuning the Piper TTS AI voice model, Bryant utilized hardware like a GPU for efficient processing, demonstrating how hardware plays a crucial role in the advancements of technology and artificial-intelligence.
