it'll usually be fine, unless you mean like a minute of data or less in which case yeah tacotron can prove too unstable with talknet it'll still sound like garbage but at least it won't have too long of a stroke

also idk if it's possible but maybe even skip training the pitch and duration predictors that way it'll use the default ljspeech ones instead of trying to optimize ones for the little data you gave it idk

not that that matters because uberduck doesn't use them anyway

Im actually trying out the Mozilla/Coqui TTS training out for a bit, but never really tried custom datasets. For the past months I've just been looking into other models, training the ljs.

oh you mean training from scratch?

yea, wait, how else would you train?

like just training the vocoder?

99% of uberduck models are made by finetuning a pre-trained ljspeech model

basically running the training loop for a few more hours with new data

wait, really?

yeah

Man I guess I've been doing it the hard way all along

pf

If you don't mind me picking your brain, I might still be dumb

but the pretrained ljs model, is that the tacotron one?

From my understanding there are two ways to produce speech, one is Text2Mel then MelToWav, and then there is end-to-end

wait let me reset and ask this

that first one is tacotron2

tacotron2 is the text2mel part waveglow or hifi-gan is the mel2wav part

gotcha

wavenet is full end-to-end but we don't use that

talknet is even further to the left than tacotron2

So, for the 99%, does everyone just use a pretrained ljs tacotron model, and then train the MelToWav

has separate pitch and duration predictors that then feed into the mel thing

or do they train the tacotron, and use a pretrained vocoder/meltowav

most people just train upon tacotron2 pretrained

usually train just text2mel yeah

gotcha

some people go the extra mile and train the vocoder, usually hifi-gan

what kinda loss am I looking for in the tacotron 2 model then before I can test it on the vocoder