as much as you have the patience to collect, wav 16-bit mono 22050 hz

how much for the voice to be decent tho, like the minimum ?

depends on how emotive the voice is

i did mafia from a hat in time with 20 minutes of audio and it turned out great because they speak somewhat normally

meanwhile the conductor turned out worse with the same amount of audio because he's quite screamy

I see, thanks

apparently chills had a very big dataset but it's not on google drive anymore so i can't check exactly how big

yeah i'm just getting familiar with how this all works - so, a binary blob of code every epoch. interesting, so it's effectively full size even at Epoch 1 and the data just shifts around. honestly for the limited data set situation someone should make manually triggered epochs after a certain low number so you can check the graph, instead of finding out afterwards that you went too far and your clone sounds like a babbling idiot.

yeah absolutely

i believe talknet has something like that for one of the elements, where it keeps the 3 best iterations so far but that probably works best with a separate validation set to detect overfitting, and we can't afford that with the size of our datasets

i'll just save it separately every 50 or so epochs though

@User are you involved with the uberduck team? you know a fair amount about the whole process. have to admit i'm a latecomer to ML and neural net terminology (and Python). so my head is swimming a bit.

nah i just like looking at debug output and understanding like half of it

lol - well have to start somewhere i guess. that's sorta how i learned C# in Unity myself...

seems like every 50 would be potentially a huge range for small datasets (admittedly my only experience with TTS). in my last run i ran it to 100 but i think Epoch 92 had the best output graph.

one question i did have - if you needed to retrain from scratch, what the best way for that? currently i just nuke the notebook and make a new copy from the main one.

just delete the model file from your google drive, or change the model name in the notebook

the graph is calculated for a different sentence each time, so if it's 9 smooth graphs and 1 messed-up one it's more likely that your network is 90% good at english and less likely that it suddenly broke the clips also have varying lengths, so if your graph is half diagonal half horizontal that might just mean the sentence is over

sorry more questions - an Epoch isn't exactly a clip is it? i got confused by the neural net analogy you gave. if the whole net changes shape every epoch i'm not sure how that relates to the graph output. when training, is the notebook feeding it sentences or phonemes or what? like if had 40 voice clips at 80 Epochs does that mean the training is 2 Epochs per clip?

an epoch is a single run through all the clips

(so with medium datasets an epoch might take a minute, with larger ones such as the /mlp/ ones it might be 5 minutes or more)

and it actually changes shape multiple times per epoch i believe it takes $batch_size clips at once and pulls the neurons a bit to make the network emulate them slightly better

i'm not sure exactly how it's trained though, but there are 2 possibilities: 1) "hey here's which character you're looking at currently as well as the previous some chunks of audio, generate the next chunk and decide where you'll be looking in the next step"; network is trained to match the clip and to look roughly forward (this is what the graph shows) b) same as above but the network is copied 200 times over, with each copy linked to the next one, and the entire blob is trained to speak the sentence correctly and the weights for each copy averaged; this would eliminate the need for having to tell it to look forward (since it'd learn that by itself) and might incentivize it to not go into

uhhh

mode more strongly but would also take up a heckton of memory i'm betting on 1) tbh, especially since i read something about tacotron 2 having

guided attention

which sounds like the "look forward" incentive

wow - that is fascinating, and it explains why my epochs were so fast generating with only 80 clips or so. by 'character' i wasn't sure if you meant like a text character (which i think would be more of a phoneme) or something more stylistic. could you clarify?

text character yeah

uberduck tacotron models are fed text directly, which is why they often mispronounce words

it is possible to train them on phonemes (the hifi-gan synthesis notebook is set to phoneme mode by default, which causes

uhhh

with character-mode models) but uberduck doesn't support that

the talknet models are trained on phonemes and that's just one way in which they're superior

which means you don't believe that - i always get a kick out of strikethroughs...

i usually use strikethroughs as a "hushed voice"/"mumble" type of thing