I suppose it actually depends on the dataset.

changing the wav files to either increase number or better wavs has a greater outcome

what do you mean?

I mean what you just said, lol

ooo, lol

making the learning value smaller can eek out a little more training, but the gain will be negligible.

i'm redoing the Aerith dataset now. The previous dataset had her yelling for some wavs like "HURRY, we have to get out of here". I'm guessing removing all of those wavs should improve talknet? Well, we'll see once the training is finished later tonight

overral, like 12% of the wavs were removed

yes yes it will

talknet dislikes changes in tone

yea, i'm hoping the removed files will improve the outcome. Tomorrow is the last day before the $50 google colab subscription ends. Trying to squeeze in as much testing as possible before tomorrow ends

afterwards, i'll probably just stick to talknet training to 200 epochs (or maybe 250 epochs) for all future datasets

around 250 is where it stops updating for every dataset I've tried so far, even for the 5000 wavs one from Atenborough

Alright, so here's the result of the Aerith retraining without the yelling wavs. noyell 200 = 200 epochs and 2000 steps. noyell 400 = 400 epochs and 4000 steps. There is some improvement, but I was hoping for more. Compare that to "talk 200" which is the previous training with 200 epoch the yelling wavs included.

I'm listening to some of Aerith's wav files and noticed a problem. All of the wavs having Aerith being "expressive" in some way. Which makes sense. Video game characters usually only say something where there's a new situation. moment, or time. And that means training may be more difficult

Speech-to-speech voice transfer/converter. It sounds okay, but far from perfect. Pretty sure one of my major bottlenecks is the model parameter size, but its already quite chunky at 200MB+ and training times are already long af. Wish GPUs weren't so expensive

singing needs some work lol

Thats Talknet

Hey guys, was TalkNET the best model for Text to Speech (TTS) and Speech to Speech (STS) at the moment? I have looked into other projects like Tacotron and FastSpeech2, but it seems the consensus is that TalkNET prevails for quality/efficiency? What were the differences between these methods that may cause a difference in quality? Was Hi-Fi GAN the best method for audio super-resolution / noise removal? I've seen other methods for improving low-quality audio here, but Hi-Fi GAN seems pretty great, considering it removes both noise and echo? Does it "upscale" the audio as well? Lastly, was there a GUI similar to YouMeBangBang's Utilities for training TalkNET? https://github.com/youmebangbang/deepvoice-model-utilities Any insights would be greatly appreciated!

talknet uses hard alignments generated by a separate pitch and duration predictors, whereas tacotron is a squishy blob that just figures things out; this means talknet is more stable (doesn't stroke) but lower quality (especially since it has way fewer neurons) it seems that there are 2 separate algorithms called hifi-gan, one for denoising (as you described) and one for converting spectrograms to audio (which is the one uberduck uses) i recall someone posting a link to another algorithm that might be better but i couldn't really tell the difference other than it being a bit smoother uberduck doesn't do supersampling, the synthesis notebooks use something else for it i think not to my knowledge, but the training notebook is easy enough to use methinks

comparing daniel from his original program to the tacotron version. the tacotron version removed a lot of his... concatenation sounds? and i lowkey like the pacing of it better. the sentence i made it read would be this: This is a fully automated tool that tunes the Dreamtonics Neural Network Inference (DNNI) engine for maximum performance. DNNI is used to accelerate AI voice rendering. By pressing start, it will run a few benchmarks and pick the best configuration for your machine. i put this in the tacotron model: This is a fully automated tool that {T UW1 N Z} the {D R IY1 M T AA2 N IH0 K S} Neural Network Inference {D IY1} {EH1 N} {EH1 N} {AY1} engine for maximum performance. {D IY1} {EH1 N} {EH1 N} {AY1} is {Y UW1 Z D} to accelerate A I voice rendering. By pressing start, it will run a few benchmarks and pick the best configuration for your {M AH0 SH IY1 N}.

Oh so that's what you meant, I thought you meant adding concatenative samples to recorded samples 💀 my bad. Otherwise I do agree the concatenative sound of his voice disappeared somewhat which is impressive

Because I don't know how Tacotron2 handles mixed speaker datasets, which I still wanted to try out honestly. At the same time I doubt it'll be any good

thats been done actually

done to death i might say

all it sounds like is if you sentence mixed with two different voices

they dont blend, they just both get pieces drawn from them

I just wonder if it works well as a pre-trained model basis or not, hence I was curious. I am working with a buddy alot on hMm synthesis and it shows a lot of success with mixed speaker trained models, especially as a pretrained model for other models to build on top of

But since tacotron works so vastly different I'm not sure what to expect