but there are some extra things

like there's a small subnetwork that makes sure the internal representation of a phoneme doesn't get correlated with speaker id·s

idk how or hwy but

is it similar to gsts

nnnot very

I also don't get the Mel spectrograms part. I kinda expected it to use those as a minor guidance but just full send go raw with directly genning audios

wha

if you mean why tacotron generates spectrograms instead of audio directly:

there is a paper where they go text to straight audio

it is easier for an ai to predict a spectrogram than to predict a sequence of samples

yes

but it doesn't work that well

and it's also easier to score it

just subtract it from the ground truth

vs with direct samples you have to make another network that learns to distinguish real audio from fake audio and have them fight

that's how hifi-gan training works, and it's why there's no useful loss value there

i think justinjohn tried training the stanley parable narrator on that

but idk how well it went

or if he switched from en-us to en-gb

oh actually I've trained thas as well

I didn't get great results

but

you know how these things go

I wouldn't say I did a rigorous eval or anything

I believe I did: https://discord.com/channels/768215836665446480/842883733195456572/938306099671601152

this is the one I was talking about https://arxiv.org/abs/2011.03568

might've been you then

how'd it go

also there's this recently https://speechresearch.github.io/naturalspeech/