how many steps do you typically train for on the pipeline?

uhhh honestly i don't know

i'd say to look at loss but i honestly can't tell the difference between 0.3 loss and 0.1 loss when synthesize

The graphs look good and the loss is low but the audio in tensorboard is really bad

epoch: 93/69420 | batch: 7/8 | loss: 0.13 | mel: 0.14 | gate: 0.002 | t: 3.94s | w: 1.14h

how indicative of how the model sounds are the samples in tensorboard?

that's because it uses a really quick and dirty vocoder, meaning the thing that turns frequencies into audio

it's there mostly so you can hear if the underlying pronunciation is good

the synthesis notebook uses hifi-gan as its vocoder which does the polishing

ah that makes sense. it sounds like it's cut all the top frequencies off

i mean that too because we synthesize in 22khz

i did another one and its saying: this is a test of the cupcakke talknet model. The quick fox jumped over the lazy dog

and sorry for ping

Yeah that's what i figured, ty

I'll check

Any idea what might be causing this in the inferencing notebook?

Downloading...
From: https://drive.google.com/uc?id=1-WDt7eZw6vpkL9SDf0-ZwWT4rHOJ6zAq
To: /content/TTSmodel.pt
100%|██████████| 353M/353M [00:05<00:00, 67.4MB/s]
Initialized Torchmoji GST
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-f838ff87474a> in <module>()
    104 
    105 device = "cpu" if not use_gpu else "cuda:0"
--> 106 taco.from_pretrained(warm_start_path="TTSmodel.pt", device="cpu")
    107 if device != "cpu":
    108     taco.cuda()

1 frames
/usr/local/lib/python3.7/dist-packages/uberduck_ml_dev/models/base.py in from_pretrained(self, warm_start_path, device, ignore_layers, model_dict)
     53         dummy_dict.update(model_dict)
     54         model_dict = dummy_dict
---> 55         self.load_state_dict(model_dict)
     56         if device == "cuda":
     57             self.cuda()

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
   1496         if len(error_msgs) > 0:
   1497             raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
-> 1498                                self.__class__.__name__, "\n\t".join(error_msgs)))
   1499         return _IncompatibleKeys(missing_keys, unexpected_keys)
   1500 

RuntimeError: Error(s) in loading state_dict for Tacotron2:
    Unexpected key(s) in state_dict: "speaker_embedding.weight".

uhh it wants you to upload a config

which is unnecessary imo

gimme a sec

ty!

use this https://colab.research.google.com/drive/1wXWuhnw2pdfFy1L-pUzHfopW10W2GiJS

Yeah it's not very understandable imo Idk if you used reference audio but if you didn't that might help I'm not super experienced with what the output on talknet sounds like given certain situations so i can't tell if it's not trained properly or if you just need more data

i didnt use reference audio

Dw, i like pings so o can check on things lol

It often makes it sound clearer if you do

here are my settings

Maybe try reference audio You can record yourself speaking, or use the output of a clear tts voice It's not a fix-all, but it can help with clearness You still probably need more data though

Disable ``reduce metallic noise``

It works well with the pony voices tho

Also try adding more data to your training dataset and retrain

got this, they're in config_v1b.json