https://uberduck.ai/ logo
Powered by
# machine-learning
  • t
    I'm going for a bilingual model though Something that competently allows a model to speak both languages
  • t
    Yeah dude! But I haven't got started on Jeff and an updated Greg though.
  • c
    Hey, I was reading more of this: https://github.com/Tomiinek/Multilingual_Text_to_Speech I've noticed they have a model for people to clone voices using it
  • c
    You weren't able to make this work right?
  • c
    I was thinking about fine-tuning a voice on it to test
  • h
    i feel like something's going very wrong because i personally was able to train a different language with an english base, and 2 languages at once with a polish base
  • c
    Using IPA or just normal text?
  • h
    the different language was text (but technically also ipa because it's written as the ipa) the 2 were mixed, polish was character-based while english was arpabet; that way i could just use the polish charset (which has arpabet for reasons unknown)
  • t
    I remember having a lot of trouble with this model; once I figured out what Japanese model would suit me, something kept stalling me in training where I had to rewrite code to less and less avail
  • t
    And has anyone else successfully trained a model using your notebook for multilingual work?
  • h
    sort of successfully yes
  • h
  • h
    there's definitely an accent though, as well as some instability (which might be because i only used an hour of data per language)
  • h
    i do know one weird trick that could fix the accent problem (the language embedding with adversarial loss that google did for their multilingual tacotron) but i do not have the knowledge necessary to implement it
  • u
    The thing is i think you'd need to train a japanese dataset and an english dataset together I don't think it would learn english otherwise
  • u
    I think i'm going to try to train a multilingual dataset with japanese and english. Should i use a japanese base model or an english base model..? And i should change the cleaners from english cleaners to something that just returns the text right? Idk how to do that with pipeline
  • u
    I'll figure it out somehow...
  • t
    Regarding the cleaners, I was instructed to use the "other" function so as to avoid confusing the machine with my Japanese input (not romaji)
  • u
    ah
  • u
    i think that uses transliteration cleaners
  • t
    Is there something else I can do methodically? I'm coming up on the anniversary of trying to make this machine and all it seems to result in is me killing time as a mere excuse for not doing anything productive while somebody else figures it out
  • u
    I think i'll test it and if it works for me i'll tell you what i did to make it work
  • h
    wait not romaji?!
  • h
    with the default symbol set the model can't read japanese characters
  • h
    you'd have to make a custom one and then do basic_cleaners
  • u
    I assumed they meant like. They did phonetic transcriptions instead of romaji But if they didn't then idk
  • c
    Thinking about it, maybe a model using IPA or ARPA and using Japanese characters would work, as they are extremelly different characters
  • u
    Like if they used kanji that literally cannot work But if they use kana (one set only) and adjusted the symbol set that would work
  • h
    arpabet + romaji is what i'd do
  • u
    Arpabet + neutalk phonetic japanese is what i'd do Romaji works too though
1...951952953...1068Latest