https://uberduck.ai/ logo
Powered by
# machine-learning
  • m
    sounds exactly like ground truth
  • m
    Copy code
    for param in model.parameters():
    param.requires_grad = False
  • m
    this will freeze ALL parameters
  • m
    as an exmaple
  • m
    lol
  • m
    its quite simple to freeze
  • m
    if u wanna freeze a layer do
  • m
    Copy code
    # we want to freeze the fc2 layer
net.fc2.weight.requires_grad = False
net.fc2.bias.requires_grad = False
  • h
    ooooh
  • h
    wait so where do i put this in the training function?
  • m
    before training
  • m
    lik
  • m
    e
  • m
    load the model
  • m
    here is reference from another codebase
  • m
  • h
    bhhh colab isn't loading so i can't check
  • m
    search for 
    freeze_transformer
  • h
    mm so i'll need to make a fork huh
  • h
    or i guess edit the module after it's downloaded
  • u
    This is pretty damn impressive:
  • u
  • h
    ok i think i succeeded? like i feel the loss is going down very slightly
  • h
    or maybe it's just oscillating in place because tacotron is non-deterministic
  • h
    ok yep yep it's going down
  • h
    for reference i'm training with only the speaker embedding unfrozen
  • h
    on an 8-speaker model because that's the best i have access to
  • h
    seems to be near apple bloom which is odd, i'd've expected twilight sparkle
  • h
    would probably make more sense to unfreeze the topmost layer
  • h
    so i think just these right
1...804805806...1068Latest