https://uberduck.ai/ logo
Powered by
# machine-learning
  • h
  • h
    oh for the tutorial you don't need a gpu at all, you'll be borrowing google's vast supply of unused cloud gpu·s
  • y
    Oh I thought that needed a membership
  • h
    granted it's literally the worst one they have but it's still pretty darn good
  • h
    you can pay for better gpu and more time per session if you want
  • h
    $5 a month gets you the tesla t4 which trains 2.5x as fast i believe
  • h
    and $50 gets you fancier stuff but it's not worth it i've heard
  • y
    Oh that's not bad. Okay now I'm excited
  • y
    I still wanna try the 2080ti though
  • y
    Also that was the most confusing article I've ever read
  • h
    faster (step size has 3 zeros after decimal point instead of 4) but far more unstable, would not recommend
  • h
    just use the default learning rate
  • h
    -oh
  • h
    well then keep it as such
  • h
    and maybe increase batch size to 20 or 25 or 30 or whatever number doesn't make it fail
  • c
    i always go with 1e-4 because the 5e-4 from the colab always gives pronunication errors (bad)
  • u
    i get it autocalculated based on batch size
  • u
    😎
  • h

    https://i.stack.imgur.com/lU3sx.png▾

  • h
    red = batch size 1, green = batch size some, blue = batch size all
  • h
    in this case the task is to get to a certain point
  • h
    and as you can see big batch does it more reliably
  • h
    i mean i'd go with a batch size that divides the dataset size evenly, e.g. 30 or 50 but idk how this notebook acts when it's not divided evenly, maybe it doesn't yeet valuable data away
  • h
    as for learning rate i've honestly never touched it
  • h
    oof
  • c
    wonder if colab has python function to restart the runtime automatically
  • c
    the closest thing ive seen of that is google automatically pulling up a restart runtime button whenever you install something with pip or something
  • m
    You have to execute a JavaScript script inside the code via ipython
  • m
    you can view a snippet example on the left of colab
  • m
    you moaned about this yesterday.... just deal with it lol
1...681682683...1068Latest