This message was deleted.
# ask-for-helps
Slackbot
09/04/2023, 2:24 PMThis message was deleted.
j
Jian Shen Yap
09/04/2023, 4:54 PM@Aaron Pham could you shed some light on some of the hardware requirements for LLM?
a
Aaron Pham
09/05/2023, 1:47 AMYou can try with GPTQ now, it will take 35GB of VRAM to run in GPTQ, so 1 A100s should be sufficient enough
Aaron Pham
09/05/2023, 1:47 AMtbh 3 T4 would also do as well
Aaron Pham
09/05/2023, 1:47 AMor even 2 A10g
m
Mykyta Holubakha
09/05/2023, 10:19 AMThanks a lot
Mykyta Holubakha
09/05/2023, 10:20 AMSo if I want to go with GPTQ (which is 4 bit), 1 should be enough, but if I want to go with 8 bit, I'll need 2
a
Aaron Pham
09/05/2023, 2:00 PMYes if you have A100s lying around 😄
2 Views