Slackbot
05/31/2023, 12:30 PMsauyon
05/31/2023, 6:01 PMAaron Pham
05/31/2023, 6:09 PMYilun Zhang
06/01/2023, 12:51 PMbfloat16
, then the saved model will also be in the same format? Maybe it’s also on transformers side that, even though config.json
specifies the model to be bfloat16
, the loaded model is still float32
. I will look further into this.sauyon
06/01/2023, 3:18 PMsave_model
, but I'm not exactly sure.Yilun Zhang
06/01/2023, 6:00 PMtorch_dtype=torch.bfloat16
while loading the model in memory. The saved model is now only 13GB 😄Aaron Pham
06/01/2023, 6:27 PMkwargs
for load_model
Aaron Pham
06/01/2023, 6:27 PMsauyon
06/01/2023, 6:28 PM