Why llama 3 8B is limited to 1024 tokens?
# ask-for-helps
Sebastián Cepeda
04/24/2024, 8:09 PMWhy llama 3 8B is limited to 1024 tokens?
c
Chaoyu
04/24/2024, 8:35 PMAre you following the BentoVLLM example project?
Chaoyu
04/24/2024, 8:35 PMit may has to do with this constant? https://github.com/bentoml/BentoVLLM/blob/main/llama3-8b-instruct/service.py#L11
Chaoyu
04/24/2024, 8:35 PMYou can change that line in code and redeploy
s
Sebastián Cepeda
04/24/2024, 9:09 PMThanks I'k try again 🙂, yes i was following the example :)