Hi, trying to deploy bentovllm-llama3-8b-insruct-service on an RTX 4060 Ti (16 GB) with 16 GB cpu ram. Any thoughts on why I get the message:
File "/opt/conda/lib/python3.10/site-packages/vllm/worker/worker.py", line 172, in initialize_cache raise_if_cache_size_invalid(num_gpu_blocks, File "/opt/conda/lib/python3.10/site-packages/vllm/worker/worker.py", line 335, in raise_if_cache_size_invalid raise ValueError("No available memory for the cache blocks. " ValueError: No available memory for the cache blocks. Try increasing
gpu_memory_utilization
when initializing the engine.