This message was deleted.

Hi Alex, can you share the command that you use and the full stack trace?

Can you send the outputs of the OpenLLM version?

Hi - please see below the command and the output. I only captured two, but all models return the same result.

ubuntu@ip-172-31-45-44:~$ openllm start llama --model-id huggyllama/llama-7b
Segmentation fault (core dumped)
ubuntu@ip-172-31-45-44:~$ openllm start opt
Segmentation fault (core dumped)

And OpenLLM version below:

ubuntu@ip-172-31-45-44:~$ openllm --version
openllm, 0.3.0 (compiled: yes)
Python (CPython) 3.8.10

I see that in previous posts someone had the same issue, and the problem is because of huggingface authentication. I already authenticate thru huggingface cli but still get the same error.

Can u try with the latest version?

I upgrade to 0.3.3, but having same error return.

ubuntu@ip-172-31-45-44:~$ openllm --version
openllm, 0.3.3 (compiled: no)
Python (CPython) 3.8.10

You need to install with -U to make sure it upgrades all dependencies

Thanks a lot! After upgrading openllm-core to 0.3.3, it’s working normally now.

I have added openllm-core to core deps for openllm such that next time

pip install -U openllm

should upgrade all transitive dependencies