BentoML

Hi, trying out openllm. When trying to serve a gated model like llama, how do I specify my HF Access Token? Can't seem to find a clear answer, no luck yet setting an env var.

What’s the error message? Have your request for access been approved?

Yeah it has been approved, I've used the access token elswhere. I don't even know where to provide it when using openllm. So when I run
```openllm serve llama3.1:8b```
It throws an error that the model is gated and I need access (don't have access to the exact text at the moment)

That’s a bit strange, how did you see the env var? 

Where do you set the env var? I too have had this problem...

This should work in design:
`source HF_TOKEN=xxxx`.
Which version are you using?

Yeah i've got it set as HF_TOKEN in a container, confirmed when I echo the value before running.
Running openllm 0.6.7

Still getting:
```Cannot access gated repo for url <https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/resolve/main/config.json>.
Access to model meta-llama/Meta-Llama-3-8B-Instruct is restricted and you are not in the authorized list. Visit <https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct> to ask for access.```

Hmm seems to be working now, not sure what changed. It's possible some dependencies may have been conflicting, though not sure why it would impact the env vars