https://promptfoo.dev/ logo
Question: We're using Anyscale, how can we safely ...
# ❓|questions
p
How can we still use GPT-4 as grader (model graded assertions) and also use Anyscale? The specific issue I'm facing is that I can't seem to find a way to safely override the apiKey without exposing the key in plaintext in yaml. I know we can override the configuration in each provider configuration as shown in https://promptfoo.dev/docs/providers/openai#configuring-parameters . But I'm wondering if there's a safer way to do this without exposing the api key in the yaml file. Great tool and thank you!
i
you can set the
OPENAI_API_KEY
environment variable, which will be sent in the Authorization header
p
but that sets the key for BOTH Anyscale AND Openai? Which is not ideal in this case because I'm using both Anyscale Inference endpoint and OpenAI for llm-rubric
i
ahh, I see what you're saying
they both use the same envar under the hood if you are just overriding the OpenAI provider. Right now the workaround would be to put the api key directly in the config, like so:
Copy code
yaml
providers:
  - id: openai:chat:gpt-4
    config:
      apiKey: xxx
and the same for anyscale
^ While this example should unblock you, this PR enables you to switch API keys for a provider without hardcoding the key: Once it lands you'll be able to do:
Copy code
yaml
providers:
  - id: openai:chat:anyscale-model-123
    config:
      apiKeyEnvar: ANYSCALE_API_KEY
in order to differentiate the anyscale provider
p
That PR looks great, exactly what I’m looking for. Thank you @ian
i
This change is published in 0.40.0, so you should be good to go! LMK if you run into issues
p
Thanks @ian , amazing work!
8 Views