https://promptfoo.dev/ logo
Distinguishing providers with the same ID
# ❓|questions
l
I am using promptfoo as a node package, with portkey. I have set up 2 providers to query gpt-4o through both Azure and OpenAI. This results in both provider IDs being identical: 'portkey:gpt-4o' (as from what I understand the ID has to contain the model name as the suffix?) This makes it difficult to distinguish the different providers in the web view results. Is there any way to add a custom label for a provider, so I can distinguish gpt-4o-oai and gpt-4o-azure? https://cdn.discordapp.com/attachments/1278281586063708234/1278281586370150483/image.png?ex=66d03bff&is=66ceea7f&hm=fb490783e1ce2216b6a2897a7029297f4b777d3cdc91722ba4c936b8105df0f1&
Associated provider configs (the portkeyConfig value does not seem to have any effect here, so I am relying on just the model names in the IDs alongside the 'provider' config property)
Copy code
{
    id: 'portkey:gpt-4o',
    config: {
      provider: 'azure',
      apiKey: process.env.PORTKEY_API_KEY,
      portkeyConfig: 'azure-config',
      apiBaseUrl: 'https://api.portkey.ai/v1/proxy',
    },
  },
{
    id: 'portkey:gpt-4o',
    config: {
      provider: 'openai',
      portkeyConfig: 'oai-config',
      apiKey: process.env.PORTKEY_API_KEY,
      apiBaseUrl: 'https://api.portkey.ai/v1/proxy',
    },
m
Hi @lazeratops , Great question! To distinguish between the two providers in the web view results, you can add a
label
property to each configuration. This label will help you easily identify which provider corresponds to Azure and which to OpenAI. Here's how you can do it:
Copy code
javascript
{
  id: 'portkey:gpt-4o',
  label: 'gpt-4o-azure',
  config: {
    provider: 'azure',
    apiKey: process.env.PORTKEY_API_KEY,
    portkeyConfig: 'azure-config',
    apiBaseUrl: 'https://api.portkey.ai/v1/proxy',
  },
},
{
  id: 'portkey:gpt-4o',
  label: 'gpt-4o-openai',
  config: {
    provider: 'openai',
    portkeyConfig: 'oai-config',
    apiKey: process.env.PORTKEY_API_KEY,
    apiBaseUrl: 'https://api.portkey.ai/v1/proxy',
  },
}
This way, you can clearly see which provider is being used when reviewing the results Please let me know if you encounter any other issues or if you have any more questions!
l
Perfect, that worked, thank you!
2 Views