Hi, from the docs it looks like the main way to authorize the websockets provider is through an Authorization header. My current setup requires the websocket to be established, then a specific message sent over the socket with the authorization token, and then promptfoo can go ahead and do its thing. Is there something I missed in the docs that would support this? Alternatively, can I programatically setup the websocket (say via extension) and then have promptfoo use it for its test?

Hello, you can implement this using a custom JavaScript provider. https://www.promptfoo.dev/docs/providers/custom-api/. Your callAPI logic could incorporate your authorization logic. The official websocket provider logic is here if you need a starting point: https://github.com/promptfoo/promptfoo/blob/main/src/providers/websocket.ts

Some minor follow up Qs: - will

callApi

be called on each prompt? - If I do some setup with extensions and set context vars will they be available in the

callApi

context?

1. Yes, callApi will be called on each prompt. 2. Yes, context vars are available in context