hey!! its awesome that openai has been added as a ...
# general
g
hey!! its awesome that openai has been added as a provider for the llm_generate expression!! is there a way to configure it so that we can use a proxy url for openai?
r
You can use the “prompt” API with a base_url option. This API supports structured outputs as well.
Copy code
import daft

# import prompt from the daft.functions library
from daft.functions import prompt

# configure the provider to "openai" with OpenAI client options like base_url
daft.set_provider("openai", base_url="...")

df = daft.from_pydict({
    "input": [
        "What sound does a cat make?",
        "What sound does a dog make?",
        "What sound does a cow make?",
        "What does the fox say?",
    ]
})

# generate responses using a chat model
df = df.with_column("response", prompt(df["input"], model="gpt-5-mini"))
Please let me know how this works for you. We recently published more documentation on these features. https://docs.daft.ai/en/stable/ai-functions/#structured-outputs-with-custom-openai-provider
g
ah nice! dint notice this fucntion!
r
It's new and snuck in 😉
❤️ 1
g
ill try it out and report
🫡 1
j
Hmm @Robert Howell thoughts on deprecating the old llm_generate in favor of
prompt
now? I believe we might be at a good spot where we can do this
(And align around the new direction for models and providers)
r
Yes, we can just make llm_generate a deprecated alias, but also will need to confirm vllm support/parity
💯 1