https://www.boundaryml.com logo
Powered by
🟡 Changing Baml Client at Runtime with User Prefe...
# questions
t
Let's say that I have a BAML function called AgentLoop, and that we allow our users to set their prefernce between whether it's called with either Sonnet or Opus. How do I change the Baml Client at runtime?
n
Copy code
py
import os
from baml_py import ClientRegistry

# Create a new client registry
client_registry = ClientRegistry()

# Add an LLM client
client_registry.add_llm_client(
    provider="openai-generic",
    name="my_model_client",
    options={
        "model": "openai/gpt-4o-mini",
        "api_key": os.getenv("OPENROUTER_API_KEY"),
        "base_url": "https://openrouter.ai/api/v1",
    },
)

# Set it as the primary client
client_registry.set_primary("my_model_client")

# Now use it with BAML functions
from baml_client import b
result = await b.SomeFunction(
    arg="value",
    baml_options=client_registry
)
you can create a client with ClientRegistry add a client and set it as primary
you can also register the clients in baml and then just set it to the one you want
t
But my clients are already created in clients.baml. I just want to use a different client that I already have not create one
n
Copy code
py
client_registry = ClientRegistry()
    baml_options = {"client_registry": client_registry}
    
    # Switch to OpenRouterClient (defined in test.baml)
    print("Using OpenRouterClient...")
    client_registry.set_primary("OpenRouterClient")
    result1 = await b.WriteStory(
        prompt="Say hello",
        baml_options=baml_options
    )
    print(f"Result: {result1}\n")
This worked for me with similar providers openai and openrouter. But failed for Gemini Models. Should work in your case!
v
yep you can call 
set_primary
on clients defined in 
.baml
files!
a
maybe it doesn't answer the question but for us setting the model names from the environment variables has worked well
a
This is what I do too
2 Views
PreviousNext
Powered by