https://www.boundaryml.com logo
Powered by
The docs say that you can use Anthropic
# suggestions
b
The docs say that you can use Anthropic on Vertex using
Copy code
client<llm> VertexClaudeSonnet {
  provider vertex-ai
  options {
    model "claude-sonnet-4-5@20250929"
    location global
    project_id my-project-id
    anthropic_version "vertex-2023-10-16"
    credentials env.GOOGLE_APPLICATION_CREDENTIALS
  }
}
but this doesnt work. it generates a url like 
https://aiplatform.googleapis.com/v1/projects/...
even though it needs to be 
https://us-east5-aiplatform.googleapis.com/v1/projects/...
tested with baml 0.220.0
i figured it out. It's when you use the 
global
location only. For us-east5 it works.
a
actually just ran into this too when working on the new compiler, can implement a fix for this soon so you can use the global endpoint
b
thanks! since the base_url doesnt interpolate env vars we cant dynamically set the gcp project for these models
a
i think project_id should accept an env.XYZ… are you trying to set it manually with base_url?
the reason for all of the weirdness here is because google doesnt support claude models through its normal endpoint and we need to do a special hack for it
looking to improve this in the future, possibly reimagining clients a bit because of stuff like this
b
yea i know it's on the GCP end, it's so stupid and poorly designed. Their devX is the worst in the industry. You can use env for something like 
project_id env.BAML_GCP_PROJECT
but you cannot do 
base_url "https://aiplatform.googleapis.com/v1/projects/${env.BAML_GCP_PROJECT}/locations/global/publishers/anthropic/models"
a
yeah their rust SDK is lacking… compared to AWS to put it mildly
i think this should become possible in the new compiler with a format string syntax, but i think setting the special fields will still be the preferred option to manual base_url
pushed a fix, should be in the next release
3 Views
PreviousNext
Powered by