https://www.boundaryml.com logo
Powered by
Anthropic models not found
# help
m
Locally the BAML tests run fine using Claude Sonnet, where the model is specified as claude-3-5-sonnet-20241022. However in prod getting the error: Request failed with status code: 404 Not Found. { "type": "error", "error": { "type": "not_found_error", "message": "model: claude-3-5-sonnet-20241022" }, "request_id": "req_011CTaCHWgmrBxuKEAPaf6ri" } Any ideas?
BTW wasn't having this issue until my recent push with nextjs.config changes from here: https://discord.com/channels/1119368998161752075/1421624181363314821
g
How is your BAML client specified? Any chance it's AWS Bedrock? One guess is that your dev machine is in a different AWS region than your prod server, and that model isn't available in your prod server's region?
m
that's my guess too! but we're in us-east-1, it should be supported
using AWS Amplify to host
g
Ahh. Are the Amplify instances themselves also in us-east-1, or could they be spread around?
m
I'm not sure...was just using the pre-defined BAML config and providing my API key. couldn't see anything region-specific in anthropic itself. Am I looking in the wrong place?
a
the baml nextjs config just makes sure it can build, but it doesnt control how your app accesses the models
whats weird is that it runs locally...
and it also calls the model fine locally right?
m
yup
a
oh wait, can you try using sonnet 3.7 or 4.0?
when you test locally, do you still authenticate to us-east-1? do you ever set the AWS_REGION env var to us-east-1?
and you dont use a 'fallback' client or retries or anything i assume
m
yup aws region is set to us-east-1 locally. and nope no fallback client specified. gpt works fine btw!
a
interesting, i can hop on a call tomorrow -- it may be easier to just screenshare our aws consoles and see if there's something off
but do try a later model id
m
will do! and that would be great!!
a
how do you authenticate locally to call bedrock on your machine? aws access keys?
m
yes
a
can you paste your baml client definition here?
the model field should look like: 
model anthropic.claude-3-5-sonnet-20240620-v1:0
i think there may be an extra colon or something in the config
and baml doesnt play nicely with those. You can try:
Copy code
client .... {
   options {
       model "anthropic.claude-3-5-sonnet-20240620-v1:0"
    }
}
although yeah if it works locally then def it's something else account related
m
client CustomSonnet { provider anthropic options { model "claude-3-5-sonnet-20241022" api_key env.ANTHROPIC_API_KEY } }
FYI claude haiku is loading just fine
a
oh anthropic deprecated this model i think
the 3-5 on that version
m
that would make sense!!!
PreviousNext
Powered by