https://www.boundaryml.com logo
🟡 Bedrock system messages and their payload forma...
# questions
a
i think you need to do
_.role("system")
i may be misunderstanding your problem though
also notably the default role on openai is
system
, but on bedrock i think it's
user
no matter what model you use
m
I see - yeah when I do "system" role in this prompt it just uses "user" and doesn't add the "system : [ "text":... ] prompt (if that makes sense) In practice, I feel like I've heard that system prompt vs user prompt doesn't make thatttt much a difference depending on the model - curious if you've seen the same for sonnet?
a
yeah that should end up in the system section, i think i fixed that on the new compiler coming soon can look into fixing it on current version ~this friday
a lot of the defaults and provider handling is historical from when it was implemented and may need to be updated or adjusted, definitely going to do a pass on that before we release the new version
and tbh i haven’t even checked for sonnet but i suspect it handles it
AWS is one of the weirder providers
m
ok nice thanks! and yeah I feel ya. appreciate all the help
2 Views