Has anyone gotten gpt-oss through vertex working? Using this config

client<llm> GptOss120b {
  provider vertex-ai
  options {
    model gpt-oss-120b-maas
    project_id PROJECT_ID
    base_url "https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/openai/models"
    generation_config {
      temperature 0.0
      seed 42
    }
  }
}

and it does make the call, but it seems to return a response in a different form, so baml fails to parse it. I can't figure out how to parse it. I tried using

openai-generic

as the client provider, thinking it would match the openai format, but that failed too with a 404. Here's an example output from the raw-curl

{
  "candidates": [
    {
      "content": {
        "role": "assistant",
        "parts": [
          {
            "text": "Why don’t skeletons fight each other?\n\nThey don’t have the guts."
          }
        ]
      },
      "finishReason": "STOP"
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 71,
    "candidatesTokenCount": 47,
    "totalTokenCount": 118
  }
}

and here's the error from baml

Unspecified error code: 2 Failed to parse into a response accepted by baml_runtime::internal::llm_client::primitive::vertex::types::VertexResponse: {"usageMetadata":{"promptTokenCount":71,"candidatesTokenCount":47,"totalTokenCount":118}}

Caused by:
    missing field `candidates`

Request options: {"generation_config":{"temperature":0.0,"seed":42}}

and when i try

openai-generic

Unspecified error code: 404 Request failed with status code: 404 Not Found, 
<no response or invalid utf-8>

Request options: {"model":"gpt-oss-120b-maas","project_id":"{}","generation_config":{"temperature":0.0,"seed":42}}

You can set the response type!

hm @Vaibhav (BAML 🐑) getting

Unspecified error code: 2 error sending request

If you haven't yet, try enabling the proxy (See API Keys button)

Request options: {"client_response_type":"openai","generation_config":{"temperature":0.0,"seed":42}}

oh what?! let me check why that isn't work

@nikitoshina it seems you got this to work -- do you use vertex?

i just updated baml from 0.214 to 0.215 and now seeing

Unspecified error code: 400 Request failed with status code: 400 Bad Request, 
{
  "error": {
    "code": 400,
    "message": "Invalid JSON payload received. Unknown name \"client_response_type\": Cannot find field.",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.BadRequest",
        "fieldViolations": [
          {
            "description": "Invalid JSON payload received. Unknown name \"client_response_type\": Cannot find field."
          }
        ]
      }
    ]
  }
}

the docs make it seem like this is maybe only allowed for openai providers?

oh interesting, ok that is def a bug! i'll patch this for a friday!

ty! no rush from my perspective, i use groq and baseten right now, just looking for more capacity haha

https://github.com/BoundaryML/baml/pull/2847/changes#diff-c19c26f94d7772f767a271cbfeac8af3df24e9a60e6ed95153e58bc109ab8d22 it'll be in fast 🙂

i use vertex for gemini. Haven't used it for oss directly. You can try OpenRouter with BYK if you want a quick workaround.

vertex is so annoying