https://livekit.io logo
Join Slack
Powered by
how to integerate vertex ai with livekit i have th...
# ask-ai
r
how to integerate vertex ai with livekit i have the api key now what code change I have to do to use the models of vertex ai
t
Thanks for reaching out, Rahul. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m responding to your question about what code changes you need to use Vertex AI models with LiveKit. Quick checklist and next steps you can apply now: • Install the Google plugin for LiveKit agents: ◦ Python: 
uv add "livekit-agents[google]~=1.2"
◦ Node.js: 
pnpm add @livekit/agents-plugin-google@1.x
• Authenticate with Vertex AI (use a service account key): ◦ Set 
GOOGLE_APPLICATION_CREDENTIALS
to your service account JSON:
Copy code
export GOOGLE_APPLICATION_CREDENTIALS="path/to/your-service-account-key.json"
• Optionally set project and location:
Copy code
export GOOGLE_CLOUD_PROJECT="your-project-id"
    export GOOGLE_CLOUD_LOCATION="us-central1"
• Update your LiveKit agent code to use the Google plugin and enable Vertex AI: ◦ Python example:
Copy code
from livekit.plugins import google

    session = AgentSession(
      llm=google.LLM(
        model="gemini-2.0-flash",
        vertexai=True,
        project="your-project-id",     # optional if env var set
        location="us-central1",        # optional if env var set
      ),
      # stt, tts, vad, etc.
    )
• Node.js example:
Copy code
import * as google from '@livekit/agents-plugin-google';

    const session = new voice.AgentSession({
      llm: google.LLM({
        model: "gemini-2.0-flash",
        vertexai: true,
        project: "your-project-id",
        location: "us-central1",
      }),
      // stt, tts, vad, etc.
    });
• For real-time speech use (Gemini Live API), use 
RealtimeModel
with 
vertexai=True
(or 
vertexai: true
in JS). Important note: the 
GOOGLE_API_KEY
is not used for Vertex AI—authentication is via the service account JSON. Do you want the full code sample for Python or Node.js (including STT/TTS/VAD setup) or help creating the service account and setting the environment variables? Sources: Gemini TTS plugin guide | LiveKit Docs | Google Gemini LLM plugin guide | LiveKit Docs | Gemini Live API plugin | LiveKit docs
r
i have service account key json
t
Great — since you have the service account JSON, here are the exact next steps to wire it into LiveKit and start using Vertex AI models. 1) Set your environment variables • Point to the service account file:
Copy code
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your-service-account-key.json"
• (Optional) Set project/location if you want to avoid passing them in code:
Copy code
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"
2) Install the Google plugin • Python:
Copy code
uv add "livekit-agents[google]~=1.2"
• Node.js:
Copy code
pnpm add @livekit/agents-plugin-google@1.x
3) Update your agent code to enable Vertex AI • Python minimal example:
Copy code
from livekit.plugins import google

session = AgentSession(
  llm=google.LLM(
    model="gemini-2.0-flash",
    vertexai=True,
    project="your-project-id",   # optional if env var set
    location="us-central1",
  ),
)
• Node.js minimal example:
Copy code
import * as google from '@livekit/agents-plugin-google';

const session = new voice.AgentSession({
  llm: google.LLM({
    model: "gemini-2.0-flash",
    vertexai: true,
    project: "your-project-id",
    location: "us-central1",
  }),
});
4) For real-time speech models (Gemini Live), use 
google.realtime.RealtimeModel
(or 
RealtimeModel
in JS) with 
vertexai=True
. 5) Test it • Run your app; authentication will use the service account JSON you pointed to. • If you hit permission errors, ensure the service account has Vertex AI roles (e.g., Vertex AI User) and the correct project is selected. Which would you like next: a full end-to-end sample (including STT/TTS/VAD), a runnable example in Python or Node.js, or help verifying service account IAM roles?
3 Views
Open in Slack
PreviousNext
Powered by