stt=stt.FallbackAdapter( [ inference.STT.from_model_string("assemblyai/universal-streaming:en"), inference.STT.from_model_string("deepgram/nova-3"), ] ), how to set language multi

Thanks for reaching out, Sajjad. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I mean for urdu

Thanks — you mean configuring the

FallbackAdapter

to transcribe Urdu. Short answer: set

language="ur"

(or

ur-PK

) on each STT instance that supports Urdu. The

FallbackAdapter

will use whatever language each model is configured with. Example (Python):

stt = stt.FallbackAdapter([
    inference.STT(model="deepgram/nova-3", language="ur"),                 # Deepgram supports Urdu
    inference.STT(model="xai/stt-1", language="ur"),                      # xAI supports Urdu
    inference.STT(model="sarvam/stt", language="ur"),                     # Sarvam supports Urdu
])

Quick question so I can tailor the exact snippet: • Which model(s) do you want to use for Urdu (Deepgram, AssemblyAI, xAI, Sarvam, or others)? Sources: Speech-to-text (STT) models overview | LiveKit Documentation | Sarvam STT plugin guide | LiveKit Documentation, with interactive elements

give full pipeline for urdu bot AgentSession

Got it — you want a full AgentSession pipeline for an Urdu bot. A couple quick questions so I give a runnable, exact example: • Which programming language/SDK do you want the snippet in (Python, Node.js, or another)? • Which STT & TTS providers do you prefer (Deepgram, xAI, Sarvam, AssemblyAI, or others)? If unsure, I can pick Deepgram (STT) + an LLM + a TTS that supports Urdu. • Do you need live streaming (real-time) or batch (recorded audio) transcription? • Should the bot respond with speech (TTS) or text only? • Any specific LLM in mind (open-source or hosted like OpenAI/xAI)? Answer those and I’ll return a complete AgentSession pipeline code sample for Urdu. , with interactive elements

python

Thanks — I’ll provide a full Python AgentSession pipeline for an Urdu bot using LiveKit Agents with Urdu STT, a Google LLM (using your Google API key), and optional TTS. I’ll assume real‑time (streaming) audio and speech responses; tell me if you want text-only or batch instead. Below is a concise, runnable example you can adapt. Replace placeholders (API keys, project/room IDs, and model names) with your actual values. 1) Install required packages (if not already):

pip install livekit
# plus any provider SDKs you use (e.g., deepgram-sdk, google-cloud)

2) Example AgentSession pipeline (Python):

from livekit import AgentSession, inference, stt as stt_module, tts as tts_module, llm as llm_module

# --- Configuration (replace these) ---
LIVEKIT_API_KEY = "LIVEKIT_API_KEY"
LIVEKIT_API_SECRET = "LIVEKIT_API_SECRET"
LIVEKIT_WS_URL = "wss://your-livekit-server"   # if needed
GOOGLE_API_KEY = "YOUR_GOOGLE_API_KEY"         # for Google LLM
ROOM_NAME = "urdu-room"
AGENT_NAME = "urdu-agent"

# --- Build STT components configured for Urdu ---
# Use Deepgram / xAI / Sarvam models that support Urdu.
# Example using generic inference.STT constructor (models that support 'ur'):
deepgram_stt = inference.STT(model="deepgram/nova-3", language="ur")
xai_stt      = inference.STT(model="xai/stt-1", language="ur")
# Fallback adapter: tries deepgram first, then xai
stt_adapter = stt_module.FallbackAdapter([deepgram_stt, xai_stt])

# --- LLM: Google (use your API key) ---
# Use an llm wrapper that accepts provider and api_key. Replace model name as appropriate.
google_llm = llm_module.GoogleLLM(api_key=GOOGLE_API_KEY, model="gemini-pro")  # adjust model string as needed

# --- Optional TTS for Urdu (if you want audio replies) ---
# Use a TTS provider that supports Urdu (e.g., xAI or Sarvam if available).
tts = tts_module.TTS(model="xai/tts-1", language="ur")  # replace with a TTS model that supports Urdu

# --- Assemble AgentSession pipeline ---
agent_session = AgentSession(
    name=AGENT_NAME,
    stt=stt_adapter,
    llm=google_llm,
    tts=tts,                   # omit or set to None if you want text-only replies
    # Additional configuration like session timeout, voice settings, etc., can go here.
)

# --- Start the AgentSession and connect to a room ---
def start_agent_session():
    # Connect (this is pseudocode; use your SDK's connect/start method)
    agent_session.connect(api_key=LIVEKIT_API_KEY, api_secret=LIVEKIT_API_SECRET, room=ROOM_NAME, ws_url=LIVEKIT_WS_URL)

    # Register event handlers (incoming audio, messages)
    @agent_session.on("transcript")
    def on_transcript(event):
        # event.text contains Urdu transcript
        print("User (transcript):", event.text)

        # Pass transcript to Google LLM to generate a response
        response = agent_session.llm.generate(prompt=event.text, language="ur")  # ensure prompt/lang handling for Urdu
        print("LLM response:", response.text)

        # If TTS configured, synthesize and send audio back to room
        if agent_session.tts:
            audio_bytes = agent_session.tts.synthesize(response.text, language="ur")
            agent_session.send_audio(audio_bytes)

        else:
            agent_session.send_text(response.text)

    # Start listening / running loop
    agent_session.start()  # SDK-specific; starts processing streams

if __name__ == "__main__":
    start_agent_session()

Notes and quick tips: • Set each STT instance’s

language="ur"

(or

ur-PK

) so Urdu is used — the FallbackAdapter will honor per-model settings. • Use a Google LLM model name compatible with your API key (adjust

model="gemini-pro"

to your allowed model). • If you don’t want voiced replies, set

tts=None

and send text responses. • Test end-to-end with short Urdu utterances and log transcripts to verify language detection. If you want, I can: • Provide a more concrete example using a specific STT/TTS SDK (Deepgram or Sarvam) with real init code. • Swap to text-only flow. • Show exact code for sending audio into and out of a LiveKit room. Which of those would you like next? , with interactive elements

I have this session = AgentSession( # stt=stt.FallbackAdapter( # [ # inference.STT.from_model_string("assemblyai/universal-streaming:en"), # inference.STT.from_model_string("deepgram/nova-3"), # ] # ), stt = stt.FallbackAdapter([ inference.STT.from_model_string("assemblyai/universal-streaming-multilingual"), inference.STT(model="deepgram/nova-3", language="multi"), ]), llm=llm.FallbackAdapter( [ inference.LLM.from_model_string("google/gemini-2.5-flash"), inference.LLM.from_model_string("openai/gpt-4.1-mini"), ] ), tts=tts.FallbackAdapter( [ inference.TTS.from_model_string("cartesia/sonic-3:9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"), inference.TTS.from_model_string("inworld/inworld-tts-1"), ] ), turn_detection=MultilingualModel(), vad=ctx.proc.userdata["vad"], preemptive_generation=True, )

Thanks — I reviewed your AgentSession and will confirm what to change to make it work for Urdu, plus a concise runnable example and notes. Changes to make • Configure each STT model with Urdu (

"ur"

or

"ur-PK"

) —

from_model_string("assemblyai/universal-streaming-multilingual")

is fine if that model includes Urdu, but specify language explicitly when possible. •

deepgram/nova-3

should be

language="ur"

(or

"multi"

if you want auto-detect, but set

"ur"

for explicit Urdu). • Ensure your Google Gemini credentials are wired to the LLM adapter (the adapter must be created with your Google API key). • Use a TTS model that supports Urdu (e.g., Sarvam or a provider you confirmed supports Urdu); set

language="ur"

for TTS. Concise Python AgentSession example (replace placeholders):

from livekit import AgentSession, inference, stt as stt_module, llm as llm_module, tts as tts_module
from livekit.turn_detection import MultilingualModel

# Replace these with your values
LIVEKIT_API_KEY = "LIVEKIT_API_KEY"
LIVEKIT_API_SECRET = "LIVEKIT_API_SECRET"
GOOGLE_API_KEY = "YOUR_GOOGLE_API_KEY"

# STT: explicit Urdu on each model
stt_adapter = stt_module.FallbackAdapter([
    inference.STT.from_model_string("assemblyai/universal-streaming-multilingual"),  # ensure this supports ur
    inference.STT(model="deepgram/nova-3", language="ur"),
    inference.STT(model="xai/stt-1", language="ur"),   # optional extra Urdu-capable model
])

# LLM: Google Gemini via API key (ensure SDK/adapter supports api_key init)
google_llm = inference.LLM.from_model_string("google/gemini-2.5-flash", api_key=GOOGLE_API_KEY)
llm_adapter = llm_module.FallbackAdapter([
    google_llm,
    inference.LLM.from_model_string("openai/gpt-4.1-mini"),  # fallback
])

# TTS: pick Urdu-capable provider; set language
tts_adapter = tts_module.FallbackAdapter([
    inference.TTS.from_model_string("sarvam/tts", language="ur"),   # example; replace with real model id
    inference.TTS.from_model_string("inworld/inworld-tts-1", language="ur"),
])

session = AgentSession(
    stt=stt_adapter,
    llm=llm_adapter,
    tts=tts_adapter,
    turn_detection=MultilingualModel(),   # supports multilingual turn detection
    vad=None,                             # set your VAD implementation or ctx.proc.userdata["vad"]
    preemptive_generation=True,
    # add other AgentSession settings as needed
)

# Connect/start — use your SDK's connection method and auth
session.connect(api_key=LIVEKIT_API_KEY, api_secret=LIVEKIT_API_SECRET)
session.start()

Quick notes • If you want automatic language detection, set Deepgram to

language="multi"

or use a multilingual AssemblyAI model that advertises auto-detect; otherwise set

language="ur"

. • The FallbackAdapter uses the language configured per model — it does not unify languages for you. • Make sure your Google/Gemini usage is permitted by your API key and that the LLM adapter accepts

api_key

during instantiation. • Test end-to-end with short Urdu utterances and log transcripts to verify STT/TTS output. If you want, I can: • Produce a version that wires your actual VAD object (

ctx.proc.userdata["vad"]

) into the session. • Swap TTS to a concrete provider you already have keys for and show the exact init calls. Which of those would you like next? , with interactive elements

follow my code style

Thanks — I’ll match your code style and show a minimal AgentSession that keeps your structure but uses a custom STT/TTS adapter for Urdu, since built-in LiveKit STT/TTS do not currently support Urdu. I’ll include Google Gemini as your LLM and keep your FallbackAdapter pattern. Before I give the code: which external Urdu STT/TTS provider do you want to use for the custom adapters (for example: a self-hosted model, AssemblyAI custom endpoint if you have one, DeepSpeech deployment, or another API you already have credentials for)? If you don’t have a provider, I’ll show a small example using a generic HTTP-based external STT/TTS API you can replace with your actual endpoints. Sources: Speech-to-text (STT) models overview | LiveKit Documentation | Building Performant Voice Agents in India | LiveKit | Deepgram TTS | LiveKit Documentation | LiveKit Inference | LiveKit Documentation | How to Set Up Speech-to-Text Input with Text-Only Output in LiveKit Agents, with interactive elements

do i need only google api key?

Short answer: No — the Google API key is needed only for the LLM (Gemini). You’ll also need additional credentials depending on other components. What you need for a full Urdu voice bot: • Google API key — required for Google Gemini LLM only. • LiveKit API key & secret — required to run/connect the AgentSession and rooms. • STT provider credentials — required if you use an external Urdu STT (LiveKit’s built‑in STT models do not currently support Urdu). Provide API key/endpoint for whichever STT you choose. • TTS provider credentials — required if you want spoken Urdu responses (built‑in TTS models don’t currently support Urdu). Provide API key/endpoint for your chosen TTS. • Optional: VAD or other service credentials if you use external VAD. If you don’t have STT/TTS providers yet, I can: • Show how to plug a generic HTTP STT/TTS adapter into your existing code style so you only add your provider keys later. • Or produce a concrete example wiring your specific STT/TTS provider (tell me which one and its auth method). Which would you like? , with interactive elements

yes please

Thanks — I’ll give the custom-adapter template in your code style so you can plug any Urdu STT/TTS later. You can still use Google for LLM; Google Cloud STT/TTS is optional if you want a single provider. Two quick choices so I return exactly what you need: • Option A — use Google Cloud STT/TTS (you'll need GOOGLE_APPLICATION_CREDENTIALS service account JSON + GOOGLE_API_KEY for Gemini). I'll output AgentSession matching your style with Google plugins. • Option B — use generic HTTP custom adapters for STT/TTS (you provide endpoints/keys later). I'll output concrete adapter classes and AgentSession in your style. Which option do you want? Sources: Speech-to-text (STT) models overview | LiveKit Documentation | Google Cloud STT plugin guide | LiveKit Documentation | Building Performant Voice Agents in India | LiveKit | Building Performant Voice Agents in India | LiveKit | Google AI and LiveKit | LiveKit Documentation, with interactive elements

204932.112 ERROR livekit.agents Error in recv_task {"room": "console"} Traceback (most recent call last): File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/utils/log.py", line 17, in async_fn_logs return await fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/inference/stt.py", line 660, in recv_task raise APIStatusError( ...<3 lines>... ) livekit.agents._exceptions.APIStatusError: message='LiveKit Inference STT returned error: failed to create session: failed to create provider instance for session 56198301-9ac5-40cc-9221-0ace7da9a0e1: language ur not supported by model: deepgram/nova-3-general, deployment: standard', status_code=2006, retryable=True, body={'type': 'error', 'session_id': '56198301-9ac5-40cc-9221-0ace7da9a0e1', 'message': 'failed to create session: failed to create provider instance for session 56198301-9ac5-40cc-9221-0ace7da9a0e1: language ur not supported by model: deepgram/nova-3-general, deployment: standard', 'code': 2006}

Thanks — that error means the Deepgram model you used does not support

language="ur"

. The provider refused to create the STT session. What to do: • Use an STT model that supports Urdu (Google Cloud STT or Sarvam are confirmed options), or remove

language="ur"

from Deepgram. • If you want Urdu voice input, switch your STT entry to a Google Cloud STT plugin (or another Urdu-capable provider) and keep Deepgram only as a fallback if you set it to

language="multi"

and it actually supports Urdu for your deployment (your error shows it doesn't). Quick fix in your code style — replace the STT block with Google Cloud STT (requires service account JSON via GOOGLE_APPLICATION_CREDENTIALS):

from livekit.plugins import google

session = AgentSession(
    stt=stt.FallbackAdapter([
        google.STT(model="cloud/speech-to-text", languages="ur"),      # Google STT supports Urdu
        inference.STT(model="deepgram/nova-3", language="multi"),    # optional fallback (keep multi if supported)
    ]),
    llm=llm.FallbackAdapter([
        inference.LLM.from_model_string("google/gemini-2.5-flash", api_key=GOOGLE_API_KEY),
        inference.LLM.from_model_string("openai/gpt-4.1-mini"),
    ]),
    tts=tts.FallbackAdapter([
        google.TTS(language="ur-PK", voice_name="ur-PK-Standard-A"),  # Google TTS supports Urdu
        inference.TTS.from_model_string("inworld/inworld-tts-1"),
    ]),
    turn_detection=MultilingualModel(),
    vad=ctx.proc.userdata["vad"],
    preemptive_generation=True,
)

Notes: • Set GOOGLE_APPLICATION_CREDENTIALS to your Google service account JSON for Cloud STT/TTS. • Keep LiveKit API key/secret as before. • If you prefer a generic HTTP adapter instead of Google, I can produce that adapter now so you can plug your provider’s endpoint and key. Do you want the Google plugin example implemented fully (showing env vars and init), or a generic HTTP adapter you can fill with your STT/TTS provider details? , with interactive elements

Traceback (most recent call last): File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/utils/log.py", line 17, in async_fn_logs return await fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/inference/stt.py", line 660, in recv_task raise APIStatusError( ...<3 lines>... ) livekit.agents._exceptions.APIStatusError: message='LiveKit Inference STT returned error: failed to create session: failed to create provider instance for session 7415edfd-9cd9-4d38-80a6-2c9db8d330e4: language ur not supported by model: deepgram/nova-3-general, deployment: standard', status_code=2006, retryable=True, body={'type': 'error', 'session_id': '7415edfd-9cd9-4d38-80a6-2c9db8d330e4', 'message': 'failed to create session: failed to create provider instance for session 7415edfd-9cd9-4d38-80a6-2c9db8d330e4: language ur not supported by model: deepgram/nova-3-general, deployment: standard', 'code': 2006}

You're still hitting the same root error: Deepgram's selected model doesn't support

language="ur"

, so LiveKit can't create the STT provider instance. Do this to fix it (keeps your code style): 1) Use Google Cloud STT (supports Urdu) as primary, keep Deepgram as fallback but without

language="ur"

. 2) Provide Google credentials: •

GOOGLE_API_KEY

for Gemini LLM. •

GOOGLE_APPLICATION_CREDENTIALS

pointing to your service account JSON for Cloud STT/TTS. 3) Example replacement (minimal, matches your style — swap env vars as needed):

from livekit.plugins import google
from livekit import AgentSession, inference, stt, llm, tts
from livekit.turn_detection import MultilingualModel

session = AgentSession(
    stt = stt.FallbackAdapter([
        google.STT(model="cloud/speech-to-text", languages="ur"),            # Google STT (Urdu)
        inference.STT(model="deepgram/nova-3", language="multi"),          # optional fallback (no explicit 'ur')
    ]),

    llm = llm.FallbackAdapter([
        inference.LLM.from_model_string("google/gemini-2.5-flash", api_key=GOOGLE_API_KEY),
        inference.LLM.from_model_string("openai/gpt-4.1-mini"),
    ]),

    tts = tts.FallbackAdapter([
        google.TTS(language="ur-PK", voice_name="ur-PK-Standard-A"),       # Google TTS (Urdu)
        inference.TTS.from_model_string("inworld/inworld-tts-1"),
    ]),

    turn_detection = MultilingualModel(),
    vad = ctx.proc.userdata["vad"],
    preemptive_generation = True,
)

If you prefer a generic HTTP adapter instead of Google (so you can plug another Urdu STT/TTS), I can provide concrete custom STT/TTS adapter classes you can drop into your session. Which do you want: Google plugin version fully wired (with env var examples) or generic HTTP adapters? , with interactive elements

I want to user urdu pakistani

Got it — you want Pakistani Urdu (ur‑PK). Use Google Cloud STT/TTS (they support ur‑PK) and Gemini for LLM. Below is a minimal AgentSession in your code style. Replace env vars/placeholders with your keys and set GOOGLE_APPLICATION_CREDENTIALS to your service account JSON. Required creds: • LIVEKIT_API_KEY / LIVEKIT_API_SECRET • GOOGLE_API_KEY (Gemini) • GOOGLE_APPLICATION_CREDENTIALS (path to service account JSON for Cloud STT/TTS) Code (copy into your style):

from livekit.plugins import google
from livekit import AgentSession, inference, stt, llm, tts
from livekit.turn_detection import MultilingualModel

# env vars: GOOGLE_API_KEY, GOOGLE_APPLICATION_CREDENTIALS, LIVEKIT_API_KEY, LIVEKIT_API_SECRET

session = AgentSession(
    stt = stt.FallbackAdapter([
        google.STT(model="cloud/speech-to-text", languages="ur-PK"),   # primary Urdu-Pakistani STT
        inference.STT(model="deepgram/nova-3", language="multi"),    # optional fallback (no explicit 'ur')
    ]),

    llm = llm.FallbackAdapter([
        inference.LLM.from_model_string("google/gemini-2.5-flash", api_key="YOUR_GOOGLE_API_KEY"),
        inference.LLM.from_model_string("openai/gpt-4.1-mini"),
    ]),

    tts = tts.FallbackAdapter([
        google.TTS(language="ur-PK", voice_name="ur-PK-Standard-A"),  # Urdu-Pakistani TTS
        inference.TTS.from_model_string("inworld/inworld-tts-1"),
    ]),

    turn_detection = MultilingualModel(),
    vad = ctx.proc.userdata["vad"],
    preemptive_generation = True,
)

Notes: • The Deepgram model you used earlier rejects

language="ur"

— that caused the errors. Keep Deepgram as a fallback but do not set it to

ur

if it doesn’t support it. • Ensure

GOOGLE_APPLICATION_CREDENTIALS

points to the service account JSON so Google STT/TTS can be created. • Test with short Urdu-Pakistani phrases and check transcripts to confirm

ur-PK

behavior. Want me to: • Show the exact env var setup and startup commands, or • Provide a generic HTTP adapter instead (if you have a non‑Google Urdu STT/TTS)?, with interactive elements

google-tts voice names

Google Cloud TTS voice names for Urdu (Pakistan) typically follow the pattern

ur-PK-Standard-<X>

and

ur-PK-Wavenet-<X>

(where X is a letter like A, B, ...). Confirm available voices in your project at: https://cloud.google.com/text-to-speech/docs/voices Example AgentSession snippet using Google TTS voice names (your code style):

from livekit.plugins import google
from livekit import AgentSession, inference, stt, llm, tts
from livekit.turn_detection import MultilingualModel

session = AgentSession(
    stt = stt.FallbackAdapter([
        google.STT(model="cloud/speech-to-text", languages="ur-PK"),
        inference.STT(model="deepgram/nova-3", language="multi"),
    ]),

    llm = llm.FallbackAdapter([
        inference.LLM.from_model_string("google/gemini-2.5-flash", api_key="YOUR_GOOGLE_API_KEY"),
        inference.LLM.from_model_string("openai/gpt-4.1-mini"),
    ]),

    tts = tts.FallbackAdapter([
        google.TTS(language="ur-PK", voice_name="ur-PK-Standard-A"),   # try Standard/Wavenet variants
        inference.TTS.from_model_string("inworld/inworld-tts-1"),
    ]),

    turn_detection = MultilingualModel(),
    vad = ctx.proc.userdata["vad"],
    preemptive_generation = True,
)

Notes: • Replace

voice_name

with whichever

ur-PK-Standard-*

or

ur-PK-Wavenet-*

voices are listed for your Google Cloud project at the voices page. • Ensure

GOOGLE_APPLICATION_CREDENTIALS

points to your service account JSON and

GOOGLE_API_KEY

is set for Gemini. • If a specific

ur-PK-<voice>

name fails, check the Cloud TTS voices page for the exact available identifiers in your region/project. Do you want me to: • List exact voices from your Google project (I can show how to query them), or • Try a recommended default like

ur-PK-Standard-A

and provide troubleshooting if it fails? Sources: Google Cloud TTS integration guide | LiveKit Documentation, with interactive elements

220627.353 ERROR livekit.agents livekit.plugins.google.tts.TTS error, switching to next TTS {"streamed": true, "room": "console"} Traceback (most recent call last): File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/google/api_core/grpc_helpers_async.py", line 107, in _wrapped_aiter async for response in self._call: # pragma: no branch yield response File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/grpc/aio/_call.py", line 366, in _fetch_stream_responses await self._raise_for_status() File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/grpc/aio/_call.py", line 274, in _raise_for_status raise _create_rpc_error( ...<2 lines>... ) grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with: status = StatusCode.PERMISSION_DENIED details = "Your application is authenticating by using local Application Default Credentials. The texttospeech.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds ." debug_error_string = "UNKNOWN:Error received from peer ipv4142.250.200.170443 {grpc_message:"Your application is authenticating by using local Application Default Credentials. The texttospeech.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds .", grpc_status:7}"

The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/plugins/google/tts.py", line 451, in _run_stream async for resp in stream: output_emitter.push(resp.audio_content) File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/google/api_core/grpc_helpers_async.py", line 110, in _wrapped_aiter raise exceptions.from_grpc_error(rpc_error) from rpc_error google.api_core.exceptions.PermissionDenied: 403 Your application is authenticating by using local Application Default Credentials. The texttospeech.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds . [reason: "SERVICE_DISABLED" domain: "googleapis.com" metadata { key: "service" value: "texttospeech.googleapis.com" } metadata { key: "consumer" value: "projects/764086051850" } , locale: "en-US" message: "Your application is authenticating by using local Application Default Credentials. The texttospeech.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds ." ] The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/tts/fallback_adapter.py", line 315, in _try_synthesize async for audio in stream: yield audio File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/tts/tts.py", line 693, in anext raise exc # noqa: B904 ^^^^^^^^^ File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/tts/tts.py", line 433, in _traceable_main_task await self._main_task() File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/agents/tts/tts.py", line 479, in _main_task await self._run(output_emitter) File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/plugins/google/tts.py", line 412, in _run await asyncio.gather(*tasks) File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/plugins/google/tts.py", line 405, in _run_segments await self._run_stream(input_stream, output_emitter, streaming_config) File "/home/athar/python_projects/pizza_assistant/.venv/lib/python3.13/site-packages/livekit/plugins/google/tts.py", line 459, in _run_stream raise APIStatusError(e.message, status_code=e.code or -1, body=f"{e.details}") from e livekit.agents._exceptions.APIStatusError: message='Your application is authenticating by using local Application Default Credentials. The texttospeech.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds .', status_code=403, retryable=False, body=[reason: "SERVICE_DISABLED" domain: "googleapis.com" metadata { key: "service" value: "texttospeech.googleapis.com" } metadata { key: "consumer" value: "projects/764086051850" } , locale: "en-US" message: "Your application is authenticating by using local Application Default Credentials. The texttospeech.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds ." ]

Thanks — that error means Google Cloud TTS refused the request because your ADC (Application Default Credentials) has no quota project set for the Text‑to‑Speech API. I’ll give the exact, minimal steps to fix it and a small code snippet matching your style. Why this happened (brief) • ADC is being used (service account or gcloud user creds). • Text‑to‑Speech requires a quota project set on ADC or use of a service account project that has the API enabled. • Error message: "requires a quota project" / SERVICE_DISABLED. Fix steps (do these in order) 1. Enable the Text‑to‑Speech API for your GCP project in Cloud Console: https://console.cloud.google.com/apis/library/texttospeech.googleapis.com 2. Use a service account key (recommended): ◦ In Google Cloud Console, create a service account in the project that has Text‑to‑Speech enabled. ◦ Grant it roles: Text-to-Speech Admin or Text-to-Speech Client (and any other needed roles). ◦ Create and download the JSON key. ◦ Set: ▪︎ export GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json" ▪︎ export GOOGLE_CLOUD_PROJECT="your-project-id" 1. If you must use user ADC (gcloud auth application-default login), set a quota project: ◦ gcloud auth application-default set-quota-project YOUR_PROJECT_ID ◦ or set env var: export GOOGLE_CLOUD_PROJECT="your-project-id" 1. Confirm the API is enabled and quotas are available for that project. Quick verification • Run a small Google TTS sample (gcloud or python) to confirm no permission error before using in LiveKit. AgentSession snippet (ur‑PK voice) — matches your style:

from livekit.plugins import google
from livekit import AgentSession, inference, stt, llm, tts
from livekit.turn_detection import MultilingualModel

# Ensure env:
# GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
# GOOGLE_CLOUD_PROJECT=your-project-id
# GOOGLE_API_KEY=your-gemini-key

session = AgentSession(
    stt = stt.FallbackAdapter([
        google.STT(model="cloud/speech-to-text", languages="ur-PK"),
        inference.STT(model="deepgram/nova-3", language="multi"),
    ]),

    llm = llm.FallbackAdapter([
        inference.LLM.from_model_string("google/gemini-2.5-flash", api_key="YOUR_GOOGLE_API_KEY"),
        inference.LLM.from_model_string("openai/gpt-4.1-mini"),
    ]),

    tts = tts.FallbackAdapter([
        google.TTS(language="ur-PK", voice_name="ur-PK-Standard-A", project="your-project-id"),
        inference.TTS.from_model_string("inworld/inworld-tts-1"),
    ]),

    turn_detection = MultilingualModel(),
    vad = ctx.proc.userdata["vad"],
    preemptive_generation = True,
)

If you still see PermissionDenied after this: • Confirm the service account belongs to the same project as GOOGLE_CLOUD_PROJECT. • Confirm Text‑to‑Speech API is enabled for that project. • Try adding

project="your-project-id"

to

google.TTS()

as in the snippet. Would you like me to: • Show the exact gcloud commands to create the service account and set roles, or • Provide a tiny test script to verify Google TTS works outside LiveKit?, with interactive elements

why do i need to create the google project

Good question — short answer: you need a Google Cloud project so Google can enable, configure, and bill for the Text‑to‑Speech (and other) APIs and associate credentials and quotas with a concrete project. Why a project is required (brief): • APIs must be enabled per project (Text‑to‑Speech must be turned on for the project you use). • Quotas and billing are tracked at the project level; the service rejects requests that don’t have a quota project. • Service account credentials belong to a project and grant that project the permission to call the API. • The client SDK uses the project (via GOOGLE_CLOUD_PROJECT or the service account) to find enabled services and apply quotas — without it you get the PermissionDenied / “requires a quota project” error you saw. What you should do next (minimal steps): 1. Create or pick a GCP project in Cloud Console. 2. Enable the Text‑to‑Speech API for that project. 3. Create a service account in the same project, grant Text‑to‑Speech roles, download the JSON key. 4. Set environment vars: ◦ GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json ◦ GOOGLE_CLOUD_PROJECT=your-project-id ◦ (Keep GOOGLE_API_KEY for Gemini if needed) 1. Retry; the PermissionDenied error should go away. If you want, I can give the exact gcloud commands to create the project, enable the API, create the service account, and set roles. Which do you prefer? , with interactive elements

I have an api key it can be billed

You can use your API key, but Google still needs a Cloud project to enable, quota, and bill the Text‑to‑Speech API. The API key is tied to a project behind the scenes — LiveKit’s Google TTS client needs the project/quota information so the Text‑to‑Speech call is allowed. What to do next (minimal): 1. Confirm which project your API key belongs to. ◦ If you created the API key in Cloud Console, note that project's ID. 1. Enable Text‑to‑Speech for that project: ◦ Visit: https://console.cloud.google.com/apis/library/texttospeech.googleapis.com and enable it for the project. 1. Provide the project to the client: ◦ Preferred: create a service account in the same project, give it TTS role, download JSON, then set: ▪︎ export GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json" ▪︎ export GOOGLE_CLOUD_PROJECT="your-project-id" ◦ Or (if using ADC with user creds) set quota project: ▪︎ gcloud auth application-default set-quota-project YOUR_PROJECT_ID 1. In your AgentSession you can pass the project (optional):

google.TTS(language="ur-PK", voice_name="ur-PK-Standard-A", project="your-project-id")

If you want, I can give: • Exact gcloud commands to enable the API, create a service account, and set roles, or • A minimal .env and startup snippet to test Google TTS in your environment. Which would you like? , with interactive elements

if i use the inference

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!

how to use google.TTS