Could anyone help me out how to setup azure plugin...
# agents
h
Could anyone help me out how to setup azure plugin for stt and tts I have azure speech services deployed and i am writing below conditions
Copy code
elif self.provider == "azure":
                azure_config = {
                    "language": self.language
                }
                speech_endpoint=os.getenv("AZURE_SPEECH_ENDPOINT")
                if speech_endpoint:
                    azure_config["speech_endpoint"] = speech_endpoint
                    azure_config["speech_key"] = os.getenv("AZURE_SPEECH_API_KEY")
                else:
                    azure_config["speech_key"] = os.getenv("AZURE_SPEECH_API_KEY")
                    azure_config["speech_region"] = os.getenv("AZURE_SPEECH_REGION")

                self.stt_service = azure.STT(
                    **azure_config
                )
now i set the endpoint and key in the env but i get this error However the stt seems to be working with same config but getting issues with tts
Copy code
{
  "message": "failed to synthesize speech, retrying in 0.1s\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.12/site-packages/livekit/agents/tts/tts.py\", line 267, in _main_task\n raise APIError(f\"no audio frames were pushed for text: {self._input_text}\")\nlivekit.agents._exceptions.APIError: no audio frames were pushed for text: Hi, I am your frindly assistant. (body=None, retryable=True)",
  "level": "WARNING",
  "name": "livekit.agents",
  "tts": "livekit.plugins.azure.tts.TTS",
  "attempt": 1,
  "streamed": false,
  "pid": 28,
  "job_id": "AJ_rJx8eDoYVaAi",
  "timestamp": "2025-11-17T10:34:32.689836+00:00"
}
l
I was a bit confused by your code snippet as it doesn't look like the examples at https://docs.livekit.io/agents/models/stt/plugins/azure/, then I realised it looks more similar to the plugin implementation at https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-azure/livekit/plugins/azure/stt.py. Are you rewriting the LK plugin? Most typically an agent would call into the plugin using the API surface given in the docs (I gave the link for STT above, and TTS is here: https://docs.livekit.io/agents/models/tts/plugins/azure/)
h
@lively-winter-88668 with same config stt is working but tts doesn't ...is this a bug in livekit ?
l
What does your agent code look like? Something like this? https://docs.livekit.io/agents/models/tts/plugins/azure/#usage
h
in this doc there is a note where 3 methods are listed, so i do not use speech region and key method but i am passing speech endpoint and speech key also the note mentioned is contradicting with what i see in the code
Copy code
class TTS(tts.TTS):
    def __init__(
        self,
        *,
        voice: str = "en-US-JennyNeural",
        language: str | None = None,
        sample_rate: int = 24000,
        prosody: NotGivenOr[ProsodyConfig] = NOT_GIVEN,
        style: NotGivenOr[StyleConfig] = NOT_GIVEN,
        speech_key: str | None = None,
        speech_region: str | None = None,
        speech_endpoint: str | None = None,
        deployment_id: str | None = None,
        speech_auth_token: str | None = None,
        http_session: aiohttp.ClientSession | None = None,
    )
there is no speech_host as the input parameter
so that's why i am using endpoint and key
Copy code
self.tts_service = azure.TTS(
                    speech_endpoint = os.getenv("AZURE_SPEECH_ENDPOINT")
                    speech_key = os.getenv("AZURE_SPEECH_API_KEY")
                    voice=self.voice_id,
                )
this same config works with azure stt but doesn't work with azure tts
l
😕 Yes, I see speech_host for TTS was removed as part of https://github.com/livekit/agents/pull/2151/files but I'm unsure why. Looks like it's still in the docs. I'll keep digging.
Do you get a response if you call the REST API directly with your endpoint / key? https://learn.microsoft.com/en-us/azure/ai-services/speech-service/rest-text-to-speech?tabs=streaming