Hi a question regarding observability feat - we ar...
# agents
w
Hi a question regarding observability feat - we are using openai realtime model with a secondary STT plugin (azure speech) and want to be able to 'select' the additional plugin as the one showing in the agent insight - is there a way to do that? https://docs.livekit.io/agents/observability/data/#session-transcripts-and-reports https://docs.livekit.io/agents/models/realtime/#scripted-speech-output
l
Are you doing this? From https://docs.livekit.io/agents/models/realtime/#separate-tts
Copy code
session = AgentSession(
    llm=openai.realtime.RealtimeModel(modalities=["text"]), # Or other realtime model plugin
    tts="cartesia/sonic-3" # Or other TTS instance of your choice
)
If so, I would expect only the separate TTS to show any data, I thought the realtime TTS was effectivley bypassed in this scenario
w
@lively-winter-88668 apologies I meant STT for better transcription - and then trying to select user transcription in agent insight coming for the second plugin not from the realtime model
realtime model transcription is pretty bad in some languages but the audio model is good in responding so we are trying to get a transcription from azure speech services that do a much better job.
l
The Azure STT plugin supports language: https://docs.livekit.io/reference/python/v1/livekit/plugins/azure/index.html#livekit.plugins.azure.STT - that doc you sent was for the realtime model. It may be easier to use the plugin model, rather than realtime, so you have greater control.
w
@lively-winter-88668 we attempted a voice pipeline, how ever for hebrew voice pipeline has limited STT options that sound good, openai tts model is ok, yet due to sentence segmentation logic intonation varies mid sentence and sounds terrible. so until 11labs v3 arrives with low latency we are kinda stuck with the realtime model. we managed to use the azure plugin in parallel to the realtime model to get better text transcription in fixed language - with some issues in matching the right message to the correct turn. It would be nice to be able to set transcription language for realtime model - the real-time models support this both on openai and azure, how ever it seems not configurable from livekit realtime plugin -it might solve the need for a separate transcript plugin.
👀 1