type='realtime_model_metrics' label='livekit.plugi...
# ask-ai
b
type='realtime_model_metrics' label='livekit.plugins.google.beta.realtime.realtime_api.RealtimeModel' request_id='GR_95e2b3404566' timestamp=1758617110.5003326 duration=10.248751640319824 ttft=0.4549386501312256 cancelled=False input_tokens=4043 output_tokens=45 total_tokens=4088 tokens_per_second=4.390778660589703 input_token_details=InputTokenDetails(audio_tokens=1, text_tokens=4042, image_tokens=0, cached_tokens=0, cached_tokens_details=CachedTokenDetails(audio_tokens=0, text_tokens=0, image_tokens=0)) output_token_details=OutputTokenDetails(text_tokens=45, audio_tokens=0, image_tokens=0) what are the atributes signifying
t
Thanks for reaching out, Aman. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thank you for sharing the log details! Let me walk you through what each attribute means to help you better understand the performance and usage metrics for real-time AI model interactions: • type: This tells us the kind of metric we're looking at, which is 'realtime_model_metrics' here. • label: Shows the source or type of the model, like 'livekit.plugins.google.beta.realtime' in this case. • request_id: A unique ID that helps us track this specific request. • timestamp: The exact time, in seconds, when the response was created. • duration: How long, in seconds, it took to complete the response. • ttft: Stands for "Time to First Token" — basically, how quickly the first token was received (0.4549 seconds here). • cancelled: Indicates whether the request was cancelled or not (here, it wasn’t). • input_tokens: The total number of input tokens involved (4043 here), covering audio, text, and image tokens. • output_tokens: Number of tokens generated in the output (45 here). • total_tokens: The sum of input and output tokens (4088 in this example). • tokens_per_second: How many output tokens are generated each second (4.39 here). • input_token_details: A detailed look at input tokens by type — audio: 1, text: 4042, image: 0. • output_token_details: Similarly, a breakdown of output tokens — text: 45, audio: 0, image: 0. These metrics are really helpful for keeping an eye on and analyzing how the real-time model interactions are performing. If you have any questions or want to dive deeper into any part, I’m here to help! Sources: agents/src/metrics/usage_collector.ts | livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/experimental/realti...
b
• output_token_details: Similarly, a breakdown of output tokens — text: 45, audio: 0, image: 0. i see audio:0 in the output_token_details even though i am using a gemini live api .
t
I'm here to help you understand why the output_token_details might be showing audio: 0 when using the Gemini Live API. This can happen for a few reasons: • The model’s response didn’t include any audio output. Sometimes, if the conversation doesn’t call for audio, or the model decides to respond with text only, this can occur. • The modalities parameter in your setup might be set to exclude audio, such as being in text-only mode. • There could be session or runtime commands that have overridden the default settings, turning off audio generation. • The model’s response may not have included audio data in the server_content.model_turn, which is necessary for generating audio tokens. To help you troubleshoot this, here are some steps you can try: 1. Double-check that the modalities parameter in your RealtimeModel configuration includes AUDIO. 2. Make sure the model isn’t being forced into text-only mode through configuration or runtime commands. 3. Review your session logic to confirm that audio isn’t being programmatically overridden. 4. Take a look at the model’s response to ensure it contains audio data in the correct format. If you’re expecting audio but it’s not coming through, it might be worth reviewing the model’s response generation or configuration settings. Please feel free to reach out if you want to go through this together—I’m here to support you every step of the way! Sources: livekit-plugins/livekit-plugins-google/livekit/plugins/google/beta/realtime... | Google integration guide | LiveKit Docs | agent/main.py | Gemini Live API integration guide | LiveKit Docs