LiveKit Community

do livekit agent sdk we can get till what audio need the user has heard to provide better context to the llm?

Thanks for reaching out, John. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.


The LiveKit Agent SDK provides metrics related to audio processing, such as the duration of audio received by the Speech-to-Text (STT) model. However, it does not explicitly track or provide information about how much audio a user has heard for use in providing context to an LLM.
If you need this functionality, you may need to implement a custom solution to track and pass this information to your LLM.
 Sources: <https://docs.livekit.io/agents-js/interfaces/agents.stt.RecognitionUsage.html|RecognitionUsage | LiveKit Agents> | <https://github.com/livekit/agents/blob/3b9965981932666b52b262a54ec9955af11e5a09/livekit-agents/livekit/agents/voice/agent_session.py#L1-L79|livekit-agents/livekit/agents/voice/agent_session.py> | <https://docs.livekit.io/agents-js/interfaces/agents.metrics.UsageSummary.html|UsageSummary | LiveKit Agents>