https://livekit.io logo
Join Slack
Powered by
This message was deleted.
# ask-ai
s
This message was deleted.
t
Thanks for reaching out, Mridul. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To process a transcript in the 
stt_node
before passing it to the LLM in LiveKit, you can override the 
stt_node
method in your custom Agent class. Here's an example of how you can achieve this:
Copy code
from livekit import rtc
from livekit.agents import ModelSettings, stt, Agent
from typing import AsyncIterable, Optional

async def stt_node(
    self, audio: AsyncIterable[rtc.AudioFrame], model_settings: ModelSettings
) -> Optional[AsyncIterable[stt.SpeechEvent]]:
    async def filtered_audio():
        async for frame in audio:
            # insert custom audio preprocessing here
            yield frame
            
    async for event in Agent.default.stt_node(self, filtered_audio(), model_settings):
        # insert custom text postprocessing here
        # This is where you can modify the transcript before it goes to the LLM
        # For example, you could remove filler words, correct common errors, etc.
        yield event
This approach allows you to: • Preprocess the audio before it's transcribed • Postprocess the transcribed text (in the form of speech events) before it's sent to the LLM The 
stt_node
is part of the STT-LLM-TTS pipeline and is responsible for converting user audio input into text that will be processed by the LLM. Let me know if you need further assistance! Sources: Voice pipeline nodes | LiveKit Docs
m
Is this correct - async def stt_node(self, text: AsyncIterable[str], model_settings: Optional[dict] = None) -> Optional[AsyncIterable[rtc.AudioFrame]]: parent_stream = super().stt_node(text, model_settings) if parent_stream is None: return None async def process_stream(): async for event in parent_stream: if hasattr(event, 'type') and str(event.type) == "SpeechEventType.FINAL_TRANSCRIPT" and event.alternatives: transcript = event.alternatives[0].text validated_trancript = self.stt_validator.validate_message(transcript) yield event return process_stream()
t
I'm sorry, but I don't have the information you need at the moment. If you would like a member of the LiveKit team to respond, please ask your question in one of the other channels.
m
So basically I have implemented a function which takes in the stt transcripts and returns the processed transcript. I want the processed transcript to be passed in LLM. How can I do this?
Open in Slack
PreviousNext
Powered by