@millions-hair-24956 I have added logs to print tts metrics emit

async def _metrics_monitor_task(
    self, event_aiter: AsyncIterable[SynthesizedAudio]
) -> None:
    """Task used to collect metrics"""
    audio_duration = 0.0
    ttfb = -1.0
    request_id = ""

    def _emit_metrics():
        nonlocal audio_duration, ttfb, request_id
        if not self._started_time:
            return

        duration = time.perf_counter() - self._started_time

        if not self._mtc_pending_texts:
            return

        text = self._mtc_pending_texts.pop(0)
        if not text:
            return

        metrics = TTSMetrics(
            timestamp=time.time(),
            request_id=request_id,
            ttfb=ttfb,
            duration=duration,
            characters_count=len(text),
            audio_duration=audio_duration,
            cancelled=self._task.cancelled(),
            label=self._tts._label,
            streamed=True,
            error=None,
        )
        print("self._tts_metrics 2", metrics)
        self._tts.emit("metrics_collected", metrics)
        print("EMITTED TTS METRICS 2")

        audio_duration = 0.0
        ttfb = -1.0
        request_id = ""
        self._started_time = 0

I see them coming self._tts_metrics 2 TTSMetrics(request_id='b53eec06f58d', timestamp=1740561184.724284, ttfb=0.5130687090568244, duration=0.5131372499745339, audio_duration=0.69578231292517, cancelled=False, characters_count=5, label='livekit.plugins.elevenlabs.tts.TTS', streamed=True, error=None) EMITTED TTS METRICS 2 But this metrics_collected is never handled in the pipeline agent later. What can be the cause? All socket handlers of "metrics_collected" do not work

The issue you're experiencing with the

metrics_collected

event not being handled in the pipeline agent could be due to several reasons. Here are a few things to check: 1. Event Subscription: Ensure that the event handler for

metrics_collected

is properly registered. According to the LiveKit documentation, you need to subscribe to the

metrics_collected

event using the

on

method. Verify that this subscription is correctly set up in your code. 2. Event Emitter: Confirm that the

emit

method is correctly implemented and that the event is being emitted on the correct instance. The

emit

method should be part of an

EventEmitter

class or similar, and the instance emitting the event should be the same one that has the event handler registered. 3. Event Loop: Check if the event loop is running and not blocked. If the event loop is blocked, it might prevent the event from being processed. 4. Debugging: Add additional logging around the event subscription and emission to ensure that the event is being emitted and that the handler is registered. This can help identify where the breakdown is occurring. Would you like more detailed guidance on setting up event handlers or debugging event emissions in LiveKit? Sources: - LiveKit documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

Suppose I use elevenlabs tts plugin inside fallback adapter. How to catch metrics collected? @millions-hair-24956

To catch

metrics_collected

events when using the ElevenLabs TTS plugin inside a fallback adapter, you need to ensure that you have properly set up event listeners for the

metrics_collected

event. Here's a general approach: 1. Subscribe to the Event: Use the

on

method to subscribe to the

metrics_collected

event. This should be done on the instance of the TTS plugin or the fallback adapter that is emitting the event. 2. Implement the Event Handler: Define a function that will handle the

metrics_collected

event. This function should take the metrics data as a parameter and process it as needed. 3. Ensure Proper Emission: Verify that the

emit

method is correctly implemented in the TTS plugin or fallback adapter, and that it is being called with the

metrics_collected

event. Here's a basic example in Python:

from livekit.agents.tts import TTS

# Assuming `tts_instance` is your ElevenLabs TTS plugin instance
tts_instance.on("metrics_collected", handle_metrics)

def handle_metrics(metrics):
    print("Metrics collected:", metrics)

Would you like more detailed guidance on setting up event handlers or using the ElevenLabs TTS plugin? Sources: - LiveKit documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

Did you ever resolve this issue @delightful-airline-96880?

yes

elevenlabs TTS doesn't seem to emit ttfb metrics for me

All I do is overriding start method to work with failovers:

class CustomVoiceAssistant(agents.voice_assistant.VoiceAssistant):
    def start(
            self, room: rtc.Room, participant: rtc.RemoteParticipant | str | None = None
    ) -> None:
        """
        Need to rewrite this method to have control over self._stt.on callbacks
        """
        if self._started:
            raise RuntimeError("voice assistant already started")

        @self._stt.on("metrics_collected")
        def _on_stt_metrics(stt_metrics: metrics.STTMetrics) -> None:
            self.emit(
                "metrics_collected",
                metrics.PipelineSTTMetrics(
                    **stt_metrics.__dict__,
                ),
            )

        def create_tts_handler(instance):
            @instance.on("metrics_collected")
            def _on_tts_metrics(tts_metrics: metrics.TTSMetrics) -> None:
                speech_data = SpeechDataContextVar.get(None)
                sequence_id = speech_data.sequence_id if speech_data else None
                self.emit(
                    "metrics_collected",
                    metrics.PipelineTTSMetrics(
                        **tts_metrics.__dict__,
                        sequence_id=sequence_id,
                    ),
                )

        if hasattr(self._tts, "_tts_instances"):
            for tts_instance in self._tts._tts_instances:
                create_tts_handler(tts_instance)
        else:
            logger.warning(f"No _tts_instances for {self._tts}")
        create_tts_handler(self._tts)

        @self._llm.on("metrics_collected")
        def _on_llm_metrics(llm_metrics: metrics.LLMMetrics) -> None:
            speech_data = SpeechDataContextVar.get(None)
            sequence_id = speech_data.sequence_id if speech_data else None
            (self.emit(
                "metrics_collected",
                metrics.PipelineLLMMetrics(
                    **llm_metrics.__dict__,
                    sequence_id=sequence_id,
                ),
            ))

        def create_llm_handlers(instance):
            @instance.on("metrics_collected")
            def _on_llm_metrics(llm_metrics: metrics.LLMMetrics) -> None:
                speech_data = SpeechDataContextVar.get(None)
                sequence_id = speech_data.sequence_id if speech_data else None
                (self.emit(
                    "metrics_collected",
                    metrics.PipelineLLMMetrics(
                        **llm_metrics.__dict__,
                        sequence_id=sequence_id,
                    ),
                ))

            @instance.on("llm_pipeline_memory_updated")
            def llm_pipeline_memory_updated(pipeline_memory: dict) -> None:
                self.session_ctx.llm_pipeline_memory = pipeline_memory

        if hasattr(self._llm, "_llm_instances"):
            for llm_instance in self._llm._llm_instances:
                create_llm_handlers(llm_instance)
        else:
            logger.warning(f"No _llm_instances for {self._llm}")
        create_llm_handlers(self._llm)

        @self._vad.on("metrics_collected")
        def _on_vad_metrics(vad_metrics: vad.VADMetrics) -> None:
            self.emit(
                "metrics_collected", metrics.PipelineVADMetrics(**vad_metrics.__dict__)
            )

        room.on("participant_connected", self._on_participant_connected)
        self._room, self._participant = room, participant

        if participant is not None:
            if isinstance(participant, rtc.RemoteParticipant):
                self._link_participant(participant.identity)
            else:
                self._link_participant(participant)
        else:
            # no participant provided, try to find the first participant in the room
            for participant in self._room.remote_participants.values():
                self._link_participant(participant.identity)
                break

        self._main_atask = asyncio.create_task(self._main_task())