I noticed that when we use React components to show audio transcripts in chat we're getting the STT conversion - is this coming from the room or being streamed by our LiveKit agent? The reason behind the question is that we want to surface tool calls and results for auditing purposes during testing and it'd be helpful to get this all highlighted on screen.

Thanks for reaching out, Arthur. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

The question here is whether the transcription is already passed in or do I need to ensure that we capture such events?

Good question — here's the short answer. The transcription is automatically streamed by the LiveKit agent on the

lk.transcription

text stream, so you do not need to separately capture the raw audio to get the transcripts. You only need to register a text-stream handler on the client (for example,

registerTextStreamHandler("lk.transcription", ...)

) and render the incoming messages; the agent includes metadata like

lk.transcribed_track_id

,

lk.segment_id

, and

lk.transcription_final

for interim/final states. If you want tool calls and results included for auditing, those are not automatically injected into the transcription stream unless you explicitly have the agent publish them (for example via

conversation_item_added

, by extending

llm_node

, or by publishing a custom text stream such as

lk.audit_log

). Do you want a small React snippet showing how to register the handler for

lk.transcription

and a custom

lk.audit_log

?

Can you first show me how to inject these in the transcript stream?

Thanks — I’ll show how to inject tool calls/results into the transcription stream and how to handle them in React. Server/agent side (publish enriched transcription messages) 1. When your agent detects a tool call or result, publish a text stream message on the same

lk.transcription

topic (so the frontend sees it alongside STT), and include a flag or metadata to identify it as an audit entry. Example fields to include: • `type`: "transcription" or "audit" • `text`: visible content • `lk.transcribed_track_id`: (for STT messages) source track id or null for audit entries • `lk.segment_id`: segment id for STT or a generated id for audit entries • `lk.transcription_final`: boolean if applicable • `audit`: object with details (tool name, inputs, outputs, timestamp, request id) 1. Example pseudocode (agent/server):

// when STT result arrives, forward as-is
agent.publishTextStream("lk.transcription", {
  type: "transcription",
  text: sttText,
  lk_transcribed_track_id: trackId,
  lk_segment_id: segmentId,
  lk_transcription_final: isFinal
});

// when a tool call/result occurs, publish an audit entry to the same topic
agent.publishTextStream("lk.transcription", {
  type: "audit",
  text: `Tool ${toolName} returned: ${summary}`,
  lk_transcribed_track_id: null,
  lk_segment_id: `audit_${Date.now()}`,
  lk_transcription_final: true,
  audit: {
    tool: toolName,
    inputs: toolInputs,
    outputs: toolOutputs,
    timestamp: new Date().toISOString(),
    request_id: toolRequestId
  }
});

Notes: • Using the same

lk.transcription

topic ensures transcripts and audit entries are delivered in the same stream and ordering as the agent publishes them. • Mark audit entries distinctly (

type: "audit"

or an

audit

field) so the client can render them differently. • Include timestamps/request IDs to make records auditable. Client side (React rendering) 1. Register a text stream handler for

lk.transcription

and update your UI state with incoming messages. 2. Minimal React example:

import { useEffect, useState } from "react";

function TranscriptViewer({ room }) {
  const [items, setItems] = useState([]);

  useEffect(() => {
    if (!room) return;

    const handler = (msg) => {
      // normalize field names depending on how you publish them
      const item = {
        id: msg.lk_segment_id ?? msg.id ?? String(Date.now()),
        type: msg.type ?? (msg.audit ? "audit" : "transcription"),
        text: msg.text,
        meta: msg.audit ?? {
          lk_transcribed_track_id: msg.lk_transcribed_track_id,
          lk_transcription_final: msg.lk_transcription_final
        },
        ts: msg.audit?.timestamp ?? new Date().toISOString()
      };
      setItems((prev) => [...prev, item]);
    };

    // registerTextStreamHandler depends on your SDK binding
    room.registerTextStreamHandler("lk.transcription", handler);

    return () => {
      room.unregisterTextStreamHandler("lk.transcription", handler);
    };
  }, [room]);

  return (
    <div>
      {items.map((it) => (
        <div key={it.id} style={{ padding: 6, margin: 4, borderRadius: 4, background: it.type === "audit" ? "#f3f4f6" : "#fff" }}>
          <div style={{ fontSize: 12, color: "#666" }}>{it.type.toUpperCase()} • {new Date(it.ts).toLocaleTimeString()}</div>
          <div>{it.text}</div>
          {it.type === "audit" && (
            <pre style={{ fontSize: 12, color: "#333" }}>{JSON.stringify(it.meta, null, 2)}</pre>
          )}
        </div>
      ))}
    </div>
  );
}

Recommendations and considerations • Maintain ordering: publish audit entries as soon as the agent has tool results, ideally with timestamps referencing the related transcription segment. • Privacy/security: ensure any tool outputs you publish are appropriate to show in the client and comply with your auditing/security policies. • If you prefer separation, you can publish to a custom topic like

lk.audit_log

instead and register a second handler; using the same topic keeps everything together for simpler UI rendering. Would you like a concrete server-side code sample for the agent framework you’re using (Node, Python, or another)?

Within a tool call itself - how do I access the transcription stream? Is it in RunContext that I pass into the tool call?

You're right to check — the live transcription stream is not directly passed into the

RunContext

. Instead: • Inside a tool call you can access transcribed text via

context.session

(the AgentSession). The session holds the conversation/chat context (

session.chat_ctx

) which contains accumulated transcribed user messages and agent responses. • Example (Python-style):

@function_tool()
async def my_tool(self, context: RunContext):
    # Get last chat message (transcribed user input)
    last_message = context.session.chat_ctx.messages[-1]
    transcript_text = last_message.content
    return f"Last user said: {transcript_text}"

• If you need real-time access to transcription events (interim/final) before or during tool execution, subscribe to session events instead of relying on

RunContext

. For example:

@session.on("user_input_transcribed")
async def on_transcript(event):
    if event.is_final:
        # process or store transcription immediately
        print("Transcribed:", event.transcript)

Recommendation: Use

context.session.chat_ctx

for tool logic that needs the latest transcribed text; use the

user_input_transcribed

session event if you need event-level or streaming/interim data for auditing or to trigger tool calls. Would you like a concrete example in Node or the exact agent framework you're using? Sources: Agents and handoffs | LiveKit docs | Tool definition and use | LiveKit docs | TTS Translator with Gladia STT | LiveKit docs | Transcriber | LiveKit docs

All we need is the result of the tool call logged (or whether it failed) into transcription for rendering / auditing by a front end use as they build workflows. Can you generate me an example of this in my Python Agents Tool call tools? We use React components on the front end

Thanks — I’ll show a concise, ready-to-use example that logs tool call results (success or failure) into the transcription stream from a Python agent, and how the React frontend can render those entries alongside STT. Python agent tool example • Use

context.session.say()

inside the tool to publish a summary to the

lk.transcription

stream. That message will be delivered to clients as part of the transcription flow.

from livekit.agents import Agent, RunContext, function_tool

class WorkflowAgent(Agent):
    @function_tool()
    async def call_tool_and_log(self, context: RunContext, query: str):
        """
        Example tool that calls an external service, then logs success/failure
        into the transcription stream via session.say().
        """
        try:
            # --- perform the tool action (replace with real call) ---
            # e.g., result = await external_api.lookup(query)
            result = {"status": "ok", "summary": f"Result for {query}"}

            # Build human-friendly audit text and metadata
            audit_text = f"Tool succeeded: {result['summary']}"
            audit_meta = {
                "tool": "example_lookup",
                "status": "success",
                "inputs": {"query": query},
                "outputs": result,
            }

            # Publish to transcription stream (spoken and text)
            # session.say publishes text that appears on lk.transcription
            await context.session.say(audit_text, metadata=audit_meta)

            return result

        except Exception as e:
            # On failure, publish a failure message for auditing
            error_text = f"Tool failed for {query}: {str(e)}"
            audit_meta = {
                "tool": "example_lookup",
                "status": "failure",
                "inputs": {"query": query},
                "error": str(e),
            }

            await context.session.say(error_text, metadata=audit_meta)
            # Re-raise or return error indicator depending on your flow
            return {"status": "error", "error": str(e)}

Notes: •

session.say()

publishes text that the agent will send to

lk.transcription

and (if TTS enabled) speak aloud. Include a

metadata

object so the frontend can render structured audit details. • You can include tool name, inputs, outputs, request IDs and timestamps in metadata for auditing. React frontend (consume transcription + audit metadata) • Register the transcription handler or use the SDK’s transcription hook to receive incoming transcription messages. Render audit entries differently when metadata indicates a tool log. Minimal React example using a generic transcription hook/handler:

import React, { useEffect, useState } from "react";

function TranscriptViewer({ room }) {
  const [items, setItems] = useState([]);

  useEffect(() => {
    if (!room) return;

    const handler = (msg) => {
      // msg fields depend on how session.say() populates text stream.
      // Normalize fields: msg.text (or msg.content), msg.metadata
      const text = msg.text ?? msg.content ?? "";
      const meta = msg.metadata ?? msg.meta ?? null;
      const id = msg.id ?? `${Date.now()}_${Math.random()}`;

      const item = {
        id,
        text,
        meta,
        ts: msg.timestamp ?? Date.now(),
      };
      setItems((prev) => [...prev, item]);
    };

    room.registerTextStreamHandler("lk.transcription", handler);
    return () => room.unregisterTextStreamHandler("lk.transcription", handler);
  }, [room]);

  return (
    <div>
      {items.map((it) => (
        <div key={it.id} style={{ marginBottom: 8, padding: 8, borderRadius: 6, background: it.meta ? "#f7fafc" : "#fff" }}>
          <div style={{ fontSize: 12, color: "#666" }}>
            {it.meta ? `${it.meta.tool} • ${it.meta.status}` : "transcript"} • {new Date(it.ts).toLocaleTimeString()}
          </div>
          <div>{it.text}</div>
          {it.meta && (
            <pre style={{ fontSize: 12, marginTop: 6 }}>{JSON.stringify(it.meta, null, 2)}</pre>
          )}
        </div>
      ))}
    </div>
  );
}

export default TranscriptViewer;

Checklist and recommendations • Ensure

session.say()

accepts a `metadata`/`metadata`-like parameter in your agent SDK version; if not, include metadata fields in the text payload or publish a parallel structured text stream entry to

lk.transcription

. • Mark audit entries clearly (metadata.tool / metadata.status) so the client can render them distinct from STT. • Include timestamps and request IDs to tie tool logs back to transcription segments for audit trails. • If you prefer to keep STT and audit logs strictly separated, publish audit entries to a separate text stream (e.g.,

lk.audit_log

) and register a second handler in the React app. Would you like this example adapted to your exact Python agent SDK version or to include request IDs and timestamp formatting? Sources: Text and transcriptions | LiveKit docs

I don't want the tool call to be read out aloud as we have TTS enabled - can it just be logged silently or directly to transcription?

Yes — you can log tool results silently (no TTS) by publishing text directly to the transcription topic instead of using

session.say()

. Python agent (silent logging) — use

context.room.local_participant.send_text(...)

to publish to `lk.transcription`:

@function_tool()
async def call_tool_and_log(self, context: RunContext, query: str):
    try:
        result = await do_tool_call(query)  # your tool call
        text = f"[Tool example_lookup succeeded] {result}"
        await context.room.local_participant.send_text(
            text,
            topic="lk.transcription",
            attributes={
                "lk_transcribed_track_id": "",       # no audio source
                "lk_transcription_final": "true",
                "tool": "example_lookup",
                "status": "success",
                "request_id": str(uuid4()),
            },
        )
        return result
    except Exception as e:
        err = f"[Tool example_lookup failed] {str(e)}"
        await context.room.local_participant.send_text(
            err,
            topic="lk.transcription",
            attributes={
                "lk_transcribed_track_id": "",
                "lk_transcription_final": "true",
                "tool": "example_lookup",
                "status": "failure",
                "request_id": str(uuid4()),
            },
        )
        return {"error": str(e)}

Notes: • Using

local_participant.send_text(..., topic="lk.transcription")

writes into the transcription stream but does not generate audio/TTS. • Include structured attributes (tool name, status, request_id, timestamps) so the frontend can treat these as audit entries and correlate them with transcribed speech. React frontend (consume and render) • Register a text stream handler for

lk.transcription

(or use

useTranscriptions

) and inspect incoming message attributes to detect tool logs vs STT. Render tool entries with different styling:

room.registerTextStreamHandler("lk.transcription", (msg) => {
  const text = msg.text ?? msg.content ?? "";
  const attrs = msg.attributes ?? {};
  const isTool = !!attrs.tool;
  addTranscript({
    id: msg.id ?? `${Date.now()}`,
    text,
    meta: attrs,
    type: isTool ? "audit" : "transcript",
  });
});

Recommendations • Add

request_id

and timestamps to attributes to correlate tool logs with segments. • If your SDK version does not expose

attributes

the same way, embed a JSON payload in the text or publish to a separate topic like

lk.audit_log

and register a second handler. • Use empty

lk_transcribed_track_id

to indicate no audio source so clients can render silently. Would you like this adapted to your exact Python agent SDK version or an example that includes timestamp formatting and request-id generation? Sources: Sending text | LiveKit docs | Text and transcriptions | LiveKit docs

I assume that anything I push to the transcript will be shown in audit. If I want to do it on event / in memory without exposing / saving data I need to do this custom on my servers side?

You're correct to confirm this. • Anything you publish to

lk.transcription

(or any public text stream) will be visible to clients subscribed to that stream and therefore available for auditing/display in the frontend. • If you need tool results handled only in memory or processed event-wise without exposing/saving them to clients, do not publish them to the transcription stream. Instead: ◦ Handle them in your agent/server memory (e.g., store in

context.session

temporary state or an in‑memory cache) and emit any audit-only records to internal logs or a secure backend endpoint. ◦ Trigger server-side events or callbacks (webhooks, internal message bus) that perform whatever processing you need without forwarding the data to

lk.transcription

. ◦ If you need limited, controlled visibility to specific clients, publish to a private/custom topic that only intended clients subscribe to (or encrypt/encode payloads and enforce access controls on your server). If you want, I can: • Provide a Python example showing an in‑memory handling pattern and separate server endpoint for secure logging, or • Show how to publish to a private topic and restrict client subscription in your React app. Which would you prefer?