In the latest Typescript agent framework (v1.0.3) - I can't get it to react to messages typed in the Agent Playground ?

Thanks for reaching out, Steve. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

LiveKit agent-js v1.0.3 (TypeScript), Real-Time Model. In Agent Playground, I type a chat. The agent does not react at all.

Hi there! It looks like the LiveKit agent-js v1.0.3 Real-Time Model might not be responding to typed chat in the Agent Playground due to some configuration settings related to text input. No worries, I’m here to help you walk through some steps to check and fix this: 1. Make sure text input is enabled: Double-check that the

textEnabled

parameter is set to

true

in the

RoomInputOptions

when you initialize the agent session. Here’s an example for you:

const roomInputOptions = {
  textEnabled: true,
  textInputCallback: DEFAULT_TEXT_INPUT_CALLBACK, // This is the default
};

1. Verify the text stream handler: Confirm that the RoomIO class is registering a handler for the

lk.chat

topic. This should happen automatically when

textEnabled

is true and you call

start()

on the RoomIO instance. 2. Check the text input callback: If you’ve customized the default callback, please make sure it calls

sess.interrupt()

and

sess.generateReply()

with the user’s input text. 3. Confirm RealtimeModel modalities: Your RealtimeModel should be set up to handle text input. By default, it supports both text and audio modalities, like this:

const DEFAULT_REALTIME_MODEL_OPTIONS = {
  modalities: ['text', 'audio'],
  // ...
};

1. Debug with logging: Take a look at the agent logs for any warnings about text stream handlers or other potential issues. 2. Verify agent state: Make sure your agent is properly started and connected to the playground. For a smoother testing experience, please ensure your agent is running in

dev

or

start

mode and is correctly connected to the playground. If you’re still facing issues after these checks, just let me know! I’m here to help you troubleshoot further and get everything working perfectly. Sources: agents/src/voice/agent.ts | Agents v0.x migration guide - Node.js | LiveKit Docs | Agents playground | LiveKit Docs

it doesn't appear that the textInputCallback is firing.

If the

textInputCallback

is not firing in LiveKit agent-js v1.0.3, don’t worry! Let’s walk through some helpful steps together to troubleshoot and get things working smoothly: 1. Enable text input: First, please double-check that

textEnabled: true

is explicitly set in your

RoomInputOptions

. Here’s a quick example to guide you:

const roomIO = new RoomIO({
  agentSession,
  room,
  inputOptions: {
    textEnabled: true, // Make sure text input is enabled
  }
});

1. Verify participant identity: It’s important that the participant sending the message matches the

participantIdentity

your agent expects. Messages from other participants might not trigger the callback. 2. Send messages to the correct topic: The frontend should send text messages to the

lk.chat

topic. Here’s how you can do that:

const text = 'Hello how are you today?';
const info = await room.localParticipant.sendText(text, {
  topic: 'lk.chat',
});

1. Check for handler conflicts: Sometimes, if another handler is already registered for the

lk.chat

topic, your registration might not go through, but without clear errors. It’s a good idea to check your logs for any warnings about handler conflicts. 2. Debug the callback: Adding some logging inside your

textInputCallback

can be really helpful to see if it’s being triggered:

textInputCallback: (sess, ev) => {
  console.log("textInputCallback triggered", ev);
  sess.interrupt();
  sess.generateReply({ userInput: ev.text });
}

1. Test manual input: To isolate the issue, try bypassing the text stream and trigger input manually like this:

session.interrupt();
session.generateReply({ userInput: "Test message" });

1. Validate room initialization: Make sure that

RoomIO.start()

is called after the room has connected, as handler registration depends on the room’s connection state. If you’ve tried these steps and the issue is still hanging around, please don’t hesitate to reach out. I’m here to help you dig deeper and find a solution together. Sources: Agents 0.x migration guide | LiveKit Docs | tests/test_agent_session.py | Docs::Agents | Getting Help | livekit/livekit_composite | DeepWiki | agents/src/voice/agent_activity.ts

Text input already enabled. Playground is reacting to speech OK. Added logging to textInputCallback. Triggered session.generateReply() successfully and can see/hear response and logging. Not sure about 'lk.chat' - my code doesn't use this.

Hi there! I’d love to help you get the LiveKit agent-js v1.0.3 framework working smoothly with your text input. This framework is designed to handle messages sent specifically to the

lk.chat

topic. If your code isn’t using this topic, the agent won’t receive those messages. To make sure everything runs perfectly, here are some friendly steps to follow: 1. Send text to the correct topic: Please use the

sendText

method with the

lk.chat

topic. Here’s a simple example:

const text = 'Hello, how are you today?';
const info = await room.localParticipant.sendText(text, {
  topic: 'lk.chat',
});

1. Verify participant identity: It’s important to make sure the participant sending the message is the same one linked to the agent. The agent only processes messages from the participant specified in

participantIdentity

. 2. Check room connection state: Before sending text, please confirm the room is fully connected. The text stream handler is set up during

RoomIO.start()

, which depends on the room connection being ready. 3. Check for topic conflicts: Double-check that no other handlers are registered for the

lk.chat

topic, as this might interfere with the agent’s handler. If you’d like, I’m here to help you with any further troubleshooting or questions you might have. Just let me know, and we can work through it together! Sources: Text and transcriptions | LiveKit Docs | Text and transcriptions | LiveKit Docs | agents/src/voice/room_io/room_io.ts | Agents v0.x migration guide - Node.js | LiveKit Docs | Agents 0.x migration guide | LiveKit Docs

I've not made any change to the Playground code - just the connected agent. It's working with the Playground - except for text messages.

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!

It advises I need to:

In your agent-JS code:
1. Register a text stream handler for the 'lk.chat' topic on your room connection
2. In the handler, read the text from the stream reader.

How is this done