https://livekit.io logo
Join Slack
Powered by
Is it possible to combined the concept of a multi-...
# ask-ai
f
Is it possible to combined the concept of a multi-user transcriber (https://github.com/livekit/agents/blob/main/examples/other/transcription/multi-user-transcriber.py) and the concept of a push-to-talk (manual turn) (https://github.com/livekit/agents/blob/main/examples/voice_agents/push_to_talk.py) -- I would like to have a full transcription of all callers in a room, but then be able to have them ask questions of the agent
t
Thanks for reaching out, Ryan. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Yes, you can combine the multi-user transcriber and push-to-talk examples to achieve your goal. Here's how you can approach it: • Multi-User Transcriber Base: Use the multi-user transcriber example to handle multiple participants. This creates an 
AgentSession
for each participant, allowing individual transcription of their audio streams. • Push-to-Talk Control: Integrate the push-to-talk functionality to control audio input. This involves using RPC methods to enable audio only when a participant triggers a turn. • Session Management: When a participant triggers push-to-talk, their 
AgentSession
is activated, enabling their audio input. Transcriptions from each session are captured and processed by the agent to formulate responses. By combining these functionalities, you can transcribe all callers in a room while allowing them to ask questions of the agent. Let me know if you need specific implementation steps or further guidance!
f
yes - are there any examples of how to do the push to talk with this transcription?
t
To combine push-to-talk functionality with multi-user transcription, here’s a friendly way to approach it: • Push-to-Talk Base: Start with the push-to-talk example, which shows how to enable audio input only when someone hits the 
start_turn
RPC method. • Integrate Transcription: Then, bring in the transcription logic from the multi-user transcriber example. This way, you get real-time transcription for everyone involved. • Session Management: Make sure each participant’s session is handled separately, so transcription only kicks in when their push-to-talk is active. For instance, you can tweak the 
start_turn
method in the push-to-talk example to turn on transcription for whoever triggered it. And don’t forget to use the transcription example’s 
on_user_turn_completed
method to log and process those transcriptions. If you want, I’m happy to share more details or specific code snippets to make this easier for you! Sources: examples/other/transcription/multi-user-transcriber.py | complex-agents/role-playing/role_playing_frontend/hooks/useChatAndTranscrip...
f
how could I put a button in my front end to trigger this?
t
Hey there! To add a push-to-talk button in your frontend, here’s a simple way to do it: 1. Create a Button Component: Build a button that toggles the push-to-talk feature. When you click it, it’ll send an RPC call to either start or stop your turn.
Copy code
import { useState } from "react";
import { useLocalParticipant } from "@livekit/components-react";

export function PushToTalkButton() {
  const { localParticipant } = useLocalParticipant();
  const [isRecording, setIsRecording] = useState(false);

  const toggleRecording = async () => {
    try {
      if (isRecording) {
        await localParticipant?.rpc("end_turn", {});
        setIsRecording(false);
      } else {
        await localParticipant?.rpc("start_turn", {});
        setIsRecording(true);
      }
    } catch (error) {
      console.error("Failed to toggle recording", error);
    }
  };

  return (
    
  );
}
2. Backend RPC Handling: On the backend, you’ll want to register RPC methods that handle starting and ending turns:
Copy code
async def start_turn(data):
    session.input.set_audio_enabled(True)

async def end_turn(data):
    session.input.set_audio_enabled(False)

localParticipant.register_rpc_method("start_turn", start_turn)
localParticipant.register_rpc_method("end_turn", end_turn)
3. Integrate the Button: Finally, pop this button into your main UI component and make sure it talks to your backend just right. If you want me to walk you through any part or need a hand with something else, just let me know—I’m here to help!
2 Views
Open in Slack
PreviousNext
Powered by