https://livekit.io logo
Join Slack
Powered by
How do i publish audio from a backend process afte...
# ask-ai
l
How do i publish audio from a backend process after i create a room for an ai agent?
t
Thanks for reaching out, Piyush. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To publish audio from a backend process after creating a room for an AI agent in LiveKit, you need to: 1. Create an 
AudioSource
with the appropriate sample rate and number of channels. 2. Create a 
LocalAudioTrack
using the 
AudioSource
. 3. Publish the 
LocalAudioTrack
to the room's local participant. 4. Continuously generate and feed audio frames to the 
AudioSource
for publishing. Here is a code example illustrating this process in Python:
Copy code
import numpy as np
from livekit import rtc

# Create an AudioSource
sample_rate = 48000
num_channels = 1
audio_source = rtc.AudioSource(sample_rate, num_channels)

# Create a LocalAudioTrack
track_name = "example-track"
local_audio_track = rtc.LocalAudioTrack.create_audio_track(track_name, audio_source)

# Publish the track
room = rtc.Room()
await room.connect("<ws://localhost:7880>", token)
publication = await room.local_participant.publish_track(local_audio_track, rtc.TrackPublishOptions())

# Generate audio frames
audio_frame = rtc.AudioFrame.create(sample_rate, num_channels, 480)
audio_data = np.frombuffer(audio_frame.data, dtype=np.int16)

while True:
    time = np.arange(480) / sample_rate
    sine_wave = (2 ** 15 - 1) * np.sin(2 * np.pi * 440 * time)
    np.copyto(audio_data, sine_wave.astype(np.int16))
    await audio_source.capture_frame(audio_frame)
This example continuously generates a 440 Hz sine wave and publishes it to the room. Adjust the audio generation logic to suit your AI agent's audio output. Make sure you have the LiveKit SDK properly installed and configured. Sources: Publishing tracks | LiveKit Docs | Audio and video | LiveKit Docs | examples/publish-wav/index.ts
l
is there a js version of the code
t
Absolutely! I'd be happy to help you with publishing audio from a backend process using JavaScript in LiveKit. Let me break down this code example for you in a friendly way.
Copy code
// Set up audio track
const source = new AudioSource(sampleRate, channels);
const track = LocalAudioTrack.createAudioTrack('audio', source);
const options = new TrackPublishOptions();
options.source = TrackSource.SOURCE_MICROPHONE;

await room.localParticipant.publishTrack(track, options);

// Send audio frames
const frame = new AudioFrame(
  buffer.slice(written, written + frameSize),
  sampleRate,
  channels,
  Math.trunc(frameSize / channels),
);
await source.captureFrame(frame);
The magic happens here: we create an 
AudioSource
, turn it into a local audio track, and then send those audio frames to the room. Just swap out 
buffer
, 
written
, and 
frameSize
with your specific audio data, and you're all set! Feel free to ask if you need any more guidance. 😊 Sources: packages/livekit-rtc/README.md
l
[184340.780] DEBUG (lk-rtc/90908): ignoring incoming text stream due to no handler for topic what is this message?
t
Hey there! I noticed you're seeing a debug message about "ignoring incoming text stream due to no handler for topic". No worries, this just means the client received a text stream, but didn't know exactly what to do with it. Here's the quick fix: In LiveKit, you'll want to register a handler for the specific topic before connecting to the room. Think of it like setting up a welcome mat for your text streams. Here's how you can do that:
Copy code
room.registerTextStreamHandler('my-topic', (reader, participantIdentity) => {
  // Process the text stream
});
Basically, if no handler is registered for a topic, the stream gets politely ignored - hence the debug message you're seeing. Pro tip: Make sure the topic you register matches the topic when you're sending text streams. It's like making sure you're speaking the same language:
Copy code
room.localParticipant.sendText('Hello, world!', 'my-topic');
Sources: src/room/Room.ts | packages/livekit-rtc/src/room.ts | livekit-rtc/livekit/rtc/room.py | Sources/LiveKit/Core/Room+DataStream.swift | README.md
l
what should be the topic name?
t
Hey there! Let's talk about choosing the right topic name when setting up a text stream handler in LiveKit. It's super important to pick a unique name because you can only have one handler per topic. While the docs don't have super strict naming rules, think of your topic name like a friendly street sign that tells everyone exactly where to go. You'll want to use names that are clear and descriptive – something like 
chat-messages
, 
ai-agent-input
, or 
transcription
. This helps keep everything organized and easy to understand. One quick heads up: make sure you're not accidentally registering multiple handlers for the same topic, as that can cause some annoying errors. Have a specific idea brewing? I'd be totally happy to brainstorm the perfect topic name with you! Sources: packages/livekit-rtc/src/room.ts | Sending text | LiveKit Docs | Runtime/Scripts/DataStream.cs | livekit-android-sdk/src/main/java/io/livekit/android/room/datastream/incomi...
l
what should be the sampling rate for audio?
sent via microphone
t
Hey there! When it comes to audio in LiveKit, we recommend using a 48 kHz sampling rate (that's 48000 Hz) through your microphone. It's the sweet spot we've found in our examples and configurations – trust me, it'll give you the best sound quality. Sources: terminal-client/client.py | Runtime/Scripts/RtcAudioSource.cs | Publishing tracks | LiveKit Docs | Audio and video | LiveKit Docs | examples/video-stream/audio_wave.py
l
if i have 8k hz audio how do i send it as a room participant
t
Hey there! I totally understand the confusion with audio sample rates in LiveKit. The knowledge base doesn't spell out exactly how to handle 8 kHz audio in a 48 kHz room, but don't worry – I've got your back! Here's the deal: most audio systems really prefer having audio at a consistent sample rate. In your case, you'll want to "upsample" your 8 kHz audio to 48 kHz before sending it into the room. Think of it like translating your audio to a language the system understands better. Would you like me to walk you through how to do this audio resampling? I'm happy to help you get your audio sounding crystal clear! Sources: Receiving and publishing tracks | LiveKit Docs | Receiving and publishing tracks | LiveKit Docs | examples/video-stream/audio_wave.py | Tests/LiveKitTests/AudioMixRecorderTests.swift | Audio and video | LiveKit Docs | Publishing tracks | LiveKit Docs
8 Views
Open in Slack
PreviousNext
Powered by