Hi guys, Have been scratching my head all day but can't seem to get this working.
I am getting TTS from eleven labs and I am trying to publish it from the server to the the room as the server participant using rtc-node. It pushes some audio with a lot of noise then throws below error.
Error: an RtcError occured: InvalidState - failed to capture frame
at AudioSource.captureFrame (/Users/suroorwijdan/Projects/livekit-interview-poc/node_modules/
@swift-school-78240/rtc-node/src/audio_source.ts
14213)
My current code:
// Convert text to speech
const audioResult = await this.elevenLabsService.textToSpeech(
text,
sessionConfig.textToSpeechConfig
);
let leftoverBuffer = Buffer.alloc(0);
for await (const audioChunk of audioResult.audioStream) {
if (audioChunk && audioChunk.length > 0) {
// 1. Combine the leftover from the previous chunk with the new chunk.
const combinedBuffer = Buffer.concat([leftoverBuffer, audioChunk]);
// 2. Calculate how many full 16-bit (2-byte) samples we can process now.
const processableLength = combinedBuffer.length - (combinedBuffer.length % 2);
if (processableLength === 0) {
// Not enough data to form a single sample, so just store it and wait for the next chunk.
leftoverBuffer = combinedBuffer;
continue;
}
// 3. Create a slice of the buffer that contains only the complete samples.
const chunkToProcess = combinedBuffer.subarray(0, processableLength);
// 4. The remainder (0 or 1 byte) is our new leftoverBuffer for the next iteration.
leftoverBuffer = combinedBuffer.subarray(processableLength);
// 5. Safely create the Int16Array from the processed chunk's buffer.
// This is now guaranteed to be safe because chunkToProcess has an even number of bytes.
const int16AudioData = new Int16Array(chunkToProcess.buffer);
// --- Your LiveKit publishing logic remains the same ---
if (int16AudioData.length > 0) {
const audioFrame = new AudioFrame(
int16AudioData,
sessionConfig.audioConfig.sampleRate,
sessionConfig.audioConfig.channels,
Math.trunc(int16AudioData.byteLength / 2)
);
const success = this.liveKitService.captureAudioFrame(
sessionConfig.alexIdentity,
sessionConfig.roomName,
this.alexAudioTrackName,
audioFrame
);
if (!success) {
this.logger.warn(
Failed to publish audio frame for ${sessionConfig.alexIdentity}
);
break;
}
}
}
}