is qwen3-tts open source model suitable for realti...
# ask-ai
p
is qwen3-tts open source model suitable for realtime voice agents?? or is it like a audio generation mode?? also can you suggest any open source TTS models that can be used for livekit agents which has less latency between responses?? ive tried XTTS_v2, chatterbox and kani TTS tell me which is best
✅ 1
t
Thanks for reaching out, Aaron. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’ll answer the specific parts you asked about and then ask a couple quick clarifying questions so I can recommend the best fit for your LiveKit agents. Short answers first • Qwen3‑tts: I don’t have internal details about that specific model here. Tell me whether you’re referring to a version labeled “open source” or a vendor release, and I’ll tailor the advice. • Real‑time suitability vs audio generation: Many TTS models are designed primarily for high‑quality audio generation (batch/streamed). Whether a TTS is suitable for real‑time voice agents depends on its streaming support, model size/compute needs, and available inference optimizations. • XTTS_v2, Chatterbox, Kani TTS: I don’t have direct performance data here to declare a single “best.” Which is best will depend on your latency targets, available CPU/GPU, and whether you can run a streaming pipeline. What matters for low‑latency LiveKit voice agents • Streaming support: Prefer models that support incremental/streamed synthesis so audio can start playing before the whole utterance finishes. • Model footprint and compute: Smaller models or models optimized with quantization/ONNX/TensorRT will have lower latency on limited hardware. • Warm/long‑lived processes: Keep the TTS server warm (avoid cold starts) and reuse GPU memory/sessions. • Audio chunk size / buffer: Use small output chunks to send audio to LiveKit as soon as possible; tune buffering to balance CPU overhead vs latency. • Sampling rate and codec: 16 kHz/mono often reduces bandwidth and encoding latency; use a fast encoder (Opus) before sending to LiveKit. Practical guidance and next steps I can help with 1. Tell me: ◦ Do you need sub‑500ms response time, or is 1–2s acceptable? ◦ What hardware will you run inference on (CPU only, NVIDIA GPU, edge device)? ◦ Which languages and voice quality level do you require? 1. If you want an immediate checklist to reduce latency: ◦ Choose a model with streaming output or implement chunked synthesis. ◦ Quantize or use an optimized runtime (ONNX/TensorRT/Triton). ◦ Keep TTS server always warm and reuse sessions. ◦ Send audio frames to LiveKit as soon as you have the first audio chunk (don’t wait for full output). ◦ Use Opus encoding and appropriate frame size to reduce network latency. 1. If you want a recommendation process (I can do this with your answers): ◦ I’ll rank XTTS_v2, Chatterbox, and Kani TTS against your latency target, hardware, and language needs and suggest a choice plus tuning steps. If you give me the three quick details from step 1 (latency target, hardware, language/quality), I’ll compare XTTS_v2, Chatterbox, and Kani TTS specifically for your LiveKit agent use case and recommend the best option with concrete tuning tips. , with interactive elements