which is the llm behind multilungual model ? plan to update?

Thanks for reaching out, Filippo. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I heard that we have new Qwen 3.5 small models : https://huggingface.co/Qwen/Qwen3.5-0.8B-Base

Thanks — I see you’re asking about the new Qwen 3.5 small model (Qwen3.5-0.8B-Base) and whether it relates to our multilingual turn-detection model or could be used as an update. Quick summary in response to your message: • Our current multilingual turn-detection model is Qwen2.5-0.5B-Instruct (distilled from Qwen2.5-7B-Instruct). • You mentioned Qwen3.5-0.8B-Base — that model exists publicly on Hugging Face. Before I dive into comparisons or recommendations, could you tell me which of these you want to explore? • Integrating Qwen3.5-0.8B into LiveKit’s turn-detection pipeline (i.e., replacing or testing it against Qwen2.5-0.5B-Instruct)? • A performance comparison (accuracy, multilingual coverage) between Qwen2.5-0.5B-Instruct and Qwen3.5-0.8B? • Resource/latency implications for running Qwen3.5-0.8B on CPU vs. the current model? • Or something else (licensing, fine-tuning, or inference tooling)? I can then: • Outline practical steps to evaluate Qwen3.5-0.8B against the current model. • Highlight likely trade-offs (model size, latency, and expected accuracy improvements). • Suggest an experimental setup and metrics to compare turn-detection performance. Which of the options above would be most useful for you?

• Integrating Qwen3.5-0.8B into LiveKit’s turn-detection pipeline (i.e., replacing or testing it against Qwen2.5-0.5B-Instruct)? • A performance comparison (accuracy, multilingual coverage) between Qwen2.5-0.5B-Instruct and Qwen3.5-0.8B? •

Thanks — I’m responding to both of your asks: integrating Qwen3.5-0.8B into LiveKit’s turn-detection pipeline, and a performance comparison vs Qwen2.5-0.5B-Instruct. I can plan a practical integration and a fair evaluation, but I need a few details to give concrete steps, recommended runtimes, and a benchmark plan. Could you please answer the items below? 1. Deployment environment ◦ Will inference run on CPU, GPU, or both? Any target instance types (cores, memory, GPU model)? 1. Inference framework & tooling ◦ Which runtime do you use or prefer (PyTorch, ONNX, GGML, vLLM, Triton, Hugging Face Inference API, etc.)? 1. Model access and licensing ◦ Do you already have the Qwen3.5-0.8B weights and license cleared for use? 1. Current pipeline details ◦ How is the existing Qwen2.5-0.5B-Instruct integrated today (API shape, batching strategy, tokenization, prompt engineering, pre/post processing)? ◦ What inputs are used for turn detection (ASR transcripts only, transcripts + timestamps, raw audio/prosody features)? 1. Evaluation data & metrics ◦ Do you have labeled test data for turn detection (which languages, how many examples)? ◦ Which metrics matter most: accuracy / F1, per-language performance, latency (p95/p99), CPU/memory usage, cost per inference? 1. Acceptance criteria & constraints ◦ Minimum acceptable latency or throughput, and the accuracy improvement you’d need to switch models? 1. Experiment scope and support ◦ Do you want a step-by-step guide you’ll run, or a benchmark run prepared with results you can review? 1. Timeline & priority ◦ Desired timeline for testing and rollout. What I’ll provide once you reply: • A recommended integration plan for Qwen3.5-0.8B tailored to your runtime (including tokenization, prompt format, batching, and runtime settings). • A concrete benchmark plan with datasets, metrics, and scripts (so you can run a fair A/B test against Qwen2.5-0.5B-Instruct). • Expected trade-offs (model size → latency and memory implications) and tuning tips (quantization, batching, prompt optimizations, and possible distillation approaches) based on your chosen runtime. Pick any subset of the questions above to start with if you don’t have everything ready. If you prefer, tell me your top priority (e.g., lowest latency on CPU or best multilingual accuracy) and I’ll tailor the plan around that.