I'm building a Unity app with voice chat using LiveKit. If multiple users are in the same room, how can I filter out the voices of nearby users?

Thanks for reaching out, Mat. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

We do not know how far apart users are in the real world.

Thanks — you're saying you don't have real-world distance data. I can help — a couple quick questions so I give the right options: 1. Do your users have avatars or positions in the Unity scene (virtual positions), or is this a pure voice app with no in-app location model? 2. Can clients share small state updates (e.g., via LiveKit data tracks or a game server) about their virtual/room position or status? If you do have virtual positions, the simplest solution is to share each participant’s Unity position over a LiveKit data track (or your game server), compute distances locally, and subscribe/unsubscribe or attenuate volume as I described earlier. If you have no position model and cannot share any positional state, you can’t filter by proximity. Alternatives: • Provide a user-controlled mute/volume UI per participant. • Use voice activity detection to prioritize closest speakers (e.g., auto-lower background participants), or implement a push-to-talk model. • Let users select “focus” participants whose audio is boosted while others are attenuated. Tell me which of the above fits your app (virtual positions available, can share positions, or neither), and whether you prefer strict unsubscribe or volume attenuation — I’ll provide concrete Unity code for that path. , with interactive elements

The problem I'm trying to solve is that, for example if a user named Adam is next to a user named Betty, Betty's microphone can pick up Adam's voice.

Thanks — I understand: Adam’s mic is physically picking up Betty’s voice (real-world bleed), and you want to reduce or remove that in your Unity + LiveKit app. Two important facts: • LiveKit cannot separate two different real-world sound sources captured by the same microphone. • You can only control what audio your client receives (subscribe/unsubscribe or adjust volume) — you cannot remove bleed that’s already mixed into a participant’s mic signal on the sender side. Given that, choose one of these practical approaches and tell me which you prefer so I can provide exact implementation steps: 1. Prevent bleed at the source (recommended) ◦ Use headset/microphone hardware or noise-cancelling mics, or position users further apart. ◦ Use sender-side noise suppression / beamforming (OS or audio driver features, or Unity plugins) so Betty’s client reduces pickup of Adam before sending audio. 1. Reduce impact server/client-side ◦ Encourage users to use push-to-talk or mute when not speaking. ◦ Implement voice activity detection (VAD) and auto-mute/attenuate other participants when one is speaking. 1. If you can obtain positions (virtual or real) ◦ Share position/state via LiveKit data tracks or a game server, then each client computes proximity and unsubscribes or attenuates remote tracks for close neighbors. Tell me: • Do you control the sender device environment (can you require headsets or enable noise suppression)? • Can you add VAD or push-to-talk, or share position/state between clients? Say which approach you want and I’ll give concrete Unity code/config and next steps. , with interactive elements

I'd like to implement noise suppression but don't know how. Please explain.

Thanks — I’ll help you add noise suppression. A couple quick questions so I give the right steps: 1. Which Unity build target are you using: WebGL (browser) or a native build (Windows/macOS/iOS/Android)? 2. Can you change sender devices/settings (require headsets or enable OS-level noise suppression) or do you need an in-app solution? Quick summary of options (pick one and I’ll provide code/config): • Browser/WebGL: rely on browser WebRTC audio processing (noise suppression/echo cancellation) — I’ll show how to ensure the browser capture uses it and how to access it via WebGL interop. • Native Unity: use a native audio-processing plugin or platform APIs for noise suppression (third-party SDKs or OS features). I can recommend plugins and show how to integrate them with LiveKit’s audio capture. • Server-side / agent processing: route audio through LiveKit Cloud agent features (Krisp or LiveKit noise cancellation) so noise is removed before distribution — I can show how to enable this if you control the backend or use LiveKit Agents. Tell me your build target and which approach you prefer and I’ll give concrete steps and code snippets. Sources: https://livekit.com/pricing.md| | Pricing | LiveKit | https://docs.livekit.io/agents/start/telephony.md| | Agents telephony integration | LiveKit Documentation | Turns overview | LiveKit Documentation, with interactive elements

This is for a native Android build.

Thanks — since this is a native Android Unity build, two practical paths can add noise suppression. Quick question before I give exact steps: Do you currently use Unity’s built-in microphone capture (UnityEngine.Microphone) and feed audio directly into LiveKit, or do you have a custom/native audio capture pipeline (e.g., a native Android plugin or Unity WebRTC package)? Answer that and I’ll give the precise implementation steps for either: • Using Android platform-level processing (

android.media.audiofx.NoiseSuppressor

) or • Integrating a third‑party SDK (e.g., RNNoise/Krisp-like plugins) or • Using Unity WebRTC's audio processing if you already use that., with interactive elements

I use UnityEngine.Microphone.

Thanks — I’ll show a practical path to add noise suppression on Android when you capture with

UnityEngine.Microphone

. Short summary:

UnityEngine.Microphone

does not expose Android’s platform audio effects. To use Android’s

NoiseSuppressor

you need to capture audio via a native Android

AudioRecord

that attaches a

NoiseSuppressor

, read PCM frames, then feed those frames into LiveKit as a local audio track. Below are the high-level steps and a minimal example flow with guidance on permissions, sample rates, and integration points. Steps overview 1. Add RECORD_AUDIO permission to AndroidManifest and request it at runtime. 2. Implement an Android Java/Kotlin plugin that: ◦ Creates an

AudioRecord

with your desired sample rate/channel config. ◦ Attaches

NoiseSuppressor.create(audioSessionId)

to the

AudioRecord

session. ◦ Reads PCM buffers in a background thread and exposes them via a JNI call or a Unity callback (e.g., using UnityPlayer.UnitySendMessage) or a shared native buffer. 1. In Unity C#: ◦ Call the plugin to start/stop capture. ◦ Receive PCM frames from the plugin, convert to the format LiveKit expects, and send them into LiveKit as a local audio source (e.g., via a RawAudioSource/local audio track API). 1. Ensure sample rate and channel layout match LiveKit expectations (typically 48 kHz mono/stereo) and perform resampling if necessary. 2. Test and tune buffer sizes and thread scheduling for low latency. Key implementation notes • Android API: use

android.media.AudioRecord

+

android.media.audiofx.NoiseSuppressor

(requires API level where NoiseSuppressor is available; check device compatibility). • Permissions: add

<uses-permission android:name="android.permission.RECORD_AUDIO"/>

and request at runtime on Android 6.0+. • Audio session: create

AudioRecord

, call

int sessionId = audioRecord.getAudioSessionId()

, then

NoiseSuppressor.create(sessionId)

. • Buffering: pick a buffer size using

AudioRecord.getMinBufferSize()

and read repeatedly on a dedicated thread. • Threading: deliver PCM to Unity on a thread-safe pathway (avoid blocking Unity main thread). • Format: Android

AudioRecord

usually gives 16-bit PCM; LiveKit SDKs accept PCM float/int16 depending on integration — convert accordingly. Minimal Java plugin pseudocode (concept)

// Create AudioRecord and NoiseSuppressor
int sampleRate = 48000;
int channelConfig = AudioFormat.CHANNEL_IN_MONO;
int audioFormat = AudioFormat.ENCODING_PCM_16BIT;
int minBuf = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
AudioRecord recorder = new AudioRecord(MediaRecorder.AudioSource.VOICE_COMMUNICATION,
                                       sampleRate, channelConfig, audioFormat, minBuf);
int sessionId = recorder.getAudioSessionId();
NoiseSuppressor ns = NoiseSuppressor.create(sessionId); // enable NS

recorder.startRecording();
byte[] buffer = new byte[minBuf];
while (running) {
    int read = recorder.read(buffer, 0, buffer.length);
    if (read > 0) {
        // send buffer to Unity via JNI/UnitySendMessage or write to native circular buffer
    }
}
recorder.stop();
ns.release();
recorder.release();

Unity-side options to feed LiveKit • If LiveKit Unity SDK accepts a custom audio source/track API (raw PCM callback), push the processed PCM into that API to create the LocalAudioTrack. • If the SDK doesn’t provide a raw-pcm hook, create a bridge that feeds processed audio into an AudioClip using `AudioClip.SetData`/`OnAudioRead` and capture that clip as the microphone source for LiveKit (note: extra latency). • Ensure you convert 16-bit PCM to float [-1..1] if the Unity/LiveKit API expects floats. Alternative (simpler) options • Use platform-level voice-optimized audio source: when creating

AudioRecord

, use

MediaRecorder.AudioSource.VOICE_COMMUNICATION

or

VOICE_RECOGNITION

which enables some OS processing automatically. This may reduce bleed without custom NoiseSuppressor handling. • Use a third-party noise-suppression native library (RNNoise, WebRTC AEC/NS modules, or commercial SDKs) integrated similarly: run native processing on the AudioRecord frames, then forward processed frames into Unity/LiveKit. • If implementing full native pipeline is heavy, require headsets or recommend OS/driver noise cancellation as a short-term mitigation. Would you like: • A complete Unity + Java plugin example with code files (Java plugin, JNI shims, Unity C# receiver) to copy into your project, or • A smaller example showing only the Android plugin and how to test it first (so you can verify suppression), or • Recommendations for third-party libraries and integration notes? Tell me which and I’ll provide the exact code/files for that path. , with interactive elements

What version of Android introduced NoiseSuppressor?

The

android.media.audiofx.NoiseSuppressor

class was added in Android API level 16 (Android 4.1, Jelly Bean). Android docs: https://developer.android.com/reference/android/media/audiofx/NoiseSuppressor , with interactive elements