This message was deleted.

👋 cc @polite-kilobyte-67570 who i think has played with this before

Figured this out. Sad that this Slack only has a 90 day search history, but for anyone interested I found how Decentraland was doing this here, using a loopback stream: https://github.com/decentraland/kernel/blob/72c45ac45a8f8bc0e9650538208fc0bf2e584219/packages/voice-chat-codec/liveKitVoiceHandler.ts

would love to have the search history, but with almost 1400 folks in here, it’s pretty $$$$

Yeah I love slack but think this is why most people use Discord now

As a product, i prefer Slack, but Discord is definitely more community friendly

hey @early-piano-21116 I think you might be interested in a new experimental option that we’ve been testing. you can set

expWebaudioMix: true

in the room options in the latest js client version, which will pipe all audio through web audio. You can then add audio nodes on a track basis by calling

remoteAudioTrack.setWebAudioPlugins([myPannerNode])

. These options are still marked as experimental and aren’t battle tested yet. would be great to hear your feedback whether this makes your life easier already. In terms of spatial support, we aim to make this even easier for people in the future, so anything you are willing to share about your specific usecase would help us make decisions also with your usecase in mind 🙂

Interesting. So currently you don't use Web Audio? I'm not 100% sure why what i've got works but I had to initialize our listener with this:

async init() {
  // code inspired by decentraland kernal repo
  // they also use livekit and to get proper positional audio it seems a loopback
  // stream is required, at least in chrome
  const listener = this.engine.audio.listener
  this.destination = listener.context.createMediaStreamDestination()
  this.destinationStream = window.navigator.userAgent.includes('Chrome')
    ? await startLoopback(this.destination.stream)
    : this.destination.stream
  this.audio = new Audio()
  this.audio.srcObject = this.destinationStream
  await this.driver.gesture
  this.audio.play()
}

After doing that then the following way of connecting tracks works:

const stream = track.mediaStream
this.source = listener.context.createMediaStreamSource(stream)
this.source.connect(this.isSpatial ? this.panner : this.gain)

by default livekit uses standard html audio elements for audio output. with the option I mentioned you can change that and don’t have to worry about the loopback stuff.

sounds perfect, but since its not the docs, what are the chances this stays in the API and remains robust?

that’s a fair concern. the idea is definitely to keep this as part of the API, we marked it experimental for now just in case the feedback from users who try it out would lead us to reconsider how that API is designed. For the sake of robustness it helps of course to have as many people as possible try it out early on 😉 But I’m pretty sure the general functionality will become part of the client-sdk in the short to midterm future. If you have it working now with the snippet you posted that’s great! Just in case you still want to try it out via our integrated path, an advantage might be that it would keep you from manually having to work around potential future browser quirks, as we’ll handle them within the package as they come up!

Great, thanks! I'll switch over to this and report back soon

that looks about right! for disconnect it shouldn’t be required for you to call

disconnectWebAudio

yourself, this is handled within the

detach

function already. the same is true for

attach

. but you’re raising a good point about not being able to set the audio context from outside via public APIs. Two things that I’m thinking about: 1. would a custom audioContext be set once via room options (e.g.

expWebAudioMix: { audioContext: myCustomContext}

or is there the need to update the context programatically on demand via something like

room.setAudioContext

2. could there be cases where individual tracks would need a different audioContext to the “main” one? If so, we could make that API public

For me personally a single Room({ audioContext }) option would work really well.

Hi there: Here are some examples of how to integrate spatial audio voice chat / positional voice chat with SFU. No LiveKit example included yet, but should help to integrate with LiveKit quite similar. https://docs.atmoky.com/examples/voicechat/overview