https://livekit.io logo
Join Slack
Powered by
This message was deleted.
# helpdesk
s
This message was deleted.
b
The way I see it there is a bug in 
Room.connect()
or much rather 
Room.acquireAudioContext()
There 
const ctx = getNewAudioContext();
should be wrapped in a 
try/catch
and the 
audioEnabled
flag should be set to 
false
and trigger the 
RoomEvent.AudioPlaybackStatusChanged
event. It is a bit strange though that still the video playback doesn’t start and the
Copy code
TrackEvent.AudioPlaybackFailed
Isn’t triggered though when trying to play back the video
Okay, so I debugged it and the issue isn’t that an exception is thrown, the warning is printed without an exception but the returned context is in 
suspended
status. --> This means that 
Room.acquireAudioContext()
should likely both have a 
try/catch
AND check for the 
status
to be running. I’ll debug the 
play()
of the actual video now to see why we don’t get a promise rejection there.
p
Hey David, thanks for looking into this! That is indeed inconsistent behaviour. 1. From the first look I would say a good fix for the audio part would be to check the audioContext status in 
acquireAudioContext
and emit the 
AudioPlaybackStatusChanged
event if it failed to start (and was 
true
before). 2. Regarding the video playback, that’s even stranger. video elements with 
muted
attribute set, should always be allowed to autoplay. Are you using the client-sdk 
attach
call ? Are you seeing this behaviour across different browsers?
b
Agreed with 1.
Are you using the client-sdk 
attach
call ?
Yes
Are you seeing this behaviour across different browsers?
I’m not having it on my OSX Chrome but (not even the warning message) but I have it on a Windows PC Chrome and on the OSX of 2 colleagues on Chrome I’ll debug it on the Windows PC and report back
🙏 1
I’m attaching both to the same 
video
element and I think I found the problem. You only ever explicitly call the 
play()
method when you are either attaching it to an 
audio
element or if you are using a 
Firefox
or 
Safari
browser (
Track.attach()
and 
Track.attachToElement()
) As far as I can see for a normal Chrome you’d simply set 
autoplay
to true but I can’t see you calling 
muted=true
anywhere. I have had the same issue in the past when we where managing this ourselves.
The way we solved this was like this: 1. Always set 
autoplay=false
2. set the 
muted
flag to 
true
if we don’t have any audio track, otherwise keep it false 3. always call 
play()
4. if 
play()
fails, do 
muted=true
and call 
play()
again 5. Display a button that displays a speaker symbol, on click change 
muted=false
p
chrome does keep an 
media interaction index
for each url, might be related: https://developer.chrome.com/blog/autoplay/ I think if the audio playback requires interaction we would get the autoplay for video for free.
b
yes but autoplay doesn’t start anything if you have 
muted=false
and you’re not allowed to play audio
p
aha! yeah, we would generally recommend to use separate media elements for audio and video tracks! any particular reason why you want to attach both to the same element?
b
In that case it will neither start audio nor video
which is why you it’s better to put 
autoplay=false
and call it yourself so you know it failed
aha! yeah, we would generally recommend to use separate media elements for audio and video tracks! any particular reason why you want to attach both to the same element?
Just because it’s easier to manage on our end that way, and I thought, why manage a separate audio element if I can do both in one… 🤷
I do agree keeping them separate makes it easier to manage off-screen users etc so I might change it to be that way once we start doing so
p
Makes sense, we did encounter issues before of audio not starting in those cases if the video data didn’t come through though. E.g. a video track is being attached to the element first, but doesn’t have any data ready, so playing is blocked. Then an audio track is attached which would already have data to play, but the playback of the video element is still blocked due to the video track lacking data.
b
Okay that’s interesting
p
The error wasn’t consistent, but there was definitely something going on with the combination of different tracks in different readystates attached to the same element. I’ll raise an issue on the GH repo about the missing audio playback event, which will be an improvement anyways and might also help with attaching both tracks to the same element.
thanks for your help debugging this 🙏
b
Okay, so I changed my implementation to use a separate 
audio
element and attach the mic track to it instead of the 
video
element. Now here’s the interesting thing: I do get the expected 
RoomEvent.AudioPlaybackStatusChanged
now and ``room.canPlayAudio=false`` BUT my video’s still don’t play
(they do play if I refresh while my camera is published so it’s not a general issue)
p
I don’t have a repro for this at hand, if you call 
videoEl.play
in the 
TrackSubscribed
callback, does that fix things?
b
If I do it via the Chrome DevTools (directly selecting the video element on the screen and then calling 
$0.play()
it solves the issue, yes.
I think it’s the problem I described above as soon as the video element doesn’t have the 
muted=true
autoplay will not start if you haven’t had a user interaction on the page
This is why we changed it to the steps I posted above
Yes this fixes it. If I create my own 
video
element, set it to 
muted=true
and then pass it into the 
attach()
method it works
p
huh, our 
attach
method also sets the muted attribute on the element 🤔
b
to be fair, I have always created the video element myself so that might be the case
huh, our 
attach
method also sets the muted attribute on the element
Looking at your 
Track
class I can’t find where you are doing the 
muted=true
though all I see is this:
The only place I find is in 
RemoteAudioTrack.attach
but in this case we’re talking about the 
RemoteVideoTrack.attach
p
You are right, I remembered incorrectly. The place where we do set some of these attributes is the 
attachToElement
function, but 
muted
is currently not part of it. It does get trickier to think about it if users would want to use the same element for both video and audio. In this case setting 
muted
would be undesired.
b
you’d have to emit your event in step 4
p
thanks, does the combination of 
muted=true
and 
autoplay=true
not work for you? do you have to call play manually?
b
sorry I mixed up step 2 I just updated it
thanks, does the combination of 
muted=true
and 
autoplay=true
not work for you?
Yes it does but if you want to be able to detect if playback worked or not you need to explicitly call 
play()
as long as you always separate audio and video into 2 elements simply muting the video element is fine
but if you want to be able to mix them then you first have to do 
muted=false
try play and if it doesn’t work 
mute=true
and play again
p
in case you have audio and video tracks combined on the same element, your step 4. would then only play the video, I assume?
I mean, you wouldn’t get audio playback that way, as the element is muted
b
in case you have audio and video tracks combined on the same element, your step 4. would then only play the video, I assume?
Yes and then it would emit the 
RoomEvent.AudioPlaybackStatusChanged
event
then the integration is responsible for getting the needed user interaction and calling the 
room.startAudio()
method
p
got it, to confirm, would https://github.com/livekit/client-sdk-js/pull/534/files address all your concerns?
b
I added 2 comments to it
👌 1
4 Views
Open in Slack
PreviousNext
Powered by