https://livekit.io logo
Join Slack
Powered by
This message was deleted.
# helpdesk
s
This message was deleted.
d
which format would you be consuming?
if you could share your use case, that'd help.
a
PCM would be awesome. I have an ML model that runs on-device that measures features of voice such as personality and mood in real time. Want to display it to the user
d
I see, I think connecting to the room with the client would be the most straight forward.. though there isn't a simple way to get the decoded audio data back today...
Egress can export PCM for each track, but it works by sending binary frames to websocket server.
a
Hmm. Does it help if I only need the audio from the speaker person talking on the device? Don't need it from anyone else in the room
d
I'm not sure how that changes anything..
a
Doesn't have to go through the livekit code at all. Audio could (in theory) be analyzed before it hits your stuff
d
I see, with the current APIs, I'm not sure if you are able to get the raw audio bytes directly before publishing to livekit.
we'd like to enable this though.. I'm just not sure if it's possible today.
a
hmm. Ok so at least it's possible... Would it be feasible for me to hire someone to do this?
d
which platform(s) are you building for?
a
ios and android using flutter
d
got it. exporting raw audio bytes is something that's on our roadmap for flutter. CCing @freezing-lifeguard-43174 and @eager-raincoat-52616 to advise.
👍 1
f
Sounds like insertable streams, I happened to be doing E2E Encryption for webrtc-sdk/flutter-webrtc this week, maybe I can see how to export/insert audio raw data.
a
That would be magical. My startup has been stalled on this issue for months
If you have a donation endpoint, I'll give what I can
e
when you mean "get the audio from the speaker", do you mean get the audio bytes from the local device's microphone, or all the audio that's output to the device's speakerphone?
a
I need only the audio bytes from the local device's microphone (or headphone mic).
e
got it, the flutter-webrtc library does have access to that at the native level, but it isn't exposed at the moment to the flutter side. (for reference @freezing-lifeguard-43174, https://github.com/flutter-webrtc/flutter-webrtc/blob/main/android/src/main/java/com/cloudwebrtc/webrtc/record/AudioSamplesInterceptor.java, iOS should have a similar hook for the audio module to get the recorded audio bytes)
f
a
@eager-raincoat-52616 is there any way to help make this happen?
d
issue filed: https://github.com/livekit/client-sdk-flutter/issues/201. @ambitious-helicopter-63561 you could follow the issue for updates.
a
Great! Thanks! I was just looking to hire someone from upwork to do this... If I were to hire someone, do you have an estimate for how long it would take? Do you have someone that you would recommend?
d
It'd be hard to give an estimate on it. It really depends on the expertise of the person working on it.
8 Views
Open in Slack
PreviousNext
Powered by