Been working on an architecture that combines Gemi...
# show-and-tell
p
Been working on an architecture that combines Gemini Embedding 2's native audio embeddings with OpenClaw's task execution framework inside a LiveKit agent pipeline — the idea being that AI meeting participants can retrieve context based on how things were said (tone, hesitation, emphasis) rather than just the transcript, and then actually do something about it (schedule follow-ups, create tickets, send docs). Wrote up the full architecture and some of the use cases we're exploring: https://eniveld.substack.com/p/when-your-meeting-agent-can-feel The part I'm most curious about from this community: has anyone experimented with embedding raw audio from LiveKit tracks rather than just STT output? Gemini Embedding 2 accepts audio natively (up to ~80s chunks), which maps pretty well to speaker turns from VAD. We're about to run a validation protocol against SER datasets (RAVDESS, CREMA-D) to see if the embeddings actually separate by affect or just by speaker identity — would be happy to share results when we have them. Also interested if anyone has thoughts on chunking strategies for real-time audio embedding. Per-speaker-turn seems natural with LiveKit's turn detection, but wondering if sliding windows with overlap would produce better retrieval.