In Node agent livekit How can it turn e2ee on base...
# ask-ai
t
In Node agent livekit How can it turn e2ee on base on roomName Currently agent could not stt when e2ee turn on
Copy code
import { cli, defineAgent, type JobContext, type JobProcess, metrics, voice, WorkerOptions } from '@livekit/agents';
import * as deepgram from '@livekit/agents-plugin-deepgram';
import * as silero from '@livekit/agents-plugin-silero';

// import * as cartesia from '@livekit/agents-plugin-cartesia';
// import * as livekit from '@livekit/agents-plugin-livekit';
// import * as openai from '@livekit/agents-plugin-openai';
// import { BackgroundVoiceCancellation } from '@livekit/noise-cancellation-node';
import dotenv from 'dotenv';
import { fileURLToPath } from 'node:url';

dotenv.config({ path: '.env.local' });

class Assistant extends voice.Agent {
  constructor() {
    super({
      instructions: `You are a helpful voice AI assistant.
      You eagerly assist users with their questions by providing information from your extensive knowledge.
      Your responses are concise, to the point, and without any complex formatting or punctuation including emojis, asterisks, or other symbols.
      `,
    });
  }
}

export default defineAgent({
  prewarm: async (proc: JobProcess) => {
    proc.userData.vad = await silero.VAD.load();
  },
  entry: async (ctx: JobContext) => {
    // Set up a voice AI pipeline using OpenAI, Cartesia, Deepgram, and the LiveKit turn detector
    const session = new voice.AgentSession({
      // A Large Language Model (LLM) is your agent's brain, processing user input and generating a response
      // See all providers at <https://docs.livekit.io/agents/integrations/llm/>
      // llm: new openai.LLM({ model: 'gpt-4o-mini' }),
      // Speech-to-text (STT) is your agent's ears, turning the user's speech into text that the LLM can understand
      // See all providers at <https://docs.livekit.io/agents/integrations/stt/>
      stt: new deepgram.STT({ model: 'nova-3' }),
      // Text-to-speech (TTS) is your agent's voice, turning the LLM's text into speech that the user can hear
      // See all providers at <https://docs.livekit.io/agents/integrations/tts/>
      // tts: new cartesia.TTS({
      //   voice: '6f84f4b8-58a2-430c-8c79-688dad597532',
      // }),
      // VAD and turn detection are used to determine when the user is speaking and when the agent should respond
      // See more at <https://docs.livekit.io/agents/build/turns>
      // turnDetection: new livekit.turnDetector.MultilingualModel(),
      // vad: ctx.proc.userData.vad! as silero.VAD,
    });

    // To use a realtime model instead of a voice pipeline, use the following session setup instead:
    // const session = new voice.AgentSession({
    //   // See all providers at <https://docs.livekit.io/agents/integrations/realtime/>
    //   llm: new openai.realtime.RealtimeModel({ voice: 'marin' }),
    // });

    // Metrics collection, to measure pipeline performance
    // For more information, see <https://docs.livekit.io/agents/build/metrics/>
    const usageCollector = new metrics.UsageCollector();
    session.on(voice.AgentSessionEventTypes.MetricsCollected, (ev) => {
      metrics.logMetrics(ev.metrics);
      usageCollector.collect(ev.metrics);
    });

    session.on(voice.AgentSessionEventTypes.UserInputTranscribed, (ev) => {
      console.log(ev);
    });

    session.on(voice.AgentSessionEventTypes.UserStateChanged, (ev) => {
      console.log(ev);
    });

    const logUsage = async () => {
      const summary = usageCollector.getSummary();
      console.log(`Usage: ${JSON.stringify(summary)}`);
    };

    ctx.addShutdownCallback(logUsage);

    // Start the session, which initializes the voice pipeline and warms up the models
    await session.start({
      agent: new Assistant(),
      room: ctx.room,
      inputOptions: {
        videoEnabled: false, audioEnabled: true,
      },
      outputOptions: {
        transcriptionEnabled: true,
        audioEnabled: false,
      }
    });


    const roomName = ctx?.job?.room?.name || '';


    await ctx.connect({
      keyProviderOptions: {
        sharedKey: new TextEncoder().encode(roomName),
      },
    });
  },
});

cli.runApp(new WorkerOptions({ agent: fileURLToPath(import.meta.url), agentName: 'transcript-agent-alpha' }));
t
Thanks for reaching out, Quinn. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To help you enable end-to-end encryption (E2EE) in your Node.js agent based on the room name, here’s a simple and clear approach you can follow: 1. Access the Room Name: Start by retrieving the room name from
ctx.room.name
. 2. Define Conditional Logic: Next, set up some logic to decide if E2EE should be enabled for the current room. For example, you might check if the room name starts with a certain prefix. 3. Configure E2EE Options: When the condition is met, create an
E2EEOptions
object with a valid
KeyProviderOptions
. Here’s an example:
Copy code
const e2eeConfig = {
  keyProviderOptions: {
    sharedKey: Buffer.from('your-secure-key-here'),
  },
};
1. Connect with E2EE: Finally, pass the
e2eeConfig
to
ctx.connect()
only when E2EE is needed:
Copy code
if (roomName.startsWith('secure-')) {
  await ctx.connect({ e2ee: e2eeConfig });
} else {
  await ctx.connect();
}
Just a quick reminder: make sure the shared key is managed securely and is the same for all participants in the room. If you have any questions or want me to walk you through any part of this, I’m here to help! Sources: End-to-end encryption | LiveKit Docs
t
here is what I set on livekit-client for local partipiant
Copy code
const keyProvider = useMemo(() => new ExternalE2EEKeyProvider(), []);
const worker = useMemo(
    () => new Worker(new URL('livekit-client/e2ee-worker', import.meta.url)),
    []
  );
      void keyProvider.setKey(roomName);
Agent should use the same key. so roomName should be shared key? But when i add
Copy code
const e2eeConfig = {
  keyProviderOptions: {
    sharedKey: Buffer.from(roomName),
  },
};
agent could not understand what user say to transcibe
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
t
what channel i should ask