Thanks — I’m responding to your question about the difference between passing the main prompt as a system instruction to a realtime (live) model versus passing it as an instruction to the agent. Short answer • Pass the prompt to the agent when you want to define the agent’s stable role, goals, tool usage, and decision-making behavior. • Pass the prompt to the realtime model (system instruction) when you want immediate control over how the model speaks and behaves during live inference (tone, prosody, short-term style). Why they differ (concise) • Scope ◦ Agent `instructions`: apply at the framework/agent level — they guide overall identity, workflow, tool selection, and long-lived reasoning. ◦ Realtime model `instructions`: apply at model inference — they directly affect the model’s conversational output for that session. • When they’re used ◦ Agent instructions remain part of the agent’s persistent context across sessions and handoffs. ◦ Model instructions are applied to the live model during a realtime session and can be changed per session. • Precedence ◦ For realtime models, model-level

instructions

take precedence for the conversational output. The agent instructions still control tool logic, handoffs, and higher-level decisions. • Best-fit use cases ◦ Use agent instructions to set identity, safety guardrails, tool constraints, and task-level behavior. ◦ Use realtime model instructions to tune tone, empathy, verbosity, speech style, or other immediate interaction nuances (especially for speech-to-speech). • Complementary use ◦ They can be combined: agent instructions define what the agent must achieve and constraints; model instructions define how the model should present that behavior in the live interaction. Recommendation (practical) • Keep durable, policy-like content (tools, goals, safety, role) in the agent

instructions

. • Put ephemeral, presentation-focused content (tone, prosody, phrasing style, session-specific modifications) in the realtime model

instructions

. • Avoid duplicating the same prompt across both; instead split responsibilities so each layer has a clear purpose. If you’d like, tell me: • The specific behavior or tone you want to enforce (e.g., “always be concise and professional” vs “be warm and informal”), and whether it should be permanent or session-specific — I can recommend an exact split of agent vs model instructions and draft example prompts for each. Sources: Prompting guide | LiveKit Documentation | OpenAI Realtime API plugin guide | LiveKit Documentation | Gemini Live API plugin | LiveKit Documentation