Can I customize how the agent (LLM) works inside L...
# ask-ai
c
Can I customize how the agent (LLM) works inside LiveKit? Like can I make is follow stricter rules through hard coding loops?
t
Thanks for reaching out, Subham. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
You're asking whether you can make an LLM agent inside LiveKit follow stricter rules — for example, by “hard-coding loops.” Short answer: yes, you can enforce stricter, deterministic agent behavior, but not by inserting traditional imperative loops into the LLM’s own reasoning. Here’s how you can achieve strict control instead. What you can do • Override pipeline nodes to inject custom logic before/after the LLM (for input validation, context shaping, constraint enforcement). • Define tools (using
@function_tool
in Python or
llm.tool()
in Node) and require the agent to call those tools for specific actions; tools can validate inputs, enforce step-by-step flows, and return deterministic results. • Build workflows that split a task into discrete, rule-driven steps or handoffs between agents to create deterministic, stateful processes. • Use session lifecycle hooks (e.g.,
on_user_turn_completed()
,
on_enter()
,
on_exit()
) to inspect/modify state and enforce entry/exit conditions. • Require structured outputs (schema validation with Zod/JSON Schema) and reject or retry outputs that don’t match the rules. How to emulate “hard-coded loops” • Use tool calls + state management to perform iterative behavior: have the agent call a tool that updates state and decides whether another step is needed, then invoke the agent/workflow again. This produces loop-like, repeatable flows while keeping control outside the LLM’s internal reasoning. Key limitation • You cannot embed `while`/`for` loops inside the LLM’s internal thought process; control has to be implemented around the model (nodes, tools, workflows, hooks). If you want, I can: • Show a short example pattern (pipeline node + tool + state) in your preferred language (Python or Node) to implement a strict, loop-like workflow. Which language would you prefer? Sources: Pipeline nodes and hooks | LiveKit docs | Tool definition and use | LiveKit docs | Building the all-in-one platform for voice AI agents
c
Can you give me a short example of same
t
Which language would you like the short example in: Python or Node?
c
python
t
Below is a short Python example that shows a pipeline node override + a strict tool + simple state management to emulate a loop-like, rule-driven agent flow.
Copy code
from livekit import Agent, pipeline, llm, StateStore
from livekit.tools import function_tool
from pydantic import BaseModel

# Simple in-memory state store for loop-like iteration tracking
state = StateStore()  # replace with your persistent store in production

# Tool that performs one step and returns whether more steps are needed
class StepResult(BaseModel):
    ok: bool
    continue_loop: bool
    message: str

@function_tool(name="perform_step", return_type=StepResult)
def perform_step(step_input: str, iteration: int) -> StepResult:
    # Strict validation and deterministic behavior
    if not step_input:
        return StepResult(ok=False, continue_loop=False, message="missing input")
    # Example deterministic rule: stop after 3 iterations
    if iteration >= 3:
        return StepResult(ok=True, continue_loop=False, message="done")
    # Perform action for this iteration
    return StepResult(ok=True, continue_loop=True, message=f"processed {iteration}")

# Pipeline node override to enforce rules before LLM runs
@pipeline.node("llm_node")
def llm_node_override(context, next_node):
    user_text = context.input_text.strip()
    # Enforce rule: reject empty user messages
    if not user_text:
        return llm.Response(content="Please provide input.", finished=True)

    # Load iteration counter from state (per-session key)
    session_id = context.session_id
    iteration = state.get(session_id, default=0)

    # Force the model to only call the tool and produce structured output
    context.tools = ["perform_step"]
    context.forced_structure = {"type": "tool_call", "tool": "perform_step"}

    # Attach iteration info to context so tool receives it
    context.metadata["iteration"] = iteration

    # Run next node (LLM) which should call the tool
    resp = next_node(context)

    # After LLM/tool run, read tool output from context.results
    tool_out = context.results.get("perform_step")
    if tool_out:
        iteration += 1
        state.set(session_id, iteration)

        if tool_out.get("continue_loop"):
            # Re-run the pipeline to emulate another loop iteration
            # Note: in real code, schedule or recursively invoke the flow
            return llm.Response(content=f"Iteration {iteration} done. Continuing...", finished=False)
        else:
            # Finalize
            return llm.Response(content=f"Finished after {iteration} iterations. {tool_out.get('message')}", finished=True)

    return resp

# Agent setup
agent = Agent(name="strict_agent")
agent.add_tool(perform_step)
agent.register_pipeline_node(llm_node_override)

# Example usage (framework-specific call)
# agent.handle_user_message(session_id="sess123", text="start process")
If you want, I can: • Turn this into a runnable example tied to your project/session API, • Show how to persist state in Redis/Postgres, • Or provide a Node version. Which would help most?