https://livekit.io logo
Join Slack
Powered by
is livekit server and AIagent should be deployed s...
# ask-ai
a
is livekit server and AIagent should be deployed separate ? What if i deploy my AI Agent, Livekit Server, TTS , STT on Same GPU server. Is it OK? If not, what is recommended approach?
t
Thanks for reaching out, Ali. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Deploying LiveKit Server, AI agents, TTS, and STT on the same GPU server is technically possible but not recommended for production environments. Here are the key considerations and recommendations: • Resource Competition: Each of these services can be resource-intensive, especially when using GPUs. Running them together on the same server may lead to resource contention, affecting performance and reliability. • Production Reliability: For high reliability, it is recommended to avoid running other applications on the same server as your LiveKit agent. This ensures that each service has dedicated resources and avoids potential conflicts. • Deployment Flexibility: LiveKit agents and servers are designed to be deployed in distributed architectures, often using container orchestration systems like Kubernetes. This allows for better scalability and resource management. • External Service Integration: Many implementations integrate with external AI services (e.g., for STT and TTS) via APIs rather than hosting these services locally. This reduces the load on your infrastructure and simplifies deployment. • Recommended Approach: Use separate servers or containers for each service type. For example: ◦ Deploy LiveKit Server on a dedicated server or container. ◦ Run AI agents on separate servers or containers, ensuring they have sufficient GPU resources. ◦ Use external APIs for TTS and STT services, or deploy them on dedicated infrastructure if local hosting is required. This distributed approach aligns with best practices for scalability, reliability, and performance. Sources: AI Agents Framework | livekit/livekit_composite | DeepWiki | Deploying to production | LiveKit Docs | Voice Assistants | livekit/livekit_composite | DeepWiki | Deploying to production | LiveKit Docs | An open source stack for real-time multimodal AI
a
What if I have to Deploy on Same GPU, should i need to use network mode as "host" or not. Like when we deploy Livekit Server on our machine ourself, the in docker-compose file network mode is set to "host"
Copy code
# This docker-compose requires host networking, which is only available on Linux
# This compose will not function correctly on Mac or Windows
services:
  caddy:
    image: livekit/caddyl4
    command: run --config /etc/caddy.yaml --adapter yaml
    restart: unless-stopped
    network_mode: "host"
    volumes:
      - ./caddy.yaml:/etc/caddy.yaml
      - ./caddy_data:/data
  livekit:
    image: livekit/livekit-server:latest
    command: --config /etc/livekit.yaml
    restart: unless-stopped
    network_mode: "host"
    volumes:
      - ./livekit.yaml:/etc/livekit.yaml
  redis:
    image: redis:7-alpine
    command: redis-server /etc/redis.conf
    restart: unless-stopped
    network_mode: "host"
    volumes:
      - ./redis.conf:/etc/redis.conf
  egress:
    image: livekit/egress:latest
    restart: unless-stopped
    environment:
      - EGRESS_CONFIG_FILE=/etc/egress.yaml
    network_mode: "host"
    volumes:
      - ./egress.yaml:/etc/egress.yaml
    cap_add:
      - CAP_SYS_ADMIN
  ingress:
    image: livekit/ingress:latest
    restart: unless-stopped
    environment:
      - INGRESS_CONFIG_FILE=/etc/ingress.yaml
    network_mode: "host"
    volumes:
      - ./ingress.yaml:/etc/ingress.yaml
  sip:
    image: livekit/sip
    network_mode: host
    environment:
      SIP_CONFIG_BODY: |
        api_key: ''
        api_secret: ''
        ws_url: '<ws://localhost:7880>'
        redis:
          address: 'localhost:6379'
        sip_port: 5060
        rtp_port: 10000-20000
        use_external_ip: true
        logging:
          level: debug
t
Hey there! Let me break down the networking setup for your LiveKit Server and related services in a way that's super clear and friendly. • LiveKit Server: Think of this like needing a direct highway for super smooth real-time communication. You'll want to use 
network_mode: host
to make sure everything runs lightning-fast and connects perfectly. It's basically giving LiveKit its own dedicated lane! • AI Agents: These guys are more low-key. They don't need a special network pass - they'll just communicate with LiveKit Server through standard WebSocket connections. Bridge networking works great here, keeping things simple and secure. • TTS and STT Services: These are a bit more flexible. Depending on how you've set them up, they might need host networking for real-time media, or they might be totally happy with bridge networking. It really depends on your specific setup. The goal here is to give LiveKit Server the network access it needs while keeping everything else neat and tidy. Got any specific details about your setup? I'd be happy to help you fine-tune things even more! Sources: docker-compose.yaml | Deploy to Kubernetes | LiveKit Docs | Self-hosting the Ingress Service | LiveKit Docs | Deployment and scaling | LiveKit Docs | tests/docker-compose.yml
Open in Slack
PreviousNext
Powered by