In my self-hosted livekit vm, caddy does not suppo...
# ask-ai
r
In my self-hosted livekit vm, caddy does not support /rtc/v1 path, it supports /rtc path, so this caused an error first on the browser before a fallback happens. I have updated my caddy.yaml to allow rewrites from /rtc/v1 to /rtc, I don't know if this is correct, can you check and validate it for me:
Copy code
logging:
  logs:
    default:
      level: INFO
storage:
  "module": "file_system"
  "root": "/data"
apps:
  tls:
    certificates:
      load_files:
        - certificate: /certs/fullchain.pem
          key: /certs/wildcard.key
  layer4:
    servers:
      main:
        listen: [":443"]
        routes:
          - match:
            - tls:
                sni:
                  - "<http://livekit-turn.example.com|livekit-turn.example.com>"
            handle:
              - handler: tls
              - handler: proxy
                upstreams:
                  - dial: ["localhost:5349"]
          - match:
              - tls:
                  sni:
                    - "<http://livekit.example.com|livekit.example.com>"
            handle:
              - handler: tls
                connection_policies:
                  - alpn: ["http/1.1"]
              - handler: route
                routes:
                  - match:
                      - header:
                          Host:
                            - "<http://livekit.example.com|livekit.example.com>"
                        path:
                          - /rtc/v1/*
                    handle:
                      - handler: rewrite
                        uri: /rtc{http.request.uri.path.1}
                      - handler: reverse_proxy
                        upstreams:
                          - dial: ["localhost:7880"]
                  - handle:
                      - handler: reverse_proxy
                        upstreams:
                          - dial: ["localhost:7880"]
          - match:
              - tls:
                  sni:
                    - "<http://livekit-whip.example.com|livekit-whip.example.com>"
            handle:
              - handler: tls
                connection_policies:
                  - alpn: ["http/1.1"]
              - handler: proxy
                upstreams:
                  - dial: ["localhost:8080"]
t
Thanks for reaching out, Josimar. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I reviewed the Caddy config you shared and it looks correct for rewriting /rtc/v1/* to /rtc/* and proxying to your LiveKit server. What your rule does: • Matches requests to Host = livekit.example.com and path = /rtc/v1/*. • Rewrites the path using
/rtc{http.request.uri.path.1}
so
/rtc/v1/room
becomes
/rtc/room
. • Reverse-proxies the rewritten request to
localhost:7880
(LiveKit server port). Why this fixes the browser error: • Many LiveKit clients request
/rtc/v1/...
. Your LiveKit server expects
/rtc/...
. The rewrite ensures the server receives the path it supports, preventing the initial 4xx/404 error before any client fallback runs. A few optional notes and small checks you may want to confirm: • Caddy version:
{http.request.uri.path.1}
expansion is supported on modern Caddy versions (v2.6+). If you run an older Caddy, confirm support for that placeholder or use a path matcher with captures. • Trailing-slash behavior: Your rewrite preserves the remainder of the path. Confirm any endpoints that rely on exact trailing-slash semantics behave as expected after rewriting. • Host header: Your route matches Host header so requests arriving under other hostnames won’t be rewritten — that appears intentional. • TLS/ALPN settings and proxies for the other hosts (turn, whip) look consistent with your intended routing. If you want, I can: • Suggest an alternative using a path matcher with a capture (more explicit) — useful if you prefer clearer semantics. • Provide a tiny validated snippet using a regex/capture for your exact Caddy version if you tell me which Caddy release you’re running. • Help check logs or test a sample request and show the expected rewritten request/response flow. Would you like the explicit capture/regex alternative or help testing this live on your VM? Sources: examples/other | livekit/livekit_composite | DeepWiki
r
Getting the following errors:
Copy code
caddy-1    | {"level":"info","ts":1771478888.1742625,"msg":"maxprocs: Leaving GOMAXPROCS=4: CPU quot
a undefined"}
caddy-1    | {"level":"info","ts":1771478888.1744928,"msg":"GOMEMLIMIT is updated","package":"github
.com/KimMachineGun/automemlimit/memlimit","GOMEMLIMIT":15047685734,"previous":9223372036854775807}
caddy-1    | {"level":"info","ts":1771478888.1746354,"msg":"using config from file","file":"/etc/cad
dy.yaml"}
caddy-1    | {"level":"info","ts":1771478888.1752107,"msg":"adapted config to JSON","adapter":"yaml"
}
caddy-1    | {"level":"info","ts":1771478888.1777651,"msg":"maxprocs: No GOMAXPROCS change to reset"
}
caddy-1    | Error: loading initial config: loading new config: loading layer4 app module: provision
 layer4: server 'main': route 1: position 1: loading module 'route': unknown module: layer4.handlers
.route
caddy-1 exited with code 1 (restarting)
t
Thanks — that error log is clear and I can help. You’re seeing: Error: loading initial config: loading new config: loading layer4 app module: provision layer4: server 'main': route 1: position 1: loading module 'route': unknown module: layer4.handlers.route What this means • Caddy does not recognize
layer4.handlers.route
because the Layer 4 app (caddy-l4) is not present in the Caddy binary you are running. • Your config uses the
layer4
app and a
route
handler inside it; that only works if Caddy was built with the caddy-l4 plugin. Why this matters for LiveKit • LiveKit’s
/rtc/v1
signaling is HTTP/WebSocket based. You do not need Layer 4 for path rewrites or HTTP/WebSocket proxying — a normal HTTP (Layer 7) server with
rewrite
+
reverse_proxy
is the correct and simpler approach for
/rtc/v1
->
/rtc
rewrites. Recommended fixes (choose one) 1) Switch to Caddy’s HTTP app (recommended) • Replace your
layer4
block with an
http
app server that matches Host + path, rewrites
/rtc/v1/*
→
/rtc/*
, and reverse-proxies to
localhost:7880
. • Example skeleton (adapt to your full config and TLS setup):
Copy code
apps:
  http:
    servers:
      srv0:
        listen: [":443"]
        routes:
          - match:
              - host: ["<http://livekit.example.com|livekit.example.com>"]
                path: ["/rtc/v1/*"]
            handle:
              - handler: rewrite
                uri: /rtc{http.request.uri.path.1}
              - handler: reverse_proxy
                upstreams:
                  - dial: "localhost:7880"
          - match:
              - host: ["<http://livekit.example.com|livekit.example.com>"]
            handle:
              - handler: reverse_proxy
                upstreams:
                  - dial: "localhost:7880"
Why this is recommended: • No special Caddy build required. • Properly supports HTTP/WebSocket which LiveKit uses. • Allows path rewrites before proxying. 2) If you truly need Layer 4: rebuild Caddy with the caddy-l4 module (advanced) • Rebuild Caddy including the
<http://github.com/mholt/caddy-l4|github.com/mholt/caddy-l4>
plugin: ◦ Example:
caddy build --with <http://github.com/mholt/caddy-l4|github.com/mholt/caddy-l4>
• Verify the module is present:
caddy list-modules | grep layer4
• Only choose this if you require raw TCP/UDP routing not supported by the HTTP app (rare for
/rtc/v1
usage). Quick checks you can run now • Confirm Caddy version:
caddy version
• Confirm modules:
caddy list-modules | grep layer4
• Look at container image: if you’re using an official Caddy image it likely lacks caddy-l4 by default. Optional refinements • If you want an explicit capture-based rewrite (more readable), I can give a validated snippet for your Caddy version. • If you paste your full intended TLS/http setup (which Caddy image/version you run), I can convert your original YAML into a ready-to-drop
http
app config. Would you like me to: • Convert your full original config into an
http
app version you can paste in (I can include the turn/whip hosts and TLS certs), or • Provide the exact commands to rebuild Caddy with caddy-l4 if you decide Layer 4 is required? Tell me which option you prefer and your Caddy version (output of
caddy version
) and I’ll prepare the exact next steps.
r
I'm using livekit/caddyl4 image v2.10.2 created on 2025-10-14T204958.795551144Z"
t
Thanks — this is helpful. You’re running the livekit/caddyl4 image (v2.10.2). That tells us two important things: • The image is intended for Layer 4 (TCP/UDP) proxying. Layer 4 handles raw streams and does not understand HTTP paths. • Your current config uses HTTP/Layer 7 constructs (path matching + rewrite for /rtc/v1), which are not valid inside a layer4 app. That mismatch is why Caddy complains: unknown module: layer4.handlers.route. What to do next (pick the path that matches your goal) 1) If you need HTTP(S) path rewrites for /rtc/v1 (most likely) • Use the standard Caddy HTTP server (the normal
caddy
image or a Caddy binary built with the
http
app). HTTP is required for matching paths, performing rewrites, and proxying WebSocket/HTTP requests. • Move your rewrite + reverse_proxy rules into an
http
app server. Example skeleton you can adapt:
Copy code
apps:
  http:
    servers:
      srv0:
        listen: [":443"]
        routes:
          - match:
              - host: ["<http://livekit.example.com|livekit.example.com>"]
                path: ["/rtc/v1/*"]
            handle:
              - handler: rewrite
                uri: /rtc{http.request.uri.path.1}
              - handler: reverse_proxy
                upstreams:
                  - dial: "localhost:7880"
          - match:
              - host: ["<http://livekit.example.com|livekit.example.com>"]
            handle:
              - handler: reverse_proxy
                upstreams:
                  - dial: "localhost:7880"
Why this is the recommended choice: • LiveKit signaling (
/rtc/v1
) uses HTTP/WebSocket. Layer 7 proxying supports path-based routing and rewrites, TLS termination, and correct Host/Upgrade headers for WebSocket. • No custom Caddy build required. 2) If you truly need Layer 4 (raw TCP/UDP) for media forwarding • Keep using the livekit/caddyl4 image, but do not include HTTP path rewriting there — Layer 4 cannot rewrite HTTP paths because it sees only raw TCP/UDP bytes. • Use layer4 only to forward ports (for TURN or raw media ports). Example use-case: forwarding UDP/TCP ports that carry RTP or TURN traffic. • For path-based HTTP traffic (signaling), terminate TLS and route at Layer 7 (see option 1). In other words, use layer4 for media tunnels and HTTP Caddy (or another HTTP reverse proxy) for signaling. 3) If you want to keep a single Caddy-based solution • You can run two Caddy containers: ◦ One built/packaged with
caddy-l4
for TCP/UDP forwarding (livekit/caddyl4). ◦ One standard
caddy
image for HTTP path rewrites and reverse proxying to LiveKit. • Or build a custom Caddy binary that includes both the HTTP app and the caddy-l4 plugin (advanced). If you go this route, ensure the binary actually includes
layer4
(verify with
caddy list-modules | grep layer4
). Quick checks you can run right now • Confirm which modules are present in the running image:
caddy list-modules | grep layer4
(inside container). If nothing prints, the binary doesn’t expose the layer4 module. • Check
caddy version
inside the container to confirm the binary release. • If you want, paste your full desired behavior (which hosts/ports you want TLS-terminated, which ports you want proxied raw) and I’ll produce a ready-to-use pair of configs: ◦ An
http
app Caddy config for /rtc/v1 rewrites + reverse proxy ◦ A simple
layer4
config for TCP/UDP forwarding (if you need it) Which do you prefer? • Convert your current YAML to a working
http
app config now, or • Keep using livekit/caddyl4 and I’ll show a correct split design (layer4 for media ports + http caddy for signaling)? Tell me which option you want and I’ll prepare the exact config and commands. Sources: Data Processing Addendum | LiveKit | Running LiveKit on AWS