Hello everyone I am new to Kubernetes and I have a Kubernete

Question

Hello everyone I am new to Kubernetes and I have a Kubernetes networking related question I am adding the details in the thread to avoid cluttering the main channel EDIT Stack Overflow question <https stackoverflow com q 72640324 7911552>

silly-thailand-99259 · Answer

I am deploying WireGuard on a pod and I have enabled host networking on this pod `hostNetwork` But after few minutes of deploying this pod the DNS stops working I have already set the `dnsPolicy` to `ClusterFirstWithHostNet` You can find some more details here <https www reddit com r kubernetes comments vd0g09 dns stops working when using hostnetwork >

rough-application-24965 · Answer

How is your cluster setup

white-napkin-85004 · Answer

which version of k8s are you using and are your kube dns pods healthy

dry-monkey-93718 · Answer

I have a feeling that <https www telepresence io |telepresence> might solve your underlying problem

silly-thailand-99259 · Answer

< rough application 24965> the cluster is running on Vultr

silly-thailand-99259 · Answer

< white napkin 85004> K8s version is `v1 23 5` and yes the kube dns pod is healthy DNS breaks only on this pod where WireGuard is running DNS is working fine on other pods

silly-thailand-99259 · Answer

hey < dry monkey 93718> I don t understand how a local environment would help me here I came across telepresence through you message

rough-application-24965 · Answer

< silly thailand 99259> are you following any article docs

dry-monkey-93718 · Answer

What is it that you re trying to achieve by installing wireguard People typically reach out for wireguard to connect to k8s services from their local environment

silly-thailand-99259 · Answer

No I am googling how to do x to get this working

silly-thailand-99259 · Answer

Since i needed to forward a subnet from another node to this node I chose adding `hostNetwork` privilege as it allows adding adding `ip route` on the node network

silly-thailand-99259 · Answer

< dry monkey 93718> I work on an opensource network management software OpenWISP The application needs to reach the devices I need WireGuard tunnels for that

rough-application-24965 · Answer

is the wiregaurd working when you directly connect to it via IP how are you saying that DNS isn t working did the DNS fail when you ran the dig command

rough-application-24965 · Answer

are these devices outside the kubernetes cluster if they are you need to use ingress

silly-thailand-99259 · Answer

When I run without host networking `hostNetwork true` the DNS works just fine the WireGuard tunnels come up and I can send traffic through these tunnels from outside When I re deploy after enabling host networking the pods works fine for some time say 5 minutes after which I get logged out from the pod if I have attached a terminal to it using `kubectl exec` After this logout this DNS stops working The service running on this WireGuard pod cannot resolve any other service FQDN

silly-thailand-99259 · Answer

gt are these devices outside the kubernetes cluster gt gt if they are you need to use ingress currently I am using NodePort for exposing the WireGuard port I am aware that I would require an ingress for this but I want the routing to working before spending time on setting up ingress

dry-monkey-93718 · Answer

Is the pod still running in the cluster `kubectl get po`

rough-application-24965 · Answer

why do you need host networking won t nodeport do the trick

silly-thailand-99259 · Answer

I need to route traffic from another pod to the WireGuard pod for a specific subnet Then that traffic is required to be routed through the wireguard interface

silly-thailand-99259 · Answer

Yes it is running < dry monkey 93718>

silly-thailand-99259 · Answer

Are you folks familiar with Calico s configuration I ve a feeling that it might be a solution to my problem But I don t understand how to configure it for my problem

rough-application-24965 · Answer

does vke use calico

silly-thailand-99259 · Answer

yes it does

rough-application-24965 · Answer

also how are you routing the traffic from another pod via wiregaurd are you using egress

silly-thailand-99259 · Answer

In the another pod I have added an `ip route` `ip route add lt subnet gt via lt private vpc ip of wireguard node gt `

rough-application-24965 · Answer

curious why are you doing this if they are in the same kubernetes won t they share the same network as the wiregaurd one

dry-monkey-93718 · Answer

I have a feeling that you re trying to solve problems on k8s like a VM and struggling because it doesn t work like that

silly-thailand-99259 · Answer

gt I have a feeling that you re trying to solve problems on k8s like a VM and struggling because it doesn t work like that I agree to this This is my first time working with K8s sweat smile

silly-thailand-99259 · Answer

gt if they are in the same kubernetes won t they share the same network as the wiregaurd one the WireGuard tunnels have completely different subnet from the VPC of the cluster the application is only aware of the WireGuard peers IP address the application sends network packets with WireGuard peer IP address

silly-thailand-99259 · Answer

e g application running on pod `app` pings the the device with WireGuard address `172 16 0 2` the ICMP request packet has to be routed from `app` pod to the `wireguard`` pod and reverse with the response packet

dry-monkey-93718 · Answer

I remember seeing a very old openVPN k8s tutorial to route traffic from the cluster to your local Let me try to find that It might help you

rough-application-24965 · Answer

can these two subnets talk to each other

dry-monkey-93718 · Answer

gt can these two subnets talk to each other That s what he s trying to do with `ip route` on the other pod + wireguard tunnel from this one

rough-application-24965 · Answer

I suggest doing it via calico instead of changing route table of the pod <https projectcalico docs tigera io networking workloads outside cluster> It s tricky with many layers involved

silly-thailand-99259 · Answer

I banged my head yesterday on this topic but I don t understand what Calico s documentation asks me to do

rough-application-24965 · Answer

You are using two subnets for security

silly-thailand-99259 · Answer

I guess so

silly-thailand-99259 · Answer

I want calico to route my traffic on the `app` pod to the `wiregaurd` pod for a specific subnet say `172 16 0 0 24`

rough-application-24965 · Answer

I think the root cause is that when you use host network you get a vm s IP for the pod and since the subnets are different and connection isn t allowed via security groups or equivalent the app pod isn t able to connect to wiregaurd pod via host networking without host networking it is working as kuberenetes overlay network is taking care of the networking part

silly-thailand-99259 · Answer

< rough application 24965> I will test your hypothesis in a bit I remember that I was able to ping the wireguard interface on the `wireguard` pod from the `app` pod yesterday I don t remember whether I was able to ping the device s IP address from the `app` pod

dry-monkey-93718 · Answer

wrt OpenVPN Nope I remembered wrong The one I d seen assumes you ll send traffic to this pod at various ports Now I m convinced that you ve to solve it at one of Overlay network CNI level Maybe read code from something like Vultr if it s possible Before that what I m trying to understand is if you need to do all of this at all If you have a single application server that needs to connect to your router devices you can probably run the wireguard container in a sidecar pattern and as far as I know you ll share network between the containers of the same pod

silly-thailand-99259 · Answer

thanks you folks for brainstorming with me over this I will update how we actually end up solving this problem

dry-monkey-93718 · Answer

Also sig network in <https slack k8s io > might be a good place to ask

silly-thailand-99259 · Answer

If you folks are curious to learn about the network management application OpenWISP I am working on this PR