Homelab Networking
Exposing the home network network to the wide open internet can be a scary thing. But even with a basic home network setup, it can be done in a relatively secure manner. This post will cover setting up CloudFlare to manage your DNS and loadbalancer. Note that it will be using CloudFlare loadbalancers which requires a subscription of at least $5/month. There are other providers that can offer the same service but (imho) CloudFlare includes a lot of features as well as cloud-native integrations.
Setup
- CloudFlare for external DNS and Loadbalancing
- K3s Kubernetes Cluster (RPis/Intel Nucs)
- MetalLB for baremetal network loadbalancing
- NGINX Loadbalancing for external only network traffic
Depending on the ISP, DDNS may be required. This guide will assume it is not required and that the home network is accessible with a static IP.
Initial Setup on Cloudflare
CloudFlare requires the domain to be under its management. If it’s managed by another entity, like AWS Route53, the registrar will need to be modified to point CloudFlare’s nameservers. That can be done by clicking on “Add a site” and following the instructions to import the domain. If the domain was originally registered with AWS Route53, the nameservers can be changed in the Route53 management console under “Registered Domains.”
To ensure end-to-end encryption, it is recommended to set strict TLS encryption.
- Click into the “SSL/TLS” tab and set Full (Strict).
- On the same page, go to “Edge Certificates” and ensure “Always Use HTTPS” is enabled.
- Also on the same page, go to “Origin Server” and create an “Origin Certificate”. This will be the certificate installed in the homelab to securely communicate to the CloudFlare proxy. Download the certificate and keep it in a secure location.
- On the “Origin Server” page, it is also highly recommended to enable “Authenticated Origin Pulls”. This will make the CloudFlare proxy present a client certificate to the homelab ingress to authenticate. The .PEM file for the CA to authenticate the client certificate can be found here: https://developers.cloudflare.com/ssl/static/authenticated_origin_pull_ca.pem
The certificates will be used when installing the second IngressClass in the homelab cluster.
Preparing Homelab
The homelab K3s cluster will need to be setup before exposing external traffic for the open internet.
Setting up MetalLB
Will not mandatory, installing MetalLB will create a single loadbalanced IP to direct client traffic to the Kubernetes cluster.
To install MetalLB, use helm
with the necessary values.yaml
required for the homelab network environment.
helm repo add metallb https://metallb.github.io/metallb
cat <<EOF | sudo tee /tmp/metallb/values.yaml
configInline:
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.x.x/28 # Replace this CIDR block with the desired list of IPs available on the LAN
EOF
helm upgrade -i --create-namespace -n metallb metallb/metallb -f /tmp/metallb/values.yaml
If the Kubernetes cluster is K3s based, MetalLB will render servicelb
unnecessary. To disable it, simply add the following configuration to the bootstrap server’s /etc/rancher/k3s/config.yaml
:
disable:
- servicelb
Once setup, any loadbalancer
type services that the cluster uses will be assigned a single IP by MetalLB, including the Traefik loadbalancer that K3s comes with. To find what IP that is, simply run kubectl -n kube-system get services -n traefik
. If desired, this could be the Ingress IP for your cluster. All that would be required is setting the default certificate to CloudFlare’s and port forwarding that Ingress IP on the home router.
Setting up the Cluster’s IngressClass
An alternative to having all Ingresses being exposed to the open Internet is having another IngressClass. This way, all administrative or local services can be exposed only to the LAN while only specific services will be exposed.
For this setup, K3s comes with Traefik which will be kept as the local and default IngressClass. The following Helm Chart config will create the Traefik IngressClass as well as other custom configuration.
# Run the following as an administrator on the boostrap server node
mkdir -p /var/lib/rancher/k3s/server/manifests
cat <<EOF | sudo tee /var/lib/rancher/k3s/server/manifests/traefik-config.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
# Specify MetalLB address pool for the loadbalancer service
service:
annotations:
metallb.universe.tf/address-pool: default
# Create default cluster IngressClass
ingressClass:
enabled: true
isDefaultClass: true
# Disable Traefik dashboard
ingressRoute:
dashboard:
enabled: false
# Ensure HTTPS traffic
ports:
web:
redirectTo: websecure
EOF
systemctl restart k3s
To create the second IngressClass, NGINX is used1. This will handle all WAN traffic to and from the CloudFlare proxy. Since a specific certificate is required, the certificate downloaded from CloudFlare above will need to be created and referenced in the NGINX deployment.
# Installing the certificate.
cat <<EOF | sudo tee /tmp/cloudflare/origin-secret.yaml
apiVersion: v1
data:
# To get the base64 of the cert/key, simply run "echo <certificate or key> | base64 -w0 && echo"
tls.crt: <base64 of tls crt>
tls.key: <base64 of tls key>
kind: Secret
metadata:
name: cloudflare-origin-tls
namespace: ingress-nginx
type: kubernetes.io/tls
EOF
kubectl apply -f /tmp/cloudflare/origin-secret.yaml
# Installing the CA
cat << EOF | sudo tee /tmp/cloudflare/ca.yaml
apiVersion: v1
data:
ca.crt: <base64 of ca crt>
kind: Secret
metadata:
name: cloudflare-auth
namespace: ingress-nginx
type: Opaque
EOF
kubectl apply -f /tmp/cloudflare/ca.yaml
# Installing NGINX
helm repo add nginx https://kubernetes.github.io/ingress-nginx
cat <<EOF | sudo tee /tmp/nginx/values.yaml
controller:
config: {}
extraArgs:
# Reference certificate created earlier
default-ssl-certificate: "ingress-nginx/cloudflare-origin-tlsn"
# Create and specify WAN IngressClass.
ingressClassResource:
name: nginx-public # Remember this! This will be the WAN IngressClass
enabled: true
default: false
service:
# Ensure HTTPS only
enableHttp: false
enableHttps: true
# Specify MetalLB address pool for the loadbalancer service
annotations:
metallb.universe.tf/address-pool: default
EOF
helm upgrade -i --create-namespace ingress-nginx nginx/ingress-nginx -f /tmp/nginx/values.yaml
Once installed, MetalLB should provision an IP address for the NGINX loadbalancer which can be identified with kubectl -n ingress-nginx get services ingress-nginx-controller
under EXTERNAL-IP
. This IP will need to be port forwarded on the home router. Each router has different configuration methods, but it should be easy find instructions by searching the specific model of router and “Port Forwarding”.
Specifying External Ingress
Now that the cluster is ready for exposing external traffic, simply set the desired Ingress with the nginx-public
ingressClassName and create annotations for client TLS authentication2. For example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: external-service
namespace: external
annotations:
# For CloudFlare Authenticated Origin Pulls
nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
nginx.ingress.kubernetes.io/auth-tls-secret: "ingress-nginx/cloudflare-auth" # Must match created CA secret
nginx.ingress.kubernetes.io/auth-tls-match-cn: "CN=origin-pull.cloudflare.net"
spec:
# This is where nginx-public must be specified
ingressClassName: nginx-public
rules:
- host: external.domain
http:
paths:
- backend:
service:
name: internal-svc
port:
number: 80
path: /
pathType: ImplementationSpecific
status:
loadBalancer:
ingress:
- ip: 192.168.x.x
# This is the IP address allocated by MetalLB for the NGINX loadbalancer
Finish Configuring CloudFlare
The final step is creating the loadbalancer on Cloudflare.
- Click into the “Traffic” tab and then into “Load Balancing”.
- Create Load Balancer.
- Set the Hostname and ensure the orange proxy setting is enabled.
- Add Origin Pool and set the IP address to the home network. To identify which IP to use, simply run
curl ifconfig.me
. NOTE: This will not work if the ISP does not use a static IP for your network. - Optionally add a Monitor which functions as a health check.
- Add any desired Traffic Steering and Customer Rules as needed.
Once the Loadbalancer is complete and active, anyone can connect to the homelab services securely!
Additional Notes
1 Deploying another Traefik helm release did not work as intended while testing, but should theoretically work as well. For convenience purposes, NGINX was used.
2 Personal testing has not shown a way to set client authentication by default and must be specified for every ingress.
Not covered in this guide but more secure configuration can be achieved from the NGINX ingress. It can be restricted to only CloudFlare proxies IPs found here: https://www.cloudflare.com/ips/