k8s clusters backed by Flux v2

Kubernetes clusters using the GitOps tool Flux.
The Git repository is the driving the state of the Kubernetes clusters.
The awesome Flux SOPS integration is used to encrypt secrets with gpg.

📂 Repository structure

The Git repository contains the following directories:

📁
├─📁 apps
│  ├─📁 all          # apps available for intallation
│  └─📁 production   # kustomization and overlays for app installations per cluster
├─📁 base
│  ├─📁 flux-system  # flux & gitops operator
│  └─📁 production   # flux configuration per cluster
├─📁 charts          # helm chart repos
├─📁 config          # configs per cluster
└─📁 crds            # custom resources required by apps

💻  Software

The following apps are installed on the clusters.

Software	Purpose
Flux2	GitOps Tool managing the cluster
Longhorn	Persistent Block Storage Provisioner
NGINX Ingress Controller	Cluster Ingress controller
MetalLB	Bare metal LoadBalancer
Cert-Manager	Letsencrypt certificates with Cloudflare DNS
ExternalDNS	Configure Cloudflare DNS Servers
kube-vip	Virtual IP Load-Balancer for Control Plane High Availability
Kube-Prometheus Stack	Prometheus & Exporters to monitor the cluster
Grafana	Monitoring & Logging Dashboard
Alertmanager	Monitoring Alerts
Grafana Loki	Log aggregation system
System Upgrade Controller	Automated k3s upgrades
Weave GitOps	Powerful WebUI extension to Flux for deployment insights
MinIO	Amazon S3 compatible high Performance Object Storage
Authelia	SSO & 2FA authentication server for Cluster Web Apps
Paperless-ngx	Document management system
Rancher	Kubernetes Management Dashboard
Homer	Static dashboard for the cluster applications
Pod-Gateway	Route traffic through a VPN gateway
Descheduler	Evicts pods to optimize scheduling
Goldilocks	Utility to help identifying good resource requests and limits
X.509 Certificate Exporter	A Prometheus exporter to monitor x509 certificates
SMB CSI Driver	CSI Driver for SMB
kured	Kubernetes Reboot Daemon (only used for Monitoring)
Kubernetes Metrics Server	Source of container resource metrics for Kubernetes

🤖  Automation

Renovate Bot makes sure the components are never outdated.

It creates PullRequests when Helm charts or Docker images have newer versions available and even keeps Flux and k3s up-to-date.

🤝  Thanks

Big shout out to k8s@home and everyone from awesome-home-kubernetes for the inspiration ❤️

📖  Notes

📍 Installation Notes

tl;dr

kubectl create namespace flux-system --dry-run=client -o yaml | kubectl apply -f -
sops -d ./base/flux-system/init/flux-sops-gpg-secret.sops.yaml | kubectl apply -f -
sops -d ./base/flux-system/init/flux-secret.sops.yaml | kubectl apply -f -
kubectl apply --kustomize=./base/flux-system
kubectl apply --kustomize=./base/staging

1. Pre-create the flux-system namespace

kubectl create namespace flux-system --dry-run=client -o yaml | kubectl apply -f -

4. Add the Flux GPG key in-order for Flux to decrypt SOPS secrets

sops -d ./base/flux-system/init/flux-sops-gpg-secret.sops.yaml | kubectl apply -f -

5. (Optional) Add the Flux SSH key in-order for Flux to pull private git repositories

sops -d ./base/flux-system/init/flux-secret.sops.yaml | kubectl apply -f -

5. Install Flux

kubectl apply --kustomize=./base/flux-system

6. Configure Flux

kubectl apply --kustomize=./base/staging