Carbon Aware Kubernetes Extensions (cake)

Enable carbon aware operations in your Kubernetes cluster

cake-node-analyzer

Consider a hypothetical k8s cluster with autoscaling node pools in 3 regions: A, B and C. The Carbon Aware Operator has gathered data from the Emissions Data Source and determined that region C is the least green. It applies a taint to the nodes in region C:

# ...
taints:
- key: greensoftware.foundation/carbon-intensity
  value: high
  effect: NoSchedule

Now, if resources are submitted to the scheduler, they will not be scheduled on any nodes in region C unless they provide a matching toleration:

# ...
tolerations:
- key: greensoftware.foundation/carbon-intensity
  operator: Exists
  effect: NoSchedule

With this we have a way of using a built-in k8s mechanism to bring carbon awareness to our daily operations. In addition to taints, the CAO could be configured to apply labels to nodes, allowing node (anti-)affinity to be carbon aware as well.

The intended way to use the tool would be to configure a set of taints and/or labels that would allow you to naturally move workloads from dirty regions to clean ones. Once utilization of dirty regions is low or gone, you can remove it from your configs entirely.

Usage

Install prereqs:

• go >=1.19 from source or gvm

# dev
go mod download
go run .
./scripts/verify.sh

# run
docker build -t my.repo/cake-node-analyzer:latest .
helm install cake-node-analyzer k8s/cake-node-analyzer --set image=my.repo/cake-node-analyzer:latest

Misc

• hackathon website
• Carbon Aware SDK
• scheduler plugin
  • https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/
  • https://kubernetes.io/docs/concepts/scheduling-eviction/scheduling-framework
  • https://medium.com/@juliorenner123/k8s-creating-a-kube-scheduler-plugin-8a826c486a1
  • https://github.com/kubernetes-sigs/scheduler-plugins/blob/master/doc/install.md
  • https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/
  • https://github.com/kubernetes-sigs/scheduler-plugins
• GKE labels
  • topology.gke.io/zone=us-central1-b
  • topology.kubernetes.io/region=us-central1
  • topology.kubernetes.io/zone=us-central1-b