Observability Collector for Kubernetes

o11y-collector-for-kubernetes is a Helm chart that creates a kubernetes daemonset along with other kubernetes objects in a kubernetes cluster to collect the cluster's logs, traces and metrics send them to Signalfx.

Log Collection

The deamonset runs fluentd with the Splunk HEC output plugin to collect logs and send them over Splunk HEC.

It does not only collects logs for applications which are running in the kubernetes cluster, but also the logs for kubernetes itself (i.e. logs from kubelet, apiserver, etc.). It reads logs from both file system with the fluentd tail plugin and systemd journal with fluent-plugin-systemd.

Trace Collection

The deamonset runs OpenTelemetry Collector with the Splunk SAPM Exporter to collect traces and send them to Splunk SignalFx Microservices APM.

Metric Collection

OpenTelemetry Collector also collects kubernetes and host metrics using the following components enabled by default:

• Kubeletstats receiver to collect metrics from Kubelet API.
• K8s cluster receiver to collect metrics from Kubernetes API.
• Host metrics receiver to collect host metrics from kubernetes node.

The metrics are sent to Splunk SignalFx Infrastructure Monitoring.

Usage

Prerequisites

The following components required to use the helm chart:

• Helm client
• Kubernetes cluster

How to install

To install o11y-collector in k8s cluster at least three parameters must be provided:

• splunkRealm (default us0): SignalFx realm to send telemetry data to.
• splunkAccessToken: Your SignalFx org access token.
• clusterName: arbitrary value that will identify your kubernetes cluster in SignalFx environment.

The project is in active development state. There are no packages released yet. In order to install helm chart you need to clone the repo first and use it locally.

$ git clone [email protected]:signalfx/o11y-collector-for-kubernetes.git
$ cd ./o11y-collector-for-kubernetes
$ helm install my-o11y-collector --set="splunkRealm=us0,splunkAccessToken=xxxxxx,clusterName=my-cluster" ./helm-charts/o11y-collector-for-kubernetes

Instead of setting helm values as arguments a yaml file can be provided:

$ helm install my-o11y-collector --values my_values.yaml ./helm-charts/o11y-collector-for-kubernetes

How to uninstall

To uninstall/delete a deployment with name my-o11y-collector:

$ helm delete my-o11y-collector

The command removes all the Kubernetes components associated with the chart and deletes the release.

Configuration

The values.yaml lists all supported configurable parameters for this chart, along with detailed explanation. Read through it to understand how to configure this chart.

At the minimum you need to configure the following values.

clusterName: my-k8s-cluster
splunkAccessToken: xxxxxx
splunkRealm: us0

Kubernetes platform

Use platform parameter to provide information about underlying kubernetes platform. It'll allow the o11y collector to automatically scrape additional cloud metadata. Supported options:

• aws - Amazon EKS or self-managed k8s cluster in AWS environment.
• gcp - Google GKE or self-managed k8s cluster in GCP environment.
• default - default configuration for other platforms.

Disable particular types of telemetry

By default all telemetry data (metrics, traces and logs) is collected from the k8s cluster. It's possible to disable any kind of telemetry with the following parameters:

• metricsEnabled: false
• tracesEnabled: false
• logsEnabled: false

For exapmle, to install o11y collector only for logs:

$ helm install my-o11y-collector --set="splunkRealm=us0,splunkAccessToken=xxxxxx,clusterName=my-cluster,metricsEnabled=false,tracesEnabled=false" ./helm-charts/o11y-collector-for-kubernetes

License

See LICENSE.