GitHub | DockerHub

Overview

collectd container to collect metrics and currently forwards them to Cloudwatch only. Currently you can reference any or all of these plugins(below) in the PLUGINS environment variable for they container to collect and forward those metrics to Cloudwatch.

• PLUGINS = "cpu,df,disk,elasticsearch,memory,swap"

How To Run container on EC2 Instance to export to Cloudwatch

EC2 instance need role with permissions to right to Cloudwatch ElasticSearch Cluster name is assumed to be elasticsearch below(change to yours, if different). This will send metrics for cpu, df, disk, elasticsearch, memroy and swap to cloudwatch.

sudo docker run -d \
  --name=collectd \
  --privileged \
  --net=host \
  -v /etc/hostname:/mnt/hostname:ro \
  -v /proc:/mnt/proc:ro \
  -v /:/hostfs:ro \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /etc:/mnt/etc:ro \
  -e ES_CLUSTER=elasticsearch \
  -e PLUGINS="cpu,df,disk,elasticsearch,memory,swap" \
  sungardas/collectd-docker:latest

or

sudo docker-compose up -d

How to view logs

docker logs collectd

How to get to bash shell in running container to troubleshoot

docker exec -it collectd bash

How To Build Container

docker build build -t sungardas/collectd-docker .

Environment Variables

• ES_CLUSTER = defaults to elasticsearch replace with your cluster name
• OUTPUTS = defaults to cloudwatch(only supported output currently)
• PLUGINS = comma separated string of plugins to use.
  • supported plugins:
    • cpu
    • df
    • disk
    • elasticsearch
    • memory
    • swap

Current whitelisted values per plugin.

To change any of the values, you will need to update the conf.tmpl files to allow for it.
Example: add java. copy an existing toml & tmpl, like plugin.cpu.conf.tmpl and plugin.cpu.conf.toml, and rename those copies to plugin.java.conf.tmpl and plugin.java.conf.toml. Change contents accordingly to fit what you are intending. Then add java into collectd.conf.tmpl and cloudwatch-whitelist.conf.tmpl. That is all that is needed to enable another plugin that is already included with collectd, that I have not included here yet.

cpu-.*

df-hostfs-data-df_complex-free
df-hostfs-data-df_complex-reserved
df-hostfs-data-df_complex-used
df-hostfs-data-percent_bytes-free
df-hostfs-data-percent_bytes-reserved
df-hostfs-data-percent_bytes-used
df-root-df_complex-free
df-root-df_complex-used
df-root-df_complex-reserved
df-root-percent_bytes-reserved
df-root-percent_bytes-free
df-root-percent_bytes-used
df-hostfs-df_complex-free
df-hostfs-df_complex-reserved
df-hostfs-df_complex-used

disk-xvd.*

elasticsearch-.*

memory--memory-used
memory--memory-buffered
memory--memory-cached
memory--memory-free

swap--swap-used
swap--swap-free
swap--swap-cached

Next Steps

• Remove templates from this container, possibly put them in there own repo, and pull it in at runtime. Or DynamoDB maybe.
• Currently template files for collectd plugins configs are templates, but no variables or logic in them yet(they are static), make them dynamic to be able to alter them without rebuilding the container.
• Add more plugins to the templates directory, and update whitelists accordingly.
• Create CodePipeline, and CodeBuild to autobuild and test the container.

License

Apache-2.0 ©

Sungard Availability Services | Labs

This project is maintained by the Labs team at Sungard Availability Services

GitHub: https://sungardas.github.io

Blog: http://blog.sungardas.com/CTOLabs/