Puppet module for configuring collectd and plugins.
The simplest use case is to use all of the configurations in the default collectd.conf file shipped with collectd. This can be done by simply including the class:
include collectd
Collectd is most useful when configured with customized plugins. This is accomplished by removing the default collectd.conf file and replacing it with a file that includes all alternative configurations. Configure a node with the following class declaration:
class { '::collectd':
purge => true,
recurse => true,
purge_config => true,
}
Set purge, recurse, and purge_config to true in order to override the default configurations shipped in collectd.conf and use custom configurations stored in conf.d. From here you can set up additional plugins as shown below.
Example of how to load plugins with no additional configuration:
collectd::plugin { 'battery': }
where 'battery' is the name of the plugin.
Parameters will vary widely between plugins. See the collectd documentation for each plugin for configurable attributes.
aggregation
(see collectd::plugin::aggregation below)amqp
(see collectd::plugin::amqp below)apache
(see collectd::plugin::apache below)bind
(see collectd::plugin::bind below)chain
(see collectd::plugin::chain below)conntrack
(see collectd::plugin::conntrack below)cpu
(see collectd::plugin::cpu below)cpufreq
(see collectd::plugin::cpufreq below)csv
(see collectd::plugin::csv below)curl
(see collectd::plugin::curl below)curl_json
(see collectd::plugin::curl_json below)df
(see collectd::plugin::df below)disk
(see collectd::plugin::disk below)entropy
(see collectd::plugin::entropy below)exec
(see collectd::plugin::exec below)filecount
(see collectd::plugin::filecount below)genericjmx
(see collectd::plugin::genericjmx below)interface
(see collectd::plugin::interface below)iptables
(see collectd::plugin::iptables below)irq
(see collectd::plugin::irq below)java
(see collectd::plugin::java below)load
(see collectd::plugin::load below)logfile
(see collectd::plugin::logfile below)libvirt
(see collectd::plugin::libvirt below)lvm
(see collectd::plugin::lvm below)memcached
(see collectd::plugin::memcached below )memory
(see collectd::plugin::memory below )mysql
(see collectd::plugin::mysql below)network
(see collectd::plugin::network below)nfs
(see collectd::plugin::nfs below)nginx
(see collectd::plugin::nginx below)ntpd
(see collectd::plugin::ntpd below)openvpn
(see collectd::plugin::openvpn below)perl
(see collectd::plugin::perl below)ping
(see collectd::plugin::ping below)postgresql
(see collectd::plugin::postgresql below)processes
(see collectd::plugin:processes below)protocols
(see collectd::plugin:protocols below)python
(see collectd::plugin::python below)redis
(see collectd::plugin::redis below)rrdcached
(see collectd::plugin::rrdcached below)rrdtool
(see collectd::plugin::rrdtool below)sensors
(see collectd::plugin::sensors below)snmp
(see collectd::plugin::snmp below)statsd
(see collectd::plugin::statsd below)swap
(see collectd::plugin::swap below)syslog
(see collectd::plugin::syslog below)tail
(see collectd::plugin::tail below)target_v5upgrade
(see collectd::plugin::target_v5upgrade below)tcpconns
(see collectd::plugin::tcpconns below)unixsock
(see collectd::plugin::unixsock below)uptime
(see collectd::plugin::uptime below)users
(see collectd::plugin::users below)varnish
(see collectd::plugin::varnish below)vmem
(see collectd::plugin::vmem below)write_graphite
(see collectd::plugin::write_graphite below)write_http
(see collectd::plugin::write_http below)write_network
(see collectd::plugin::write_network below)write_riemann
(see collectd::plugin::write_riemann below)zfs_arc
(see collectd::plugin::zfs_arc below)
####Class: collectd::plugin::aggregation
collectd::plugin::aggregation::aggregator {
cpu':
plugin => 'cpu',
type => 'cpu',
groupby => ["Host", "TypeInstance",],
calculateaverage => true,
}
You can as well configure this plugin with a parameterized class :
class { 'collectd::plugin::aggregation':
aggregators => {
cpu' => {
plugin => 'cpu',
type => 'cpu',
groupby => ["Host", "TypeInstance",],
calculateaverage => true,
},
},
}
####Class: collectd::plugin::amqp
class { 'collectd::plugin::amqp':
amqphost => '127.0.0.1',
amqpvhost => 'myvirtualhost',
graphiteprefix => 'collectdmetrics',
amqppersistent => true,
}
####Class: collectd::plugin::apache
class { 'collectd::plugin::apache':
instances => {
'apache80' => {
'url' => 'http://localhost/mod_status?auto', 'user' => 'collectd', 'password' => 'hoh2Coo6'
},
'lighttpd8080' => {
'url' => 'http://localhost:8080/mod_status?auto'
}
},
}
####Class: collectd::plugin::bind
class { 'collectd::plugin::bind':
url => 'http://localhost:8053/',
}
####Class: collectd::plugin::chain
class { 'collectd::plugin::chain':
chainname => "PostCache",
defaulttarget => "write",
rules => [
{
'match' => {
'type' => 'regex',
'matches' => {
'Plugin' => "^cpu$",
'PluginInstance' => "^[0-9]+$",
},
},
'targets' => [
{
'type' => "write",
'attributes' => {
"Plugin" => "aggregation",
},
},
{
'type' => "stop",
},
],
},
],
}
####Class: collectd::plugin::conntrack
class { 'collectd::plugin::conntrack':
}
####Class: collectd::plugin::cpu
reportbystate
available from collectd version >= 5.5reportbycpu
available from collectd version >= 5.5valuespercentage
available from collectd version >= 5.5
See collectd plugin_cpu documentation for more details.
class { 'collectd::plugin::cpu':
reportbystate => true,
reportbycpu => true,
valuespercentage => true,
}
####Class: collectd::plugin::cpufreq
class { 'collectd::plugin::cpufreq':
}
####Class: collectd::plugin::csv
class { 'collectd::plugin::csv':
datadir => '/etc/collectd/var/lib/collectd/csv',
storerates => false,
}
####Class: collectd::plugin::curl
collectd::plugin::curl::page {
'stock_quotes':
url => 'http://finance.google.com/finance?q=NYSE%3AAMD',
user => 'foo',
password => 'bar',
matches => [
{
'dstype' => 'GaugeAverage',
'instance' => 'AMD',
'regex' => ']*> *([0-9]*\\.[0-9]+) *',
'type' => 'stock_value',
}],
}
You can as well configure this plugin with a parameterized class :
class { 'collectd::plugin::curl':
pages => {
'stock_GM' => {
url => 'http://finance.google.com/finance?q=NYSE%3AGM',
user => 'foo',
password => 'bar',
matches => [
{
'dstype' => 'GaugeAverage',
'instance' => 'AMD',
'regex' => ']*> *([0-9]*\\.[0-9]+) *',
'type' => 'stock_value',
},
],
},
},
}
####Class: collectd::plugin::curl_json
collectd::plugin::curl_json {
'rabbitmq_overview':
url => 'http://localhost:55672/api/overview',
instance => 'rabbitmq_overview',
keys => {
'message_stats/publish' => {'type' => 'gauge'},
}
}
####Class: collectd::plugin::df
class { 'collectd::plugin::df':
mountpoints => ['/u'],
fstypes => ['nfs','tmpfs','autofs','gpfs','proc','devpts'],
ignoreselected => true,
}
####Class: collectd::plugin::disk
class { 'collectd::plugin::disk':
disks => ['/^dm/'],
ignoreselected => true,
udevnameattr => 'DM_NAME',
}
####Class: collectd::plugin::entropy
class { 'collectd::plugin::entropy':
}
####Class: collectd::plugin::exec
collectd::plugin::exec::cmd {
'dummy':
user => nobody,
group => nogroup,
exec => ["/bin/echo", "PUTVAL myhost/foo/gauge-flat N:1"],
}
####Class: collectd::plugin::filecount
collectd::plugin::filecount::directory {'foodir':
path => '/path/to/dir',
pattern => '*.conf',
mtime => '-5m',
recursive => true,
includehidden => false
}
You can also configure this plugin with a parameterized class:
class { 'collectd::plugin::filecount':
directories => {
'foodir' => {
'path' => '/path/to/dir',
'pattern' => '*.conf',
'mtime' => '-5m',
'recursive' => true,
'includehidden' => false
},
},
}
For backwards compatibility:
class { 'collectd::plugin::filecount':
directories => {
'active' => '/var/spool/postfix/active',
'incoming' => '/var/spool/postfix/incoming'
},
}
####Class: collectd::plugin::genericjmx
include collectd::plugin::genericjmx
collectd::plugin::genericjmx::mbean {
'garbage_collector':
object_name => 'java.lang:type=GarbageCollector,*',
instance_prefix => 'gc-',
instance_from => 'name',
values => [
{
type => 'invocations',
table => false,
attribute => 'CollectionCount',
},
{
type => 'total_time_in_ms',
instance_prefix => 'collection_time',
table => false,
attribute => 'CollectionTime',
},
];
}
collectd::plugin::genericjmx::connection {
'java_app':
host => $fqdn,
service_url => 'service:jmx:rmi:///jndi/rmi://localhost:3637/jmxrmi',
collect => [ 'memory-heap', 'memory-nonheap','garbage_collector' ],
}
####Class: collectd::plugin::interface
class { 'collectd::plugin::interface':
interfaces => ['lo'],
ignoreselected => true
}
####Class: collectd::plugin::irq
class { 'collectd::plugin::irq':
irqs => ['7', '23'],
ignoreselected => true,
}
####Class: collectd::plugin::iptables
class { 'collectd::plugin::iptables':
chains => {
'nat' => 'In_SSH',
'filter' => 'HTTP'
},
}
####Class: collectd::plugin::java
class { 'collectd::plugin::java': }
####Class: collectd::plugin::load
class { 'collectd::plugin::load':
}
####Class: collectd::plugin::logfile
class { 'collectd::plugin::logfile':
log_level => 'warning',
log_file => '/var/log/collected.log'
}
####Class: collectd::plugin::libvirt
The interface_format parameter was introduced in collectd 5.0 and will therefore be ignored (with a warning) when specified with older versions.
class { 'collectd::plugin::libvirt':
connection => 'qemu:///system',
interface_format => 'address'
}
####Class: collectd::plugin::lvm
class { 'collectd::plugin::lvm': }
####Class: collectd::plugin::memcached
class { 'collectd::plugin::memcached':
host => '192.168.122.1',
port => 11211,
}
####Class: collectd::plugin::memory
class { 'collectd::plugin::memory':
}
####Class: collectd::plugin::mysql
collectd::plugin::mysql::database { 'betadase':
host => 'localhost',
username => 'stahmna',
password => 'secret',
port => '3306',
masterstats => true,
}
####Class: collectd::plugin::network
collectd::plugin::network::server{'hostname':
port => 25826,
}
collectd::plugin::network::listener{'hostname':
port => 25826,
}
You can as well configure this plugin with a parameterized class :
class { 'collectd::plugin::network':
timetolive => '70',
maxpacketsize => '42',
forward => false,
reportstats => true,
servers => { 'hostname' => {
'port' => '25826',
'interface' => 'eth0',
'securitylevel' => '',
'username' => 'foo',
'password' => 'bar',},
},
listeners => { 'hostname' => {
'port' => '25826',
'interface' => 'eth0',
'securitylevel' => '',
'authfile' => '/etc/collectd/passwd',},
},
}
####Class: collectd::plugin::nfs
class { 'collectd::plugin::nfs':
}
####Class: collectd::plugin::nginx
class { 'collectd::plugin::nginx':
url => 'https://localhost:8433',
user => 'stats',
password => 'uleePi4A',
}
####Class: collectd::plugin::ntpd
class { 'collectd::plugin::ntpd':
host => 'localhost',
port => 123,
reverselookups => false,
includeunitid => false,
}
####Class: collectd::plugin::openvpn
statusfile
(String or Array) Status file(s) to collect data from. (Default/etc/openvpn/openvpn-status.log
)improvednamingschema
(Bool) When enabled, the filename of the status file will be used as plugin instance and the client's "common name" will be used as type instance. This is required when reading multiple status files. (Default:false
)collectcompression
Sets whether or not statistics about the compression used by OpenVPN should be collected. This information is only available in single mode. (Defaulttrue
)collectindividualusers
Sets whether or not traffic information is collected for each connected client individually. If set to false, currently no traffic data is collected at all because aggregating this data in a save manner is tricky. (Defaulttrue
)collectusercount
When enabled, the number of currently connected clients or users is collected. This is especially interesting when CollectIndividualUsers is disabled, but can be configured independently from that option. (Defaultfalse
)
Watch multiple statusfile
s:
class { 'collectd::plugin::openvpn':
statusfile => [ '/etc/openvpn/openvpn-status-tcp.log', '/etc/openvpn/openvpn-status-udp.log' ],
collectindividualusers => false,
collectusercount => true,
}
Watch the single default statusfile
:
class { 'collectd::plugin::openvpn':
collectindividualusers => false,
collectusercount => true,
}
####Class: collectd::plugin::perl
This class has no parameters and will load the actual perl plugin.
It will be automatically included if any perl::plugin
is defined.
#####Example:
include collectd::plugin::perl
####Define: collectd::plugin::perl::plugin
This define will load a new perl plugin.
#####Parameters:
module
(String): name of perl module to load (mandatory)enable_debugger
(False or String): whether to load the perl debugger. See collectd-perl manpage for more details.include_dir
(String or Array): directories to add to @INCprovider
("package"
,"cpan"
,"file"
orfalse
): method to get the plugin codesource
(String): this parameter is consumed by the provider to infer the source of the plugin codedestination (String or false): path to plugin code if
provideris
file`. Ignored otherwise.order
(String containing numbers): order in which the plugin should be loaded. Defaults to"00"
config
(Hash): plugin configuration in form of a hash. This will be converted to a suitable structure understood by liboconfig which is the collectd configuration parser. Defaults to{}
#####Examples:
######Using a preinstalled plugin:
collectd::plugin::perl::plugin { 'foo':
module => 'Collectd::Plugins::Foo',
enable_debugger => "",
include_dir => '/usr/lib/collectd/perl5/lib',
}
######Using a plugin from a file from source:
collectd::plugin::perl::plugin { 'baz':
module => 'Collectd::Plugins::Baz',
provider => 'file',
source => 'puppet:///modules/myorg/baz_collectd.pm',
destination => '/path/to/my/perl5/modules'
}
######Using a plugin from cpan (requires the puppet cpan module):
collectd::plugin::perl::plugin {
'openafs_vos':
module => 'Collectd::Plugins::OpenAFS::VOS',
provider => 'cpan',
source => 'Collectd::Plugins::OpenAFS',
config => {'VosBin' => '/usr/afsws/etc/vos'},
}
######Using a plugin from package source:
collectd::plugin::perl::plugin {
'bar':
module => 'Collectd::Plugins::Bar',
provider => 'package',
source => 'perl-Collectd-Plugins-Bar',
config => {'foo' => 'bar'},
}
####Class: collectd::plugin::ping
collectd::plugin::ping {
'example':
hosts => ['example.com'],
}
####Class: collectd::plugin::postgresql
collectd::plugin::postgresql::database{'monitoring_node1':
name => 'monitoring',
port => '5433',
instance => 'node1',
host => 'localhost',
user => 'collectd',
password => 'collectd',
query => 'log_delay',
}
collectd::plugin::postgresql::query{'log_delay':
statement => 'SELECT * FROM log_delay_repli;',
results => [{
type => 'gauge',
instanceprefix => 'log_delay',
instancesfrom => 'inet_server_port',
valuesfrom => 'log_delay',
}],
}
collectd::plugin::postgresql::writer{'sqlstore':
statement => 'SELECT collectd_insert($1, $2, $3, $4, $5, $6, $7, $8, $9);',
storerates => 'true',
}
You can as well configure this plugin as a parameterized class :
class { 'collectd::plugin::postgresql':
databases => {
'postgres' => {
'host' => '/var/run/postgresql/',
'user' => 'postgres',
'password' => 'postgres',
'sslmode' => 'disable',
'query' => [ 'query_plans', 'queries', 'table_states', 'disk_io' ],
},
'devdb' => {
'host' => 'host.example.com',
'port' => '5432',
'user' => 'postgres',
'password' => 'secret',
'sslmode' => 'prefer',
}
}
}
####Class: collectd::plugin::processes
You can either specify processes / process matches at once:
class { 'collectd::plugin::processes':
processes => ['process1', 'process2'],
process_matches => [
{ name => 'process-all', regex => 'process.*' }
],
}
####Class: collectd::plugin::protocols
values
is an array ofProtocol
names,Protocol:ValueName
pairs, or a regex- see
/proc/net/netstat
and/proc/net/snmp
for a list ofProtocol
targets
See [collectd.conf documentation] (https://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_protocols) for details
class { 'collectd::plugin::protocols':
values => ['/^Tcp:*/', '/^Udp:*/', 'Icmp:InErrors' ],
ignoreselected => false,
}
Or define single processes / process matches:
collectd::plugin::processes::process { 'collectd' : }
collectd::plugin::processes::processmatch { 'elasticsearch' :
regex => '.*java.*org.elasticsearch.bootstrap.Elasticsearch'
}
####Class: collectd::plugin::python
modulepaths
is an array of paths where will be Collectd looking for Python modules, Puppet will ensure that each of specified directories exists and it is owned byroot
(andchmod 0750
). If you don't specify anymodulepaths
a default value for given distribution will be used.modules
a Hash containing configuration of Python modules, where the key is the module nameglobals
Unlike most other plugins, this one should setGlobals true
. This will cause collectd to export the name of all objects in the Python interpreter for all plugins to see. If you don't do this or your platform does not support it, the embedded interpreter will start anyway but you won't be able to load certain Python modules, e.g. "time".interactive
whentrue
it will launch an interactive Python interpreter that reads from and writes to the terminal (default:false
)logtraces
if a Python script throws an exception it will be logged by collectd with the name of the exception and the message (default:false
)
See collectd-python documentation for more details.
NOTE: Since v3.4.0
the syntax of this plugin has changed. Make sure to update your existing configuration. Now you can specify multiple Python modules at once:
class { 'collectd::plugin::python':
modulepaths => ['/usr/share/collectd/python'],
modules => {
'elasticsearch': {
'script_source' => 'puppet:///modules/myorg/elasticsearch_collectd_python.py',
'config' => {'Cluster' => 'elasticsearch'},
},
'another-module': {
'config' => {'Verbose' => 'true'},
}
}
logtraces => true,
interactive => false
}
When script_source
provided, a file called {module}.py
will be created in $modulepath/$module.py
.
Or define single module:
collectd::plugin::python::module {'zk-collectd':
script_source => 'puppet:///modules/myorg/zk-collectd.py',
config => {
'Hosts' => "localhost:2181"
}
}
Each plugin might use different modulepath
, however make sure that all paths are included in collectd::plugin::python
variable modulepaths
. If no modulepath
is specified, OS default will be used.
collectd::plugin::python::module {'my-module':
modulepath => '/var/share/collectd',
script_source => 'puppet:///modules/myorg/my-module.py',
config => {
'Key' => "value"
}
}
####Class: collectd::plugin::redis
class { 'collectd::plugin::redis':
nodes => {
'node1' => {
'host' => 'localhost',
},
'node2' => {
'host' => 'node2.example.com',
'port' => '6380',
'timeout' => 3000,
}
}
}
####Class: collectd::plugin::rrdcached
class { 'collectd::plugin::rrdcached':
daemonaddress => 'unix:/var/run/rrdcached.sock',
datadir => '/var/lib/rrdcached/db/collectd',
}
####Class: collectd::plugin::rrdtool
class { 'collectd::plugin::rrdtool':
datadir => '/var/lib/collectd/rrd',
createfilesasync => false,
rrarows => 1200,
rratimespan => [3600, 86400, 604800, 2678400, 31622400],
xff => 0.1,
cacheflush => 900,
cachetimeout => 120,
writespersecond => 50
}
####Class: collectd::plugin::sensors
class {'collectd::plugin::sensors':
ignoreselected => false,
}
####Class: collectd::plugin::snmp
class {'collectd::plugin::snmp':
data => {
amavis_incoming_messages => {
'Type' => 'counter',
'Table' => false,
'Instance' => 'amavis.inMsgs',
'Values' => ['AMAVIS-MIB::inMsgs.0']
}
},
hosts => {
debianvm => {
'Address' => '127.0.0.1',
'Version' => 2,
'Community' => 'public',
'Collect' => ['amavis_incoming_messages'],
'Interval' => 10
}
},
}
####Class: collectd::plugin::statsd
class { 'collectd::plugin::statsd':
host => '0.0.0.0',
port => 8125,
deletecounters => false,
deletetimers => false,
deletegauges => false,
deletesets => false,
timerpercentile => 50,
}
####Class: collectd::plugin::swap
class { 'collectd::plugin::swap':
reportbydevice => false,
reportbytes => true
}
####Class: collectd::plugin::syslog
class { 'collectd::plugin::syslog':
log_level => 'warning'
}
####Class: collectd::plugin::target_v5upgrade
class { 'collectd::plugin::target_v5upgrade':
}
####Class: collectd::plugin::tcpconns
class { 'collectd::plugin::tcpconns':
localports => ['25', '12026'],
remoteports => ['25'],
listening => false,
}
####Class: collectd::plugin::tail
collectd::plugin::tail::file { 'exim-log':
filename => '/var/log/exim4/mainlog',
instance => 'exim',
matches => [
{
regex => 'S=([1-9][0-9]*)',
dstype => 'CounterAdd',
type => 'ipt_bytes',
instance => 'total',
},
{
regex => '\\<R=local_user\\>',
dstype => 'CounterInc',
type => 'counter',
instance => 'local_user',
}
]
}
####Class: collectd::plugin::unixsock
class {'collectd::plugin::unixsock':
socketfile => '/var/run/collectd-sock',
socketgroup => 'nagios',
socketperms => '0770',
deletesocket => false,
}
####Class: collectd::plugin::uptime
class {'collectd::plugin::uptime':
}
####Class: collectd::plugin::users
class {'collectd::plugin::users':
}
####Class: collectd::plugin::varnish
class { 'collectd::plugin::varnish':
instances => {
'instanceName' => {
'CollectCache' => 'true',
'CollectBackend' => 'true',
'CollectConnections' => 'true',
'CollectSHM' => 'true',
'CollectESI' => 'false',
'CollectFetch' => 'true',
'CollectHCB' => 'false',
'CollectTotals' => 'true',
'CollectWorkers' => 'true',
}
},
}
####Class: collectd::plugin::vmem
class { 'collectd::plugin::vmem':
verbose => true,
}
####Class: collectd::plugin::write_graphite
The write_graphite
plugin writes data to Graphite, an open-source metrics storage and graphing project.
collectd::plugin::write_graphite::carbon {'my_graphite':
graphitehost => 'graphite.example.org',
graphiteport => 2003,
graphiteprefix => '',
protocol => 'tcp'
}
You can define multiple Graphite backends where will be metrics send. Each backend should have unique title:
collectd::plugin::write_graphite::carbon {'secondary_graphite':
graphitehost => 'graphite.example.org',
graphiteport => 2004,
graphiteprefix => '',
protocol => 'udp',
escapecharacter => '_',
alwaysappendds => true,
storerates => true,
separateinstances => false,
logsenderrors => true
}
####Class: collectd::plugin::write_http
class { 'collectd::plugin::write_http':
urls => {
'collect1.example.org' => { 'format' => 'JSON' },
'collect2.example.org' => {},
}
}
####Class: collectd::plugin::write_network
Deprecated
class { 'collectd::plugin::write_network':
servers => {
'collect1.example.org' => { 'serverport' => '25826' },
'collect2.example.org' => { 'serverport' => '25826' }
}
}
####Class: collectd::plugin::write_riemann
class { 'collectd::plugin::write_riemann':
riemann_host => 'riemann.example.org',
riemann_port => 5555,
}
####Class: collectd::plugin::zfs_arc
class { 'collectd::plugin::zfs_arc':
}
##Limitations
See metadata.json for supported platforms
##Known issues
Some plugins will need two runs of Puppet to fully generate the configuration for collectd. See this issue.
##Development
This project contains tests for rspec-puppet.
Quickstart:
gem install bundler
bundle install
bundle exec rake lint
bundle exec rake validate
bundle exec rake spec SPEC_OPTS='--format documentation'
Some plugins or some options in plugins are only available for recent versions of collectd.
This module shall not use unsupported configuration directives. Look at templates/loadplugin.conf.erb for a hands-on example.
Please make use of the search by branch/tags on the collectd github to see when a function has been first released.
Reading the collectd.conf.pod file is good, validating the presence of the code in the .c files is even better.