Agent Check: Amazon MSK

Overview

This check monitors Amazon Managed Streaming for Apache Kafka (Amazon MSK) through the Datadog Agent.

Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

Installation

1. Create a client machine if one does not already exist
2. Ensure the client machine has been granted the permission policy arn:aws:iam::aws:policy/AmazonMSKReadOnlyAccess or equivalent credentials are available
3. Enable open monitoring with Prometheus on the MSK side to enable the JmxExporter and the NodeExporter.
4. Install the Datadog Agent

Configuration

1. Edit the amazon_msk.d/conf.yaml file, in the conf.d/ folder at the root of your Agent's configuration directory to start collecting your Amazon MSK performance data. See the sample amazon_msk.d/conf.yaml for all available configuration options.

   Note: If you reuse the sample file, change the file name from conf.yaml.example to conf.yaml.

2. Restart the Agent.

Validation

Run the Agent's status subcommand and look for amazon_msk under the Checks section.

Data Collected

Metrics

See metadata.csv for a list of metrics provided by this check.

Service Checks

aws.msk.can_connect:
Returns CRITICAL if the Agent is unable to discover nodes of the MSK cluster. Otherwise, returns OK.

aws.msk.prometheus.health:
Returns CRITICAL if the check cannot access a metrics endpoint. Otherwise, returns OK.

When using the Agent 7+ implementation by setting use_openmetrics to true:

aws.msk.openmetrics.health:
Returns CRITICAL if the Agent is unable to connect to the OpenMetrics endpoint, otherwise returns OK.

Events

The Amazon MSK check does not include any events.

Troubleshooting

Need help? Contact Datadog support.