Skip to content

Latest commit

 

History

History

amazon_msk

README.md

Agent Check: Amazon MSK

Overview

Amazon Managed Streaming for Apache Kafka (MSK) is a fully managed service that makes it easy to build and run applications that use Apache Kafka to process streaming data.

You can collect metrics from this integration in two ways-with the Datadog Agent or with a Crawler that collects metrics from CloudWatch.

Setup

The Agent check monitors Amazon Managed Streaming for Apache Kafka (Amazon MSK) through the Datadog Agent.

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

This OpenMetrics-based integration has a latest mode (use_openmetrics: true) and a legacy mode (use_openmetrics: false). To get all the most up-to-date features, Datadog recommends enabling the latest mode. For more information, see Latest and Legacy Versioning For OpenMetrics-based Integrations.

Installation

  1. Create a client machine if one does not already exist.
  2. Ensure the client machine has been granted the permission policy arn:aws:iam::aws:policy/AmazonMSKReadOnlyAccess or equivalent credentials are available.
  3. Enable open monitoring with Prometheus on the MSK side to enable the JmxExporter and the NodeExporter.
  4. Install the Datadog Agent on the client machine just created.

Configuration

  1. Edit the amazon_msk.d/conf.yaml file, in the conf.d/ folder at the root of your Agent's configuration directory to start collecting your Amazon MSK performance data.

    Include custom tags that attach to every metric and service check provided by this integration.

    tags:
  - <KEY_1>:<VALUE_1>
  - <KEY_2>:<VALUE_2>

    See the sample amazon_msk.d/conf.yaml for all available configuration options for the latest mode. For the legacy mode of this integration, see the legacy example.

  2. Restart the Agent.

Validation

Run the Agent's status subcommand and look for amazon_msk under the Checks section.

Data Collected

Metrics

See metadata.csv for a list of metrics provided by this check.

Events

The Amazon MSK check does not include any events.

Service Checks

See service_checks.json for a list of service checks provided by this integration.

Troubleshooting

Need help? Contact Datadog support.

Further Reading

Additional helpful documentation, links, and articles: