Agent Check: Fly.io

This integration is in public beta. Use caution if enabling it on production workloads.

Overview

This check monitors Fly.io metrics through the Datadog Agent.

Setup

Follow the instructions below to install and configure this check for an Agent running on a Fly application.

Installation

The Fly.io check is included in the Datadog Agent package. We recommend running the Fly.io check on the Datadog Agent in a Fly.io application. The Agent collects Prometheus metrics as well as some additional data from the Machines API. Additionally, you can configure the Agent to receive traces and custom metrics from all of your Fly.io applications inside the organization.

Deploying the Agent as a Fly.io application

1. Create a new application in Fly.io with the image set as the Datadog Agent when launching, or provide the image in the fly.toml file:

   [build]
       image = 'gcr.io/datadoghq/agent:7'
   

2. Set a secret for your Datadog API key called DD_API_KEY, and optionally your site as DD_SITE.

3. In your app's directory, create a conf.yaml file for the Fly.io integration, configure the integration, and mount it in the Agent's conf.d/fly_io.d/ directory as conf.yaml:

   instances:
   - empty_default_hostname: true
     headers:
         Authorization: Bearer <YOUR_FLY_TOKEN>
     machines_api_endpoint: http://_api.internal:4280
     org_slug: <YOUR_ORG_SLUG>
   

4. Deploy your app.

Note: To collect traces and custom metrics from your applications, see Application traces.

Configuration

1. Edit the fly_io.d/conf.yaml file, located in the conf.d/ folder at the root of your Agent's configuration directory, to start collecting your Fly.io performance data. See the sample fly_io.d/conf.yaml for all available configuration options.

2. Restart the Agent.

Validation

Run the Agent's status subcommand and look for fly_io under the Checks section.

Data Collected

Metrics

See metadata.csv for a list of metrics provided by this integration.

Events

The Fly.io integration does not include any events.

Service Checks

The Fly.io integration does not include any service checks.

Application traces

Follow these steps to collect traces for an application in your Fly.io environment.

1. Instrument your application.

2. Deploy the Datadog Agent as a Fly.io application.

3. Set the required environment variables in the fly.toml or Dockerfile of your application and deploy the app.

   Set the following as an environment variable to submit metrics to the Datadog Agent application:

   [env]
       DD_AGENT_HOST="<YOUR_AGENT_APP_NAME>.internal"
   
   

   Set the following environment variable to ensure you report the same host for logs and metrics:

   DD_TRACE_REPORT_HOSTNAME="true"
   

   To utilize unified service tagging, set these environment variables:

   DD_SERVICE="APP_NAME"
   DD_ENV="ENV_NAME"
   DD_VERSION="VERSION"
   

   To correlate logs and traces, follow these steps and set this environment variable:

   DD_LOGS_INJECTION="true"
   

4. Set the following environment variables in your Datadog Agent application's fly.toml and deploy the app:

   [env]
       DD_APM_ENABLED = "true"
       DD_APM_NON_LOCAL_TRAFFIC = "true"
       DD_DOGSTATSD_NON_LOCAL_TRAFFIC = "true"
       DD_BIND_HOST = "fly-global-services"
   

Note: Ensure that the settings on your Fly.io instances do not publicly expose the ports for APM and DogStatsD, if enabled.

Log collection

Use the fly_logs_shipper to collect logs from your Fly.io applications.

1. Clone the logs shipper project.

2. Modify the vector-configs/vector.toml file to set the logs source as fly_io:

   [transforms.log_json]
   type = "remap"
   inputs = ["nats"]
   source  = '''
   . = parse_json!(.message)
   .ddsource = 'fly-io'
   .host = .fly.app.instance
   .env = <YOUR_ENV_NAME>
   '''
   

This configuration will parse basic fly-specific log attributes. To fully parse all log attributes, set ddsource to a known logs integration on a per-app basis using vector transforms.

3. Set secrets for NATS: ORG and ACCESS_TOKEN

4. Set secrets for Datadog: DATADOG_API_KEY and DATADOG_SITE

5. Deploy the logs shipper app.

Troubleshooting

Need help? Contact Datadog support.