Skip to content

Latest commit

 

History

History
104 lines (85 loc) · 5.81 KB

batch-diagnostics.md

File metadata and controls

104 lines (85 loc) · 5.81 KB
title description services documentationcenter author manager editor ms.assetid ms.service ms.devlang ms.topic ms.tgt_pltfrm ms.workload ms.date ms.author
Azure Batch diagnostic logging | Microsoft Docs
Record and analyze diagnostic log events for Azure Batch account resources like pools and tasks.
batch
tamram
timlt
e14e611d-12cd-4671-91dc-bc506dc853e5
batch
na
article
multiple
big-compute
10/12/2016
tamram

Azure Batch diagnostic logging

As with many Azure services, the Batch service emits log events for certain resources during the lifetime of the resource. You can enable Azure Batch diagnostic logs to record events for resources like pools and tasks, and then use the logs for diagnostic evaluation and monitoring. Events like pool create, pool delete, task start, task complete, and others are included in Batch diagnostic logs.

Note

This article discusses logging events for Batch account resources themselves, not job and task output data. For details on storing the output data of your jobs and tasks, see Persist Azure Batch job and task output.

Prerequisites

Enable diagnostic logging

Diagnostic logging is not enabled by default for your Batch account. You must explicitly enable diagnostic logging for each Batch account you want to monitor:

How to enable collection of Diagnostic Logs

We recommend that you read the full Overview of Azure Diagnostic Logs article to gain an understanding of not only how to enable logging, but the log categories supported by the various Azure services. For example, Azure Batch currently supports one log category: Service Logs.

Service Logs

Azure Batch Service Logs contain events emitted by the Azure Batch service during the lifetime of a Batch resource like a pool or task. Each event emitted by Batch is stored in the specified Storage account in JSON format. For example, this is the body of a sample pool create event:

{
    "poolId": "myPool1",
    "displayName": "Production Pool",
    "vmSize": "Small",
    "cloudServiceConfiguration": {
        "osFamily": "4",
        "targetOsVersion": "*"
    },
    "networkConfiguration": {
        "subnetId": " "
    },
    "resizeTimeout": "300000",
    "targetDedicated": 2,
    "maxTasksPerNode": 1,
    "vmFillType": "Spread",
    "enableAutoscale": false,
    "enableInterNodeCommunication": false,
    "isAutoPool": false
}

Each event body resides in a .json file in the specified Azure Storage account. If you want to access the logs directly, you may wish to review the schema of Diagnostic Logs in the storage account.

Service Log events

The Batch service currently emits the following Service Log events. This list may not be exhaustive, since additional events may have been added since this article was last updated.

Service Log events
Pool create
Pool delete start
Pool delete complete
Pool resize start
Pool resize complete
Task start
Task complete
Task fail

Next steps

In addition to storing diagnostic log events in an Azure Storage account, you can also stream Batch Service Log events to an Azure Event Hub, and send them to Azure Log Analytics.

  • Stream Azure Diagnostic Logs to Event Hubs

    Stream Batch diagnostic events to the highly scalable data ingress service, Event Hubs. Event Hubs can ingest millions of events per second, which you can then transform and store using any real-time analytics provider.

  • Analyze Azure diagnostic logs using Log Analytics

    Send your diagnostic logs to Log Analytics where you can analyze them in the Operations Management Suite (OMS) portal, or export them for analysis in Power BI or Excel.