diff --git a/sdk/monitor/azure-monitor-query/README.md b/sdk/monitor/azure-monitor-query/README.md index cf10b3b421ce0..4fc990b60fab2 100644 --- a/sdk/monitor/azure-monitor-query/README.md +++ b/sdk/monitor/azure-monitor-query/README.md @@ -1,31 +1,34 @@ -# Azure Monitor query client library for Java +# Azure Monitor Query client library for Java -Azure Monitor helps you maximize the availability and performance of your applications and services. It delivers a -comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments. +The Azure Monitor Query client library is used to execute read-only queries against [Azure Monitor][azure_monitor_overview]'s two data platforms: -All data collected by Azure Monitor fits into one of two fundamental types, metrics and logs. Metrics are numerical -values that describe some aspect of a system at a particular point in time. They are lightweight and capable of -supporting near real-time scenarios. Logs contain different kinds of data organized into records with different sets of -properties for each type. Telemetry such as events and traces are stored as logs in addition to performance data so that -it can all be combined for analysis. +- [Logs](https://docs.microsoft.com/azure/azure-monitor/logs/data-platform-logs) - Collects and organizes log and performance data from monitored resources. Data from different sources such as platform logs from Azure services, log and performance data from virtual machines agents, and usage and performance data from apps can be consolidated into a single [Azure Log Analytics workspace](https://docs.microsoft.com/azure/azure-monitor/logs/data-platform-logs#log-analytics-workspaces). The various data types can be analyzed together using the [Kusto Query Language][kusto_query_language]. +- [Metrics](https://docs.microsoft.com/azure/azure-monitor/essentials/data-platform-metrics) - Collects numeric data from monitored resources into a time series database. Metrics are numerical values that are collected at regular intervals and describe some aspect of a system at a particular time. Metrics are lightweight and capable of supporting near real-time scenarios, making them particularly useful for alerting and fast detection of issues. -This client library provides access to query metrics and logs collected by Azure Monitor. +**Resources:** -[Source code][source_code] | [Product Documentation][product_documentation] | [Samples][samples_readme] +- [Source code][source] +- [Package (Maven)][package] +- [Service documentation][azure_monitor_overview] +- [Samples][samples] +- [Change log][changelog] ## Getting started ### Prerequisites -- A [Java Development Kit (JDK)][jdk_link], version 8 or later. -- [Azure Subscription][azure_subscription] +- A [Java Development Kit (JDK)][jdk_link], version 8 or later +- An [Azure subscription][azure_subscription] +- To query Logs, you need an [Azure Log Analytics workspace][azure_monitor_create_using_portal]. +- To query Metrics, you need an Azure resource of any kind (Storage Account, Key Vault, Cosmos DB, etc.). -### Include the Package +### Install the package + +Install the Azure Monitor Query client library for Java by adding the following to your *pom.xml* file: [//]: # ({x-version-update-start;com.azure:azure-monitor-query;current}) ```xml - com.azure azure-monitor-query @@ -35,7 +38,11 @@ This client library provides access to query metrics and logs collected by Azure [//]: # ({x-version-update-end}) -### Create Logs query client +### Create the client + +An authenticated client is required to query Logs or Metrics. The library includes both synchronous and asynchronous forms of the clients. To authenticate, the following examples use `DefaultAzureCredentialBuilder` from the [com.azure:azure-identity](https://search.maven.org/artifact/com.azure/azure-identity) package. + +#### Synchronous clients ```java @@ -44,16 +51,6 @@ LogsQueryClient logsQueryClient = new LogsQueryClientBuilder() .buildClient(); ``` -### Create Logs query async client - - -```java -LogsQueryAsyncClient logsQueryAsyncClient = new LogsQueryClientBuilder() - .credential(new DefaultAzureCredentialBuilder().build()) - .buildAsyncClient(); -``` -### Create Metrics query client - ```java MetricsQueryClient metricsQueryClient = new MetricsQueryClientBuilder() @@ -61,7 +58,14 @@ MetricsQueryClient metricsQueryClient = new MetricsQueryClientBuilder() .buildClient(); ``` -### Create Metrics query async client +#### Asynchronous clients + + +```java +LogsQueryAsyncClient logsQueryAsyncClient = new LogsQueryClientBuilder() + .credential(new DefaultAzureCredentialBuilder().build()) + .buildAsyncClient(); +``` ```java @@ -72,81 +76,41 @@ MetricsQueryAsyncClient metricsQueryAsyncClient = new MetricsQueryClientBuilder( ## Key concepts -### Logs - -Azure Monitor Logs is a feature of Azure Monitor that collects and organizes log and performance data from monitored -resources. Data from different sources such as platform logs from Azure services, log and performance data from virtual -machines agents, and usage and performance data from applications can be consolidated into a single workspace so they -can be analyzed together using a sophisticated query language that's capable of quickly analyzing millions of records. -You may perform a simple query that just retrieves a specific set of records or perform sophisticated data analysis to -identify critical patterns in your monitoring data. - -#### Log Analytics workspaces - -Data collected by Azure Monitor Logs is stored in one or more Log Analytics workspaces. The workspace defines the -geographic location of the data, access rights defining which users can access data, and configuration settings such as -the pricing tier and data retention. - -You must create at least one workspace to use Azure Monitor Logs. A single workspace may be sufficient for all of your -monitoring data, or may choose to create multiple workspaces depending on your requirements. For example, you might have -one workspace for your production data and another for testing. - -#### Logs queries - -Data is retrieved from a Log Analytics workspace using a log query which is a read-only request to process data and -return results. Log queries are written -in [Kusto Query Language (KQL)](https://docs.microsoft.com/azure/data-explorer/kusto/query/), which is the same query -language used by Azure Data Explorer. You can write log queries in Log Analytics to interactively analyze their results, -use them in alert rules to be proactively notified of issues, or include their results in workbooks or dashboards. -Insights include prebuilt queries to support their views and workbooks. +### Logs query rate limits and throttling -#### Logs query rate limits and throttling +Each Azure Active Directory user is able to make up to 200 requests per 30 seconds, with no cap on the total calls per day. If requests are made at a rate higher than this, these requests will receive HTTP status code 429 (Too Many Requests) along with the `Retry-After: ` header. The header indicates the number of seconds until requests to this app are likely to be accepted. -Each AAD user is able to make up to 200 requests per 30 seconds, with no cap on the total calls per day.If requests are -being made at a rate higher than this, then these requests will receive HTTP status code 429 -(Too Many Requests) along with the Retry-After: header which indicates the number of seconds until -requests to this application are likely to be accepted. +In addition to call rate limits and daily quota caps, there are limits on queries themselves. Queries cannot: -As well as call rate limits and daily quota caps, there are also limits on queries themselves: +- Return more than 500,000 rows. +- Return more than 64,000,000 bytes (~61 MiB total data). +- Run longer than 10 minutes by default. See this for details. -- Queries cannot return more than 500,000 rows -- Queries cannot return more than 64,000,000 bytes (~61 MiB total data) -- Queries cannot run longer than 10 minutes by default. See this for details. +### Metrics data structure -### Metrics - -Azure Monitor Metrics is a feature of Azure Monitor that collects numeric data from monitored resources into a time -series database. Metrics are numerical values that are collected at regular intervals and describe some aspect of a -system at a particular time. Metrics in Azure Monitor are lightweight and capable of supporting near real-time scenarios -making them particularly useful for alerting and fast detection of issues. You can analyze them interactively with -metrics explorer, be proactively notified with an alert when a value crosses a threshold, or visualize them in a -workbook or dashboard. - -#### Metrics data structure - -Data collected by Azure Monitor Metrics is stored in a time-series database which is optimized for analyzing -time-stamped data. Each set of metric values is a time series with the following properties: +Each set of metric values is a time series with the following characteristics: - The time the value was collected -- The resource the value is associated with +- The resource associated with the value - A namespace that acts like a category for the metric - A metric name - The value itself -- Some metrics may have multiple dimensions as described in Multi-dimensional metrics. Custom metrics can have up to 10 - dimensions. +- Some metrics may have multiple dimensions as described in multi-dimensional metrics. Custom metrics can have up to 10 dimensions. ## Examples -* [Get logs for a query](#get-query "Get logs for a query") -* [Get logs for a query and read the response as a model type](#get-query-model "Get logs for a query and read the response as a model type") -* [Get logs for a batch for queries](#get-batch-query "Get logs for a batch of queries") -* [Get logs for a query with server timeout](#get-query-server-timeout "Get logs for a query with server timeout") -* [Get logs from multiple workspaces](#get-query-multiple-workspaces "Get logs from multiple workspaces") -* [Get metrics](#get-metrics "Get metrics") -* [Get average and count metrics ](#get-aggregation-metrics "Get average and count metrics") - +- [Logs query](#logs-query) + - [Map logs query results to a model](#map-logs-query-results-to-a-model) + - [Handle logs query response](#handle-logs-query-response) +- [Batch logs query](#batch-logs-query) +- [Advanced logs query scenarios](#advanced-logs-query-scenarios) + - [Set logs query timeout](#set-logs-query-timeout) + - [Query multiple workspaces](#query-multiple-workspaces) +- [Metrics query](#metrics-query) + - [Handle metrics query response](#handle-metrics-query-response) + - [Get average and count metrics](#get-average-and-count-metrics) -### Get logs for a query +### Logs query ```java @@ -162,7 +126,7 @@ for (LogsTableRow row : queryResults.getTable().getRows()) { } ``` -### Get logs for a query and read the response as a model type +#### Map logs query results to a model ```java @@ -179,6 +143,7 @@ public class CustomLogModel { } } ``` + ```java LogsQueryClient logsQueryClient = new LogsQueryClientBuilder() @@ -192,7 +157,29 @@ for (CustomLogModel customLogModel : customLogModels) { System.out.println(customLogModel.getOperationName() + " " + customLogModel.getResourceGroup()); } ``` -### Get logs for a batch of queries + +#### Handle logs query response + +The `query` API returns the `LogsQueryResult`, while the `queryBatch` API returns the `LogsBatchQueryResult`. Here's a hierarchy of the response: + +``` +LogsQueryResult / LogsBatchQueryResult +|---id (this exists in `LogsBatchQueryResult` object only) +|---status (this exists in `LogsBatchQueryResult` object only) +|---statistics +|---visualization +|---error +|---tables (list of `LogsTable` objects) + |---name + |---rows (list of `LogsTableRow` objects) + |--- rowIndex + |--- rowCells (list of `LogsTableCell` objects) + |---columns (list of `LogsTableColumn` objects) + |---name + |---type +``` + +### Batch logs query ```java @@ -224,7 +211,9 @@ if (query3Result.hasFailed()) { } ``` -### Get logs for a query with server timeout +### Advanced logs query scenarios + +#### Set logs query timeout ```java @@ -240,7 +229,9 @@ Response response = logsQueryClient.queryWithResponse("{workspa "{kusto-query}", new TimeInterval(Duration.ofDays(2)), options, Context.NONE); ``` -### Get logs from multiple workspaces +#### Query multiple workspaces + +To run the same query against multiple Log Analytics workspaces, use the `LogsQueryOptions.setAdditionalWorkspaces` method: When multiple workspaces are included in the query, the logs in the result table are not grouped according to the workspace from which it was retrieved. To identify the workspace of a row in the result table, you can inspect the @@ -260,30 +251,7 @@ Response response = logsQueryClient.queryWithResponse("{workspa LogsQueryResult result = response.getValue(); ``` -#### Response structure for Logs Query - -The `query` API returns the `LogsQueryResult` while the `queryBatch` API returns the `LogsBatchQueryResult`. - -Here is a hierarchy of the response: - -``` -LogsQueryResult / LogsBatchQueryResult -|---id (this exists in `LogsBatchQueryResult` object only) -|---status (this exists in `LogsBatchQueryResult` object only) -|---statistics -|---visualization -|---error -|---tables (list of `LogsTable` objects) - |---name - |---rows (list of `LogsTableRow` objects) - |--- rowIndex - |--- rowCells (list of `LogsTableCell` objects) - |---columns (list of `LogsTableColumn` objects) - |---name - |---type -``` - -### Get metrics +### Metrics query A resource ID, as denoted by the `{resource-id}` placeholder in the sample below, is required to query metrics. To find the resource ID: @@ -311,7 +279,34 @@ for (MetricResult metric : metricsQueryResult.getMetrics()) { } ``` -### Get average and count metrics +#### Handle metrics query response + +The metrics query API returns a `MetricsQueryResult` object. The `MetricsQueryResult` object contains properties such as a list of `MetricResult`-typed objects, `granularity`, `namespace`, and `timeInterval`. The `MetricResult` objects list can be accessed using the `metrics` param. Each `MetricResult` object in this list contains a list of `TimeSeriesElement` objects. Each `TimeSeriesElement` contains `data` and `metadata_values` properties. In visual form, the object hierarchy of the response resembles the following structure: + +``` +MetricsQueryResult +|---granularity +|---timeInterval +|---cost +|---namespace +|---resourceRegion +|---metrics (list of `MetricResult` objects) + |---id + |---type + |---name + |---unit + |---timeSeries (list of `TimeSeriesElement` objects) + |---metadata (dimensions) + |---metricValues (list of data points represented by `MetricValue` objects) + |--- timeStamp + |--- count + |--- average + |--- total + |--- maximum + |--- minimum +``` + +#### Get average and count metrics ```java @@ -338,37 +333,6 @@ for (MetricResult metric : metricsQueryResult.getMetrics()) { } } ``` -### Response structure for Metrics query - -The metrics query API returns a `MetricsQueryResult` object. The `MetricsQueryResult` object contains properties such as a -list of `MetricResult`-typed objects, `granularity`, `namespace`, and `timeInterval`. The `MetricResult` objects list -can be accessed using the `metrics` param. Each `MetricResult` object in this list contains a list of `TimeSeriesElement` objects. -Each `TimeSeriesElement` contains `data` and `metadata_values` properties. The `MetricResult` list is ordered -according to the list of metrics names provided in the request. In visual form, the object hierarchy of the -response resembles the following structure: - -``` -MetricsQueryResult -|---granularity -|---timeInterval -|---cost -|---namespace -|---resourceRegion -|---metrics (list of `MetricResult` objects) - |---id - |---type - |---name - |---unit - |---timeSeries (list of `TimeSeriesElement` objects) - |---metadata (dimensions) - |---metricValues (list of data points represented by `MetricValue` objects) - |--- timeStamp - |--- count - |--- average - |--- total - |--- maximum - |--- minimum -``` ## Troubleshooting @@ -378,7 +342,7 @@ You can set the `AZURE_LOG_LEVEL` environment variable to view logging statement example, setting `AZURE_LOG_LEVEL=2` would show all informational, warning, and error log messages. The log levels can be found here: [log levels][log_levels]. -### Default HTTP Client +### Default HTTP client All client libraries by default use the Netty HTTP client. Adding the above dependency will automatically configure the client library to use the Netty HTTP client. Configuring or changing the HTTP client is detailed in the @@ -393,44 +357,37 @@ reduce the dependency size, refer to the [performance tuning][performance_tuning ## Next steps -- Samples are explained in detail [here][samples_readme]. +To learn more about Azure Monitor, see the [Azure Monitor service documentation][azure_monitor_overview]. ## Contributing -This project welcomes contributions and suggestions. Most contributions require you to agree to -a [Contributor License Agreement (CLA)][cla] declaring that you have the right to, and actually do, grant us the rights -to use your contribution. +This project welcomes contributions and suggestions. Most contributions require you to agree to a [Contributor License Agreement (CLA)][cla] declaring that you have the right to, and actually do, grant us the rights to use your contribution. When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. -This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For more information see +This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For more information, see the [Code of Conduct FAQ][coc_faq] or contact [opencode@microsoft.com][coc_contact] with any additional questions or comments. -[source_code]: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/monitor/azure-monitor-query/src - -[samples_readme]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/monitor/azure-monitor-query/src/samples/java/README.md - +[azure_monitor_create_using_portal]: https://docs.microsoft.com/azure/azure-monitor/logs/quick-create-workspace +[azure_monitor_overview]: https://docs.microsoft.com/azure/azure-monitor/overview [azure_subscription]: https://azure.microsoft.com/free/java - +[changelog]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/monitor/azure-monitor-query/CHANGELOG.md [jdk_link]: https://docs.microsoft.com/java/azure/jdk/?view=azure-java-stable - -[product_documentation]: https://docs.microsoft.com/azure/azure-monitor/overview - +[kusto_query_language]: https://docs.microsoft.com/azure/data-explorer/kusto/query/ [log_levels]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/util/logging/ClientLogger.java - +[package]: https://search.maven.org/artifact/com.azure/azure-monitor-query +[samples]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/monitor/azure-monitor-query/src/samples/java/README.md +[source]: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/monitor/azure-monitor-query/src [performance_tuning]: https://github.com/Azure/azure-sdk-for-java/wiki/Performance-Tuning [cla]: https://cla.microsoft.com - [coc]: https://opensource.microsoft.com/codeofconduct/ - [coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/ - [coc_contact]: mailto:opencode@microsoft.com ![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-java%2Fsdk%2Fmonitor%2Fazure-monitor-query%2FREADME.png)