Skip to content

Files

Latest commit

author
orspod
Fixed blocking and non-blocking changes
Nov 27, 2018
250428a · Nov 27, 2018

History

History
91 lines (62 loc) · 5.2 KB

connect-from-databricks.md

File metadata and controls

91 lines (62 loc) · 5.2 KB
title description services author ms.author ms.reviewer ms.service ms.topic ms.date
Connect to Azure Data Explorer from Azure Databricks using Python
This topic will show you how to use Python library in Azure Databricks to access data from Azure Data Explorer (ADX) using one of two authentication methods.
data-explorer
orspod
v-orspod
mblythe
data-explorer
conceptual
11/27/2018

Connect to Azure Data Explorer from Azure Databricks using Python

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. This article shows you how to use Python library in Azure Databricks to access data from Azure Data Explorer (ADX). There are several ways to authenticate with ADX including device login and Azure Active Directory (Azure AD) App.

Prerequisites

Create cluster

Install Python library on your Azure Databricks cluster

To install Python library on your Azure Databricks cluster:

  1. Go to your Azure Databricks workspace and Create a Library
  2. Upload a Python PyPI package or Python Egg
    • Upload, install, and attach the library to your Databricks cluster.
    • Enter the PyPi name: azure-kusto-data

Connect to ADX using device login

Import a Notebook using the Query-ADX-device-login notebook to connect to ADX using your credentials.

Connect to ADX using Azure AD App

  1. Create Azure AD App by Provisioning an AAD application.

  2. Grant access to your Azure AD App on your Azure Data Explorer database as follows:

    .set database <DB Name> users ('aadapp=<AAD App ID>;<AAD Tenant ID>') 'AAD App to connect Spark to ADX
    DB Name your database name
    AAD App ID your Azure AD App ID
    AAD Tenant ID your Azure AD Tenant ID

Find your Azure AD Tenant ID

To authenticate an application, Azure Data Explorer uses your Azure AD tenant ID. To find your tenant ID, use the following URL, substituting your domain for YourDomain.

https://login.windows.net/<YourDomain>/.well-known/openid-configuration/

For example, if your domain is contoso.com, the URL is: https://login.windows.net/contoso.com/.well-known/openid-configuration/. Click this URL to see the results; the first line is as follows.

"authorization_endpoint":"https://login.windows.net/6babcaad-604b-40ac-a9d7-9fd97c0b779f/oauth2/authorize"

Your tenant ID is 6babcaad-604b-40ac-a9d7-9fd97c0b779f.

Store and secure your Azure AD App ID and Key

Store and secure your Azure AD App ID and Key using Azure Databricks Secrets as follows:

  1. Set up the CLI

  2. Install the CLI

  3. Set up authentication.

  4. Configure the Secrets using the following sample commands:

    databricks secrets create-scope --scope adx

    databricks secrets put --scope adx --key myaadappid

    databricks secrets put --scope adx --key myaadappkey

    databricks secrets list --scope adx

Import a Notebook

Import a Notebook using the Query-ADX-AAD-App notebook to connect to ADX. Update the placeholder values with your cluster name, database name, and Azure AD Tenant ID.