title | description | services | author | manager | editor | ms.service | ms.workload | ms.tgt_pltfrm | ms.devlang | ms.topic | ms.date | ms.author |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Azure Data Factory service identity | Microsoft Docs |
Learn about data factory service identity in Azure Data Factory. |
data-factory |
linda33wj |
craigg |
data-factory |
data-services |
na |
na |
conceptual |
11/28/2018 |
jingwang |
This article helps you understand what is data factory service identity and how it works.
When creating a data factory, a service identity can be created along with factory creation. The service identity is a managed application registered to Azure Activity Directory, and represents this specific data factory.
Data factory service identity benefits the following features:
- Store credential in Azure Key Vault, in which case data factory service identity is used for Azure Key Vault authentication.
- Connectors including Azure Blob storage, Azure Data Lake Storage Gen1, Azure Data Lake Storage Gen2, Azure SQL Database, and Azure SQL Data Warehouse.
- Web activity.
Data factory service identity is generated as follows:
- When creating data factory through Azure portal or PowerShell, service identity will always be created automatically.
- When creating data factory through SDK, service identity will be created only if you specify "Identity = new FactoryIdentity()" in the factory object for creation. See example in .NET quickstart - create data factory.
- When creating data factory through REST API, service identity will be created only if you specify "identity" section in request body. See example in REST quickstart - create data factory.
If you find your data factory doesn't have a service identity associated following retrieve service identity instruction, you can explicitly generate one by updating the data factory with identity initiator programmatically:
- Generate service identity using PowerShell
- Generate service identity using REST API
- Generate service identity using an Azure Resource Manager template
- Generate service identity using SDK
Note
- Service identity cannot be modified. Updating a data factory which already have a service identity won't have any impact, the service identity is kept unchanged.
- If you update a data factory which already have a service identity without specifying "identity" parameter in the factory object or without specifying "identity" section in REST request body, you will get an error.
- When you delete a data factory, the associated service identity will be deleted along.
Call Set-AzureRmDataFactoryV2 command again, then you see "Identity" fields being newly generated:
PS C:\WINDOWS\system32> Set-AzureRmDataFactoryV2 -ResourceGroupName <resourceGroupName> -Name <dataFactoryName> -Location <region>
DataFactoryName : ADFV2DemoFactory
DataFactoryId : /subscriptions/<subsID>/resourceGroups/<resourceGroupName>/providers/Microsoft.DataFactory/factories/ADFV2DemoFactory
ResourceGroupName : <resourceGroupName>
Location : East US
Tags : {}
Identity : Microsoft.Azure.Management.DataFactory.Models.FactoryIdentity
ProvisioningState : Succeeded
Call below API with "identity" section in the request body:
PATCH https://management.azure.com/subscriptions/<subsID>/resourceGroups/<resourceGroupName>/providers/Microsoft.DataFactory/factories/<data factory name>?api-version=2017-09-01-preview
Request body: add "identity": { "type": "SystemAssigned" }.
{
"name": "<dataFactoryName>",
"location": "<region>",
"properties": {},
"identity": {
"type": "SystemAssigned"
}
}
Response: service identity is created automatically, and "identity" section is populated accordingly.
{
"name": "<dataFactoryName>",
"tags": {},
"properties": {
"provisioningState": "Succeeded",
"loggingStorageAccountKey": "**********",
"createTime": "2017-09-26T04:10:01.1135678Z",
"version": "2017-09-01-preview"
},
"identity": {
"type": "SystemAssigned",
"principalId": "765ad4ab-XXXX-XXXX-XXXX-51ed985819dc",
"tenantId": "72f988bf-XXXX-XXXX-XXXX-2d7cd011db47"
},
"id": "/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.DataFactory/factories/ADFV2DemoFactory",
"type": "Microsoft.DataFactory/factories",
"location": "<region>"
}
Template: add "identity": { "type": "SystemAssigned" }.
{
"contentVersion": "1.0.0.0",
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"resources": [{
"name": "<dataFactoryName>",
"apiVersion": "2018-06-01",
"type": "Microsoft.DataFactory/factories",
"location": "<region>",
"identity": {
"type": "SystemAssigned"
}
}]
}
Call the data factory create_or_update function with Identity=new FactoryIdentity(). Sample code using .NET:
Factory dataFactory = new Factory
{
Location = <region>,
Identity = new FactoryIdentity()
};
client.Factories.CreateOrUpdate(resourceGroup, dataFactoryName, dataFactory);
You can retrieve the service identity from Azure portal or programmatically. The following sections show some samples.
Tip
If you don't see the service identity, generate service identity by updating your factory.
You can find the service identity information from Azure portal -> your data factory -> Settings -> Properties:
- SERVICE IDENTITY ID
- SERVICE IDENTITY TENANT
- SERVICE IDENTITY APPLICATION ID > copy this value
The service identity principal ID and tenant ID will be returned when you get a specific data factory as follows:
PS C:\WINDOWS\system32> (Get-AzureRmDataFactoryV2 -ResourceGroupName <resourceGroupName> -Name <dataFactoryName>).Identity
PrincipalId TenantId
----------- --------
765ad4ab-XXXX-XXXX-XXXX-51ed985819dc 72f988bf-XXXX-XXXX-XXXX-2d7cd011db47
Copy the principal ID, then run below Azure Active Directory command with principal ID as parameter to get the ApplicationId, which you use to grant access:
PS C:\WINDOWS\system32> Get-AzureRmADServicePrincipal -ObjectId 765ad4ab-XXXX-XXXX-XXXX-51ed985819dc
ServicePrincipalNames : {76f668b3-XXXX-XXXX-XXXX-1b3348c75e02, https://identity.azure.net/P86P8g6nt1QxfPJx22om8MOooMf/Ag0Qf/nnREppHkU=}
ApplicationId : 76f668b3-XXXX-XXXX-XXXX-1b3348c75e02
DisplayName : ADFV2DemoFactory
Id : 765ad4ab-XXXX-XXXX-XXXX-51ed985819dc
Type : ServicePrincipal
See the following topics which introduce when and how to use data factory service identity:
- Store credential in Azure Key Vault
- Copy data from/to Azure Data Lake Store using managed identities for Azure resources authentication
See Managed Identities for Azure Resources Overview for more background on managed identities for Azure resources, which data factory service identity is based upon.