| title | description | services | author | ms.service | ms.topic | ms.date | ms.author | ms.custom |
|---|---|---|---|---|---|---|---|---|
include file |
include file |
data-factory |
linda33wj |
data-factory |
include |
06/20/2018 |
jingwang |
include file |
Data factory is a multi-tenant service that has the following default limits in place to make sure customer subscriptions are protected from each other's workloads. Many of the limits can be easily raised for your subscription up to the maximum limit by contacting support.
| Resource | Default Limit | Maximum Limit |
|---|---|---|
| Data factories in an Azure subscription | 50 | Contact support |
| Total number of entities (Pipeline, Datasets, Triggers, Linked Services, Integration runtimes) within a data factory | 5000 | Contact support |
| Total CPU cores for Azure-SSIS Integration Runtime(s) under one subscription | 128 | Contact support |
| Concurrent pipeline runs per pipeline | 100 | Contact support |
| Concurrent pipeline runs per data factory | 10,000 | Contact support |
| Max activities per pipeline (includes inner activities for containers) | 40 | 40 |
| Max parameters per pipeline | 50 | 50 |
| ForEach items | 100,000 | 100,000 |
| ForEach parallelism | 20 | 50 |
| Characters per expression | 8,192 | 8,192 |
| Minimum Tumbling Window Trigger interval | 15 min | 15 min |
| Max Timeout for pipeline activity runs | 7 days | 7 days |
| Bytes per object for pipeline objects 1 | 200 KB | 200 KB |
| Bytes per object for dataset and linked service objects 1 | 100 KB | 2000 KB |
| Data integration units per copy activity run 3 | 256 | Contact support |
| Write API calls | 2500/hr This limit is imposed by Azure Resource Manager, not Azure Data Factory. |
Contact support. |
| Read API calls | 12,500/hr This limit is imposed by Azure Resource Manager, not Azure Data Factory. |
Contact support |
| Monitoring queries per minute | 1000 | Contact support |
| Entity CRUD operations per minute | 50 | Contact support |
| Resource | Default Limit | Maximum Limit |
|---|---|---|
| Data factories in an Azure subscription | 50 | Contact support |
| Pipelines within a data factory | 2500 | Contact support |
| Datasets within a data factory | 5000 | Contact support |
| Concurrent slices per dataset | 10 | 10 |
| Bytes per object for pipeline objects 1 | 200 KB | 200 KB |
| Bytes per object for dataset and linked service objects 1 | 100 KB | 2000 KB |
| HDInsight on-demand cluster cores within a subscription 2 | 60 | Contact support |
| Cloud data movement units per copy activity run 3 | 32 | Contact support |
| Retry count for pipeline activity runs | 1000 | MaxInt (32 bit) |
1 Pipeline, dataset, and linked service objects represent a logical grouping of your workload. Limits for these objects do not relate to amount of data you can move and process with the Azure Data Factory service. Data factory is designed to scale to handle petabytes of data.
2 On-demand HDInsight cores are allocated out of the subscription that contains the data factory. As a result, the above limit is the Data Factory enforced core limit for on-demand HDInsight cores and is different from the core limit associated with your Azure subscription.
3 Data Integration Unit (DIU) for v2 or Cloud Data Movement Unit (DMU) for v1 is being used in a cloud-to-cloud copy operation. It is a measure that represents the power (a combination of CPU, memory, and network resource allocation) of a single unit in Data Factory. You can achieve higher copy throughput by using more DMUs for some scenarios. Refer to Data integration units (V2) and Cloud data movement units (V1) section on details, and Azure Data Factory pricing page for billing implication.
4 The Integration Runtime (IR) is the compute infrastructure used by Azure Data Factory to provide the following data integration capabilities across different network environments: data movement, dispatching activities to compute services, execution of SSIS packages. For more information, see Integration Runtime overview.
| Resource | Default lower limit | Minimum limit |
|---|---|---|
| Scheduling interval | 15 minutes | 15 minutes |
| Interval between retry attempts | 1 second | 1 second |
| Retry timeout value | 1 second | 1 second |
Azure Resource Manager has limits for API calls. You can make API calls at a rate within the Azure Resource Manager API limits.