Skip to content

Latest commit

 

History

History
51 lines (37 loc) · 2.6 KB

Lab1.md

File metadata and controls

51 lines (37 loc) · 2.6 KB

Lab 1: Load Data into Azure SQL Data Warehouse using Azure Data Factory Pipelines

In this lab you will configure the Azure environment to allow relational data to be transferred from a SQL Server 2017 database to an Azure SQL Data Warehouse database using Azure Data Factory. The dataset you will use contains data about motor vehicle collisions that happened in New Your City from 2012 to 2019. You will use Power BI to visualise collision data loaded from Azure SQL Data Warehouse.

Lab Architecture

Lab Architecture

Step Description
1 Restore SQL Server backup from Azure Storage and Configure Azure Data Factory Self-Hosted Integration Runtime
2 Build an Azure Data Factory Pipeline to copy data from a SQL Server table
3 Use Azure Storage as a staging area for Polybase
4 Load data to an Azure SQL Data Warehouse table using Polybase
5 Visualize data from Azure SQL Data Warehouse using Power BI

IMPORTANT: Some of the Azure services provisioned by Lab0 require globally unique name and a “-suffix” has been added to their names to ensure this uniqueness. Please take note of the suffix generated as you will need it for the following resources:

Name Type
mdwcosmosdb-suffix Cosmos DB account
MDWDataFactory-suffix Data Factory (V2)
mdwdatalakesuffix Storage Account
MDWEventHubs-suffix Event Hubs Namespace
MDWKeyVault-suffix Key vault
mdwsqlvirtualserver-suffix SQL server
MDWStreamAnalytics-suffix Stream Analytics job

Connect to MDWDesktop

In this section you are going to establish a Remote Desktop Connection to MDWDesktop virtual machine.

IMPORTANT
Execute these steps on your host computer
  1. In the Azure Portal, navigate to the MDW-Lab resource group and click the MDWDesktop virtual machine.
  2. On the MDWDesktop blade, from the Overview menu, click the Connect button.

  1. On the Connect to virtual machine blade, click Download RDP File. This will download a .rdp file that you can use to establish a Remote Desktop Connection with the virtual machine.

Install required software onto MDWDesktop

In this section you are going to install Power BI Desktop and Azure Data Studio on MDWDesktop.

IMPORTANT
Execute these steps inside the MDWDesktop remote desktop connection