Skip to content

Latest commit

 

History

History
222 lines (152 loc) · 10.1 KB

tutorial-deploy-stream-analytics.md

File metadata and controls

222 lines (152 loc) · 10.1 KB
title description services keywords author manager ms.author ms.date ms.topic ms.service
Deploy Azure Stream Analytics with Azure IoT Edge | Microsoft Docs
Deploy Azure Stream Analytics as a module to an edge device
iot-edge
msebolt
timlt
v-masebo
11/15/2017
article
iot-edge

Deploy Azure Stream Analytics as an IoT Edge module - preview

IoT devices can produce large quantities of data. Sometimes this data has to be analyzed or processed before reaching the cloud to reduce the size of uploaded data or to eliminate the round-trip latency of an actionable insight.

Azure Stream Analytics (ASA) provides a richly structured query syntax for data analysis both in the cloud and on IoT Edge devices. For more information about ASA on IoT Edge, see ASA documentation.

This tutorial walks you through the creation of an Azure Stream Analytics job, and its deployment on an IoT Edge device in order to process a local telemetry stream directly on the device, and generate alerts to drive immediate action on the device. There are two modules involved in this tutorial. A simulated temperature sensor module (tempSensor) that generates temperature data from 20 to 120 degrees, incremented by 1 every 5 seconds, and an ASA module that filters out temperatures greater than 100 degrees. The ASA module also resets the tempSensor when the 30 seconds average reaches 100.

You learn how to:

[!div class="checklist"]

  • Create an ASA job to process data on the Edge
  • Connect the new ASA job with other IoT Edge modules
  • Deploy the ASA job to an IoT Edge device

Prerequisites

  • An IoT Hub
  • The device that you created and configured in the quickstart or in Deploy Azure IoT Edge om a simulated device in Windows and Linux.
  • Docker on your IoT Edge device
  • Python 2.7.x on your IoT Edge device
    • Install Python 2.7 on Windows.
    • Most Linux distributions, including Ubuntu, already have Python 2.7 installed. Use the following command to make sure pip is installed: sudo apt-get install python-pip.

Note

Note your device connection string and IoT Edge device ID will be necessary for this tutorial.

IoT Edge takes advantage of pre-built Azure Service IoT Edge modules for quick deployment and Azure Stream Analytics (ASA) is one such module. You can create an ASA job from its portal, then come to IoT Hub portal to deploy it as an IoT Edge Module.

For more information on Azure Stream Analytics, see the Overview section of the Stream Analytics Documentation.

Create an ASA job

In this section, you create an Azure Stream Analytics job to take data from your IoT hub, query the sent telemetry data from your device, and forward the results to an Azure Storage Container (BLOB). For more information, see the Overview section of the Stream Analytics Documentation.

Note

An Azure Storage account is required to provide an endpoint to be used as an output in your ASA job. The example below uses the BLOB storage type. For more information, see the Blobs section of the Azure Storage Documentation.

  1. In the Azure portal, navigate to Create a resource -> Storage, click See all, and click Storage account - blob, file, table, queue.

  2. Enter a name for your storage account, and select the same location where your IoT Hub is stored. Click Create. Note the name for later.

    new storage account

  3. In the Azure portal, navigate to the storage account that you just created. Click Browse blobs under Blob Service.

  4. Create a new container for the ASA module to store data. Set the access level to Container. Click OK.

    storage settings

  5. In the Azure portal, navigate to Create a resource > Internet of Things and select Stream Analytics Job.

  6. Enter a name, choose Edge as the Hosting environment, and use the remaining default values. Click Create.

    [!NOTE] Currently, there are a limited number of regions that support ASA jobs on IoT Edge. Select one of the following as the location: Central US, West Central US, North Central US, East US, Canada East, Japan East, East Asia, North Europe, UK South, Australia East, Brazil South.

    ASA create

  7. Go into the created job, under Job Topology, select Inputs, click Add.

  8. Enter name temperature, choose Data stream as the source type, and use defaults for the other parameters. Click Create.

    ASA input

    [!NOTE] Additional inputs can include IoT Edge specific endpoints.

  9. Under Job Topology, select Outputs, click Add.

  10. Enter name alert and use defaults. Click Create.

    ASA output

  11. Under Job Topology, select Query, and enter the following:

    SELECT  
    'reset' AS command 
INTO 
   alert 
FROM 
   temperature TIMESTAMP BY timeCreated 
GROUP BY TumblingWindow(second,30) 
HAVING Avg(machine.temperature) > 100

Deploy the job

You are now ready to deploy the ASA job on your IoT Edge device.

  1. In the Azure portal, in your IoT Hub, navigate to IoT Edge (preview) and open your {deviceId}'s blade.

  2. Select Set modules, then select Import Azure Service IoT Edge Module.

  3. Select the subscription and the ASA Edge job that you created. Then select your storage account. Click Save.

    set module

  4. Click Add IoT Edge Module to add the temperature sensor module. Enter tempSensor for name, microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview for Image URL. Leave the other settings unchanged, and click Save.

    temperature module

  5. Copy the name of the ASA module. Click Next to configure routes.

  6. Copy the following to Routes. Replace {moduleName} with the module name you copied:

    {
    "routes": {                                                               
      "telemetryToCloud": "FROM /messages/modules/tempSensor/* INTO $upstream", 
      "alertsToCloud": "FROM /messages/modules/{moduleName}/* INTO $upstream", 
      "alertsToReset": "FROM /messages/modules/{moduleName}/* INTO BrokeredEndpoint(\"/modules/tempSensor/inputs/control\")", 
      "telemetryToAsa": "FROM /messages/modules/tempSensor/* INTO BrokeredEndpoint(\"/modules/{moduleName}/inputs/temperature\")" 
    }
}

  7. Click Next.

  8. In the Review Template step, click Submit.

  9. Return to the device details page and click Refresh. You should see the new {moduleName} module running along with the IoT Edge agent module and the IoT Edge hub.

    module output

View data

Now you can go to your IoT Edge device to check out the interaction between the ASA module and the tempSensor module.

  1. At a command prompt, configure the runtime with your IoT Edge device connection string:

    iotedgectl setup --connection-string "{device connection string}" --auto-cert-gen-force-no-passwords

  2. Run the command to start the runtime:

    iotedgectl start

  3. Run the command to see the modules running:

    docker ps

    docker output

  4. Run the command to see all system logs and metrics data. Use the module name from above:

    docker logs -f {moduleName}

    docker log

  5. In the Azure portal, in your Storage account, under Blob Service, click Browse blobs, select your container, and select newly created JSON file.

  6. Click Download and view the results.

Next steps

In this tutorial, you configured an Azure Storage container and a Streaming Analytics job to analyze data from your IoT Edge device. You then loaded a custom ASA module to move data from your device, through the stream, into a BLOB for download. You can continue on to other tutorials to further see how Azure IoT Edge can create solutions for your business.

[!div class="nextstepaction"] Deploy an Azure Machine Learning model as a module