Skip to content

Files

Latest commit

Sep 27, 2018
9ce8393 · Sep 27, 2018

History

History
293 lines (207 loc) · 14 KB

logic-apps-batch-process-send-receive-messages.md

File metadata and controls

293 lines (207 loc) · 14 KB
title description services ms.service ms.suite author ms.author ms.reviewer ms.topic ms.date
Batch process messages as a group or collection - Azure Logic Apps | Microsoft Docs
Send and receive messages as batches in Azure Logic Apps
logic-apps
logic-apps
integration
divyaswarnkar
divswa
estfan, jonfan, LADocs
article
08/19/2018

Send, receive, and batch process messages in Azure Logic Apps

To send and process messages together in a specific way as groups, you can create a batching solution that collects messages into a batch until your specified criteria are met for releasing and processing the batched messages. Batching can reduce how often your logic app processes messages. This article shows how to build a batching solution by creating two logic apps within the same Azure subscription, Azure region, and following this specific order:

  • The "batch receiver" logic app, which accepts and collects messages into a batch until your specified criteria is met for releasing and processing those messages.

    Make sure you first create the batch receiver so you can later select the batch destination when you create the batch sender.

  • One or more "batch sender" logic apps, which send the messages to the previously created batch receiver.

    You can also specify a unique key, such as a customer number, that partitions or divides the target batch into logical subsets based on that key. That way, the receiver app can collect all items with the same key and process them together.

Make sure your batch receiver and batch sender share the same Azure subscription and Azure region. If they don't, you can't select the batch receiver when you create the batch sender because they're not visible to each other.

Prerequisites

To follow this example, you need these items:

Create batch receiver

Before you can send messages to a batch, that batch must first exist as the destination where you send those messages. So first, you must create the "batch receiver" logic app, which starts with the Batch trigger. That way, when you create the "batch sender" logic app, you can select the batch receiver logic app. The batch receiver continues collecting messages until your specified criteria is met for releasing and processing those messages. While batch receivers don't need to know anything about batch senders, batch senders must know the destination where they send the messages.

  1. In the Azure portal or Visual Studio, create a logic app with this name: "BatchReceiver"

  2. In Logic Apps Designer, add the Batch trigger, which starts your logic app workflow. In the search box, enter "batch" as your filter. Select this trigger: Batch messages

    Add "Batch messages" trigger

  3. Set the batch receiver properties:

    Property Description
    Batch Mode - Inline: For defining release criteria inside the batch trigger
    - Integration Account: For defining multiple release criteria configurations through an integration account. With an integration account, you can maintain these configurations all in one place rather than in separate logic apps.
    Batch Name The name for your batch, which is "TestBatch" in this example, and applies only to Inline batch mode
    Release Criteria Applies only to Inline batch mode and selects the criteria to meet before processing each batch:

    - Message count based: The number of messages to collect in the batch, for example, 10 messages
    - Size based: The maximum batch size in bytes, for example, 100 MB
    - Schedule based: The interval and frequency between batch releases, for example, 10 minutes. The minimum recurrence is 60 seconds or 1 minute. Fractional minute values are effectively rounded up to 1 minute. To specify a start date and time, choose Show advanced options.
    - Select all: Use all the specified criteria.

    This example selects all the criteria:

    Provide Batch trigger details

  4. Now add one or more actions that process each batch.

    For this example, add an action that sends an email when the batch trigger fires. The trigger runs and sends an email when the batch either has 10 messages, reaches 10 MB, or after 10 minutes pass.

    1. Under the batch trigger, choose New step.

    2. In the search box, enter "send email" as your filter. Based on your email provider, select an email connector.

      For example, if you have a personal account, such as @outlook.com or @hotmail.com, select the Outlook.com connector. If you have a Gmail account, select the Gmail connector. This example uses Office 365 Outlook.

    3. Select this action: Send an email - <email provider>

      For example:

      Select "Send an email" action for your email provider

  5. If prompted, sign in to your email account.

  6. Set the properties for the action you added.

    • In the To box, enter the recipient's email address. For testing purposes, you can use your own email address.

    • In the Subject box, when the dynamic content list appears, select the Partition Name field.

      From the dynamic content list, select "Partition Name"

      In a later section, you can specify a unique partition key that divides the target batch into logical subsets to where you can send messages. Each set has a unique number that's generated by the batch sender logic app. This capability lets you use a single batch with multiple subsets and define each subset with the name that you provide.

      [!IMPORTANT] A partition has a limit of 5,000 messages or 80 MB. If either condition is met, Logic Apps might release the batch, even when your defined release condition isn't met.

    • In the Body box, when the dynamic content list appears, select the Message Id field.

      The Logic Apps Designer automatically adds a "For each" loop around the send email action because that action treats the output from the previous action as a collection, rather than a batch.

      For "Body", select "Message Id"

  7. Save your logic app. You've now created a batch receiver.

    Save your logic app

  8. If you're using Visual Studio, make sure you deploy your batch receiver logic app to Azure. Otherwise, you can't select the batch receiver when you create the batch sender.

Create batch sender

Now create one or more batch sender logic apps that send messages to the batch receiver logic app. In each batch sender, you specify the batch receiver and batch name, message content, and any other settings. You can optionally provide a unique partition key to divide the batch into logical subsets for collecting messages with that key.

  • Make sure you've already created your batch receiver so when you create your batch sender, you can select the existing batch receiver as the destination batch. While batch receivers don't need to know anything about batch senders, batch senders must know where to send messages.

  • Make sure your batch receiver and batch sender share the same Azure region and Azure subscription. If they don't, you can't select the batch receiver when you create the batch sender because they're not visible to each other.

  1. Create another logic app with this name: "BatchSender"

    1. In the search box, enter "recurrence" as your filter. Select this trigger: Recurrence - Schedule

      Add the "Recurrence - Schedule" trigger

    2. Set the frequency and interval to run the sender logic app every minute.

      Set frequency and interval for recurrence trigger

  2. Add a new action for sending messages to a batch.

    1. Under the recurrence trigger, choose New step.

    2. In the search box, enter "batch" as your filter. Select the Actions list, and then select this action: Choose a Logic Apps workflow with batch trigger - Send messages to batch

      Select "Choose a Logic Apps workflow with batch trigger"

    3. Select your batch receiver logic app that you previously created.

      Select "batch receiver" logic app

      [!NOTE] The list also shows any other logic apps that have batch triggers.

      If you're using Visual Studio, and you don't see any batch receivers to select, check that you deployed your batch receiver to Azure. If you haven't, learn how to deploy your batch receiver logic app to Azure.

    4. Select this action: Batch_messages - <your-batch-receiver>

      Select this action: "Batch_messages - <your-logic-app>"

  3. Set the batch sender's properties:

    Property Description
    Batch Name The batch name defined by the receiver logic app, which is "TestBatch" in this example

    Important: The batch name gets validated at runtime and must match the name specified by the receiver logic app. Changing the batch name causes the batch sender to fail.

    Message Content The content for the message you want to send

    For this example, add this expression, which inserts the current date and time into the message content that you send to the batch:

    1. Click inside the Message Content box.

    2. When the dynamic content list appears, choose Expression.

    3. Enter the expression utcnow(), and then choose OK.

      In "Message Content", choose "Expression", enter "utcnow()", and choose "OK".

  4. Now set up a partition for the batch. In the "BatchReceiver" action, choose Show advanced options and set these properties:

    Property Description
    Partition Name An optional unique partition key to use for dividing the target batch into logical subsets and collect messages based on that key
    Message Id An optional message identifier that is a generated globally unique identifier (GUID) when empty

    For this example, in the Partition Name box, add an expression that generates a random number between one and five. Leave the Message Id box empty.

    1. Click inside the Partition Name box so that the dynamic content list appears.

    2. In the dynamic content list, choose Expression.

    3. Enter the expression rand(1,6), and then choose OK.

      Set up a partition for your target batch

      This rand function generates a number between one and five. So you are dividing this batch into five numbered partitions, which this expression dynamically sets.

  5. Save your logic app. Your sender logic app now looks similar to this example:

    Save your sender logic app

Test your logic apps

To test your batching solution, leave your logic apps running for a few minutes. Soon, you start getting emails in groups of five, all with the same partition key.

Your batch sender logic app runs every minute, generates a random number between one and five, and uses this generated number as the partition key for the target batch where messages are sent. Each time the batch has five items with the same partition key, your batch receiver logic app fires and sends mail for each message.

Important

When you're done testing, make sure that you disable the BatchSender logic app to stop sending messages and avoid overloading your inbox.

Next steps