title | description | services | ms.service | ms.suite | author | ms.author | ms.reviewer | ms.topic | ms.date |
---|---|---|---|---|---|---|---|---|---|
Batch process messages as a group or collection - Azure Logic Apps | Microsoft Docs |
Send and receive messages as batches in Azure Logic Apps |
logic-apps |
logic-apps |
integration |
divyaswarnkar |
divswa |
estfan, jonfan, LADocs |
article |
08/19/2018 |
To send and process messages together in a specific way as groups, you can create a batching solution that collects messages into a batch until your specified criteria are met for releasing and processing the batched messages. Batching can reduce how often your logic app processes messages. This article shows how to build a batching solution by creating two logic apps within the same Azure subscription, Azure region, and following this specific order:
-
The "batch receiver" logic app, which accepts and collects messages into a batch until your specified criteria is met for releasing and processing those messages.
Make sure you first create the batch receiver so you can later select the batch destination when you create the batch sender.
-
One or more "batch sender" logic apps, which send the messages to the previously created batch receiver.
You can also specify a unique key, such as a customer number, that partitions or divides the target batch into logical subsets based on that key. That way, the receiver app can collect all items with the same key and process them together.
Make sure your batch receiver and batch sender share the same Azure subscription and Azure region. If they don't, you can't select the batch receiver when you create the batch sender because they're not visible to each other.
To follow this example, you need these items:
-
An Azure subscription. If you don't have a subscription, you can start with a free Azure account. Or, sign up for a Pay-As-You-Go subscription.
-
An email account with any email provider supported by Azure Logic Apps
-
Basic knowledge about how to create logic apps
-
To use Visual Studio rather than the Azure portal, make sure you set up Visual Studio for working with Logic Apps.
Before you can send messages to a batch, that batch must first exist as the destination where you send those messages. So first, you must create the "batch receiver" logic app, which starts with the Batch trigger. That way, when you create the "batch sender" logic app, you can select the batch receiver logic app. The batch receiver continues collecting messages until your specified criteria is met for releasing and processing those messages. While batch receivers don't need to know anything about batch senders, batch senders must know the destination where they send the messages.
-
In the Azure portal or Visual Studio, create a logic app with this name: "BatchReceiver"
-
In Logic Apps Designer, add the Batch trigger, which starts your logic app workflow. In the search box, enter "batch" as your filter. Select this trigger: Batch messages
-
Set the batch receiver properties:
Property Description Batch Mode - Inline: For defining release criteria inside the batch trigger
- Integration Account: For defining multiple release criteria configurations through an integration account. With an integration account, you can maintain these configurations all in one place rather than in separate logic apps.Batch Name The name for your batch, which is "TestBatch" in this example, and applies only to Inline batch mode Release Criteria Applies only to Inline batch mode and selects the criteria to meet before processing each batch: - Message count based: The number of messages to collect in the batch, for example, 10 messages
- Size based: The maximum batch size in bytes, for example, 100 MB
- Schedule based: The interval and frequency between batch releases, for example, 10 minutes. The minimum recurrence is 60 seconds or 1 minute. Fractional minute values are effectively rounded up to 1 minute. To specify a start date and time, choose Show advanced options.
- Select all: Use all the specified criteria.This example selects all the criteria:
-
Now add one or more actions that process each batch.
For this example, add an action that sends an email when the batch trigger fires. The trigger runs and sends an email when the batch either has 10 messages, reaches 10 MB, or after 10 minutes pass.
-
Under the batch trigger, choose New step.
-
In the search box, enter "send email" as your filter. Based on your email provider, select an email connector.
For example, if you have a personal account, such as @outlook.com or @hotmail.com, select the Outlook.com connector. If you have a Gmail account, select the Gmail connector. This example uses Office 365 Outlook.
-
Select this action: Send an email - <email provider>
For example:
-
-
If prompted, sign in to your email account.
-
Set the properties for the action you added.
-
In the To box, enter the recipient's email address. For testing purposes, you can use your own email address.
-
In the Subject box, when the dynamic content list appears, select the Partition Name field.
In a later section, you can specify a unique partition key that divides the target batch into logical subsets to where you can send messages. Each set has a unique number that's generated by the batch sender logic app. This capability lets you use a single batch with multiple subsets and define each subset with the name that you provide.
[!IMPORTANT] A partition has a limit of 5,000 messages or 80 MB. If either condition is met, Logic Apps might release the batch, even when your defined release condition isn't met.
-
In the Body box, when the dynamic content list appears, select the Message Id field.
The Logic Apps Designer automatically adds a "For each" loop around the send email action because that action treats the output from the previous action as a collection, rather than a batch.
-
-
Save your logic app. You've now created a batch receiver.
-
If you're using Visual Studio, make sure you deploy your batch receiver logic app to Azure. Otherwise, you can't select the batch receiver when you create the batch sender.
Now create one or more batch sender logic apps that send messages to the batch receiver logic app. In each batch sender, you specify the batch receiver and batch name, message content, and any other settings. You can optionally provide a unique partition key to divide the batch into logical subsets for collecting messages with that key.
-
Make sure you've already created your batch receiver so when you create your batch sender, you can select the existing batch receiver as the destination batch. While batch receivers don't need to know anything about batch senders, batch senders must know where to send messages.
-
Make sure your batch receiver and batch sender share the same Azure region and Azure subscription. If they don't, you can't select the batch receiver when you create the batch sender because they're not visible to each other.
-
Create another logic app with this name: "BatchSender"
-
Add a new action for sending messages to a batch.
-
Under the recurrence trigger, choose New step.
-
In the search box, enter "batch" as your filter. Select the Actions list, and then select this action: Choose a Logic Apps workflow with batch trigger - Send messages to batch
-
Select your batch receiver logic app that you previously created.
[!NOTE] The list also shows any other logic apps that have batch triggers.
If you're using Visual Studio, and you don't see any batch receivers to select, check that you deployed your batch receiver to Azure. If you haven't, learn how to deploy your batch receiver logic app to Azure.
-
Select this action: Batch_messages - <your-batch-receiver>
-
-
Set the batch sender's properties:
Property Description Batch Name The batch name defined by the receiver logic app, which is "TestBatch" in this example Important: The batch name gets validated at runtime and must match the name specified by the receiver logic app. Changing the batch name causes the batch sender to fail.
Message Content The content for the message you want to send For this example, add this expression, which inserts the current date and time into the message content that you send to the batch:
-
Now set up a partition for the batch. In the "BatchReceiver" action, choose Show advanced options and set these properties:
Property Description Partition Name An optional unique partition key to use for dividing the target batch into logical subsets and collect messages based on that key Message Id An optional message identifier that is a generated globally unique identifier (GUID) when empty For this example, in the Partition Name box, add an expression that generates a random number between one and five. Leave the Message Id box empty.
-
Click inside the Partition Name box so that the dynamic content list appears.
-
In the dynamic content list, choose Expression.
-
Enter the expression
rand(1,6)
, and then choose OK.This rand function generates a number between one and five. So you are dividing this batch into five numbered partitions, which this expression dynamically sets.
-
-
Save your logic app. Your sender logic app now looks similar to this example:
To test your batching solution, leave your logic apps running for a few minutes. Soon, you start getting emails in groups of five, all with the same partition key.
Your batch sender logic app runs every minute, generates a random number between one and five, and uses this generated number as the partition key for the target batch where messages are sent. Each time the batch has five items with the same partition key, your batch receiver logic app fires and sends mail for each message.
Important
When you're done testing, make sure that you disable the BatchSender logic app to stop sending messages and avoid overloading your inbox.