The Azure Batch task type allows for the easy processing of data using Microsoft’s Azure Batch computing service. With Loome Integrate Online you are able to easily interact with Azure Batch without requiring a development background or Azure Batch Shipyard.
The Azure Batch task type is also incredibly flexible, allowing for simple pool provisioning thanks to cluster definitions and integration with your existing Azure Blob connections.
The Azure Batch task type requires an Azure Batch Account connection, learn more in the connections reference.
In order to create an Azure Batch task you will need to have a few things ready in advance.
Create a new task in your chosen job.
Select Azure Batch Task as your task type.
Choose a cluster definition. Your available cluster definitions can be chosen from this dropdown. (Learn more about cluster definitions here. Or learn how to create one for Azure Batch Pool here.)
The details of your chosen cluster definition will display below.
Provide the script for your Azure Batch task.
You can also choose a file share connection. (This connection is optional.)
Selecting a connection here will allow for the file share storage to be mounted and treated as a local drive on a compute node, when that node joins a pool.
You can still set up an external file share in your script if you do not want to choose a connection here.
The Auto Scaling Evaluation Interval will define the time interval that the compute nodes will be periodically reviewed when you provide an Auto Scaling Formula.
Provide an interval in minutes in this field.
Provide a whole number that is between 5 minutes and 10080 minutes.
When you provide a value here, for example 20, the compute nodes in the pool will be reviewed every 20 minutes by the Auto Scaling Formula.
If you provide a value here, you will also need to provide an Auto Scaling Formula in the next field.
If you submit an empty value, the default interval of 5 minutes will be used.
If you submit an empty value for both the Auto Scaling Evaluation Interval and the Auto Scaling Formula, Auto Scaling Evaluation will not apply to this Azure Batch task.
The formula is a string value of statements that will be assigned to your pool and the Batch service will use this formula to review compute nodes in the pool at each interval.
Learn more about Auto Scale Formulas here.
Provide your formula in this field.
Any parameters added to the task will be usable inside the Azure Batch script at runtime.
If you provide any Task Parameters, they will take precedence over the Auto Scaling Evaluation Interval and Auto Scaling Formula.
Submit the task and your task is ready to run.