Azure Batch libraries for python
Overview
Run large-scale parallel and high-performance computing applications efficiently in the cloud with Azure Batch.
To get started with Azure Batch, see Create a Batch account with the Azure portal.
Install the libraries
Client library
The Azure Batch client libraries let you configure compute nodes and pools, define tasks and configure them to run in jobs, and set up a job manager to control and monitor job execution. Learn more about using these objects to run large-scale parallel compute solutions.
pip install azure-batch
Example
Set up a pool of Linux compute nodes in a batch account:
import azure.batch
from azure.batch import batch_auth, BatchServiceClient, models
# create the batch client for an account using its URI and keys
creds = batch_auth.SharedKeyCredentials(account, key)
client = BatchServiceClient(creds, batch_url)
# Create the VirtualMachineConfiguration, specifying
# the VM image reference and the Batch node agent to
# be installed on the node.
vmc = models.VirtualMachineConfiguration(
image_reference = models.ImageReference(
publisher='Canonical',
offer='UbuntuServer',
sku='18.04-LTS'
),
node_agent_sku_id = "batch.node.ubuntu 18.04")
# Assign the virtual machine configuration to the pool
new_pool = models.PoolAddParameter(
id = 'new_pool',
vm_size='standard_d2_v2',
virtual_machine_configuration = vmc
)
# Create pool in the Batch service
client.pool.add(new_pool)
Management API
Use the Azure Batch management libraries to create and delete batch accounts, read and regenerate batch account keys, and manage batch account storage.
pip install azure-mgmt-batch
Example
Create an Azure Batch account and configure a new application and Azure storage account for it.
from azure.mgmt.batch import BatchManagementClient
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.storage import StorageManagementClient
LOCATION ='eastus'
GROUP_NAME ='batchresourcegroup'
STORAGE_ACCOUNT_NAME ='batchstorageaccount'
# Create Resource group
print('Create Resource Group')
resource_client.resource_groups.create_or_update(GROUP_NAME, {'location': LOCATION})
# Create a storage account
storage_async_operation = storage_client.storage_accounts.create(
GROUP_NAME,
STORAGE_ACCOUNT_NAME,
StorageAccountCreateParameters(
sku=Sku(SkuName.standard_ragrs),
kind=Kind.storage,
location=LOCATION
)
)
storage_account = storage_async_operation.result()
# Create a Batch Account, specifying the storage account we want to link
storage_resource = storage_account.id
batch_account_parameters = azure.mgmt.batch.models.BatchAccountCreateParameters(
location=LOCATION,
auto_storage=azure.mgmt.batch.models.AutoStorageBaseProperties(storage_resource)
)
creating = batch_client.batch_account.begin_create('MyBatchResourceGroup', 'MyBatchAccount', batch_account_parameters)
creating.wait()
Azure SDK for Python
Feedback
https://aka.ms/ContentUserFeedback.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see:Submit and view feedback for