Create an Azure Managed Lustre file system using Azure Resource Manager templates

You can automate Azure Managed Lustre file system creation by using Azure Resource Manager templates. This article explains the basic procedure and gives examples of the files you need.

For more information about the templates, see Azure Resource Manager templates.

This article gives examples of two different methods for creating Azure Resource Manager templates:

• Use JSON to create Azure Resource Manager templates directly. See JSON template syntax.
• Use Bicep, which uses simpler syntax to supply the configuration details. When you deploy the template, the Bicep files are converted into Azure Resource Manager template files. See the Bicep documentation.

To learn more about your options, see Comparing JSON and Bicep for templates.

Choose file system type and size

Before you write a template, you must make some decisions about your Azure Managed Lustre file system. To learn more about the configuration options, see the setup details at Create an Azure Managed Lustre file system.

When you use a template, specify a SKU name to define the basic type of Azure Managed Lustre system to create. If you use the Azure portal to create your Azure Managed Lustre, you specify the system type indirectly by selecting its capabilities.

In Azure, the term SKU defines a set of features for the resource being created. For an Azure Managed Lustre file system, the SKU sets system qualities such as the type of disks used, the amount of storage supported, and the maximum throughput capacity.

Currently, the following SKUs are supported:

• AMLFS-Durable-Premium-40
• AMLFS-Durable-Premium-125
• AMLFS-Durable-Premium-250
• AMLFS-Durable-Premium-500

These SKUs create a file system that uses durable SSD storage. The following table shows the throughput and storage size values for each SKU:

¹ If you require storage values larger than the listed maximum, you can open a support ticket to explore options.

You can use the create workflow in Azure portal to check SKU capabilities. SKU-specific settings are on the Basics tab under File system details.

Create a template file

Once you decide on configuration options, you can create a template file. The template file is a JSON or Bicep file that contains the configuration details for your Azure Managed Lustre file system. This section explains the required and optional information to include in your template file.

For example files that contain all possible configuration options, see Sample JSON files and Sample Bicep file.

Required information

This section explains the information you need to include in your Azure Resource Manager template files to create an Azure Managed Lustre file system. The exact syntax is different between Bicep and JSON, so consult the examples for each language type for the literal values.

• Resource type to create - This value tells Azure Resource Manager that you're creating an Azure Managed Lustre file system by passing a combination of the value Microsoft.StorageCache/amlFileSystems and the API version.

  There are several ways to create the resource type:

  • In this article's JSON example, the resource type value is passed literally in the template.json file, but the API version value is read from the parameters.json file.
  • In the Bicep example, the resource type and API version are passed together at the beginning of the template file.

• API version - The version of Azure Managed Lustre to create.

  To find the current API version:

  az provider show --namespace Microsoft.StorageCache --query "resourceTypes[?resourceType=='amlFilesystems'].apiVersions" --out table
  

• SKU name - The performance model for the file system, either AMLFS-Durable-Premium-125 or AMLFS-Durable-Premium-250.

  Use the following command to find available SKUs (use the current API version):

  az rest --url https://management.azure.com/subscriptions/<subscription_id>/providers/Microsoft.StorageCache/skus/?api-version=<version> | jq '.value[].name' | grep AMLFS| uniq
  

• Location - The name of the Azure region where the file system is created.

  To find the regions and availability zones where Azure Managed Lustre is supported:

  az provider show --namespace Microsoft.StorageCache --query "resourceTypes[?resourceType=='amlFilesystems'].zoneMappings[].{location: location, zones: to_string(zones)}" --out table
  

  Note

  This command outputs the display names of Azure regions; you should use the shorter name value (for example, use "eastus" instead of "East US").

  This command returns the short name from the display name. West US is an example; this command returns westus:

  az account list-locations --query "[?displayName=='West US'].name" --output tsv
  

• Availability zone - The availability zone to use within the Azure region.

  Use the previous command in Location to find availability zones. Specify a single availability zone for your system.

• File system name - The user-visible name for this Azure Managed Lustre file system.

• File system subnet - The subnet that the file system uses. Provide the subnet URI; for example, /subscriptions/<SubscriptionID>/resourceGroups/<VnetResourceGroupName>/providers/Microsoft.Network/virtualNetworks/<VnetName>/subnets/<SubnetName>.

• Storage capacity - The size of your Azure Managed Lustre cluster, in TiB. Values depend on the SKU. For more information, see Choose file system type and size.

• Maintenance period - Requires two values that set the maintenance period. These values define a 30-minute period weekly during which system updates can be done.

  • Day of the week (for example, Sunday)
  • Time of day (UTC) (for example, 22:00)

Optional information

The parameters in this section are either optional, or required only if you're using specific features.

• Tags - Use this option if you want to set Azure resource metadata tags.

• Blob integration settings - Supply these values to use an integrated Blob Storage container with this file system. For more information, see Blob integration.

  • Container - The resource ID of the blob container to use for Lustre hierarchical storage management (HSM).
  • Logging container - The resource ID of a separate container to hold import and export logs. The logging container must be in the same storage account as the data container.
  • Import prefix (optional) - If this value is provided, only blobs beginning with the import prefix string are imported into the Azure Managed Lustre File System. If you don't provide it, the default value is /, which specifies that all blobs in the container are imported.

• Customer-managed key settings - Supply these values if you want to use an Azure Key Vault to control the encryption keys that are used to encrypt your data in the Azure Managed Lustre system. By default, data is encrypted using Microsoft-managed encryption keys.

  • Identity type - set this to UserAssigned to turn on customer-managed keys.
  • Encryption Key Vault - The resource ID of the Azure Key Vault that stores the encryption keys.
  • Encryption key URL - The identifier for the key to use to encrypt your data.
  • Managed identity - A user-assigned managed identity that the Azure Managed Lustre file system uses to access the Azure Key Vault. For more information, see Use customer-managed encryption keys.

Deploy the file system using the template

These example steps use Azure CLI commands to create a new resource group and create an Azure Managed Lustre file system in it.

Before you deploy, make sure you complete the following steps:

• Choose a file system type and size
• Create a template file
• Ensure that all prerequisites are met.

Follow these steps to deploy the file system using the template:

1. Set your default subscription:

   az account set --subscription "<SubscriptionID>"
   az account show
   

2. Optionally, create a new resource group for your Azure Managed Lustre file system. If you want to use an existing resource group, skip this step and provide the name of the existing resource group when you execute the template command.

   az group create --name <ResourceGroupName> --location <RegionShortname>
   

   Your file system can use resources outside of its own resource group as long as they are in the same subscription.

3. Deploy the Azure Managed Lustre file system by using the template. The syntax is different depending on whether you're using JSON or Bicep files, and the number of files you use.

   You can deploy both Bicep and JSON templates as single files or multiple files. For more information and to see the exact syntax for each option, see the Azure Resource Manager templates documentation.

   Example JSON command:

   az deployment group create \
     --name <ExampleDeployment> \
     --resource-group <ResourceGroupName> \
     --template-file azlustre-template.json \
     --parameters @azlustre-parameters.json
   

   Example Bicep command:

   az deployment group create \
    --resource-group <ResourceGroupName> \
    --template-file azlustre.bicep
   

Sample JSON files

This section shows sample contents for a template file and a separate parameters file. These files contain all possible configuration options. You can remove optional parameters when creating your own Azure Resource Manager template.

Template file

This section shows example contents of a template file:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "location": {
            "type": "String"
        },
        "apiVersion": {
            "type": "String"
        },
        "fileSystemName": {
            "type": "String"
        },
        "availabilityZone": {
            "type": "Array"
        },
        "subnetId": {
            "type": "String"
        },
        "storageCapacityTiB": {
            "type": "Int"
        },
        "container": {
            "type": "String"
        },
        "loggingContainer": {
            "type": "String"
        },
        "importPrefix": {
            "type": "String"
        },
        "dayOfWeek": {
            "type": "String"
        },
        "timeOfDay": {
            "type": "String"
        },
        "encryptionKeyUrl": {
            "type": "String"
        },
        "encryptionVault": {
            "type": "String"
        }
    },
    "resources": [
        {
            "type": "Microsoft.StorageCache/amlFileSystems",
            "apiVersion": "[parameters('apiVersion')]",
            "name": "[parameters('fileSystemName')]",
            "location": "[parameters('location')]",
            "tags": {
                "MyTagName": "TagValue"
            },
            "sku": {
                "name": "AMLFS-Durable-Premium-250"
            },
            "identity": {
                "type": "UserAssigned",
                "userAssignedIdentities": {
                    "/subscriptions/<subscription_id>/resourcegroups/<identity_resource_group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name_of_identity>": {}
                }
            },
            "properties": {
                "storageCapacityTiB": "[parameters('storageCapacityTiB')]",
                "zones": "[parameters('availabilityZone')]",
                "filesystemSubnet": "[parameters('subnetId')]",
                "hsm": {
                    "settings": {
                        "container": "[parameters('container')]",
                        "loggingContainer": "[parameters('loggingContainer')]",
                        "importPrefix": "[parameters('importPrefix')]"
                    }
                },
                "maintenanceWindow": {
                    "dayOfWeek": "[parameters('dayOfWeek')]",
                    "timeOfDay": "[parameters('timeOfDay')]"
                },
                "encryptionSettings": {
                    "keyEncryptionKey": {
                        "keyUrl": "[parameters('encryptionKeyUrl')]",
                        "sourceVault": {
                            "id": "[parameters('encryptionVault')]"
                        }
                    }
                }
            }
        }
    ],
    "outputs": {}
}

Parameters file

This section shows example contents of a parameters file:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "location": {
            "value": "eastus"
        },
        "apiVersion": {
            "value": "2023-05-01"
        },
        "fileSystemName": {
            "value": "amlfs-example"
        },
        "availabilityZone": {
            "value": [
                "1"
            ]
        },
        "subnetId": {
            "value": "/subscriptions/<subscription_id>/resourceGroups/<vnet_resource_group>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>"
        },
        "storageCapacityTiB": {
            "value": 4
        },
        "container": {
            "value": "/subscriptions/<subscription_id>/resourceGroups/<storage_account_resource_group>/providers/Microsoft.Storage/storageAccounts/<storage_account_name>/blobServices/default/containers/<container_name>"
        },
        "loggingContainer": {
            "value": "/subscriptions/<subscription_id>/resourceGroups/<storage_account_resource_group>/providers/Microsoft.Storage/storageAccounts/<storage_account_name>/blobServices/default/containers/<log_container_name>"
        },
        "importPrefix": {
            "value": ""
        },
        "dayOfWeek": {
            "value": "Saturday"
        },
        "timeOfDay": {
            "value": "16:45"
        },
        "encryptionKeyUrl": {
            "value": "<encryption_key_URL>"
        },
        "encryptionVault": {
            "value": "/subscriptions/<subscription_id>/resourceGroups/<keyvault_resource_group>/providers/Microsoft.KeyVault/vaults/<keyvault_name>"
        }
    }
}

Sample Bicep file

This example includes all of the possible values in an Azure Managed Lustre template. When creating your template, remove any optional values, you don't want.

resource fileSystem 'Microsoft.StorageCache/amlFileSystems@2023-05-01' = {
  name: 'fileSystemName'
  location: 'eastus'
  tags: {
    'test-tag': 'test'
  }
  sku: {
    name: 'AMLFS-Durable-Premium-250'
  }
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '/subscriptions/<subscription_id>/resourcegroups/<identity_resource_group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name_of_identity>': {
      }
    }
  }
  properties: {
    storageCapacityTiB: 8
    zones: [ 1 ]
    filesystemSubnet: '/subscriptions/<subscription_id>/resourceGroups/<vnet_resource_group>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>'
    hsm: {
      settings: {
        container: '/subscriptions/<subscription_id>/resourceGroups/<storage_account_resource_group>/providers/Microsoft.Storage/storageAccounts/<storage_account_name>/blobServices/default/containers/<container_name>'
        loggingContainer: '/subscriptions/<subscription_id>/resourceGroups/<storage_account_resource_group>/providers/Microsoft.Storage/storageAccounts/<storage_account_name>/blobServices/default/containers/<log_container_name>'
        importPrefix: ''
      }
    }
    maintenanceWindow: {
      dayOfWeek: 'Friday'
      timeOfDay: '21:00'
    }
    encryptionSettings: {
      keyEncryptionKey: {
        keyUrl: '<encryption_key_URL>'
        sourceVault: {
          id: '/subscriptions/<subscription_id>/resourceGroups/<keyvault_resource_group>/providers/Microsoft.KeyVault/vaults/<keyvault_name>'
        }
      }
    }
  }
}

Next steps

• Azure Managed Lustre File System overview