Working with Azure OpenAI models

Article
02/26/2024

Azure OpenAI Service is powered by a diverse set of models with different capabilities and price points. Model availability varies by region.

You can get a list of models that are available for both inference and fine-tuning by your Azure OpenAI resource by using the Models List API.

Model updates

Azure OpenAI now supports automatic updates for select model deployments. On models where automatic update support is available, a model version drop-down is visible in Azure OpenAI Studio under Create new deployment and Edit deployment:

You can learn more about Azure OpenAI model versions and how they work in the Azure OpenAI model versions article.

Auto update to default

When you set your deployment to Auto-update to default, your model deployment is automatically updated within two weeks of a change in the default version. For a preview version, it updates automatically when a new preview version is available starting two weeks after the new preview version is released.

If you're still in the early testing phases for inference models, we recommend deploying models with auto-update to default set whenever it's available.

Specific model version

As your use of Azure OpenAI evolves, and you start to build and integrate with applications you might want to manually control model updates. You can first test and validate that your application behavior is consistent for your use case before upgrading.

When you select a specific model version for a deployment, this version remains selected until you either choose to manually update yourself, or once you reach the retirement date for the model. When the retirement date is reached the model will automatically upgrade to the default version at the time of retirement.

Viewing retirement dates

For currently deployed models, from Azure OpenAI Studio select Deployments:

To view retirement dates for all available models in a given region from Azure OpenAI Studio, select Models > Column options > Select Deprecation fine tune and Deprecation inference:

Model deployment upgrade configuration

You can check what model upgrade options are set for previously deployed models in Azure OpenAI Studio. Select Deployments > Under the deployment name column select one of the deployment names that are highlighted in blue.

Selecting a deployment name opens the Properties for the model deployment. You can view what upgrade options are set for your deployment under Version update policy:

The corresponding property can also be accessed via REST, Azure PowerShell, and Azure CLI.

Option	Read	Update
REST	Yes. If `versionUpgradeOption` is not returned, it means it is `null`	Yes
Azure PowerShell	Yes.`VersionUpgradeOption` can be checked for `$null`	Yes
Azure CLI	Yes. It shows `null` if `versionUpgradeOption` is not set.	No. It is currently not possible to update the version upgrade option.

There are three distinct model deployment upgrade options:

Name	Description
`OnceNewDefaultVersionAvailable`	Once a new version is designated as the default, the model deployment automatically upgrades to the default version within two weeks of that designation change being made.
`OnceCurrentVersionExpired`	Once the retirement date is reached the model deployment automatically upgrades to the current default version.
`NoAutoUpgrade`	The model deployment never automatically upgrades. Once the retirement date is reached the model deployment stops working. You need to update your code referencing that deployment to point to a nonexpired model deployment.

Note

null is equivalent to OnceCurrentVersionExpired. If the Version update policy option is not present in the properties for a model that supports model upgrades this indicates the value is currently null. Once you explicitly modify this value, the property is visible in the studio properties page as well as via the REST API.

Review the Azure PowerShell getting started guide to install Azure PowerShell locally or you can use the Azure Cloud Shell.

The steps below demonstrate checking the VersionUpgradeOption option property as well as updating it:

// Step 1: Get Deployment
$deployment = Get-AzCognitiveServicesAccountDeployment -ResourceGroupName {ResourceGroupName} -AccountName {AccountName} -Name {DeploymentName}
 
// Step 2: Show Deployment VersionUpgradeOption
$deployment.Properties.VersionUpgradeOption
 
// VersionUpgradeOption can be null - one way to check is
$null -eq $deployment.Properties.VersionUpgradeOption
 
// Step 3: Update Deployment VersionUpgradeOption
$deployment.Properties.VersionUpgradeOption = "NoAutoUpgrade"
New-AzCognitiveServicesAccountDeployment -ResourceGroupName {ResourceGroupName} -AccountName {AccountName} -Name {DeploymentName} -Properties $deployment.Properties -Sku $deployment.Sku
 
// repeat step 1 and 2 to confirm the change.
// If not sure about deployment name, use this command to show all deployments under an account
Get-AzCognitiveServicesAccountDeployment -ResourceGroupName {ResourceGroupName} -AccountName {AccountName}

// To update to a new model version

// Step 1: Get Deployment
$deployment = Get-AzCognitiveServicesAccountDeployment -ResourceGroupName {ResourceGroupName} -AccountName {AccountName} -Name {DeploymentName}

// Step 2: Show Deployment Model properties
$deployment.Properties.Model.Version

// Step 3: Update Deployed Model Version
$deployment.Properties.Model.Version = "0613"
New-AzCognitiveServicesAccountDeployment -ResourceGroupName {ResourceGroupName} -AccountName {AccountName} -Name {DeploymentName} -Properties $deployment.Properties -Sku $deployment.Sku

// repeat step 1 and 2 to confirm the change.

To query the current model deployment settings including the deployment upgrade configuration for a given resource use Deployments List. If the value is null, you won't see a versionUpgradeOption property.

GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.CognitiveServices/accounts/{accountName}/deployments?api-version=2023-05-01

Path parameters

Parameter	Type	Required?	Description
`acountname`	string	Required	The name of your Azure OpenAI Resource.
`resourceGroupName`	string	Required	The name of the associated resource group for this model deployment.
`subscriptionId`	string	Required	Subscription ID for the associated subscription.
`api-version`	string	Required	The API version to use for this operation. This follows the YYYY-MM-DD format.

Supported versions

2023-05-01 Swagger spec

Example response

{
  "value": [
    {
      "id": "/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeeb/resourceGroups/az-test-openai/providers/Microsoft.CognitiveServices/accounts/aztestopenai001/deployments/gpt-35-turbo",
      "type": "Microsoft.CognitiveServices/accounts/deployments",
      "name": "gpt-35-turbo",
      "sku": {
        "name": "Standard",
        "capacity": 80
      },
      "properties": {
        "model": {
          "format": "OpenAI",
          "name": "gpt-35-turbo",
          "version": "0301"
        },
        "versionUpgradeOption": "OnceNewDefaultVersionAvailable",
        "capabilities": {
          "completion": "true",
          "chatCompletion": "true"
        },
        "raiPolicyName": "Microsoft.Default",
        "provisioningState": "Succeeded",
        "rateLimits": [
          {
            "key": "request",
            "renewalPeriod": 10,
            "count": 80
          },
          {
            "key": "token",
            "renewalPeriod": 60,
            "count": 80000
          }
        ]
      },
      "systemData": {
        "createdBy": "docs@contoso.com",
        "createdByType": "User",
        "createdAt": "2023-07-31T16:45:32.622404Z",
        "lastModifiedBy": "docs@contoso.com",
        "lastModifiedByType": "User",
        "lastModifiedAt": "2023-10-31T13:59:34.4978286Z"
      },
      "etag": "\"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee\""
    }
  ]
}

You can then take the settings from this list to construct an update model REST API call as described below if you want to modify the deployment upgrade configuration.

Update & deploy models via the API

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.CognitiveServices/accounts/{accountName}/deployments/{deploymentName}?api-version=2023-05-01

Path parameters

Parameter	Type	Required?	Description
`acountname`	string	Required	The name of your Azure OpenAI Resource.
`deploymentName`	string	Required	The deployment name you chose when you deployed an existing model or the name you would like a new model deployment to have.
`resourceGroupName`	string	Required	The name of the associated resource group for this model deployment.
`subscriptionId`	string	Required	Subscription ID for the associated subscription.
`api-version`	string	Required	The API version to use for this operation. This follows the YYYY-MM-DD format.

Supported versions

2023-05-01 Swagger spec

Request body

This is only a subset of the available request body parameters. For the full list of the parameters, you can refer to the REST API reference documentation.

Parameter	Type	Description
versionUpgradeOption	String	Deployment model version upgrade options: `OnceNewDefaultVersionAvailable` `OnceCurrentVersionExpired` `NoAutoUpgrade`
capacity	integer	This represents the amount of quota you are assigning to this deployment. A value of 1 equals 1,000 Tokens per Minute (TPM)

Example request

curl -X PUT https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/resource-group-temp/providers/Microsoft.CognitiveServices/accounts/docs-openai-test-001/deployments/gpt-35-turbo?api-version=2023-05-01 \
  -H "Content-Type: application/json" \
  -H 'Authorization: Bearer YOUR_AUTH_TOKEN' \
  -d '{"sku":{"name":"Standard","capacity":120},"properties": {"model": {"format": "OpenAI","name": "gpt-35-turbo","version": "0613"},"versionUpgradeOption":"OnceCurrentVersionExpired"}}'

Note

There are multiple ways to generate an authorization token. The easiest method for initial testing is to launch the Cloud Shell from the Azure portal. Then run az account get-access-token. You can use this token as your temporary authorization token for API testing.

Example response

 {
  "id": "/subscriptions/{subscription-id}/resourceGroups/resource-group-temp/providers/Microsoft.CognitiveServices/accounts/docs-openai-test-001/deployments/gpt-35-turbo",
  "type": "Microsoft.CognitiveServices/accounts/deployments",
  "name": "gpt-35-turbo",
  "sku": {
    "name": "Standard",
    "capacity": 120
  },
  "properties": {
    "model": {
      "format": "OpenAI",
      "name": "gpt-35-turbo",
      "version": "0613"
    },
    "versionUpgradeOption": "OnceCurrentVersionExpired",
    "capabilities": {
      "chatCompletion": "true"
    },
    "provisioningState": "Succeeded",
    "rateLimits": [
      {
        "key": "request",
        "renewalPeriod": 10,
        "count": 120
      },
      {
        "key": "token",
        "renewalPeriod": 60,
        "count": 120000
      }
    ]
  },
  "systemData": {
    "createdBy": "docs@contoso.com",
    "createdByType": "User",
    "createdAt": "2023-02-28T02:57:15.8951706Z",
    "lastModifiedBy": "docs@contoso.com",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2023-10-31T15:35:53.082912Z"
  },
  "etag": "\"GUID\""
}

Share via

Working with Azure OpenAI models

Model updates

Auto update to default

Specific model version

Viewing retirement dates

Model deployment upgrade configuration

Examples

Example response

Update & deploy models via the API

Example request

Example response

Next steps

Feedback

Feedback

Additional resources