Import an Azure OpenAI API
APPLIES TO: All API Management tiers
This article shows two options to import an Azure OpenAI Service API into an Azure API Management instance as a REST API:
- Import an Azure OpenAI API directly from Azure OpenAI Service
- Download and add the OpenAPI specification for Azure OpenAI and add it to API Management as an OpenAPI API.
Prerequisites
An existing API Management instance. Create one if you haven't already.
Access granted to Azure OpenAI in the desired Azure subscription. You can apply for access to Azure OpenAI by completing the form at https://aka.ms/oai/access. Open an issue on this repo to contact us if you have an issue.
An Azure OpenAI resource with a model deployed. For more information about model deployment, see the resource deployment guide.
Make a note of the ID (name) of the deployment. You'll need it when you test the imported API in API Management.
Permissions to grant access to the Azure OpenAI resource from the API Management instance.
Option 1. Import API from Azure OpenAI Service
You can import an Azure OpenAI API directly to API Management from the Azure OpenAI Service. When you import the API, API Management automatically configures:
- Operations for each of the Azure OpenAI REST API endpoints.
- A system-assigned identity with the necessary permissions to access the Azure OpenAI resource.
- A backend resource and set-backend-service policy that direct API requests to the Azure OpenAI Service endpoint.
- An authentication-managed-identity policy that can authenticate to the Azure OpenAI resource using the instance's system-assigned identity.
- (optionally) Policies to help you monitor and manage token consumption by the Azure OpenAI API.
To import an Azure OpenAI API to API Management:
In the Azure portal, navigate to your API Management instance.
In the left menu, under APIs, select APIs > + Add API.
Under Create from Azure resource, select Azure OpenAI Service.
On the Basics tab:
Select the Azure OpenAI resource that you want to import.
Optionally select an Azure OpenAI API version. If you don't select one, the latest production-ready REST API version is used by default.
Enter a Display name and optional Description for the API.
In Base URL, append a path that your API Management instance uses to access the Azure OpenAI API endpoints. If you enable Ensure OpenAI SDK compatibility (recommended),
/openai
is automatically appended to the base URL.For example, if your API Management gateway endpoint is
https://contoso.azure-api.net
, set a Base URL similar tohttps://contoso.azure-api.net/my-openai-api/openai
.Optionally select one or more products to associate with the API. Select Next.
On the Policies tab, optionally enable policies to monitor and manage Azure OpenAI API token consumption. If selected, enter settings or accept defaults that define the
azure-openai-token-limit
andazure-openai-emit-token-metric
policies for your API. You can also set or update the policy configuration later. Select Review + Create.After settings are validated, select Create.
Option 2. Add an OpenAPI specification to API Management
Alternatively, manually download the OpenAPI specification for the Azure OpenAI REST API and add it to API Management as an OpenAPI API.
Download the OpenAPI specification
Download the OpenAPI specification for the Azure OpenAI REST API, such as the 2024-02-01 GA version.
In a text editor, open the specification file that you downloaded.
In the
servers
element in the specification, substitute the name of your Azure OpenAI Service endpoint in the placeholder values ofurl
anddefault
endpoint in the specification. For example, if your Azure OpenAI Service endpoint iscontoso.openai.azure.com
, update theservers
element with the following values:- url:
https://contoso.openai.azure.com/openai
- default endpoint:
contoso.openai.azure.com
[...] "servers": [ { "url": "https://contoso.openai.azure.com/openai", "variables": { "endpoint": { "default": "contoso.openai.azure.com" } } } ], [...]
- url:
Make a note of the value of the API
version
in the specification. You'll need it to test the API. Example:2024-02-01
.
Add OpenAPI specification to API Management
- In the Azure portal, navigate to your API Management instance.
- In the left menu, select APIs > + Add API.
- Under Define a new API, select OpenAPI. Enter a Display name and Name for the API.
- Enter an API URL suffix ending in
/openai
to access the Azure OpenAI API endpoints in your API Management instance. Example:my-openai-api/openai
. - Select Create.
The API is imported and displays operations from the OpenAPI specification.
Configure authentication to Azure OpenAI API
To authenticate to the Azure OpenAI API, you supply an API key or a managed identity. If you imported the Azure OpenAI API directly to your API Management instance, authentication using the API Management instance's managed identity is automatically configured.
If you added the Azure OpenAI API from its OpenAPI specification, you need to configure authentication. For more information about configuring authentication using API Management policies, see Authenticate and authorize to Azure OpenAI API.
Test the Azure OpenAI API
To ensure that your Azure OpenAI API is working as expected, test it in the API Management test console. You need to supply a model deployment ID (name) configured in the Azure OpenAI resource to test the API.
Select the API you created in the previous step.
Select the Test tab.
Select an operation that's compatible with the model you deployed in the Azure OpenAI resource. The page displays fields for parameters and headers.
In Template parameters, enter the following values:
Enter other parameters and headers as needed. Depending on the operation, you may need to configure or update a Request body.
Note
In the test console, API Management automatically populates an Ocp-Apim-Subscription-Key header, and configures the subscription key of the built-in all-access subscription. This key enables access to every API in the API Management instance. Optionally display the Ocp-Apim-Subscription-Key header by selecting the "eye" icon next to the HTTP Request.
Select Send.
When the test is successful, the backend responds with a successful HTTP response code and some data. Appended to the response is token usage data to help you monitor and manage your Azure OpenAI API consumption.
Caching policies for Azure OpenAI APIs
In addition to the azure-openai-token-limit
and azure-openai-emit-token-metric
policies that you can configure when importing an Azure OpenAI Service API, API Management provides the following caching policies to help you optimize performance and reduce latency for Azure OpenAI APIs:
azure-openai-semantic-cache-store
azure-openai-semantic-cache-lookup
Related content
- API Management policy reference
- Azure OpenAI Service as a central capability with Azure API Management
- Azure API Management - Azure OpenAI sample
Related topics
- API import limitations
- Import an OpenAPI specification
- Import a SOAP API
- Import a SOAP API and convert to REST
- Import an App Service API
- Import a Container App API
- Import a WebSocket API
- Import a GraphQL API
- Import a GraphQL schema and set up field resolvers
- Import an Azure Function App
- Import an Azure Logic App
- Import a Service Fabric service
- Import an Azure OpenAI API
- Import an OData API
- Import SAP OData metadata
- Import a gRPC API
- Edit an API
Feedback
https://aka.ms/ContentUserFeedback.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see:Submit and view feedback for