Intelligent applications with Azure SQL Database

Applies to: Azure SQL Database

This article provides an overview of using artificial intelligence (AI) options, such as OpenAI and vectors, to build intelligent applications with Azure SQL Database.

For samples and examples, please visit the SQL AI Samples repository.

Watch this video in the Azure SQL Database essentials series for a brief overview of building an AI ready application:

Overview

Large language models (LLMs) enable developers to create AI-powered applications with a familiar user experience.

Using LLMs in applications brings greater value and an improved user experience when the models can access the right data, at the right time, from your application's database. This process is known as Retrieval Augmented Generation (RAG) and Azure SQL Database has many features that support this new pattern, making it a great database to build intelligent applications.

The following links provide sample code of various Azure SQL Database options to build intelligent applications:

Key concepts

This section includes key concepts that are critical for implementing RAG with Azure SQL Database and Azure OpenAI.

Retrieval Augmented Generation (RAG)

RAG is a technique that enhances the LLM's ability to produce relevant and informative responses by retrieving additional data from external sources. For example, RAG can query articles or documents that contain domain-specific knowledge related to the user's question or prompt. The LLM can then use this retrieved data as a reference when generating its response. For example, a simple RAG pattern using Azure SQL Database could be:

1. Insert data into an Azure SQL Database table.
2. Link Azure SQL Database to Azure AI Search.
3. Create an Azure OpenAI GPT4 model and connect it to Azure AI Search.
4. Chat and ask questions about your data using the trained Azure OpenAI model from your application and from Azure SQL Database.

The RAG pattern, with prompt engineering, serves the purpose of enhancing response quality by offering more contextual information to the model. RAG enables the model to apply a broader knowledgebase by incorporating relevant external sources into the generation process, resulting in more comprehensive and informed responses. For more information on grounding LLMs, see Grounding LLMs - Microsoft Community Hub.

Prompts and prompt engineering

A prompt refers to specific text or information that serves as an instruction to an LLM, or as contextual data that the LLM can build upon. A prompt can take various forms, such as a question, a statement, or even a code snippet.

Sample prompts that can be used to generate a response from an LLM:

• Instructions: provide directives to the LLM
• Primary content: gives information to the LLM for processing
• Examples: help condition the model to a particular task or process
• Cues: direct the LLM's output in the right direction
• Supporting content: represents supplemental information the LLM can use to generate output

The process of creating good prompts for a scenario is called prompt engineering. For more information about prompts and best practices for prompt engineering, see Azure OpenAI Service.

Tokens

Tokens are small chunks of text generated by splitting the input text into smaller segments. These segments can either be words or groups of characters, varying in length from a single character to an entire word. For instance, the word hamburger would be divided into tokens such as ham, bur, and ger while a short and common word like pear would be considered a single token.

In Azure OpenAI, input text provided to the API is turned into tokens (tokenized). The number of tokens processed in each API request depends on factors such as the length of the input, output, and request parameters. The quantity of tokens being processed also impacts the response time and throughput of the models. There are limits to the number of tokens each model can take in a single request/response from Azure OpenAI. To learn more, see Azure OpenAI Service quotas and limits.

Vectors

Vectors are ordered arrays of numbers (typically floats) that can represent information about some data. For example, an image can be represented as a vector of pixel values, or a string of text can be represented as a vector or ASCII values. The process to turn data into a vector is called vectorization.

Embeddings

Embeddings are vectors that represent important features of data. Embeddings are often learned by using a deep learning model, and machine learning and AI models utilize them as features. Embeddings can also capture semantic similarity between similar concepts. For example, in generating an embedding for the words person and human, we would expect their embeddings (vector representation) to be similar in value since the words are also semantically similar.

Azure OpenAI features models to create embeddings from text data. The service breaks text out into tokens and generates embeddings using models pretrained by OpenAI. To learn more, see Creating embeddings with Azure OpenAI.

Vector search

Vector search refers to the process of finding all vectors in a dataset that are semantically similar to a specific query vector. Therefore, a query vector for the word human searches the entire dictionary for semantically similar words, and should find the word person as a close match. This closeness, or distance, is measured using a similarity metric such as cosine similarity. The closer vectors are in similarity, the smaller is the distance between them.

Consider a scenario where you run a query over millions of document to find the most similar documents in your data. You can create embeddings for your data and query documents using Azure OpenAI. Then, you can perform a vector search to find the most similar documents from your dataset. However, performing a vector search across a few examples is trivial. Performing this same search across thousands, or millions, of data points becomes challenging. There are also trade-offs between exhaustive search and approximate nearest neighbor (ANN) search methods including latency, throughput, accuracy, and cost, all of which depends on the requirements of your application.

Since Azure SQL Database embeddings can be efficiently stored and queried using to columnstore index support, allowing exact nearest neighbor search with great performance, you don't have to decide between accuracy and speed: you can have both. Storing vector embeddings alongside the data in an integrated solution minimizes the need to manage data synchronization and accelerates your time-to-market for AI application development.

Azure OpenAI

Embedding is the process of representing the real world as data. Text, images, or sounds can be converted into embeddings. Azure OpenAI models are able to transform real-world information into embeddings. The models are available as REST endpoints and thus can be easily consumed from Azure SQL Database using the sp_invoke_external_rest_endpoint system stored procedure:

DECLARE @retval INT, @response NVARCHAR(MAX);
DECLARE @payload NVARCHAR(MAX);

SET @payload = JSON_OBJECT('input': @text);

EXEC @retval = sp_invoke_external_rest_endpoint @url = 'https://<openai-url>/openai/deployments/<model-name>/embeddings?api-version=2023-03-15-preview',
    @method = 'POST',
    @credential = [https://<openai-url>/openai/deployments/<model-name>],
    @payload = @payload,
    @response = @response OUTPUT;

SELECT CAST([key] AS INT) AS [vector_value_id],
    CAST([value] AS FLOAT) AS [vector_value]
FROM OPENJSON(JSON_QUERY(@response, '$.result.data[0].embedding'));

Using a call to a REST service to get embeddings is just one of the integration options you have when working with SQL Database and OpenAI. You can let any of the available models access data stored in Azure SQL Database to create solutions where your users can interact with the data, such as the following example.

For additional examples on using SQL Database and OpenAI, see the following articles:

• Generate images with Azure OpenAI Service (DALL-E) and Azure SQL Database
• Using OpenAI REST Endpoints with Azure SQL Database

Vectors

Although Azure SQL Database doesn't have a native vector type, a vector is nothing more than an ordered tuple, and relational databases are great at managing tuples. You can think of a tuple as the formal term for a row in a table.

Azure SQL Database also supports columnstore indexes and batch mode execution. A vector-based approach is used for batch mode processing, which means that each column in a batch has its own memory location where it's stored as a vector. This allows for faster and more efficient processing of data in batches.

The following example shows how a vector can be stored in SQL Database:

CREATE TABLE [dbo].[wikipedia_articles_embeddings_titles_vector]
(
    [article_id] [int] NOT NULL,
    [vector_value_id] [int] NOT NULL,
    [vector_value] [float] NOT NULL
)
GO

CREATE CLUSTERED COLUMNSTORE INDEX ixc
    ON dbo.wikipedia_articles_embeddings_titles_vector
    ORDER (article_id);
GO

For an example that uses a common subset of Wikipedia articles with embeddings already generated using OpenAI, see Vector similarity search with Azure SQL Database and OpenAI.

Another option for leveraging Vector Search in Azure SQL database is integration with Azure AI using the integrated vectorization capabilities: Vector Search with Azure SQL Database and Azure AI Search

Azure AI Search

Implement RAG-patterns with Azure SQL Database and Azure AI Search. You can run supported chat models on data stored in Azure SQL Database, without having to train or fine-tune models, thanks to the integration of Azure AI Search with Azure OpenAI and Azure SQL Database. Running models on your data enables you to chat on top of, and analyze, your data with greater accuracy and speed.

• Azure OpenAI on your data
• Retrieval Augmented Generation (RAG) in Azure AI Search
• Vector Search with Azure SQL Database and Azure AI Search

Intelligent applications

Azure SQL Database can be used to build intelligent applications that include AI features, such as recommenders, and Retrieval Augmented Generation (RAG) as the following diagram demonstrates:

For an end-to-end sample to build a AI-enabled application using sessions abstract as a sample dataset, see:

• How I built a session recommender in 1 hour using Open AI.
• Using Retrieval Augmented Generation to build a conference session assistant

LangChain integration

LangChain is a well-known framework for developing applications powered by language models.

For an example that shows how LangChain can be used to create a Chatbot on your own data, see Building your own DB Copilot for Azure SQL with Azure OpenAI GPT-4.

Semantic Kernel integration

Semantic Kernel is an open-source SDK that lets you easily build agents that can call your existing code. As a highly extensible SDK, you can use Semantic Kernel with models from OpenAI, Azure OpenAI, Hugging Face, and more! By combining your existing C#, Python, and Java code with these models, you can build agents that answer questions and automate processes.

• Semantic Kernel & Kernel Memory - SQL Connector - Provides a connection to a SQL database for the Semantic Kernel for the memories.
• OpenAI Embeddings Sample: An example that shows how to use Semantic Kernel and Kernel Memory to work with embeddings in a .NET application using SQL Server as Vector Database.

Microsoft Copilot skills in Azure SQL Database

Microsoft Copilot skills in Azure SQL Database (preview) is a set of AI-assisted experiences designed to streamline the design, operation, optimization, and health of Azure SQL Database-driven applications. Copilot can improve productivity by offering natural language to SQL conversion and self-help for database administration.

Copilot provides relevant answers to user questions, simplifying database management by leveraging database context, documentation, dynamic management views, Query Store, and other knowledge sources. For example:

• Database administrators can independently manage databases and resolve issues, or learn more about the performance and capabilities of your database.
• Developers can ask questions about their data as they would in text or conversation to generate a T-SQL query. Developers can also learn to write queries faster through detailed explanations of the generated query.

The preview of Copilot for Azure SQL Database includes two Azure portal experiences:

For more information, see Frequently asked questions about Microsoft Copilot skills in Azure SQL Database (preview).

Related content

• Create and deploy an Azure OpenAI Service resource
• Embeddings models
• SQL AI Samples and Examples
• Frequently asked questions about Microsoft Copilot skills in Azure SQL Database (preview)
• Responsible AI FAQ for Microsoft Copilot for Azure (preview)