Build resilient HTTP apps: Key development patterns

Building robust HTTP apps that can recover from transient fault errors is a common requirement. This article assumes that you've already read Introduction to resilient app development, as this article extends the core concepts conveyed. To help build resilient HTTP apps, the Microsoft.Extensions.Http.Resilience NuGet package provides resilience mechanisms specifically for the HttpClient. This NuGet package relies on the Microsoft.Extensions.Resilience library and Polly, which is a popular open-source project. For more information, see Polly.

Get started

To use resilience-patterns in HTTP apps, install the Microsoft.Extensions.Http.Resilience NuGet package.

dotnet add package Microsoft.Extensions.Http.Resilience --version 8.0.0

<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="8.0.0" />

For more information, see dotnet add package or Manage package dependencies in .NET applications.

Add resilience to an HTTP client

To add resilience to an HttpClient, you chain a call on the IHttpClientBuilder type that is returned from calling any of the available AddHttpClient methods. For more information, see IHttpClientFactory with .NET.

There are several resilience-centric extensions available. Some are standard, thus employing various industry best practices, and others are more customizable. When adding resilience, you should only add one resilience handler and avoid stacking handlers. If you need to add multiple resilience handlers, you should consider using the AddResilienceHandler extension method, which allows you to customize the resilience strategies.

Add standard resilience handler

The standard resilience handler uses multiple resilience strategies stacked atop one another, with default options to send the requests and handle any transient errors. The standard resilience handler is added by calling the AddStandardResilienceHandler extension method on an IHttpClientBuilder instance.

var services = new ServiceCollection();

var httpClientBuilder = services.AddHttpClient<ExampleClient>(
    configureClient: static client =>
    {
        client.BaseAddress = new("https://jsonplaceholder.typicode.com");
    });

The preceding code:

• Creates a ServiceCollection instance.
• Adds an HttpClient for the ExampleClient type to the service container.
• Configures the HttpClient to use "https://jsonplaceholder.typicode.com" as the base address.
• Creates the httpClientBuilder that's used throughout the other examples within this article.

A more real-world example would rely on hosting, such as that described in the .NET Generic Host article. Using the Microsoft.Extensions.Hosting NuGet package, consider the following updated example:

using Http.Resilience.Example;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

HostApplicationBuilder builder = Host.CreateApplicationBuilder(args);

IHttpClientBuilder httpClientBuilder = builder.Services.AddHttpClient<ExampleClient>(
    configureClient: static client =>
    {
        client.BaseAddress = new("https://jsonplaceholder.typicode.com");
    });

The preceding code is similar to the manual ServiceCollection creation approach, but instead relies on the Host.CreateApplicationBuilder() to build out a host that exposes the services.

The ExampleClient is defined as follows:

using System.Net.Http.Json;

namespace Http.Resilience.Example;

/// <summary>
/// An example client service, that relies on the <see cref="HttpClient"/> instance.
/// </summary>
/// <param name="client">The given <see cref="HttpClient"/> instance.</param>
internal sealed class ExampleClient(HttpClient client)
{
    /// <summary>
    /// Returns an <see cref="IAsyncEnumerable{T}"/> of <see cref="Comment"/>s.
    /// </summary>
    public IAsyncEnumerable<Comment?> GetCommentsAsync()
    {
        return client.GetFromJsonAsAsyncEnumerable<Comment>("/comments");
    }
}

The preceding code:

• Defines an ExampleClient type that has a constructor that accepts an HttpClient.
• Exposes a GetCommentsAsync method that sends a GET request to the /comments endpoint and returns the response.

The Comment type is defined as follows:

namespace Http.Resilience.Example;

public record class Comment(
    int PostId, int Id, string Name, string Email, string Body);

Given that you've created an IHttpClientBuilder (httpClientBuilder), and you now understand the ExampleClient implementation and corresponding Comment model, consider the following example:

httpClientBuilder.AddStandardResilienceHandler();

The preceding code adds the standard resilience handler to the HttpClient. Like most resilience APIs, there are overloads that allow you to customize the default options and applied resilience strategies.

Standard resilience handler defaults

The default configuration chains five resilience strategies in the following order (from the outermost to the innermost):

Add standard hedging handler

The standard hedging handler wraps the execution of the request with a standard hedging mechanism. Hedging retries slow requests in parallel.

To use the standard hedging handler, call AddStandardHedgingHandler extension method. The following example configures the ExampleClient to use the standard hedging handler.

httpClientBuilder.AddStandardHedgingHandler();

The preceding code adds the standard hedging handler to the HttpClient.

Standard hedging handler defaults

The standard hedging uses a pool of circuit breakers to ensure that unhealthy endpoints aren't hedged against. By default, the selection from the pool is based on the URL authority (scheme + host + port).

The preceding code adds the standard hedging handler to the IHttpClientBuilder. The default configuration chains five resilience strategies in the following order (from the outermost to the innermost):

Customize hedging handler route selection

When using the standard hedging handler, you can customize the way the request endpoints are selected by calling various extensions on the IRoutingStrategyBuilder type. This can be useful for scenarios such as A/B testing, where you want to route a percentage of the requests to a different endpoint:

httpClientBuilder.AddStandardHedgingHandler(static (IRoutingStrategyBuilder builder) =>
{
    // Hedging allows sending multiple concurrent requests
    builder.ConfigureOrderedGroups(static options =>
    {
        options.Groups.Add(new UriEndpointGroup()
        {
            Endpoints =
            {
                // Imagine a scenario where 3% of the requests are 
                // sent to the experimental endpoint.
                new() { Uri = new("https://example.net/api/experimental"), Weight = 3 },
                new() { Uri = new("https://example.net/api/stable"), Weight = 97 }
            }
        });
    });
});

The preceding code:

• Adds the hedging handler to the IHttpClientBuilder.
• Configures the IRoutingStrategyBuilder to use the ConfigureOrderedGroups method to configure the ordered groups.
• Adds an EndpointGroup to the orderedGroup that routes 3% of the requests to the https://example.net/api/experimental endpoint and 97% of the requests to the https://example.net/api/stable endpoint.
• Configures the IRoutingStrategyBuilder to use the ConfigureWeightedGroups method to configure the

To configure a weighted group, call the ConfigureWeightedGroups method on the IRoutingStrategyBuilder type. The following example configures the IRoutingStrategyBuilder to use the ConfigureWeightedGroups method to configure the weighted groups.

httpClientBuilder.AddStandardHedgingHandler(static (IRoutingStrategyBuilder builder) =>
{
    // Hedging allows sending multiple concurrent requests
    builder.ConfigureWeightedGroups(static options =>
    {
        options.SelectionMode = WeightedGroupSelectionMode.EveryAttempt;

        options.Groups.Add(new WeightedUriEndpointGroup()
        {
            Endpoints =
            {
                // Imagine A/B testing
                new() { Uri = new("https://example.net/api/a"), Weight = 33 },
                new() { Uri = new("https://example.net/api/b"), Weight = 33 },
                new() { Uri = new("https://example.net/api/c"), Weight = 33 }
            }
        });
    });
});

The preceding code:

• Adds the hedging handler to the IHttpClientBuilder.
• Configures the IRoutingStrategyBuilder to use the ConfigureWeightedGroups method to configure the weighted groups.
• Sets the SelectionMode to WeightedGroupSelectionMode.EveryAttempt.
• Adds a WeightedEndpointGroup to the weightedGroup that routes 33% of the requests to the https://example.net/api/a endpoint, 33% of the requests to the https://example.net/api/b endpoint, and 33% of the requests to the https://example.net/api/c endpoint.

For more information, see Polly docs: Hedging resilience strategy.

It's common to configure either an ordered group or weighted group, but it's valid to configure both. Using ordered and weighted groups is helpful in scenarios where you want to send a percentage of the requests to a different endpoint, such is the case with A/B testing.

Add custom resilience handlers

To have more control, you can customize the resilience handlers by using the AddResilienceHandler API. This method accepts a delegate that configures the ResiliencePipelineBuilder<HttpResponseMessage> instance that is used to create the resilience strategies.

To configure a named resilience handler, call the AddResilienceHandler extension method with the name of the handler. The following example configures a named resilience handler called "CustomPipeline".

httpClientBuilder.AddResilienceHandler(
    "CustomPipeline",
    static builder =>
{
    // See: https://www.pollydocs.org/strategies/retry.html
    builder.AddRetry(new HttpRetryStrategyOptions
    {
        // Customize and configure the retry logic.
        BackoffType = DelayBackoffType.Exponential,
        MaxRetryAttempts = 5,
        UseJitter = true
    });

    // See: https://www.pollydocs.org/strategies/circuit-breaker.html
    builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
    {
        // Customize and configure the circuit breaker logic.
        SamplingDuration = TimeSpan.FromSeconds(10),
        FailureRatio = 0.2,
        MinimumThroughput = 3,
        ShouldHandle = static args =>
        {
            return ValueTask.FromResult(args is
            {
                Outcome.Result.StatusCode:
                    HttpStatusCode.RequestTimeout or
                        HttpStatusCode.TooManyRequests
            });
        }
    });

    // See: https://www.pollydocs.org/strategies/timeout.html
    builder.AddTimeout(TimeSpan.FromSeconds(5));
});

The preceding code:

• Adds a resilience handler with the name "CustomPipeline" as the pipelineName to the service container.
• Adds a retry strategy with exponential backoff, five retries, and jitter preference to the resilience builder.
• Adds a circuit breaker strategy with a sampling duration of 10 seconds, a failure ratio of 0.2 (20%), a minimum throughput of three, and a predicate that handles RequestTimeout and TooManyRequests HTTP status codes to the resilience builder.
• Adds a timeout strategy with a timeout of five seconds to the resilience builder.

There are many options available for each of the resilience strategies. For more information, see the Polly docs: Strategies. For more information about configuring ShouldHandle delegates, see Polly docs: Fault handling in reactive strategies.

Dynamic reload

Polly supports dynamic reloading of the configured resilience strategies. This means that you can change the configuration of the resilience strategies at run time. To enable dynamic reload, use the appropriate AddResilienceHandler overload that exposes the ResilienceHandlerContext. Given the context, call EnableReloads of the corresponding resilience strategy options:

httpClientBuilder.AddResilienceHandler(
    "AdvancedPipeline",
    static (ResiliencePipelineBuilder<HttpResponseMessage> builder,
        ResilienceHandlerContext context) =>
    {
        // Enable reloads whenever the named options change
        context.EnableReloads<HttpRetryStrategyOptions>("RetryOptions");

        // Retrieve the named options
        var retryOptions =
            context.GetOptions<HttpRetryStrategyOptions>("RetryOptions");

        // Add retries using the resolved options
        builder.AddRetry(retryOptions);
    });

The preceding code:

• Adds a resilience handler with the name "AdvancedPipeline" as the pipelineName to the service container.
• Enables the reloads of the "AdvancedPipeline" pipeline whenever the named RetryStrategyOptions options change.
• Retrieves the named options from the IOptionsMonitor<TOptions> service.
• Adds a retry strategy with the retrieved options to the resilience builder.

For more information, see Polly docs: Advanced dependency injection.

This example relies on an options section that is capable of change, such as an appsettings.json file. Consider the following appsettings.json file:

{
    "RetryOptions": {
        "Retry": {
            "BackoffType": "Linear",
            "UseJitter": false,
            "MaxRetryAttempts": 7
        }
    }
}

Now imagine that these options were bound to the app's configuration, binding the HttpRetryStrategyOptions to the "RetryOptions" section:

var section = builder.Configuration.GetSection("RetryOptions");

builder.Services.Configure<HttpStandardResilienceOptions>(section);

For more information, see Options pattern in .NET.

Example usage

Your app relies on dependency injection to resolve the ExampleClient and its corresponding HttpClient. The code builds the IServiceProvider and resolves the ExampleClient from it.

IHost host = builder.Build();

ExampleClient client = host.Services.GetRequiredService<ExampleClient>();

await foreach (Comment? comment in client.GetCommentsAsync())
{
    Console.WriteLine(comment);
}

The preceding code:

• Builds the IServiceProvider from the ServiceCollection.
• Resolves the ExampleClient from the IServiceProvider.
• Calls the GetCommentsAsync method on the ExampleClient to get the comments.
• Writes each comment to the console.

Imagine a situation where the network goes down or the server becomes unresponsive. The following diagram shows how the resilience strategies would handle the situation, given the ExampleClient and the GetCommentsAsync method:

The preceding diagram depicts:

• The ExampleClient sends an HTTP GET request to the /comments endpoint.
• The HttpResponseMessage is evaluated:
  • If the response is successful (HTTP 200), the response is returned.
  • If the response is unsuccessful (HTTP non-200), the resilience pipeline employs the configured resilience strategies.

While this is a simple example, it demonstrates how the resilience strategies can be used to handle transient errors. For more information, see Polly docs: Strategies.