Application Insights for Worker Service applications (non-HTTP applications)

Application Insights SDK for Worker Service is a new SDK, which is best suited for non-HTTP workloads like messaging, background tasks, and console applications. These types of applications don't have the notion of an incoming HTTP request like a traditional ASP.NET/ASP.NET Core web application. For this reason, using Application Insights packages for ASP.NET or ASP.NET Core applications isn't supported.

The new SDK doesn't do any telemetry collection by itself. Instead, it brings in other well-known Application Insights auto collectors like DependencyCollector, PerfCounterCollector, and ApplicationInsightsLoggingProvider. This SDK exposes extension methods on IServiceCollection to enable and configure telemetry collection.

Supported scenarios

The Application Insights SDK for Worker Service is best suited for non-HTTP applications no matter where or how they run. If your application is running and has network connectivity to Azure, telemetry can be collected. Application Insights monitoring is supported everywhere .NET Core is supported. This package can be used in the newly introduced .NET Core Worker Service, background tasks in ASP.NET Core, and console apps like .NET Core and .NET Framework.

Prerequisites

You must have a valid Application Insights connection string. This string is required to send any telemetry to Application Insights. If you need to create a new Application Insights resource to get a connection string, see Connection Strings.

Use Application Insights SDK for Worker Service

1. Install the Microsoft.ApplicationInsights.WorkerService package to the application. The following snippet shows the changes that must be added to your project's .csproj file:

       <ItemGroup>
           <PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.22.0" />
       </ItemGroup>
   

2. Configure the connection string in the APPLICATIONINSIGHTS_CONNECTION_STRING environment variable or in configuration (appsettings.json).

   

3. Retrieve an ILogger instance or TelemetryClient instance from the Dependency Injection (DI) container by calling serviceProvider.GetRequiredService<TelemetryClient>(); or by using Constructor Injection. This step will trigger setting up of TelemetryConfiguration and auto-collection modules.

Specific instructions for each type of application are described in the following sections.

.NET Core Worker Service application

The full example is shared at the NuGet website.

1. Download and install the .NET SDK.

2. Create a new Worker Service project either by using a Visual Studio new project template or the command line dotnet new worker.

3. Add the Microsoft.ApplicationInsights.WorkerService package to the application.

4. Add services.AddApplicationInsightsTelemetryWorkerService(); to the CreateHostBuilder() method in your Program.cs class, as in this example:

       public static IHostBuilder CreateHostBuilder(string[] args) =>
           Host.CreateDefaultBuilder(args)
               .ConfigureServices((hostContext, services) =>
               {
                   services.AddHostedService<Worker>();
                   services.AddApplicationInsightsTelemetryWorkerService();
               });
   

5. Modify your Worker.cs as per the following example:

       using Microsoft.ApplicationInsights;
       using Microsoft.ApplicationInsights.DataContracts;
   
       public class Worker : BackgroundService
       {
           private readonly ILogger<Worker> _logger;
           private TelemetryClient _telemetryClient;
           private static HttpClient _httpClient = new HttpClient();
   
           public Worker(ILogger<Worker> logger, TelemetryClient tc)
           {
               _logger = logger;
               _telemetryClient = tc;
           }
   
           protected override async Task ExecuteAsync(CancellationToken stoppingToken)
           {
               while (!stoppingToken.IsCancellationRequested)
               {
                   _logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
   
                   using (_telemetryClient.StartOperation<RequestTelemetry>("operation"))
                   {
                       _logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher is captured by Application Insights");
                       _logger.LogInformation("Calling bing.com");
                       var res = await _httpClient.GetAsync("https://bing.com");
                       _logger.LogInformation("Calling bing completed with status:" + res.StatusCode);
                       _telemetryClient.TrackEvent("Bing call event completed");
                   }
   
                   await Task.Delay(1000, stoppingToken);
               }
           }
       }
   

6. Set up the connection string.

   

   Note

   We recommend that you specify the connection string in configuration. The following code sample shows how to specify a connection string in appsettings.json. Make sure appsettings.json is copied to the application root folder during publishing.

       {
           "ApplicationInsights":
           {
               "ConnectionString" : "InstrumentationKey=00000000-0000-0000-0000-000000000000;"
           },
           "Logging":
           {
               "LogLevel":
               {
                   "Default": "Warning"
               }
           }
       }
   

Alternatively, specify the connection string in the APPLICATIONINSIGHTS_CONNECTION_STRING environment variable.

Typically, APPLICATIONINSIGHTS_CONNECTION_STRING specifies the connection string for applications deployed to web apps as web jobs.

ASP.NET Core background tasks with hosted services

This document describes how to create background tasks in an ASP.NET Core application.

The full example is shared at this GitHub page.

1. Install the Microsoft.ApplicationInsights.WorkerService package to the application.

2. Add services.AddApplicationInsightsTelemetryWorkerService(); to the ConfigureServices() method, as in this example:

       public static async Task Main(string[] args)
       {
           var host = new HostBuilder()
               .ConfigureAppConfiguration((hostContext, config) =>
               {
                   config.AddJsonFile("appsettings.json", optional: true);
               })
               .ConfigureServices((hostContext, services) =>
               {
                   services.AddLogging();
                   services.AddHostedService<TimedHostedService>();
   
                   // connection string is read automatically from appsettings.json
                   services.AddApplicationInsightsTelemetryWorkerService();
               })
               .UseConsoleLifetime()
               .Build();
   
           using (host)
           {
               // Start the host
               await host.StartAsync();
   
               // Wait for the host to shutdown
               await host.WaitForShutdownAsync();
           }
       }
   

   The following code is for TimedHostedService, where the background task logic resides:

       using Microsoft.ApplicationInsights;
       using Microsoft.ApplicationInsights.DataContracts;
   
       public class TimedHostedService : IHostedService, IDisposable
       {
           private readonly ILogger _logger;
           private Timer _timer;
           private TelemetryClient _telemetryClient;
           private static HttpClient httpClient = new HttpClient();
   
           public TimedHostedService(ILogger<TimedHostedService> logger, TelemetryClient tc)
           {
               _logger = logger;
               this._telemetryClient = tc;
           }
   
           public Task StartAsync(CancellationToken cancellationToken)
           {
               _logger.LogInformation("Timed Background Service is starting.");
   
               _timer = new Timer(DoWork, null, TimeSpan.Zero,
                   TimeSpan.FromSeconds(1));
   
               return Task.CompletedTask;
           }
   
           private void DoWork(object state)
           {
               _logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
   
               using (_telemetryClient.StartOperation<RequestTelemetry>("operation"))
               {
                   _logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher is captured by Application Insights");
                   _logger.LogInformation("Calling bing.com");
                   var res = httpClient.GetAsync("https://bing.com").GetAwaiter().GetResult();
                   _logger.LogInformation("Calling bing completed with status:" + res.StatusCode);
                   _telemetryClient.TrackEvent("Bing call event completed");
               }
           }
       }
   

3. Set up the connection string. Use the same appsettings.json from the preceding .NET Worker Service example.

.NET Core/.NET Framework console application

As mentioned in the beginning of this article, the new package can be used to enable Application Insights telemetry from even a regular console application. This package targets netstandard2.0, so it can be used for console apps in .NET Core or higher, and .NET Framework or higher.

The full example is shared at this GitHub page.

1. Install the Microsoft.ApplicationInsights.WorkerService package to the application.

2. Modify Program.cs as shown in the following example:

       using Microsoft.ApplicationInsights;
       using Microsoft.ApplicationInsights.DataContracts;
       using Microsoft.ApplicationInsights.WorkerService;
       using Microsoft.Extensions.DependencyInjection;
       using Microsoft.Extensions.Logging;
       using System;
       using System.Net.Http;
       using System.Threading.Tasks;
   
       namespace WorkerSDKOnConsole
       {
           class Program
           {
               static async Task Main(string[] args)
               {
                   // Create the DI container.
                   IServiceCollection services = new ServiceCollection();
   
                   // Being a regular console app, there is no appsettings.json or configuration providers enabled by default.
                   // Hence instrumentation key/ connection string and any changes to default logging level must be specified here.
                   services.AddLogging(loggingBuilder => loggingBuilder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>("Category", LogLevel.Information));
                   services.AddApplicationInsightsTelemetryWorkerService((ApplicationInsightsServiceOptions options) => options.ConnectionString = "InstrumentationKey=<instrumentation key here>");
   
                   // To pass a connection string
                   // - aiserviceoptions must be created
                   // - set connectionstring on it
                   // - pass it to AddApplicationInsightsTelemetryWorkerService()
   
                   // Build ServiceProvider.
                   IServiceProvider serviceProvider = services.BuildServiceProvider();
   
                   // Obtain logger instance from DI.
                   ILogger<Program> logger = serviceProvider.GetRequiredService<ILogger<Program>>();
   
                   // Obtain TelemetryClient instance from DI, for additional manual tracking or to flush.
                   var telemetryClient = serviceProvider.GetRequiredService<TelemetryClient>();
   
                   var httpClient = new HttpClient();
   
                   while (true) // This app runs indefinitely. Replace with actual application termination logic.
                   {
                       logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
   
                       // Replace with a name which makes sense for this operation.
                       using (telemetryClient.StartOperation<RequestTelemetry>("operation"))
                       {
                           logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher is captured by Application Insights");
                           logger.LogInformation("Calling bing.com");                    
                           var res = await httpClient.GetAsync("https://bing.com");
                           logger.LogInformation("Calling bing completed with status:" + res.StatusCode);
                           telemetryClient.TrackEvent("Bing call event completed");
                       }
   
                       await Task.Delay(1000);
                   }
   
                   // Explicitly call Flush() followed by sleep is required in console apps.
                   // This is to ensure that even if application terminates, telemetry is sent to the back-end.
                   telemetryClient.Flush();
                   Task.Delay(5000).Wait();
               }
           }
       }
   

This console application also uses the same default TelemetryConfiguration. It can be customized in the same way as the examples in earlier sections.

Run your application

Run your application. The workers from all the preceding examples make an HTTP call every second to bing.com and also emit few logs by using ILogger. These lines are wrapped inside the StartOperation call of TelemetryClient, which is used to create an operation. In this example, RequestTelemetry is named "operation."

Application Insights collects these ILogger logs, with a severity of Warning or above by default, and dependencies. They're correlated to RequestTelemetry with a parent-child relationship. Correlation also works across process/network boundaries. For example, if the call was made to another monitored component, it's correlated to this parent as well.

This custom operation of RequestTelemetry can be thought of as the equivalent of an incoming web request in a typical web application. It isn't necessary to use an operation, but it fits best with the Application Insights correlation data model. RequestTelemetry acts as the parent operation and every telemetry generated inside the worker iteration is treated as logically belonging to the same operation.

This approach also ensures all the telemetry generated, both automatic and manual, will have the same operation_id. Because sampling is based on operation_id, the sampling algorithm either keeps or drops all the telemetry from a single iteration.

The following sections list the full telemetry automatically collected by Application Insights.

Live metrics

Live metrics can be used to quickly verify if application monitoring with Application Insights is configured correctly. Telemetry can take a few minutes to appear in the Azure portal, but the live metrics pane shows CPU usage of the running process in near real time. It can also show other telemetry like requests, dependencies, and traces.

ILogger logs

Logs emitted via ILogger with the severity Warning or greater are automatically captured. To change this behavior, explicitly override the logging configuration for the provider ApplicationInsights, as shown in the following code. The following configuration allows Application Insights to capture all Information logs and more severe logs.

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    },
    "ApplicationInsights": {
      "LogLevel": {
        "Default": "Information"
      }
    }
  }
}

It's important to note that the following example doesn't cause the Application Insights provider to capture Information logs. It doesn't capture it because the SDK adds a default logging filter that instructs ApplicationInsights to capture only Warning logs and more severe logs. Application Insights requires an explicit override.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information"
    }
  }
}

For more information, follow ILogger docs to customize which log levels are captured by Application Insights.

Dependencies

Dependency collection is enabled by default. The article Dependency tracking in Application Insights explains the dependencies that are automatically collected and also contains steps to do manual tracking.

EventCounter

EventCounterCollectionModule is enabled by default, and it will collect a default set of counters from .NET apps. The EventCounter tutorial lists the default set of counters collected. It also has instructions on how to customize the list.

Manually track other telemetry

Although the SDK automatically collects telemetry as explained, in most cases, you'll need to send other telemetry to Application Insights. The recommended way to track other telemetry is by obtaining an instance of TelemetryClient from Dependency Injection and then calling one of the supported TrackXXX() API methods on it. Another typical use case is custom tracking of operations. This approach is demonstrated in the preceding worker examples.

Configure the Application Insights SDK

The default TelemetryConfiguration used by the Worker Service SDK is similar to the automatic configuration used in an ASP.NET or ASP.NET Core application, minus the telemetry initializers used to enrich telemetry from HttpContext.

You can customize the Application Insights SDK for Worker Service to change the default configuration. Users of the Application Insights ASP.NET Core SDK might be familiar with changing configuration by using ASP.NET Core built-in dependency injection. The Worker Service SDK is also based on similar principles. Make almost all configuration changes in the ConfigureServices() section by calling appropriate methods on IServiceCollection, as detailed in the next section.

Use ApplicationInsightsServiceOptions

You can modify a few common settings by passing ApplicationInsightsServiceOptions to AddApplicationInsightsTelemetryWorkerService, as in this example:

using Microsoft.ApplicationInsights.WorkerService;

public void ConfigureServices(IServiceCollection services)
{
    var aiOptions = new ApplicationInsightsServiceOptions();
    // Disables adaptive sampling.
    aiOptions.EnableAdaptiveSampling = false;

    // Disables live metrics (also known as QuickPulse).
    aiOptions.EnableQuickPulseMetricStream = false;
    services.AddApplicationInsightsTelemetryWorkerService(aiOptions);
}

The ApplicationInsightsServiceOptions in this SDK is in the namespace Microsoft.ApplicationInsights.WorkerService as opposed to Microsoft.ApplicationInsights.AspNetCore.Extensions in the ASP.NET Core SDK.

The following table lists commonly used settings in ApplicationInsightsServiceOptions.

For the most up-to-date list, see the configurable settings in ApplicationInsightsServiceOptions.

Sampling

The Application Insights SDK for Worker Service supports both fixed-rate sampling and adaptive sampling. Adaptive sampling is enabled by default. Sampling can be disabled by using the EnableAdaptiveSampling option in ApplicationInsightsServiceOptions.

To configure other sampling settings, you can use the following example:

using Microsoft.ApplicationInsights.AspNetCore.Extensions;
using Microsoft.ApplicationInsights.Extensibility;

var builder = WebApplication.CreateBuilder(args);

builder.Services.Configure<TelemetryConfiguration>(telemetryConfiguration =>
{
   var telemetryProcessorChainBuilder = telemetryConfiguration.DefaultTelemetrySink.TelemetryProcessorChainBuilder;

   // Using adaptive sampling
   telemetryProcessorChainBuilder.UseAdaptiveSampling(maxTelemetryItemsPerSecond: 5);

   // Alternately, the following configures adaptive sampling with 5 items per second, and also excludes DependencyTelemetry from being subject to sampling:
   // telemetryProcessorChainBuilder.UseAdaptiveSampling(maxTelemetryItemsPerSecond:5, excludedTypes: "Dependency");
});

builder.Services.AddApplicationInsightsTelemetry(new ApplicationInsightsServiceOptions
{
   EnableAdaptiveSampling = false,
});

var app = builder.Build();

For more information, see the Sampling document.

Add telemetry initializers

Use telemetry initializers when you want to define properties that are sent with all telemetry.

Add any new telemetry initializer to the DependencyInjection container and the SDK automatically adds them to TelemetryConfiguration.

    using Microsoft.ApplicationInsights.Extensibility;

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddSingleton<ITelemetryInitializer, MyCustomTelemetryInitializer>();
        services.AddApplicationInsightsTelemetryWorkerService();
    }

Remove telemetry initializers

Telemetry initializers are present by default. To remove all or specific telemetry initializers, use the following sample code after calling AddApplicationInsightsTelemetryWorkerService().

   public void ConfigureServices(IServiceCollection services)
   {
        services.AddApplicationInsightsTelemetryWorkerService();
        // Remove a specific built-in telemetry initializer.
        var tiToRemove = services.FirstOrDefault<ServiceDescriptor>
                            (t => t.ImplementationType == typeof(AspNetCoreEnvironmentTelemetryInitializer));
        if (tiToRemove != null)
        {
            services.Remove(tiToRemove);
        }

        // Remove all initializers.
        // This requires importing namespace by using Microsoft.Extensions.DependencyInjection.Extensions;
        services.RemoveAll(typeof(ITelemetryInitializer));
   }

Add telemetry processors

You can add custom telemetry processors to TelemetryConfiguration by using the extension method AddApplicationInsightsTelemetryProcessor on IServiceCollection. You use telemetry processors in advanced filtering scenarios to allow for more direct control over what's included or excluded from the telemetry you send to Application Insights. Use the following example:

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddApplicationInsightsTelemetryWorkerService();
        services.AddApplicationInsightsTelemetryProcessor<MyFirstCustomTelemetryProcessor>();
        // If you have more processors:
        services.AddApplicationInsightsTelemetryProcessor<MySecondCustomTelemetryProcessor>();
    }

Configure or remove default telemetry modules

Application Insights uses telemetry modules to automatically collect telemetry about specific workloads without requiring manual tracking.

The following auto-collection modules are enabled by default. These modules are responsible for automatically collecting telemetry. You can disable or configure them to alter their default behavior.

• DependencyTrackingTelemetryModule
• PerformanceCollectorModule
• QuickPulseTelemetryModule
• AppServicesHeartbeatTelemetryModule (There's currently an issue involving this telemetry module. For a temporary workaround, see GitHub Issue 1689.)
• AzureInstanceMetadataTelemetryModule

To configure any default telemetry module, use the extension method ConfigureTelemetryModule<T> on IServiceCollection, as shown in the following example:

    using Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse;
    using Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector;

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddApplicationInsightsTelemetryWorkerService();

            // The following configures QuickPulseTelemetryModule.
            // Similarly, any other default modules can be configured.
            services.ConfigureTelemetryModule<QuickPulseTelemetryModule>((module, o) =>
            {
                module.AuthenticationApiKey = "keyhere";
            });

            // The following removes PerformanceCollectorModule to disable perf-counter collection.
            // Similarly, any other default modules can be removed.
            var performanceCounterService = services.FirstOrDefault<ServiceDescriptor>
                                        (t => t.ImplementationType == typeof(PerformanceCollectorModule));
            if (performanceCounterService != null)
            {
                services.Remove(performanceCounterService);
            }
    }

Configure the telemetry channel

The default channel is ServerTelemetryChannel. You can override it as the following example shows:

using Microsoft.ApplicationInsights.Channel;

    public void ConfigureServices(IServiceCollection services)
    {
        // Use the following to replace the default channel with InMemoryChannel.
        // This can also be applied to ServerTelemetryChannel.
        services.AddSingleton(typeof(ITelemetryChannel), new InMemoryChannel() {MaxTelemetryBufferCapacity = 19898 });

        services.AddApplicationInsightsTelemetryWorkerService();
    }

Disable telemetry dynamically

If you want to disable telemetry conditionally and dynamically, you can resolve the TelemetryConfiguration instance with an ASP.NET Core dependency injection container anywhere in your code and set the DisableTelemetry flag on it.

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddApplicationInsightsTelemetryWorkerService();
    }

    public void Configure(IApplicationBuilder app, IHostingEnvironment env, TelemetryConfiguration configuration)
    {
        configuration.DisableTelemetry = true;
        ...
    }

Frequently asked questions

This section provides answers to common questions.

Which package should I use?

Can HostedServices inside a .NET Core app using the AspNetCore package have TelemetryClient injected to it?

Yes. The configuration will be shared with the rest of the web application.

How can I track telemetry that's not automatically collected?

Get an instance of TelemetryClient by using constructor injection and call the required TrackXXX() method on it. We don't recommend creating new TelemetryClient instances. A singleton instance of TelemetryClient is already registered in the DependencyInjection container, which shares TelemetryConfiguration with the rest of the telemetry. Creating a new TelemetryClient instance is recommended only if it needs a configuration that's separate from the rest of the telemetry.

Can I use Visual Studio IDE to onboard Application Insights to a Worker Service project?

Visual Studio IDE onboarding is currently supported only for ASP.NET/ASP.NET Core applications. This document will be updated when Visual Studio ships support for onboarding Worker Service applications.

Can I enable Application Insights monitoring by using tools like Azure Monitor Application Insights Agent (formerly Status Monitor v2)?

No. Azure Monitor Application Insights Agent currently supports .NET only.

Are all features supported if I run my application in Linux?

Yes. Feature support for this SDK is the same in all platforms, with the following exceptions:

• Performance counters are supported only in Windows except for Process CPU/Memory shown in live metrics.

• Even though ServerTelemetryChannel is enabled by default, if the application is running in Linux or macOS, the channel doesn't automatically create a local storage folder to keep telemetry temporarily if there are network issues. Because of this limitation, telemetry is lost when there are temporary network or server issues. To work around this issue, configure a local folder for the channel:

  using Microsoft.ApplicationInsights.Channel;
  using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;
  
      public void ConfigureServices(IServiceCollection services)
      {
          // The following will configure the channel to use the given folder to temporarily
          // store telemetry items during network or Application Insights server issues.
          // User should ensure that the given folder already exists
          // and that the application has read/write permissions.
          services.AddSingleton(typeof(ITelemetryChannel),
                                  new ServerTelemetryChannel () {StorageFolder = "/tmp/myfolder"});
          services.AddApplicationInsightsTelemetryWorkerService();
      }
  

Sample applications

.NET Core console application: Use this sample if you're using a console application written in either .NET Core (2.0 or higher) or .NET Framework (4.7.2 or higher).

ASP.NET Core background tasks with HostedServices: Use this sample if you're in ASP.NET Core and creating background tasks in accordance with official guidance.

.NET Core Worker Service: Use this sample if you have a .NET Worker Service application in accordance with official guidance.

Open-source SDK

Read and contribute to the code.

For the latest updates and bug fixes, see the Release Notes.

Next steps

• Use the API to send your own events and metrics for a detailed view of your app's performance and usage.
• Track more dependencies not automatically tracked.
• Enrich or filter auto-collected telemetry.
• Dependency Injection in ASP.NET Core.