Query based on run-time state

In most LINQ queries, the general shape of the query is set in code. You might filter items using a where clause, sort the output collection using orderby, group items, or perform some computation. Your code might provide parameters for the filter, or the sort key, or other expressions that are part of the query. However, the overall shape of the query can't change. In this article, you learn techniques to use System.Linq.IQueryable<T> interface and types that implement it to modify the shape of a query at run time.

You use these techniques to build queries at run time, where some user input or run-time state changes the query methods you want to use as part of the query. You want to edit the query by adding, removing, or modifying query clauses.

Consider code that defines an IQueryable or an IQueryable<T> against a data source:

string[] companyNames = [
    "Consolidated Messenger", "Alpine Ski House", "Southridge Video",
    "City Power & Light", "Coho Winery", "Wide World Importers",
    "Graphic Design Institute", "Adventure Works", "Humongous Insurance",
    "Woodgrove Bank", "Margie's Travel", "Northwind Traders",
    "Blue Yonder Airlines", "Trey Research", "The Phone Company",
    "Wingtip Toys", "Lucerne Publishing", "Fourth Coffee"
];

// Use an in-memory array as the data source, but the IQueryable could have come
// from anywhere -- an ORM backed by a database, a web request, or any other LINQ provider.
IQueryable<string> companyNamesSource = companyNames.AsQueryable();
var fixedQry = companyNames.OrderBy(x => x);

Every time you run the preceding code, the same exact query is executed. Let's learn how to modify the query extend it or modify it. Fundamentally, an IQueryable has two components:

• Expression—a language-agnostic and datasource-agnostic representation of the current query's components, in the form of an expression tree.
• Provider—an instance of a LINQ provider, which knows how to materialize the current query into a value or set of values.

In the context of dynamic querying, the provider usually remains the same; the expression tree of the query differs from query to query.

Expression trees are immutable; if you want a different expression tree—and thus a different query—you need to translate the existing expression tree to a new one. The following sections describe specific techniques for querying differently in response to run-time state:

• Use run-time state from within the expression tree
• Call more LINQ methods
• Vary the expression tree passed into the LINQ methods
• Construct an Expression<TDelegate> expression tree using the factory methods at Expression
• Add method call nodes to an IQueryable's expression tree
• Construct strings, and use the Dynamic LINQ library

Each of techniques enables more capabilities, but at a cost of increased complexity.

Use run-time state from within the expression tree

The simplest way to query dynamically is to reference the run-time state directly in the query via a closed-over variable, such as length in the following code example:

var length = 1;
var qry = companyNamesSource
    .Select(x => x.Substring(0, length))
    .Distinct();

Console.WriteLine(string.Join(",", qry));
// prints: C, A, S, W, G, H, M, N, B, T, L, F

length = 2;
Console.WriteLine(string.Join(",", qry));
// prints: Co, Al, So, Ci, Wi, Gr, Ad, Hu, Wo, Ma, No, Bl, Tr, Th, Lu, Fo

The internal expression tree—and thus the query—isn't modified; the query returns different values only because the value of length changed.

Call more LINQ methods

Generally, the built-in LINQ methods at Queryable perform two steps:

• Wrap the current expression tree in a MethodCallExpression representing the method call.
• Pass the wrapped expression tree back to the provider, either to return a value via the provider's IQueryProvider.Execute method; or to return a translated query object via the IQueryProvider.CreateQuery method.

You can replace the original query with the result of an System.Linq.IQueryable<T>-returning method, to get a new query. You can use run-time state, as in the following example:

// bool sortByLength = /* ... */;

var qry = companyNamesSource;
if (sortByLength)
{
    qry = qry.OrderBy(x => x.Length);
}

Vary the expression tree passed into the LINQ methods

You can pass in different expressions to the LINQ methods, depending on run-time state:

// string? startsWith = /* ... */;
// string? endsWith = /* ... */;

Expression<Func<string, bool>> expr = (startsWith, endsWith) switch
{
    ("" or null, "" or null) => x => true,
    (_, "" or null) => x => x.StartsWith(startsWith),
    ("" or null, _) => x => x.EndsWith(endsWith),
    (_, _) => x => x.StartsWith(startsWith) || x.EndsWith(endsWith)
};

var qry = companyNamesSource.Where(expr);

You might also want to compose the various subexpressions using another library such as LinqKit's PredicateBuilder:

// This is functionally equivalent to the previous example.

// using LinqKit;
// string? startsWith = /* ... */;
// string? endsWith = /* ... */;

Expression<Func<string, bool>>? expr = PredicateBuilder.New<string>(false);
var original = expr;
if (!string.IsNullOrEmpty(startsWith))
{
    expr = expr.Or(x => x.StartsWith(startsWith));
}
if (!string.IsNullOrEmpty(endsWith))
{
    expr = expr.Or(x => x.EndsWith(endsWith));
}
if (expr == original)
{
    expr = x => true;
}

var qry = companyNamesSource.Where(expr);

Construct expression trees and queries using factory methods

In all the examples up to this point, you know the element type at compile time—string—and thus the type of the query—IQueryable<string>. You might add components to a query of any element type, or to add different components, depending on the element type. You can create expression trees from the ground up, using the factory methods at System.Linq.Expressions.Expression, and thus tailor the expression at run time to a specific element type.

Constructing an Expression<TDelegate>

When you construct an expression to pass into one of the LINQ methods, you're actually constructing an instance of System.Linq.Expressions.Expression<TDelegate>, where TDelegate is some delegate type such as Func<string, bool>, Action, or a custom delegate type.

System.Linq.Expressions.Expression<TDelegate> inherits from LambdaExpression, which represents a complete lambda expression like the following example:

Expression<Func<string, bool>> expr = x => x.StartsWith("a");

A LambdaExpression has two components:

1. A parameter list—(string x)—represented by the Parameters property.
2. A body—x.StartsWith("a")—represented by the Body property.

The basic steps in constructing an Expression<TDelegate> are as follows:

1. Define ParameterExpression objects for each of the parameters (if any) in the lambda expression, using the Parameter factory method.

   ParameterExpression x = Parameter(typeof(string), "x");
   

2. Construct the body of your LambdaExpression, using the ParameterExpression defined, and the factory methods at Expression. For instance, an expression representing x.StartsWith("a") could be constructed like this:

   Expression body = Call(
       x,
       typeof(string).GetMethod("StartsWith", [typeof(string)])!,
       Constant("a")
   );
   

3. Wrap the parameters and body in a compile-time-typed Expression<TDelegate>, using the appropriate Lambda factory method overload:

   Expression<Func<string, bool>> expr = Lambda<Func<string, bool>>(body, x);
   

The following sections describe a scenario in which you might want to construct an Expression<TDelegate> to pass into a LINQ method. It provides a complete example of how to do so using the factory methods.

Construct a full query at run time

You want to write queries that work with multiple entity types:

record Person(string LastName, string FirstName, DateTime DateOfBirth);
record Car(string Model, int Year);

For any of these entity types, you want to filter and return only those entities that have a given text inside one of their string fields. For Person, you'd want to search the FirstName and LastName properties:

string term = /* ... */;
var personsQry = new List<Person>()
    .AsQueryable()
    .Where(x => x.FirstName.Contains(term) || x.LastName.Contains(term));

But for Car, you'd want to search only the Model property:

string term = /* ... */;
var carsQry = new List<Car>()
    .AsQueryable()
    .Where(x => x.Model.Contains(term));

While you could write one custom function for IQueryable<Person> and another for IQueryable<Car>, the following function adds this filtering to any existing query, irrespective of the specific element type.

// using static System.Linq.Expressions.Expression;

IQueryable<T> TextFilter<T>(IQueryable<T> source, string term)
{
    if (string.IsNullOrEmpty(term)) { return source; }

    // T is a compile-time placeholder for the element type of the query.
    Type elementType = typeof(T);

    // Get all the string properties on this specific type.
    PropertyInfo[] stringProperties = elementType
        .GetProperties()
        .Where(x => x.PropertyType == typeof(string))
        .ToArray();
    if (!stringProperties.Any()) { return source; }

    // Get the right overload of String.Contains
    MethodInfo containsMethod = typeof(string).GetMethod("Contains", [typeof(string)])!;

    // Create a parameter for the expression tree:
    // the 'x' in 'x => x.PropertyName.Contains("term")'
    // The type of this parameter is the query's element type
    ParameterExpression prm = Parameter(elementType);

    // Map each property to an expression tree node
    IEnumerable<Expression> expressions = stringProperties
        .Select(prp =>
            // For each property, we have to construct an expression tree node like x.PropertyName.Contains("term")
            Call(                  // .Contains(...) 
                Property(          // .PropertyName
                    prm,           // x 
                    prp
                ),
                containsMethod,
                Constant(term)     // "term" 
            )
        );

    // Combine all the resultant expression nodes using ||
    Expression body = expressions
        .Aggregate((prev, current) => Or(prev, current));

    // Wrap the expression body in a compile-time-typed lambda expression
    Expression<Func<T, bool>> lambda = Lambda<Func<T, bool>>(body, prm);

    // Because the lambda is compile-time-typed (albeit with a generic parameter), we can use it with the Where method
    return source.Where(lambda);
}

Because the TextFilter function takes and returns an IQueryable<T> (and not just an IQueryable), you can add further compile-time-typed query elements after the text filter.

var qry = TextFilter(
        new List<Person>().AsQueryable(),
        "abcd"
    )
    .Where(x => x.DateOfBirth < new DateTime(2001, 1, 1));

var qry1 = TextFilter(
        new List<Car>().AsQueryable(),
        "abcd"
    )
    .Where(x => x.Year == 2010);

Add method call nodes to the IQueryable<TDelegate>'s expression tree

If you have an IQueryable instead of an IQueryable<T>, you can't directly call the generic LINQ methods. One alternative is to build the inner expression tree as shown in the previous example, and use reflection to invoke the appropriate LINQ method while passing in the expression tree.

You could also duplicate the LINQ method's functionality, by wrapping the entire tree in a MethodCallExpression that represents a call to the LINQ method:

IQueryable TextFilter_Untyped(IQueryable source, string term)
{
    if (string.IsNullOrEmpty(term)) { return source; }
    Type elementType = source.ElementType;

    // The logic for building the ParameterExpression and the LambdaExpression's body is the same as in the previous example,
    // but has been refactored into the constructBody function.
    (Expression? body, ParameterExpression? prm) = constructBody(elementType, term);
    if (body is null) { return source; }

    Expression filteredTree = Call(
        typeof(Queryable),
        "Where",
        [elementType],
        source.Expression,
        Lambda(body, prm!)
    );

    return source.Provider.CreateQuery(filteredTree);
}

In this case, you don't have a compile-time T generic placeholder, so you use the Lambda overload that doesn't require compile-time type information, and which produces a LambdaExpression instead of an Expression<TDelegate>.

The Dynamic LINQ library

Constructing expression trees using factory methods is relatively complex; it's easier to compose strings. The Dynamic LINQ library exposes a set of extension methods on IQueryable corresponding to the standard LINQ methods at Queryable, and which accept strings in a special syntax instead of expression trees. The library generates the appropriate expression tree from the string, and can return the resultant translated IQueryable.

For instance, the previous example could be rewritten as follows:

// using System.Linq.Dynamic.Core

IQueryable TextFilter_Strings(IQueryable source, string term)
{
    if (string.IsNullOrEmpty(term)) { return source; }

    var elementType = source.ElementType;

    // Get all the string property names on this specific type.
    var stringProperties =
        elementType.GetProperties()
            .Where(x => x.PropertyType == typeof(string))
            .ToArray();
    if (!stringProperties.Any()) { return source; }

    // Build the string expression
    string filterExpr = string.Join(
        " || ",
        stringProperties.Select(prp => $"{prp.Name}.Contains(@0)")
    );

    return source.Where(filterExpr, term);
}