| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
See Milestones for release notes.
Detects overly complex Entity Framework Core queries and either logs them or throws. Each check has two levels: the level a query is logged at, which has defaults, and the level a query throws at, which is opt in.
Entity Framework has no limits of its own on the size or shape of a query, and the team closed the request for unbounded result set warnings as not planned. Limits like these usually sit in front of Entity Framework, in an OData or GraphQL layer, and so only cover queries that arrive that way.
An API that lets the client shape the query, such as GraphQL or OData, hands part of the query to whoever sends the request. One small request can ask for every row of a table, nest navigations many levels deep, or send a Contains list with thousands of values. Each costs the database and the server far more than it costs the client to send, so a handful of them, repeated, is enough to make the API slow or unavailable. This is a denial of service by resource exhaustion, and it needs no bug in the API to work, only a query the API did not expect.
The checks bound what one query can ask for, whichever layer built it:
| Attack | Check |
|---|---|
| Requesting every row | RejectUnbounded, MaxTake |
| Deeply nested or very large queries | MaxNodes, MaxDepth, MaxOperators |
| Long navigation chains and includes, which multiply joins | MaxNavigationDepth, MaxIncludes, MaxIncludeDepth |
| Several collections in one query, which multiply rows | MaxSingleQueryCollections |
| Huge IN lists | MaxInValues |
| A query that passes every check but is still expensive | SQL Server cost limit |
Only throw levels stop a query. Log levels report it and let it run. A practical rollout is to log first, see which levels real clients reach, then set throw levels above that.
These checks limit the cost of each query. They do not replace limits on how often a client can send one, so still use authentication, rate limiting, and request and command timeouts.
Background:
https://nuget.org/packages/EfQueryComplexity/
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder.UseQueryComplexity();With no levels passed, a query is logged when it exceeds QueryComplexityLimits.LogDefaults, and no query throws.
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder.UseQueryComplexity(
logAt: QueryComplexityLimits.LogDefaults with
{
MaxTake = 500,
RejectUnbounded = false
});Throwing is opt in, and every level has to be given, so nothing is enforced by accident:
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder.UseQueryComplexity(
throwAt: new(
MaxNodes: 4096,
MaxDepth: 64,
MaxOperators: 64,
MaxNavigationDepth: 4,
MaxIncludes: 16,
MaxIncludeDepth: 4,
MaxTake: 1000,
MaxInValues: 1000,
RejectUnbounded: true));A query that exceeds a throw level throws QueryComplexityException, which carries every level it exceeded in Violations.
A query is checked against the throw levels before the log levels, so a throw level below its log level would leave that log level unreachable. That is rejected as the context is constructed, rather than quietly logging nothing. Passing the same levels for both is allowed, and is how to say "only throw".
| Level | Counts | Measured | Log default |
|---|---|---|---|
| MaxNodes | Expression nodes in the query | While compiled | 1000 |
| MaxDepth | Nesting depth of the query expression | While compiled | 50 |
| MaxOperators | LINQ operators, including in subqueries | While compiled | 30 |
| MaxNavigationDepth | Navigations in one member access chain | While compiled | 3 |
| MaxIncludes | Include calls | While compiled | 6 |
| MaxIncludeDepth | Navigations in one Include chain | While compiled | 3 |
| MaxSingleQueryCollections | Collections one SQL query loads | While compiled | 1 |
| MaxTake | The value passed to Take | Every execution | 1000 |
| MaxInValues | Values in the largest list the query sends | Every execution | 1000 |
| RejectUnbounded | A query returning rows with no Take or lookup by key | While compiled | All |
A check fires when the measured value is greater than the level. A level of null turns that check off.
A single SQL query joins every collection it loads, so each multiplies the rows returned for the others. Loading 10 departments with 50 employees and 20 projects each returns 10,000 rows for 710 entities. This is a cartesian explosion.
MaxSingleQueryCollections counts:
It does not count:
The log default of 1 matches the point where Entity Framework logs MultipleCollectionIncludeWarning. That warning only covers Include, and only when no splitting behavior is configured.
A query is bounded when it cannot return more rows than a Take, or a lookup by key, allows:
The message names the types of the rows returned without a Take, and so does QueryComplexityViolation.RowTypes. A row type is the entity a query reads, not what it projects to, so Employees.Select(_ => _.Name) returns Employee rows. A query that joins in another sequence returns rows of both types: Departments.SelectMany(_ => _.Employees) returns Department and Employee rows.
A Where that compares the key of its rows with a value, like Employees.Where(_ => _.Id == id), returns at most one row, so it needs no Take:
Some apps have no large table at all. An admin or workflow app where every table holds hundreds or thousands of rows can return all of them, and this check only reports queries that are fine. Turn it off, and keep the rest:
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder.UseQueryComplexity(
logAt: QueryComplexityLimits.LogDefaults with
{
// Every table is small, so a query with no Take is fine
RejectUnbounded = false
});The other checks are unaffected, so query size, depth, operators, navigations, includes, Take counts and IN list sizes are still reported. Turn it on again if a table starts growing, and use Only to name that table.
Most apps know which tables stay small and which grow. RejectUnbounded takes an UnboundedEntities, so the check can cover only the types where returning every row is a problem. true converts to UnboundedEntities.All and false to UnboundedEntities.None.
Check every type except the ones known to have few rows:
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder.UseQueryComplexity(
logAt: QueryComplexityLimits.LogDefaults with
{
// Few rows, so returning all of them is fine
RejectUnbounded = UnboundedEntities.AllExcept(
typeof(User),
typeof(AccessGroup))
});Or check only the types known to have many rows:
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder.UseQueryComplexity(
logAt: QueryComplexityLimits.LogDefaults with
{
// Many rows, so every query for them needs a Take
RejectUnbounded = UnboundedEntities.Only(
typeof(Commitment))
});These values only exist while a query runs, so they are checked for every execution rather than once per query. That check needs Entity Framework's internal query compiler, which is registered when MaxTake or MaxInValues is set at either level. It is also registered whenever throwAt is passed, since it caches a query that throws, so the query is not measured again for every execution. Consequences:
A value that is over a log level is logged the first time a compiled query exceeds it, rather than on every execution. A value over a throw level throws every time.
Skip every check for one query:
// Skips every check for this query
var employees = await context.Employees
.IgnoreQueryComplexity()
.ToListAsync();Or replace levels for one query:
// Replaces levels for this query only
var employees = await context.Employees
.WithQueryComplexity(
new()
{
MaxTake = 5000
})
.Take(5000)
.ToListAsync();A marker is read from the query being executed, not from a subquery inside it. The levels are for the whole query, so a marker on a queryable that is later used inside another query would change the levels of that whole query, and every query composed over it would skip checks it never asked to skip. One there throws rather than being honored or quietly dropped.
Every level left null keeps the configured value, and a level that is set replaces both the log and the throw level for that check. An override never starts throwing for a context that was not given throw levels. To turn one check off for a query use int.MaxValue. RejectUnbounded is a bool for a query: true checks every type and false none, whichever types were configured.
MaxTake and MaxInValues can only be changed for a query when the configured levels set one of them, since the value checks are otherwise not set up at all. An override that sets one anyway throws.
Each distinct set of levels is a constant in the query, so a query using them is compiled and checked separately.
Warnings are logged as QueryComplexityEventId.LimitExceeded through the Entity Framework pipeline, so they reach LogTo, an ILoggerFactory, and a DiagnosticSource. That also means the usual configuration applies, including turning the warning into an error:
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder
.UseQueryComplexity()
.ConfigureWarnings(
_ => _.Throw(QueryComplexityEventId.LimitExceeded));Use Ignore instead of Throw to silence it.
The same configuration can turn the warning into an error without mentioning it. A context whose warnings all throw, such as one built by EfLocalDb, which uses Default(WarningBehavior.Throw), throws InvalidOperationException for any query over a log level, and the message starts with "An error was generated for warning 'EfQueryComplexity.LimitExceeded'". A behavior set for one event takes precedence over the default, so to keep only logging:
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder
.UseQueryComplexity()
.ConfigureWarnings(
_ => _
.Default(WarningBehavior.Throw)
.Log(QueryComplexityEventId.LimitExceeded));The message names every level that was exceeded and then prints the query, bounded to 1000 characters. The query that broke a level is the one that prints long, and without a bound every log line reporting it would carry the whole expression tree.
The levels above bound the shape of a query, not what it costs to run. An allowed query over a large unindexed table is still expensive. SQL Server can make that call itself:
protected override void OnConfiguring(DbContextOptionsBuilder builder) =>
builder
.UseSqlServer("connection-string")
.UseQueryComplexity(sqlServerCostLimit: 300);SET QUERY_GOVERNOR_COST_LIMIT is applied to every connection as it opens, and SQL Server then refuses any statement whose estimated plan cost is greater than the limit, with error 8649.
Shape is measured only when a query is compiled, which happens once for each distinct query. After that, each execution pays only for the value checks: reading the Take count, and counting each list the query sends, such as a Contains list.
Two benchmarks in src/Benchmarks run a query that is already compiled, with a Take and a 100 value Contains list, creating a new context for each execution, as each request does. Every configuration other than the baseline calls UseQueryComplexity:
DatabaseExecutionBenchmarks executes it against LocalDB, so these are the numbers for a whole request, including the round trip and materializing the rows:
| Configuration | Mean | Allocated |
|---|---|---|
| Baseline (no UseQueryComplexity) | 1.092 ms | 252.42 KB |
| Shape checks only | 1.088 ms (-4 μs) | 252.83 KB (+420 bytes) |
| Log defaults | 1.093 ms (+1 μs) | 252.84 KB (+427 bytes) |
| Log and throw at the defaults | 1.093 ms (+1 μs) | 252.84 KB (+427 bytes) |
Each configuration ran in three processes, and against LocalDB the time varied between them by up to 22 μs, more than the checks cost. That is also why shape checks only measures faster than the baseline. So ExecutionOverheadBenchmarks measures the cost without a database. It calls ToQueryString(), which runs the same cached query and value checks without connecting. It does different work from an execution, so only what each configuration adds is shown, compared with the baseline:
| Configuration | Time added | Memory added |
|---|---|---|
| Shape checks only | 0.3 μs | 407 bytes |
| Log defaults | -0.1 μs | 415 bytes |
| Log and throw at the defaults | 1.4 μs | 415 bytes |
Without a database the processes varied by about 2.5 μs, and every configuration is within 1.5 μs of the baseline, so the checks add no time that can be measured, against a request of about 1.1 ms. The memory comes with creating each context, most of it what Entity Framework allocates for any options extension and interceptor. The value checks allocate nothing on each execution, and add 8 bytes to shape checks only. Memory is the lowest of the three processes: BenchmarkDotNet counts allocations on every thread, and some processes allocated up to about 290 bytes more, including with UseQueryComplexity not called. Measured on an AMD Ryzen 9 5900X, .NET 10, BenchmarkDotNet 0.15.8.
Compiling a query that has not been seen before costs one extra pass over its expression tree, and that happens once per distinct query.
Pattern from The Noun Project
| Back | FazBrowse Home | New Git URL |