Draft scope

This is the original exploratory branch from the incremental update benchmark investigation. It is intentionally broad and should be reviewed as a bundle of instrumentation, benchmark tooling, and candidate performance changes rather than as a narrow production fix.

The branch currently contains 32 commits over main across 43 files.

What it contains

Instrumentation and benchmark tooling

• Adds opt-in perf tracing helpers for DB live queries and db-ivm operators.
• Adds trace aggregation coverage for the incremental query path, including hashing/operator timings.
• Adds scripts/bench/incremental-update.ts for reproducing the issue-style benchmark shapes and source-size experiments.
• Adds the investigation docs:
  • INCREMENTAL_UPDATE_INSTRUMENTATION_PLAN.md
  • INCREMENTAL_UPDATE_FINDINGS.md

DB collection and sync hot paths

• Avoids full row origin snapshots and avoids storing implicit remote row origins.
• Adds faster single-op and two-row sync commit paths.
• Reduces redundant synced-data/index map lookups during common insert/update/delete paths.
• Trims virtual prop enrichment and simple sync update checks when they are not needed.
• Reduces incremental change delivery allocations and skips empty filtered subscription batches.
• Makes Transaction.applyMutations linear rather than repeatedly shifting work from the front of an array.

Query/live graph hot paths

• Runs immediate live query graph work directly when there is no transaction context.
• Skips redundant pending graph checks.
• Avoids full nested include scans in the benchmarked include shapes.
• Trims live query update allocations.

Index/sorted map/hash/db-ivm changes

• Adds fast paths for key-sorted SortedMap updates and duplicate lookup avoidance.
• Caches compiled index expression evaluators and reduces index timestamp allocation overhead.
• Reduces group aggregate hashing cost.
• Reuses group aggregate entries and specializes simple group aggregates.
• Adds fast paths for single-row joins.

Why

The benchmark shared during the investigation showed very slow steady-state incremental updates for query shapes that look like filtered lists plus joins/includes/counts. The main question was whether the time was in source collection update application, hashing, live query scheduling, or the db-ivm query graph.

This branch keeps the instrumentation used to answer that question and the candidate optimizations that were explored from those traces.

Review notes

• This is a draft PR for review/discussion and not currently shaped as a minimal merge candidate.
• Some production-ready fixes from this investigation have already been split out elsewhere. This branch preserves the full exploratory thread for comparison and review.
• The diff includes tracing and benchmark docs/tooling by design.

Validation performed during the investigation

• Ran the local incremental benchmark repeatedly across multiple source sizes, including the larger issue/user/comment datasets used to approximate the shared benchmark.
• Compared baseline, this branch, the external performance PR branch, and the combined branch.
• Added targeted tests for trace aggregation and db-ivm groupBy incremental behavior.

No fresh full CI run was performed immediately before opening this draft PR.