| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
optimizations to improve performance, add optional ormar-utils packag…
…e written in rust
Optimize hot paths with caching and Rust reverse alias map
Profile-driven optimizations targeting the most expensive ormar functions. Key changes: - Cache alias<->field_name mappings per model class, using Rust build_reverse_alias_map for O(1) lookups (was O(n) linear scan, called 406K times in profiling) - Cache (col_name, field_name) pairs to avoid repeated SA column iteration in own_table_columns and extract_prefixed_table_columns - Use set instead of list for selected_columns membership checks - Cache get_name(lower=True), extract_db_own_fields, ormar_fields_set, and ForeignKey constructors dict - Use frozenset for RelationProxy method check End-to-end benchmark improvements: - iterate: 24-30% faster - first: 26-36% faster - get_all: 18-19% faster - saving: 17-25% faster - select_related: 12-17% faster Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
chore: regenerate lock and reorder imports after rebase
Post-rebase tidying: poetry lock bumped to 2.3.3 plus mkdocstrings patch bump, and ruff reordered the third-party `ormar_rust_utils` import in queryset/utils.py.
perf: cache & specialize _process_kwargs hot path (#1649)
Profile-driven refactor of NewBaseModel._process_kwargs, which fires on every Model.__init__ (user construction and row hydration). Removes redundant per-init work and skips no-op conversion calls for fields that are neither JSON nor bytes. Changes: - Cache _pydantic_field_names, _extra_is_ignore, _allowed_kwarg_names on the class (lazy-populated on first init); installed via metaclass add_cached_properties alongside the existing _json_fields/_bytes_fields caches. - Replace nested _convert_to_bytes(_convert_json(...)) wrapping with an explicit dispatch loop. Common path (regular ormar field, no JSON, no bytes) avoids the function-call overhead entirely. - Inline _remove_extra_parameters_if_they_should_be_ignored behind the cached _extra_is_ignore bool; remove the method (no external callers). - Remove now-unused _convert_to_bytes / _convert_json methods and the orphaned _convert_json entry in quick_access_views. Behavior unchanged: same ModelError messaging on unknown fields, same JSON encoding, same base64 bytes handling. 628 tests pass at 100 % coverage. Benchmark deltas (median, pytest-benchmark): - test_initializing_models[250] -34.9% - test_initializing_models[500] -8.8% - test_iterate[500] / test_iterate[1000] -7.9% / -7.4% - test_get_all_with_related_models[40] -3.2% - I/O-bound get_one / first / get_or_none within noise cProfile (all_with_related): _process_kwargs tottime 0.365s -> 0.238s (-35%). The line-397 dict-comp is no longer a separate hotspot. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
perf: replace RelationProxy.__getattribute__ with explicit count/clear (
#1650) Profile-driven removal of the per-attribute-access Python override on RelationProxy. The previous __getattribute__ existed solely to redirect two list-method names ("count", "clear") to the QuerysetProxy versions; every other attribute access paid the cost of a Python-level __getattribute__ call only to fall through to super. cProfile (all_with_related, 40 000 row hydrations): __getattribute__ was 0.354 s tottime / 6.2 % of scenario time, with 420 000 calls. After this change it is no longer in the top 25 by tottime — attribute access goes through the C-level lookup path. Replacement: define count() and clear() as async methods directly on RelationProxy. They shadow list.count / list.clear by virtue of MRO and delegate to queryset_proxy after self._initialize_queryset(). The observable async semantics are unchanged: callers always did ``await proxy.count(...)`` / ``await proxy.clear(...)``; the previous override returned a bound async method, the new methods are themselves async — both produce the same coroutine. Behavior unchanged: same signatures (distinct=True / keep_reversed=True defaults), same delegation path, same QuerysetProxy initialization trigger. 628 tests pass at 100 % coverage. Benchmark deltas (median, pytest-benchmark, --warmup=on, 10+ rounds): - test_get_all_with_related_models[10] -17.2 % - test_get_all_with_related_models[20] -15.6 % - test_get_all_with_related_models[40] -7.7 % - test_get_all[250] -13.1 % - test_get_all[500] -9.9 % - test_get_all[1000] -6.4 % - test_iterate[*] within noise - single-row get_one / first I/O-dominated, ±15 % noise Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Merge remote-tracking branch 'upstream/master' into check-optimisatio…
…ns-and-rs # Conflicts: # poetry.lock
perf: defer QuerysetProxy construction in RelationProxy
Every model materialized from a row eagerly built a QuerysetProxy per reverse/m2m relation, even though the queryset machinery is rarely touched on read paths. Make `RelationProxy.queryset_proxy` a lazy property so the allocation only happens when something actually queries through the relation. Benchmarks (sqlite, on-disk): - init[1000]: 9.74 ms -> 8.42 ms (-13.6%) - get_all[1000]: 15.64 ms -> 14.20 ms (-9.2%) - iterate[1000]: 22.83 ms -> 21.33 ms (-6.6%)
perf: defer RelationProxy construction in Relation
Reverse / many-to-many relations allocated their RelationProxy in Relation.__init__ for every model, even when the relation was never read. Construct on first add/get instead, and read related_models directly from the hash-cache update path so a PK change doesn't force materialization of every reverse proxy on the model. Combined with the previous QuerysetProxy change, vs baseline: - init[1000]: 9.74 ms -> 7.06 ms (-27.5%) - get_all[1000]: 15.64 ms -> 12.06 ms (-22.9%) - iterate[1000]: 22.83 ms -> 18.66 ms (-18.3%)
perf: defer Relation construction in RelationsManager
RelationsManager.__init__ used to build a Relation for every declared FK on every Model.__init__. Most of those Relation instances are never read — they exist only to be checked for membership or never touched at all on row-materialization paths. Build them on demand in _get(name) using a precomputed name->field lookup instead. Combined with the previous lazy-RelationProxy and lazy-QuerysetProxy changes, vs baseline: - init[1000]: 9.74 ms -> 6.67 ms (-31.6%) - get_all[1000]: 15.64 ms -> 11.58 ms (-26.0%) - iterate[1000]: 22.83 ms -> 18.83 ms (-17.5%) - init[250]: 2.65 ms -> 1.66 ms (-37.4%)
perf: cache row-extraction plan in from_row (#1653)
from_row used to recompute, for every row × every join level, the selected-columns set, the prefixed column key strings, and the exclude set. All three depend only on (model_cls, table_prefix, excludable), not on the row, so they can be built once per query and reused across rows. - New RowExtractionPlan dataclass holds the precomputed work. - build_row_extraction_plan / get_or_build_row_plan / apply_row_plan split the lifecycle so callers can build once and apply many. - _process_query_result_rows allocates a per-call plan cache; iterate shares one cache across all yielded chunks so 1-row chunks still amortize. - prefetch_query._instantiate_models builds the plan once before the row loop. - _construct_with_excluded widened to AbstractSet[str] so the plan can store the exclude set as a hashable frozenset. Benchmarks vs the lazy-relation baseline: - get_all[1000]: 11.58 ms -> 9.11 ms (-21.3%) - get_all[500]: 6.80 ms -> 4.88 ms (-28.3%) - iterate[1000]: 18.83 ms -> 15.67 ms (-16.8%) - iterate[250]: 5.62 ms -> 4.54 ms (-19.2%)
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff master...lazy-queryset-proxy
| Back | FazBrowse Home | New Git URL |