FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Make `udaf_default_*` display/schema name helpers non-generic (214 KB smaller binary) by alamb · Pull Request #24728 · apache/datafusion · GitHub

Make udaf_default_* display/schema name helpers non-generic (214 KB smaller binary) - #24728

Open
alamb wants to merge 3 commits into
apache:mainfrom
alamb:udaf-default-helpers-dyn
Open

Make udaf_default_* display/schema name helpers non-generic (214 KB smaller binary)#24728
alamb wants to merge 3 commits into
apache:mainfrom
alamb:udaf-default-helpers-dyn

Conversation

alamb commented Aug 27, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

The default udaf_default_* display / schema name helpers in datafusion/expr/src/udaf.rs are instantiated for each UDAF

What changes are included in this PR?

  1. Move the default implementations of the five display / schema name methods into non-generic builder structs so each body is compiled only once
  2. Update the AggregateUDFImpl default methods to use the builders
  3. Mark existing functions as deprecated
  4. Add a doc example

Code size

Measured per the methodology in #24727, at merge base d66f4b9 and on this branch (39d0e53).

cargo llvm-lines --release -p datafusion-functions-aggregate --lib (29 AggregateUDFImpl implementations):

main (d66f4b9) this branch change
udaf_default_* display/schema name helpers 212,048 IR lines (2,755 functions) 0 -212,048 (-100%)
Crate total 2,112,337 IR lines (39,704 copies) 1,900,949 IR lines (36,936 copies) -211,388 (-10.0%)

cargo llvm-lines --release -p datafusion-expr --lib (the builder bodies now compile here exactly once):

main (d66f4b9) this branch change
Crate total 1,265,863 IR lines (26,406 copies) 1,229,636 IR lines (25,925 copies) -36,227 (-2.9%)

cargo build --release --bin datafusion-cli:

main (d66f4b9) this branch (39d0e53) change
Binary size 81,580,368 bytes 81,365,536 bytes -214,832 bytes (-0.26%)

Are these changes tested?

Covered by existing tests

Are there any user-facing changes?

Yes, but not breaking:

alamb added the api change Changes the API exposed to users of the crate label Aug 27, 2026
github-actions Bot added documentation Improvements or additions to documentation logical-expr Logical plan and expressions labels Aug 27, 2026

This comment was marked as outdated.

github-actions Bot added the auto detected api change Auto detected API change label Aug 27, 2026
alamb force-pushed the udaf-default-helpers-dyn branch from 580bb8a to 39d0e53 Compare August 27, 2026 15:16
github-actions Bot removed the documentation Improvements or additions to documentation label Aug 27, 2026
alamb changed the title Make udaf_default_* display/schema name helpers non-generic to reduce code size Make udaf_default_* display/schema name helpers non-generic (214 KB smaller binary) Aug 27, 2026

alamb left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This Diff looks lage, but if you ignore whitespace via https://github.com/apache/datafusion/pull/24728/changes?w=1 it is mich smaller (basically adding a bunch of docs and a builder)

/// The default implementation is provided by [`UdafSchemaNameBuilder`]
fn schema_name(&self, params: &AggregateFunctionParams) -> Result<String> {
udaf_default_schema_name(self, params)
UdafSchemaNameBuilder::new(self.name(), params)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

The core rationale is to use a single non generic function -- I also made a struct like builder to make the code easier to see too

null_treatment,
} = params;

// exclude the first function argument(= column) in ordered set aggregate function,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

this logic is all the same -- there is just now some more builder ceremony logic

}
}

/// Builds the default implementation of [`AggregateUDFImpl::schema_name`].

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Here is a new builder for schema name -- it is basically the same pattern for all of the other types

alamb removed the auto detected api change Auto detected API change label Aug 27, 2026
alamb marked this pull request as ready for review August 27, 2026 16:16
alamb mentioned this pull request Aug 27, 2026
12 tasks
github-actions Bot added the auto detected api change Auto detected API change label Aug 27, 2026

codecov-commenter commented Aug 27, 2026
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.39698% with 41 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.47%. Comparing base (c56a7ab) to head (3ef58be).

Files with missing lines Patch % Lines
datafusion/expr/src/udaf.rs 79.39% 21 Missing and 20 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24728      +/-   ##
==========================================
- Coverage   81.47%   81.47%   -0.01%     
==========================================
  Files        1122     1122              
  Lines      404140   404185      +45     
  Branches   404140   404185      +45     
==========================================
+ Hits       329284   329292       +8     
- Misses      55546    55577      +31     
- Partials    19310    19316       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

alamb commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Extended teset failure seems to be failing on main too

    joins::hash_join::exec::tests::test_partitioned_null_equal_dynamic_filter_keeps_probe_nulls_for_build_logical_null

Here is a ticket

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api change Changes the API exposed to users of the crate auto detected api change Auto detected API change logical-expr Logical plan and expressions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL