FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat(dataframe): add limit, distinct, dropColumns, withColumnRenamed by andygrove · Pull Request #2 · andygrove/datafusion-java · GitHub

Repository navigation

feat(dataframe): add limit, distinct, dropColumns, withColumnRenamed - #2

Closed
andygrove wants to merge 5 commits into
mainfrom
feat/dataframe-methods-batch
Closed

andygrove wants to merge 5 commits into
mainfrom
feat/dataframe-methods-batch

Conversation

Copy link
Copy Markdown
Owner

Which issue does this PR close?

  • Closes #.

Rationale for this change

The Java DataFrame API currently exposes only select and filter as
transformations. This PR rounds out the trivially-implementable subset of
DataFusion's Rust DataFrame API so users can build small pipelines without
falling back to SQL strings.

What changes are included in this PR?

Four new methods on DataFrame, each backed by a thin JNI bridge that calls
the corresponding DataFusion operator:

  • limit(int fetch) / limit(int skip, int fetch) — take the first N rows, optionally after skipping. Negative arguments throw IllegalArgumentException.
  • distinct() — deduplicate rows across all columns.
  • dropColumns(String... columnNames) — inverse of select. Unknown column names are silently ignored (matches DataFusion's drop_columns semantics).
  • withColumnRenamed(String oldName, String newName) — rename a column. Unknown old names are a no-op (matches DataFusion's with_column_renamed semantics).

All four follow the existing pattern: throw IllegalStateException on closed/collected handles, and leave the receiver usable so chaining off a shared source is non-destructive.

Are these changes tested?

Yes. DataFrameTransformationsTest gains 12 new tests covering happy paths,
non-destructive semantics, invalid arguments, and the silent-ignore behaviors
for dropColumns/withColumnRenamed. The existing methodsThrowAfterClose
and methodsThrowAfterCollect lifecycle tests are extended to cover the new
methods.

Are there any user-facing changes?

Yes — four new public methods on org.apache.datafusion.DataFrame. No
breaking changes.

Copy link
Copy Markdown
Owner Author

Closing — meant to open against apache/datafusion-java.

andygrove closed this May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL