FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

WAYANG-678: Implement cardinality tracking in DataStreamChannel by Harshakamalur · Pull Request #729 · apache/wayang · GitHub

/ wayang Public

WAYANG-678: Implement cardinality tracking in DataStreamChannel - #729

Open
Harshakamalur wants to merge 1 commit into
apache:mainfrom
Harshakamalur:fix-datastreamchannel-cardinality
Open

Harshakamalur wants to merge 1 commit into
apache:mainfrom
Harshakamalur:fix-datastreamchannel-cardinality

Conversation

Copy link
Copy Markdown

Fixes #678.

Implements cardinality tracking in DataStreamChannel.Instance using AtomicLong, following the same pattern as RddChannel in the Spark platform.

Changes:

  • Replaces the always-zero size field with an AtomicLong counter
  • Counter increments via a filter on the DataStream when instrumentation is enabled
  • Counter is read in both getMeasuredCardinality() and doDispose()
  • Mirrors the established pattern from RddChannel in wayang-spark

All 25 existing Flink tests pass.

Copy link
Copy Markdown
Author

Hi @apache/wayang-committers, I've now submitted two PRs: this one for the README links and #728 for the TensorFlow test fixes.

Could you please assign both to me and approve the CI workflows so I can ensure all tests pass? Thanks!

mspruc commented May 11, 2026

Copy link
Copy Markdown
Contributor

Can you also provide an integration test for getting the cardinality?

Copy link
Copy Markdown
Member

@Harshakamalur - dod you see the latest comment from @mspruc ?

Copy link
Copy Markdown
Member

@Harshakamalur - please follow up.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

this.size is currently always 0

3 participants


Back | FazBrowse Home | New Git URL