Connected components performance & Spark Connect fixes

• The randomized_contraction connected-components algorithm now benchmarks better than two_phase with AQE and uses roughly half the memory
• pageRank no longer raises an AttributeError on Spark Connect
• Tutorial dependencies are published as PEP 621 extras, installable with pip install graphframes-py[tutorials]
• click moved to the main dependencies so the graphframes CLI works without the tutorials extra

What's Changed

• chore(deps-dev): bump setuptools from 80.9.0 to 83.0.0 in /python by @dependabot[bot] in #875
• Fix: move click to main dependencies and lazy-import tutorials in console.py by @rjurney with @Copilot in #881
• Fix: restore the broken star history chart by @PingouinFerreux in #883
• fix: publish tutorials dependencies as PEP 621 extras by @james-willis in #885
• feat: GQL-1 -- update py7zr by @SemyonSinchenko in #867
• chore: add test for 873 by @SemyonSinchenko in #886
• fix: resolve AttributeError in pageRank on Spark Connect (issue #889) by @MCBoarder289 in #890
• perf: better wcc by @SemyonSinchenko in #888

New Contributors

• @rjurney with @Copilot made their first contribution in #881
• @PingouinFerreux made their first contribution in #883
• @MCBoarder289 made their first contribution in #890

Full Changelog: v0.12.1...v0.12.2

PySpark Client fixes & compatibility

• py4j imports were relocated
• small fixes across API

What's Changed

• fix: doc build by @SemyonSinchenko in #860
• feat: support of pyspark-client by @SemyonSinchenko in #863
• ci: add codespell spell check and extend python linting to dev scripts by @james-willis in #864

Full Changelog: v0.12.0...v0.12.1

What's Changed

• fix: revert upload-pages-artifact to v4 in docs workflow by @james-willis in #820
• doc: sync docs with code by @SemyonSinchenko in #822
• chore: fix docs build by @SemyonSinchenko in #821
• docs: add Wherobots CTA to 0.11.0 release post by @james-willis in #823
• add reversed implementation to Pregel API by @slavlotski in #813
• chore(deps-dev): bump pytest from 8.4.1 to 9.0.3 in /python by @dependabot[bot] in #826
• Potential fix for code scanning alert no. 1: Workflow does not contain permissions by @SemyonSinchenko in #832
• Potential fix for code scanning alert no. 2: Workflow does not contain permissions by @SemyonSinchenko in #831
• Potential fix for code scanning alert no. 3: Workflow does not contain permissions by @SemyonSinchenko in #830
• chore(deps-dev): bump urllib3 from 2.6.3 to 2.7.0 in /python by @dependabot[bot] in #837
• chore(deps-dev): bump idna from 3.10 to 3.15 in /python by @dependabot[bot] in #838
• chore(deps-dev): bump pyarrow from 20.0.0 to 23.0.1 in /python by @dependabot[bot] in #844
• Add Spark CBO config tips for boosting motif finding performance by @WeichenXu123 in #845
• Optimization: Add pruning for connected component 2 phase algorithm by @WeichenXu123 in #846
• chore: README update by @SemyonSinchenko in #850
• chore: sitemap generation by @SemyonSinchenko in #852
• chore: sbt-eclipse plugin by @SemyonSinchenko in #848
• feat: add NeighborhoodAwareCDLP community detection algorithm by @SemyonSinchenko in #825
• feat: all paths between two nodes by @SemyonSinchenko in #828
• fix: use git modification time as docs mtime by @SemyonSinchenko in #855
• feat: pregel memory optimizations by @SemyonSinchenko in #847
• feat: HyperANF implementation by @SemyonSinchenko in #841
• feat: 012 release blogpost by @SemyonSinchenko in #857

New Contributors

• @slavlotski made their first contribution in #813

Full Changelog: v0.11.0...v0.12.0

What's Changed

• doc: add Maven Central coordinates section to installation guide by @SemyonSinchenko in #774
• fix: Support string IDs in GraphFrame powerIterationClustering by @SemyonSinchenko in #773
• fix: fix links in docs by @SemyonSinchenko in #777
• feat: more tests for fromEdges by @SemyonSinchenko in #778
• fix: allow chaining with fixed-length patterns by @goungoun in #771
• feat: update SVDPP docs and code by @SemyonSinchenko in #779
• feat: Randomized Contraction CC by @SemyonSinchenko in #776
• fix: allow undirected fixed length pattern by @goungoun in #781
• fix: anonymous vertex suppot for variable length pattern by @goungoun in #783
• feat: run tests agains spark 4.1 by @SemyonSinchenko in #787
• feat: remove numpy and nose by @SemyonSinchenko in #788
• feat: add approximate triangle counting algorithm using DataSketches by @SemyonSinchenko in #789
• Update README.md by @Yorko in #793
• chore(deps-dev): bump protobuf from 5.29.5 to 5.29.6 in /python by @dependabot[bot] in #794
• chore: switch to LDBC parquets for benchmarks + refactoring of them by @SemyonSinchenko in #799
• feat: pyspark property graphs by @SemyonSinchenko in #798
• docs: add AGENTS.md for AI agent development guidance by @james-willis in #800
• fix(kcore): swap sendMsgToSrc/sendMsgToDst column references by @DavidGi in #802
• feat: random walks and embeddings by @SemyonSinchenko in #752
• feat: refactor ConnectedComponents API by @SemyonSinchenko in #803
• feat(pregel): automatically skip second join when dst columns not needed by @james-willis in #795
• chore: Fix typos in doc/05-blog/998-graphframes-010-release by @juanli16 in #811
• chore(deps-dev): bump requests from 2.32.4 to 2.33.0 in /python by @dependabot[bot] in #812
• feat: aggregate nbrs api by @SemyonSinchenko in #792
• chore(deps-dev): bump pygments from 2.19.2 to 2.20.0 in /python by @dependabot[bot] in #815
• fix: use trim instead f strip by @SemyonSinchenko in #816
• fix: Java 8 compatibility for KMinSampling and benchmarks by @james-willis in #817
• docs: add 0.11.0 release blog post by @james-willis in #814
• chore: bump versions for publishing to latest by @SemyonSinchenko in #818
• docs: add future work section to 0.11.0 release post by @james-willis in #819

New Contributors

• @Yorko made their first contribution in #793
• @DavidGi made their first contribution in #802
• @juanli16 made their first contribution in #811

Full Changelog: v0.10.1...v0.11.0

What's Changed

• chore(deps-dev): bump brotli from 1.1.0 to 1.2.0 in /python by @dependabot[bot] in #751
• chore(deps-dev): bump urllib3 from 2.5.0 to 2.6.0 in /python by @dependabot[bot] in #756
• add more LDBC benchmark datasets by @james-willis in #764
• feat(pregel): add requiredSrcColumns/requiredDstColumns API for triplet memory optimization by @joelrobin18 in #763
• fix: mixing directed and undirected edges in motif throws parse error by @goungoun in #754
• Fix: Throw IllegalArgumentException for null IDs/Src/Dst in toGraphX (Fixes #765) by @JeganVG in #766
• chore(deps-dev): bump urllib3 from 2.6.0 to 2.6.3 in /python by @dependabot[bot] in #767
• Update README.md to fix broken network motif tutorial link by @russelljurney-upside in #770
• revert ConnectedComponents back to 0.9.3 version to fix perf regression by @james-willis in #772

New Contributors

• @JeganVG made their first contribution in #766
• @russelljurney-upside made their first contribution in #770

Full Changelog: v0.10.0...v0.10.1

What's Changed

• feat: initial commit with a copy of GraphX code by @SemyonSinchenko in #680
• chore: update image paths in README files to new directory location by @SemyonSinchenko in #683
• feat: performance improvement in GraphX CDLP by @SemyonSinchenko in #681
• feat: fix memory leaks in GraphX by @SemyonSinchenko in #687
• feat: improvements in GF-based algorithms performance by @SemyonSinchenko in #690
• feat: add LDBC Connected Components example by @SemyonSinchenko in #692
• hotfix: java 8 did not have Path.of by @SemyonSinchenko in #694
• feat: variable length pattern matching by @goungoun in #685
• doc: update docs by @SemyonSinchenko in #699
• feat: performance and API sync by @SemyonSinchenko in #698
• feat: add validate method by @SemyonSinchenko in #695
• feat: add Rocha–Thatte cycle detection algorithm by @SemyonSinchenko in #700
• feat: aggregatemessages multiple columns by @SemyonSinchenko in #701
• chore: update Dockerfile by @SemyonSinchenko in #702
• feat: change default branch to "main" by @SemyonSinchenko in #703
• feat: add a test for pagerank by @SemyonSinchenko in #707
• feat: method to transform graph to undirected by @SemyonSinchenko in #705
• feat: simplified implementation of triplets by @SemyonSinchenko in #708
• feat: better TriangleCount by @SemyonSinchenko in #704
• doc: Documentation on setting up the environment by @joelrobin18 in #719
• chore: Unpersist cached data from the test suite by @joelrobin18 in #720
• feat: refactoring of python && docs && sync pyspark with core by @SemyonSinchenko in #711
• feat: add bidirectional motif pattern by @goungoun in #717
• feat: add undirected edge motif pattern by @goungoun in #712
• Improve result formatting for variable length pattern and undirected pattern by @goungoun in #728
• bug: prevent negated bidirectional edge by @goungoun in #739
• Add Graphframe Type Degree Methods by @joelrobin18 in #742
• fix: make py4j pyspark api robust by @SemyonSinchenko in #743
• feat: K-Core by @SemyonSinchenko in #740
• feat: reduce memory consumption of cycles detection by @SemyonSinchenko in #731
• feat: handle edges directions in SP by @SemyonSinchenko in #737
• feat: maximal independent set by @SemyonSinchenko in #744
• feat: prepare the release 0.10.0 by @SemyonSinchenko in #725

New Contributors

• @goungoun made their first contribution in #685
• @joelrobin18 made their first contribution in #719

Full Changelog: v0.9.3...v0.10.0

What's Changed

• bug: Don't publish root project by @Kimahriman in #661
• feat: add tests for the case from #375 by @SemyonSinchenko in #663
• feat: localCheckpoints + docs by @SemyonSinchenko in #662
• feat: replace UDF by built-in function in skewedJoin by @SemyonSinchenko in #666
• feat: initial top-level structure of PropertyGraph by @SemyonSinchenko in #613
• fix: update javadocs to fix the build by @SemyonSinchenko in #670
• feat: S-sized benchmarks by @SemyonSinchenko in #671
• feat: updates in SBT and new site engine by @SemyonSinchenko in #673
• hotfix: fix broken links in docs by @SemyonSinchenko in #675
• feat: prepare 0.9.3 release by @SemyonSinchenko in #677

Full Changelog: v0.9.2...v0.9.3

Overview

• fix publishing issues
• fix docs publishing issues

What's Changed

• fix: publish connect sources and javadocs by @SemyonSinchenko in #654
• feat: fix docs path by @SemyonSinchenko in #656
• feat: 0.9.2 release by @SemyonSinchenko in #658

Full Changelog: v0.9.1...v0.9.2

Overview

Mostly this release is about publishing fixes (version string, README, PySpark Classic dependencies, etc.). There are no changes in core or API.

What's Changed

• Fix versioning by @Kimahriman in #641
• bug: Let classic python version work without grpc deps by @Kimahriman in #647
• fix: resolve gf-connect is not published by @SemyonSinchenko in #645
• feat: prepare the 0.9.1 release by @SemyonSinchenko in #650

Full Changelog: v0.9.0...v0.9.1

Overview

• Spark Connect support
• Spark 4.x support
• Performance improvements in CC
• Updated API for Pregel
• GraphX-free impl of CDLP and SP
• New groupId io.graphframes
• New PyPi ID: graphframes-py
• A lot of additional changes and fixes
• Updated documentation

What's Changed

• Add JVM args to fix sbt.ForkMain error by @EnricoMi in #438
• Add Spark 3.5.0 support by @EnricoMi in #436
• Add log4j2.properties to test resources by @EnricoMi in #437
• Fix connected component by @WeichenXu123 in #454
• Update doc and releasing guide by @WeichenXu123 in #455
• Update quick-start.md to use latest build of GraphFrames jar by @rjurney in #461
• Java opt MaxPermSize --> MaxMetaspaceSize, includes for Java 17+ sun.nio.ch, java.lang ALL-UNNAMED by @rjurney in #464
• Upgrade github actions by @bjornjorgensen in #471
• Update to spark-version: 3.5.4 and fix Connected component suite by @bjornjorgensen in #478
• Add Type hints for Python code by @bjornjorgensen in #480
• Fix the bug I introduced previously when adapting build.sbt for JDK 11 by @rjurney in #476
• Update scala code by @bjornjorgensen in #479
• Use vertices SparkSession by @Nassizouz in #474
• Update .mailmap to include me by @rjurney in #482
• Update .mailmap to include rjurney by @rjurney in #483
• [DOCS] Adding new super cool logo to the docs by @SauronShepherd in #495
• adding favicon by @SauronShepherd in #497
• chore: drop sbt-spark-packages && update plugins by @SemyonSinchenko in #487
• Add Pull request template by @bjornjorgensen in #472
• doc: update badges in README by @SemyonSinchenko in #509
• chore: add scalafmt and re-format all the code by @SemyonSinchenko in #507
• 1 of 3: Build a graphframes Python package during the build process by @rjurney in #512
• Rjurney/motif tutorial code min by @rjurney in #520
• 3 of 3: Documentation cleanup and update. Added a motif finding tutorial. by @rjurney in #511
• chore: replace setup-scala action by @SemyonSinchenko in #522
• chore: suppress warnings about incomplete patterns by @SemyonSinchenko in #526
• chore: testing against both java-11 and java-17 by @SemyonSinchenko in #525
• chore: force python code style check in CI by @SemyonSinchenko in #527
• chore: pre-commit hook by @SemyonSinchenko in #534
• Convert nose / unittest tests to pytest by @rjurney in #524
• fixed a minor nit usage issue by @dmatrix in #535
• feat: power iteration clustering by @SemyonSinchenko in #529
• Add a missing image in preparation for docs release by @rjurney in #537
• feat: SparkConnect support by @SemyonSinchenko in #506
• [ConnectedComponents] Memory leak with unpersisted DataFrames in the last round by @SauronShepherd in #552
• feat: add early stopping to Pregel by @SemyonSinchenko in #550
• Changes the output label of CC to min of original ID by @architch in #320
• Quote column names with backticks by @EnricoMi in #370
• feat: add an experimental implementation of SP in GF by @SemyonSinchenko in #558
• feat: add LDBC tests by @SemyonSinchenko in #570
• chore: add "-Xlint" and scalafix checks && fix all errors && update pre-commit by @SemyonSinchenko in #575
• chore: automate docs publishing by @SemyonSinchenko in #566
• feat: add vertices "voting" ability and optional flags to Pregel by @SemyonSinchenko in #581
• feat: python publish by @SemyonSinchenko in #577
• feat: add a new mixin WithMaxIter and propogate it to GraphX CC by @SemyonSinchenko in #590
• Allow setting default ConnectedComponents configurations from spark config by @james-willis in #563
• Bump setuptools from 78.1.0 to 78.1.1 in /python by @dependabot[bot] in #596
• Removing unused file as per #585 by @Conor0Callaghan in #592
• feat: LabelPropagation without GraphX by @SemyonSinchenko in #587
• feat: sbt-ci-release by @SemyonSinchenko in #598
• Bump requests from 2.32.3 to 2.32.4 in /python by @dependabot[bot] in #601
• Bump protobuf from 5.29.4 to 5.29.5 in /python by @dependabot[bot] in #606
• Bump urllib3 from 2.4.0 to 2.5.0 in /python by @dependabot[bot] in #607
• feat: Scala publish CI GitHub Action by @SemyonSinchenko in #600
• feat: Spark 4.0 support by @Kimahriman in #608
• chore: Add issue templates by @SemyonSinchenko in #605
• chore: Cleanup assembly and shading by @Kimahriman in #617
• chore: fix shading problems and slightly update CI by @SemyonSinchenko in #615
• fix: try to fix publishing by @SemyonSinchenko in #619
• chore: add useLabelsAsComponents configuration for connected components by @SemyonSinchenko in #623
• doc: update README with details on nightly builds by @SemyonSinchenko in #622
• README and .gitignore update to add GraphFrames classes by @rjurney in #629
• allow useLabelsAsComponents to be set locally on the ConnectedComponents instance by @james-willis in #632
• feat: set plugin and update pyproject by @SemyonSinchenko in #634
• feat: planning release 0.9.0 by @SemyonSinchenko in #626
• fix: remove version.sbt by @SemyonSinchenko in #637

New Contributors

• @bjornjorgensen made their first contribution in #471
• @Nassizouz made their first contribution in #474
• @SauronShepherd made their first contribution in #495
• @SemyonSinchenko made their first contribution in #487
• @dmatrix made their first contribution in #535
• @architch made their first contribution in #320
• @james-willis made their first contribution in #563
• @dependabot[bot] made their first contribution in #596
• @Conor0Callaghan made their first contribution in #592
• @Kimahriman made their first contribution in #608

Full Changelog: v0.8.3...v0.9.0