3 of 3: Documentation cleanup and update. Added a motif finding tutorial. by rjurney · Pull Request #511 · graphframes/graphframes · GitHub

This is a documentation only change, the docs part of #473. The docs build and look okay when I serve them.

What changes were proposed in this pull request?

I added a network motif tutorial and updated / cleaned the docs generally. I added the new google group and the new logo to the images folder.

Why are the changes needed?

The docs were wrong in a few ways and there weren't even instructions in the README, which stopped new users from onboarding.

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (rjurney/motif-tutorial-code-min@0b566d0). Learn more about missing BASE report.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@                        Coverage Diff                         @@
##             rjurney/motif-tutorial-code-min     #511   +/-   ##
==================================================================
  Coverage                                   ?   90.73%           
==================================================================
  Files                                      ?       18           
  Lines                                      ?      896           
  Branches                                   ?      103           
==================================================================
  Hits                                       ?      813           
  Misses                                     ?       83           
  Partials                                   ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…s.txt and split out requirements-dev.txt. Version bumps.

This project should be merged following #512 and #513, with #513.

…ney/build-upgrades

…mes/graphframes into rjurney/motif-tutorial-docs

Okay @bjornjorgensen @SauronShepherd @SemyonSinchenko I am done here. I would maybe suggest reviewing it by reading docs/README.md and running jekyll to view the documentation. Give it a once over. I went through and fixed anything that was old, broken, out of date version, etc. I added new community resources and the graphframes-py package I reserved that will go out with 0.8.5. It's a big PR but this is english not code, and it cross-refers to itself and depends on the changes in the other two PRs this is based on: #512 <-- #518 <-- #511

…graphframes into rjurney/motif-tutorial-docs

@SemyonSinchenko this one is hard to review... lots of text changes. I have double checked it, but appreciate any review you can give :)

LGTM overall, thank you! The only important comment is about licenses. Other comments are minor.

Let's add a note about deprecation of GraphX in Spark 4.0? Something like "With deprecation of GraphX in Apache Spark Core GraphFrames is the recommended approach for the distributed graphs processing."?

@SemyonSinchenko for the moment - and we can get your own PR to do this if you want - I prefer to pretend that didn't happen. It isn't good for the project and we have a ways to go before merging with Spark.

Where is this one taken? I can assume that it is from a scientific paper that may have a license like Creative Commons or a proprietary one. I think it is important to maintain IP clearance if we are going to move GraphFrames to the Spark core one day. Could you please add a source or even better to add a NOTICE.md file to the root of project with mentioning of the source of that picture and how is it licensed?

@SemyonSinchenko It's from this paper, which the caption and image itself lead to if you click: https://www.computer.org/csdl/journal/tb/2017/06/07501518/13rRUxcsYKf

The thing is... there is no open license image of graphlets, let alone directed graphlets. What if I include it now and seek permission in the meanwhile? Then I can figure out some alternative and in the meanwhile people can still learn about graphlets? Worst case I can generate them all in networkx or something.

I reached out to the authors, asking for permission. It would be best to secure their permission. In reading this guide to fair use, I believe this image may qualify: https://www.copyrightlaws.com/simple-guide-fair-use/

I got permission to use the image :)

The same

Co-authored-by: Sem <ssinchenko@apache.org>

Documentation cleanup and update. Added a Motif Finding tutorial.

bfa76c3

rjurney requested a review from WeichenXu123 February 16, 2025 04:20

rjurney added 3 commits February 15, 2025 20:25

Updated latest version to v0.8.4, which I verified is in Spark-Packages

60d2ff3

Added GraphFrames and Maven / sbt instructions in README

42a13b0

v0.8.5 --> 0.8.4

1370a1a

rjurney changed the title Documentation cleanup and update. Added a Motif Finding tutorial. Documentation cleanup and update. Added a motif finding tutorial. Feb 16, 2025

rjurney mentioned this pull request Feb 16, 2025

Adding motif finding tutorial using the stats.meta.stackexchange.com data dump #473

Closed

rjurney added 3 commits February 15, 2025 20:58

Converted tests to pytest. Build a Python package. Update requirement…

f4e9cdb

…s.txt and split out requirements-dev.txt. Version bumps.

Restore Python .gitignore

c256244

Extra newline removed

6c3df0b

rjurney mentioned this pull request Feb 16, 2025

New Python tutorial module graphframes.tutorial #513

Closed

rjurney added 2 commits February 15, 2025 21:48

Fixed mistaken reference to stackexchange.py where motif.py is required

a583d57

Had XML package when I needed GraphFrames

73f6cc2

rjurney mentioned this pull request Feb 16, 2025

1 of 3: Build a graphframes Python package during the build process #512

Merged

rjurney added 14 commits February 16, 2025 11:46

Merge branch 'master' of github.com:graphframes/graphframes into rjur…

b2838d2

…ney/build-upgrades

Added VERSION file set to 0.8.5

caf5091

isort; fiex edgesDF variable name.

7cfa2d1

Merge branch 'master' of github.com:graphframes/graphframes into rjur…

2ca9a15

…ney/build-upgrades

Back out Dockerfile changes

a8bf0be

Back out version change in build.sbt

54a942d

Backout changes to config and run-tests

8b0e346

Back out pytest conversion

46c2b93

Back out version changes to make nose tests pass

18b5da0

Remove changes to requirements

8eca097

Put nose back in requirements.txt

277c06f

Remove version bump to version.sbt

b55ee48

Remove packages related to testing

f8a8fd9

Remove old setup.py / setup.cfg

bc2cb36

Merge branch 'rjurney/motif-tutorial-code-min' of github.com:graphfra…

1497e65

…mes/graphframes into rjurney/motif-tutorial-docs

rjurney changed the base branch from master to rjurney/motif-tutorial-code-min February 17, 2025 21:32

rjurney added 10 commits February 17, 2025 13:50

Rewrote README section to reflect new graphframes-py package

f84b746

Minor README cleanup

f8bb23d

White background logo added to README

73d85c6

Move image up top left

6410de4

Try this logo thing without to restore other iamges

011eeb8

Image replaces # graphframes

3165305

Logo 300->400px

2e51004

+note about GraphFrames is back

ec40c7c

note about Sphinx, removed all sudoes

704876a

Added sections on Community Resources

0662970

rjurney changed the title Documentation cleanup and update. Added a motif finding tutorial. 3 of 3: Documentation cleanup and update. Added a motif finding tutorial. Feb 18, 2025

rjurney added 4 commits February 21, 2025 12:30

Updated poetry.lock

bd5c91e

+sphinx to dev dependencies to build docs

dc75ad8

Merge branch 'rjurney/motif-tutorial-docs' of github.com:graphframes/…

f0f1e42

…graphframes into rjurney/motif-tutorial-docs

Forgot sphinx requires Python 3.10

3aaee45

rjurney mentioned this pull request Feb 21, 2025

Rjurney/motif tutorial code min #520

Merged

Merged rjurney/motif-tutorial-code-min

7b364f3

rjurney changed the base branch from rjurney/motif-tutorial-code-min to master February 21, 2025 12:32

Updated to use 'graphframes stackexchange' command

9aff7d7

SemyonSinchenko reviewed Feb 21, 2025

View reviewed changes

rjurney and others added 2 commits February 21, 2025 15:31

Rewrite of intro sentence in README.md

8859f95

Co-authored-by: Sem <ssinchenko@apache.org>

Rewrite of GraphFrames is back paragraph

e2328d6

Co-authored-by: Sem <ssinchenko@apache.org>

rjurney merged commit 7c08186 into master Feb 21, 2025

rjurney mentioned this pull request Mar 18, 2025

Merge Mercury Graph with GraphFrames BBVA/mercury-graph#40

Open

SemyonSinchenko mentioned this pull request Mar 28, 2025

Add a motif finding tutorial #493

Closed

rjurney deleted the rjurney/motif-tutorial-docs branch April 15, 2025 00:32

-              API for motif finding.  The user also benefits from DataFrame performance optimizations
-              within the Spark SQL engine.
+              This is a package for DataFrame-based graphs on top of Apache Spark. Users can write highly expressive queries by leveraging the DataFrame API, combined with a new API for network motif finding. The user also benefits from DataFrame performance optimizations within the Spark SQL engine. GraphFrames works in Java, Scala, and Python.

3 of 3: Documentation cleanup and update. Added a motif finding tutorial. by rjurney · Pull Request #511 · graphframes/graphframes · GitHub

Uh oh!

3 of 3: Documentation cleanup and update. Added a motif finding tutorial. - #511

3 of 3: Documentation cleanup and update. Added a motif finding tutorial.#511rjurney merged 78 commits intomastergraphframes/graphframes:masterfrom rjurney/motif-tutorial-docsgraphframes/graphframes:rjurney/motif-tutorial-docsCopy head branch name to clipboard

Conversation

rjurney commented Feb 16, 2025

What changes were proposed in this pull request?

Why are the changes needed?

Uh oh!

codecov-commenter commented Feb 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rjurney commented Feb 16, 2025

Uh oh!

rjurney commented Feb 17, 2025

Uh oh!

rjurney commented Feb 21, 2025

Uh oh!

SemyonSinchenko left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rjurney Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rjurney Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

3 of 3: Documentation cleanup and update. Added a motif finding tutorial.#511
rjurney merged 78 commits into
masterfrom
rjurney/motif-tutorial-docs

codecov-commenter commented Feb 16, 2025 •
edited
Loading

rjurney Feb 21, 2025 •
edited
Loading

rjurney Feb 21, 2025 •
edited
Loading