FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Circle: Build docs in parallel. by QuLogic · Pull Request #20718 · matplotlib/matplotlib · GitHub

  1. matplotlib
  2. matplotlib

Repository navigation

Circle: Build docs in parallel. - #20718

Merged
tacaswell merged 2 commits into
matplotlib:mainfrom
QuLogic:circle-parallel
Aug 9, 2022
ConversationCommits (2)ChecksFiles changed
Merged

Circle: Build docs in parallel.#20718
tacaswell merged 2 commits into
matplotlib:mainfrom
QuLogic:circle-parallel

Conversation

QuLogic commented Jul 22, 2021

Copy link
Copy Markdown
Member

PR Summary

We are supposedly on medium Docker VMs, which offer 2 vCPUs, so we should try to build in parallel.

PR Checklist

  • [n/a] Has pytest style unit tests (and pytest passes).
  • Is Flake 8 compliant (run flake8 on changed files to check).
  • [n/a] New features are documented, with examples if plot related.
  • Documentation is sphinx and numpydoc compliant (the docs should build without error).
  • [n/a] Conforms to Matplotlib style conventions (install flake8-docstrings and run flake8 --docstring-convention=all).
  • [n/a] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
  • [n/a] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).

jklymak commented Jul 22, 2021

Copy link
Copy Markdown
Member

Looks like the build ran out of memory?

jklymak marked this pull request as draft July 27, 2021 13:53

QuLogic commented Aug 4, 2022 •
edited
Loading

Copy link
Copy Markdown
Member Author

On recent Circle builds, I noticed it said we have access to a higher resource class, so I bumped it up to large, which will hopefully mean it won't run out of RAM this time, and will be a little bit faster with double the vCPU.

QuLogic added 2 commits August 4, 2022 22:53
Circle: Increase resource class to large
9ac36f3 
This doubles the vCPU and RAM, which will hopefully be enough to run in
parallel.
Circle: Build docs in parallel.
afef2e6 
We are now on large Docker VMs, which offer 4 vCPUs, so we should try to
build in parallel.

QuLogic commented Aug 5, 2022

Copy link
Copy Markdown
Member Author

Re-reading the resource class docs again, it looks like the system really has 32 cores, and since I passed -jauto, it probably was running in 32 processes, which is likely to OOM on any amount of RAM. Testing locally, single-core takes 2.4GB, 2-process takes 2.5GB, 3-process takes 2.65GB, and 4-process takes 3.1GB. The times are 19.6min, 17.1min, 15.7min, and 15.3min, respectively. Unfortunately, sphinx-gallery takes half the time and doesn't run in parallel, so we can't save too much.

But we might as well save the 5 minutes by going to -j4.

QuLogic marked this pull request as ready for review August 5, 2022 04:18
tacaswell added this to the v3.6.0 milestone Aug 9, 2022
tacaswell merged commit d8005dd into matplotlib:main Aug 9, 2022
QuLogic deleted the circle-parallel branch August 9, 2022 19:31
StefRe mentioned this pull request Sep 30, 2022
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

timhoffm timhoffm approved these changes

Assignees

No one assigned

Labels

Build Documentation

Projects

None yet

Milestone

v3.6.0

Development

Successfully merging this pull request may close these issues.

4 participants

Back | FazBrowse Home | New Git URL