FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

ENH, DOC: Add support for interactive examples for NumPy with `jupyterlite-sphinx` by agriyakhetarpal · Pull Request #26745 · numpy/numpy · GitHub

/ numpy Public

ENH, DOC: Add support for interactive examples for NumPy with jupyterlite-sphinx - #26745

Merged
rgommers merged 61 commits into
numpy:mainfrom
agriyakhetarpal:interactive-docs
Jan 28, 2025
Merged

rgommers merged 61 commits into
numpy:mainfrom
agriyakhetarpal:interactive-docs

Conversation

agriyakhetarpal commented Jun 18, 2024
edited
Loading

Copy link
Copy Markdown
Contributor

Description

This PR adds support for WASM-powered interactive examples with JupyterLite and jupyterlite-sphinx in the NumPy documentation.

A similar enhancement was made for the documentation for SciPy (scipy/scipy#20019) and that for PyWavelets (PyWavelets/pywt#728).

How to test this

Tip

I have a GitHub Pages website deployed at https://agriyakhetarpal.github.io/numpy/ through which these changes can be experimented with. The CircleCI deployment does not render the examples properly, however, a fix is available as reported in #26745 (comment) and will be fixed upstream in jupyterlite/jupyterlite-sphinx#182.

Update, 24/07/2024: this fix is now available in jupyterlite-sphinx version 0.16.2, and the CircleCI deployment should render the examples properly as well.

Key changes (so far)

  1. Configuration for jupyterlite-sphinx in doc/source/conf.py and doc/Makefile, coupled with changes to styling in doc/source/_static/numpy.css
  2. Updates to the API documentation under files in numpy/ under the "Examples" section in the docstrings for various modules

Note

I have currently left out the following: _datasource.py, _utils_impl.py, _version.py, _array_api_info.py, ctypeslib.py, files under numpy/_typing/, and numpy/_core/numerictypes.py.

Tip

The "NumPy user guide" section has not been modified at this time due to the size of the PR (in terms of the number of files modified). However, it can be modified or adapted to a Jupytext notebook similar to scipy/scipy#20303 to make it work well in an interactive set-up.

Important

The NumPy documentation will have to be rebuilt and reuploaded in https://github.com/numpy/doc/ for these changes to show up in the official documentation.

What issue does this PR address?

Closes #25969

Additional context

charris commented Jun 18, 2024

Copy link
Copy Markdown
Member

This could use a release note.

Comment on lines +470 to +485
.. try_examples::

>>> import numpy as np

>>> a = np.memmap('newfile.dat', dtype=float, mode='w+', shape=1000)
>>> a[10] = 10.0
>>> a[30] = 30.0
>>> del a

>>> b = np.fromfile('newfile.dat', dtype=float)
>>> print(b[10], b[30])
10.0 30.0

>>> a = np.memmap('newfile.dat', dtype=float)
>>> print(a[10], a[30])
10.0 30.0

agriyakhetarpal Jun 18, 2024
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I haven't tested whether every example works because there are way too many, but I did test this and this is one of the examples that doesn't, and fails with the error

ValueError: mmap length is greater than file size

and reducing the shape from 1000 to 100 doesn't help: OSError: [Errno 43] No such device (edit: this error occurs on subsequent runs on the same cell because newfile.dat cannot be loaded)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm not sure if there is a way to enable the Pyodide file system or interact with Pyodide directly through a Pythonic interface like this.

Copy link
Copy Markdown
Contributor Author

This could use a release note.

@charris, I addressed this in 484fe56. Please let me know if a "Highlight" note would be okay or if I should switch this to new_feature.

[skip cirrus] [skip azp]

Copy link
Copy Markdown
Contributor Author

I forgot to skip Azure Pipelines and Cirrus in my previous commit – please don't mind the force-push, my apologies

agriyakhetarpal marked this pull request as ready for review June 18, 2024 17:48

agriyakhetarpal commented Jun 18, 2024
edited
Loading

Copy link
Copy Markdown
Contributor Author

I notice that for some reason the CircleCI build isn't able to load the examples, most likely because the files in source/contents/ weren't copied or that jupyterlite-pyodide-kernel was missing. I shall investigate.

Edit: I think I figured it out – CircleCI is storing the notebooks in a different path, but GitHub Pages does not do so.

Please use my GitHub Pages site to test things out: https://agriyakhetarpal.github.io/numpy/reference/maskedarray.generic.html, I have another branch at agriyakhetarpal#2 and I shall keep both of them in sync based on further commits that I push here.

melissawm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I also don't see the try examples button locally - should I do anything differently other than spin docs?

Comment thread doc/source/_static/numpy.css Outdated

Copy link
Copy Markdown
Contributor Author

I also don't see the try examples button locally - should I do anything differently other than spin docs?

@melissawm, I'm not sure – I think you'll need to refresh your conda/mamba environment, to load the new dependencies or install requirements/doc_requirements.txt, perhaps? I am able to see the buttons locally.

Copy link
Copy Markdown
Member

Nevermind - I do see it now!

Copy link
Copy Markdown
Contributor Author

d818f8c disables source maps for JupyterLite, based on scikit-learn/scikit-learn#26246. This brings the size of the JupyterLite build in the doc/build/lite/ folder from 63.3 MiB (out of the total docs size of 180.7 MiB) to 22.4 MiB (with a total docs size of 140.2 MiB), which is a 22.4% reduction in build size. While doing this has the disadvantage of making debugging slightly harder for end users, it is favourable for the aspect of saving space, especially for GitHub Pages and https://github.com/numpy/doc. I've employed the same configuration for sympy/sympy#27419 as well.

This reverts commit ef3f14b.

[skip actions] [skip azp] [skip cirrus]

Copy link
Copy Markdown
Contributor Author

Hi @rgommers, we published https://github.com/jupyterlite/jupyterlite-sphinx/releases/tag/v0.18.0 some moments ago (thanks @steppi!), and I've updated the versions of jupyterlite-sphinx and jupyterlite-pyodide-kernel in 0713994 and 369e27d – we are good to proceed now, thanks!

[skip actions] [skip azp] [skip cirrus]
[skip actions] [skip azp] [skip cirrus]

Copy link
Copy Markdown
Contributor Author

Commit e7972a5 bumps to https://github.com/jupyterlite/pyodide-kernel/releases/tag/v0.5.1 that was released today, which brings the recent https://github.com/pyodide/pyodide/releases/tag/0.27.1.

(Apologies for the force push!)

melissawm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Hi @agriyakhetarpal ! The changes look good to me, I've successfully built this PR locally and it seems to work (although the jupyterlite blocks don't work locally).

The CircleCI version of the "Try it" works fine, but when I click "Open in new tab" I see this:

Is this expected? (Maybe it's because it's in CI)

agriyakhetarpal commented Jan 21, 2025
edited
Loading

Copy link
Copy Markdown
Contributor Author

Hi @melissawm, thanks for the review! I don't think I can reproduce it with CircleCI. Could you please retry with incognito mode, or with a different internet connection or mobile data if you can? This is what I get:



For reference, I tried this link: https://output.circle-artifacts.com/output/job/4b11fff4-e43b-4010-af4e-de2e84deed66/artifacts/0/doc/build/html/lite/notebooks/index.html?path=88efbc39_8e7b_4e21_86c0_519bb66f4f6a.ipynb

Based on your screenshot, I guess you're using Firefox. I tried with it as well, both with incognito mode and without, and the JupyterLite UI loads perfectly.

although the jupyterlite blocks don't work locally

They should work locally as well. Could you please share any errors or output that you might be seeing?

Copy link
Copy Markdown
Member

I think they had a partial outage eariler today so that may have been just bad luck. I'll wait for the check to finish here and test again.

melissawm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Now it works fine! Thanks @agriyakhetarpal

rgommers left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This looks pretty clean, nice work @agriyakhetarpal! Time to give this a go. Thanks to @melissawm and everyone else who contributed as well.

@agriyakhetarpal could you update us on the next steps here? I think it includes upgrading to NumPy 2.2 in Pyodide and enabling use of nightlies, and probably also enabling functionality on the User Guide? Anything else?

rgommers merged commit 62b43b9 into numpy:main Jan 28, 2025
rgommers added this to the 2.3.0 release milestone Jan 28, 2025
agriyakhetarpal deleted the interactive-docs branch January 28, 2025 21:45

agriyakhetarpal commented Jan 28, 2025
edited
Loading

Copy link
Copy Markdown
Contributor Author

@agriyakhetarpal could you update us on the next steps here? I think it includes upgrading to NumPy 2.2 in Pyodide and enabling use of nightlies, and probably also enabling functionality on the User Guide? Anything else?

Thanks, @rgommers! The following steps here are indeed those, and also, we need to add some docs to resolve gh-27310, similar to @melissawm's SciPy PR (scipy/scipy#22385, which I'll be reviewing and we can adopt similar messaging for NumPy when done).

Copy link
Copy Markdown
Member

Thanks Agriya. Docs would be a nice next step here indeed, leveraging the work Melissa did for SciPy (the setups and considerations are almost identical). It'd also be fine with me to have a shorter explanation and link to the SciPy docs on the topic for more details.

Upgrading to NumPy 2.2 in Pyodide was last blocked due to [...]

This would be good to prioritize. It seems like there was a way forward (or two possibly) there, by reverting making symbols private for example.

Re User Guide: agreed with waiting for better tracebacks and doing 1-2 pages first.

charris commented Dec 1, 2025

Copy link
Copy Markdown
Member

@agriyakhetarpal try_examples.json is giving a parsing error when I try updating circleci to Python 3.13. It looks like perfectly good json, so I wonder where the problem is. Maybe the environment? Any ideas?

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DOC: Add interactive examples with jupyterlite-sphinx

8 participants


Back | FazBrowse Home | New Git URL