Search: "codex" term returns no results on devdocs · Issue #29542 · matplotlib/matplotlib · GitHub

On both devdocs and stable, when I include "codex" in the search I get no results
https://matplotlib.org/devdocs/search.html?q=stem+codex
https://matplotlib.org/stable/search.html?q=stem+codex

There is no problem for v3.9
https://matplotlib.org/3.9.3/search.html?q=stem+codex

or indeed for v3.10 if I explicitly use that url
https://matplotlib.org/3.10.0/search.html?q=stem+codex

Maybe you need to refresh the cached copy? I get the same results from your stable and 3.10.0 links.

Oh yeah, stable is fine after a hard refresh. I had somehow managed to break it on three different devices!

I still get no results on devdocs. Is that expected?

No, this isn't expected on devdocs either. We patch in codex to all examples:

matplotlib/doc/conf.py

Lines 350 to 373 in 91d115e

    
           # Monkey-patching gallery header to include search keywords 
        
           gen_rst.EXAMPLE_HEADER = """ 
        
           .. DO NOT EDIT. 
        
           .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. 
        
           .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: 
        
           .. "{0}" 
        
           .. LINE NUMBERS ARE GIVEN BELOW. 
        
           .. only:: html 
        
               .. meta:: 
        
                   :keywords: codex 
        
               .. note:: 
        
                   :class: sphx-glr-download-link-note 
        
                   :ref:`Go to the end <sphx_glr_download_{1}>` 
        
                   to download the full example code.{2} 
        
           .. rst-class:: sphx-glr-example-title 
        
           .. _sphx_glr_{1}: 
        
           """

but this is missing on devdocs:

--- a/stem_plot.html	2025-01-24 18:09:21.000000000 -0500
+++ b/stem_plot.html	2025-01-28 17:45:54.000000000 -0500
@@ -7,9 +7,8 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
-<meta content="codex" name="keywords" />
 
-    <title>Stem plot &#8212; Matplotlib 3.10.0 documentation</title>
+    <title>Stem plot &#8212; Matplotlib 3.11.0.dev398+g91d115ec16 documentation</title>

It could be a dependency change, but I don't see any change in Sphinx or pydata-sphinx-theme based on the page footers. It could be sphinx-gallery, but they haven't had a particularly recent release. @ksunden what version did you use to build 3.10.0 docs?

I used sphinx-gallery 0.18.0 (which is the most recent) I use a fresh venv when building docs, installing the doc dependencies and mpl from pypi.

Is there something different about the devdocs build?

Nothing major, that I can tell. For devdocs, we enable release mode, but even on PR builds (which are not in release mode), there's no keywords meta tag.

This now seems broken on published docs since 3.10.1 (no matter how much I hard-refresh!) If I build locally with a new conda environment, there is no issue 😕

Why has this worked at all? I don't find any documentation on .. meta:: or :keywords: in the Sphinx docs.
It's in https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#html-metadata
which is actually a docutils feature: https://docutils.sourceforge.io/docs/ref/rst/directives.html#meta

Should we just remove the feature? I don't think it's terribly important and not quite intuitive.

Anyway, having the docs claim that it works when it doesn't is not a good state.

I don't think it's terribly important and not quite intuitive.

I have used it quite a lot in the past, and have been frustrated since it stopped working.

The issue appears to be something to do with the parallel build. Locally, with a newly built mpl-dev conda environment

make clean html gives me docs where the "codex" search term is effective
make clean html O="-j4" gives me docs where the "codex" search term results in zero results

So the first check is whether the keyword is correctly placed in the .rst sources; e.g. grep codex gallery/lines_bars_and_markers/axline.rst should give :keywords: codex.

Also you can try to adding keywords in other plain rst doc files to exclude the influence of sphinx-gallery.

"codex" does not appear in any of the gallery rst files.
This change

index 259bd2f35e..82f8c4de6e 100644
--- a/doc/release/prev_whats_new/whats_new_3.9.0.rst
+++ b/doc/release/prev_whats_new/whats_new_3.9.0.rst
@@ -1,5 +1,8 @@
 .. redirect-from:: /users/prev_whats_new/whats_new_3.9.0
 
+.. meta::
+    :keywords: wibble
+
 =============================================
 What's new in Matplotlib 3.9.0 (May 15, 2024)
 =============================================

does insert the keyword into the header of the expected page

<!DOCTYPE html>


<html lang="en" data-content_root="../../" >

  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<meta content="wibble" name="keywords" />

    <title>What&#39;s new in Matplotlib 3.9.0 (May 15, 2024) &#8212; Matplotlib 3.11.0.dev1549+g4edc215ed.d20251214 documentation</title>

and searching for "wibble" gives me that page.

So it seems like sphinx-gallery in parallel mode is not letting us patch its header. Is that patching something that sphinx-gallery encourages or more of a hack on our part?

AFAICS there's no official mentioning of EXAMPLE_HEADER in the sphinx-gallery docs, so I believe it's just us monkey-patching sphinx-gallery.

So likely what is happening is that from conf.py we patch EXAMPLE_HEADER in the main thread, but the worker threads load the clean sphinx-gallery - config is passed into them as an object, and conf.py is not executed there. Most likely, we broke our monkey-patch ourselves through enabling parallel builds in #28617.

Working on it ...

I have established that we can re-instate it by monkey-patching for each example using the "reset_modules" option.

DOC: reinstate "codex" search term

Ah, that's clever. I've tried to do this with a custom sphinx extension, but that did not work since the paralleliztion of the write is not not the sphinx side. Instead it's a direct joblib.Parallel in sphinx_gallery, and since it spawns new processes it always has the unpatched version.

Mid-term we might want to see whether we can push something upstream to sphinx-gallery. But that'll need some consideration. Do we want customization of the example header? Do we want keyword support? If so, is it enough to define global keywords?

As an end user, I'd be pretty happy if sphinx-gallery injected the same keyword in all of its examples. Then once I know the keyword I can use it consistently in Matplotlib, Cartopy and various other packages' docs.

I think a globally hard-coded keyword is of limited use. The only suitable name would be "gallery". But this is the technical term of the project, not quite obvious or meaningful to end-users. (or probably even more correct and less handy "gallery-entry").

A sematic name would be better, but is not general, at there very minimum there are usecases for "example" and "tutorial".

I think the minimal viable approach would be to have a global config entry for the keyword.

Second thought: Maybe we can convince sphinx-gallery to hard-coded add the keyword "example" for a start. Even though sphinx-gallery wants to move away form the "example" terminology" because it does not fit all use cases. That terminology is still quite engrained and it may be ok to still put this in a keyword.

Search: "codex" term returns no results on devdocs · Issue #29542 · matplotlib/matplotlib · GitHub

Repository navigation

Search: "codex" term returns no results on devdocs #29542

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	# Monkey-patching gallery header to include search keywords
	gen_rst.EXAMPLE_HEADER = """
	.. DO NOT EDIT.
	.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
	.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
	.. "{0}"
	.. LINE NUMBERS ARE GIVEN BELOW.

	.. only:: html

	.. meta::
	:keywords: codex

	.. note::
	:class: sphx-glr-download-link-note

	:ref:`Go to the end <sphx_glr_download_{1}>`
	to download the full example code.{2}

	.. rst-class:: sphx-glr-example-title

	.. _sphx_glr_{1}:

	"""

Search: "codex" term returns no results on devdocs · Issue #29542 · matplotlib/matplotlib · GitHub

Repository navigation

Search: "codex" term returns no results on devdocs #29542

Description

Activity

QuLogic commented on Jan 28, 2025

rcomer commented on Jan 28, 2025

QuLogic commented on Jan 29, 2025

ksunden commented on Jan 30, 2025

QuLogic commented on Jan 30, 2025

rcomer commented on Jun 30, 2025

timhoffm commented on Dec 13, 2025

rcomer commented on Dec 14, 2025

timhoffm commented on Dec 14, 2025

rcomer commented on Dec 14, 2025

timhoffm commented on Dec 14, 2025

rcomer commented on Dec 14, 2025

timhoffm commented on Dec 14, 2025

rcomer commented on Dec 14, 2025

timhoffm commented on Dec 15, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions