Doc building is failing. There is a warning about missing files, but I don't think that is the cause:

WARNING: Path to light image logo does not exist: numpylogo.svgumpy.asarray_chkfinite
WARNING: Path to dark image logo does not exist: numpylogo_dark.svg

I couldn't rerun only the circleci CI.

Yeah, no idea... just closing/reopening to see if that helps.

Still beats, me... @tupui maybe a long shot, but do you happen to know if this is known from pydata-sphinx-theme or so?! I don't see why this PR should include anything causing the build to trip building. The error is:

updating environment: [new config] 2587 added, 0 changed, 0 removed
WARNING: Path to light image logo does not exist: numpylogo.svgumpy.asarray_chkfinite

(Which is clearly a malformed path, but one that seems unrelated to anything here.)

array_chkfinite is just the first function docs/html we build, I believe. I don't understand why none of the other PRs seem to fail. Could there be some caching that means they simply don't rebuild or are getting other dependencies, or is there some odd cause here?

Does document building work locally?

Ok while debugging this I hit a nasty segmentation fault which completely froze my computer. I'll try to describe what I did:

1. Building locally following the usual steps in https://numpy.org/devdocs/dev/howto_build_docs.html doesn't work for me. I get errors about Matplotlib (I think?) being compiled with a different Python ABI. I realized this also has to do with a python=3.9 pin on our environment.yml file that I think needs to be updated (see MAINT: Update environment.yml to match *_requirements.txt #25308)
2. Next, I created a virtualenv environment to match what is described in https://github.com/numpy/numpy/blob/main/.circleci/config.yml This got me to the error we see here in CI. The logo warning is legitimate but shouldn't block the build (see DOC: Fix path to svg logo files #25309). The next error is the meaningful one:

Traceback (most recent call last):
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/cmd/build.py", line 298, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/application.py", line 355, in build
    self.builder.build_update()
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 293, in build_update
    self.build(to_build,
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 313, in build
    updated_docnames = set(self.read())
                           ^^^^^^^^^^^
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 418, in read
    self._read_parallel(docnames, nproc=self.app.parallel)
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 474, in _read_parallel
    tasks.join()
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/util/parallel.py", line 102, in join
    if not self._join_one():
           ^^^^^^^^^^^^^^^^
  File "/home/circleci/repo/venv/lib/python3.11/site-packages/sphinx/util/parallel.py", line 120, in _join_one
    exc, logs, result = pipe.recv()
                        ^^^^^^^^^^^
  File "/home/circleci/.pyenv/versions/3.11.4/lib/python3.11/multiprocessing/connection.py", line 249, in recv
    buf = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "/home/circleci/.pyenv/versions/3.11.4/lib/python3.11/multiprocessing/connection.py", line 413, in _recv_bytes
    buf = self._recv(4)
          ^^^^^^^^^^^^^
  File "/home/circleci/.pyenv/versions/3.11.4/lib/python3.11/multiprocessing/connection.py", line 382, in _recv
    raise EOFError
EOFError

Exception occurred:
  File "/home/circleci/.pyenv/versions/3.11.4/lib/python3.11/multiprocessing/connection.py", line 382, in _recv
    raise EOFError
EOFError
The full traceback has been saved in /tmp/sphinx-err-agt51dsa.log, if you want to report the issue to the developers.

So, because I've seen similar issues before, I tried building the docs with no parallel execution (removed -j2 from the docs build command). This gave me a segmentation fault the first time I tried it, and froze my linux computer completely the second time. No idea what this is but it looks like something coming from sphinx...

Docs seem to build fine with this PR checked out on an M3 Mac running Mac OS Sonoma.

That all seems very reasonable. The only one of these that is used in SciPy is PyArrayFlags_Type (once):

https://github.com/scipy/scipy/blob/fa9f13e6906e7d00510d593f7f982db30e4e4f14/scipy/sparse/linalg/_dsolve/_superlumodule.c#L352-L355

I can't say that it's immediately obvious to me what is going on there.

Well, let's see if rebasing magically fixes the doc build issue... I have not really tried to dig deeper into it after Melissa had a look.

The build is still failing. My guess is that there is a segfault (maybe a header is still there but the implementation is missing?) in the worker thread, it dies without reporting anything, and the main thread can no longer communicate with it. Perhaps changing the doc build to non-parallel might expose the segfault? We are building with

sphinx-build -b html -WT --keep-going -d build/doctrees  -j2 -n source build/html

maybe the -j2 could be forced to j1

Are we just being stupid? We have scipy.linalg imports in NumPy. And those do that nonsense INCREF that Ralf found. That is bound to segfault.

We change API to break ABI, and in the rare case where it actually matters, our doc builds are bound to be broken until downstream is recompiled.

EDIT: also scipy.signal import seems to be sufficient, likely more.

Are we just being stupid?

Probably :)

Let's see if scipy/scipy#19656 is merged quickly. But I think I am willing to gamble and live with a briefly broken doc build until new downstream nightlies are uploaded.

I'll get that SciPy PR merged and will see if I can get some new wheels up. There are a bunch of other failures preventing successful wheel uploads right now - we're kinda in circular dependencies territory at the moment.

OK, scipy update is through and sufficient to make the docs build pass here.

Btw. I think this one is ready to go in now.

Thanks @seberg

Hi - I just ran into this when attempting to compile ml_dtypes against the nightly numpy release. ml_dtypes package uses PyArray_TypeNumFromName here: https://github.com/jax-ml/ml_dtypes/blob/6f7b695c95004c9b9b1d070918a5091920624e0c/ml_dtypes/_src/custom_float.h#L873-L882 Hopefully the long comment gives enough context – can you suggest an alternative API to use for this case?

It might be that our best option is to simply remove the highlighted code block, as the TODO there has been satisfied by the creation of the ml_dtypes package.

I thought you can just grab it from Python via PyArray_DescrConverter(unicode_string("name")) or similar. If we support names for this, it seems like they need to work for Python also, and not be its own mechanism?

Thanks @seberg - I'll give that a try.