0.8.0: make the numba acceleration discoverable, and add accelerated() - #23
Merged
andremsouza merged 3 commits intoJul 31, 2026
Merged
0.8.0: make the numba acceleration discoverable, and add accelerated()#23andremsouza merged 3 commits into
andremsouza merged 3 commits into
Conversation
Copy link
Copy Markdown
Owner
Sorry, something went wrong.
0.7.0's acceleration is invisible when it works, and there was no supported way to tell whether it had taken effect. A draft of the how-to reached for `from python_som._accelerate import bmu_kernel`, which is a private module and not something to publish. A function rather than a constant. `NUMBA_AVAILABLE` would have to resolve at import time, forcing the numba import on every `import python_som`; that costs about 80 ms and 0.7.0 deferred it deliberately. A function defers it to whoever asked for the answer. Named for the capability rather than the backend, so replacing numba later is not a rename. The docs still say numba where they tell you what to install. Calling it imports numba but does not compile the kernel: njit is lazy, so the roughly 500 ms compile happens on the first call that trains. Both figures are measured and in the docstring. Tested in both directions, in the two files that can each see only one of them: test_numba_kernel.py asserts True where numba is installed, test_core_boundary.py asserts False where it is not and skips if numba is unexpectedly present. A predicate checked one way round would pass while always returning the same answer. Two boundary assertions come with it. Calling accelerated() must not import numba when numba is absent, and `import python_som` must still not import it when present, which is exactly the invariant a new public name could break. The pinned public surface in test_core_boundary.py grows by one entry, so the addition is a decision rather than an accident.
0.7.0's Install section listed four pip commands and numba was not one of them. Its only user-facing mention was a bullet in the feature list, and the documentation site mentioned numba exactly once, in a caveat about MiniSom's development branch. Nothing told a reader the option existed. The Install section now carries the command, the NumPy consequence of running it, and a link to a new how-to. A packaging test asserts that section rather than the description as a whole, which is the distinction that matters here: the string was already present in 0.7.0, just nowhere a reader deciding what to install would look. The new page is a how-to and stays one. An earlier draft explained why numba is not an extra, quoted Kohonen on batch convergence, and described which phase of training dominates; all three are rationale, and a how-to that teaches is the failure the mode exists to avoid. Each now links to the explanation page that already covers it. What remains is four steps in order of effect and what each costs the reader. Three claims in that draft were wrong and are corrected. The confirmation snippet imported a private module. "A one-off compile of about a second" is 80 ms to import numba and 500 ms to compile, measured separately. "A 100x100 map is roughly 25 times the work of a 20x20 one" was invented and wrong in shape, since the update grows faster than node count; it now points at the measured table. The Kohonen rule of thumb was checked against Section 3.6, p. 56, which reads "about 50 items per node on the average". "About", not "at least".
Minor rather than patch: accelerated() is new public API, and semver puts new functionality in a minor release. Nothing else about it is minor-shaped, so the entry says plainly that no behaviour and no numerical results change, and there is no reproducibility note for the first time in three releases. The release exists because a PyPI description cannot be edited after upload, which is the same reason 0.6.1 existed. 0.7.0's description mentions numba only in its feature list, and the Install section a reader consults is what needed correcting.
Copy link
Copy Markdown
There was a problem hiding this comment.
Pull request overview
This PR prepares the 0.8.0 release by making the optional numba-accelerated BMU kernel discoverable to users and adding a new public predicate, python_som.accelerated(), to check whether the compiled kernel will be used.
Changes:
- Add public python_som.accelerated() (backed by the existing lazy bmu_kernel() resolver) and pin it into the public surface.
- Update README + MkDocs site with an explicit pip install numba install path and a new “Speed up training” how-to page.
- Add tests to assert accelerated() behavior in both “numba present” and “numba absent” environments, plus packaging tests that validate the README Install section content.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file| File | Description |
|---|---|
| uv.lock | Bumps local package version metadata to 0.8.0. |
| pyproject.toml | Bumps project version to 0.8.0. |
| src/python_som/_version.py | Bumps __version__ to 0.8.0. |
| src/python_som/_accelerate.py | Exposes accelerated() and updates module exports. |
| src/python_som/init.py | Re-exports accelerated and adds it to __all__. |
| README.md | Makes numba installation + accelerated() discoverable in Install section. |
| mkdocs.yml | Adds the new how-to page to navigation. |
| docs/how-to/speed-up-training.md | New how-to documenting practical training speed levers including numba. |
| CHANGELOG.md | Adds 0.8.0 release notes describing the new API and docs. |
| tests/test_numba_kernel.py | Adds tests for accelerated() in the numba-enabled environment. |
| tests/test_core_boundary.py | Pins accelerated in the public surface and tests “no numba” behavior + import boundaries. |
| tests/test_packaging.py | Adds tests asserting README Install section includes all installable options and numpy constraint note. |
src/python_som/_accelerate.py:37
- bmu_kernel() docstring still refers to a non-existent "fast" extra, but this module (and the README/docs) describe enabling acceleration via installing numba directly. This can mislead readers about how to enable the kernel.
__all__ = ["accelerated", "bmu_kernel"] @functools.cache def bmu_kernel() -> BmuKernel | None:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Sorry, something went wrong.
Comment thread
src/python_som/_accelerate.py
| from ._core._protocols import BmuKernel | ||
|
|
||
| __all__ = ["bmu_kernel"] | ||
| __all__ = ["accelerated", "bmu_kernel"] |
Comment thread
tests/test_packaging.py
Comment on lines
+106
to
+115
| def _readme_section(name: str) -> str: | ||
| """Return one ``## `` section of the README, which is also the PyPI description. | ||
|
|
||
| :param name: Heading text, without the ``## ``. | ||
| :return: The section body. | ||
| """ | ||
| lines = (PROJECT_ROOT / "README.md").read_text(encoding="utf-8").splitlines() | ||
| start = next(n for n, line in enumerate(lines) if line.strip() == f"## {name}") | ||
| end = next((n for n in range(start + 1, len(lines)) if lines[n].startswith("## ")), len(lines)) | ||
| return "\n".join(lines[start:end]) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
0.8.0. 0.7.0 shipped optional numba acceleration that a user had no way to find or to check. This fixes both. No behaviour changes and no numerical changes.
Why it needs a release rather than a docs push
The README is the PyPI long_description, and a published description cannot be edited. 0.7.0's Install section lists four pip install lines and numba is not among them; its only user-facing mention is one bullet in the feature list. The documentation site was worse: it mentioned numba exactly once, in a caveat about MiniSom's development branch.
Same reason 0.6.1 existed.
python_som.accelerated()
A function, not a constant. NUMBA_AVAILABLE would have to resolve at import time, forcing the numba import on every import python_som — 80 ms, which 0.7.0 deferred deliberately. A function defers it to whoever asked. Named for the capability rather than the backend, so replacing numba later is not a rename.
Calling it imports numba but does not compile the kernel: njit is lazy, so the ~500 ms compile happens on the first call that trains. Both figures measured.
This is why the release is a minor and not the patch originally proposed: new public API is new functionality.
Three things the draft how-to got wrong
Caught while planning, before shipping:
One citation was verified and held: Kohonen §3.6, p. 56 reads "about 50 items per node on the average". The draft's "about" was right; "at least" would not have been.
The how-to is a how-to
documentation-writer's own "a how-to that teaches" test flagged three drifts in my draft: it explained why numba is not an extra, quoted Kohonen on batch convergence, and described which training phase dominates. All rationale. Each now links to the explanation page that already covers it, leaving four steps in order of effect and what each costs the reader.
Tests
Verification
703 tests, 100% coverage. All five gates pass at each of the three commits independently, not just the tip. documentation-writer and llm-writing-patterns run over the new page and every changed file: zero severity-5 findings, zero em dashes. The built wheel's METADATA was unpacked and its Install section confirmed to contain the command and the numpy<2.5 caveat.
Correction to something I said earlier
I claimed the 0.7.0 PyPI publish self-approved and that my tag push caused an upload you had not authorised. That was wrong — the approval record shows state: approved, user: andremsouza. The pypi environment gate fired and you approved it. No workflow change is needed and none is in this PR.