Add a "doc" extra for documentation build dependencies by EliahKagan · Pull Request #1872 · gitpython-developers/GitPython · GitHub

Much as GitPython has a test extra to aid development by installing testing and--currently--most linting dependencies, this adds a doc extra that installs Sphinx and its theme and plugins. The test extra is built by parsing test-requirements.txt, and similarly the new doc extra is built by parsing doc/requirements.txt. The previous approach of running pip install -r doc/requirements.txt continues to work.

The following are updated to use the new extra:

The Read the Docs configuration in .readthedocs.yaml.
Documentation build testing step in pythonpackage.yml.
Documentation building tox environment in tox.ini.

Before doing this, I factored out shared logic in the way metadata files are read in setup.py, wrote that code in a more streamlined way, and removed what appear to be small unintentional complexities in how two unusual situations are handled:

The entire content of VERSION, which should always be exactly one line, is now used (except leading and trailing whitespace) as the version.
The requirements files and readme are, as the VERSION file already had been, now found relative to the setup.py file rather than the current directory, in the rare case that those two locations are different.

You might decide you actually only want that setup.py cleanup, and not the new doc extra. I believe it makes sense to have a doc extra because it is a convenient way to install documentation, and because adding a new extra for that is readily feasible without even though the project definition is not declarative, because the information is available for static inspection in its requirements file, which already exist. (In contrast, I have not proposed a new lint extra at this time, though I think we should have that, because it would currently involve either adding another requirements file or making it harder to statically inspect the dependencies with automated tools.)

However, to a large extent this comes down to personal preference, and there is a reason many projects use a requirements file for their documentation dependencies: it tends to be a reasonably okay way to pin them all exactly, including indirect (transitive) dependencies, for greater stability. Read the Docs advocates this. But GitPython is not doing this. Currently documentation builld dependencies deliberately support ranges to support building on different Python versions. That is compatible with pinning if a tool like poetry is used (when multiple sets of precisely pinned direct and transitive dependencies, for multiple versions and sometimes platforms, have to be maintained, one wants a tool that fully automates doing so). Also, I think there is no reason to think GitPython will pin documentation build dependencies in the future yet not pin other extra/optional dependencies such as those for testing and linting. I think there is some symmetry between these things, for GitPython (and for many other projects).

Theory you don't have to read that also isn't correct in real life

In principle:

Applications use a requirements.txt file, or something conceptually similar like Pipfile.lock, that lists all their Python package dependencies fully pinned, including indirect dependencies (which, for Python packages, are the same thing as transitive dependencies), and typically express unpinned or less pinned direct dependencies separately in a requirements.in file, or something conceptually similar like Pipfile.

Libraries define their dependencies in setup.py, setup.cfg, or pyproject.toml, sometimes in a way affected by the choice of build backend, sometimes with the aid of dependency management tools such as poetry and sometimes not. The idea is that applications need specific pinned dependency versions to guarantee stability, while in contrast libraries need to accommodate the varying versions of whatever applications use them, either indirectly or through other libraries.

The problems are that the distinction between an application and a library is more permeable than it may seem at first, and that there are various benefits of defining an application like a library (for example, a requirements.txt file will not facilitate automatically installing executable scripts, and will not facilitate finding the code from a different location, even a subdirectory within its source tree), and also that when automated tests are run on a library, it is being used like an application and there can even be a benefit to precisely pinning the test runner and other test dependencies.

However, if you prefer not to take the addition of the "doc" extra, but do want the other changes to setup.py, then I can drop the commit that actually does the "doc" extra and the PR can be retitled accordingly.

I have locally tested the parts of this change that are not automatically exercised by GitHub Actions checks and Read the Docs pull request builds:

python -m build to build GitPython itself.
Editable and non-editable installations.
git.__version__ when imported from a non-editable installation, both from a prebuilt wheel and with pip omitting -e.
Ability to install locally with the doc extra on a Windows environment set up dissimilarly to how Windows is set up on GitHub Actions runners.
Ability to run make -C doc html successfully when the only installation command run was pip install -e '.[doc]' (as it would be used locally).

Here's the Read the Docs build status for this pull request (passing at 74df5a8).

While setuptools has official status, it is not actually part of the standard library (and since Python 3.12 cannot be treated as if it is, since it is not installed by default), so its imports belong in the second group rather than the first, per PEP-8.

This is partly a refactoring, reducing logic duplication and slightly decreasing overall length of the code that reads values from files. However, it does make two small behavioral changes: - In addition to the VERSION file for which this was already the case, requirements files and the readme are also taken relative to the directory that contains the setup.py file, rather than relative to the current directory if different. Supporting reading the readme and dependencies realtive to a different directory doesn't seem intended, and could lead to confusion in some cases. - The VERSION file is now read entirely and whitespace stripped, rather than reading just the first line. The VERSION file should always contain exactly one line, and this logic is simpler. This changes code for the reading of metadata only. It does not make any changes to the version stamping code.

And use it in: - GitHub Actions CI checks - Read the Docs configuration - tox (for the "html" environment) (The tox "html" environment was not previously overriding "extras" to empty it out of dependencies needed only for linting or testing, so this happens to make `tox -e html` faster.)

Thanks a lot for this great simplification!

I absolutely trust you to make the right decisions on how to do things, particularly when it comes to the 'meta' of the project like tooling. You know much better than me, I really am not part of the python world anymore and you are the expert.

Thanks as well for the primer on how python applications and libraries use 'lock' files or something akin to these. It's the same in the Rust world, even though gitoxide applies a mix of both. Cargo.lock is used for stable builds and is thus tracked by Git, even though versions in Cargo.toml files are only marking the minimal usable version so applications will be getting the most recent compatible version of dependencies accordingly.

EliahKagan added 3 commits March 13, 2024 17:09

Group setup.py imports

1b43166

While setuptools has official status, it is not actually part of the standard library (and since Python 3.12 cannot be treated as if it is, since it is not installed by default), so its imports belong in the second group rather than the first, per PEP-8.

EliahKagan force-pushed the doc-extra branch from d36038c to 74df5a8 Compare March 13, 2024 23:54

EliahKagan marked this pull request as ready for review March 14, 2024 00:04

Byron approved these changes Mar 14, 2024

View reviewed changes

Byron merged commit 883b03a into gitpython-developers:main Mar 14, 2024

EliahKagan deleted the doc-extra branch March 14, 2024 13:39

Add a "doc" extra for documentation build dependencies by EliahKagan · Pull Request #1872 · gitpython-developers/GitPython · GitHub

Uh oh!

Uh oh!

Add a "doc" extra for documentation build dependencies - #1872

Add a "doc" extra for documentation build dependencies#1872
Byron merged 3 commits into
gitpython-developers:mainfrom
EliahKagan:doc-extra

EliahKagan commented Mar 13, 2024 •
edited
Loading

Uh oh!

Uh oh!

Byron left a comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Add a "doc" extra for documentation build dependencies by EliahKagan · Pull Request #1872 · gitpython-developers/GitPython · GitHub

Uh oh!

Uh oh!

Add a "doc" extra for documentation build dependencies - #1872

Add a "doc" extra for documentation build dependencies#1872Byron merged 3 commits intogitpython-developers:maingitpython-developers/GitPython:mainfrom EliahKagan:doc-extraEliahKagan/GitPython:doc-extraCopy head branch name to clipboard

Conversation

EliahKagan commented Mar 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Byron left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Add a "doc" extra for documentation build dependencies#1872
Byron merged 3 commits into
gitpython-developers:mainfrom
EliahKagan:doc-extra

EliahKagan commented Mar 13, 2024 •
edited
Loading