Codex/assertion text diff blocks by hamza-mobeen · Pull Request #14425 · pytest-dev/pytest · GitHub

Description

This PR introduces a new configuration option, assertion_text_diff_style, to allow users to customize how multiline string assertion failures are displayed.

As highlighted in #6757, when pytest uses ndiff to display differences for multiline strings (like output from capsys), it can sometimes become an unreadable "soup", especially when changes involve indentation or leading/trailing whitespace. The current behavior often results in a confusing line-by-line diff that is difficult to parse.

This PR adds an optional block diff style that displays the exact contents of the "Left" and "Right" text blocks intact and sequentially, making it significantly easier to read the literal multiline strings being compared without the interspersed ndiff noise.

Key changes:

Added assertion_text_diff_style configuration option. Defaults to ndiff to preserve backwards compatibility.
Implemented the block diff style (_diff_text_block) for multiline string comparisons, solving the readability issue raised in assertion diffs for multiline-string can become unreadable soup #6757.
Added fallback logic so that single-line comparisons still use ndiff even if block is selected.
Added strict validation to ensure assertion_text_diff_style is either ndiff or block.
Included extensive test coverage in testing/test_assertion.py verifying block formatting, fallback logic, blank line handling, and configuration validation.

Checklist

Include documentation when adding new features.
Include new tests or update existing tests when applicable.
Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.
Add text like closes #XYZW to the PR description and/or commits (where XYZW is the issue number).
If AI agents were used, they are credited in Co-authored-by commit trailers.
Create a new changelog file in the changelog directory, with a name like <ISSUE NUMBER>.<TYPE>.rst.
Add yourself to AUTHORS in alphabetical order.

Closes pytest-dev#8395 Co-authored-by: Antigravity <antigravity@google.com>

for more information, see https://pre-commit.ci

@RonnyPfannschmidt I'm still waiting on your feedback :)

This. Technically looks good but is also at odds with a number of comments on the original issue

As such id like to defer to @nicoddemus

Thanks @hamza-mobeen for the PR, overall great work!

I left some comments, please take a look.

Why is this necessary? Is the config being passed here different from the one already in util._config?

Also, given this is the only place where util.assertrepr_compare is called, consider just passing the text diff style directly to util.assertrepr_compare.

Not sure we should special case being multiline text or not; I think it is reasonable to always honor the block configuration, even for single line texts.

Instead of typing this as str, let's use Literal["ndiff", "block"]. This way we are more explicit, plus we can use match below.

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

for more information, see https://pre-commit.ci

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

Co-authored-by: Codex <codex@openai.com>

Thanks!

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

for more information, see https://pre-commit.ci

Hi,
I know this PR was merged over a month ago, but when I updated some packages, it broke one of my Pytest plugins:

# Use a different formatting than the default.
left_repr = format(left)
right_repr = format(right)

if left_repr != right_repr:
    highlighter = config.get_terminal_writer()._highlight
    verbose = config.get_verbosity(Config.VERBOSITY_ASSERTIONS)
    return ["", *_compare_eq_any(left_repr, right_repr, highlighter, verbose)] # missing argument

I know it's a bit of a minor issue, but please can you add default arguments that map to the previous version in the future:

def _compare_eq_any(
    left: object,
    right: object,
    highlighter: _HighlightFunc,
    verbose: int,
    assertion_text_diff_style: _AssertionTextDiffStyle = "ndiff", # added default argument
) -> Iterator[str]:

I can add a PR to add this change, but I'd be interested if you have other thoughts.

Add missing argument introduced in pytest-dev/pytest#14425

Private API is private for a reason. If you use it and it changes, that's on you and not on pytest.

@George-Ogden do you require supporting old versions of pytest? If not, just require the latest one and update the call to pass the argument explicitly.

Otherwise, one solution is to use different calls based on version, similar to pytest-dev/pytest-mock@1d42981.

@The-Compiler is right, we don't provide any guarantee about private APIs, so plugins need to be aware that they might change during releases (even patch releases). Adding a new default value won't help much, because then 9.1.0 and 9.1.1 would still contain the old code, which can cause some confusion.

@nicoddemus, thanks for your reply. I agree it's a bit late to add a default argument, and I've just changed the required pytest version for this plugin. It's not ideal to rely on a private API, but, for now, it's my best option for this use case.

Hopefully, you can bear this in mind in the future, as I'm not the only person who relies on the internal API. A quick search for _compare_eq_any on GitHub turns up quite a lot of results.

I hope you also enjoy this comic:

Hopefully, you can bear this in mind in the future, as I'm not the only person who relies on the internal API. A quick search for _compare_eq_any on GitHub turns up quite a lot of results.

For what it's worth, most of those seem to be copies of pytest's source code (e.g. people accidentally comitting their virtualenvs). Searching for the import or ignoring pytest copies I see pytest-dbg, pytest-unordered (utapyngo/pytest-unordered#22), and maybe two standalone Python scripts which might be affected by this.

hamza-mobeen and others added 2 commits April 23, 2026 12:41

Add datetime/timedelta support to pytest.approx (pytest-dev#8395 )

e720397

Closes pytest-dev#8395 Co-authored-by: Antigravity <antigravity@google.com>

Add block text diffs

5a2a014

psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Apr 27, 2026

pre-commit-ci Bot and others added 8 commits April 27, 2026 10:36

[pre-commit.ci] auto fixes from pre-commit.com hooks

2a40e38

for more information, see https://pre-commit.ci

Fix pre-commit lint and mypy errors

ef90179

Merge branch 'main' into codex/assertion-text-diff-blocks

b9e20c4

Add test to cover getini KeyError in mock_config

499aaa9

Merge main

e5f209e

[pre-commit.ci] auto fixes from pre-commit.com hooks

84dc2f4

for more information, see https://pre-commit.ci

Merge branch 'main' into codex/assertion-text-diff-blocks

4529bf4

Merge branch 'main' into codex/assertion-text-diff-blocks

d093267

RonnyPfannschmidt reviewed May 14, 2026

View reviewed changes

nicoddemus requested changes May 16, 2026

View reviewed changes

hamza-mobeen and others added 8 commits May 18, 2026 08:37

Update doc/en/how-to/output.rst

c1d79c6

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

[pre-commit.ci] auto fixes from pre-commit.com hooks

588e1a4

for more information, see https://pre-commit.ci

Update src/_pytest/assertion/util.py

1fe4671

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

Update doc/en/reference/reference.rst

68564ae

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

Update doc/en/reference/reference.rst

6ab0d7d

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

Address assertion text diff style review

b9d11e8

Co-authored-by: Codex <codex@openai.com>

Merge branch 'main' into codex/assertion-text-diff-blocks

f9bf927

Cover ndiff assertion text diff style

0eafb5f

Co-authored-by: Codex <codex@openai.com>

nicoddemus approved these changes May 18, 2026

View reviewed changes

Comment thread src/_pytest/assertion/util.py

Comment thread src/_pytest/assertion/util.py

nicoddemus and others added 3 commits May 18, 2026 12:52

Apply suggestions from code review

4e11a6b

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

[pre-commit.ci] auto fixes from pre-commit.com hooks

c051132

for more information, see https://pre-commit.ci

Update util.py

4cc27d7

nicoddemus reviewed May 18, 2026

View reviewed changes

Comment thread src/_pytest/assertion/util.py Outdated

Apply suggestion from @nicoddemus

8a69623

nicoddemus merged commit d6dbe52 into pytest-dev:main May 18, 2026
33 checks passed

George-Ogden mentioned this pull request Jun 20, 2026

Fix breaking change in Pytest 9.1 George-Ogden/pytest-dbg#40

Merged

George-Ogden added a commit to George-Ogden/pytest-dbg that referenced this pull request Jun 20, 2026

Fix breaking change in Pytest 9.1 (#40 )

7d0dcf9

Add missing argument introduced in pytest-dev/pytest#14425

Zac-HD mentioned this pull request Jul 22, 2026

Weird output for long string equality assert diffs #7127

Closed

+              ) -> list[str]:
+                  if (
+                      assertion_text_diff_style == ASSERTION_TEXT_DIFF_STYLE_BLOCK
+                      and _is_multiline_text(left, right)

		saved_config = util._config
		util._config = config

Codex/assertion text diff blocks by hamza-mobeen · Pull Request #14425 · pytest-dev/pytest · GitHub

Uh oh!

Uh oh!

Codex/assertion text diff blocks - #14425

Codex/assertion text diff blocks#14425nicoddemus merged 22 commits intopytest-dev:mainpytest-dev/pytest:mainfrom hamza-mobeen:codex/assertion-text-diff-blockshamza-mobeen/pytest:codex/assertion-text-diff-blocksCopy head branch name to clipboard

Conversation

hamza-mobeen commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

hamza-mobeen commented May 14, 2026

Uh oh!

RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

Uh oh!

nicoddemus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nicoddemus May 16, 2026

Choose a reason for hiding this comment

Uh oh!

nicoddemus May 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nicoddemus May 16, 2026

Choose a reason for hiding this comment

Uh oh!

nicoddemus May 16, 2026

Choose a reason for hiding this comment

Uh oh!

nicoddemus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

George-Ogden commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

The-Compiler commented Jun 20, 2026

Uh oh!

nicoddemus commented Jun 20, 2026

Uh oh!

George-Ogden commented Jun 20, 2026

Uh oh!

George-Ogden commented Jun 20, 2026

Uh oh!

The-Compiler commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Codex/assertion text diff blocks#14425
nicoddemus merged 22 commits into
pytest-dev:mainfrom
hamza-mobeen:codex/assertion-text-diff-blocks

hamza-mobeen commented Apr 27, 2026 •
edited
Loading

George-Ogden commented Jun 20, 2026 •
edited
Loading

The-Compiler commented Jun 20, 2026 •
edited
Loading