Repository navigation
fix(stats): preserve Unicode line separators in filenames - #2301
Closed
BHARTIYAYASH wants to merge 1 commit into
Closed
BHARTIYAYASH wants to merge 1 commit into
BHARTIYAYASH wants to merge 1 commit into
Conversation
Copy link
Copy Markdown
Sorry, something went wrong.
With `core.quotePath=false`, Git emits U+0085, U+2028 and U+2029 directly in filenames. `Commit.stats` and `Stats._list_from_string` used `str.splitlines()`, treating those filename characters as record endings and raising `ValueError` instead of returning statistics. Split only at the line feeds used by Git's protocol. Keep empty commits and the generated text's trailing newline working by ignoring empty records. No filename encoding or public return types change. The regression uses real Git output for all three Unicode separators and covers root, later and empty commits. It failed before this change. The complete commit/statistics modules passed: 44 tests and 8 subtests. Ruff lint/format, mypy for both changed source files, and `git diff --check` also passed.
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused parsing changes address the reported failure, include relevant regression coverage, and have no identified blocking issues.
No open issues 🎉
What changed in this PRThis PR fixes statistics parsing for valid filenames containing Unicode line separators without changing public types or filename encodings.
Changes:
- Split Git output only on line feeds, preserving Unicode separators within filenames.
- Skip empty records to handle empty commits and trailing newlines.
- Add regression coverage for all three separators across root commits, modifications, and an empty commit.
| File | Description |
|---|---|
| test/test_commit.py | Adds regression coverage using actual Git output. |
| git/util.py | Preserves Unicode separators when parsing statistics records. |
| git/objects/commit.py | Uses line-feed-only splitting for both commit statistics paths. |
Review effort: Balanced
Sorry, something went wrong.
Byron
pushed a commit
that referenced
this pull request
Oct 24, 2026
Port PR #2301's regression for U+0085, U+2028 and U+2029 in filenames with `core.quotePath=false`. Both backends raised `ValueError` because `Commit.stats` and `Stats._list_from_string()` treated those characters as record separators in Git's output. Split only at LF and ignore empty records, preserving filenames and empty commit results. Keep the existing Gix filename-quoting CLI fallback; this change adapts Git's output and does not implement native diff behavior. Clarify that fallback in `doc/gix-backend.md` and record the user-visible fix. The scripted fixture creates root, modified and empty commits through Git. Direct NUL-delimited `git diff-tree --numstat` and `--name-status` queries validate the expected names, counts and change types before the GitPython assertions. The regression fails before the fix with both backends; all 42 commit/statistics tests and 21 subtests pass afterward on each backend. Ruff lint/format, changed-source mypy, ShellCheck and `git diff --check` pass.
Byron
pushed a commit
that referenced
this pull request
Oct 24, 2026
Port PR #2301's regression for U+0085, U+2028 and U+2029 in filenames with `core.quotePath=false`. Both backends raised `ValueError` because `Commit.stats` and `Stats._list_from_string()` treated those characters as record separators in Git's output. Split only at LF and ignore empty records, preserving filenames and empty commit results. Keep the existing Gix filename-quoting CLI fallback; this change adapts Git's output and does not implement native diff behavior. Clarify that fallback in `doc/gix-backend.md` and record the user-visible fix. The scripted fixture creates root, modified and empty commits through Git. Direct NUL-delimited `git diff-tree --numstat` and `--name-status` queries validate the expected names, counts and change types before the GitPython assertions. The regression fails before the fix with both backends; all 42 commit/statistics tests and 21 subtests pass afterward on each backend. Ruff lint/format, changed-source mypy, ShellCheck and `git diff --check` pass.
Copy link
Copy Markdown
Member
|
Codex, operating on behalf of Byron: This PR has been superseded by #2274. Its regression coverage has been integrated there, with assertions checked against Git’s behavior and fixes included where needed. |
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With core.quotePath=false, Git emits U+0085, U+2028 and U+2029 directly in filenames. Commit.stats and Stats._list_from_string use str.splitlines(), which treats those filename characters as record boundaries. Statistics for these valid paths raise ValueError.
Change
Split Git's records only at line feeds, preserving filename characters. Skip empty records so empty commits and the generated text's trailing newline continue to work. No public types or filename encodings change.
The regression uses actual Git output and covers all three Unicode separators, root commits, subsequent modifications and an empty commit. Blobs are stored directly in the index, following the existing filename-whitespace regression.
Verification
Prepared and submitted by OpenAI Codex, an AI agent operating through BHARTIYAYASH's account. Human review is pending.