FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

fix(stats): preserve Unicode line separators in filenames · gitpython-developers/GitPython@f7cb262 · GitHub

Repository navigation

Commit f7cb262

Browse files
authored andcommitted
fix(stats): preserve Unicode line separators in filenames
Port PR #2301's regression for U+0085, U+2028 and U+2029 in filenames with `core.quotePath=false`. Both backends raised `ValueError` because `Commit.stats` and `Stats._list_from_string()` treated those characters as record separators in Git's output. Split only at LF and ignore empty records, preserving filenames and empty commit results. Keep the existing Gix filename-quoting CLI fallback; this change adapts Git's output and does not implement native diff behavior. Clarify that fallback in `doc/gix-backend.md` and record the user-visible fix. The scripted fixture creates root, modified and empty commits through Git. Direct NUL-delimited `git diff-tree --numstat` and `--name-status` queries validate the expected names, counts and change types before the GitPython assertions. The regression fails before the fix with both backends; all 42 commit/statistics tests and 21 subtests pass afterward on each backend. Ruff lint/format, changed-source mypy, ShellCheck and `git diff --check` pass.
1 parent 00eb12d commit f7cb262

6 files changed

Lines changed: 54 additions & 7 deletions

File tree

‎doc/gix-backend.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -366,7 +366,7 @@ repository-format validation gap.
366366
| Committer identity, `var` | Native process/config identity from `Repository.committer()` | Per-command identity overrides, other variables and normalization/identity errors; GIX-21 |
367367
| Selected `set_object` / reference deletion, `update_ref` | Native ref edits with GitPython's existing validation | Strict create/CAS, branch-target validation, symbolic aliases other than HEAD, symbolic detachment, active-branch HEAD log, unchanged targets and log-message cleanup; GIX-9/10/21/24 |
368368
| Tree/commit `Diffable.diff` | Raw change records and unambiguous exact renames | Patches, paths, index/worktree/root diff, inexact/ambiguous renames; GIX-11 |
369-
| `Commit.stats` | Native Myers line counts, binary counts, first-parent tree comparison | Diff attributes, other algorithms, gitlinks, quoted filenames, large blobs; GIX-2/18 |
369+
| `Commit.stats` | Native Myers line counts, binary counts, first-parent tree comparison | Diff attributes, other algorithms, gitlinks, non-ASCII or quoted filenames, large blobs; GIX-2/18 |
370370
| `Repo.is_dirty` | Index/tree/worktree status, configured submodule checks and pathspecs | Custom/sparse indexes and non-root command directories |
371371
| `Repo.untracked_files` | Native directory walk | Extra status options and non-root command directories |
372372
| `Repo.ignored` | Native excludes with tracked-file suppression | Symlink/submodule traversal, unsupported path normalization, and unavailable path metadata |

‎doc/source/changes.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,10 @@ Git executable and safety
5151
API changes
5252
-----------
5353

54+
* ``Commit.stats`` preserves Unicode line separators in filenames when
55+
``core.quotePath=false``. Git's LF-delimited output remains intact for root,
56+
subsequent and empty commits, including the Gix backend's existing filename
57+
quoting fallback.
5458
* ``GitConfigParser[section] = values`` again replaces a nonempty section,
5559
restoring Kas's Git identity cleanup. Git performs each mutation and locks
5660
it independently; replacement is not one atomic transaction. Empty sections

‎git/objects/commit.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -512,6 +512,8 @@ def stats(self) -> Stats:
512512
def process_lines(lines: List[str]) -> str:
513513
text = ""
514514
for file_info, line in zip(lines, lines[len(lines) // 2 :]):
515+
if not line:
516+
continue
515517
change_type = file_info.split("\t")[0][-1]
516518
(insertions, deletions, filename) = line.split("\t")
517519
text += "%s\t%s\t%s\t%s\n" % (change_type, insertions, deletions, filename)
@@ -520,12 +522,12 @@ def process_lines(lines: List[str]) -> str:
520522
if not self.parents:
521523
lines = self.repo.git._call_process_safe(
522524
"diff_tree", self.hexsha, "--", numstat=True, no_renames=True, root=True, raw=True
523-
).splitlines()[1:]
525+
).split("\n")[1:]
524526
text = process_lines(lines)
525527
else:
526528
lines = self.repo.git._call_process_safe(
527529
"diff", self.parents[0].hexsha, self.hexsha, "--", numstat=True, no_renames=True, raw=True
528-
).splitlines()
530+
).split("\n")
529531
text = process_lines(lines)
530532
return Stats._list_from_string(self.repo, text)
531533

‎git/util.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1051,7 +1051,9 @@ def _list_from_string(cls, repo: "Repo", text: str) -> "Stats":
10511051
"total": {"insertions": 0, "deletions": 0, "lines": 0, "files": 0},
10521052
"files": {},
10531053
}
1054-
for line in text.splitlines():
1054+
for line in text.split("\n"):
1055+
if not line:
1056+
continue
10551057
(change_type, raw_insertions, raw_deletions, filename) = line.split("\t")
10561058
insertions = raw_insertions != "-" and int(raw_insertions) or 0
10571059
deletions = raw_deletions != "-" and int(raw_deletions) or 0

‎test/test_commit.py‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,40 @@ def test_stats_preserve_filename_spaces(self):
191191
}
192192
assert removed_stats.total == {"insertions": 0, "deletions": 30, "lines": 30, "files": 5}
193193

194+
def test_stats_preserve_unicode_line_separators(self):
195+
paths = [f"before{separator}after.txt" for separator in ("\x85", "\u2028", "\u2029")]
196+
with fixture_repo("commit.sh", "unicode-separators") as repo:
197+
for revision, change_type in (("update-1", "A"), ("update-2", "M")):
198+
assert repo.git.diff_tree(
199+
"--root",
200+
"--no-commit-id",
201+
"--no-renames",
202+
"--numstat",
203+
"-z",
204+
revision,
205+
"--",
206+
stdout_as_string=False,
207+
) == b"".join(f"1\t0\t{path}\0".encode("utf-8") for path in paths)
208+
assert repo.git.diff_tree(
209+
"--root",
210+
"--no-commit-id",
211+
"--no-renames",
212+
"--name-status",
213+
"-z",
214+
revision,
215+
"--",
216+
stdout_as_string=False,
217+
) == b"".join(f"{change_type}\0{path}\0".encode("utf-8") for path in paths)
218+
stats = repo.commit(revision).stats
219+
assert stats.total == {"insertions": 3, "deletions": 0, "lines": 3, "files": 3}
220+
assert stats.files == {
221+
path: {"insertions": 1, "deletions": 0, "lines": 1, "change_type": change_type} for path in paths
222+
}
223+
assert repo.git.diff_tree("--no-commit-id", "--numstat", "-z", "empty", "--") == ""
224+
empty_stats = repo.commit("empty").stats
225+
assert empty_stats.files == {}
226+
assert empty_stats.total == {"insertions": 0, "deletions": 0, "lines": 0, "files": 0}
227+
194228
def test_renames(self):
195229
commit = self.rorepo.commit("185d847ec7647fd2642a82d9205fb3d07ea71715")
196230
files = commit.stats.files

‎tests/fixtures/commit.sh‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,18 @@ set -eu
33
git init --initial-branch=master repo
44
cd repo
55
case "$1" in
6-
spaces)
6+
spaces|unicode-separators)
77
# Construct trees directly: Windows rejects trailing spaces in index paths.
8+
paths=(name ' name' 'name ' ' name ' ' ')
9+
if [ "$1" = unicode-separators ]; then
10+
git config core.quotePath false
11+
paths=($'before\302\205after.txt' $'before\342\200\250after.txt' $'before\342\200\251after.txt')
12+
fi
813
for multiplier in 1 2; do
914
tree=$(
1015
count=0
11-
for path in name ' name' 'name ' ' name ' ' '; do
12-
count=$((count + 1))
16+
for path in "${paths[@]}"; do
17+
if [ "$1" = spaces ]; then count=$((count + 1)); else count=1; fi
1318
oid=$(for ((line=0; line<count*multiplier; line++)); do printf 'line\n'; done | git hash-object -w --stdin)
1419
printf '100644 blob %s\t%s\0' "$oid" "$path"
1520
done | git mktree -z

0 commit comments

Comments
 (0)

Back | FazBrowse Home | New Git URL