| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
1 parent 21a6c6a commit af1de6a
20 files changed
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -31,6 +31,19 @@ Attributing AI assistance in commit metadata, for example with a `Co-authored-by | |||
| 31 | 31 | trailer, is welcome but not required. Code is reviewed the same way regardless of its | |
| 32 | 32 | origin. | |
| 33 | 33 | ||
| 34 | + ## Temporary test directories | ||
| 35 | + | ||
| 36 | + Use `test.cleanup.TemporaryDirectory` for isolated temporary directories and | ||
| 37 | + `test.cleanup.cleanup_directory` when disposing of directories created by test | ||
| 38 | + fixtures. Close repository handles before removing their files. Cleanup is | ||
| 39 | + best-effort: it logs filesystem errors, removes whatever it can, and leaves | ||
| 40 | + locked files behind without failing or skipping a test. The shared writable | ||
| 41 | + repository decorators still keep failed tests' directories for debugging. | ||
| 42 | + | ||
| 43 | + Deletions and renames that exercise library behavior or prepare a fixture for | ||
| 44 | + reuse must remain strict. If a cached fixture cannot be cleaned up, rebuild it | ||
| 45 | + at a fresh location before handing it to another test. | ||
| 46 | + | ||
| 34 | 47 | ## Fuzzing Test Specific Documentation | |
| 35 | 48 | ||
| 36 | 49 | For details related to contributing to the fuzzing test suite and OSS-Fuzz integration, please | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -54,9 +54,11 @@ The prepared environments in this checkout are `.venv` (CLI) and `.tox/gix` | |||
| 54 | 54 | ``` | |
| 55 | 55 | ||
| 56 | 56 | The runner uses local version tags, creates an isolated Git configuration, | |
| 57 | - prepares the historical test fixture inside a temporary shared clone, and | ||
| 58 | - disables package-index access. Tests use local repositories, including the | ||
| 59 | - tutorial example. The suite needs loopback sockets for its Git daemon and | ||
| 57 | + prepares the historical test fixture inside a temporary shared clone, gives | ||
| 58 | + pytest a separate temporary root for each run, and disables package-index | ||
| 59 | + access. Cleanup of these isolated directories is best-effort, so a locked | ||
| 60 | + leftover cannot change pytest's exit status. Tests use local repositories, | ||
| 61 | + including the tutorial example. The suite needs loopback sockets for its Git daemon and | ||
| 60 | 62 | permission to inspect its own child processes. It does not need a remote Git | |
| 61 | 63 | server. Missing local tags or packages are errors, not invitations to download. | |
| 62 | 64 | ||
@@ -581,7 +583,7 @@ candidates, not proof that a shared fixture is safe in every execution order. | |||
| 581 | 583 | | --- | --- | --- | | |
| 582 | 584 | | 166 | `TExc` (157) and `TestActor` (9) inherit repository-building `TestBase`. | Use the existing `TestCase` base without repository setup. | | |
| 583 | 585 | | 182 | Three submodule rejection bodies repeatedly build `movable_submodule`, then check snapshots for no mutation. | Prepare logical-name baselines once and copy the parent per case, retaining fresh wrappers and independent writable files. | | |
| 584 | - | 51 | Six submodule rejection bodies prepare nested metadata, separate metadata, intermediate/leaf symlinks, or retained metadata before checking rejection. | Cache ten prepared layouts and restore complete copies at their original paths, preserving absolute Git links and symlinks. Cleanup removes the active copy even after failure. | | ||
| 586 | + | 51 | Six submodule rejection bodies prepare nested metadata, separate metadata, intermediate/leaf symlinks, or retained metadata before checking rejection. | Cache ten prepared layouts and restore complete copies at their original paths, preserving absolute Git links and symlinks. Cleanup is best-effort; a locked active copy invalidates the layout so the next case rebuilds at a fresh path. | | ||
| 585 | 587 | | 15 | Eight revision-query bodies rebuild the same four-commit graph, refs, index, and reflogs through `rev_parse_repo`. | Prepare the graph once and copy it for every consumer, including mutating cases; recreate repository, branch and commit wrappers. | | |
| 586 | 588 | | 8 | Tree lookup bodies clone and check out `0.3.2.1` through `with_rw_repo`. | Read the historical tree directly through the existing class repository, removing clones and checkouts. | | |
| 587 | 589 | ||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -460,8 +460,8 @@ def _clone_repo( | |||
| 460 | 460 | except FileNotFoundError: | |
| 461 | 461 | pass | |
| 462 | 462 | raise | |
| 463 | - cls._connect_module(module_checkout_path, module_abspath) | ||
| 464 | 463 | clone.close() | |
| 464 | + cls._connect_module(module_checkout_path, module_abspath) | ||
| 465 | 465 | clone = git.Repo(module_checkout_path) | |
| 466 | 466 | ||
| 467 | 467 | return clone | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -0,0 +1,81 @@ | |||
| 1 | + # This module is part of GitPython and is released under the | ||
| 2 | + # 3-Clause BSD License: https://opensource.org/license/bsd-3-clause/ | ||
| 3 | + | ||
| 4 | + """Best-effort disposal of isolated test directories, including on Python 3.8. | ||
| 5 | + | ||
| 6 | + Use these helpers only when discarding a directory owned by a test. Deleting or | ||
| 7 | + moving files to exercise Git behavior, or to prepare a reusable fixture, must | ||
| 8 | + still report errors. | ||
| 9 | + | ||
| 10 | + Keep this module independent of GitPython and pytest so the local test runner | ||
| 11 | + can use it before configuring the test environment. | ||
| 12 | + """ | ||
| 13 | + | ||
| 14 | + import logging | ||
| 15 | + import os | ||
| 16 | + import shutil | ||
| 17 | + import stat | ||
| 18 | + import sys | ||
| 19 | + import tempfile | ||
| 20 | + import weakref | ||
| 21 | + | ||
| 22 | + _logger = logging.getLogger(__name__) | ||
| 23 | + | ||
| 24 | + | ||
| 25 | + def cleanup_directory(path): | ||
| 26 | + """Try to remove an owned test directory; return False and log on filesystem errors.""" | ||
| 27 | + errors = [] | ||
| 28 | + | ||
| 29 | + def onerror(function, filename, exception): | ||
| 30 | + if isinstance(exception, FileNotFoundError): | ||
| 31 | + return | ||
| 32 | + if sys.platform == "win32" and function in (os.unlink, os.rmdir) and isinstance(exception, PermissionError): | ||
| 33 | + try: | ||
| 34 | + # Git files and test directories may be read-only. Never chmod a | ||
| 35 | + # symlink or junction target, which may be outside the owned tree. | ||
| 36 | + if not os.lstat(filename).st_file_attributes & stat.FILE_ATTRIBUTE_REPARSE_POINT: | ||
| 37 | + os.chmod(filename, stat.S_IWUSR) | ||
| 38 | + function(filename) | ||
| 39 | + return | ||
| 40 | + except FileNotFoundError: | ||
| 41 | + return | ||
| 42 | + except OSError as retry_error: | ||
| 43 | + exception = retry_error | ||
| 44 | + errors.append(exception) | ||
| 45 | + | ||
| 46 | + try: | ||
| 47 | + if sys.version_info >= (3, 12): | ||
| 48 | + shutil.rmtree(path, onexc=onerror) | ||
| 49 | + else: | ||
| 50 | + shutil.rmtree(path, onerror=lambda function, filename, excinfo: onerror(function, filename, excinfo[1])) | ||
| 51 | + except FileNotFoundError: | ||
| 52 | + pass | ||
| 53 | + except OSError as error: | ||
| 54 | + errors.append(error) | ||
| 55 | + | ||
| 56 | + if errors: | ||
| 57 | + _logger.warning("Could not fully remove temporary test directory %r: %s", os.fspath(path), errors[0]) | ||
| 58 | + return not errors | ||
| 59 | + | ||
| 60 | + | ||
| 61 | + class TemporaryDirectory: | ||
| 62 | + """A test-owned temporary directory whose cleanup cannot fail on file locks. | ||
| 63 | + | ||
| 64 | + Supports context management, ``name``, and explicit ``cleanup()``. A finalizer | ||
| 65 | + also attempts cleanup if a test drops the object without closing it. Explicit | ||
| 66 | + cleanup detaches that finalizer, but may be called again to retry later. | ||
| 67 | + """ | ||
| 68 | + | ||
| 69 | + def __init__(self, suffix=None, prefix=None, dir=None): | ||
| 70 | + self.name = tempfile.mkdtemp(suffix=suffix, prefix=prefix, dir=dir) | ||
| 71 | + self._finalizer = weakref.finalize(self, cleanup_directory, self.name) | ||
| 72 | + | ||
| 73 | + def __enter__(self): | ||
| 74 | + return self.name | ||
| 75 | + | ||
| 76 | + def __exit__(self, *args): | ||
| 77 | + self.cleanup() | ||
| 78 | + | ||
| 79 | + def cleanup(self): | ||
| 80 | + self._finalizer.detach() | ||
| 81 | + cleanup_directory(self.name) | ||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -42,10 +42,10 @@ | |||
| 42 | 42 | import venv | |
| 43 | 43 | from typing import Union, Type, Tuple | |
| 44 | 44 | ||
| 45 | - import gitdb | ||
| 46 | 45 | import pytest | |
| 47 | 46 | ||
| 48 | - from git.util import rmtree, cwd | ||
| 47 | + from git.util import cwd | ||
| 48 | + from test.cleanup import TemporaryDirectory, cleanup_directory | ||
| 49 | 49 | ||
| 50 | 50 | TestCase = unittest.TestCase | |
| 51 | 51 | SkipTest = unittest.SkipTest | |
@@ -107,7 +107,10 @@ def wait(self, stderr=None): | |||
| 107 | 107 | ||
| 108 | 108 | def with_rw_directory(func): | |
| 109 | 109 | """Create a temporary directory which can be written to, remove it if the | |
| 110 | - test succeeds, but leave it otherwise to aid additional debugging.""" | ||
| 110 | + test succeeds, but leave it otherwise to aid additional debugging. | ||
| 111 | + | ||
| 112 | + Cleanup is best-effort: a locked file must not change the test result. | ||
| 113 | + """ | ||
| 111 | 114 | ||
| 112 | 115 | @wraps(func) | |
| 113 | 116 | def wrapper(self, *args, **kwargs): | |
@@ -132,7 +135,7 @@ def wrapper(self, *args, **kwargs): | |||
| 132 | 135 | # though this is not the case here unless we collect explicitly. | |
| 133 | 136 | gc.collect() | |
| 134 | 137 | if not keep: | |
| 135 | - rmtree(path) | ||
| 138 | + cleanup_directory(path) | ||
| 136 | 139 | ||
| 137 | 140 | return wrapper | |
| 138 | 141 | ||
@@ -179,13 +182,10 @@ def repo_creator(self): | |||
| 179 | 182 | raise | |
| 180 | 183 | finally: | |
| 181 | 184 | os.chdir(prev_cwd) | |
| 182 | - rw_repo.git.clear_cache() | ||
| 185 | + rw_repo.close() | ||
| 183 | 186 | rw_repo = None | |
| 184 | 187 | if repo_dir is not None: | |
| 185 | - gc.collect() | ||
| 186 | - gitdb.util.mman.collect() | ||
| 187 | - gc.collect() | ||
| 188 | - rmtree(repo_dir) | ||
| 188 | + cleanup_directory(repo_dir) | ||
| 189 | 189 | # END rm test repo if possible | |
| 190 | 190 | # END cleanup | |
| 191 | 191 | ||
@@ -202,29 +202,17 @@ def git_daemon_launched(base_path, ip, port): | |||
| 202 | 202 | ||
| 203 | 203 | gd = None | |
| 204 | 204 | try: | |
| 205 | - if sys.platform == "win32": | ||
| 206 | - # On MINGW-git, daemon exists in Git\mingw64\libexec\git-core\, | ||
| 207 | - # but if invoked as 'git daemon', it detaches from parent `git` cmd, | ||
| 208 | - # and then CANNOT DIE! | ||
| 209 | - # So, invoke it as a single command. | ||
| 210 | - daemon_cmd = [ | ||
| 211 | - osp.join(Git()._call_process("--exec-path"), "git-daemon"), | ||
| 212 | - "--enable=receive-pack", | ||
| 213 | - "--listen=%s" % ip, | ||
| 214 | - "--port=%s" % port, | ||
| 215 | - "--base-path=%s" % base_path, | ||
| 216 | - base_path, | ||
| 217 | - ] | ||
| 218 | - gd = Git().execute(daemon_cmd, as_process=True) | ||
| 219 | - else: | ||
| 220 | - gd = Git().daemon( | ||
| 221 | - base_path, | ||
| 222 | - enable="receive-pack", | ||
| 223 | - listen=ip, | ||
| 224 | - port=port, | ||
| 225 | - base_path=base_path, | ||
| 226 | - as_process=True, | ||
| 227 | - ) | ||
| 205 | + # Killing a `git daemon` wrapper can leave its child listening on any OS. | ||
| 206 | + daemon_cmd = [ | ||
| 207 | + osp.join(Git()._call_process("--exec-path"), "git-daemon"), | ||
| 208 | + "--enable=receive-pack", | ||
| 209 | + "--reuseaddr", | ||
| 210 | + "--listen=%s" % ip, | ||
| 211 | + "--port=%s" % port, | ||
| 212 | + "--base-path=%s" % base_path, | ||
| 213 | + base_path, | ||
| 214 | + ] | ||
| 215 | + gd = Git().execute(daemon_cmd, as_process=True) | ||
| 228 | 216 | ||
| 229 | 217 | # Wait until git daemon listens for connections. | |
| 230 | 218 | for _attempt in range(1, 30): | |
@@ -241,7 +229,7 @@ def git_daemon_launched(base_path, ip, port): | |||
| 241 | 229 | Probably test will fail subsequently. | |
| 242 | 230 | ||
| 243 | 231 | BUT you may start *git-daemon* manually with this command:" | |
| 244 | - git daemon --enable=receive-pack --listen=%s --port=%s --base-path=%s %s | ||
| 232 | + git daemon --enable=receive-pack --reuseaddr --listen=%s --port=%s --base-path=%s %s | ||
| 245 | 233 | You may also run the daemon on a different port by passing --port=<port>" | |
| 246 | 234 | and setting the environment variable GIT_PYTHON_TEST_GIT_DAEMON_PORT to <port> | |
| 247 | 235 | """ | |
@@ -257,6 +245,7 @@ def git_daemon_launched(base_path, ip, port): | |||
| 257 | 245 | try: | |
| 258 | 246 | _logger.debug("Killing git-daemon...") | |
| 259 | 247 | gd.proc.kill() | |
| 248 | + gd.proc.wait(timeout=5) | ||
| 260 | 249 | except Exception as ex: | |
| 261 | 250 | # Either it has died (and we're here), or it won't die, again here... | |
| 262 | 251 | _logger.debug("Hidden error while Killing git-daemon: %s", ex, exc_info=1) | |
@@ -346,17 +335,14 @@ def remote_repo_creator(self): | |||
| 346 | 335 | raise | |
| 347 | 336 | ||
| 348 | 337 | finally: | |
| 349 | - rw_repo.git.clear_cache() | ||
| 350 | - rw_daemon_repo.git.clear_cache() | ||
| 338 | + rw_repo.close() | ||
| 339 | + rw_daemon_repo.close() | ||
| 351 | 340 | del rw_repo | |
| 352 | 341 | del rw_daemon_repo | |
| 353 | - gc.collect() | ||
| 354 | - gitdb.util.mman.collect() | ||
| 355 | - gc.collect() | ||
| 356 | 342 | if rw_repo_dir: | |
| 357 | - rmtree(rw_repo_dir) | ||
| 343 | + cleanup_directory(rw_repo_dir) | ||
| 358 | 344 | if rw_daemon_repo_dir: | |
| 359 | - rmtree(rw_daemon_repo_dir) | ||
| 345 | + cleanup_directory(rw_daemon_repo_dir) | ||
| 360 | 346 | # END cleanup | |
| 361 | 347 | ||
| 362 | 348 | # END bare repo creator | |
@@ -416,7 +402,7 @@ def setUpClass(cls): | |||
| 416 | 402 | ||
| 417 | 403 | @classmethod | |
| 418 | 404 | def tearDownClass(cls): | |
| 419 | - cls.rorepo.git.clear_cache() | ||
| 405 | + cls.rorepo.close() | ||
| 420 | 406 | cls.rorepo.git = None | |
| 421 | 407 | ||
| 422 | 408 | def _make_file(self, rela_path, data, repo=None): | |
@@ -496,7 +482,7 @@ def symlinks_supported() -> bool: | |||
| 496 | 482 | Developer Mode or SeCreateSymbolicLinkPrivilege, and an unprivileged process gets | |
| 497 | 483 | OSError (WinError 1314) instead. | |
| 498 | 484 | """ | |
| 499 | - with tempfile.TemporaryDirectory(prefix="gitpython-symlink-check-") as temp_dir: | ||
| 485 | + with TemporaryDirectory(prefix="gitpython-symlink-check-") as temp_dir: | ||
| 500 | 486 | link_path = osp.join(temp_dir, "link") | |
| 501 | 487 | try: | |
| 502 | 488 | os.symlink("missing-target", link_path) | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -10,8 +10,8 @@ | |||
| 10 | 10 | ||
| 11 | 11 | from git import Repo | |
| 12 | 12 | from git.db import GitCmdObjectDB, GitDB | |
| 13 | - from git.util import rmtree | ||
| 14 | 13 | ||
| 14 | + from test.cleanup import cleanup_directory | ||
| 15 | 15 | from test.lib import TestBase | |
| 16 | 16 | ||
| 17 | 17 | # { Invariants | |
@@ -51,9 +51,9 @@ def setUp(self): | |||
| 51 | 51 | self.puregitrorepo = Repo(repo_path, odbt=GitDB, search_parent_directories=True) | |
| 52 | 52 | ||
| 53 | 53 | def tearDown(self): | |
| 54 | - self.gitrorepo.git.clear_cache() | ||
| 54 | + self.gitrorepo.close() | ||
| 55 | 55 | self.gitrorepo = None | |
| 56 | - self.puregitrorepo.git.clear_cache() | ||
| 56 | + self.puregitrorepo.close() | ||
| 57 | 57 | self.puregitrorepo = None | |
| 58 | 58 | ||
| 59 | 59 | ||
@@ -72,12 +72,15 @@ def setUp(self): | |||
| 72 | 72 | ||
| 73 | 73 | def tearDown(self): | |
| 74 | 74 | super().tearDown() | |
| 75 | + dirname = None | ||
| 75 | 76 | if self.gitrwrepo is not None: | |
| 76 | - rmtree(self.gitrwrepo.working_dir) | ||
| 77 | - self.gitrwrepo.git.clear_cache() | ||
| 77 | + dirname = self.gitrwrepo.working_dir | ||
| 78 | + self.gitrwrepo.close() | ||
| 78 | 79 | self.gitrwrepo = None | |
| 79 | - self.puregitrwrepo.git.clear_cache() | ||
| 80 | + self.puregitrwrepo.close() | ||
| 80 | 81 | self.puregitrwrepo = None | |
| 82 | + if dirname is not None: | ||
| 83 | + cleanup_directory(dirname) | ||
| 81 | 84 | ||
| 82 | 85 | ||
| 83 | 86 | # } END base classes | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -21,6 +21,7 @@ | |||
| 21 | 21 | class TestPerformance(TestBigRepoRW, TestCommitSerialization): | |
| 22 | 22 | def tearDown(self): | |
| 23 | 23 | gc.collect() | |
| 24 | + super().tearDown() | ||
| 24 | 25 | ||
| 25 | 26 | # ref with about 100 commits in its history. | |
| 26 | 27 | ref_100 = "0.1.6" | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -9,12 +9,13 @@ | |||
| 9 | 9 | import socket | |
| 10 | 10 | import subprocess | |
| 11 | 11 | import sys | |
| 12 | - import tempfile | ||
| 12 | + | ||
| 13 | + from cleanup import TemporaryDirectory | ||
| 13 | 14 | ||
| 14 | 15 | ||
| 15 | 16 | def main(): | |
| 16 | 17 | root = Path(__file__).resolve().parent.parent | |
| 17 | - with tempfile.TemporaryDirectory(prefix="gitpython-local-tests-") as directory: | ||
| 18 | + with TemporaryDirectory(prefix="gitpython-local-tests-") as directory: | ||
| 18 | 19 | temporary = Path(directory) | |
| 19 | 20 | config = temporary / "gitconfig" | |
| 20 | 21 | config.write_text("[user]\nname = GitPython Tests\nemail = tests@example.invalid\n", encoding="utf-8") | |
@@ -51,7 +52,11 @@ def git(*args, cwd=root): | |||
| 51 | 52 | with socket.socket() as listener: | |
| 52 | 53 | listener.bind(("127.0.0.1", 0)) | |
| 53 | 54 | env["GIT_PYTHON_TEST_GIT_DAEMON_PORT"] = str(listener.getsockname()[1]) | |
| 54 | - return subprocess.call([sys.executable, "-m", "pytest", *sys.argv[1:]], cwd=root, env=env) | ||
| 55 | + return subprocess.call( | ||
| 56 | + [sys.executable, "-m", "pytest", "--basetemp", str(temporary / "pytest"), *sys.argv[1:]], | ||
| 57 | + cwd=root, | ||
| 58 | + env=env, | ||
| 59 | + ) | ||
| 55 | 60 | ||
| 56 | 61 | ||
| 57 | 62 | if __name__ == "__main__": | |
| Back | FazBrowse Home | New Git URL |
0 commit comments