Migrate to `pathlib.Path` instead of `str` (first phase) by deruyter92 · Pull Request #3350 · DeepLabCut/DeepLabCut · GitHub

Motivation

We currently use pathlib.Path and str mixed inconsistently throughout the codebase. Settling on pathlib.Path could make the codebase more uniform, and fix issues with brittle path resolvement, (e.g. see #3348).

This PR builds on top of #3349, which introduces safer path resolution.

Policy

Internal: use pathlib.Path for path math and state.
Public API: accept str | Path at entry; normalize with Path(...).absolute() internally.
Boundaries: convert with str() / os.fspath() only where required:
- Qt widgets (QLineEdit, QFileDialog, list item text)
- video_sets dict keys in config.yaml
- OpenCV / PIL / VideoReader / subprocess / pandas paths
- (legacy helpers that still expect str -> should be replaced on long-term as well)

Scope

This PR starts with addressing many places to clearly enforce Path-centric API, but leaves quite some legacy helpers untouched for now, to avoid a single big switch. See details (autogenerated) below:

Done in this PR

create_project (2D core)

new.py, add.py, demo_data.py: Path internally; os.fspath at video_sets / VideoReader / copy-mklink boundaries
create_new_project: returns Path | Literal["nothingcreated"] (was str)
Removed legacy realpath / readlink fallback for video_sets keys; use Path(video).absolute()

GUI

window.py: config_path: Path | None, files: set[Path]; derived folders (project_folder, models_folder, …) return Path; os.fspath for Qt line edits / recent files
widgets.py: ConfigEditor stores Path; DragDropListView keeps str in Qt items (by design), normalizes on add
components.py: video/snapshot/conditions selection uses Path; os.fspath at QFileDialog
Tabs updated: open_project, manage_project, create_project, analyze_videos, modelzoo
Other tabs mostly pass root.config_path (Path) through to DLC APIs — no per-tab refactor

pose_estimation_pytorch (partial)

Loaders / metadata: Loader.project_root, DLCLoader.project_path, PoseMetadata.project_path are Path
APIs: use cfg.project_path via ProjectConfig instead of Path(cfg["project_path"]) in apis/utils, analyze_images, tracklets, evaluation, visualization; dlcloader.load_ground_truth / load_split
Model zoo: modelzoo/config.py reads super-animal project config via ProjectConfig.from_yaml
Boundaries kept as str: COCO file_name, cv2.imread, Image.open, VideoReader, etc.
Public functions still typed config: str | Path at entry

Partially done / not in this PR

Area	Status
create_project/modelzoo.py	Mixed: still str() for train paths and some config writes
create_project/new_3d.py	Path for dirs; project_path written as str to YAML
pose_estimation_pytorch CTD conditions typing	Separate PR (heterogeneous YAML shape)
pose_estimation_pytorch/metrics/scoring.py	Still str-only hints
deeplabcut.utils.*	Largely unchanged; many functions still str-centric internally
deeplabcut.compat	Accepts str \| Path at API, coerces internally — not fully Path-native
generate_training_dataset, refine_training_dataset	Some Path usage, no systematic migration
pose_estimation_tensorflow	Out of scope — still os.path / str patterns
Top-level analyze_videos, create_labeled_video, etc.	Not refactored in this slice

Intentional exceptions

Qt list / line-edit text stays str
video_sets keys in YAML stay str
create_new_project return type changes from str → Path (callers using read_config / path math are fine; strict isinstance(x, str) would break)

…h Path(..).absolute()

Strongly approve that this is a worthwhile effort, let me know if I can help!

Pull request overview

This PR advances the codebase-wide shift toward consistent pathlib.Path usage (reducing brittle string-based path handling) and enables Ruff’s PTH rules to help enforce that direction.

Changes:

Enable Ruff PTH lint rules in pyproject.toml.
Replace many os.path/open() call sites with Path operations (joins, exists checks, mkdir, open, glob) across utilities, TensorFlow/PyTorch pipelines, model zoo, GUI, and benchmark code.
Update several public API type hints/signatures to accept Path (and convert inputs to Path early in functions).

Reviewed changes

Copilot reviewed 107 out of 107 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
pyproject.toml	Enable Ruff PTH rules to encourage pathlib usage
deeplabcut/utils/visualization.py	Convert joins/saves to Path-based operations
deeplabcut/utils/skeleton.py	Replace os filesystem ops with Path
deeplabcut/utils/pseudo_label.py	Switch JSON/path/glob handling to Path
deeplabcut/utils/plotting.py	Use Path for output paths; accept Path inputs in API
deeplabcut/utils/make_labeled_video.py	Convert path handling to Path; accept Path inputs
deeplabcut/utils/conversioncode.py	Use Path for labeled-data file operations; update type hints
deeplabcut/utils/auxiliaryfunctions_3d.py	Replace glob/path ops with Path where applicable
deeplabcut/utils/auxfun_videos.py	Replace os.path usage with Path in video utilities
deeplabcut/utils/auxfun_multianimal.py	Use Path.open and Path joins for multi-animal utils
deeplabcut/utils/auxfun_models.py	Replace directory scanning with Path.iterdir and Path joins
deeplabcut/refine_training_dataset/tracklets.py	Use Path.open for pickle loading
deeplabcut/refine_training_dataset/stitch.py	Use Path.open and Path-based paths; update type hints
deeplabcut/refine_training_dataset/outlier_frames.py	Use Path for file IO/paths; update type hints
deeplabcut/post_processing/filtering.py	Update function type hints to accept Path
deeplabcut/post_processing/analyze_skeleton.py	Use Path for filesystem checks; update type hints
deeplabcut/pose_tracking_pytorch/train_dlctransreid.py	Replace glob/path logic with Path.glob
deeplabcut/pose_tracking_pytorch/processor/processor.py	Use Path for checkpoint/results output
deeplabcut/pose_tracking_pytorch/create_dataset.py	Use Path for pickles/feature files and checks
deeplabcut/pose_tracking_pytorch/config/init.py	Use Path for locating reid_cfg.yaml
deeplabcut/pose_tracking_pytorch/apis.py	Switch checkpoint handling to Path
deeplabcut/pose_estimation_tensorflow/visualizemaps.py	Replace many joins/cwd/exists with Path
deeplabcut/pose_estimation_tensorflow/util/logging.py	Remove redundant os.path.join for log filename
deeplabcut/pose_estimation_tensorflow/training.py	Use Path for model/config paths and CWD handling
deeplabcut/pose_estimation_tensorflow/predict_videos.py	Replace many joins/checks with Path
deeplabcut/pose_estimation_tensorflow/predict_multianimal.py	Use Path.open for pickles
deeplabcut/pose_estimation_tensorflow/modelzoo/api/superanimal_inference.py	Replace glob/os.path with Path for configs/weights/outputs
deeplabcut/pose_estimation_tensorflow/modelzoo/api/spatiotemporal_adapt.py	Replace glob/os.path with Path; mkdir via Path.mkdir
deeplabcut/pose_estimation_tensorflow/export.py	Use Path for export directory/files and checkpoint copying
deeplabcut/pose_estimation_tensorflow/datasets/pose_tensorpack.py	Use Path joins for dataset/image paths
deeplabcut/pose_estimation_tensorflow/datasets/pose_multianimal_imgaug.py	Use Path for config/dataset paths and IO
deeplabcut/pose_estimation_tensorflow/datasets/pose_imgaug.py	Use Path for dataset paths and pickle reading
deeplabcut/pose_estimation_tensorflow/datasets/pose_deterministic.py	Use Path for dataset/image paths and IO
deeplabcut/pose_estimation_tensorflow/core/train.py	Use Path.cwd() and Path.open for stats CSV
deeplabcut/pose_estimation_tensorflow/core/train_multianimal.py	Use Path.cwd() and Path.open; simplify snapshot prefix
deeplabcut/pose_estimation_tensorflow/core/test.py	Replace os directory/file ops with Path
deeplabcut/pose_estimation_tensorflow/core/openvino/session.py	Replace os.path usage with Path joins/parents
deeplabcut/pose_estimation_tensorflow/core/evaluate.py	Use Path for model/data/evaluation paths
deeplabcut/pose_estimation_tensorflow/core/evaluate_multianimal.py	Use Path for evaluation/data paths
deeplabcut/pose_estimation_tensorflow/config.py	Use Path.open when reading YAML config
deeplabcut/pose_estimation_tensorflow/backbones/mobilenet.py	Replace string path splitting with Path operations
deeplabcut/pose_estimation_tensorflow/backbones/efficientnet_builder.py	Use Path for model param file path
deeplabcut/pose_estimation_pytorch/utils.py	Use Path for folder creation
deeplabcut/pose_estimation_pytorch/runners/logger.py	Use Path.open for YAML/CSV logging IO
deeplabcut/pose_estimation_pytorch/modelzoo/utils.py	Treat DLC root as Path consistently
deeplabcut/pose_estimation_pytorch/modelzoo/memory_replay.py	Use Path.open and Path joins for JSON IO
deeplabcut/pose_estimation_pytorch/modelzoo/inference.py	Use Path.open for JSON writing
deeplabcut/pose_estimation_pytorch/modelzoo/fmpose_3d/inference.py	Use Path.open for JSON writing
deeplabcut/pose_estimation_pytorch/modelzoo/config.py	Use Path.mkdir/Path.open for config output
deeplabcut/pose_estimation_pytorch/metrics/scoring.py	Use Path.open for pickle IO
deeplabcut/pose_estimation_pytorch/data/dlcloader.py	Use Path.open for dataset pickle IO
deeplabcut/pose_estimation_pytorch/data/ctd.py	Use Path.open for conditions JSON/pickle
deeplabcut/pose_estimation_pytorch/data/cocoloader.py	Use Path for COCO JSON path resolution and IO
deeplabcut/pose_estimation_pytorch/config/utils.py	Treat DLC root as Path consistently
deeplabcut/pose_estimation_pytorch/apis/videos.py	Accept/coerce Path inputs; use Path.open for pickles
deeplabcut/pose_estimation_pytorch/apis/utils.py	Update type hints to accept Path
deeplabcut/pose_estimation_pytorch/apis/tracklets.py	Accept Path inputs; use Path.cwd() and Path.open
deeplabcut/pose_estimation_pytorch/apis/tracking_dataset.py	Accept Path inputs; update destfolder type hints
deeplabcut/pose_estimation_pytorch/apis/analyze_images.py	Use Path.open and Path for output folders/paths
deeplabcut/pose_estimation_3d/triangulation.py	Use Path for existence checks and output paths; update type hints
deeplabcut/pose_estimation_3d/plotting3D.py	Replace glob/os.path with Path and Path.glob
deeplabcut/pose_estimation_3d/camera_calibration.py	Replace glob/os.path/symlink ops with Path
deeplabcut/modelzoo/video_inference.py	Accept Path dest folder; use Path.open for JSON IO
deeplabcut/modelzoo/utils.py	Replace glob/os.path with Path and Path.glob
deeplabcut/modelzoo/generalized_data_converter/utils.py	Replace glob/os.path with Path and Path joins
deeplabcut/modelzoo/generalized_data_converter/datasets/single_dlc.py	Replace joins with Path for image paths
deeplabcut/modelzoo/generalized_data_converter/datasets/single_dlc_dataframe.py	Replace joins with Path for labeled-data access
deeplabcut/modelzoo/generalized_data_converter/datasets/materialize.py	Replace joins/symlink/mkdir with Path
deeplabcut/modelzoo/generalized_data_converter/datasets/ma_dlc.py	Replace joins with Path for image paths
deeplabcut/modelzoo/generalized_data_converter/datasets/ma_dlc_dataframe.py	Replace joins with Path for labeled-data access
deeplabcut/modelzoo/generalized_data_converter/datasets/coco.py	Use Path for annotation/image path handling
deeplabcut/modelzoo/generalized_data_converter/datasets/base.py	Replace os.sep splitting with Path-based name extraction
deeplabcut/modelzoo/generalized_data_converter/datasets/base_dlc.py	Use Path for config/dataset doc IO
deeplabcut/gui/window.py	Replace some os.path usage with Path in GUI paths
deeplabcut/gui/widgets.py	Replace file/dir checks and path joins with Path
deeplabcut/gui/tracklet_toolbox.py	Use Path for labeled-data export paths and IO
deeplabcut/gui/tabs/train_network.py	Replace logo path derivation with Path
deeplabcut/gui/tabs/refine_tracklets.py	Use Path for destination/datafile paths
deeplabcut/gui/tabs/open_project.py	Use Path.cwd() and Path for logo paths
deeplabcut/gui/tabs/modelzoo.py	Replace icon path joins with Path
deeplabcut/gui/tabs/manage_project.py	Use Path.cwd() for file dialogs
deeplabcut/gui/tabs/label_frames.py	Accept Path image_folder; use Path for directory iteration
deeplabcut/gui/tabs/evaluate_network.py	Use Path for evaluation maps directory and listing
deeplabcut/gui/tabs/create_training_dataset.py	Use Path for training artifact checks
deeplabcut/gui/tabs/create_project.py	Use Path for video suffix checks and default paths
deeplabcut/gui/launch_script.py	Use Path for asset paths and stylesheet loading
deeplabcut/gui/displays/shuffle_metadata_viewer.py	Use Path.open for reading metadata
deeplabcut/gui/displays/selected_shuffle_display.py	Pass Path to read_plainconfig consistently
deeplabcut/gui/components.py	Replace abspath/basename with Path.resolve()/.name
deeplabcut/gui/init.py	Represent BASE_DIR as a Path
deeplabcut/generate_training_dataset/multiple_individuals_trainingsetmanipulation.py	Use Path joins and Path.unlink for dataset recreation
deeplabcut/generate_training_dataset/metadata.py	Use Path.open and Path.exists consistently
deeplabcut/generate_training_dataset/frame_extraction.py	Accept Path config; use Path glob/iterdir/unlink
deeplabcut/create_project/new.py	Update API to accept Path; use Path for config/symlinks
deeplabcut/create_project/new_3d.py	Use Path for config output path
deeplabcut/create_project/modelzoo.py	Use Path.open for template read/write; use Path.cwd()
deeplabcut/create_project/demo_data.py	Accept Path config; use Path for demo video paths
deeplabcut/create_project/add.py	Accept Path; use Path for symlink/readlink
deeplabcut/core/inferenceutils.py	Use Path.open for assembly pickle IO
deeplabcut/core/crossvalutils.py	Replace path splitting/joining and pickle IO with Path
deeplabcut/core/config.py	Use Path.open for YAML config read/write
deeplabcut/compat.py	Coerce config/videos/destfolder to Path; update API type hints
deeplabcut/benchmark/utils.py	Use Path.open for /dev/null redirection
deeplabcut/benchmark/metrics.py	Use Path for file IO/path assembly in benchmarking
deeplabcut/benchmark/init.py	Use Path for data/cache locations and IO

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Fix path handling when loading the dataset pickle. The code previously attempted Path(project_path) / file_name even though file_name already included the project path, resulting in an incorrect path. This changes the context manager to call file_name.open('rb') directly, removing the redundant join and preventing path errors when loading the pickle.

Replace direct ruamel_file.dump(...open('w')) call with a with-statement to open pose_cfg.yaml. This ensures the file handle is properly closed after writing and avoids potential resource leaks.

Convert video_path to str when constructing cv2.VideoCapture to ensure pathlib.Path objects are handled correctly and avoid type errors when opening videos. This makes VideoReader more robust when given Path instances.

Replace the list-comprehension side-effect used to delete temporary frame files with an explicit for-loop that checks p.is_file() before calling unlink. This avoids building an unused list, prevents errors from trying to unlink non-file entries (e.g., directories), and improves readability and safety.

Here is a first pass, I addressed a few things I thought were not too discussion-worthy directly, please see the latest commits!

PoseConfig.model.backbone is currently a dict-type but freeze_bn_stats were set as attribute. This commit fixes that and treats backbone correctly as dict.

Important

Merge #3349 before this PR.

I've changed the base to main to trigger all functional tests, but this PR builds on top of #3349 which should be merged first.

@C-Achard, I tried to scope it a bit to a stable first phase (see PR description). The migration can be a bit tricky still if downstream code receives Path now instead of str. But those cases will be easy to fix in case they pop up.

Let me know what you think.

We can also try to go all the way all at once, and do everything (even including the TensorFlow branch)

Apologies, I thought I had approved from VSCode, not sure where that went. I think the current scope is good, thanks for the effort! We can keep converting later, better to chunk it down imo.

deruyter92 added 3 commits May 29, 2026 09:51

fix path resolution on network drives. Replace Path(..).resolve() wit…

3b8c44a

…h Path(..).absolute()

Add safe_resolve helper for path resolution

fddcbaa

replace str with pathlib.Path

2403b72

deruyter92 changed the base branch from main to jaap/safe_resolve_paths May 29, 2026 08:51

deruyter92 mentioned this pull request May 29, 2026

read_config() project path resolution can break path #3348

Closed

2 tasks

deruyter92 added 4 commits May 29, 2026 11:16

fix redundant resolution of resolved path in memory_replay.py

65f8dfb

fix open -> os.stat

6bbe26d

replace os.stat -> open for files, listdir for dirs.

3f93a76

fix TestReadConfigProjectPath

87cd6aa

deruyter92 and others added 3 commits May 29, 2026 15:10

Update deeplabcut/utils/auxiliaryfunctions.py

8be1c26

additional replacements: str ->Path

f2bb35d

Update function signatures str -> str | Path

810406a

deruyter92 force-pushed the jaap/pathlib_path_policy branch from b9de7c2 to 810406a Compare June 4, 2026 14:55

Add ruff linting rule PTH

c7835bc

C-Achard reviewed Jun 5, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

C-Achard assigned deruyter92 Jun 15, 2026

C-Achard added the enhancement New feature or request label Jun 15, 2026

C-Achard mentioned this pull request Jun 15, 2026

Update GUI assets import #3370

Merged

C-Achard requested a review from Copilot June 15, 2026 16:05

Copilot started reviewing on behalf of C-Achard June 15, 2026 16:06 View session

Copilot AI reviewed Jun 15, 2026

View reviewed changes

C-Achard added 4 commits June 15, 2026 13:04

Use context manager when writing pose_cfg.yaml

156c25f

Replace direct ruamel_file.dump(...open('w')) call with a with-statement to open pose_cfg.yaml. This ensures the file handle is properly closed after writing and avoids potential resource leaks.

Pass string path to cv2.VideoCapture

d854aa5

Convert video_path to str when constructing cv2.VideoCapture to ensure pathlib.Path objects are handled correctly and avoid type errors when opening videos. This makes VideoReader more robust when given Path instances.

C-Achard reviewed Jun 15, 2026

View reviewed changes

Fix CircleCI failure

e896398

C-Achard mentioned this pull request Jun 15, 2026

Fix PySide6 stylesheet usage #3371

Merged

fix Matplotlib Path import shadowing ->MPLPath

132b882

deruyter92 added 3 commits July 7, 2026 15:51

GUI use os.fspath for str conversions where required

af613f3

use pathlib Path in create_project

1586e2a

use pathlib Path across pose_estimation_pytorch internally

c49d392

deruyter92 removed request for AlexEMG and MMathisLab July 7, 2026 14:43

deruyter92 changed the title Enforce pathlib.Path instead of str Migrate to pathlib.Path instead of str (first phase) Jul 7, 2026

deruyter92 added 7 commits July 9, 2026 13:39

use centralized yaml dumper (which normalizes Path -> str)

6a85abb

fix materialize.py downstream old str.split -> Path

33c7b58

fix testsscripts str usage

efebf8d

fix old str usage in trainingsetmanipulation

971f381

allow str type at TF boundary

292b251

allow str formatting in tensorflow code

1330eab

fix config update in train_from_coco.py

84b5c83

PoseConfig.model.backbone is currently a dict-type but freeze_bn_stats were set as attribute. This commit fixes that and treats backbone correctly as dict.

deruyter92 changed the base branch from jaap/safe_resolve_paths to main July 9, 2026 12:42

deruyter92 requested a review from C-Achard July 9, 2026 15:03

C-Achard approved these changes Jul 14, 2026

View reviewed changes

deruyter92 requested review from AlexEMG and MMathisLab July 15, 2026 12:15

deruyter92 assigned AlexEMG and MMathisLab Jul 15, 2026

MMathisLab approved these changes Jul 15, 2026

View reviewed changes

deruyter92 mentioned this pull request Jul 15, 2026

GUI: Config validation errors handler dialog & layout fixes #3397

Merged

3 tasks

deruyter92 merged commit 30a8155 into main Jul 15, 2026
30 checks passed

deruyter92 deleted the jaap/pathlib_path_policy branch July 15, 2026 15:53

C-Achard mentioned this pull request Jul 16, 2026

GUI: Fix redundant signal calls, streamline path selections #3395

Merged

This was referenced Jul 17, 2026

Fix network-drive path corruption in read_config() and related call sites #3349

Closed

Harden path-handling after pathlib migration #3421

Merged

version bump 3.0.1 #3423

Merged

Migrate to `pathlib.Path` instead of `str` (first phase) by deruyter92 · Pull Request #3350 · DeepLabCut/DeepLabCut · GitHub

Uh oh!

Uh oh!

Migrate to pathlib.Path instead of str (first phase) - #3350

Migrate to pathlib.Path instead of str (first phase)#3350deruyter92 merged 51 commits intomainDeepLabCut/DeepLabCut:mainfrom jaap/pathlib_path_policyDeepLabCut/DeepLabCut:jaap/pathlib_path_policyCopy head branch name to clipboard

Conversation

deruyter92 commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Policy

Scope

Done in this PR

create_project (2D core)

GUI

pose_estimation_pytorch (partial)

Partially done / not in this PR

Intentional exceptions

Uh oh!

C-Achard commented May 29, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

C-Achard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

deruyter92 commented Jul 9, 2026

Uh oh!

deruyter92 commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

C-Achard left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Migrate to pathlib.Path instead of str (first phase)#3350
deruyter92 merged 51 commits into
mainfrom
jaap/pathlib_path_policy

deruyter92 commented May 29, 2026 •
edited
Loading

deruyter92 commented Jul 9, 2026 •
edited
Loading

C-Achard left a comment •
edited
Loading