FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

evaluation refactor by n-poulsen · Pull Request #2679 · DeepLabCut/DeepLabCut · GitHub

/ DeepLabCut Public

evaluation refactor - #2679

Merged
n-poulsen merged 15 commits into
pytorch_dlcfrom
niels/eval_oom
Jul 19, 2024
ConversationCommits (15)ChecksFiles changed
Merged

evaluation refactor#2679
n-poulsen merged 15 commits into
pytorch_dlcfrom
niels/eval_oom

Conversation

n-poulsen commented Jul 19, 2024

Copy link
Copy Markdown
Contributor

Evaluation refactor

Improvements to the evaluation code, as well as new tests to ensure that mAP scores match the pycocotools implementation.

Change list:

  • Moved all metric computation code to a deeplabcut/core/metrics folder (as metrics are computed with numpy)
  • Cleaned metric computation code so the prediction/ground truth matching always happens
    • Refactored in a way such that no OOM errors should occur, even on very large datasets (>60k images)
  • Multi-animal RMSE: only compute RMSE using (ground-truth, detection) matches with non-zero RMSE
  • Add compute_detection_rmse to compute "detection" RMSE, matching the DeepLabCut 2.X implementation
  • Fixed the bug for PAF models documented in Evaluation error with PAF heads: ValueError: matrix contains invalid numeric entries #2631

n-poulsen requested review from MMathisLab and jeylau July 19, 2024 12:12
MMathisLab approved these changes Jul 19, 2024
View reviewed changes

MMathisLab left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

lgtm, although I did not test

MMathisLab approved these changes Jul 19, 2024
View reviewed changes

MMathisLab left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

but see suggested changes to docstrings

Comment thread deeplabcut/core/metrics/api.py Outdated
Comment thread deeplabcut/core/metrics/api.py
Comment thread deeplabcut/core/metrics/api.py Outdated
MMathisLab approved these changes Jul 19, 2024
View reviewed changes

MMathisLab left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

maybe in main docs we need to add information about these new metrics as well @n-poulsen

n-poulsen merged commit dc908b9 into pytorch_dlc Jul 19, 2024
n-poulsen deleted the niels/eval_oom branch July 19, 2024 15:11
n-poulsen mentioned this pull request Jul 19, 2024
Closed
2 tasks
n-poulsen mentioned this pull request Aug 13, 2024
Closed
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

MMathisLab MMathisLab approved these changes
jeylau Awaiting requested review from jeylau

Assignees

No one assigned

Labels

DLC3.0🔥

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

Back | FazBrowse Home | New Git URL