FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

ci: install deepeval and autoevals for evaluation integ tests by jariy17 · Pull Request #608 · aws/bedrock-agentcore-sdk-python · GitHub

ci: install deepeval and autoevals for evaluation integ tests - #608

Merged
jariy17 merged 1 commit into
mainfrom
fix/eval-integ-third-party-deps
Jul 31, 2026
Merged

ci: install deepeval and autoevals for evaluation integ tests#608
jariy17 merged 1 commit into
mainfrom
fix/eval-integ-third-party-deps

Conversation

jariy17 commented Jul 30, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Problem

Test (evaluation) in the Secure Integration test workflow is failing on main (run 30494098538):

ERROR tests_integ/evaluation/test_third_party_adapters.py::TestDeepEvalAdapterIntegration::test_bias_metric_passes - ModuleNotFoundError: No module named 'deepeval'
ERROR tests_integ/evaluation/test_third_party_adapters.py::TestAutoEvalsAdapterIntegration::test_factuality_scorer - ModuleNotFoundError: No module named 'autoevals'
======= 36 passed, 1 skipped, 41 warnings, 6 errors in 78.52s (0:01:18) ========

test_third_party_adapters.py (added in #568) imports deepeval and autoevals, but the evaluation matrix group only installed strands-agents-evals. The bare import calls in the autouse fixtures raise ModuleNotFoundError, which pytest reports as errors rather than skips, so the job exits 1.

Both packages are already declared as optional extras in pyproject.toml ([deepeval], [autoevals]) — they were simply never installed in that job.

Change

One line — add both packages to the evaluation group's extra-deps.

Testing

  • python3 -c "yaml.safe_load(...)" — workflow YAML parses, matrix entry correct
  • Installed the full evaluation dep set on Python 3.10 (CI's version); deepeval==4.1.4 and autoevals both resolve and import cleanly, no conflicts with strands-agents-evals==1.0.3
  • Confirmed the two ModuleNotFoundErrors are gone, and confirmed the residual OpenAI-credential failures documented above

tests_integ/evaluation/test_third_party_adapters.py imports deepeval and
autoevals, but the evaluation matrix group only installed
strands-agents-evals. The bare imports in the autouse fixtures raised
ModuleNotFoundError, which pytest reports as errors (not skips), failing
the job with exit code 1.

Both packages are already declared as optional extras in pyproject.toml.
jariy17 requested a review from a team July 30, 2026 22:33
github-actions Bot added the size/xs PR size: XS label Jul 30, 2026
agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jul 30, 2026

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

jariy17 merged commit 5d23292 into main Jul 31, 2026
29 of 34 checks passed
jariy17 deleted the fix/eval-integ-third-party-deps branch July 31, 2026 14:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs PR size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL