Add test & coverage reporting to PR comments and master baseline - #4292
Merged
andimarek merged 16 commits intoMar 5, 2026
Merged
Add test & coverage reporting to PR comments and master baseline#4292andimarek merged 16 commits into
andimarek merged 16 commits into
Conversation
Copy link
Copy Markdown
Member
Sorry, something went wrong.
Enables automatic test coverage analysis via the JaCoCo Gradle plugin. Generates HTML and XML reports, excluding generated ANTLR and shaded code. Usage: ./gradlew test jacocoTestReport https://claude.ai/code/session_01R1nvwA9Gc7gv8Uxwyvy83u
Add Madrapps/jacoco-report action (pinned to commit SHA for supply chain safety) to post a sticky coverage summary comment on PRs. The action runs only in the matrix job that generates the JaCoCo report and updates the same comment on each new commit. https://claude.ai/code/session_01R1nvwA9Gc7gv8Uxwyvy83u
Copy link
Copy Markdown
Contributor
Test Results 336 files +1 336 suites +1 5m 14s ⏱️ +10s Results for commit 90b8d6c. ± Comparison against base commit 5b63591. This pull request removes 196 and adds 181 tests. Note that renamed tests count towards both. ?
, expected: combo-\"\\\b\f\n\r\t, #4]
__schema { types { fields { args { type { name fields { name }}}}}}
__schema { types { fields { type { name fields { name }}}}}
__schema { types { inputFields { type { inputFields { name }}}}}
__schema { types { interfaces { fields { type { interfaces { name } } } } } }
__schema { types { name} }
__type(name : "t") { name }
a1: __schema { types { name} }
a1: __type(name : "t") { name }
…
graphql.AssertTest ‑ assertFalse with different number of error args but false does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_21prov0_closure23@78072f48 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0] graphql.AssertTest ‑ assertFalse with different number of error args but false does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_21prov0_closure24@578249f7 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2, #1] graphql.AssertTest ‑ assertFalse with different number of error args but false does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_21prov0_closure25@5ed1438a delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2 arg3, #2] graphql.AssertTest ‑ assertFalse with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_20prov0_closure20@2960a7fe delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0] graphql.AssertTest ‑ assertFalse with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_20prov0_closure21@487edcba delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2, #1] graphql.AssertTest ‑ assertFalse with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_20prov0_closure22@33c9c0f0 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2 arg3, #2] graphql.AssertTest ‑ assertNotNull with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_5prov0_closure3@349cb820 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0] graphql.AssertTest ‑ assertNotNull with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_5prov0_closure4@186bbc12 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2, #1] graphql.AssertTest ‑ assertNotNull with different number of error args throws assertions [toRun: <graphql.AssertTest$__spock_feature_0_5prov0_closure5@34545f5f delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1 arg2 arg3, #2] graphql.AssertTest ‑ assertNotNull with different number of error args with non null does not throw assertions [toRun: <graphql.AssertTest$__spock_feature_0_6prov0_closure6@40889d01 delegate=inaccessible owner=inaccessible thisObject=inaccessible resolveStrategy=inaccessible directive=inaccessible parameterTypes=inaccessible maximumNumberOfParameters=inaccessible bcw=inaccessible thisType=inaccessible>, expectedMessage: error arg1, #0] … ♻️ This comment has been updated with latest results. |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
|
Sorry, something went wrong.
Swap the third-party JaCoCo action for a custom github-script step that parses the JaCoCo XML and posts a per-package coverage breakdown (line, branch, method) as a PR comment. Also uploads the full HTML report as a build artifact for class-level drill-down. https://claude.ai/code/session_01R1nvwA9Gc7gv8Uxwyvy83u
- Remove EnricoMi/publish-unit-test-result-action - Restructure build matrix to use labeled includes (java11/17/21/25) - Each test job parses JUnit XML results and uploads stats as artifact - New test-summary job collects all stats, compares against committed baseline (test-stats-baseline.json), and posts a PR comment showing per-version totals with deltas (passed/failed/errors/skipped) - The summary job commits the updated baseline to the PR branch so it arrives on master through the normal merge flow - Uses [skip ci] on baseline commits to prevent infinite loops https://claude.ai/code/session_01R1nvwA9Gc7gv8Uxwyvy83u
Consolidate the two separate PR comments (test results + JaCoCo coverage) into one combined comment. Baseline is now only updated on master, never on PR branches. - Rename test-stats-baseline.json → test-baseline.json with tests + coverage sections - PR workflow: upload JaCoCo XML as artifact, post single unified comment - PR workflow: change contents permission to read (no more bot commits on PRs) - Master workflow: add labeled matrix, test parsing, JaCoCo generation - Master workflow: add update-baseline job that commits test-baseline.json Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor
Test ReportTest Results
Code Coverage (Java 25)
Changed Class Coverage (951 classes)
|
Sorry, something went wrong.
Parse per-class counters from JaCoCo XML and compare against master baseline. The PR comment now shows a collapsible table of classes where line, branch, or method coverage changed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Test results: green for more total/passed, red for more failed/errors. Coverage deltas: green for increase, red for decrease. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
e.g. graphql.execution.instrumentation.Foo$Bar → g.e.i.Foo$Bar Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Strip $InnerClass suffixes from class names and remove emoji from per-class delta cells. The +/- sign already conveys direction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Inner class names are kept but wrapped onto a second line at the $ boundary using <br> to keep the table narrow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The testng task was silently broken: useJUnitPlatform() in the global
tasks.withType(Test) block overrode useTestNG(), and the task lacked
testClassesDirs/classpath config. Guard useJUnitPlatform() to skip
testng tasks, fix the testng task config, add testngWithJava{11,17,21}
tasks, and wire them into both CI workflows with proper result parsing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a coverage gate in the test-summary job that checks line, branch, and method coverage against the master baseline. The PR comment is posted first so developers always see the report, then core.setFailed() is called if any metric decreased. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The TCK uses thread-locals to track recursion. When supplyAsync runs on the shared ForkJoinPool, tasks can complete on the same thread causing false spec303 failures. Use Executors.newSingleThreadExecutor() for all remaining TCK test classes, matching the earlier fix in dcb9b05. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Files changed
🤖 Generated with Claude Code