Add controlled JDK cache benchmark scenario - #10
Merged
brunoborges merged 7 commits intoAug 5, 2026
Merged
Conversation
Copy link
Copy Markdown
Collaborator
Sorry, something went wrong.
Compare pinned baseline and candidate setup-java commits across isolated cold and warm Microsoft JDK runs, with raw timing and cache-size artifacts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Temporarily route the registered focused benchmark workflow through the reusable JDK cache benchmark for branch execution. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the temporary branch-dispatch wiring now that JDK cache samples are complete, leaving the reusable benchmark harness intact for retention. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new benchmark harness to measure the impact of a default-on JDK cache in actions/setup-java by running pinned “baseline” vs “candidate” commits in isolated cold/warm jobs, then generating a summarized report and artifacts.
Changes:
- Added a new jdk-cache.yml workflow that runs baseline/candidate cold+warm jobs and uploads benchmark artifacts.
- Added scripts/report-jdk-cache.mjs to fetch job timings + cache sizes from the GitHub API and write JSON/CSV/markdown summaries.
- Extended tests and documentation to cover the new job-name parsing and benchmark description.
| File | Description |
|---|---|
| scripts/report.test.mjs | Adds unit test coverage for parsing JDK cache benchmark job names. |
| scripts/report-jdk-cache.mjs | Implements report generation for the new JDK cache benchmark, including cache discovery and optional cleanup. |
| README.md | Documents the new JDK cache benchmark workflow and its sequential sampling/cleanup requirement. |
| .github/workflows/jdk-cache.yml | Defines the new benchmark workflow and report/artifact upload steps. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 2
- Review effort level: Lite
Sorry, something went wrong.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain the stricter seed/read-only implementation while joining the prior benchmark history.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why seed then measure
JDK cache keys are content-derived from the JDK identity and source, so they cannot be namespaced with a per-sample marker. A normal cold/warm matrix would let one sample's JDK entry contaminate another. The prepare job deletes existing setup-java-jdk- entries, each arm then runs one seed, and every measurement job uses cache-read-only: true. Measurement jobs can restore caches but cannot save, collide on, or race for the shared JDK key. Dependency and Maven Wrapper caches remain isolated with arm- and run-specific marker content.
A setup-java runner tool-cache hit returns before JDK cache restore/save registration. Every seed and measurement job therefore removes Java_* from $RUNNER_TOOL_CACHE first. Microsoft Build of OpenJDK is the default; Temurin remains available as an input to exercise the same forced-miss path.
This PR consolidates the earlier pinned baseline/candidate harness and PR #11. The earlier harness was successfully sampled five times, but those runs predate this same-ref, seed/read-only design; the consolidated workflow itself has intentionally not been dispatched from the PR.
How to run
Open Actions > JDK cache > Run workflow. For the primary comparison, use:
Validation