Measure overlay-base database size at the clear cleanup level - #3979
Merged
henrymercer merged 4 commits intoJun 30, 2026
Merged
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
This lets us compare the storage cost of overlay-base and trimmed databases for the same commit.
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Warning
- Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.
Pull request overview
This PR adds telemetry to measure what an overlay-base database would have compressed to if it had been cleaned with CleanupLevel.Clear (instead of CleanupLevel.Overlay), enabling apples-to-apples storage comparisons on the same commit. The measurement runs only when overlay-base uploads are enabled (feature-flagged) and on the default branch, and is skipped in debug mode.
Changes:
- Extend DatabaseUploadResult with optional fields to record the clear-cleanup zipped size and the time spent measuring it.
- After overlay-base uploads complete, perform a clear cleanup and re-bundle each language to record the “clear” zipped size (non-fatal, telemetry-only).
- Add unit tests covering the overlay-base measurement path and the skip conditions (regular uploads, debug mode).
| File | Description |
|---|---|
| src/database-upload.ts | Adds post-upload clear-cleanup size measurement and new telemetry fields for overlay-base runs. |
| src/database-upload.test.ts | Adds tests validating measurement behavior and skip conditions. |
| lib/entry-points.js | Generated output (excluded from diff by policy); expected to track corresponding TypeScript changes. |
Review details
Files excluded by content exclusion policy (1)- lib/entry-points.js
- Files reviewed: 2/3 changed files
- Comments generated: 2
- Review effort level: Low
Sorry, something went wrong.
henrymercer
marked this pull request as ready for review
June 26, 2026 17:43
mbg
previously approved these changes
Jun 30, 2026
mbg
left a comment
Copy link
Copy Markdown
Member
There was a problem hiding this comment.
The changes look reasonable. I had a couple of questions and one minor comment, but nothing blocking.
Sorry, something went wrong.
Comment thread
src/database-upload.ts
Comment thread
src/database-upload.ts
Comment thread
src/database-upload.ts
mbg
approved these changes
Jun 30, 2026
This was referenced Jul 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This lets us compare the storage cost of overlay-base and trimmed databases for the same commit.
This only happens when the feature flag for uploading overlay DBs to the GitHub API is enabled (currently staff only) and when running on the default branch. There is some performance penalty to taking this measurement, but since it's on the default branch, this seems justifiable for the time period that we are rolling out uploading overlay DBs to the GitHub API. We could also just compare the zipped DB size with and without the feature flag, but this change has the benefit of giving us the numbers on the same commit, which improves data quality.
Risk assessment
For internal use only. Please select the risk level of this change:
Which use cases does this change impact?
Workflow types:
Products:
Environments:
How did/will you validate this change?
If something goes wrong after this change is released, what are the mitigation and rollback strategies?
How will you know if something goes wrong after this change is released?
Are there any special considerations for merging or releasing this change?
Merge / deployment checklist