add first_party_analysis boolean to all status reports - #2111
Merged
nickfyson merged 4 commits intoFeb 21, 2024
Merged
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
nickfyson
force-pushed
the
nickfyson/first-or-third-party
branch
from
febf195 to
a7dc229
Compare
February 20, 2024 14:58
nickfyson
marked this pull request as ready for review
February 20, 2024 16:05
henrymercer
previously approved these changes
Feb 21, 2024
henrymercer
left a comment
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Nice!
Sorry, something went wrong.
Comment thread
src/environment.ts
Outdated
| HAS_WARNED_ABOUT_DISK_SPACE = "CODEQL_ACTION_HAS_WARNED_ABOUT_DISK_SPACE", | ||
|
|
||
| /** Whether the init action has been run. */ | ||
| INIT_ACTION_HAS_RUN = "CODEQL_INIT_ACTION_HAS_RUN", |
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Minor: Doesn't read as well but it's nice to have everything the Action defines prefixed by CODEQL_ACTION_
Suggested change
| INIT_ACTION_HAS_RUN = "CODEQL_INIT_ACTION_HAS_RUN", | |
| INIT_ACTION_HAS_RUN = "CODEQL_ACTION_INIT_HAS_RUN", |
Sorry, something went wrong.
Comment thread
src/status-report.test.ts
Outdated
Comment on lines
+66
to
+77
| process.env["CODEQL_ACTION_ANALYSIS_KEY"] = "analysis-key"; | ||
| process.env["GITHUB_REF"] = "refs/heads/main"; | ||
| process.env["GITHUB_REPOSITORY"] = "octocat/HelloWorld"; | ||
| process.env["GITHUB_RUN_ATTEMPT"] = "2"; | ||
| process.env["GITHUB_RUN_ID"] = "100"; | ||
| process.env["GITHUB_SHA"] = "a".repeat(40); | ||
| process.env["ImageVersion"] = "2023.05.19.1"; | ||
| process.env["RUNNER_OS"] = "macOS"; | ||
| process.env["RUNNER_TEMP"] = tmpDir; | ||
|
|
||
| const getRequiredInput = sinon.stub(actionsUtil, "getRequiredInput"); | ||
| getRequiredInput.withArgs("matrix").resolves("input/matrix"); |
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Minor: consider creating a function for all of this as it's shared across a few tests
Sorry, something went wrong.
angelapwen
approved these changes
Feb 21, 2024
angelapwen
left a comment
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Looks great to me ✨ thanks for the care in thinking about misconfigured workflows!
Sorry, something went wrong.
This was referenced Feb 21, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In order to define more granular metrics and define more appropriate SLOs we add a new field to the status reports uploaded by the CodeQL Action.
This field first_party_analysis is based on whether the init action has been used, which is only used for first party analysis. When a SARIF file has been generated by other means and submitted using the upload action, this is considered to be a third party analysis and will be treated differently when calculating SLOs. To ensure misconfigured workflows are not treated as third party, only the upload-sarif action can submit status reports that are not first-party.
See back-linked internal issue for example status reports, showing these changes in action.
Merge / deployment checklist