Add support for validation jobs to `sync.ts`, and refactor by mbg · Pull Request #3541 · github/codeql-action · GitHub

The primary goal of this PR is to add support for additional, validation jobs to sync.ts. My motivation for this is #3519 (review). To add a PR check which does that, the most sensible way of testing it would be to run a second job after the main one which verifies that the artifact was successfully uploaded.

This PR doesn't yet add or modify a PR check which makes use of this new functionality. I will add that as part of #3519 once this PR is merged.

I have refactored a few things in the first commits to break up the horrible long main function that we had initially ported from the earlier sync.py script as-is in #3526.

The refactoring has introduced some minor changes to the generated workflow files because the order of the various setup-* steps is now alphabetical, rather than effectively random. This should not have any impact on the behaviour of the workflows.

Notes for reviewers

Best reviewed commit-by-commit. Since these changes do not affect the CodeQL Action itself, a relatively lightweight review which focuses on the following is acceptable:

The high-level design of this new functionality. I.e. are we happy to support "validation jobs" in this way?
Are there any issues arising from the slight re-ordering of the setup steps / inputs in the workflows?
Any stylistic issues with the changes in this PR?

I have locally verified that the changes here broadly work for what I have in mind for the new workflow for #3526. Any small adjustments that may be required to support the full workflow could be added in that PR.

Risk assessment

For internal use only. Please select the risk level of this change:

Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Environments:

Testing/None - This change does not impact any CodeQL workflows in production.

How did/will you validate this change?

End-to-end tests - I am depending on PR checks (i.e. tests in pr-checks).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

Development/testing only - This change cannot cause any failures in production.

How will you know if something goes wrong after this change is released?

N/A

Are there any special considerations for merging or releasing this change?

No special considerations - This change can be merged at any time.

Merge / deployment checklist

Confirm this change is backwards compatible with existing workflows.
Consider adding a changelog entry for this change.
Confirm the readme and docs have been updated if necessary.

Pull request overview

This PR refactors pr-checks/sync.ts to make workflow generation more modular, and introduces first-class support for generating additional “validation jobs” that run after the primary PR-check job.

Changes:

Refactor workflow generation into helpers (matrix generation, shared setup-step generation, job generation).
Add validationJobs support in check specifications to generate dependent post-check jobs.
Reorder generated workflow setup steps/inputs (notably alphabetizing setup steps), causing widespread regenerated workflow diffs.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
pr-checks/sync.ts	Adds validation-job generation and refactors setup-step/matrix/job generation into reusable helpers.
.github/workflows/__with-checkout-path.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__upload-sarif.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__upload-ref-sha-input.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__unset-environment.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__swift-custom-build.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__split-workflow.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__remote-config.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-inputs-js.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-config-js.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-config-inputs-js.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-codescanning-config-inputs-js.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__multi-language-autodetect.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__local-bundle.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__go.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__go-custom-queries.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__export-file-baseline-information.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__config-input.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__build-mode-manual.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__analyze-ref-input.yml	Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__all-platform-bundle.yml	Regenerated output reflecting reordered setup steps/inputs.

…tion`

…ion-jobs

LGTM.
Meta: We are getting closer to designing a full workflow generation framework, but I think this is an effective sweet spot.
My comments are mostly on maintainability, the generalizations themselves are fine.

How about:

Suggested change

      
                  checkSpecification[setupSpec.specProperty] !== true
          
                  !checkSpecification[setupSpec.specProperty]

Personal preference, but I generally prefer the explicit check due to JS weirdness about truthiness.

- Add explicit `setup-python` step with condition to the workflow that was still using it - This allows simplifying the logic in `sync.ts`

The behaviour of `installPython` now mirrors other `install*` options

…he same specifications

For be7fe2b, I bumped the lib version in the tsconfig.json for pr-checks (since apparently the union method requires the latest ES version). That should be OK since the scripts in pr-checks are only used in development or CI environments.

This required me to make a subsequent change to the pr-checks.yml workflow as well, since the pr-checks script checks were run as part of the (matrixed) unit-tests job there, which runs with both Node 20 and Node 24. That makes sense for the unit tests etc. for the main codebase, but less so for the pr-checks scripts where it now caused a failure. I am guessing that this was just an oversight when we added the node-versions matrix to run the tests with the node versions we use for both v3 and v4.

LGTM. Thanks for the follow-ups.

mbg added 6 commits March 3, 2026 14:15

Extract JobSpecification type from Specification

5db3a9e

Organise language-specific setup information

97a3705

Refactor matrix generation into its own function

369d73b

Refactor job generation into generateJob

92ab799

Move yq setup code into getSetupSteps

95fc2f1

Add support for additional, validation jobs

2b60771

mbg self-assigned this Mar 4, 2026

mbg requested a review from a team as a code owner March 4, 2026 11:43

Copilot AI review requested due to automatic review settings March 4, 2026 11:43

github-actions Bot added the size/L May be hard to review label Mar 4, 2026

Copilot started reviewing on behalf of mbg March 4, 2026 11:44 View session

Copilot AI reviewed Mar 4, 2026

View reviewed changes

Comment thread pr-checks/sync.ts Outdated

Comment thread pr-checks/sync.ts Outdated

Comment thread pr-checks/sync.ts Outdated

Comment thread pr-checks/sync.ts Outdated

mbg added 7 commits March 5, 2026 15:39

Merge branch 'main' into mbg/pr-checks/validation-jobs

56ebdff

Add tsconfig.json for pr-checks

3d47812

Fix generateValidationJobs typing

79fdef7

Make it more explicit that getSetupSteps just needs a `JobSpecifica…

103db93

…tion`

Fix condition

2a00604

Run prepare-test after setup steps

da11f44

Install python if there is no matrix.version

5ddbbbe

github-actions Bot added size/XL May be very hard to review and removed size/L May be hard to review labels Mar 9, 2026

Merge remote-tracking branch 'origin/main' into mbg/pr-checks/validat…

8ba8180

…ion-jobs

esbena reviewed Mar 11, 2026

View reviewed changes

mbg added 6 commits March 11, 2026 11:55

Remove last use of installPython

b9b42be

- Add explicit `setup-python` step with condition to the workflow that was still using it - This allows simplifying the logic in `sync.ts`

Remove installPython condition in sync.ts

2e1f08f

The behaviour of `installPython` now mirrors other `install*` options

Make it more explicit by construction that known inputs always have t…

be7fe2b

…he same specifications

Extend base tsconfig.json

6570ad3

Use version in error message

89f6321

Verify PR checks in a different job, with newer Node

d1a7580

mbg requested a review from esbena March 11, 2026 12:46

esbena approved these changes Mar 11, 2026

View reviewed changes

mbg added this pull request to the merge queue Mar 11, 2026

Merged via the queue into main with commit 1a97b0f Mar 11, 2026
246 checks passed

mbg deleted the mbg/pr-checks/validation-jobs branch March 11, 2026 15:06

mbg mentioned this pull request Mar 12, 2026

Add PR check for CSRA artifact upload #3571

Draft

This was referenced Mar 16, 2026

Merge main into releases/v4 #3574

Merged

Merge releases/v4 into releases/v3 #3577

Merged

Add support for validation jobs to `sync.ts`, and refactor by mbg · Pull Request #3541 · github/codeql-action · GitHub

Uh oh!

Add support for validation jobs to sync.ts, and refactor - #3541

Add support for validation jobs to sync.ts, and refactor#3541
mbg merged 20 commits into
mainfrom
mbg/pr-checks/validation-jobs

mbg commented Mar 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

esbena left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

esbena Mar 11, 2026

Uh oh!

mbg Mar 11, 2026

Uh oh!

Uh oh!

Uh oh!

mbg commented Mar 11, 2026

Uh oh!

esbena left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	checkSpecification[setupSpec.specProperty] !== true
	!checkSpecification[setupSpec.specProperty]

Add support for validation jobs to `sync.ts`, and refactor by mbg · Pull Request #3541 · github/codeql-action · GitHub

Uh oh!

Add support for validation jobs to sync.ts, and refactor - #3541

Add support for validation jobs to sync.ts, and refactor#3541mbg merged 20 commits intomaingithub/codeql-action:mainfrom mbg/pr-checks/validation-jobsgithub/codeql-action:mbg/pr-checks/validation-jobsCopy head branch name to clipboard

Conversation

mbg commented Mar 4, 2026

Risk assessment

Which use cases does this change impact?

How did/will you validate this change?

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

How will you know if something goes wrong after this change is released?

Are there any special considerations for merging or releasing this change?

Merge / deployment checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

esbena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

esbena Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

mbg Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mbg commented Mar 11, 2026

Uh oh!

esbena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add support for validation jobs to sync.ts, and refactor#3541
mbg merged 20 commits into
mainfrom
mbg/pr-checks/validation-jobs