Add `resolve-environment` action by mbg · Pull Request #1684 · github/codeql-action · GitHub

Summary

This PR adds a new resolve-environment action which calls the new CLI resolve build-environment command to try and infer a configuration for the build environment that is suitable for the autobuilder. Example workflow (assuming a repository with a Go project):

jobs:
  analyze-linux:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - uses: github/codeql-action/resolve-environment@mbg/add-resolve-environment
        id: resolve-environment
        with:
          language: go

      - run: echo "${{ steps.resolve-environment.outputs.environment }}"

The action performs the following work:

Downloads the CodeQL CLI (optionally from the tools input); same behaviour as the init action.
Runs the resolve build-environment command and stores the JSON result received from that in the environment output

This is looking in good shape!

I think it's worth writing a CHANGELOG note for observant customers who are watching the repo to say that this Action is an internal experiment, is subject to change and shouldn't be used in production.
It'd be good to have a basic integration test of this once we have a CLI to test it against, just to make sure everything is wired up correctly.
We will need to make some internal changes to start capturing telemetry properly — for instance we need to add resolve-environment to an allowlist in the monolith. Though there's no rush to make this change.

Are there any known user errors that we should ignore? Eg- trying to resolve an unknown language? Trying to avoid spurious triggers of our SLOs.

I think this will probably get addressed by the proposed handling of CLI errors in general: #1684 (comment)

Nice! Some minor suggestions to improve this before we merge.

Co-authored-by: Henry Mercer <henry.mercer@me.com>

In addition to the basic integration test included in this PR, we also have an internal repo with a more comprehensive test matrix which covers most cases that the Go autobuilder distinguishes between when resolving build environments. That forms an end-to-end test for all the new components related to this work.

One improvement around better CLI error handling for old CLIs, otherwise this LGTM.

Checks are currently failing since nightly-latest doesn't have a version number of 2.13.4+ yet. I suggest we point to the new 2.13.4 bundle directly (stable-v2.13.4) or alternatively wait until #1721 is merged and use latest in the checks.

Thanks for addressing the feedback on the naming of the check step! This LGTM

mbg self-assigned this May 18, 2023

mbg force-pushed the mbg/add-resolve-environment branch from 1dca7fb to bbaaef5 Compare May 26, 2023 10:40

henrymercer reviewed May 26, 2023

View reviewed changes

mbg force-pushed the mbg/add-resolve-environment branch 3 times, most recently from a01d376 to f95520d Compare June 6, 2023 11:22

mbg force-pushed the mbg/add-resolve-environment branch from 6a72d37 to f95520d Compare June 13, 2023 11:06

mbg marked this pull request as ready for review June 13, 2023 11:11

mbg requested a review from a team as a code owner June 13, 2023 11:11

aeisenberg reviewed Jun 13, 2023

View reviewed changes

henrymercer reviewed Jun 13, 2023

View reviewed changes

mbg force-pushed the mbg/add-resolve-environment branch from fa9ebea to 0485950 Compare June 13, 2023 19:15

mbg and others added 18 commits June 13, 2023 20:45

Add initial stub for resolve-environment action

dce63d8

Add resolveBuildEnvironment to CodeQL type

4322324

Update OS representation

360d2b7

Add resolve-environment to ActionName

cc95e67

Add runResolveBuildEnvironment

4356c16

Add initial workflow code dependent on init

ee80b30

Add token and matrix inputs

0930314

Fix: rename output

4518431

Add working-directory input

527ea94

Fix formatting for runResolveBuildEnvironment

59a9e3a

Install CodeQL as part of the action

8216c3a

Apply suggestions from code review

aef016d

Co-authored-by: Henry Mercer <henry.mercer@me.com>

Rename actionName to ACTION_NAME

ddb13a1

Remove BuildEnvironmentOS

fce65da

Propagate --working-dir argument

6f07b50

Include CLI output in JSON error

e08f131

Do not download CLI

829bba8

Report success

64fa96a

mbg and others added 9 commits June 13, 2023 20:45

Add integration test for resolve-environment

0b8c812

Use nightly-latest for test for now

ef4bf4a

Apply suggestions from code review

8e75e1a

Co-authored-by: Henry Mercer <henry.mercer@me.com>

Remove validateWorkflow call

16f3f1a

Handle case where workingDir is undefined

6419f99

Update action description

efe0e21

Update working-directory description

1c73235

Add CHANGELOG entry

ba64fee

Improve changelog wording

c878505

mbg force-pushed the mbg/add-resolve-environment branch from 0485950 to c878505 Compare June 13, 2023 19:47

Rebuild integration test workflow

f239f49

Continue after CLI errors

899b5a2

mbg requested review from aeisenberg and henrymercer June 14, 2023 15:04

henrymercer reviewed Jun 14, 2023

View reviewed changes

Comment thread src/resolve-environment-action.ts

Don't invoke CLI when command not supported

7a9f099

mbg requested a review from henrymercer June 15, 2023 09:15

Reformat

eb9703e

henrymercer previously approved these changes Jun 15, 2023

View reviewed changes

Use stable-v2.13.4 for integration test

7e5bafc

mbg dismissed henrymercer’s stale review via 7e5bafc June 15, 2023 10:04

Improve step name in integration test

d4006d9

henrymercer approved these changes Jun 15, 2023

View reviewed changes

mbg merged commit 0ac1815 into main Jun 15, 2023

mbg deleted the mbg/add-resolve-environment branch June 15, 2023 16:50

github-actions Bot mentioned this pull request Jun 21, 2023

Merge main into releases/v2 #1736

Merged

6 tasks

aliscco mentioned this pull request Jun 24, 2023

[Snyk] Fix for 1 vulnerabilities aliscco/codeql-action#704

Open

+                  );
+                  core.setOutput("environment", result);
+                } catch (unwrappedError) {
+                  const error = wrapError(unwrappedError);

Add `resolve-environment` action by mbg · Pull Request #1684 · github/codeql-action · GitHub

Add resolve-environment action - #1684

Add resolve-environment action#1684mbg merged 33 commits intomaingithub/codeql-action:mainfrom mbg/add-resolve-environmentgithub/codeql-action:mbg/add-resolve-environmentCopy head branch name to clipboard

Conversation

mbg commented May 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

henrymercer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aeisenberg Jun 13, 2023

Choose a reason for hiding this comment

Uh oh!

mbg Jun 14, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

henrymercer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mbg commented Jun 13, 2023

Uh oh!

henrymercer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

henrymercer left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

henrymercer left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add resolve-environment action#1684
mbg merged 33 commits into
mainfrom
mbg/add-resolve-environment

mbg commented May 18, 2023 •
edited
Loading

henrymercer left a comment •
edited
Loading