| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Behind a feature flag
|
I think I'm running a newer version of npm on my machine which is reformatting the package.json files. |
Sorry, something went wrong.
|
Pushed a commit to update the checked-in dependencies. Please mark the PR as ready for review to trigger PR checks. |
Sorry, something went wrong.
There was a problem hiding this comment.
Looks good to me 👍 thanks for adding the tests!
Sorry, something went wrong.
| [Feature.ZstdBundleStreamingExtraction]: { | ||
| defaultValue: false, | ||
| envVar: "CODEQL_ACTION_ZSTD_BUNDLE_STREAMING_EXTRACTION", | ||
| minimumVersion: undefined, | ||
| }, |
There was a problem hiding this comment.
💡 This flag should be set only when the ZstdBundle flag is also set to true, right? What happens in the case the streaming flag is set and the ZstdBundle isn't — we just fall back to gzip regardless, right?
Sorry, something went wrong.
There was a problem hiding this comment.
Yeah, if the ZstdBundle flag isn't set, this flag has no effect.
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
We can save time installing the bundle by streaming the download and extraction phases, so that CPU time previously spent waiting on the network can now be spent extracting what we've received so far.
Some implementation notes:
Streaming is enabled behind a feature flag.
We implement streaming only for Zstandard so that in the event of a bug with streaming, we can fallback to a download first then extract method.
The Actions toolkit HTTP client downloads the full response body, so we can't use it to stream the extraction. Therefore, we use the lower-level Node APIs. But these don't handle redirects, so we add a dependency on follow-redirects to avoid implementing error-prone redirect handling ourselves.
We add some performance measurements so we can see how much faster streaming is compared to downloading first then extracting.
This PR also adds some logging improvements to the setup CodeQL step, removing the verbose tar extraction output, and only echoing commands for which we're streaming the output.
Merge / deployment checklist