The only problem is that it will be green if there are no changes in test/internet, so it might not be practical to know if it really ran or not IMO. A solution would be (with fetch-depth: 0):

      - name: Get changed test/internet files
        id: check-internet
        run: echo "::set-output name=changed::$(git diff --name-only origin/${{ github.base_ref || 'master' }} HEAD -- test/internet)"
      - name: Test Internet
        # Only run if files in test/internet have changed.
        if: steps.check-internet.outputs.changed
        run: make test-internet -j2 V=1

But at this point, an action like https://github.com/tj-actions/changed-files could even be used.

Refs (-j2 -> -j): #40080 (comment)

The only problem is that it will be green if there are no changes in test/internet, so it might not be practical to know if it really ran or not IMO.

I'm OK with that, especially if it keeps the workflow simple and light on dependencies.

Refs (-j2 -> -j): #40080 (comment)

I left a comment there but I'm pretty sure we don't want to do that.

The multi-line suggestion is a good one. Right now, this isn't working though because of the way the workflow fetches from origin. So that if stuff might be going away. If not, I'll do it your way for sure. Thanks! Pondering the right solution...

What made you add --exit-code?

What made you add --exit-code?

git diff always exits with code 0, even if diff is empty.
With --exit-code it exits with code 1 if changes, otherwise 0.

I don't think we need that. The if [[ $( COMMAND ) ]] means "true if there is any output from COMMAND, otherwise false".

Added it as a daily/on-demand workflow for now. We can later improve it to run on pull requests and pushes when appropriate only (or not).

I want to say that it's almost useless if you do it that way. It would just be impossible to run this workflow on a PR that changes the internet tests, and that's a shame, it loses all its usefulness IMO.

The additions you're proposing here may be worth doing, but we should probably keep it on a schedule too. Other changes may break internet tests (such as changes to cares or dns.js) and we might not notice for a long time. If it's a month later, it will be very useful to have daily results to go back to so we can immediately narrow the issue down to a single day's worth of commits.

My idea was that someone can manually launch the job on PRs if the PR branch is in the repo. (Or they can launch it on their own branch if it's in their repo.)

By the way, the "run once a day and run it manually" mirrors what we do on Jenkins now, so they idea is that this would allow us to get rid of the Jenkins job.

(Although it does seem like the Jenkins job runs much faster. Maybe I'm doing something wrong in the workflow and there's a way to have a persistent compile cache to speed things up or something.)

I have updated the suggestion.

If it's OK with you, I think my preference is to land this as-is and then incrementally add these files/paths over time. We've gone years and years without having these jobs run automatically, and there's always been certain concerns around running them routinely in CI. These concerns (don't want to hit a DNS, web, or other external service too much from our CI, for example) may not be well-founded, but just in case, I think an incremental approach is still the way to go.

It's OK with me.