FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Fix flaky compactor tests: ignore userIndexUpdateLoop error logs in exact log-set assertions by Copilot · Pull Request #7859 · cortexproject/cortex · GitHub

Fix flaky compactor tests: ignore userIndexUpdateLoop error logs in exact log-set assertions - #7859

Open
CharlieTLe with Copilot wants to merge 2 commits into
masterfrom
copilot/fix-flaky-compactor-test
Open

CharlieTLe with Copilot wants to merge 2 commits into
masterfrom
copilot/fix-flaky-compactor-test

Conversation

Copilot AI commented Sep 22, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

What this PR does:

TestCompactor_ShouldCompactAllUsersOnShardingEnabledButOnlyOneInstanceRunning fails intermittently on the arm64 -race CI job: the background userIndexUpdateLoop ticks every ~100ms in tests (via prepare()'s user-index strategy) and logs a ring error whenever no healthy instance exists — before the lifecycler is ACTIVE or during LEAVING at shutdown. Any tick landing in the test window leaks an extra line into the exact log-set assertion, failing with elements differ.

removeIgnoredLogs already suppresses two of the loop's log lines via map entries; the two error lines were missed because their variable err= payloads require regexes. This adds them:

ignoredLogStringsRegexList := []*regexp.Regexp{
	regexp.MustCompile(`^level=(info|debug|warn) component=cleaner .+$`),
	regexp.MustCompile(`^level=info component=compactor msg="set state" .+$`),
	regexp.MustCompile(`^level=error component=compactor msg="failed to check if compactor owns updating user index" err=.+$`),
	regexp.MustCompile(`^level=error component=compactor msg="failed to update user index" err=.+$`),
}

Since all 14 exact log-set assertions in compactor_test.go and compactor_paritioning_test.go funnel through removeIgnoredLogs, this single change covers every sharding-enabled test exposed to the same race. Regexes were verified against real go-kit logfmt output, including the exact err="at least 1 live replicas required, could only find 0" line from the failed CI jobs. Test-only change; no production behavior modified.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
  • docs/configuration/v1-guarantees.md updated if this PR introduces experimental flags

Co-authored-by: CharlieTLe <3375195+CharlieTLe@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix flaky test for compactor log output assertion Fix flaky compactor tests: ignore userIndexUpdateLoop error logs in exact log-set assertions Sep 22, 2026
Copilot AI requested a review from CharlieTLe September 22, 2026 18:56
CharlieTLe marked this pull request as ready for review September 22, 2026 21:52
CharlieTLe requested a review from a team as a code owner September 22, 2026 21:52

This branch has not been deployed

No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

2 participants


Back | FazBrowse Home | New Git URL