FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

adapter: read the config sync file once per tick and cache its parse by claude[bot] · Pull Request #39574 · MaterializeInc/materialize · GitHub

Repository navigation

adapter: read the config sync file once per tick and cache its parse - #39574

Open
claude[bot] wants to merge 1 commit into
mainfrom
configmap-read-once-cache
Open

claude[bot] wants to merge 1 commit into
mainfrom
configmap-read-once-cache

Conversation

claude Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Requested by Moritz Hoffmann · Slack thread

Motivation

First half of a two-PR stack. #38208 (cluster- and replica-scoped parameters in the config-sync file) is stacked on this PR. This part changes how the existing flat ConfigMap is read and nothing about what it means, so it can be reviewed and landed on its own.

Before: the file-backed frontend re-read and re-parsed the file once per synchronized parameter on every tick, and logged a warning per parameter per tick while the file was missing or malformed. A missing or unparseable file was handed to the scoped reconcile as a complete, empty desired state, which pruned every scoped override.

After: the file is read once per tick, and the parse is cached until the file's bytes change, so the warnings fire once per edit. A tick whose read failed or did not parse skips the scoped reconcile, so an unreadable file leaves the scoped overrides alone.

Description

  • SystemParameterFrontend::pull reads the file once, through refresh_config_file, which compares the bytes against the cached read and re-parses only on a change.
  • has_scoped_desired_state reports whether the latest read parsed (always true for LaunchDarkly). sync_scoped_params returns early while it is false.
  • Values for a flat file come out as they do today: the same JSON-to-string conversion, null means no opinion, and a parameter the file does not mention, or every parameter while the file is unreadable, keeps its current value.

On main the file client produces no scoped overrides, so skipping the reconcile only matters for rows written some other way. It matters once #38208 gives the file scoped rules, and it belongs with the read path.

Verification

New unit tests in src/adapter/src/config/frontend.rs:

  • test_parse_flat_file_is_environment_wide, test_parse_rejects_non_object_document: flat-file parsing and the JSON-to-string conversion.
  • test_config_file_cached_until_it_changes: unchanged bytes reuse the parse.
  • test_unreadable_file_has_no_scoped_desired_state: a missing file, invalid JSON, or a non-object document reports no desired state. An empty object does report one.
  • test_pull_from_file: pull against a real file, covering values, null, and a file that turns unparseable and then goes missing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KdNahqJ7ASK2z2EZVEvr36


Generated by Claude Code

The file-backed system parameter frontend re-read and re-parsed the file
once per synchronized parameter on every tick, and logged a warning per
parameter per tick while the file was missing or malformed. It also
could not tell the scoped reconcile that it knew nothing, so a missing
or unparseable file was reconciled as a complete desired state of "no
scoped overrides", pruning every one.

Read the file once per tick, keep the parse until the file's bytes
change, and warn once per change. Report whether the latest read parsed
through `has_scoped_desired_state`, and skip the scoped reconcile while
it did not, leaving existing overrides in place.

Values for a flat file are unchanged: the same JSON-to-string
conversion, `null` means no opinion, and a parameter the file does not
mention, or every parameter while the file is unreadable, keeps its
current value.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KdNahqJ7ASK2z2EZVEvr36

claude Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

buildkite/test/postgres-cdc-3 failed on 716ca37 (build 137699, exit status 1). I don't think this PR caused it:

  • This PR only changes how the system-parameter frontend reads the config-sync file. environmentd builds that frontend only when --config-sync-file-path or a LaunchDarkly SDK key is set (src/environmentd/src/lib.rs, the (None, None) => None arm). test/pg-cdc sets neither, so the changed code never runs there.
  • The other five postgres-cdc shards and postgres-cdc-against-a-physical-standby passed on the same build. On the other ~20 open PRs that ran it today, postgres-cdc-3 passed.

I couldn't read the job log: Buildkite is blocked from the environment I work in. I also can't retry the job from here. Could someone with Buildkite access re-run that one job?

🤖 Generated with Claude Code

antiguru added this pull request to stack #39575 October 6, 2026 08:31
antiguru requested a review from aljoscha October 6, 2026 09:04

This branch has not been deployed

No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL