FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Cherry pick #115991 to 26.3: Validate the Keeper path when converting a table with ATTACH AS REPLICATED by robot-clickhouse-ci-1 · Pull Request #121285 · ClickHouse/ClickHouse · GitHub

Cherry pick #115991 to 26.3: Validate the Keeper path when converting a table with ATTACH AS REPLICATED - #121285

Open
robot-clickhouse-ci-1 wants to merge 6 commits into
backport/26.3/115991from
cherrypick/26.3/115991
Open

robot-clickhouse-ci-1 wants to merge 6 commits into
backport/26.3/115991from
cherrypick/26.3/115991

Conversation

Copy link
Copy Markdown
Contributor

Original pull-request #115991

Do not merge this PR manually

This pull-request is a first step of an automated backporting.
It contains changes similar to calling git cherry-pick locally.
If you intend to continue backporting the changes, then resolve all conflicts if any.
Otherwise, if you do not want to backport them, then just close this pull-request.

The check results does not matter at this step - you can safely ignore them.

Before you resolve anything

Conflicts are often caused by a prerequisite change that has not been backported yet, rather than by a real divergence. The bot re-tries this cherry-pick against the release branch on every run, so if that is the case here it will merge itself as soon as the prerequisite lands, and you will see a comment saying so. Manual resolution is only needed while the conflict persists.

Troubleshooting

If the conflicts were resolved in a wrong way

If this cherry-pick PR is completely screwed by a wrong conflicts resolution, and you want to recreate it:

  • delete the pr-cherrypick label from the PR
  • delete this branch from the repository

You also need to check the Original pull-request for pr-backports-created label, and delete if it's presented there

The PR source

The PR is created in the CI job

groeneai and others added 6 commits August 23, 2026 05:55
…CATED

A plain MergeTree table has no ZooKeeper path, so ATTACH TABLE ... AS REPLICATED
mints one out of the server's default_replica_path and the table's own name. It
then marks the synthesized definition attach_short_syntax, which disables
validate_substitutions in the storage factory and so skips the path checks added
in #114006. That exemption exists because an ordinary short ATTACH re-derives the
path of a table that already exists and must keep loading whatever the stored
path expands to. A conversion is the opposite case: the path is brand new.

On a server configured with a name-based template, which the engine's own
documentation recommends and which is not the {uuid}-based default, a table named
`victim/replicas/ghost` therefore resolves to a path inside victim's own subtree.
Converting it and running SYSTEM RESTORE REPLICA registers a second replica of
victim, whose ALTER ... SETTINGS alter_sync = 2 then fails with a
KEEPER_EXCEPTION over the stray replica's missing log_pointer while SELECT and
INSERT keep working. Both statements are covered by GRANT ALL ON <database>.*, so
no privilege on the affected table is required, and only an administrator can
clear the stray replica.

Add a side-effect-free helper next to the existing checkReplicaPathExists that
resolves the prospective path and replica name through TableZnodeInfo::resolve
with validation enabled, and call it from both conversion routes, which are the
only two callers of setMergeTreeEngine. No new validation logic is introduced.

The call site in the interpreter is deliberately ahead of all three of that
function's irreversible steps. clearTransactionMetadata removes txn_version.txt
from every part on every disk with no rollback, and per
04492_attach_as_replicated_clears_tmp_txn_version a part missing that file is
read back as a rolled-back transaction and discarded as Outdated, so a rejection
sited after it could lose rows. Measured: with the check moved down, the
conversion is rejected identically but every txn_version.txt is already gone.

On the convert_to_replicated flag file route the rejection happens during
startup, so the server refuses to start. That matches what the neighbouring
checkReplicaPathExists already does on the same route, and the recovery is the
same: delete the flag file, after which the table loads unchanged.

Closes: #115967
The module was written but never executed, and it had two deterministic
failures. Both are fixed and the module now runs green under Docker, with its
rows proven to redden on a neutralized binary.

Plant and count txn_version.txt over system.parts instead of a directory
listing. A MergeTree table's data directory always contains a top-level
detached directory, so `find -maxdepth 1 -type d` counted one entry more than
the parts and the fixed expectation of 2 failed against an actual 3. The
expectation is now derived from the number of active parts, with an arming
assertion that it is non-zero, and merges are pinned off so the count is
stable across the conversion. The validated shell harness this was ported from
already enumerated system.parts; the port lost it.

Read the table's data path while the server is up in the convert-flag test.
It was read after start_clickhouse(expected_to_fail=True), which returns only
once there is provably no server process, so the query behind get_table_path
could not be answered and the test died before asserting anything, leaving the
convert flag set. The sibling test_zk_path_exists.py captures the path before
stopping the server; do the same.

Restate the T1c comment. It claimed a part losing txn_version.txt is read as a
rolled-back transaction and discarded. VersionMetadataOnDisk::loadMetadata
returns a NonTransactionalTID/NonTransactionalCSN committed part when neither
the file nor its .tmp is present; the rolled-back path needs a surviving .tmp,
which clearTransactionMetadata removes alongside the main file. The row count
never discriminated anything and was constant across every measured arm. What
justifies the call site is that the removal is irreversible, which is what the
comment now says.

Add a control for the reverse conversion direction. ATTACH ... AS NOT
REPLICATED mints no Keeper path, so an unsafe name must not block it. The
table has to be replicated and unsafely named at once, which a conversion
cannot produce, so it is created directly with an explicit path. Deleting the
to_replicated guard turns this row red, so it asserts the guard rather than
decorating it.

No source change: the diff against the reviewed tree under src/ is empty.
A conversion mints a Keeper path the table never had, so the substituted
{database}/{table} value has to be validated as strictly as a CREATE
validates it. Requesting LoadingStrictnessLevel::CREATE for that also
imposed normalizeZooKeeperPath's requirement that the path begin with
'/', which is a separate rule that only ever applied to a genuinely new
table. A server configured with a relative default_replica_path used to
convert with a deprecation warning; it would have started failing with
BAD_ARGUMENTS, and on the convert_to_replicated flag-file route it would
have refused to boot.

SECONDARY_CREATE keeps the whole rejection: mode is read at exactly two
places in TableZnodeInfo::resolve, and both checkSubstitutedValues and
checkPathComponents are gated on validate_substitutions alone and take
no mode at all. Only the leading-slash predicate at
extractZooKeeperPath differs between the two levels. Measured on both
levels against one relative-path server: the conversion succeeds with
the deprecation warning under SECONDARY_CREATE and fails BAD_ARGUMENTS
under CREATE, while an unsafe table name is rejected identically by
both.

Also write $CLICKHOUSE_TEST_ZOOKEEPER_PREFIX inline in the engine
argument of 04853. The style check greps for the token on the engine
line itself, so assigning it to a shell variable first does not
satisfy check_style's various_checks.sh. The prefix expands to
"${CLICKHOUSE_TEST_NAME}_${CLICKHOUSE_DATABASE}", a strict superset of
the database name it replaces, so parallel copies of the test still get
distinct Keeper paths.
The cases were labelled T1/T4 and C1/C4, which only mean something to a reader
who has the fix plan in front of them. Each label now says which shape of name
or path the case exercises, so an assertion reads on its own.

The stateless labels are emitted output, so the reference moves with them. The
integration test labels were comment-only, so its behaviour is unchanged.
The two rows grepped the output for BAD_ARGUMENTS, so any other failure of the
short ATTACH or of SYSTEM RESTART REPLICA was reported as success. Both routes
register the table in the catalog before calling startup(), so the table-count
assertions that follow cannot see such a failure either.

Verified by pointing the exempted ATTACH at a missing table: the row now fails
the test with the client status (rc=134 and rc=60), where the previous form
printed ATTACHED and passed.
…d-keeper-path-115967

Validate the Keeper path when converting a table with ATTACH AS REPLICATED
robot-clickhouse-ci-1 added pr-cherrypick Cherry-pick of merge-commit before backporting. Do not use manually - automated use only! do not test disable testing on pull request pr-bugfix Pull request with bugfix, not backported by default labels Sep 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not test disable testing on pull request pr-bugfix Pull request with bugfix, not backported by default pr-cherrypick Cherry-pick of merge-commit before backporting. Do not use manually - automated use only!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL