| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
…CATED A plain MergeTree table has no ZooKeeper path, so ATTACH TABLE ... AS REPLICATED mints one out of the server's default_replica_path and the table's own name. It then marks the synthesized definition attach_short_syntax, which disables validate_substitutions in the storage factory and so skips the path checks added in #114006. That exemption exists because an ordinary short ATTACH re-derives the path of a table that already exists and must keep loading whatever the stored path expands to. A conversion is the opposite case: the path is brand new. On a server configured with a name-based template, which the engine's own documentation recommends and which is not the {uuid}-based default, a table named `victim/replicas/ghost` therefore resolves to a path inside victim's own subtree. Converting it and running SYSTEM RESTORE REPLICA registers a second replica of victim, whose ALTER ... SETTINGS alter_sync = 2 then fails with a KEEPER_EXCEPTION over the stray replica's missing log_pointer while SELECT and INSERT keep working. Both statements are covered by GRANT ALL ON <database>.*, so no privilege on the affected table is required, and only an administrator can clear the stray replica. Add a side-effect-free helper next to the existing checkReplicaPathExists that resolves the prospective path and replica name through TableZnodeInfo::resolve with validation enabled, and call it from both conversion routes, which are the only two callers of setMergeTreeEngine. No new validation logic is introduced. The call site in the interpreter is deliberately ahead of all three of that function's irreversible steps. clearTransactionMetadata removes txn_version.txt from every part on every disk with no rollback, and per 04492_attach_as_replicated_clears_tmp_txn_version a part missing that file is read back as a rolled-back transaction and discarded as Outdated, so a rejection sited after it could lose rows. Measured: with the check moved down, the conversion is rejected identically but every txn_version.txt is already gone. On the convert_to_replicated flag file route the rejection happens during startup, so the server refuses to start. That matches what the neighbouring checkReplicaPathExists already does on the same route, and the recovery is the same: delete the flag file, after which the table loads unchanged. Closes: #115967
The module was written but never executed, and it had two deterministic failures. Both are fixed and the module now runs green under Docker, with its rows proven to redden on a neutralized binary. Plant and count txn_version.txt over system.parts instead of a directory listing. A MergeTree table's data directory always contains a top-level detached directory, so `find -maxdepth 1 -type d` counted one entry more than the parts and the fixed expectation of 2 failed against an actual 3. The expectation is now derived from the number of active parts, with an arming assertion that it is non-zero, and merges are pinned off so the count is stable across the conversion. The validated shell harness this was ported from already enumerated system.parts; the port lost it. Read the table's data path while the server is up in the convert-flag test. It was read after start_clickhouse(expected_to_fail=True), which returns only once there is provably no server process, so the query behind get_table_path could not be answered and the test died before asserting anything, leaving the convert flag set. The sibling test_zk_path_exists.py captures the path before stopping the server; do the same. Restate the T1c comment. It claimed a part losing txn_version.txt is read as a rolled-back transaction and discarded. VersionMetadataOnDisk::loadMetadata returns a NonTransactionalTID/NonTransactionalCSN committed part when neither the file nor its .tmp is present; the rolled-back path needs a surviving .tmp, which clearTransactionMetadata removes alongside the main file. The row count never discriminated anything and was constant across every measured arm. What justifies the call site is that the removal is irreversible, which is what the comment now says. Add a control for the reverse conversion direction. ATTACH ... AS NOT REPLICATED mints no Keeper path, so an unsafe name must not block it. The table has to be replicated and unsafely named at once, which a conversion cannot produce, so it is created directly with an explicit path. Deleting the to_replicated guard turns this row red, so it asserts the guard rather than decorating it. No source change: the diff against the reviewed tree under src/ is empty.
A conversion mints a Keeper path the table never had, so the substituted
{database}/{table} value has to be validated as strictly as a CREATE
validates it. Requesting LoadingStrictnessLevel::CREATE for that also
imposed normalizeZooKeeperPath's requirement that the path begin with
'/', which is a separate rule that only ever applied to a genuinely new
table. A server configured with a relative default_replica_path used to
convert with a deprecation warning; it would have started failing with
BAD_ARGUMENTS, and on the convert_to_replicated flag-file route it would
have refused to boot.
SECONDARY_CREATE keeps the whole rejection: mode is read at exactly two
places in TableZnodeInfo::resolve, and both checkSubstitutedValues and
checkPathComponents are gated on validate_substitutions alone and take
no mode at all. Only the leading-slash predicate at
extractZooKeeperPath differs between the two levels. Measured on both
levels against one relative-path server: the conversion succeeds with
the deprecation warning under SECONDARY_CREATE and fails BAD_ARGUMENTS
under CREATE, while an unsafe table name is rejected identically by
both.
Also write $CLICKHOUSE_TEST_ZOOKEEPER_PREFIX inline in the engine
argument of 04853. The style check greps for the token on the engine
line itself, so assigning it to a shell variable first does not
satisfy check_style's various_checks.sh. The prefix expands to
"${CLICKHOUSE_TEST_NAME}_${CLICKHOUSE_DATABASE}", a strict superset of
the database name it replaces, so parallel copies of the test still get
distinct Keeper paths.
The cases were labelled T1/T4 and C1/C4, which only mean something to a reader who has the fix plan in front of them. Each label now says which shape of name or path the case exercises, so an assertion reads on its own. The stateless labels are emitted output, so the reference moves with them. The integration test labels were comment-only, so its behaviour is unchanged.
The two rows grepped the output for BAD_ARGUMENTS, so any other failure of the short ATTACH or of SYSTEM RESTART REPLICA was reported as success. Both routes register the table in the catalog before calling startup(), so the table-count assertions that follow cannot see such a failure either. Verified by pointing the exempted ATTACH at a missing table: the row now fails the test with the client status (rc=134 and rc=60), where the previous form printed ATTACHED and passed.
…d-keeper-path-115967 Validate the Keeper path when converting a table with ATTACH AS REPLICATED
| Back | FazBrowse Home | New Git URL |
Original pull-request #115991
Do not merge this PR manually
This pull-request is a first step of an automated backporting.
It contains changes similar to calling git cherry-pick locally.
If you intend to continue backporting the changes, then resolve all conflicts if any.
Otherwise, if you do not want to backport them, then just close this pull-request.
The check results does not matter at this step - you can safely ignore them.
Before you resolve anything
Conflicts are often caused by a prerequisite change that has not been backported yet, rather than by a real divergence. The bot re-tries this cherry-pick against the release branch on every run, so if that is the case here it will merge itself as soon as the prerequisite lands, and you will see a comment saying so. Manual resolution is only needed while the conflict persists.
Troubleshooting
If the conflicts were resolved in a wrong way
If this cherry-pick PR is completely screwed by a wrong conflicts resolution, and you want to recreate it:
You also need to check the Original pull-request for pr-backports-created label, and delete if it's presented there
The PR source
The PR is created in the CI job