| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
…sync cancel Connector half of the API+CUJ audit fixes (kernel half: databricks-sql-kernel PR #121). Bumps KERNEL_REV to pick up the kernel surface. Staging fail-loud (kernel/client.py): - Volume/staging PUT/GET/REMOVE silently no-op'd on the kernel path (KernelResultSet.is_staging_operation is always False, so the connector's _handle_staging_operation never fired and no file was transferred). Detect the leading verb in execute_command and raise NotSupportedError so ETL fails loud instead of ingesting stale data. Error context (_errors.py): - Forward display_message / diagnostic_info / error_details_json (now exposed across the pyo3 boundary in #121) onto the re-raised PEP-249 exception, and populate ServerOperationError.context with "diagnostic-info" (Spark stack trace) + "operation-id" — matching the Thrift backend so callers reading err.context work identically. Sync cancel wiring (client.py, kernel/client.py): - cursor.cancel() was a silent no-op for the default blocking execute() (active_command_id is None until execute returns). The kernel backend now registers a detached StatementCanceller (keyed by the cursor) before the blocking execute and exposes cancel_running_cursor(cursor). Cursor.cancel() routes to that hook via getattr when there's no command id yet — opt-in, so Thrift/SEA backends are unaffected. Tests: unit (staging fail-loud, _is_staging_statement, cancel registry + routing, error context/diagnostic-info forwarding) and e2e (tz-aware TIMESTAMP, scientific DECIMAL, staging NotSupportedError, diagnostic-info context, cross-thread sync cancel interrupts a running query). All e2e verified live against dogfood with use_kernel=True. Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
|
🟠 P1 (Important — should fix)
🟡 P2 (Minor)
|
Sorry, something went wrong.
…c cancel Review fixes on PR #825 (+ KERNEL_REV bump to the amended kernel #121 which now folds the original message on attach failure and bounds error_details_json): P1 #1 — staging fail-loud missed comment-prefixed statements: _is_staging_statement took the first whitespace token without stripping SQL comments, so "-- upload\nPUT ..." / "/* c */ PUT ..." (common in ETL) classified as non-staging and slipped into the silent-no-op bug. Added _strip_leading_sql_comments (handles leading -- line and /* */ block comments, multiple/mixed) before extracting the verb. Tests for both comment forms, mixed, and verb-only-in-comment (must NOT match). P1 #2 — sync cancel could raise out of cursor.cancel(): cursor.cancel() is best-effort per PEP-249, but cancel_running_cursor re-raised a canceller failure (e.g. an early cancel before the server statement id is observed, or a transport hiccup) via the public cancel(). Now swallow+log and still return True (a canceller was present and attempted) so Cursor doesn't emit the misleading "no executing command" warning. Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
|
Thanks @gopalldb — addressed in 4413040 (+ KERNEL_REV bumped to the amended kernel #121). P1 #1 — staging fail-loud missed comment-prefixed statements ✅ Fixed. _is_staging_statement now calls _strip_leading_sql_comments first, which skips leading -- … line and /* … */ block comments (multiple/mixed) before extracting the verb. So -- upload\nPUT … and /* c */ PUT … are correctly classified as staging and fail loud. Added unit cases for both comment forms, mixed comments, leading-whitespace, and the inverse (-- PUT …\nSELECT 1 / /* PUT */ SELECT 1 must NOT match), plus a live e2e (test_comment_prefixed_staging_put_raises_not_supported). P1 #2 — early-cancel could propagate out of cursor.cancel() ✅ Fixed. cancel_running_cursor is now tolerant: it swallows + logs (exc_info=True) a canceller failure instead of re-raising via _wrap_kernel_exception, matching cancel_command's tolerance and the PEP-249 expectation that cursor.cancel() doesn't raise. It still returns True (a canceller was present and attempted) so the Cursor doesn't emit the misleading "no executing command" warning. (Also: the kernel-side StatementCanceller.cancel() is a documented no-op before the id is observed, so the common early-cancel case doesn't even reach this path.) Added a unit test asserting a raising canceller is swallowed and True is returned. P2s — all acknowledged:
|
Sorry, something went wrong.
#121 (tz-aware/scientific param binds, error context, sync cancel + Ctrl-C) is merged to kernel main. Re-pin from the orphaned branch HEAD (f62d941) to the merged squash SHA (cbeaf44) — content-identical, but reachable from main so no orphan-SHA risk. Verified against a wheel built from cbeaf44: connector unit + kernel e2e (tz-aware TIMESTAMP, scientific DECIMAL, staging fail-loud incl. comment-prefixed, diagnostic-info context) all pass. Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
| Back | FazBrowse Home | New Git URL |
What
Connector half of the use_kernel API+CUJ gap audit fixes. The kernel half is databricks-sql-kernel#121; this PR wires it through and bumps KERNEL_REV.
1. Staging fail-loud (kernel/client.py)
Volume/staging PUT/GET/REMOVE silently no-op'd on the kernel path — KernelResultSet.is_staging_operation is always False, so the connector's _handle_staging_operation never fired and no file was transferred (ETL silently ingests stale/missing data). Now detects the leading verb in execute_command and raises NotSupportedError, so it fails loud. (Real staging support tracked separately — needs a kernel staging marker.)
2. Error context (_errors.py)
Forward display_message / diagnostic_info / error_details_json (now exposed across the pyo3 boundary in #121) onto the re-raised PEP-249 exception, and populate ServerOperationError.context with "diagnostic-info" (the Spark stack trace) + "operation-id" — matching the Thrift backend so except ServerOperationError as e: e.context["diagnostic-info"] works identically on use_kernel.
3. Sync cancel wiring (client.py, kernel/client.py)
cursor.cancel() was a silent no-op for the default blocking execute() (the documented cross-thread shape) — active_command_id is None until execute returns. The kernel backend now registers a detached StatementCanceller (keyed by the cursor) before the blocking execute and exposes cancel_running_cursor(cursor). Cursor.cancel() routes to that hook via getattr when there's no command id yet — opt-in, so the Thrift/SEA backends (which set active_command_id before blocking) are unaffected.
Testing
Depends on #121 (kernel). black (pinned 22.3.0) + unit suites green.
This pull request and its description were written by Isaac.