| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
For all-purpose-compute Thrift connections on SPOG (custom-URL) hosts the http_path is /sql/protocolv1/o/<workspace-id>/<cluster-id> and the workspace ID is encoded in the path itself. PoPP routes the Thrift request correctly off the /o/<wsid>/ segment, so the connection succeeds without an explicit ?o= query parameter. Other requests on the same connection (telemetry uploads to /telemetry-ext, feature-flag fetches, SEA REST calls) hit different paths that don't carry the workspace ID. Previously _extract_spog_headers only looked at ?o= in the http_path, so the x-databricks-org-id header was never set for cluster URLs without ?o=. On SPOG hosts PoPP then had no workspace context for these requests and redirected them to /login, silently dropping telemetry. Extend _extract_spog_headers to also extract the workspace ID from the cluster path segment as a fallback when ?o= is absent. Priority order: explicit caller header > ?o= query param > /o/<wsid>/ path segment. Adds five unit tests covering the new cluster-path extraction, leading slash, query-param-wins priority, explicit-header-wins priority, and a warehouse-path regression guard. Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
|
🟡 P2 (Minor — fast-follow)
|
Sorry, something went wrong.
There was a problem hiding this comment.
minor comments
Sorry, something went wrong.
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
## Summary - Fixes silent telemetry loss on SPOG (custom-URL) hosts when connecting to an all-purpose cluster via an httpPath like `sql/protocolv1/o/<workspace-id>/<cluster-id>`. - `extractSpogHeaders` now extracts the workspace ID from the `/o/<wsid>/` path segment as a fallback when `?o=<wsid>` is not present, and emits an `x-databricks-org-id` header so the wrapped `telemetryClient` / feature-flag client can route correctly on SPOG. - Priority order preserved: `?o=` in `httpPath` ▶ `/sql/protocolv1/o/<wsid>/` path segment. Caller-set request headers still win, including mixed-case `X-Databricks-Org-Id`. ## Why On a SPOG host the workspace identity has to be in either the URL or the `x-databricks-org-id` header for PoPP to route a request to the correct workspace. For all-purpose cluster Thrift this is free — the workspace ID is in the `/o/<wsid>/` segment of httpPath, so PoPP routes Thrift via `routing_reason=workspace-id` and the session opens fine without an explicit `?o=`. The telemetry and feature-flag transports built by `connector.Connect` wrap `c.client` with `withSpogHeaders` only when `extractSpogHeaders(c.cfg.HTTPPath)` returns a non-nil map. Before this PR that only happened when `?o=` was present, so cluster URLs without `?o=` produced no `x-databricks-org-id` header, PoPP fell back to default (account) routing on `/telemetry-ext`, and the responses were 303 redirects to `/login` — silently dropping telemetry on SPOG. ## What changes `connector.go` - New `clusterPathOrgIDPattern` regex (`(?:^|/)sql/protocolv1/o/(\d+)/[^/?]+`) compiled once at package init. - `extractSpogHeaders` checks `?o=` first (existing behavior), then falls back to the cluster path segment, then returns nil. The malformed-query-string path now also falls through to path inspection instead of returning early. Log messages indicate which source produced the workspace ID. - Updated the connection comment to describe both query-param and cluster-path extraction. - `regexp` added to imports. `connector_spog_test.go` - Four new table entries under `TestExtractSpogHeaders`: - Cluster path without `?o=` → header extracted from `/o/<wsid>/` - Cluster path with leading `/` → same - Cluster path with `?o=` → query-param value wins - Warehouse path without `?o=` → still nil (regression guard: the new regex must not match warehouse paths) - New transport regression test proving a mixed-case caller-set `X-Databricks-Org-Id` is not overwritten by the SPOG wrapper. ## Test plan - [x] `go test -run 'TestExtractSpogHeaders|TestHeaderInjectingTransport' -v .` — focused SPOG extraction and transport tests pass. - [x] `go test -short .` — root package suite passes. - [x] Behavior validated on the OSS JDBC equivalent fix against Prod SPOG (`peco.azuredatabricks.net`) all-purpose cluster: telemetry POST `/telemetry-ext` flipped from `HTTP 303 → /login` to `HTTP 200 OK` once the path-segment extraction populated `x-databricks-org-id`. Same shape of fix here. ## Out of scope - The corresponding OSS JDBC driver fix is opened as databricks/databricks-jdbc#1475. - The Python connector (`databricks/databricks-sql-python`) has a sibling upstream-head PR for the same fix: databricks/databricks-sql-python#817. - The Node.js connector (`databricks/databricks-sql-nodejs`) already extracts org ID from both query param and path segment (see `extractWorkspaceId` in `lib/DBSQLClient.ts`). This pull request and its description were written with assistance from Claude Code. --------- Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
| Back | FazBrowse Home | New Git URL |
Summary
Why
On a SPOG host the workspace identity has to be in either the URL or the x-databricks-org-id header so PoPP can route a request to the right workspace. For all-purpose cluster Thrift this is free — the workspace ID is in the /o/<wsid>/ segment of the http_path, so PoPP routes Thrift via routing_reason=workspace-id and the session opens fine without ?o=.
Connection-scoped HTTP clients used for telemetry, feature flags, and the SEA backend talk to different paths (/telemetry-ext, /api/...) that do not carry the workspace ID. The previous extraction only looked at ?o= in the query string, so on a cluster http_path without ?o= no x-databricks-org-id header was ever attached. PoPP fell back to default (account) routing on those endpoints and responded with a 303 redirect to /login — silently dropping every telemetry batch.
What changes
src/databricks/sql/session.py
tests/unit/test_session.py
Test plan
Out of scope
This pull request and its description were written with assistance from Claude Code.