test(e2e): add e2e tests for workbooks, datasources, views, pagination, jobs, tagging, projects, favorites, metadata, site-admin by jacalata · Pull Request #1823 · tableau/server-client-python · GitHub

Summary

Adds e2e tests for 10 TSC functional areas: workbooks CRUD, datasources CRUD, views export, pagination, jobs/async, tagging, projects admin, favorites, metadata GraphQL, and site-admin operations
Consolidates shared boilerplate (project lookup, server fixture, SSL bypass) into conftest.py; adds server_admin, default_project, project_id session-scoped fixtures and e2e_admin marker
All tests validated against a live Tableau server; real failures found and fixed (SSL, xdist incompatibility, extract refresh 403, tagging 404 in finally block, workbook filter flakiness)

Test plan

Run pytest test_e2e/ -v -n 0 -m e2e against a Tableau server with TABLEAU_SERVER, TABLEAU_TOKEN_NAME, TABLEAU_TOKEN, TABLEAU_SITE, TABLEAU_PROJECT, TABLEAU_VERIFY_SSL set — 41 passed, 1 known flake (test_datasources_get name-filter timing), 3 skipped pending Add Projects.get_by_path() for hierarchy-aware project lookup #1804 merge (test_projects_get_by_path)
Run pytest test_e2e/test_site_admin.py -v -n 0 -m e2e_admin with additional TABLEAU_SITEADMIN_TOKEN_NAME / TABLEAU_SITEADMIN_TOKEN env vars — skipped, test server does not support user creation via REST API
Confirm no regressions in existing tests under test/ — CI passing

🤖 Generated with Claude Code

Adds integration tests covering: - Workbooks CRUD (create, publish, get, update, download, delete) - Datasources CRUD (publish, get, update connections, refresh, delete) - Views export (PDF, PNG, CSV, Excel) - Tag quoting (spaces and commas in tag names) - Projects get_by_path - Pagination (Pager and QuerySet, order_by, pagesize) - Jobs and async operations (trigger refresh, poll until complete) - Projects admin CRUD and permissions (SiteAdmin) - Users and groups (SiteAdmin) - Workbook permissions (grant, verify, revoke) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Move default_project, project_id, and server_admin into conftest.py so individual test files don't each inline the same TABLEAU_PROJECT env var lookup. Saves ~130 lines of boilerplate across 5 files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- tagging: guard finally delete_tags with try/except (tag already deleted by test body) - jobs: skip on 403180 when workbook has no extract and refresh is disallowed - workbooks: improve filter assertion error message; use QuerySet filter instead of get() - conftest: add TABLEAU_VERIFY_SSL env var to skip SSL verification for on-prem servers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…rkbook asset Restores content from old jac/e2e-tests branch (commit cf59ca4): - test_favorites.py: workbook/view/datasource favorites - test_metadata.py: GraphQL metadata API queries - test_site_admin.py: schedules, webhooks, connection update, data freshness, workbook move, user/group management (requires e2e_admin marker) - assets/WorkbookWithExtract.twbx: binary asset for extract refresh tests - test_jobs.py: add extract_workbook fixture and test_extract_workbook_refresh_completes - test_workbooks_crud.py: add test_workbook_populate_preview_image Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Coverage Report

File	Stmts	Miss	Cover	Missing
tableauserverclient
__init__.py	5	0	100%
config.py	15	0	100%
datetime_helpers.py	25	1	1	96%
exponential_backoff.py	20	0	100%
filesys_helpers.py	31	0	100%
namespace.py	26	3	3	88%
tableauserverclient/bin
__init__.py	2	0	100%
_version.py	358	212	212	41%
tableauserverclient/helpers
__init__.py	1	0	100%
logging.py	2	0	100%
strings.py	31	1	1	97%
tableauserverclient/models
__init__.py	46	0	100%
collection_item.py	41	7	7	83%
column_item.py	55	32	32	42%
connection_credentials.py	35	11	11	69%
connection_item.py	94	14	14	85%
custom_view_item.py	144	21	21	85%
data_acceleration_report_item.py	54	1	1	98%
data_alert_item.py	158	4	4	97%
data_freshness_policy_item.py	155	15	15	90%
database_item.py	207	36	36	83%
datasource_item.py	300	12	12	96%
dqw_item.py	104	5	5	95%
exceptions.py	4	0	100%
extensions_item.py	132	4	4	97%
extract_item.py	44	4	4	91%
favorites_item.py	69	8	8	88%
fileupload_item.py	19	0	100%
flow_item.py	149	10	10	93%
flow_run_item.py	71	0	100%
group_item.py	89	6	6	93%
groupset_item.py	49	7	7	86%
interval_item.py	182	32	32	82%
job_item.py	192	10	10	95%
linked_tasks_item.py	79	1	1	99%
location_item.py	29	2	2	93%
metric_item.py	129	13	13	90%
oidc_item.py	63	3	3	95%
pagination_item.py	34	1	1	97%
permissions_item.py	111	12	12	89%
project_item.py	207	31	31	85%
property_decorators.py	100	18	18	82%
reference_item.py	26	2	2	92%
revision_item.py	59	1	1	98%
schedule_item.py	209	6	6	97%
server_info_item.py	37	7	7	81%
site_item.py	636	13	13	98%
subscription_item.py	101	2	2	98%
table_item.py	119	18	18	85%
tableau_auth.py	61	25	25	59%
tableau_types.py	27	1	1	96%
tag_item.py	15	0	100%
target.py	6	0	100%
task_item.py	56	2	2	96%
user_item.py	310	18	18	94%
view_item.py	220	16	16	93%
virtual_connection_item.py	64	8	8	88%
webhook_item.py	69	1	1	99%
workbook_item.py	362	16	16	96%
tableauserverclient/server
__init__.py	9	0	100%
exceptions.py	4	0	100%
filter.py	29	1	1	97%
pager.py	33	1	1	97%
query.py	143	15	15	90%
request_factory.py	1335	195	195	85%
request_options.py	386	5	5	99%
server.py	188	23	23	88%
sort.py	6	0	100%
tableauserverclient/server/endpoint
__init__.py	35	0	100%
auth_endpoint.py	77	11	11	86%
custom_views_endpoint.py	152	12	12	92%
data_acceleration_report_endpoint.py	21	0	100%
data_alert_endpoint.py	94	23	23	76%
databases_endpoint.py	111	30	30	73%
datasources_endpoint.py	323	33	33	90%
default_permissions_endpoint.py	44	3	3	93%
dqw_endpoint.py	45	16	16	64%
endpoint.py	212	20	20	91%
exceptions.py	77	6	6	92%
extensions_endpoint.py	31	0	100%
favorites_endpoint.py	94	22	22	77%
fileuploads_endpoint.py	51	0	100%
flow_runs_endpoint.py	62	9	9	85%
flow_task_endpoint.py	21	2	2	90%
flows_endpoint.py	198	53	53	73%
groups_endpoint.py	126	9	9	93%
groupsets_endpoint.py	72	7	7	90%
jobs_endpoint.py	67	9	9	87%
linked_tasks_endpoint.py	37	0	100%
metadata_endpoint.py	88	14	14	84%
metrics_endpoint.py	55	6	6	89%
oidc_endpoint.py	42	1	1	98%
permissions_endpoint.py	44	3	3	93%
projects_endpoint.py	178	24	24	87%
resource_tagger.py	127	35	35	72%
schedules_endpoint.py	119	11	11	91%
server_info_endpoint.py	36	10	10	72%
sites_endpoint.py	130	27	27	79%
subscriptions_endpoint.py	56	14	14	75%
tables_endpoint.py	110	36	36	67%
tasks_endpoint.py	63	6	6	90%
users_endpoint.py	183	8	8	96%
views_endpoint.py	150	9	9	94%
virtual_connections_endpoint.py	113	10	10	91%
webhooks_endpoint.py	54	9	9	83%
workbooks_endpoint.py	338	22	22	93%
TOTAL	12007	1423	88%

…urce tests SampleDS.tds is a live connection to postgres91.test.tsi.lan which Tableau Server tries to validate on publish, causing the fixture to fail if the host is unreachable. WorldIndicators.tdsx is an extract-based datasource that publishes without a connection check. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Claude code report:

Code Review

Reviewed the diff via gh pr diff 1823. Overview: adds ~1,650 lines of e2e tests across 10 functional areas (workbooks CRUD, datasources CRUD, views export, pagination, jobs, tagging expansion, projects admin, favorites, metadata GraphQL, site-admin), plus binary fixture assets and conftest.py consolidation (server_admin, default_project, project_id fixtures, new e2e_admin marker). Solid, well-organized addition to test_e2e/, which is correctly excluded from the default pytest test -n auto run.

Blocking: will fail CI black check

test_e2e/test_views_export.py (a_view fixture): if not views: pytest.fail("Published workbook has no views") is a single-line if. Confirmed locally with black --check --line-length 120 — this gets reformatted to a multi-line block, so meta-checks.yml's black --check will fail on this line as written.

Suggestions (non-blocking)

test_workbook_permissions.py's workbook_and_user fixture still does manual TABLEAU_PROJECT project-lookup instead of using the new project_id/default_project fixtures this PR introduced — worth using them for consistency with the dedup work done elsewhere (e.g. test_tagging.py).
Several except Exception: pass best-effort cleanup blocks (test_datasources_crud.py, test_pagination.py's datasource fixture, test_users_groups.py, etc.) silently swallow teardown failures. test_pagination.py's workbook cleanup already does warnings.warn(...) on failure — consider applying that pattern more consistently instead of bare pass.
Minor stylistic inconsistency: SAMPLE_DATASOURCE is passed as a bare Path in some tests and wrapped in str(...) in others (e.g. test_favorites.py, test_jobs.py). Not a bug since publish()'s PathOrFileR accepts both, but worth picking one convention.

Other notes

e2e_admin marker is registered via conftest.py's pytest_configure, but not added to pyproject.toml's [tool.pytest.ini_options] markers list (which only has e2e and packaging) — fine as-is, just flagging for consistency if marker registration is ever centralized there.
test_get_by_path_raises_for_empty_or_root_path in test_projects_get_by_path.py intentionally has no pytestmark/marker (documented in a module comment, since the file mixes e2e and e2e_admin tests) — harmless since it doesn't use server fixtures, but worth being aware it'll run under any marker selection.
No security concerns: no hardcoded credentials, tokens come from env vars per existing convention, TABLEAU_VERIFY_SSL defaults to "true" (secure by default).
Coverage is a clear net positive — exercises real server behavior (SSL, xdist incompatibility, live 403/404s) that mocked unit tests can't catch.

🤖 Generated with Claude Code

@jacalata overall this looks good. Take a look at the issues Claude noted to see if any are worth fixing/improving.

- Reformat single-line `if` in test_views_export.py's a_view fixture so black --check passes CI (was blocking meta-checks.yml). - Use conftest.default_project in test_workbook_permissions.workbook_and_user, eliminating 10 lines of duplicated project-lookup logic. - Replace bare `except Exception: pass` in teardown blocks with warnings.warn(...) for consistency with test_pagination.py's workbook cleanup. - Use bare Path for SAMPLE_DATASOURCE calls across test_favorites.py and test_jobs.py (was inconsistently wrapped in str()). - Register the e2e_admin marker in pyproject.toml so it matches conftest.py's pytest_configure. All from bcantoni's review comment on #1823. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Thanks — fixes applied in 3a4fe13:

Split the single-line if not views: pytest.fail(...) in test_views_export.py's a_view fixture so black --check passes CI.
test_workbook_permissions.workbook_and_user now takes the default_project fixture from conftest.py; dropped 15 lines of duplicated project-lookup logic.
Replaced 7 bare except Exception: pass teardown blocks across test_datasources_crud.py, test_pagination.py, test_projects_admin.py, test_tagging.py with warnings.warn(...), matching the pattern in test_pagination.py's workbook cleanup.
Dropped str(...) wrapping on SAMPLE_DATASOURCE in test_favorites.py and test_jobs.py; bare Path is consistent with the rest of the branch now.
Registered the e2e_admin marker in pyproject.toml matching conftest.py.

Skipped Claude's suggestion to also touch test_workbook_permissions.py:149 — that one's a test-body finally, not a fixture teardown.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a comprehensive end-to-end (E2E) pytest suite that exercises Tableau Server Client (TSC) APIs against a real Tableau server, including a separate set of SiteAdmin-only tests.

Changes:

Introduces multiple new E2E test modules covering CRUD, permissions, exports, jobs, pagination, favorites, metadata, and admin operations.
Adds e2e_admin marker plus a server_admin authenticated fixture and shared project fixtures for consistent publishing targets.
Expands tagging tests to include datasource tagging scenarios and additional space/comma edge cases.

Reviewed changes

Copilot reviewed 15 out of 17 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test_e2e/conftest.py	Adds admin-auth session fixture, shared project selection, and SSL verify configuration for E2E runs.
test_e2e/test_workbooks_crud.py	New workbook CRUD E2E coverage (publish/get/update/populate/download/delete).
test_e2e/test_workbook_permissions.py	New workbook/view permissions E2E coverage using SiteAdmin credentials.
test_e2e/test_views_export.py	New view export E2E coverage (PDF/PNG/CSV).
test_e2e/test_users_groups.py	New SiteAdmin E2E coverage for user/group lifecycle and membership operations.
test_e2e/test_tagging.py	Updates tagging tests; adds datasource tagging and more tag edge-case coverage.
test_e2e/test_site_admin.py	New SiteAdmin E2E coverage for schedules, webhooks, connection updates, freshness policy, project moves, and groups.
test_e2e/test_projects_get_by_path.py	New E2E/admin split tests for projects.get_by_path() behavior.
test_e2e/test_projects_admin.py	New SiteAdmin project CRUD + default permissions E2E coverage.
test_e2e/test_pagination.py	New E2E coverage for TSC.Pager and QuerySet pagination behavior.
test_e2e/test_metadata.py	New E2E smoke tests for Metadata GraphQL queries.
test_e2e/test_jobs.py	New E2E coverage for refresh jobs and wait_for_job() completion paths.
test_e2e/test_favorites.py	New E2E coverage for favorites on workbooks/views/datasources.
test_e2e/test_datasources_crud.py	New datasource CRUD E2E coverage (publish/get/update/connections/tags/download/delete).
pyproject.toml	Registers the new e2e_admin marker.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot review findings on #1823. (1) test_users_groups.py: every user/group name is now suffixed with a uuid via a _unique(prefix) helper. Fixed names caused 409 conflicts when two runs executed against the same site in parallel, or when a prior run crashed before its finally block removed the resource. (2) test_workbook_permissions.py: wrap the delete_permission call in test_update_permissions_appears_on_populate's finally block in try/except, matching the sibling test_delete_permission_removes_rule. Unwrapped cleanup can raise (e.g. rule already gone, transient server error) and mask the original assertion failure with a confusing "delete_permission raised 500" instead of the actual test failure. No production code touched.

Pull request overview

Copilot reviewed 15 out of 17 changed files in this pull request and generated no new comments.

Suppressed comments (2)

test_e2e/conftest.py:80

default_project depends on the server fixture (publisher PAT). This makes e2e_admin tests that need default_project/project_id implicitly require TABLEAU_TOKEN + TABLEAU_TOKEN_NAME, contradicting the admin-test run instructions (SiteAdmin-only). Consider resolving the project using whichever credentials are available (publisher PAT preferred, else SiteAdmin PAT) so admin-only runs can work.

def default_project(server):
    """Return a project to publish into, shared across all test modules.

    Uses TABLEAU_PROJECT env var if set (default: "Default"). Falls back to
    "Personal Work" then the first available project.

test_e2e/test_workbook_permissions.py:29

The permissions fixture creates a fixed username ("tsc-e2e-perm-testuser"). If a prior run crashed before cleanup, or two runs hit the same site, users.add() can 409-conflict and fail the whole module. Use a per-run unique username (e.g., UUID suffix) like the users/groups tests do.

    try:
        user = TSC.UserItem("tsc-e2e-perm-testuser", TSC.UserItem.Roles.Viewer)
        user = server_admin.users.add(user)
    except Exception:

- conftest.py: warn when TABLEAU_VERIFY_SSL=false so a disabled SSL check is loud, not silent. - test_favorites.py::test_favorites_view: skip instead of IndexError if the published workbook has no views. - test_jobs.py::test_workbook_refresh_job_completes: skip on JobFailedException; a missing extract is a precondition, not a bug. - test_site_admin.py: name ScheduleItem kwargs (priority=50/60 vs bare positional ints) so the schedule intent reads at a glance. - test_views_export.py::a_view fixture: skip instead of fail when the workbook has no views. Not addressed here (in-line justifications for the PR replies): - BytesIO as filepath in test_workbook_download_returns_nonempty_bytes: the download() @overload accepts FileObjectW (any writable), so a BytesIO buffer is a valid caller and this test IS the coverage. - _unique() sweep to test_workbooks_crud / test_datasources_crud: those publish with PublishMode.Overwrite so cross-run collisions self-heal; uuid suffixes would defeat orphan cleanup after a crashed run.

Second-pass review sweep — 866049f addresses the remaining Copilot findings:

conftest: warns on TABLEAU_VERIFY_SSL=false so an insecure run isn't silent.
test_favorites.py::test_favorites_view: pytest.skip if workbook.views is empty, so no IndexError.
test_site_admin.py: schedule ScheduleItem(...) calls now use kwargs (priority=50/60) so intent reads at a glance. Held off on a named constant since the two values are only used once each and mean different things — happy to add one if you'd rather see it.
test_jobs.py::test_workbook_refresh_job_completes and test_views_export.py::a_view fixture: switched pytest.fail to pytest.skip for missing-extract / missing-views preconditions. Not a Copilot finding, but same shape as the views[0] guard.

One Copilot finding rejected: BytesIO() as filepath= in test_workbook_download_returns_nonempty_bytes — the workbooks.download() @overload block (workbooks_endpoint.py:400-407) explicitly accepts a FileObjectW (writable file-like), so a BytesIO buffer is a supported caller and this test is the coverage.

jacalata and others added 5 commits July 21, 2026 15:53

test: add WorldIndicators.tdsx asset for datasource e2e tests

966ca5e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

bcantoni mentioned this pull request Jul 23, 2026

feat: consolidate and export public type aliases (HyperAction, FilePath, AddResponse, IDP types) #1825

Open

2 tasks

jacalata and others added 2 commits July 28, 2026 00:21

Merge origin/development into jac/e2e-tests

424c7b4

jacalata requested a lite review from Copilot August 6, 2026 21:18

Copilot AI reviewed Aug 6, 2026

View reviewed changes

Comment thread test_e2e/test_workbooks_crud.py

Comment thread test_e2e/test_workbook_permissions.py Outdated

Comment thread test_e2e/conftest.py Outdated

Comment thread test_e2e/test_users_groups.py Outdated

Comment thread test_e2e/test_favorites.py

Comment thread test_e2e/test_site_admin.py

jacalata requested a lite review from Copilot August 17, 2026 20:01

Copilot started reviewing on behalf of jacalata August 17, 2026 20:02 View session

Copilot AI reviewed Aug 17, 2026

View reviewed changes

test(e2e): add e2e tests for workbooks, datasources, views, pagination, jobs, tagging, projects, favorites, metadata, site-admin by jacalata · Pull Request #1823 · tableau/server-client-python · GitHub

Uh oh!

test(e2e): add e2e tests for workbooks, datasources, views, pagination, jobs, tagging, projects, favorites, metadata, site-admin - #1823

test(e2e): add e2e tests for workbooks, datasources, views, pagination, jobs, tagging, projects, favorites, metadata, site-admin#1823
jacalata wants to merge 10 commits into
developmentfrom
jac/e2e-tests

jacalata commented Jul 22, 2026 •
edited
Loading

Uh oh!

Uh oh!

github-actions Bot commented Jul 22, 2026 •
edited
Loading

Uh oh!

Uh oh!

bcantoni commented Jul 23, 2026 •
edited
Loading

Uh oh!

Uh oh!

bcantoni commented Jul 23, 2026

Uh oh!

jacalata commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

jacalata commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

test(e2e): add e2e tests for workbooks, datasources, views, pagination, jobs, tagging, projects, favorites, metadata, site-admin by jacalata · Pull Request #1823 · tableau/server-client-python · GitHub

Uh oh!

test(e2e): add e2e tests for workbooks, datasources, views, pagination, jobs, tagging, projects, favorites, metadata, site-admin - #1823

Conversation

jacalata commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

github-actions Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bcantoni commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review

Blocking: will fail CI black check

Suggestions (non-blocking)

Other notes

Uh oh!

bcantoni commented Jul 23, 2026

Uh oh!

jacalata commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

jacalata commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jacalata commented Jul 22, 2026 •
edited
Loading

github-actions Bot commented Jul 22, 2026 •
edited
Loading

bcantoni commented Jul 23, 2026 •
edited
Loading