FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

fix: remove redundant MetricsCapture from trace_call by waiho-gumloop · Pull Request #1522 · googleapis/python-spanner · GitHub

This repository was archived by the owner on Jun 8, 2026. It is now read-only.
/ python-spanner Public archive

fix: remove redundant MetricsCapture from trace_call - #1522

Closed
waiho-gumloop wants to merge 2 commits into
googleapis:mainfrom
waiho-gumloop:fix/remove-redundant-metrics-capture-from-trace-call
ConversationCommits (2)ChecksFiles changed
Closed

fix: remove redundant MetricsCapture from trace_call#1522
waiho-gumloop wants to merge 2 commits into
googleapis:mainfrom
waiho-gumloop:fix/remove-redundant-metrics-capture-from-trace-call

Conversation

waiho-gumloop commented Mar 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

trace_call() wraps every Spanner operation with a bare MetricsCapture() that creates a MetricsTracer without project_id or instance_id. Since every caller of trace_call already provides its own MetricsCapture(resource_info) with correct labels, the one inside trace_call is redundant and harmful.

The redundant tracer records operation metrics (via record_operation_completion()) with incomplete resource labels on every operation. Because OpenTelemetry uses cumulative aggregation, these orphan data points persist for the process lifetime and get re-exported every 60 seconds by the PeriodicExportingMetricReader. Cloud Monitoring rejects them with:

INVALID_ARGUMENT: One or more TimeSeries could not be written:
timeSeries[...]: the set of resource labels is incomplete, missing (instance_id)

Root cause

When Python evaluates with trace_call(...) as span, MetricsCapture(resource_info):, the execution order is:

  1. trace_call.__enter__() → creates internal bare MetricsCapture() → tracer_A (no project_id/instance_id)
  2. Caller's MetricsCapture(resource_info).__enter__() → tracer_B (has correct labels, overwrites tracer_A in context var)
  3. Body runs — gRPC calls use tracer_B via MetricsInterceptor
  4. Caller's MetricsCapture.__exit__() → records correct metrics on tracer_B, resets context to tracer_A
  5. trace_call.__exit__() → records metrics on tracer_A with incomplete labels

This creates persistent orphan aggregation buckets in the OpenTelemetry SDK that are re-exported every 60s.

History

  • PR feat: Add Attempt, Operation and GFE Metrics #1302 introduced the metrics system. All MetricsCapture() instances were bare — both the one inside trace_call and the ones at caller sites. The design relied on MetricsInterceptor to populate labels during gRPC calls. At this point, the trace_call MetricsCapture was not redundant.
  • PR feat: implement native asyncio support via Cross-Sync #1509 added the _resource_info property and changed all caller sites from MetricsCapture() to MetricsCapture(self._resource_info) for eager label propagation. However, the bare MetricsCapture() inside trace_call was not removed during this large refactor (225 files changed), making it redundant and harmful.

Fix

Remove the bare MetricsCapture() from trace_call. All ~27 call sites already provide their own MetricsCapture with correct resource labels.

Testing

All existing unit tests pass (46/46). The change only removes the redundant context manager; span/trace behavior is unchanged.

Fixes googleapis/google-cloud-python#16173

fix: remove redundant MetricsCapture from trace_call
bc87c88 
trace_call() wraps every Spanner operation with a bare MetricsCapture()
that creates a MetricsTracer without project_id or instance_id. Since
every caller of trace_call already provides its own MetricsCapture with
resource_info, this inner one is redundant.

The redundant tracer records operation metrics with incomplete resource
labels on every operation. Because OpenTelemetry uses cumulative
aggregation, these orphan data points persist for the process lifetime
and get re-exported every 60 seconds. Cloud Monitoring rejects them
with INVALID_ARGUMENT (missing instance_id), producing repeated error
logs.

Removing the bare MetricsCapture from trace_call eliminates the orphan
metric data points entirely. Callers continue to provide their own
MetricsCapture(resource_info) with correct labels.

Fixes: googleapis#1319
waiho-gumloop requested review from a team as code owners March 25, 2026 02:11
product-auto-label Bot added the size: s Pull request size is small. label Mar 25, 2026
product-auto-label Bot added the api: spanner Issues related to the googleapis/python-spanner API. label Mar 25, 2026

gemini-code-assist Bot commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical issue where a redundant OpenTelemetry MetricsCapture instance within the trace_call function was causing incorrect and incomplete metrics to be exported. By removing this unnecessary MetricsCapture, the system now correctly uses the caller-provided metrics context, resolving INVALID_ARGUMENT errors in Cloud Monitoring and preventing the accumulation of orphan metric data points.

Highlights

  • Redundant MetricsCapture Removed: Eliminated a superfluous MetricsCapture() within trace_call() that was causing incomplete metrics to be exported to Cloud Monitoring, leading to INVALID_ARGUMENT errors.
  • Metric Labeling Issue Resolved: Fixed a bug where an internal MetricsCapture in trace_call would record operation metrics with missing project_id and instance_id labels, persisting as orphan aggregation buckets.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist Bot reviewed Mar 25, 2026
View reviewed changes

gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Code Review

This pull request removes the MetricsCapture context manager from the trace_call function in _opentelemetry_tracing.py. This change discontinues the use of MetricsCapture within the OpenTelemetry tracing context for Spanner client calls. There is no feedback to provide on the changes.

This was referenced Mar 25, 2026
Open
Open

parthea commented Mar 31, 2026

Copy link
Copy Markdown
Contributor

Hi @waiho-gumloop, The code in this repository has moved to https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-spanner. Please could you open a new PR in google-cloud-python?

parthea closed this Mar 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer
gemini-code-assist[bot] gemini-code-assist[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

steotia

Labels

api: spanner Issues related to the googleapis/python-spanner API. size: s Pull request size is small.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Redundant MetricsCapture in trace_call produces orphan metrics with incomplete resource labels

3 participants

Back | FazBrowse Home | New Git URL