feat(server-utils): Map Mastra classifier evaluations to gen_ai.evaluate spans by andreiborza · Pull Request #25044 · getsentry/sentry-javascript · GitHub

What

Mastra classifier evaluations (Jev calls through Mastra's Classifier, its processors and scorers) now show up as gen_ai.evaluate spans with model, provider and token usage. When gen_ai recording is on, they also record the evaluated state, questions and answers.

Why

These calls skip the AI SDK hook, and our Mastra exporter dropped their spans, so they were invisible.

Closes: #25031

JS-3877

size-limit report 📦

Path	Size	% Change	Change
@sentry/browser	29.8 kB	-	-
@sentry/browser - with treeshaking flags	27.92 kB	-	-
@sentry/browser - with treeshaking flags tracing without tracing	27.84 kB	-	-
@sentry/browser (incl. Tracing)	51.85 kB	-	-
@sentry/browser (incl. Tracing + Span Streaming)	51.87 kB	-	-
@sentry/browser (incl. Tracing, Profiling)	54.8 kB	-	-
@sentry/browser (incl. Tracing, Replay)	91.58 kB	-	-
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags	80.45 kB	-	-
@sentry/browser (incl. Tracing, Replay with Canvas)	96.29 kB	-	-
@sentry/browser (incl. Tracing, Replay, Feedback)	109.27 kB	-	-
@sentry/browser (incl. Feedback)	47.32 kB	-	-
@sentry/browser (incl. sendFeedback)	34.84 kB	-	-
@sentry/browser (incl. FeedbackAsync)	39.95 kB	-	-
@sentry/browser (incl. Metrics)	30.82 kB	-	-
@sentry/browser (incl. Logs)	31.11 kB	-	-
@sentry/browser (incl. Metrics & Logs)	31.75 kB	-	-
@sentry/react	31.63 kB	-	-
@sentry/react (incl. Tracing)	54.17 kB	-	-
@sentry/vue	37.84 kB	-	-
@sentry/vue (incl. Tracing)	54.78 kB	-	-
@sentry/svelte	29.83 kB	-	-
@sentry/remix (Remix 3 client bundle)	56.83 kB	-	-
CDN Bundle	31.53 kB	-	-
CDN Bundle (incl. Tracing)	52.34 kB	-	-
CDN Bundle (incl. Logs, Metrics)	33.72 kB	-	-
CDN Bundle (incl. Tracing, Logs, Metrics)	54.3 kB	-	-
CDN Bundle (incl. Replay, Logs, Metrics)	74.65 kB	-	-
CDN Bundle (incl. Tracing, Replay)	90.01 kB	-	-
CDN Bundle (incl. Tracing, Replay, Logs, Metrics)	91.96 kB	-	-
CDN Bundle (incl. Tracing, Replay, Feedback)	96.17 kB	-	-
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics)	98.15 kB	-	-
CDN Bundle - uncompressed	92.92 kB	-	-
CDN Bundle (incl. Tracing) - uncompressed	155.46 kB	-	-
CDN Bundle (incl. Logs, Metrics) - uncompressed	99.46 kB	-	-
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed	161.41 kB	-	-
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed	229.5 kB	-	-
CDN Bundle (incl. Tracing, Replay) - uncompressed	275.63 kB	-	-
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed	281.57 kB	-	-
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed	289.33 kB	-	-
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed	295.26 kB	-	-
@sentry/nextjs (client)	56.53 kB	-	-
@sentry/sveltekit (client)	52.23 kB	-	-
@sentry/core/server	40.86 kB	-	-
@sentry/core/browser	13.71 kB	-	-
@sentry/node	146.94 kB	+0.34%	+492 B 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection)	83.41 kB	+0.04%	+31 B 🔺
@sentry/node - without tracing	93.79 kB	+0.06%	+47 B 🔺
@sentry/node - without channel injection	125.11 kB	+0.4%	+490 B 🔺
@sentry/aws-serverless	102 kB	+0.04%	+37 B 🔺
@sentry/cloudflare (withSentry) - minified	209.77 kB	+0.04%	+64 B 🔺
@sentry/cloudflare (withSentry)	520.1 kB	+0.04%	+156 B 🔺
@sentry/nextjs/cloudflare (withSentry) - minified	227.44 kB	+0.03%	+64 B 🔺

View base workflow run

Stale Bugbot comment from a previous run.

not specific to this PR, but I think we have an object that looks (almost) exactly like this in most or all of our integrations. we should probably refactor that at some point too

Agreed, filed JS-3892 to add one shared orchestrion payload type.

m/h: it seems mastra default-configures some filters that run on span events, for instance a sensitive data filter: https://mastra.ai/reference/observability/tracing/processors/sensitive-data-filter

iiuc this message object that is copied to the outputs here is stored from the original input so these filters are probably not applied yet? I am concerned this could leak sensitive data

Good catch. I ended up running their processors over the data before storing it in f12213b.

I was debating wether we should reconstruct the data at all, but figured we are in the o11y business and it makes sense to do so. I'll file an issue upstream.

Stale Bugbot comment from a previous run.

ja passt

Cursor Bugbot has reviewed your changes and found 1 potential issue.

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c3f2fa5. Configure here.

…ate spans Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

## What Adds a node-mastra e2e test that runs a real Jev evaluation through a Mastra `Classifier` (via OpenRouter) and checks the `gen_ai.evaluate` span. ## Why Covers the classifier instrumentation from #25044 against the real provider, not only a mock model.

cursor Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread dev-packages/node-integration-tests/suites/tracing/mastra/test.ts Outdated

andreiborza force-pushed the ab/mastra-classifier-evaluate-spans branch from d3c4b79 to 1b15020 Compare October 7, 2026 08:15

andreiborza added this pull request to stack #25116 October 7, 2026 08:16

andreiborza mentioned this pull request Oct 7, 2026

test(e2e): Add Mastra Jev classifier test to node-mastra #25115

Merged

andreiborza marked this pull request as ready for review October 7, 2026 08:35

andreiborza requested a review from a team as a code owner October 7, 2026 08:35

andreiborza requested review from RulaKhaled, chargome, isaacs, mydea and nicohrubec and removed request for a team, isaacs and mydea October 7, 2026 08:35

andreiborza force-pushed the ab/mastra-classifier-evaluate-spans branch from 1b15020 to 4c71676 Compare October 7, 2026 09:11

sentry Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread packages/server-utils/src/integrations/mastra-classifier.ts

cursor Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread packages/server-utils/src/integrations/mastra-classifier.ts

andreiborza force-pushed the ab/mastra-classifier-evaluate-spans branch from 549d28a to 01f4a44 Compare October 7, 2026 09:40

andreiborza requested a review from JPeer264 October 8, 2026 08:09

nicohrubec reviewed Oct 8, 2026

View reviewed changes

JPeer264 approved these changes Oct 8, 2026

View reviewed changes

cursor Bot reviewed Oct 8, 2026

View reviewed changes

Comment thread packages/server-utils/src/integrations/mastra-classifier.ts

andreiborza requested a review from nicohrubec October 8, 2026 11:04

andreiborza mentioned this pull request Oct 8, 2026

Add a shared channel context type for orchestrion integrations #25173

Open

nicohrubec approved these changes Oct 8, 2026

View reviewed changes

andreiborza force-pushed the ab/mastra-classifier-evaluate-spans branch from 15ef1ec to c3f2fa5 Compare October 8, 2026 11:45

cursor Bot reviewed Oct 8, 2026

View reviewed changes

Comment thread packages/server-utils/src/ai/mastra/index.ts

feat(server-utils): Map Mastra classifier evaluations to gen_ai.evalu…

2d08f72

…ate spans Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

andreiborza added 11 commits October 8, 2026 14:16

Extract Mastra span identifier helper

6dc33ca

Use GEN_AI_EVALUATE op from conventions

c110bf2

Use conventions constants in Mastra exporter tests

8824cd6

Record Mastra classifier inputs and answers

4495ee8

Use conventions constants in Mastra integration tests

35dd163

Test failed Mastra classifier evaluations

2b9662c

Follow exporter recording options for classifier data

01cf050

Run classifier data through Mastra span output processors

9e56d15

Resolve Mastra recording options once per event

3dd55f2

Move evaluation message helpers to shared AI utils

5599733

Assert classifier tests on streamed spans

17acb4b

andreiborza force-pushed the ab/mastra-classifier-evaluate-spans branch from c3f2fa5 to 17acb4b Compare October 8, 2026 12:16

andreiborza merged commit 5bb9a68 into develop Oct 8, 2026
531 of 533 checks passed

andreiborza deleted the ab/mastra-classifier-evaluate-spans branch October 8, 2026 12:50

sentry-release-bot Bot mentioned this pull request Oct 8, 2026

publish: getsentry/sentry-javascript@11.6.0 getsentry/publish#9809

Closed

47 tasks

github-actions Bot mentioned this pull request Oct 8, 2026

Map Mastra classifier_evaluation spans to gen_ai.evaluate #25031

Closed

+                  safeChannelCallback(() => {
+                    const call = calls.get(message as object);
+                    if (call) {
+                      finishClassifierEvaluation(call, message as ClassifierEvaluateChannelContext);

+              import { safeChannelCallback } from '../tracing-channel';
+              import * as diagnosticsChannel from '../utils/diagnosticsChannel';
+              interface ClassifierEvaluateChannelContext {

feat(server-utils): Map Mastra classifier evaluations to gen_ai.evaluate spans by andreiborza · Pull Request #25044 · getsentry/sentry-javascript · GitHub

Repository navigation

feat(server-utils): Map Mastra classifier evaluations to gen_ai.evaluate spans - #25044

feat(server-utils): Map Mastra classifier evaluations to gen_ai.evaluate spans#25044andreiborza merged 12 commits intodevelopgetsentry/sentry-javascript:developfrom ab/mastra-classifier-evaluate-spansgetsentry/sentry-javascript:ab/mastra-classifier-evaluate-spansCopy head branch name to clipboard

Conversation

andreiborza commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Uh oh!

linear-code Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

size-limit report 📦

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nicohrubec Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

andreiborza Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nicohrubec Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

andreiborza Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nicohrubec left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat(server-utils): Map Mastra classifier evaluations to gen_ai.evaluate spans#25044
andreiborza merged 12 commits into
developfrom
ab/mastra-classifier-evaluate-spans

andreiborza commented Oct 5, 2026 •
edited
Loading

github-actions Bot commented Oct 5, 2026 •
edited
Loading

cursor Bot left a comment •
edited
Loading

cursor Bot left a comment •
edited
Loading

andreiborza Oct 8, 2026 •
edited
Loading

cursor Bot left a comment •
edited
Loading