perf: add preloadSchemas() and warm wire schemas eagerly on workerd - #2483
perf: add preloadSchemas() and warm wire schemas eagerly on workerd#2483felixweinberger merged 1 commit into
Conversation
Sorry, something went wrong.
🦋 Changeset detectedLatest commit: fca3843 The changes in this PR will be included in the next version bump. This PR includes changesets to release 9 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Sorry, something went wrong.
|
@modelcontextprotocol/client
npm i https://pkg.pr.new/@modelcontextprotocol/client@2483
npm i https://pkg.pr.new/@modelcontextprotocol/codemod@2483
npm i https://pkg.pr.new/@modelcontextprotocol/core@2483
npm i https://pkg.pr.new/@modelcontextprotocol/server@2483
npm i https://pkg.pr.new/@modelcontextprotocol/server-legacy@2483
npm i https://pkg.pr.new/@modelcontextprotocol/express@2483
npm i https://pkg.pr.new/@modelcontextprotocol/fastify@2483
npm i https://pkg.pr.new/@modelcontextprotocol/hono@2483
npm i https://pkg.pr.new/@modelcontextprotocol/node@2483 commit: fca3843 |
Sorry, something went wrong.
The wire schemas are built lazily since the memoized-factory change, which is the right default on process-per-invocation runtimes where module evaluation is boot latency. On isolate-based serverless platforms the accounting inverts: module scope evaluates during deployment/isolate warm-up outside any request's billed CPU, so lazy construction lands inside the first request each fresh isolate serves - observable in production as a per-request CPU regression with a fresh-isolate-first-request signature. - preloadSchemas() (core-internal wire/preload.ts): synchronous, idempotent; forces both era schema factories and every memoized lookup layer above them (the 2025 registry maps, the 2026 in-band input-request maps, the 2026 wire-result wrappers), so post-preload validation constructs nothing. Exported as public API from the client and server package roots for platforms that bill request CPU but not module evaluation. - The workerd shims call it at module scope, so Cloudflare Workers deployments get eager construction automatically. Node and browser builds stay lazy; the server package gains a dedicated browser shim (its browser export condition previously reused the workerd shim, which would have leaked the eager call into browser bundles). - Dist-level pins in both packages: the workerd entries must carry the module-scope call, node/browser entries must not, and the shim must import preloadSchemas from the same shared chunk as the root entry (a duplicated definition would warm a twin module graph and leave the real one cold). - The server dist suites now share a single-flight build helper so two test files cannot race tsdown's clean step on a cold checkout; the client helper's dist sentinels now cover the shim entries. Verified against packed tarballs in a scratch consumer: a bare import constructs nothing (first preloadSchemas call does the work, second is free); a wrangler bundle resolves shimsWorkerd and contains exactly one module-scope call; an esbuild browser bundle resolves shimsBrowser and contains none; the Cloudflare Workers integration test stays green.
| // Explicit opt-in to eager wire-schema construction, for platforms that bill | ||
| // request CPU but not module evaluation (isolate-based edge/serverless | ||
| // runtimes). The package's workerd build calls it automatically at module | ||
| // scope; other builds stay lazy unless the application calls it itself. | ||
| export { preloadSchemas } from '@modelcontextprotocol/core-internal'; | ||
|
|
There was a problem hiding this comment.
🟡 The new public API preloadSchemas() (exported from both @modelcontextprotocol/client and @modelcontextprotocol/server with a minor changeset) has no prose documentation — no docs/**/*.md file or README mentions it, only the JSDoc and the changeset. Since the PR description itself names a module-scope preloadSchemas() call as "the intended opt-in" for Lambda-style billed-CPU serverless Node deployments, a short section in docs/serving/web-standard.md (or an advanced/performance page) covering the lazy default, the automatic workerd warm-up, and the manual opt-in would make that guidance discoverable.
Extended reasoning...What's missing
This PR adds preloadSchemas() as new public API on both the client and server packages (packages/client/src/index.ts:111, packages/server/src/index.ts:106), ships it with a minor changeset (.changeset/workerd-schema-preload.md), and checks the "I have added or updated documentation as needed" box — but zero docs/**/*.md files are touched. A grep for preloadSchemas across docs/, examples/, and every package README returns no hits; the only prose about the API lives in the JSDoc on packages/core-internal/src/wire/preload.ts and in the changeset, neither of which is guide documentation. The repo review conventions explicitly require, for a new feature: verify prose documentation is added (not just JSDoc).
Why it matters more than usual here
The intended usage is discoverability-sensitive in a way most new exports aren't:
-
Serverless Node (Lambda-style) users get no automatic behavior. The PR description states these deployments default to lazy because Node resolution cannot distinguish them from a laptop, and that calling preloadSchemas() at module scope is "the intended opt-in for those deployments." That guidance currently exists only in the PR description and JSDoc — the exact audience it targets (someone noticing a p99 CPU regression on first-invocation Lambdas after upgrading) will never find it, because a user who doesn't already know the function's name can't discover its JSDoc.
-
Workers users can't learn about the auto-warm. docs/serving/web-standard.md is the Cloudflare Workers/Deno/Bun-facing serving guide, and it says nothing about the lazy-by-default schema construction the sibling changesets introduce or the automatic workerd module-scope warm-up this PR adds. A Workers consumer reading the guide has no way to know their bundle behaves differently from Node.
Concrete walkthrough
Consider a team running an MCP server on AWS Lambda with per-request CPU billing. They upgrade past this release: #2476's laziness moves both eras' schema-graph construction from (cheap, amortized) init into the first billed invocation of every cold start — the exact tail-heavy regression the Workers consumer reported and this PR fixes for workerd only. The Lambda team sees the p99 regression, checks the docs — nothing about lazy schemas or preloadSchemas() exists anywhere under docs/ — and has no discoverable path to the one-line fix (preloadSchemas(); at module scope) that this PR explicitly built for them.
Why nothing prevents it
The dist-level pin tests (workerdSchemaPreload.test.ts in both packages) rigorously guard the runtime behavior — module-scope call present in workerd shims, absent from node/browser, single shared memo chunk — but there is no doc-coverage check, so the docs gap merges silently.
How to fix
Add a short section to docs/serving/web-standard.md (or an advanced/performance page) covering three things: (a) wire schemas are constructed lazily by default (right trade for process-per-invocation runtimes), (b) the Cloudflare Workers builds call preloadSchemas() automatically at module scope so no action is needed there, and (c) deployments on platforms that bill per-request CPU but not module evaluation (e.g. Lambda-style serverless Node) should opt in by calling preloadSchemas() at module scope themselves. The JSDoc example in packages/core-internal/src/wire/preload.ts can be reused nearly verbatim.
Severity
Nit: nothing breaks at runtime if this merges as-is — the API works, the workerd auto-warm needs no docs to function, and Node/browser behavior is unchanged. This is a checklist-mandated discoverability gap, not a blocking defect.
Sorry, something went wrong.
Stacked on #2458 / #2476 / #2477 — the diff below includes them until they merge; the commit to review is the tip (60d7f1f). Will rebase onto main and mark ready once the three land.
Motivation and Context
#2476 defers era wire-schema construction from import time to first validation. On long-lived Node processes that is a pure win. On isolate platforms with per-request CPU accounting (Cloudflare Workers), it moves the one-time construction cost out of module evaluation — which is unbilled and amortized by isolate pre-warming — into the first billed request on every fresh isolate. A production Workers consumer measured the effect as a tail-heavy per-request CPU regression (p99-dominant, matching the fresh-isolate-first-request signature).
The fix makes eagerness platform-conditional and consumer-controllable:
How Has This Been Tested?
Breaking Changes
None. New exported function; no existing API or behavior changes on Node/browser. Workers consumers regain the pre-#2476 CPU accounting automatically.
Types of changes
Checklist
Additional context