Beyond rate limiting — govern rate, concurrency, and cost, provably. Meter what your LLM spends. Prove what your fleet admits. Counting requests is the easy 10%; ThrottleKit governs the three axes a real request must clear — rate, concurrency, and cost — each behind a bound you can prove.

Two engines do the hard part. GALE (provable distributed leasing) holds global overshoot to a fleet-size-independent bound, machine-checked in TLA⁺. TALE (token-budget escrow) meters an LLM's output tokens — known only as they stream — and bounds them anyway. All on one small core: a 169 ns in-process check (5.9M ops/s, effectively allocation-free), zero runtime dependencies, proven bit-identical across six stores.

Most limiters just count requests — fine in one process, "probably fine" across a fleet, and silent on the two axes that actually cost you money: concurrency and tokens. ThrottleKit states its bounds, and proves them.

And the bound is only the start. GALE and TALE ship as real features, not heuristics — window-coupled leasing, adaptive lease sizing, weighted-fair escrow, distributed adaptive concurrency, unified rate × concurrency × cost admission, and an LLM token-budget stack — each a checked guarantee, every one byte-identically verified. See how GALE & TALE work →

Site: throttlekit.in · Docs: Wiki · Benchmarks · Stability · Design · Component design · Formal model · Scoreboard · Changelog

npm i throttlekit

Zero runtime dependencies. Peer deps are optional — install only what your stores and adapters use (ioredis/node-redis for throttlekit/redis, pg for throttlekit/postgres, express, …). The Web fetch adapter needs none (Node 18+, Cloudflare, Deno, Bun).

In-memory, no infrastructure — and a real synchronous fast path:

import { rateLimit, gcra } from "throttlekit";

const limiter = rateLimit({
  strategy: gcra({ limit: 100, periodMs: 60_000, burst: 20 }), // 100/min, instantaneous burst 20
});

const decision = limiter.checkSync(userId); // sync + allocation-free — or `await limiter.check(userId)`
if (!decision.allowed) {
  throw new Error(`rate limited; retry in ${decision.retryAfterMs}ms`);
}

Every check returns an immutable Decision — { allowed, limit, remaining, resetAt, retryAfterMs }. Read state without spending it via limiter.peek(key) / limiter.forecast(key).

Going distributed is the same algorithm with a store and a trade-off — lease a batch of credits in one round trip, then serve them locally at in-process speed:

import { twoTier, gcra } from "throttlekit";
import { RedisStore } from "throttlekit/redis";

const limiter = twoTier({
  strategy: gcra({ limit: 10_000, periodMs: 60_000, burst: 500 }),
  l2: new RedisStore({ client }),               // or Postgres, DynamoDB, Deno KV, Cloudflare
  mode: "leased",                               // "strict" | "cached-deny" | "leased"
  lease: { batch: 50, windowCoupled: true },    // ~1 round trip / 50 requests; admitted ≤ Limit, any fleet size
});

• A proven overshoot bound, independent of fleet size. Two-tier leasing is model-checked in TLA⁺/TLC: worst-case global admissions are exactly Limit + N·(Batch−1) (tight by counterexample), and windowCoupled collapses that to exactly Limit — no matter how many nodes, and across regions too (federate(...), bound independent of region count K). The checker re-runs in CI. Most limiters can't state a bound at all.
• Sub-microsecond synchronous checks. checkSync returns a complete decision in 169 ns/op (5.9M ops/s), effectively allocation-free — a true sync API, uncommon among JS limiters, for hot paths that shouldn't pay for an await. Benchmarks →
• One algorithm, six backends, proven bit-identical. The same GCRA (or token-bucket, sliding-window, …) runs in memory, on Redis (one atomic EVALSHA), and on Postgres (advisory-lock transaction — no Redis needed). A dual-path conformance suite — including a 200-way concurrent read-modify-write — proves the JavaScript and Lua decisions agree, so local and distributed limiters can't silently drift.
• Two engines, shipped as features — not heuristics. GALE (provable distributed leasing) and TALE (LLM token-budget escrow) land as real, byte-identically-verified APIs: window-coupled leasing, weighted-fair escrow, adaptive lease sizing, unified rate × concurrency × cost admission, distributed adaptive concurrency, and the cost-axis token-budget stack.
• Batteries included, dependencies not. 24 entry points — 8 strategies, 6 storage backends, 13 framework/transport adapters — and zero runtime dependencies. First-class types, ESM + CJS, tree-shakeable subpaths.
• Honest about where it loses. Every benchmark is reproducible on your hardware, including the cases an incumbent wins. See BENCH.md and SCOREBOARD.md.

The incumbents are good at what they do — this is what ThrottleKit adds on top. Every row is a shipped, tested ThrottleKit feature; the comparison reflects each library's documented capabilities.

This table is about distributed-correctness guarantees and breadth — the benchmarks (incl. the rows an incumbent wins) are reproducible on your hardware: BENCH.md.

In-process, single hot key, Node 24 / AMD Ryzen AI 9 HX 370 (full methodology, caveats, and head-to-head in BENCH.md):

The honest head-to-head (npm run bench:compare): on Redis, level with rate-limiter-flexible (both one atomic Lua round trip) with a tighter tail; the async in-memory GCRA path edges past it (301 vs 331 ns) while computing a full Decision; a single Postgres check trails a one-statement upsert by design, but twoTier(leased) turns that into a ~35× throughput win under load.

gcra (default — tiny state, smooth pacing, controlled bursts) · tokenBucket · fixedWindow · slidingWindow · slidingWindowLog · leakyBucket (traffic shaping) · adaptiveConcurrency (backpressure when the right rate is unknown) · quota (billing-period budgets: calendar-month/-week/-day, fixed, rolling — leap-year-correct). → Strategies

Express · Fastify · Koa · Hono · Next · NestJS · SvelteKit · Remix · Elysia · Web fetch (edge) · AWS Lambda · tRPC · gRPC — each its own subpath sharing one options surface and standards headers, plus createEnforcer() for anything else (queues, job runners, custom protocols). Serverless/edge/RPC bindings are dependency-free.

import { expressRateLimit } from "throttlekit/express"; // + /fastify /koa /hono /next /nest /sveltekit /remix /elysia
import { withRateLimit } from "throttlekit/fetch";       // Web fetch — Cloudflare / Deno / Bun / Next edge

→ Frameworks & the edge

Front a distributed store (L2) with a local in-process tier (L1) and choose the trade-off:

leased trades exactness for throughput with an overshoot you choose: carryover gives admitted ≤ Limit + N·(Batch−1); windowCoupled: true expires credits at the L2 window so admitted ≤ Limit, independent of fleet size N. Set lease.adaptive and each key's batch is sized online at the EOQ optimum (GALE Pillar 2), bound untouched.

Formally verified, not claimed. A TLA⁺ spec is model-checked with TLC (overshoot exactly Limit + N·(Batch−1), tight by counterexample); window-coupling tightens it to exactly Limit, and a Java-free exhaustive checker reproduces both in CI. Details in docs/FORMAL-MODEL.md.

When a store goes down: apply rejects (never silently allows/denies), your fail: "open" | "closed" policy decides, no write lands partially, durable backends never lose committed counts, and twoTier(leased) keeps serving local credits through a brief L2 blip. Full matrix: docs/FAILURE-MODES.md. → Distributed & provable

Primitives that sit upstream of per-key limiters — overload, fairness, and cost:

• adaptiveThrottle — Google-SRE client-side load-shedding from a backend's recent accept rate.
• weightedMaxMin / weightedFairShare — exact, work-conserving weighted max-min sharing so a greedy tenant can't starve the rest (GALE's Weighted Fair Escrow, machine-checked); federatedWeightedFairEscrow lifts it to a global guarantee across regions — held within one process, or across a fleet of separate region processes via the store-backed RedisRegionFairPool (the Lua grant is verified byte-identical to the in-process pool).
• tokenBudget / distributedTokenBudget — a streaming token-budget meter for post-hoc costs (LLM output tokens, known only as they stream): overshoot bounded by debit granularity — Δ = 0 per token, independent of concurrency (TALE Layer 1).
• learnedReservation / predictiveReservation — pace LLM admission over a budget with an online newsvendor learner (O(√T) regret); the predictive variant blends an output-length hint with unconditional safety (the meter holds the bound, not the prediction).
• unifiedAdmission — compose rate ∧ concurrency ∧ cost into one decision, sequential or atomic Lua-fused; opt into a revenue-management bid-price filter (policy: "joint-lp").
• sketchRateLimit — cap an unbounded key universe in ~7.4 KB with a Count-Min Sketch that provably never over-admits.

→ Overload, fairness & DDoS

Changing a limit in production is usually a blind bet — ship it and watch the logs. throttlekit/policy makes it a terraform plan for rate / cost limits: replay your own recorded traffic against a candidate policy and read the exact, per-policy, per-key allow↔deny diff before you deploy.

import { recordLimiter } from "throttlekit/testkit";
import { policy, policySet, corpusFromRecordings, plan, renderPlan, assertPlanAcceptable } from "throttlekit/policy";

// record real traffic against today's limiter…
const rec = recordLimiter({ strategy: "fixedWindow", limit: 3, windowMs: 1000 });
for (let i = 0; i < 6; i++) rec.limiter.checkSync("tenant-a");

// …then ask what tightening to limit 2 would have done — before shipping it
const current   = policySet([policy("api", { strategy: "fixedWindow", limit: 3, windowMs: 1000 })]);
const candidate = policySet([policy("api", { strategy: "fixedWindow", limit: 2, windowMs: 1000 })]);
const result = plan(current, candidate, corpusFromRecordings({ api: rec }));

console.log(renderPlan(result));                  // "api: 1 allow→deny, 0 deny→allow over 6 arrival(s)…"
assertPlanAcceptable(result, { maxAllowToDeny: 0 }); // throws in CI — the change would 429 a live request

The baseline is your current policy replayed cold over the recorded arrival timing — not a guess, and never a comparison against a warm production node's exact decisions (a cold replay can't reproduce those). Leaf rate + cost limiters diff exactly; a concurrency / escrow axis is reported not-replayable (observe it live), never faked. Deterministic, and built entirely on throttlekit/testkit — no new moving parts.

ThrottleKit Lens is a built-in, zero-dependency monitoring dashboard that runs right in your terminal — throttlekit-server --config x.yaml --tui, alongside gRPC, with no browser and no metrics backend. It gives every policy the full ops board across eight tabbed views — Overview · Latency · Fairness · Capacity · Guarantee · Cost Room · Replay · Plan — plus the one view no other rate-limiter dashboard renders: live binding-axis attribution. Because unifiedAdmission composes rate × concurrency × cost in one decision, the Lens shows which axis is throttling each key right now — rate, concurrency, cost, or the joint-LP policy lane.

throttlekit-server --config .throttlekit.yaml --tui
#  → gRPC on :50051  +  ThrottleKit Lens in your terminal (q quit · 1-8/Tab views · p pause)

The Lens watches the server's decisions, so it works for Python / Go / any-language clients too. Need that state remotely, or from another language? It's also a read-only gRPC service — the Monitor door (throttlekit.v1.Monitor: GetSnapshot + Watch) — with a Prometheus /metrics endpoint and standard gRPC health, on by default and loopback-bound until you set a secret. A TUI owns the terminal, so the Lens is opt-in (--tui) and needs an interactive TTY; for fully headless monitoring, emit OpenTelemetry → Grafana, including throttlekit.denies_by_axis{lane} (the deliberate axis escape hatch from 1.2.0). → Monitoring

The old browser-based throttlekit-lens npm package is deprecated — monitoring now lives in the terminal (--tui) and over the Monitor door.

Reach the limiter from non-Node services without re-implementing a single algorithm: the Node core stays the only thing that computes a decision, and every other surface is a thin pipe conformance-checked against one set of language-neutral golden vectors.

• Service door — throttlekit-server runs the core and answers a small gRPC contract (throttlekit.proto); any language becomes a trivial stub, and a denial is a normal decision, never an RPC error. Configure a policy as federated: / fleetBudget: / distributedConcurrency: / federatedFairEscrow: and every client — Node, Python, edge — gets globally-coordinated, fleet-wide decisions over the existing RPCs, with no client change.
• Fleet lease (the scale ceiling) — a very high-throughput client leases a chunk of a global budget from the service's additive Fleet.Reserve door and spends it locally with the core LeaseSpender (throttlekit/twotier), round-tripping only to refresh. The server stays the one oracle — it sizes the grant; the client's local spend is a verbatim port of the leased-L1 path, pinned byte-for-byte by golden lease vectors.
• Monitor door — the server's read-only Monitor gRPC service (+ Prometheus /metrics) makes the same live operational state ThrottleKit Lens renders in the terminal readable remotely, from any language.
• Direct door — a client runs the core's own vendored Lua against the same Redis your Node fleet uses (one hop, no extra service).
• throttlekit-py reaches every axis from Python — rate (check), cost (debit), two-tier leased (check, transparently), and concurrency + unified admission (admit, with crash-safe leases) — plus the Fleet lease and Monitor doors — and its RedisBackend replays the full golden vectors through real Redis to reproduce this core bit-for-bit.

The proto is the stable polyglot contract; the raw Lua wire is behavior-locked but deliberately not frozen. The full walkthrough is the Polyglot & Python wiki page; design + decision records live in research/polyglot/DESIGN.md.

Dual-path conformance (JS ≡ Lua over thousands of generated timelines) + shrinkable fast-check property passes, atomicity tests (N concurrent checks at limit K ⇒ exactly K allowed), the TLA⁺/TLC model re-checked by an exhaustive JS checker in CI, and federation BFS-twin coverage. All time-dependent tests use ManualClock, so the 1,500+ test suite is deterministic; CI is green across Node 20/22/24, and a bench-regression gate blocks hot-path slowdowns.

ThrottleKit is 1.0 — the public API is frozen under SemVer. STABILITY.md is the promise: the stable core (algorithms, stores, adapters, federation, unified-admission core) only grows additively, the experimental frontier (the joint-LP policy, distributed-concurrency tuning knobs, the learned escrow/sketch layer) is opt-in and carved out, and the freeze is mechanically enforced (type-level surface tests + attw + publint in CI). MIT-licensed, developed in the open.