FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat(ratelimit): credit counting at any budget scope by jscaltreto · Pull Request #996 · erpc/erpc · GitHub

/ erpc Public

feat(ratelimit): credit counting at any budget scope - #996

Open
jscaltreto wants to merge 2 commits into
erpc:mainfrom
jscaltreto:feat/ratelimit-weighted-credit-budgets
Open

feat(ratelimit): credit counting at any budget scope#996
jscaltreto wants to merge 2 commits into
erpc:mainfrom
jscaltreto:feat/ratelimit-weighted-credit-budgets

Conversation

jscaltreto commented Jul 13, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Rewritten. This PR predated #1016 and originally shipped a parallel weighting mechanism (weighted: true + methodCosts + a per-descriptor HitsAddend). Now that vendor credit accounting is merged, that design duplicated its vocabulary, its resolution logic and its addend for no gain. This is a rebuild on top of it.

Stacked on #1058#1059. Review those first; this branch contains both commits. They are independent bugfixes and stand on their own.

Problem

Credit-weighted rate limiting only exists at upstream scope, where an upstream on rateLimitCountMode: credit prices each call from its vendor's credit-unit table.

A project-, network- or auth-scoped budget has no vendor, so it can only count requests — a range scan and a chain-ID lookup cost the same. The operator is then forced to size a per-IP cap for the worst case, which throttles cheap traffic in order to guard against expensive traffic. There is no way to say "this client may spend N units of work per minute, however they choose to spend it".

Change

Let a budget price its own methods.

rateLimiters:
  budgets:
    - id: frontend
      creditUnits:
        "*": 20
        eth_chainId: 0        # exempt
        eth_call: 26
        eth_getLogs: 60
      rules:
        - method: "*"
          countMode: credit   # the wallet: one pool shared by all methods
          maxCount: 20000
          period: minute
          perIP: true
        - method: eth_chainId # exempt from the wallet, still capped
          countMode: request
          maxCount: 100
          period: second
          perIP: true

rules[].countMode: credit charges the method's cost from the budget's creditUnits table instead of a flat 1, turning maxCount into a compute allowance. Resolution reuses ResolveCreditUnits, so precedence is the one operators already know from upstream.creditUnits: exact method, then "*", then 1. A method priced 0 is exempt.

countMode is per rule, not per budget. That is what makes exemption usable: price a method 0 to keep it out of the wallet, and cover it with a countMode: request rule so it still has a ceiling. A rule that states its mode keeps it even when the caller counts credits; only an unset mode inherits, so existing upstream credit budgets behave exactly as before.

Design decisions

No built-in cost table. The original version shipped DefaultRateLimitMethodCosts merged in SetDefaults. Dropping it is what allows ResolveCreditUnits to be reused unchanged: with defaults == nil the precedence collapses to override[method] → override["*"] → 1. With a defaults table it does not, because ResolveCreditUnits ranks defaults[method] above the operator's own "*" catch-all — so creditUnits: {"*": 1, eth_getLogs: 50} would have its global policy silently defeated by a table the operator never wrote and cannot see. A built-in table is also an unpinnable behaviour surface: a patch release adjusting it would change every budget's effective capacity with nothing visible in the marshalled config. The docs carry a starting table to copy instead.

This is also the weaker hypothesis in the sense of CLAUDE.md's razor — pricing is operator data, not something eRPC should commit to on their behalf.

The two cost sources never mix. An upstream on rateLimitCountMode: credit prices from its vendor; pointing it at a budget that also defines creditUnits or rule-level countMode is rejected at validation rather than resolved by silent precedence. Two plausible cost sources for one counter is the kind of ambiguity that surfaces six months later as "why is my wallet draining faster than my table says".

Exempt methods skip the rule rather than sending a 0 addend. Worth flagging, because the original version of this PR justified the same behaviour with the wrong reason. Check-only-at-zero is a property of the per-descriptor addend, which this rebuild no longer uses; the request-level addend coerces 0 to 1 (GetHitsAddends), so a 0 there would charge a full hit. The consequence is documented as a footgun: a 0-priced method keeps being served after the pool is empty, which is why the exemption pattern above pairs it with a flat rule.

countMode joins the rule identity and the metric labels (rate_limiter_budget_max_count), so rules differing only in mode cannot share a counter or overwrite each other's gauge. Today they cannot collide anyway — a credit rule drops the method entry a request rule keeps — but that non-collision is incidental and holds only while method-omission is the sole descriptor difference between the modes.

Testing

Nine behaviour tests covering: pricing from the budget table with pooling, zero-cost exemption with a flat rule still enforcing, an explicit countMode: request rule surviving a credit caller, nested wildcard pools, #1016 inheritance compatibility, vendor weight winning over the budget table, no-table behaviour as a cross-method request cap, and "*": 0 exemption. Plus config round-trip and rejection cases.

The regression guards were checked for teeth rather than assumed. Restoring the naive permit.Mode || rule.CountMode resolution fails the mixed-budget test immediately:

--- FAIL: TestCreditRule_ExplicitRequestModeSurvivesCreditCaller
    request 1 is within the flat ceiling of 2

That is the whole failure mode in one line: a credit-mode upstream attaching to the budget would force every rule credit, so the flat per-method caps silently stop being flat.

Verified green: upstream, common, auth, erpc. gofmt and go vet clean. Benchmarks show identical allocation counts against the #1059 baseline.

Notes

One item deliberately not in this PR: the auto-tuner is enabled by default for any upstream with a budget and clamps maxCount to maxBudget (default 100000), so on main today a rateLimitCountMode: credit upstream with a 300M CU plan is reduced to 100k on the first healthy adjustment tick. That is a pre-existing issue with the merged feature rather than anything this PR introduces, so it belongs in its own fix.

subweave Bot commented Jul 13, 2026
edited
Loading

Copy link
Copy Markdown

This looks like a really powerful way to manage budgets! 🤯
Explore here →

Carefully crafted by Subweave · 🧶 used ~709k LLM tokens

jscaltreto marked this pull request as draft August 7, 2026 23:54
jscaltreto force-pushed the feat/ratelimit-weighted-credit-budgets branch from 048ee57 to 00329c6 Compare August 10, 2026 15:49
jscaltreto changed the title feat(ratelimit): weighted (credit-based) rate limit rules feat(ratelimit): credit counting at any budget scope Aug 10, 2026
jscaltreto marked this pull request as ready for review August 10, 2026 15:55
An upstream on `rateLimitCountMode: credit` charges the request's vendor
credit-unit estimate as the hit weight, so a budget of
`maxCount: 300000000, period: month` reads as the provider plan it is
meant to model: 300M CU/month. It did not behave that way. The `method`
descriptor entry was always present, so the budget was enforced once per
method and the real ceiling was 300M CU multiplied by the number of
distinct methods in traffic.

Per-method partitioning is right for request counting, where every hit
is worth 1 and `maxCount: 100` on a `*` rule sensibly means 100 requests
per method. It is wrong for credit counting: the per-method cost is
already carried by the addend, so partitioning by method as well prices
each method into its own wallet and makes the total meaningless.

Omit the `method` entry when the permit is credit-counted. All methods
then draw from one pool and each spends its own cost from it. Scope
entries are unaffected, so a perIP credit rule still gives every client
its own wallet. Request counting is untouched.

Carrying the count mode down to descriptor construction needs it at the
call site, so the optional `hitsAddend ...uint32` becomes
`cost ...PermitCost` holding both the weight and its unit. The weight
alone is not enough: inferring credit mode from `hits != 1` would
misread a 1-CU method as request counting and silently switch that
call's pool.

Depends on the per-rule counter isolation fix. Without it, dropping
`method` makes a `*` credit rule and an exact-method credit rule at the
same scope collide on one counter.

Note the resulting sharp edge, documented as a footgun: a budget used by
both credit-mode and request-mode upstreams now keeps two counters per
rule, since the two modes build different descriptors.
Credit-weighted rate limiting only existed at upstream scope, where an
upstream on `rateLimitCountMode: credit` prices each call from its
vendor's credit-unit table. A project-, network- or auth-scoped budget
has no vendor, so it could only count requests: a range scan and a
chain-ID lookup cost the same. A per-IP cap then has to be sized for the
worst case and throttles cheap traffic to guard against expensive
traffic.

Let a budget price its own methods. `rules[].countMode: credit` makes a
rule charge the method's cost from a new budget-level `creditUnits`
table instead of a flat 1, turning `maxCount` into a compute allowance.
Resolution reuses `ResolveCreditUnits`, so precedence matches what
operators know from `upstream.creditUnits`: exact method, then "*", then
1. A method priced 0 is exempt.

`countMode` is per rule, not per budget, so one budget can hold a pooled
wallet and flat per-method caps together. That is what makes exempting a
method usable: price it 0 to keep it out of the wallet, and cover it
with a `countMode: request` rule so it still has a ceiling. A rule that
states its mode keeps it even when the caller counts credits; only an
unset mode inherits, so existing upstream credit budgets are unchanged.

The two cost sources never mix. An upstream on
`rateLimitCountMode: credit` prices from its vendor, and pointing it at
a budget that also defines `creditUnits` or rule-level `countMode` is
rejected at validation rather than resolved by silent precedence.

Exempt methods skip the rule rather than sending a 0 addend, because the
request-level `HitsAddend` coerces 0 to 1 (`GetHitsAddends`) and would
charge a full hit. Note the consequence, documented as a footgun: a
0-priced method keeps being served after the pool is empty.

`countMode` joins the rule identity used for counter isolation and the
`rate_limiter_budget_max_count` label set, so rules differing only in
mode cannot share a counter or overwrite each other's gauge.

This replaces an earlier `weighted` + `methodCosts` design that predated
vendor credit accounting and duplicated its vocabulary, its resolution
logic and its addend. No built-in cost table ships with it: that would
rank eRPC's own per-method values above an operator's "*" catch-all and
silently defeat their policy. The docs carry a starting table to copy.

- common/config.go: RateLimitBudgetConfig.CreditUnits,
  RateLimitRuleConfig.CountMode, CountModeString, RateLimiterBudget
  lookup (+ regenerated generated.ts, index.ts re-exports).
- common/validation.go: countMode enum, credit needs maxCount > 0,
  non-negative creditUnits, upstream/budget cost-source conflict.
- upstream/ratelimiter_budget.go: resolveCost per rule, mode-driven
  method pooling, per-rule exempt skip, countMode in the rule key.
- docs: schema rows, worked example, three footguns.
jscaltreto force-pushed the feat/ratelimit-weighted-credit-budgets branch from 00329c6 to 3684d17 Compare August 12, 2026 22:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL