A flexible, goroutine-safe, per-key rate limiter for Go, built as a thin manager around the token-bucket implementation in golang.org/x/time/rate.

Each key (a user ID, API key, or IP address) gets its own independent token bucket, so one client exhausting its budget never affects another. Limiters are created lazily on first use and automatically evicted after they have been idle for a configurable duration.

New to token buckets? See docs/TOKEN_BUCKET.md for a thorough, from-scratch explanation of the algorithm, how to choose limit and burst, and how the HTTP rate-limit headers work.

Upgrading from an earlier version? See docs/MIGRATION.md for a before/after guide to the breaking API changes.

• Per-key isolation — a separate token bucket per identifier; keys never share a budget.
• Token bucket algorithm — sustained average rate plus configurable bursts, backed by the battle-tested golang.org/x/time/rate.
• Goroutine-safe — designed for concurrent use; key creation is atomic (no duplicate limiters under races).
• Automatic idle eviction — a single background goroutine removes limiters that have not been used for deleteAfter; active keys are kept alive. Stop it with Close().
• Type-safe & generic — Storage[K, V] and BucketLimiter[K] use Go generics (Go 1.26+).
• Extensible — implement the Storage interface for a custom in-process store, or the Limiter interface for a custom algorithm (optionally adding Reserver for accurate Retry-After, even with a Redis/Valkey backend).
• HTTP middleware example — with accurate Retry-After and RateLimit-* response headers, driven by the backend-agnostic Reserver interface.

BucketLimiter is a thin manager: it maps each key to its own Limiter, builds new ones on demand through a factory, persists them in a pluggable Storage, and runs a single background goroutine that evicts idle keys.

flowchart TD
    subgraph caller["Your code"]
        C["GetOrAdd(key)"]
    end

    subgraph manager["BucketLimiter[K]"]
        direction TB
        F["newLimiter func() Limiter<br/>(factory)"]
        A["access map<br/>K → last-use time"]
        S["sweepLoop goroutine<br/>evicts idle keys"]
    end

    subgraph store["Storage[K, Limiter]"]
        direction LR
        K1["user-123 → bucket"]
        K2["user-456 → bucket"]
        K3["10.0.0.7 → bucket"]
    end

    C -->|"1. Load / LoadOrStore"| store
    C -.->|"2. build on miss"| F
    F -.->|"fresh *rate.Limiter"| store
    C -->|"3. touch"| A
    S -->|"Delete idle"| store
    S -->|"Delete idle"| A

    K1 & K2 & K3 -->|"independent<br/>token buckets"| RL["golang.org/x/time/rate"]

Each key owns an independent token bucket, so one client draining its budget has no effect on any other. A typical GetOrAdd(key).Allow() call:

sequenceDiagram
    autonumber
    participant App as Your code
    participant BL as BucketLimiter
    participant St as Storage
    participant Lim as Limiter (bucket)

    App->>BL: GetOrAdd(key)
    BL->>St: Load(key)
    alt key exists
        St-->>BL: existing Limiter
    else first use of key
        BL->>BL: newLimiter()
        BL->>St: LoadOrStore(key, fresh)
        Note over BL,St: atomic — racing callers<br/>share one instance
        St-->>BL: stored Limiter
    end
    BL->>BL: touch(key) — refresh idle timer
    BL-->>App: Limiter
    App->>Lim: Allow()
    alt token available
        Lim-->>App: true (consume 1 token)
    else bucket empty
        Lim-->>App: false (rate limited → 429)
    end

go get github.com/slashdevops/ratelimiter

Requires Go 1.26 or newer.

package main

import (
	"fmt"
	"time"

	"github.com/slashdevops/ratelimiter"
	"golang.org/x/time/rate"
)

func main() {
	// A store that keeps one Limiter per key.
	storage := ratelimiter.NewInMemoryStorage[string, ratelimiter.Limiter]()

	// A factory that builds an independent bucket for each new key:
	// 5 requests/second sustained, absorbing bursts of up to 10.
	newLimiter := ratelimiter.NewRateLimiterFunc(rate.Limit(5), 10)

	// The manager. Limiters idle for 1 minute are evicted.
	manager := ratelimiter.NewBucketLimiter(newLimiter, time.Minute, storage)
	defer manager.Close() // stop the background eviction goroutine

	// Each key has its own bucket.
	if manager.GetOrAdd("user-123").Allow() {
		fmt.Println("allowed")
	} else {
		fmt.Println("rate limited")
	}
}

• limit (rate.Limit) — sustained refill rate in tokens per second. Use rate.Every(d) for "one every d", or rate.Inf for unlimited.
• burst (int) — bucket capacity: the largest number of requests allowed in a single instant.

See docs/TOKEN_BUCKET.md for guidance on choosing values.

lim := manager.GetOrAdd(key)

// Non-blocking: drop work when over the limit (e.g. HTTP 429).
if !lim.Allow() {
	// reject
}

// Blocking: shape work by waiting for a token (pass a ctx with a deadline).
if err := lim.Wait(ctx); err != nil {
	// ctx cancelled/expired
}

When you need the exact delay until the next token — to set a Retry-After or RateLimit-Reset header — use the optional Reserver capability. Feature-detect it so your code works with any limiter and degrades gracefully:

lim := manager.GetOrAdd(key)

if r, ok := lim.(ratelimiter.Reserver); ok {
	res := r.Reserve()
	if res.OK() && res.Delay() == 0 {
		// proceed now
	} else {
		res.Cancel()                 // return the token
		retryAfter := res.Delay()    // tell the client exactly how long to wait
	}
} else {
	_ = lim.Allow() // limiter without reservation support: no timing info
}

The default RateLimiter from NewRateLimiterFunc implements Reserver, and a custom (e.g. Redis/Valkey-backed) Limiter can too — so the same middleware produces accurate headers regardless of backend. See docs/CUSTOM_STORAGE.md.

The examples/middleware program limits requests per client IP and sets standard response headers:

• RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset (IETF draft names, plus legacy X-RateLimit-*).
• Retry-After on 429, computed from the actual reservation delay.

It also extracts the client IP with net.SplitHostPort, so it is correct for both IPv4 and IPv6.

go run ./examples/middleware -limit 2 -burst 1

See examples/key for a minimal, dependency-free demo of per-key isolation and bucket refill over time:

go run ./examples/key -limit 1 -burst 3

BucketLimiter talks to the Storage[K, V] interface, never a concrete map, and you inject the implementation at construction time. InMemoryStorage is just the bundled default — implement the interface to bring your own in-process store (for example a size-bounded LRU to cap memory instead of, or in addition to, time-based eviction). Implementations must be safe for concurrent use, and LoadOrStore must be atomic.

type Storage[K comparable, V any] interface {
	Store(key K, value V)
	Load(key K) (value V, ok bool)
	LoadOrStore(key K, value V) (actual V, loaded bool)
	Delete(key K)
	Range(f func(key K, value V) bool)
}

A complete, runnable size-bounded LRU store lives in examples/customstorage:

go run ./examples/customstorage -cap 2

docs/CUSTOM_STORAGE.md is a full guide: the method contracts, how to test atomicity, and — importantly — why a custom Storage is in-process only, plus the correct pattern for distributed limiting with Redis / Valkey: a datastore-backed Limiter, given its key by WithLimiterFactoryForKey.

A Limiter is bound to exactly one key — Allow() takes no arguments, so the instance is the bucket. WithLimiterFactoryForKey builds each one from its key, which is what a limiter needs when its counter lives somewhere else:

bl := ratelimiter.NewBucketLimiter(nil, time.Minute,
    ratelimiter.NewInMemoryStorage[string, ratelimiter.Limiter](),
    ratelimiter.WithLimiterFactoryForKey(func(key string) ratelimiter.Limiter {
        if shared != nil {
            return sharedLimiter{client: shared, key: "rl:" + key}
        }
        return ratelimiter.RateLimiter{Limiter: rate.NewLimiter(limit, burst)}
    }),
)

That is the whole seam for "shared budget when the datastore is there, in-process when it is not" — and the Storage stays the ordinary in-memory one in both branches, because it only caches handles.

Given the same parameters they behave identically — they are duals, and any comparison implying otherwise has quietly changed the burst between the two sides. Measured with this package:

token bucket, 10/s burst 1  → 1 admitted now, 1 more after 250ms
leaky bucket, 100ms cap 1   → 1 admitted now, 1 more after 250ms

What differs is how you say it, and that is worth something:

"No more than 1000 an hour" is a budget → token bucket. "No more than one call every 100ms" is a pace → leaky bucket, where strict pacing is the obvious configuration rather than a non-obvious burst: 1.

Pick one by value, so it can come from a config file or a database column:

strategy, err := ratelimiter.ParseStrategy("leaky_bucket") // or "token_bucket"
limit := ratelimiter.Limit{Requests: 60, Period: time.Minute, Burst: 1}

newLimiter, err := ratelimiter.NewLimiterFunc(strategy, limit)
bl := ratelimiter.NewBucketLimiter(newLimiter, time.Minute, storage)

One Limit describes both; the strategy decides how it is enforced. ParseStrategy rejects an unrecognised value rather than defaulting — a typo that silently became token_bucket would admit bursts you specifically asked it not to.

docs/TOKEN_BUCKET.md and docs/LEAKY_BUCKET.md are the full guides.

Storage holds limiters in this process. Backend holds the count, wherever you want it: an in-process map, Valkey, Redis, DynamoDB, Postgres.

type Backend interface {
	Take(ctx context.Context, key string, limit Limit, cost int) (Decision, error)
}

That is the whole interface. Implement it and every process shares one budget.

backend := ratelimiter.NewMemoryBackend()   // or your own
limit   := ratelimiter.Limit{Requests: 100, Period: time.Minute}

bl := ratelimiter.NewBucketLimiter(nil, time.Minute,
	ratelimiter.NewInMemoryStorage[string, ratelimiter.Limiter](),
	ratelimiter.WithLimiterFactoryForKey(
		ratelimiter.NewBackendLimiterFunc(backend, limit,
			ratelimiter.WithFallback(local)),
	),
)

It is Take-shaped rather than Get/Set-shaped for a reason: a token bucket's update is a read-modify-write, and split across a network that is a race in which the limit silently becomes 2×. The decision has to run where the state lives, so the interface is the decision, not the storage.

BackendLimiter adds the three things nobody should have to reimplement:

• a local fallback, because neither refusing nor allowing is an acceptable answer to "the datastore is down" on its own;
• a circuit breaker, because falling back without one makes every request during an outage pay a failed round trip first;
• a degraded signal, because a limiter silently enforcing N× the intended limit is invisible from a request.

docs/BACKENDS.md is the full guide, including a ~25-line Valkey implementation. Runnable example:

go run ./examples/backend

Token state lives in memory inside each *rate.Limiter, so the default limiter enforces limits within one process. Use a Backend to share one budget across instances. Running N instances behind a load balancer yields an effective global limit of up to N × limit. Global, cross-instance limiting requires a distributed algorithm (e.g. a Redis script) and is out of scope. The Storage interface is for custom in-process stores, not for synchronizing token state across machines. See docs/TOKEN_BUCKET.md.

go test -race ./...
go test -bench . -benchmem ./...

Apache License 2.0. See LICENSE.