Caching

TL;DR

• A cache is a fast, in-memory store that intercepts reads before they reach the database. The cache hit rate is the single number that determines how hard your database has to work.
• At 95% hit rate, your DB sees 20× less read traffic. At 99%, 100× less. That compounding effect is why caching is the first line of defense against every read-scaling problem.
• The core trade-off is consistency vs. latency: your cache is always a snapshot of the DB at a point in time. TTL and invalidation strategies control how stale that snapshot can get — and how much complexity you accept in exchange.
• Cache-aside is the default read pattern. Write-through for data that must be fresh. Write-behind for high-throughput telemetry. Write-around for infrequently read, one-off writes.
• Cache invalidation — knowing when to evict a stale entry — is the hardest operational problem in caching. Get it wrong and you serve wrong data silently at scale.

The Problem It Solves

It's Black Friday. Your e-commerce platform has 500,000 concurrent users. Eighty percent of them are looking at the same 100 product pages — the viral deals.

Your PostgreSQL database has 12 million products, but those 100 rows are being read roughly 4,000 times per second each. Each SELECT takes 5ms. Simple math: 400,000 reads/second × 5ms = you need 2,000 database connections just to avoid queuing. Your connection pool maxes out at 200.

The query time climbs from 5ms to 50ms as connection contention sets in, then to 500ms as the connection pool queue fills. Your monitoring dashboard shows CPU at 95%, I/O wait climbing. At 10:03 a.m. the DB falls over.

Not because the data is complex to retrieve — because you're fetching the same 100 rows over and over, from disk, for every request. I'll often see engineers reach for read replicas here first, but that misses the point: the problem isn't read capacity, it's redundant computation.

flowchart TD
  subgraph Internet["🌐 Black Friday — 500K Concurrent Users"]
    Users(["👤 Users\n500K concurrent\n80% reading same 100 products"])
  end

  subgraph AppTier["⚙️ App Tier — 2,000 Servers Still Not Enough"]
    AS1["⚙️ App Server 1\nEvery read → DB\n400K queries/min"]
    AS2["⚙️ App Server 2–N\nSame pattern — no deduplication"]
  end

  subgraph DBTier["💥 Database Under Siege"]
    DB[("🗄️ PostgreSQL\n400K queries/s\nQuery time: 5ms → 500ms\nConnections maxed · 503s firing")]
  end

  Users -->|"500K HTTP requests/s"| AS1 & AS2
  AS1 & AS2 -->|"Every request → same 100 product rows\nNo deduplication · No reuse"| DB

The fix isn't more database replicas or more app servers. The fix is answering the same question from memory instead of computing it fresh every single time.

What Is It?

A cache is a faster storage layer that sits between your application and your primary data store, holding copies of recently or frequently accessed data so future requests for that data can be served without touching the original source.

Analogy: Think of a coffee shop that serves 300 customers a day, and 90% of them order the same three drinks. The barista could grind beans fresh for every single cup. Or, they could brew a large batch of the popular drip coffees at the start of the hour and pour from it.

Most customers get their coffee in 10 seconds instead of 3 minutes. The three popular coffees are the cache. The bean grinder is the database.

The batch brew is the cache population. Stale coffee that's sat for 4 hours is the invalidation problem — at some point, you throw out the old batch and brew a fresh one.

flowchart TD
  subgraph Internet["🌐 500K Concurrent Users"]
    Users(["👤 Users\n500K concurrent requests"])
  end

  subgraph AppTier["⚙️ Stateless App Tier"]
    AS1["⚙️ App Server 1"]
    AS2["⚙️ App Server 2–N"]
  end

  subgraph CacheTier["⚡ Cache Tier"]
    Redis["⚡ Redis Cluster\n< 1ms · ~95%+ hit rate\nProduct data · Sessions · Hot reads"]
  end

  subgraph DBTier["🗄️ Database Tier"]
    DB[("🗄️ PostgreSQL\nWrites + 5% of reads\n~20K queries/s · manageable")]
  end

  Users -->|"HTTPS requests"| AS1 & AS2
  AS1 & AS2 -->|"Cache read\n(95% hit)"| Redis
  Redis -.->|"Cache miss (5%)\nfetch + populate"| DB
  AS1 & AS2 -->|"Writes + cache miss reads"| DB

With a 95% cache hit rate, your database now handles 20,000 queries/second instead of 400,000. The same database that was collapsing under Black Friday traffic runs comfortably. The cache tier absorbs the fan-out; the database sees only the residual misses and all writes.

My recommendation here is to anchor every caching discussion in the hit rate number — "95% hit rate = 20× less DB traffic" is the sentence that shows you understand the mechanism, not just the concept. The database literally doesn't know 95% of your users exist.

How It Works

Here's what happens on every request when cache-aside (the default pattern) is in use:

1. Client sends a request — e.g., GET /products/7429. The app server needs product data.
2. Check the cache first — The app constructs the cache key (product:7429) and fires a Redis GET. This takes < 1ms.
3. Cache hit → return immediately — If the key exists, deserialize the value and return it. The database is never involved. Cost: ~0.5ms.
4. Cache miss → fetch from source — Redis returns nil. The app queries the database (SELECT * FROM products WHERE id = 7429). Cost: ~5ms.
5. Populate the cache — The app takes the DB result, serializes it, and writes it to Redis with SET product:7429 <data> EX 300 (5-minute TTL). Future requests for this product skip the DB for the next 300 seconds.
6. Return the result — First caller gets database latency. Every subsequent caller until TTL expires gets cache latency.

async function getProduct(productId: string): Promise<Product> {
  const cacheKey = `product:${productId}`;

  // Step 1: Check cache (~0.5ms on hit)
  const cached = await cache.get(cacheKey);
  if (cached !== null) {
    return JSON.parse(cached) as Product; // Cache hit
  }

  // Step 2: Cache miss — fetch from database (~5ms)
  const product = await db.queryOne<Product>(
    'SELECT id, name, price, description FROM products WHERE id = $1',
    [productId]
  );

  if (!product) throw new NotFoundError(`Product ${productId} not found`);

  // Step 3: Populate cache with TTL (5 minutes)
  // Fire-and-forget: don't block the response on the cache write
  cache.set(cacheKey, JSON.stringify(product), { EX: 300 }).catch(console.error);

  return product; // DB miss latency — only the first caller pays this cost
}

sequenceDiagram
    participant C as 👤 Client
    participant A as ⚙️ App Server
    participant R as ⚡ Redis Cache
    participant D as 🗄️ Database

    Note over C,D: Cache HIT path — 95%+ of requests
    C->>A: GET /products/7429
    A->>R: GET product:7429
    R-->>A: {"id":7429,"name":"...","price":99} · < 1ms
    A-->>C: HTTP 200 ─ product data

    Note over C,D: Cache MISS path<br/>first request or after TTL expires
    C->>A: GET /products/1337
    A->>R: GET product:1337
    R-->>A: (nil) — not in cache
    activate D
    A->>D: SELECT * FROM products WHERE id=1337
    D-->>A: row data · ~5ms
    deactivate D
    A->>R: SET product:1337 {data} EX 300
    Note over R: Populated for next 300s
    A-->>C: HTTP 200 ─ product data

The first caller to request product 1337 after a miss (or after TTL expiry) pays database latency. Every caller for the next 5 minutes gets cache latency. At any read-heavy traffic profile, this asymmetry is enormously valuable.

I'll often walk through both paths step by step in an interview — it forces you to articulate the hit path and the miss path, which is where all the interesting tradeoffs live. The hit path is free. The miss path is where your design choices matter.

Key Components

Cache Layers

Every production system has multiple cache layers operating simultaneously. Each layer is physically closer to the user and faster than the one below it — at the cost of smaller capacity and lower consistency guarantees.

The browser and CDN layers are covered in depth in the CDN article. The remainder of this article focuses on the application cache layer — the one you design, own, and debug.

In interviews, I typically sketch just three layers: browser/CDN, app cache (Redis), and database — with the latency numbers at each. That's enough context to show you understand the hierarchy without burning five minutes explaining the full stack. The application layer is the only one where your architecture decisions actually live.

Read Strategies

How the cache gets populated is a design decision. These two patterns cover 95% of use cases:

Cache-Aside (Lazy Loading)

The application manages the cache directly. On a miss, the app fetches from the database and populates the cache itself. This is the pattern shown in the "How It Works" code above.

Characteristics: Only data that's actually been requested is cached — no wasted memory on cold data. The first request to any key always pays database latency; subsequent requests get cache latency. Stale data is bounded by TTL unless you explicitly invalidate on write, and it works with any data source since the cache has no knowledge of where the data lives.

The mistake I see most often is teams skipping cache-aside in favor of a fancier pattern — but cache-aside is correct precisely because it's explicit and simple.

Cache-aside is the correct default for most read-heavy systems.

Read-Through

The cache itself is responsible for loading data from the database on a miss. The application talks only to the cache and never directly queries the DB on a read path.

Characteristics: Simpler application code — one data access layer handles everything. The cache must be configured with your DB schema and connection, often via libraries like Spring Cache or AWS DAX for DynamoDB. The first-request latency and thundering-herd exposure are identical to cache-aside.

In practice, reach for read-through only if your framework naturally supports it — cache-aside is easier to reason about and debug.

Write Strategies

When data changes, what gets updated and in what order? This is where most caching bugs originate. The mistake I see most often is engineers treating caching as a read concern and never thinking about the write path.

Write-Through

Data is written to the cache and the database synchronously in the same operation. Both writes must succeed before the caller receives an acknowledgement.

async function updateProductPrice(productId: string, newPrice: number): Promise<void> {
  // Both writes happen serially before we return — latency = DB write + cache write
  await db.query('UPDATE products SET price = $1 WHERE id = $2', [newPrice, productId]);
  await cache.set(`product:${productId}`, JSON.stringify({ price: newPrice }), { EX: 300 });
  // Cache is now immediately consistent with DB
}

Use when: Read-heavy data that must always be immediately fresh (user account details, inventory counts). Writes are infrequent enough that the doubled write latency is acceptable.

Avoid when: Write-heavy workloads (analytics events, counters) — the cache write adds latency to every single write.

Write-Behind (Write-Back)

Data is written to the cache first and acknowledged to the caller immediately. The cache flushes changes to the database asynchronously via a background process or queue.

async function recordPageView(articleId: string): Promise<void> {
  // Increment in-memory counter — responds in < 1ms
  await cache.incr(`views:${articleId}`);

  // Enqueue DB flush — batched, happens out-of-band
  await queue.enqueue({ type: 'flush_views', articleId });
  // If the cache node crashes before flush, view counts since last flush are lost
}

Use when: Very high write throughput where small data loss windows are acceptable — view counters, click events, IoT telemetry. Losing 500ms of analytics data in a crash is fine. Losing 500ms of payment records is not.

Avoid when: Data represents money, inventory, or any state that must survive a cache failure with zero loss.

Write-Around

Data is written directly to the database, bypassing the cache entirely. The cache entry is not updated or touched — it becomes stale until TTL expires and a read repopulates it.

async function archiveOldOrder(orderId: string): Promise<void> {
  // Direct DB write — this data is read rarely; caching it wastes memory
  await db.query('UPDATE orders SET archived = true WHERE id = $1', [orderId]);
  // No cache interaction — cache either holds stale data until TTL expires,
  // or this key was never cached in the first place
}

Use when: Data that is written once and read rarely (archived records, audit logs, cold historical data). Caching it would consume memory with zero hit-rate benefit on the hot path.

Cache Invalidation on Write (Most Common Pattern)

In practice, the most common pattern across all write types is: write to the database, then delete the cache key. The next read miss repopulates with fresh data.

async function updateUserProfile(userId: string, profile: UserProfile): Promise<void> {
  // 1. Write to source of truth first
  await db.query(
    'UPDATE users SET name = $1, email = $2 WHERE id = $3',
    [profile.name, profile.email, userId]
  );

  // 2. Invalidate — DELETE the key, never SET a new value here
  // Setting a value here introduces a race: two concurrent writes can leave
  // the older write's value as the current cache entry
  await cache.del(`user:${userId}`);
}

Write to the database first. Delete the cache key after. That rule eliminates the entire class of write-race bugs.

Cache Invalidation

Phil Karlton's famous quip — "There are only two hard things in Computer Science: cache invalidation and naming things" — is funny because it's operationally true at every scale.

Invalidation is the question: given that the database has changed, when does the cache know to discard its stale copy?

TTL-Based Invalidation

The simplest approach. Every entry has a fixed expiry. After TTL seconds, Redis automatically evicts the key; the next read repopulates with current DB data.

If your data changes frequently enough for users to notice staleness, TTL alone won't save you — you'll need event-driven invalidation.

Event-Driven Invalidation

On every write, proactively delete relevant cache keys. The cache is stale for at most the propagation delay between the DB write and the invalidation call — typically < 5ms same-region. The challenge: your write path must know every cache key that depends on the changed data.

A users row change might need to invalidate user:{id}, feed:{id}, profile:{id}, and recommendations:{id}. Miss any one of those and stale data persists silently. I've seen teams discover this dependency graph for the first time only after a production incident.

Version-Based Invalidation

Embed a version number in the cache key. When data changes, increment the version. Old keys become unreachable and eventually expire.

// Version-stamped key: product:7429:v3
const cacheKey = `product:${productId}:v${product.version}`;

// After an update, the new version key is used; the old key simply expires via TTL
// No explicit invalidation needed — the old key is unreachable from new reads

Elegant for immutable-snapshot use cases. The downside: stale keys linger until TTL — memory overhead grows proportional to how often data is updated.

Invalidation is the part of caching that will bite you in production — TTL, events, or versioning, you need a strategy before you ship.

Eviction Policies

When Redis runs out of memory, the eviction policy determines which keys are dropped to accommodate new writes.

Trade-offs

The fundamental tension here is consistency vs. performance. A cache is an explicitly stale replica of your database. Every performance gain from caching exists because you are, by design, serving data that might be slightly behind the source of truth.

The engineering challenge is deciding how stale is acceptable for each type of data — and building the invalidation mechanics that enforce that bound. Get the staleness tolerance wrong and your users will notice.

When to Use It / When to Avoid It

So when does this actually matter? The short answer: any time the same data is read far more often than it's written, a cache will dramatically reduce your database load. Here's the full breakdown.

Use caching when:

• Your read traffic is 5× or more than your write traffic for a given dataset.
• The same data is read repeatedly within a window shorter than your acceptable staleness window.
• Database query response time is a meaningful fraction of total request latency.
• You need to scale reads past what a single primary or read replica can handle.
• You store derived or computed values that are expensive to recompute (aggregations, rendered templates, ML inference results).

Avoid caching (or be very careful) when:

• Data is write-heavy and read-once — audit logs, financial ledger entries. Cache pollution with zero hit benefit.
• Data must be instantly visible to all users immediately after a write — for example, a payment acknowledgement shown to the paying user AND billed to their account simultaneously. TTL-based caching requires explicit invalidation for this.
• You're prototyping. Cache adds complexity that masks performance problems. Measure DB performance first, then cache proven bottlenecks.
• Cache failure would cause silently incorrect behavior. If your app serves wrong inventory or wrong prices when the cache breaks, verify your invalidation logic is complete before deploying.

If your application is serving the same read-heavy data to many concurrent users, a cache isn't optional — it's the difference between a database that survives the load and one that doesn't.

The Thundering Herd Problem

Caching's most dangerous failure mode has nothing to do with the cache being full or down. It happens when a TTL expires on a popular key. I'll often see candidates skip this failure mode entirely in interviews — it's the one that bites production caches hardest.

The scenario: Your most popular product (product:viral-item) is cached with a 60-second TTL. At 09:00:00.000, the key expires. Within the next 50ms, 1,000 requests arrive for that product — every single one gets a cache miss.

Every single one fires a database query simultaneously. Your database spikes from 50 queries/second to 1,050 queries/second in 50ms. If your DB connection pool has 100 connections, 900 queries queue immediately.

Latency climbs. Cache population slows. The spike persists until repopulation finishes — which takes longer because the DB is already overloaded.

For any high-traffic key, TTL expiry is a ticking clock to a DB spike — the only question is how many concurrent misses you'll absorb when it fires.

sequenceDiagram
    participant R1 as ⚙️ Request 1
    participant R2 as ⚙️ Request 2
    participant Rn as ⚙️ Request 1,000
    participant C as ⚡ Redis Cache
    participant D as 🗄️ Database

    Note over R1,D: TTL expires: "product:viral"<br/>at 09:00:00.000

    R1->>C: GET product:viral
    R2->>C: GET product:viral
    Rn->>C: GET product:viral

    C-->>R1: (nil) — cache miss
    C-->>R2: (nil) — cache miss
    C-->>Rn: (nil) — cache miss

    Note over D: 1,000 simultaneous DB queries

    R1->>D: SELECT * FROM products
    R2->>D: SELECT * FROM products
    Rn->>D: SELECT * FROM products

    D-->>R1: row data · now 200ms (contention)
    D-->>Rn: row data · now 200ms

    R1->>C: SET product:viral {data} EX 60
    Rn->>C: SET product:viral {data} EX 60
    Note over C: 1,000 requests<br/>all re-populate the same key

Fixing the Thundering Herd

Option 1: Mutex lock — Only the first requester that observes a miss acquires a lock and fetches from DB. All others wait, then return the populated value.

function getWithMutex(key, fetchFn):
  cached = cache.get(key)
  if cached exists → return cached

  lockKey = "lock:" + key
  acquired = cache.set(lockKey, "1", NX=true, EX=10)
  // NX = "set only if key does not exist" — atomically claims the lock

  if acquired:
    try:
      value = fetchFn()             // only this one request hits the database
      cache.set(key, value, EX=300)
      return value
    finally:
      cache.del(lockKey)            // always release the lock, even on error

  // Another process is already fetching — wait and retry
  sleep(50ms)
  return getWithMutex(key, fetchFn) // lock is gone; value should be in cache now

Option 2: Probabilistic Early Expiration (PER) — Before TTL fully expires, randomly refresh the cache with probability rising as expiry approaches. No lock contention; stale data is briefly served while the background refreshes.

function getWithEarlyExpiration(key, ttl, fetchFn):
  value     = cache.get(key)
  remaining = cache.ttl(key)

  if value is null → return null   // complete miss — caller handles fallback

  // Randomly trigger a background refresh before the key fully expires.
  // Probability rises as remaining TTL shrinks toward zero.
  threshold = ttl × 0.20           // begin considering refresh at 20% TTL left
  if remaining < threshold AND random() < 1.5 × (1 - remaining / threshold):
    background: cache.set(key, fetchFn(), EX=ttl)  // async — does not block caller

  return value   // always return current cached value immediately

Option 3: TTL jitter — Instead of a fixed TTL across a batch of keys, add random variance at population time. Expiry events spread out across a window instead of firing simultaneously.

const BASE_TTL = 300;
const JITTER = 60; // ± 1 minute

const ttl = BASE_TTL + Math.floor(Math.random() * JITTER * 2) - JITTER;
await cache.set(cacheKey, value, { EX: ttl }); // TTL between 240–360 seconds

TTL jitter is free to implement and eliminates the synchronized expiry problem without any lock contention — add it by default to any batch of keys populated together.

Real-World Examples

Twitter (X) — 99% of reads served from cache

Twitter's home timeline API serves hundreds of thousands of requests per second. Their underlying storage (Manhattan, their distributed KV store) handles only writes and cache-miss fallback reads — roughly 1–2% of total read traffic. Twitter uses a two-level cache: an in-process LRU cache (L1) within each app server for the absolute hottest content, backed by a distributed Memcached cluster (L2). The L1 cache eliminates the Redis network round-trip entirely for the most popular tweets. When a celebrity with 50M followers posts, the tweet is synchronously fan-out-written into the pre-cached timelines of followers via a fanout write service — so by the time users refresh their feed, the data is already cache-warm with no miss possible. The lesson: at Twitter's scale, the cache is the primary read path. The database is the backup.

I bring up Twitter's fan-out-on-write design whenever an interviewer asks about social feeds — it's a concrete example of caching as an architectural decision, not just a performance optimization.

Facebook (Meta) — "Scaling Memcache at Facebook" (2013)

Facebook published this paper after running one of the largest Memcached deployments in the world — tens of thousands of nodes. Their most surprising finding: cache invalidation, not cache misses, was the primary operational challenge. At their scale, a single DB write needed to invalidate cache entries across thousands of servers simultaneously. The naive approach — write to DB, then delete from cache — had a race condition: a stale read could repopulate the cache between the write and the delete. To prevent this, they built a "lease" mechanism: on a miss, the server issues a lease token to the first requester.

If a concurrent write invalidates the key while that requester is fetching from DB, the lease is revoked and the requester must re-read from cache. Without this, Facebook's caches would silently serve stale data after every write burst.

Stack Overflow — 10M pageviews/day from a handful of servers

Stack Overflow serves enormous traffic from remarkably few servers, and aggressive two-level caching is the core reason. They run an in-process L1 cache within each ASP.NET worker (MemoryCache) and a shared L2 Redis cluster. The in-process cache eliminates the Redis network round-trip for the hottest queries — top questions, user reputation, tag pages — reducing latency to nanoseconds. Stack Overflow uses an active pre-warming strategy: background jobs continuously refresh popular cached values before TTL expires, so DB queries are only ever executed by background workers, never by user-traffic request handlers. A question page that takes 40ms to render from DB cold is served from the in-process cache in under 1ms for 99.9% of loads.

Every one of these examples proves the same thing: at scale, caching is a consistency and invalidation problem far more than a performance problem.

How This Shows Up in Interviews

Here's the honest answer on what separates a good caching answer from a forgettable one: it's not knowing Redis exists — it's narrating the impact in numbers. My recommendation is to always follow "I'd add a cache here" with the hit rate and its consequence on DB load. That one habit alone moves you from junior to mid-level in the interviewer's mental model.

Depth expected at senior/staff level:

• Name your cache hit rate and calculate the DB impact: "95% hit rate → DB sees 5% of reads → 20× less traffic — that's the difference between needing 5 read replicas and needing none."
• Explain the write strategy choice. Write-through for critical user data; write-behind for high-throughput counters; always-delete-never-set on explicit invalidation.
• Know the thundering herd problem and two mitigations: mutex lock (serialize misses) or probabilistic early expiration (background refresh before TTL expires).
• Address Redis as a SPOF: replication (Redis Sentinel) and a graceful degradation path (fall through to DB with a circuit breaker) when cache fails.
• Distinguish eviction policy (Redis choosing what to drop when full) from invalidation strategy (you proactively evicting stale data after a write). These are separate mechanisms that work together.

Common follow-up questions and strong answers:

Know these cold — cache invalidation, thundering herd, and Redis SPOF handling come up in nearly every system design interview at the senior level.

Test Your Understanding

Quick Recap

1. A cache is a fast in-memory store that intercepts reads before they hit the database — cache hit rate, not server count, is what determines how hard your database actually works.
2. In cache-aside (the default read pattern), your application checks the cache first, falls back to the database on a miss, and populates the cache on return. Write-through keeps cache and DB synchronous; write-behind allows fast writes at the cost of durability; always delete (never update) the cache key on a write to avoid race conditions.
3. Every cache entry must have a TTL. Short TTLs maximise freshness at the cost of hit rate; long TTLs improve performance but risk serving stale data. Jitter TTL values across hot batches to prevent a Cache Avalanche when they expire simultaneously.
4. Cache invalidation on write is always DEL key, never SET key newValue. Updating the cache value after a write introduces a race condition where two concurrent writes leave the cache holding the older value.
5. The thundering herd fires when a popular key's TTL expires and hundreds or thousands of requests simultaneously miss — use mutex locks or probabilistic early expiration to limit the DB burst to a single re-population fetch.
6. Redis is a potential SPOF. Deploy with replication (Redis Sentinel for HA, Redis Cluster for horizontal scale) and build a graceful degradation path — fall through to the database with a rate limiter when the cache is down.
7. In every interview, state your hit rate numerically and calculate its DB impact: "95% hit rate → DB handles 5% of reads → 20× less traffic than without a cache" is the sentence that signals you understand what caching actually does, not just that it exists.

Related Concepts

• Load balancing — Load balancers route traffic across app servers; caches reduce the database traffic those app servers generate. Both are required for a horizontally scaled system — neither alone is sufficient.
• CDN — A CDN is a geographically distributed cache for static and edge-cacheable content. Understanding browser cache headers and CDN invalidation is the natural extension of application-layer caching concepts.
• Databases — Caching exists to protect databases from read fan-out. Understanding database connection pooling, query cost, and replica lag helps calibrate the right TTL and hit-rate targets for any caching strategy.
• Replication — Database read replicas and Redis replication are two different answers to the same read-throughput problem. Read replicas suit complex queries; caches suit repeated simple reads. Knowing when to use each prevents over-engineering.
• Rate limiting — Redis is the canonical store for distributed rate limit counters. The atomic INCR and Lua scripting patterns from this article apply directly to building correct, race-condition-free rate limiters.