Trending Topics

What is a trending articles system?

A trending articles system tracks the K most-shared articles over a configurable sliding time window, such as the last hour, 24 hours, or 7 days. The hidden engineering challenge is time-bounded counting: share events that fall outside the window must stop contributing to an article's score, which means counts age out continuously rather than only accumulating.

This is harder than a generic Top-K because the answer changes every second as old events drop off. I find this one of the best interview questions because candidates initially assume it is just "count stuff and sort," then realize the sliding window fundamentally changes the data structure choices. The design gets pushed toward bucket-level granularity, approximate counting, and precomputation.

Functional Requirements

Core Requirements

1. Users can share an article, generating a share event that increments the article's trending score.
2. The system exposes the top-K trending articles for configurable sliding windows (last 1 hour, 24 hours, 7 days). K is configurable, defaulting to 10, with a maximum of 100.
3. Trending results can be filtered by category (tech, sports, politics).

Below the Line (out of scope)

• Article content storage and serving
• User notifications when a shared article enters the trending list
• Trending people, hashtags, or topics (articles only)
• Real-time push of trending updates to all connected clients

The hardest part in scope: Maintaining a correct leaderboard across a sliding time window at 100,000 share events per second. Counts must age out continuously as the window moves, which rules out naive counters and requires a bucket-based aggregation strategy combined with precomputed results.

Article content storage is below the line because it solves a completely different problem. To add it, I would store article metadata in a relational database with a full-text search index, separate from the counter pipeline we're designing here.

User notifications are below the line because they introduce a fan-out write pattern orthogonal to the counter design. To add them, I would publish a Kafka event when an article first enters the top-K, consumed by a dedicated notification service that fans out to interested users asynchronously.

Real-time push of trending updates is below the line because it adds WebSocket fan-out complexity without changing the storage design. To add it, I would use Server-Sent Events: when the precomputed top-K changes, the Read Service publishes a diff event that SSE connections consume, keeping the push path entirely separate from the read path.

Non-Functional Requirements

Core Requirements

• Write throughput: 100,000 share events per second at peak, during viral events when a single article dominates a window. This rules out any design that routes all writes through a single storage key.
• Read throughput: 500M DAU with 30 page loads per day each, producing roughly 170,000 trending reads per second. This demands precomputed results, not live aggregation.
• Read latency: Trending list returns in under 50ms p99. A real-time ZUNIONSTORE across 60 Redis sorted sets on every request is too slow at this read rate.
• Write latency: Share event acknowledged in under 100ms. The ingestion path must stay thin and delegate aggregation work to an async pipeline.
• Freshness: Top-K results are at most 30 seconds stale. This is the key tolerance that enables precomputation and CDN caching.
• Availability: 99.99% uptime. Availability over consistency: a slightly stale trending list is always preferable to an error response.

Below the Line

• Sub-5ms global read latency via CDN edge (achievable but not a core NFR in this design)
• Exactly-once share counting (at-least-once with deduplication is sufficient)

Read/write ratio: At steady state, reads outpace writes roughly 50:1. During viral spikes, write rate briefly spikes 10x. The system must handle both extremes independently: the write path must absorb bursts without affecting read latency, and the read path at 170,000 requests/second requires precomputed answers rather than live computation.

The 30-second freshness tolerance is the most consequential NFR in this design. It gives us permission to compute the top-K in a background job and cache the result, rather than aggregating on every read. Without this tolerance, the architecture would require a much more complex real-time aggregation pipeline.

The 100,000 writes per second for a single viral article is the worst-case write multiplier. Any design that routes all writes for one article to a single Redis key will saturate a Redis node, since Redis serializes all operations on a single key on one thread.

Core Entities

• ShareEvent: A record that a specific user shared a specific article at a timestamp. Contains article_id, user_id, and shared_at. This is the raw event that drives all downstream counting.
• Article: An external content item identified by article_id, with a category field used for filtered trending queries. This service does not own article content or metadata beyond the category.
• TrendingBucket: An aggregated score for an article within a specific one-minute time bucket. Contains bucket_ts, article_id, category, and share_count. This is the fundamental unit of the sliding window implementation.
• TrendingResult: A precomputed cached list of the top-K articles for a given window and optional category. Refreshed every 30 seconds by the background Trending Compute Job.

Full schema, bucket key design, and Redis data structures are covered in the deep dives. These four entities are sufficient to drive the API and high-level design.

API Design

Three functional requirements drive the API shape.

FR 1 - Record a share event:

POST /articles/{article_id}/shares
Authorization: Bearer <token>

Response: 202 Accepted

202 Accepted over 200 OK because the write is asynchronous: the ingestion service accepts the event to Kafka and returns immediately. A 200 would incorrectly imply the count was atomically updated, which it is not.

FR 2 - Get the top-K trending articles:

GET /trending?window=1h&k=10&category=tech&cursor=<opaque_cursor>

Response: {
  "articles": [
    { "article_id": "a1b2c3", "title": "...", "share_count": 45231, "rank": 1 }
  ],
  "window": "1h",
  "computed_at": "2026-03-29T00:00:00Z",
  "next_cursor": "..."
}

The computed_at field is the explicit freshness signal to clients. It tells product teams that results are up to 30 seconds stale by design, preventing them from building features that depend on exact real-time counts.

Cursor-based pagination handles large K values. Even at K=100, the result set is small, but the cursor enables incremental loading in mobile UIs. The category filter is optional; when omitted, the response covers all categories. When specified, the Read Service fetches a category-specific precomputed result at no extra aggregation cost.

FR 3 - Trending articles filtered by category:

This uses the same GET /trending endpoint shown above with the category query parameter. No separate endpoint is needed because category filtering is resolved at precompute time, not at query time.

High-Level Design

1. Naive approach: share events directly to the database

The simplest design writes every share to a share_events table and queries it for the trending list at read time.

Components:

• Client: Sends POST /articles/{id}/shares for writes and GET /trending for reads.
• Share Service: Accepts requests, inserts share rows, and runs aggregate queries for the trending list.
• Database: Stores all share events. Top-K query uses GROUP BY article_id ORDER BY count DESC LIMIT K with a time range filter.

Request walkthrough:

1. Client sends POST /articles/42/shares.
2. Share Service extracts user_id from the auth token.
3. Service inserts (article_id=42, user_id=891, shared_at=NOW()) into the share_events table.
4. Client receives 202 Accepted.
5. Client calls GET /trending?window=1h.
6. Share Service runs SELECT article_id, COUNT(*) FROM share_events WHERE shared_at > NOW() - INTERVAL '1 hour' GROUP BY article_id ORDER BY count DESC LIMIT 10 against the database.
7. Client receives the top-10 list.

This is the complete naive system. It is correct at low traffic. I always start here in an interview because it takes 30 seconds to draw and immediately sets up the scaling conversation. It fails at scale in two specific ways.

Why the naive approach breaks

The GROUP BY + ORDER BY + LIMIT query scans every row in the last hour's window. At 100,000 share events per second, the share_events table accumulates 360 million rows per hour. Even with an index on shared_at, the aggregation scan takes seconds per query, which immediately violates the 50ms p99 read latency NFR.

The sliding window makes this worse. Because the answer changes every second as old events drop off, there is no safe way to cache the query result. An in-process cache would return stale counts and could not predict when a given article's count changed.

2. Evolved write path: Kafka ingestion with Redis sorted set buckets

The key insight is to separate the write path from the aggregation path. Writes publish to Kafka instantly and return. A Stream Processor consumes from Kafka, maintaining per-minute sorted sets in Redis, while reads serve precomputed results rather than live aggregation.

New components:

• Share Ingestion Service: Accepts the share event and publishes to Kafka. Returns 202 immediately after Kafka acknowledgment. Stateless and horizontally scalable.
• Kafka: Durable ordered log of share events, partitioned by article_id to ensure events for the same article are consumed in order. Decouples write rate from processing rate.
• Stream Processor: Kafka consumer group. Reads share events and issues ZINCRBY trending:{bucket_ts} 1 article:{id} on Redis. Bucket key design and hot-key mitigation are deferred to the deep dives.
• Redis Cluster: Holds per-minute sorted sets keyed by bucket timestamp. Each bucket expires via TTL after it falls outside the longest configured window.

Request walkthrough (write path):

1. Client sends POST /articles/42/shares.
2. Share Ingestion Service publishes { article_id: 42, user_id: 891, ts: <now> } to Kafka topic share_events.
3. Client receives 202 Accepted in under 100ms.
4. Stream Processor reads the event from Kafka.
5. Processor computes bucket_ts = floor(event_ts / 60) * 60 (rounds down to the current minute).
6. Processor issues ZINCRBY trending:{bucket_ts} 1 article:42 on Redis.

This establishes the write path. I like to pause here in an interview and confirm with the interviewer: "the write path is fully async now, and I'll build the read path next." It signals deliberate pacing.

The read path and sliding window aggregation are covered in the next section.

3. Adding the read path: precomputed results from a background job

Serving 170,000 reads per second requires a read path that never triggers live aggregation. A background Trending Compute Job runs every 30 seconds, unions the last N minute buckets into a result key, and caches it. All trending reads fetch from this precomputed key.

New components:

• Trending Compute Job: Runs every 30 seconds. Issues ZUNIONSTORE across the last N bucket keys for each window and category. Stores the aggregated sorted set at a well-known result key with a 35-second TTL.
• Read Service: Accepts GET /trending requests. Fetches the precomputed result key from Redis via ZREVRANGE. On cold start or cache miss, triggers an on-demand compute before responding.

Request walkthrough (read path):

1. Client sends GET /trending?window=1h&k=10.
2. Read Service fetches the precomputed key trending:result:1h from Redis via ZREVRANGE trending:result:1h 0 9 WITHSCORES.
3. Read Service enriches the article IDs with metadata from the database (title, category URL).
4. Client receives the top-10 list in under 50ms.

This is the complete high-level design. For your interview: if you can draw the three-stage pipeline (Kafka ingest, Redis buckets, precomputed read key) and explain the 30-second freshness tradeoff, you have covered the core design. The deep dives below address the sliding window implementation, hot-key mitigation, approximate counting, and read scalability at full production load.

Potential Deep Dives

1. How do we implement a correct sliding window?

The core question is: when a share event falls outside the window, how does its contribution stop counting toward the score? A naive approach either rescans raw events or holds a sorted set that never shrinks cleanly.

There are three constraints to design against:

• Old scores must age out as the window slides, not all at once when a cache expires.
• The implementation must support multiple window sizes (1h, 24h, 7d) without duplicating all writes.
• Write latency to update the leaderboard must stay sub-millisecond so the Stream Processor does not become a bottleneck.

2. How do we handle a viral article generating a hot key?

At 100,000 share events per second for one article, every write targets ZINCRBY trending:{bucket_ts} 1 article:viral_id. Redis serializes all operations on the same key through one thread. The current-minute bucket for a viral article saturates a Redis node before reaching the 100K write/second target.

3. Approximate vs exact Top-K: which matters here?

This system operates on aggregate counts at high volume across millions of articles. The question is whether the trending list needs exact counts, or whether approximate counts with bounded error are acceptable.

This is one of the more interesting design choices in trending systems. I've seen candidates spend ten minutes debating exact vs. approximate when the real question is simpler: the counts displayed to users can tolerate small errors, but the rank order near the cut-off boundary matters most.

4. How do we serve 170,000 reads per second efficiently?

At 170,000 trending read requests per second, even a fast Redis ZREVRANGE on a precomputed key costs CPU and network. Running a live ZUNIONSTORE across 60 sorted sets on every read is out of the question. The read path must be fully decoupled from aggregation.

Final Architecture

The complete system integrates all components from the High-Level Design and the deep dives.

Component summary:

• CDN Edge: Caches the precomputed trending list for 30 seconds per window and category. Reduces origin traffic from 170,000 to under 10 requests per second at the Read Service and Redis.
• API Gateway: Handles auth validation and route distribution. Stateless and horizontally scalable.
• Share Ingestion Service: Publishes share events to Kafka and returns 202 immediately. Handles write bursts via Kafka backpressure, not application-layer queuing. Adding instances scales write ingestion linearly.
• Kafka: Durable ordered log partitioned by article_id. Ensures that all writes for a given article are processed in order by the same Stream Processor partition. Enables replay for backfilling new window sizes.
• Stream Processor: Kafka consumer group. Applies the CMS gate and issues ZINCRBY to sharded bucket sorted sets. Horizontally scalable by adding consumer instances and Kafka partitions together.
• Trending Compute Job: Runs every 30 seconds. Issues ZUNIONSTORE across all bucket shards for each window and category, writes the aggregated result to a well-known Redis key with a 35-second TTL.
• Redis Cluster: Dual purpose. Raw bucket sorted sets (trending:{bucket_ts}:s{shard}) are the compute job input. Precomputed result keys (trending:result:{window}:{category}) are the read path output.
• Read Service: Serves ZREVRANGE on the precomputed result key for CDN misses and direct API traffic. Falls back to on-demand ZUNIONSTORE on cold start only.
• PostgreSQL: Stores article metadata (title, category, canonical URL) for enriching trending results. Also receives durable count flushes from the Stream Processor for audit and window backfill.

Interview Cheat Sheet

• A trending articles system is Top-K constrained to a time window. The key difference from a generic Top-K system is that share counts must age out as the window slides. Events that fall outside the window must stop contributing to the score, which requires bucket-level granularity rather than a simple accumulating counter.
• The sliding window uses per-minute tumbling buckets. Each share event writes to the current minute's Redis sorted set (trending:{bucket_ts}:s{shard}). Old buckets age out by exclusion from the ZUNIONSTORE key list; their TTL ensures automatic cleanup. No explicit subtraction logic is required.
• Redis ZINCRBY maintains real-time per-bucket leaderboards. The background Trending Compute Job issues ZUNIONSTORE across all bucket keys for a window to produce an aggregated sorted set. ZREVRANGE on the aggregated result returns the top-K in O(log N + K) time.
• Viral articles create hot keys because 100,000 writes per second all target the same sorted set member. Consistent-hash sharding by article_id % S routes each article to a fixed shard. With S=10, each shard receives at most 10,000 writes per second for any one article, well within a Redis sorted set's throughput ceiling.
• A Count-Min Sketch gates promotion into the sorted set leaderboard. Only articles above a minimum share threshold enter the sorted set, keeping it small regardless of total article corpus size. The CMS uses roughly 30KB for 10M articles at 1% error rate.
• Precomputing the top-K every 30 seconds eliminates live aggregation from the read path. Every read becomes a single ZREVRANGE on a precomputed key. The background job pays the ZUNIONSTORE cost once per 30-second cycle; reads pay nothing.
• CDN caching with Cache-Control: max-age=30 reduces origin read load from 170,000 requests per second to under 10. Each CDN PoP caches the result for one full precompute cycle. stale-while-revalidate: 10 prevents thundering herd on TTL expiry.
• The 30-second freshness NFR is the enabling constraint for the entire read architecture. Confirm this tolerance early in the interview because it directly unlocks precomputation and CDN caching. Without it, the system would require continuous real-time aggregation streaming.
• Kafka decouples write acknowledgment from processing latency. The Share Ingestion Service acknowledges writes in under 100ms because it only publishes to Kafka. Stream Processor lag does not affect user-facing write latency.
• Category filtering uses parallel bucket keys (trending:{cat}:{bucket_ts}:s{shard}). The Stream Processor writes to both the global and category-specific bucket from the same event using a Redis pipeline. The Trending Compute Job precomputes separate result keys per category.
• Window eviction is implicit. Old buckets are excluded from the ZUNIONSTORE key list before their TTL expiry. No cleanup job is required because the window is defined by which bucket keys the compute job includes, not by what is stored.
• At interview time, start by confirming the freshness tolerance and read/write ratio, draw the naive approach (write to DB, read with GROUP BY), then introduce Kafka for decoupling, Redis per-minute buckets for the sliding window, and the background precompute job. Defer hot-key sharding and CMS to deep dives if time allows.