Fan-out patterns

The Feed Problem

When a user posts on a social platform, their followers need to see the post. The question is: do you distribute the post at write time or at read time?

• Fan-out on write: when a user posts, push the post (or a reference to it) to every follower's feed at write time
• Fan-out on read: when a user reads their feed, pull posts from all the accounts they follow and merge

The choice has massive implications for storage, latency, and infrastructure at scale.

TL;DR

• Fan-out on write (push model) precomputes each user's feed at post time by writing to every follower's inbox. Reads are fast (O(1)), but writes are expensive (O(followers)).
• Fan-out on read (pull model) assembles the feed at read time by querying all followed accounts and merging. Writes are cheap (O(1)), but reads are expensive (O(following)).
• Neither pure model works at scale. The real solution is hybrid: push for normal accounts, pull for high-follower accounts (celebrities).
• The celebrity/hotspot problem is the defining challenge: a user with 50M followers generates 50M writes per post under fan-out on write.
• This pattern appears in nearly every "Design Twitter/Instagram/Facebook" interview question. Know when to use each model and how to handle the celebrity threshold.

The Problem

When a user posts on a social platform, their followers need to see the post. This sounds simple until you look at the numbers.

Consider a platform with 500M users, where the average user follows 200 accounts and the average user posts twice per day. A celebrity user has 50M followers. When that celebrity posts, how do you get the post into 50M feeds?

Option 1: Write the post reference to all 50M follower inboxes at post time. That's 50M writes for one post. If the celebrity posts 10 times a day, that's 500M write operations just for one account.

Option 2: When any of those 50M followers opens their feed, query all accounts they follow, fetch recent posts, and merge them. If each user follows 200 accounts, that's 200 lookups and an N-way merge, computed on every feed refresh, for every user, multiple times per day.

Both options are expensive. The question isn't which one to use, it's when to use which.

The forces in tension: write amplification (fan-out on write makes posts expensive to create) vs read latency (fan-out on read makes feeds expensive to load). You can't eliminate both. Every feed system engineering team eventually confronts this trade-off, and the answer is always "it depends on the follower distribution."

One-Line Definition

Fan-out patterns distribute content to consumers either at write time (push) or at read time (pull), trading write amplification for read performance or vice versa.

Analogy

Think about a newspaper delivery service. There are two models.

Push model (fan-out on write): The printing press runs. A truck delivers a copy of the newspaper to every subscriber's doorstep at 5 a.m. When subscribers wake up, the paper is already there. Fast read experience. But if you have 1 million subscribers and a breaking story needs an extra edition, you need 1 million more deliveries.

Pull model (fan-out on read): No home delivery. Subscribers go to the newsstand when they want to read. The newsstand has a copy of every newspaper from every publisher. The subscriber picks the ones they want and reads them. No delivery cost, but the subscriber has to travel to the newsstand and browse every time.

Hybrid: High-volume newspapers get delivered to your door. Niche publications are only available at the newsstand. Most subscribers get 90% of their reading delivered and occasionally visit the newsstand for the rest. The decision of what to deliver vs what to leave at the newsstand is the "threshold" in our hybrid fan-out model.

Solution Walkthrough

Fan-Out on Write (Push Model)

At post time, iterate the posting user's follower list and write a reference to the new post into each follower's feed inbox.

async function post(userId: string, content: string): Promise<void> {
  const postId = await createPost(userId, content);
  const followers = await getFollowers(userId); // could be millions

  for (const followerId of followers) {
    await feedInbox.prepend(followerId, postId); // write to each inbox
  }
}

async function getFeed(userId: string): Promise<Post[]> {
  const postIds = await feedInbox.getRecent(userId, 50); // pre-computed
  return fetchPosts(postIds); // simple batch lookup
}

Read path: A user's feed is just reading their inbox. Pre-computed, sorted, fast. O(1) read (ignoring pagination). Feed load times are under 50ms because there's no computation at read time.

Write path: O(followers) writes per post. A user with 10M followers generates 10M writes per post. This is write amplification.

Storage: Every post reference is duplicated in N inboxes. If a celebrity posts and has 50M followers, that post ID is written 50M times. Assuming 100 bytes per feed entry (postId + metadata + timestamp), one celebrity post consumes 50M x 100B = 5GB of feed storage.

Best for: Accounts with small-to-medium follower counts (under 10K) where read performance matters more than write cost.

Fan-Out on Read (Pull Model)

At read time, look up all accounts the user follows, fetch recent posts from each, and merge-sort them.

async function post(userId: string, content: string): Promise<void> {
  await createPost(userId, content); // one write, done
}

async function getFeed(userId: string): Promise<Post[]> {
  const following = await getFollowing(userId); // 200 accounts
  const postLists = await Promise.all(
    following.map(id => getRecentPosts(id, 20))
  );
  return mergeSorted(postLists).slice(0, 50);
}

Read path: O(following) fetches + N-way merge. If a user follows 200 accounts, that's 200 parallel queries and a 200-way merge at read time. Even with parallelism and caching, this takes 100-500ms depending on infrastructure.

Write path: One write: create the post. No fan-out. O(1). Instant.

Storage: Each post is stored once. No duplication. A celebrity with 50M followers and 10 posts/day has the same storage footprint as any other user.

Best for: Systems where write cost must be minimized. Also works when follower counts are very high (celebrities) because there's zero write amplification regardless of follower count. The trade-off is clear: you pay nothing at write time but pay dearly at read time.

The Hybrid Approach

Neither pure model works at scale. The real solution is to use both, switching between them based on a follower count threshold.

const CELEBRITY_THRESHOLD = 10_000; // accounts with >10K followers

async function post(userId: string, content: string): Promise<void> {
  const postId = await createPost(userId, content);
  const followerCount = await getFollowerCount(userId);

  if (followerCount <= CELEBRITY_THRESHOLD) {
    // Normal user: fan-out on write (push to all follower inboxes)
    const followers = await getFollowers(userId);
    await fanOutWorker.enqueue({ postId, followers });
  } else {
    // Celebrity: mark for pull at read time
    await markAsCelebrityPost(postId, userId);
  }
}

async function getFeed(userId: string): Promise<Post[]> {
  // Step 1: Read pre-computed inbox (pushed posts from normal accounts)
  const pushedPosts = await feedInbox.getRecent(userId, 50);

  // Step 2: Pull celebrity posts in real-time
  const celebsFollowed = await getCelebrityFollowing(userId);
  const celebPosts = await Promise.all(
    celebsFollowed.map(id => getRecentPosts(id, 10))
  );

  // Step 3: Merge and return
  return mergeSorted([pushedPosts, ...celebPosts]).slice(0, 50);
}

The threshold is the key design decision. Twitter reportedly uses around 10K followers as the dividing line. Instagram's threshold may differ. The exact number depends on your system's write throughput capacity and acceptable fan-out latency.

Why this works: Most accounts have under 10K followers. The 99% case uses fan-out on write (fast reads). Only celebrities (the top fraction of a percent) use fan-out on read. At read time, the user's feed merges the pre-computed inbox with a small number of real-time celebrity fetches. If a user follows 5 celebrities and 195 normal accounts, the read path does 1 inbox read + 5 celebrity fetches. That's 6 operations instead of 200.

SNS/SQS Fan-Out (Infrastructure Pattern)

The same fan-out concept appears at the infrastructure level. AWS SNS + SQS implements fan-out on write: when a message is published to an SNS topic with N SQS subscribers, SNS delivers a copy to each queue. This is commonly used for decoupling microservices.

SNS Topic: "order-created"
  → SQS Queue: "order-fulfillment" (warehouse team)
  → SQS Queue: "order-analytics" (data team)
  → SQS Queue: "order-notifications" (comms team)

Kafka consumer groups are the pull-side equivalent: a single topic, multiple consumer groups, each group independently reading all messages. The broker handles the fan-out internally.