How YouTube keeps comment counts accurate at scale

The Problem Statement

Interviewer: "A viral YouTube video is getting 50,000 new comments per second. You need to display an accurate comment count next to the video. How do you keep that counter accurate without melting your database?"

This question tests three things: your understanding of write contention on a single database row, your knowledge of distributed counter patterns like sharded counters, and whether you can articulate the tradeoff between perfect accuracy and system availability at massive scale.

I love this question because it seems trivially simple on the surface. "Just increment a counter, right?" But the moment you try to increment a single row at 50K writes per second, you discover why every large-scale system uses a completely different approach. The naive solution is the one that breaks production.

Counting is one of the hardest problems in distributed systems. Not because the math is hard, but because single-row mutations at high throughput create lock contention that cascades into timeouts, retries, and eventually an outage. Understanding this is what separates a textbook answer from a production-ready one.

Clarifying the Scenario

You: "Good question. Let me scope this before I dive in."

You: "When you say 'accurate,' are we talking about exactly right at every millisecond, or within a few seconds of the true count? Those lead to very different architectures."

Interviewer: "Eventually consistent is fine. Within 5 seconds of the true value."

You: "Got it. And is this just comment counts, or should I also handle like counts, view counts, and reply counts?"

Interviewer: "Focus on comment counts, but I want to see a pattern that generalizes."

You: "One more: should I handle the case where comments are deleted or marked as spam? That means decrements, not just increments."

Interviewer: "Yes, include that."

You: "OK. I will structure my answer in three parts: why a single-row counter breaks at scale, sharded counters that spread writes across many rows, and the anti-spam reconciliation flow that handles the count going down when spam is removed."

My Approach

I break this into five parts:

Why single-row counters fail: Lock contention on a single row turns 50K writes/second into a serialized bottleneck where each write waits for the previous one to release the row lock
Sharded counters: Instead of one counter row, create N counter shards. Each write picks a random shard and increments it. Reads sum all shards
Periodic rollup: A background job sums all shards into a snapshot, then resets them. This keeps reads fast and prevents shard proliferation
Anti-spam reconciliation: When spam is detected retroactively, the count must go down. This introduces negative shard values, batch re-counts, and timing challenges
Eventual consistency and read divergence: Why two users looking at the same video see different counts, and why that is acceptable

The mental model is straightforward: spread writes across many rows to avoid contention, then consolidate reads with background aggregation. This is the same pattern Google uses internally with Spanner, and it shows up in DynamoDB, Bigtable, and every large-scale counter system.

YouTube processes over 500 hours of video uploaded per minute and billions of comments per day. A single viral video can receive 50,000+ comments per second during a live event. At this scale, even a well-indexed single-row counter creates a write hotspot that cascades into broader database contention.

The Architecture

Here is the full counter architecture showing how writes fan out across shards and how reads reconstruct the count.

The walkthrough:

User A posts a comment. The write handler picks a random shard (say shard 2) and increments it by 1. There is no lock contention because shard 2 is only one of N shards, so the probability of two writes hitting the same shard simultaneously is 1/N.
User C reads the comment count. The read handler fetches the snapshot value (9,847,231, consolidated 5 seconds ago) and adds the sum of all active shards (+14 +9 +11 -2 +7 = +39). It returns 9,847,270.
The rollup job fires every 5 seconds. It sums all active shards (+39), adds that to the snapshot (9,847,231 + 39 = 9,847,270), writes the new snapshot, and resets all shards to 0.
Spam detection removes comments. When the ML classifier flags a comment as spam, the system decrements a random shard by 1. Shard 4 shows -2 because two spam comments were removed since the last rollup. The math works cleanly because the rollup sums all values, positive and negative.

For the interview: say you would use N=64 shards for a hot video. At 50K writes/sec, each shard sees about 780 writes/sec, which is well within the throughput limit of a single Bigtable or Spanner row.

Sharded Counters and the Hot-Key Problem

This is the core of the solution and the part that distinguishes a strong answer from a weak one. The hot-key problem is simple: when many concurrent writes target the same database row, they serialize on the row lock. At 50K writes/sec, each write waits for the previous lock to release. Effective throughput drops to maybe 1,000 to 2,000 writes/sec, and everything else queues up, times out, and retries, creating a cascading failure.

Sharded counters solve this by splitting the single hot key into N independent keys. Each write picks one shard at random and increments only that shard. The write throughput is N times higher because contention is divided by N.

The key design decisions:

How many shards? Too few and you still have contention. Too many and reads become expensive (summing more rows). Start with 64 shards for high-traffic videos and 4 shards for normal videos. Dynamically adjust based on write rate: a background monitor detects high write latency and doubles the shard count. When traffic subsides, halve them.

Random vs hash-based shard selection? Random distributes writes more evenly, which is what we want for counters. Hash-based (on request ID or user ID) is deterministic but can produce skew if certain hash buckets are overrepresented. Use random.

What storage system? Bigtable and Spanner are natural fits because they handle high write throughput per row. DynamoDB works with its atomic increment operations. Redis is fast but not durable by default, so a restart loses the count. Use a durable store for the authoritative counter, Redis as a read cache.

The rollup. A background job runs every 5 seconds: read all shards atomically, sum them, add the sum to the snapshot row, reset all shards to 0. Between rollups, the read path computes snapshot + SUM(active shards). Maximum staleness equals the rollup interval (5 seconds), which is imperceptible to users.

A common interview mistake: "Just use Redis INCR." Redis handles 100K+ increments per second on a single key, which sounds sufficient. But Redis is single-threaded per shard, so a hot key blocks other operations on that shard. More importantly, Redis is not durable by default. A restart loses the count. Sharded counters in a durable store (Bigtable, Spanner) are the correct production answer. Use Redis as a read cache, not as the primary counter.

Anti-Spam Reconciliation: When Counts Go Down

Here is where it gets interesting. Comments do not only accumulate. They also get deleted, flagged as spam by ML classifiers, removed by moderators, or retracted by the author. Every removal must decrement the counter, and the timing of that decrement creates subtle consistency challenges.

The anti-spam pipeline runs asynchronously. An ML classifier evaluates each new comment and may flag it as spam minutes, hours, or even days after it was posted. When a batch of comments is removed, the counter must be adjusted downward.

The decrement path works just like the increment path but in reverse. When the spam classifier flags a comment, the system picks a random shard and decrements it by 1. The shard can go negative, which is fine. The rollup sums all values (positive and negative) into the snapshot.

Batch removal is more interesting. When the ML classifier does a sweep and flags 10,000 comments as spam at once, you have two choices: issue 10,000 individual decrements (one per shard pick), or issue a single decrement of -10,000 to one shard. The batch approach is more efficient but requires the classifier to know the exact count being removed. I prefer the batch approach with a safeguard: the reconciliation job catches any discrepancies.

The reconciliation job is the safety net. It runs hourly for popular videos, daily for others. It queries the comments table directly: SELECT COUNT(*) FROM comments WHERE video_id = 'abc' AND status = 'visible'. If the counter value differs from the actual row count by more than a threshold (say, 0.1%), the reconciliation job writes a correction delta to a shard.

My recommendation for interviews: mention the reconciliation job proactively. It shows you understand that distributed counters can drift from reality due to race conditions, crashed processes, or double-counting. The reconciliation job is your "source of truth audit."

The reconciliation job is "extra credit" in interviews. Most candidates describe sharded counters and stop. Mentioning reconciliation shows you understand that distributed systems drift, and you design for that reality. Say: "I would reconcile hourly against the source of truth to correct any counter drift from failed increments or race conditions."

Eventual Consistency and Why Your Count Differs from Mine

Here is the part that trips up candidates who think "eventually consistent" means "sloppy." Eventual consistency is a deliberate architectural choice that buys you 1000x write throughput. Two users looking at the same video at the same second may see different comment counts. This is by design, and it is perfectly acceptable.

There are five reasons counts diverge between users:

CDN caching. The comment count endpoint is cached at CDN edge locations with a 2 to 5 second TTL. User A in New York hits a CDN PoP with a cache entry from 3 seconds ago. User B in London hits a different PoP with a cache entry from 1 second ago. They see different counts.

Rollup timing. The rollup fires every 5 seconds. Between rollups, the read path sums snapshot plus active shards. If User A reads 1 second after a rollup and User B reads 4 seconds after, the shard values have accumulated differently.

Database replica lag. Reads may hit different database replicas with slightly different replication states. Even with strong consistency on the primary, read replicas lag by 10 to 100ms.

Spam filter timing. The ML classifier runs asynchronously. User A might see the count before a spam batch is removed. User B reads after the removal. The count went down for User B but not for User A yet.

Read-your-own-writes for the commenter. When you post a comment, you should see the count go up by 1 immediately. Other users do not see your comment reflected until the next rollup.

The read-your-own-writes pattern deserves specific attention. When User A posts a comment, I store a per-session adjustment in a fast cache (Redis or even in-memory on the app server). When User A reads the count, the app server adds their personal adjustment to the global count. The adjustment has a TTL slightly longer than the rollup interval (10 seconds for a 5-second rollup). After the rollup absorbs User A's increment, the adjustment expires naturally and the global count reflects their comment.

This technique is documented in Google Cloud Spanner's official guidance for sharded counters. It works for any eventually consistent counter system.

The negative count bug. Here is a subtle edge case: if spam removal decrements the count below zero, you should floor it at 0 in the display layer. A race between a comment addition and spam batch deletion can temporarily produce a negative shard sum. The storage layer allows negative values (necessary for correct math), but the API layer should never return a negative comment count. Add a MAX(0, computed_count) guard in the read path.

The "views" counter on YouTube uses a different approach than comments. Views are counted approximately using HyperLogLog (probabilistic cardinality estimation) because exact deduplication of billions of views is prohibitively expensive. Comment counts need to be exact (eventually) because users mentally verify them: "I posted a comment, the count should go up by 1." Views are too large for anyone to notice a 0.1% error.

The Tricky Parts

Counter drift after crashes. If a shard increment succeeds but the comment write fails (or vice versa), the counter drifts from reality. I use a two-phase approach: write the comment first, then increment the shard. If the increment fails, the reconciliation job catches the discrepancy. This makes the counter slightly lag reality rather than lead it, which is safer.
Hot video shard rebalancing. A newly viral video starts with 4 shards. Once it goes viral, 4 shards cannot handle 50K writes/sec. The system needs a "shard expansion" mechanism: a background monitor detects high write latency and doubles the count. Old shards remain active (rolled up normally). New writes go to the expanded set. This must be seamless with no counter loss.
Comment count vs reply count vs thread count. YouTube displays multiple counters per video. Each needs its own shard set. Reply counts need per-parent-comment sharding. I use a compound shard key: video:abc:shard:7 for top-level comments, comment:xyz:shard:3 for replies to a specific comment.
Displaying counts across time zones and CDN caches. Users in different regions see different counts because they hit different CDN PoPs with different cache ages. A short TTL (2 to 5 seconds) on count endpoints minimizes divergence while still protecting the counter store.
The reconciliation query is expensive. SELECT COUNT(*) WHERE video_id = ? AND status = 'visible' can be slow for videos with millions of comments. I maintain a secondary exact counter (updated within a database transaction alongside the comment insert/delete) as the reconciliation source of truth. This trades write complexity for fast reconciliation reads.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Single-row counter	"Just increment a counter column"	Row lock contention at 50K writes/sec causes cascading timeouts	"Shard the counter across N rows, each write picks a random shard"
Redis as primary store	"Use Redis INCR, it handles 100K ops"	Redis is not durable, a restart loses the count, hot key blocks the shard	"Durable store (Bigtable, Spanner) for sharded counters. Redis as a read cache"
Ignoring decrements	"Just increment on new comments"	Spam removal, moderation, and user deletions make the count drift upward	"Shards support negative values. Deletes decrement a random shard. Reconciliation corrects drift"
Over-engineering consistency	"Use distributed transactions for every increment"	Transactions kill throughput at this scale	"Accept 5-second eventual consistency. Read-your-own-writes for the commenter"
No rollup strategy	"Sum all shards on every read"	64+ shard reads per request at 10M reads/sec overwhelms the store	"Periodic rollup consolidates into a snapshot. Reads are snapshot plus active delta"

How I Would Communicate This in an Interview

Here is how I would actually say this:

"The core problem is lock contention. If 50,000 comments per second all try to increment the same database row, you get a serialized bottleneck. Each write waits for the row lock. Effective throughput drops to maybe 2,000 writes/sec, and everything else queues up and times out.

The solution is sharded counters. Instead of one counter row per video, I create 64 counter shards. Each comment write picks a random shard and increments it. This spreads 50K writes across 64 rows, so each row sees about 780 writes/sec, well within tolerance.

For reads, I do not sum all 64 shards every time. A background rollup job runs every 5 seconds, sums all shards, writes the total into a snapshot row, and resets the shards. Reads return snapshot plus active shard delta. The count is at most 5 seconds stale.

For the commenter, I add read-your-own-writes: store a per-session adjustment so they see their own comment reflected immediately, even before the next rollup.

Spam removal is the interesting wrinkle. When the ML classifier flags comments retroactively, the system decrements a shard just like it increments one. Shards can go negative. An hourly reconciliation job compares the counter against the actual comment row count and corrects any drift."

Interview Cheat Sheet

Trigger: "Counter at scale" or "How would you count X" say "sharded counters with periodic rollup."
Hot-key problem: "A single counter row becomes a write bottleneck due to row-level locking. Shard the counter across N rows."
Shard count: "64 shards for hot resources, 4 for cold. Scale dynamically based on write velocity."
Rollup interval: "5-second rollup balances read freshness against overhead. YouTube and Instagram use 1 to 10 second intervals."
Read path formula: "Displayed count = snapshot + SUM(active shards). Maximum staleness equals the rollup interval."
Decrements: "Deletes and spam removal decrement a random shard. Shards can go negative. The math works cleanly."
Reconciliation: "Hourly job compares counter value against SELECT COUNT(*) from comments. Corrects drift from failed increments or race conditions."
Read-your-own-writes: "Per-user session adjustment with TTL of 2x the rollup interval. Added to the global count for that user only."
Storage choice: "Bigtable or Spanner for durable counters. Redis as read-through cache. Never Redis as the sole counter store."
CDN caching: "2 to 5 second TTL on count endpoints. Acceptable staleness avoids thundering herd on the counter store."

Test Your Understanding

Quick Recap

Single-row counters at high write throughput create lock contention that cascades into outages. This is the fundamental problem.
Sharded counters spread writes across N independent rows, dividing contention by N. Use 64 shards for hot videos, 4 for cold.
A periodic rollup job (every 5 seconds) consolidates shard values into a snapshot, keeping reads fast and shard values small.
The displayed count is snapshot + SUM(active shards), at most one rollup interval stale.
Anti-spam reconciliation handles retroactive comment removal by decrementing shards (which can go negative) and running hourly drift-correction against the source of truth.
Read-your-own-writes uses a per-session adjustment with TTL to give the commenter immediate feedback without breaking the eventually consistent model.
CDN caching with a 2 to 5 second TTL protects the counter store from thundering-herd reads while keeping displayed counts near real-time.
Counter drift (from failed increments, race conditions, or crashed processes) is corrected by the reconciliation job, which compares the counter against actual row counts.

Distributed counters in Spanner explores the same sharded counter pattern in Google's globally distributed database, where TrueTime enables consistent rollup coordination across regions.
CRDTs (Conflict-free Replicated Data Types) provide a theoretical foundation for counters that merge without coordination, which is the academic basis for grow-only and positive-negative counter patterns.
Write-behind caching applies a similar "buffer writes, flush periodically" strategy to general-purpose cache updates, not just counters.
Event sourcing takes the counter problem further by storing every increment as an event and deriving the count from the event log, giving full auditability at the cost of storage.
Hot key mitigation addresses the broader problem of traffic concentration on a single key, which is exactly what sharded counters solve for the counter-specific case.

title: "How YouTube comment counts stay accurate at 10M comments" description: "How YouTube avoids counting every write as a database increment using sharded counters, periodic rollups, and eventual-consistency read paths that converge within seconds." tags:

"situational"
"youtube"
"counters"
"scale" difficulty: "medium" category: "situational/architecture" order: 122 publishedAt: "2026-04-12" relatedArticles: []

The Problem Statement

Interviewer: "A viral YouTube video is getting 50,000 new comments per second. You need to display an accurate comment count next to the video. How do you keep that counter accurate without melting your database?"

This question tests three things: your understanding of write contention on a single database row, your knowledge of distributed counter patterns like sharded counters and periodic rollups, and whether you can articulate the tradeoff between perfect accuracy and system availability at massive scale.

The real lesson here is that counting is one of the hardest problems in distributed systems. Not because the math is hard, but because single-row mutations at high throughput create lock contention that cascades into timeouts, retries, and ultimately an outage. Understanding this is what separates a textbook answer from a production-ready one.

Clarifying the Scenario

You: "Good question. Let me make sure I scope this correctly before I dive in."

You: "When you say 'accurate,' are we talking about exactly right at every millisecond, or within a few seconds of the true count? Because those lead to very different architectures."

Interviewer: "Eventually consistent is fine. Within 5 seconds of the true value."

You: "Got it. And is this just comment counts, or should I also think about like counts, view counts, and reply counts?"

Interviewer: "Focus on comment counts, but I want to see a pattern that generalizes."

You: "One more: should I handle the case where comments are deleted or marked as spam? That would mean decrements, not just increments."

Interviewer: "Yes, include that."

You: "OK. I will structure my answer in three parts: first, why a single-row counter breaks at scale. Second, sharded counters that spread writes across many rows. Third, periodic rollups that consolidate shards into a stable snapshot, plus how the commenter sees their own comment reflected immediately."

My Approach

I break this into five parts:

Why single-row counters fail: Lock contention on a single row turns 50K writes/second into a serialized bottleneck where each write waits for the previous one to release the row lock.
Sharded counters: Instead of one counter row, create N counter shards. Each write picks a random shard and increments it. Reads sum all shards.
Periodic rollup: A background job periodically sums all shards into a snapshot value, then resets the shards. This keeps reads fast and prevents shard proliferation.
Handling decrements: Deletes and spam removals need to decrement the counter. This introduces negative shard values and rollup timing challenges.
Read-your-own-writes for the commenter: When I post a comment, I should see the count go up by 1 immediately, even if the global count is eventually consistent.

YouTube processes over 500 hours of video uploaded per minute and billions of comments per day. At this scale, even a well-indexed single-row counter would create a write hotspot. Google solved this problem with sharded counters in Bigtable and Spanner, and the pattern is now standard across the industry.

The Architecture

Here is the full counter architecture showing how writes fan out across shards and how reads reconstruct the count.

The walkthrough:

User A posts a comment. The write handler picks a random shard (say shard 2) and increments it by 1. There is no lock contention because shard 2 is only one of N shards, so the probability of two writes hitting the same shard simultaneously is 1/N.
User C reads the comment count. The read handler fetches the snapshot value (9,847,231, which was consolidated 5 seconds ago) and adds the sum of all active shards (+14 +9 +11 -2 +7 = +39). It returns 9,847,270 to the client.
The rollup job fires every 5 seconds. It sums all active shards (+39), adds that to the snapshot (9,847,231 + 39 = 9,847,270), writes the new snapshot, and resets all shards to 0.
Spam removal happens via shard 4. When a spam filter removes a comment, the system picks a shard and decrements it by 1. The shard can go negative. The rollup handles this correctly because it sums all values, positive and negative.

For the interview: say you would use N=64 shards for a video with millions of comments. At 50K writes/sec, each shard sees about 780 writes/sec, which is well within the throughput limit of a single Bigtable row.

Sharded Counter Architecture

This is the core of the solution, and the part that distinguishes a strong answer from a weak one. Let me walk through exactly how the sharding works, why the shard count matters, and what happens at the edges.

The key decisions in shard design:

How many shards? Too few and you still have contention. Too many and reads become expensive because you sum more rows. I would start with 64 shards for high-traffic videos and 4 shards for normal videos. You can dynamically adjust shard count based on write rate.

Random vs hash-based shard selection? Random selection distributes writes more evenly. Hash-based selection (on request ID or user ID) is more deterministic. I would use random for counters because we do not need to route the same user to the same shard.

What storage system? Bigtable and Spanner are natural fits because they handle high write throughput on individual rows. DynamoDB works too with its atomic increment operations. Redis also works for ephemeral counters, but you lose durability.

A common interview mistake is saying "just use Redis INCR." Redis can handle 100K+ increments per second on a single key, which sounds sufficient. But Redis is single-threaded per shard, so a hot key still blocks other operations on that shard. More importantly, Redis is not durable by default, so a restart loses the count. Sharded counters in a durable store (Bigtable, Spanner) are the correct production answer.

Periodic Rollup and Convergence

Sharded counters solve the write problem, but they create a read problem: summing 64 shards on every read adds latency and load. The rollup job solves this by periodically consolidating shards into a single snapshot, keeping reads fast.

The convergence model works like this:

Between rollups, the displayed count is snapshot + SUM(active_shards). This is always at most 5 seconds stale (the rollup interval), which YouTube considers acceptable for displayed counts.

During a rollup, there is a brief window where a write could land on a shard that was just read but not yet reset. To handle this, the rollup uses a "read-then-reset" atomic operation. In Bigtable, this is a ReadModifyWrite. In Spanner, this is a read-write transaction.

If the rollup job crashes, the shards accumulate but nothing is lost. The next rollup picks up all accumulated values. The read path still works because it always sums snapshot + shards. The count is correct, just slightly more expensive to read.

Read-Your-Own-Writes for the Commenter

Here is the subtle part that separates a good answer from a great one. When I post a comment, I want to see the count go up by 1 immediately. I do not want to wait 5 seconds for the rollup. This is the "read-your-own-writes" consistency requirement.

The challenge: the global read path returns an eventually consistent count. If I post a comment and immediately refresh, the count might not reflect my comment yet because it was written to a shard that has not been rolled up.

The technique: when a user posts a comment, the app server stores a per-session adjustment in a local cache (Redis or even in-memory). When that same user reads the count, the app adds their personal adjustment to the global count. The adjustment has a TTL slightly longer than the rollup interval (10 seconds for a 5-second rollup). After the rollup absorbs their increment, the adjustment expires and the global count naturally reflects their comment.

This is exactly how Google Cloud Spanner's official documentation recommends handling read-your-own-writes for sharded counters. The pattern works for any eventually consistent counter system.

The read-your-own-writes pattern is the "extra credit" answer in interviews. Most candidates stop at sharded counters and rollups. Mentioning session-local adjustments shows you understand the user experience implications of eventual consistency, which is exactly what senior and staff-level interviews look for.

The Tricky Parts

Handling deletes and spam removal. When a comment is deleted or flagged as spam, the counter must decrement. Shards can go negative, which is fine mathematically (the rollup sums all values). But if the rollup fires between the comment deletion and the shard decrement, the snapshot includes the now-deleted comment. The fix: always decrement through a shard, never modify the snapshot directly. The next rollup absorbs the negative value.
Counter drift after crashes. If a shard increment succeeds but the comment write fails (or vice versa), the counter drifts from reality. I would use a two-phase approach: write the comment first, then increment the shard. If the shard increment fails, a background reconciliation job compares actual comment count (via COUNT query) against the counter value and corrects drift. Run this hourly for popular videos, daily for others.
Hot video shard rebalancing. A newly viral video may start with 4 shards that were adequate when it was quiet. Once it goes viral, 4 shards cannot handle 50K writes/sec. The system needs a "shard expansion" mechanism: a background monitor detects high write latency on a video's shards and doubles the shard count. The old shards remain active (they will be rolled up normally), and new writes go to expanded shards. This must be seamless, with no counter loss.
Displaying counts across time zones and CDN caches. The rollup snapshot is cached at CDN edges for read efficiency. If different CDN PoPs cache different snapshot versions, users in different regions see different counts for the same video. This is usually acceptable (within seconds of convergence), but it confuses users in the same room comparing screens. The fix: use a short CDN TTL (2-5 seconds) for count endpoints.
Comment count vs reply count vs thread count. YouTube displays multiple counters (total comments, replies to a specific comment, thread counts). Each needs its own set of shards. Naive implementations share one counter, but reply counts need per-parent-comment sharding. I would use a compound shard key: video:abc:shard:7 for top-level comments, and comment:xyz:shard:3 for replies to a specific comment.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Single-row counter	"Just increment a counter in the database"	Row lock contention at 50K writes/sec causes cascading timeouts	"Shard the counter across N rows, each write picks a random shard"
Redis as primary counter	"Use Redis INCR, it handles 100K ops"	Redis is not durable by default, and a hot key still blocks the shard	"Use a durable store like Bigtable with sharded counters. Redis works as a read cache"
Ignoring decrements	"Just increment on new comments"	Comments get deleted, marked as spam, or moderated. Counter drifts	"Shards support negative values. Deletes decrement a random shard"
Over-engineering consistency	"Use a distributed transaction to keep the count exactly right"	Distributed transactions kill throughput at this scale	"Accept 5-second eventual consistency. Use read-your-own-writes for the commenter"
No rollup strategy	"Just sum all shards on every read"	Summing 64+ shards per read at 10M reads/sec overwhelms the store	"Periodic rollup consolidates shards into a snapshot. Reads are snapshot + active delta"

How I Would Communicate This in an Interview

Here is how I would actually say this:

"The core problem with counting at scale is lock contention. If 50,000 comments per second all try to increment the same database row, you get a serialized bottleneck where each write waits for the row lock. The effective throughput drops to maybe 2,000 writes/sec, and everything else queues up and times out.

The solution is sharded counters. Instead of one counter row per video, I create 64 counter shards. Each comment write picks a random shard and increments it. This spreads the 50K writes across 64 rows, so each row sees about 780 writes/sec, which is well within tolerance.

For reads, I do not sum all 64 shards every time. That would be expensive at scale. Instead, a background rollup job runs every 5 seconds, sums all the shards, writes the total into a snapshot row, and resets the shards to zero. Reads return snapshot plus the sum of any shards accumulated since the last rollup. The count is at most 5 seconds stale, which is perfectly acceptable for a displayed count.

The one refinement I would add is read-your-own-writes for the commenter. When someone posts a comment, I store a per-session adjustment so they see their own comment reflected immediately, even before the next rollup."

Interview Cheat Sheet

Trigger: "Counter at scale" or "How would you count X" → say "sharded counters with periodic rollup."
Single row hotspot: "A single counter row becomes a write bottleneck due to row-level locking. The fix is to shard the counter across N rows."
Shard count guideline: "Start with 64 shards for hot resources, 4 for cold. Scale dynamically based on write velocity."
Rollup interval: "5-second rollup balances read freshness against rollup overhead. YouTube and Instagram use 1-10 second intervals."
Read path formula: "Displayed count = snapshot + SUM(active shards). Maximum staleness equals the rollup interval."
Decrements: "Deletes and spam removal decrement a random shard. Shards can go negative. The math works cleanly."
Read-your-own-writes: "Store a per-user session adjustment with TTL slightly longer than the rollup interval. Add it to the global count for that user only."
Counter drift: "If writes and increments are not atomically paired, a reconciliation job periodically compares COUNT(*) against the counter value and corrects drift."
Storage choice: "Bigtable or Spanner for durable sharded counters. Redis as a read-through cache for the snapshot value."
CDN caching: "Cache the count endpoint with a 2-5 second TTL at the CDN edge. Acceptable staleness avoids thundering herd on the counter store."

Test Your Understanding

Quick Recap

Single-row counters at high write throughput create lock contention that cascades into outages.
Sharded counters spread writes across N independent rows, dividing contention by N.
A periodic rollup job (every 5 seconds) consolidates shard values into a snapshot, keeping reads fast.
The displayed count is snapshot + SUM(active shards), which is at most one rollup interval stale.
Decrements (deletes, spam removal) work identically: write a negative value to a random shard.
Read-your-own-writes uses a per-session adjustment with TTL to give the commenter immediate feedback.
Dynamic shard allocation scales shard count up for viral videos and down for quiet ones.
Counter drift is corrected by periodic reconciliation jobs that compare the counter against actual row counts.

Distributed counters in Spanner explore the same sharded counter pattern in Google's globally distributed database, where TrueTime enables consistent rollup coordination.
CRDTs (Conflict-free Replicated Data Types) offer a theoretical foundation for counters that merge without coordination, which is the academic basis for the grow-only and positive-negative counter patterns used here.
Write-behind caching applies a similar "buffer writes, flush periodically" strategy to general-purpose cache updates, not just counters.
Event sourcing takes the counter problem further by storing every increment as an event and deriving the count from the event log, which gives you full auditability at the cost of storage.
Hot key mitigation addresses the broader problem of traffic concentration on a single key, which is exactly what sharded counters solve for the counter-specific case.

The Problem Statement

Interviewer: "A viral YouTube video is getting 50,000 new comments per second. You need to display an accurate comment count next to the video. How do you keep that counter accurate without melting your database?"

Clarifying the Scenario

You: "Good question. Let me scope this before I dive in."

You: "When you say 'accurate,' are we talking about exactly right at every millisecond, or within a few seconds of the true count? Those lead to very different architectures."

Interviewer: "Eventually consistent is fine. Within 5 seconds of the true value."

You: "Got it. And is this just comment counts, or should I also handle like counts, view counts, and reply counts?"

Interviewer: "Focus on comment counts, but I want to see a pattern that generalizes."

You: "One more: should I handle the case where comments are deleted or marked as spam? That means decrements, not just increments."

Interviewer: "Yes, include that."

My Approach

I break this into five parts:

Why single-row counters fail: Lock contention on a single row turns 50K writes/second into a serialized bottleneck where each write waits for the previous one to release the row lock
Sharded counters: Instead of one counter row, create N counter shards. Each write picks a random shard and increments it. Reads sum all shards
Periodic rollup: A background job sums all shards into a snapshot, then resets them. This keeps reads fast and prevents shard proliferation
Anti-spam reconciliation: When spam is detected retroactively, the count must go down. This introduces negative shard values, batch re-counts, and timing challenges
Eventual consistency and read divergence: Why two users looking at the same video see different counts, and why that is acceptable

The Architecture

Here is the full counter architecture showing how writes fan out across shards and how reads reconstruct the count.

The walkthrough:

User A posts a comment. The write handler picks a random shard (say shard 2) and increments it by 1. There is no lock contention because shard 2 is only one of N shards, so the probability of two writes hitting the same shard simultaneously is 1/N.
User C reads the comment count. The read handler fetches the snapshot value (9,847,231, consolidated 5 seconds ago) and adds the sum of all active shards (+14 +9 +11 -2 +7 = +39). It returns 9,847,270.
The rollup job fires every 5 seconds. It sums all active shards (+39), adds that to the snapshot (9,847,231 + 39 = 9,847,270), writes the new snapshot, and resets all shards to 0.
Spam detection removes comments. When the ML classifier flags a comment as spam, the system decrements a random shard by 1. Shard 4 shows -2 because two spam comments were removed since the last rollup. The math works cleanly because the rollup sums all values, positive and negative.

Sharded Counters and the Hot-Key Problem

The key design decisions:

Anti-Spam Reconciliation: When Counts Go Down

Eventual Consistency and Why Your Count Differs from Mine

There are five reasons counts diverge between users:

Database replica lag. Reads may hit different database replicas with slightly different replication states. Even with strong consistency on the primary, read replicas lag by 10 to 100ms.

Read-your-own-writes for the commenter. When you post a comment, you should see the count go up by 1 immediately. Other users do not see your comment reflected until the next rollup.

This technique is documented in Google Cloud Spanner's official guidance for sharded counters. It works for any eventually consistent counter system.

The Tricky Parts

Counter drift after crashes. If a shard increment succeeds but the comment write fails (or vice versa), the counter drifts from reality. I use a two-phase approach: write the comment first, then increment the shard. If the increment fails, the reconciliation job catches the discrepancy. This makes the counter slightly lag reality rather than lead it, which is safer.
Hot video shard rebalancing. A newly viral video starts with 4 shards. Once it goes viral, 4 shards cannot handle 50K writes/sec. The system needs a "shard expansion" mechanism: a background monitor detects high write latency and doubles the count. Old shards remain active (rolled up normally). New writes go to the expanded set. This must be seamless with no counter loss.
Comment count vs reply count vs thread count. YouTube displays multiple counters per video. Each needs its own shard set. Reply counts need per-parent-comment sharding. I use a compound shard key: video:abc:shard:7 for top-level comments, comment:xyz:shard:3 for replies to a specific comment.
Displaying counts across time zones and CDN caches. Users in different regions see different counts because they hit different CDN PoPs with different cache ages. A short TTL (2 to 5 seconds) on count endpoints minimizes divergence while still protecting the counter store.
The reconciliation query is expensive. SELECT COUNT(*) WHERE video_id = ? AND status = 'visible' can be slow for videos with millions of comments. I maintain a secondary exact counter (updated within a database transaction alongside the comment insert/delete) as the reconciliation source of truth. This trades write complexity for fast reconciliation reads.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Single-row counter	"Just increment a counter column"	Row lock contention at 50K writes/sec causes cascading timeouts	"Shard the counter across N rows, each write picks a random shard"
Redis as primary store	"Use Redis INCR, it handles 100K ops"	Redis is not durable, a restart loses the count, hot key blocks the shard	"Durable store (Bigtable, Spanner) for sharded counters. Redis as a read cache"
Ignoring decrements	"Just increment on new comments"	Spam removal, moderation, and user deletions make the count drift upward	"Shards support negative values. Deletes decrement a random shard. Reconciliation corrects drift"
Over-engineering consistency	"Use distributed transactions for every increment"	Transactions kill throughput at this scale	"Accept 5-second eventual consistency. Read-your-own-writes for the commenter"
No rollup strategy	"Sum all shards on every read"	64+ shard reads per request at 10M reads/sec overwhelms the store	"Periodic rollup consolidates into a snapshot. Reads are snapshot plus active delta"

How I Would Communicate This in an Interview

Here is how I would actually say this:

For the commenter, I add read-your-own-writes: store a per-session adjustment so they see their own comment reflected immediately, even before the next rollup.

Interview Cheat Sheet

Trigger: "Counter at scale" or "How would you count X" say "sharded counters with periodic rollup."
Hot-key problem: "A single counter row becomes a write bottleneck due to row-level locking. Shard the counter across N rows."
Shard count: "64 shards for hot resources, 4 for cold. Scale dynamically based on write velocity."
Rollup interval: "5-second rollup balances read freshness against overhead. YouTube and Instagram use 1 to 10 second intervals."
Read path formula: "Displayed count = snapshot + SUM(active shards). Maximum staleness equals the rollup interval."
Decrements: "Deletes and spam removal decrement a random shard. Shards can go negative. The math works cleanly."
Reconciliation: "Hourly job compares counter value against SELECT COUNT(*) from comments. Corrects drift from failed increments or race conditions."
Read-your-own-writes: "Per-user session adjustment with TTL of 2x the rollup interval. Added to the global count for that user only."
Storage choice: "Bigtable or Spanner for durable counters. Redis as read-through cache. Never Redis as the sole counter store."
CDN caching: "2 to 5 second TTL on count endpoints. Acceptable staleness avoids thundering herd on the counter store."

Test Your Understanding

Quick Recap

Single-row counters at high write throughput create lock contention that cascades into outages. This is the fundamental problem.
Sharded counters spread writes across N independent rows, dividing contention by N. Use 64 shards for hot videos, 4 for cold.
A periodic rollup job (every 5 seconds) consolidates shard values into a snapshot, keeping reads fast and shard values small.
The displayed count is snapshot + SUM(active shards), at most one rollup interval stale.
Anti-spam reconciliation handles retroactive comment removal by decrementing shards (which can go negative) and running hourly drift-correction against the source of truth.
Read-your-own-writes uses a per-session adjustment with TTL to give the commenter immediate feedback without breaking the eventually consistent model.
CDN caching with a 2 to 5 second TTL protects the counter store from thundering-herd reads while keeping displayed counts near real-time.
Counter drift (from failed increments, race conditions, or crashed processes) is corrected by the reconciliation job, which compares the counter against actual row counts.

Distributed counters in Spanner explores the same sharded counter pattern in Google's globally distributed database, where TrueTime enables consistent rollup coordination across regions.
CRDTs (Conflict-free Replicated Data Types) provide a theoretical foundation for counters that merge without coordination, which is the academic basis for grow-only and positive-negative counter patterns.
Write-behind caching applies a similar "buffer writes, flush periodically" strategy to general-purpose cache updates, not just counters.
Event sourcing takes the counter problem further by storing every increment as an event and deriving the count from the event log, giving full auditability at the cost of storage.
Hot key mitigation addresses the broader problem of traffic concentration on a single key, which is exactly what sharded counters solve for the counter-specific case.

"situational"
"youtube"
"counters"
"scale" difficulty: "medium" category: "situational/architecture" order: 122 publishedAt: "2026-04-12" relatedArticles: []

The Problem Statement

Interviewer: "A viral YouTube video is getting 50,000 new comments per second. You need to display an accurate comment count next to the video. How do you keep that counter accurate without melting your database?"

Clarifying the Scenario

You: "Good question. Let me make sure I scope this correctly before I dive in."

You: "When you say 'accurate,' are we talking about exactly right at every millisecond, or within a few seconds of the true count? Because those lead to very different architectures."

Interviewer: "Eventually consistent is fine. Within 5 seconds of the true value."

You: "Got it. And is this just comment counts, or should I also think about like counts, view counts, and reply counts?"

Interviewer: "Focus on comment counts, but I want to see a pattern that generalizes."

You: "One more: should I handle the case where comments are deleted or marked as spam? That would mean decrements, not just increments."

Interviewer: "Yes, include that."

My Approach

I break this into five parts:

Why single-row counters fail: Lock contention on a single row turns 50K writes/second into a serialized bottleneck where each write waits for the previous one to release the row lock.
Sharded counters: Instead of one counter row, create N counter shards. Each write picks a random shard and increments it. Reads sum all shards.
Periodic rollup: A background job periodically sums all shards into a snapshot value, then resets the shards. This keeps reads fast and prevents shard proliferation.
Handling decrements: Deletes and spam removals need to decrement the counter. This introduces negative shard values and rollup timing challenges.
Read-your-own-writes for the commenter: When I post a comment, I should see the count go up by 1 immediately, even if the global count is eventually consistent.

The Architecture

Here is the full counter architecture showing how writes fan out across shards and how reads reconstruct the count.

The walkthrough:

User A posts a comment. The write handler picks a random shard (say shard 2) and increments it by 1. There is no lock contention because shard 2 is only one of N shards, so the probability of two writes hitting the same shard simultaneously is 1/N.
User C reads the comment count. The read handler fetches the snapshot value (9,847,231, which was consolidated 5 seconds ago) and adds the sum of all active shards (+14 +9 +11 -2 +7 = +39). It returns 9,847,270 to the client.
The rollup job fires every 5 seconds. It sums all active shards (+39), adds that to the snapshot (9,847,231 + 39 = 9,847,270), writes the new snapshot, and resets all shards to 0.
Spam removal happens via shard 4. When a spam filter removes a comment, the system picks a shard and decrements it by 1. The shard can go negative. The rollup handles this correctly because it sums all values, positive and negative.

Handling deletes and spam removal. When a comment is deleted or flagged as spam, the counter must decrement. Shards can go negative, which is fine mathematically (the rollup sums all values). But if the rollup fires between the comment deletion and the shard decrement, the snapshot includes the now-deleted comment. The fix: always decrement through a shard, never modify the snapshot directly. The next rollup absorbs the negative value.
Counter drift after crashes. If a shard increment succeeds but the comment write fails (or vice versa), the counter drifts from reality. I would use a two-phase approach: write the comment first, then increment the shard. If the shard increment fails, a background reconciliation job compares actual comment count (via COUNT query) against the counter value and corrects drift. Run this hourly for popular videos, daily for others.
Hot video shard rebalancing. A newly viral video may start with 4 shards that were adequate when it was quiet. Once it goes viral, 4 shards cannot handle 50K writes/sec. The system needs a "shard expansion" mechanism: a background monitor detects high write latency on a video's shards and doubles the shard count. The old shards remain active (they will be rolled up normally), and new writes go to expanded shards. This must be seamless, with no counter loss.
Displaying counts across time zones and CDN caches. The rollup snapshot is cached at CDN edges for read efficiency. If different CDN PoPs cache different snapshot versions, users in different regions see different counts for the same video. This is usually acceptable (within seconds of convergence), but it confuses users in the same room comparing screens. The fix: use a short CDN TTL (2-5 seconds) for count endpoints.
Comment count vs reply count vs thread count. YouTube displays multiple counters (total comments, replies to a specific comment, thread counts). Each needs its own set of shards. Naive implementations share one counter, but reply counts need per-parent-comment sharding. I would use a compound shard key: video:abc:shard:7 for top-level comments, and comment:xyz:shard:3 for replies to a specific comment.

What Most People Get Wrong

Mistake	What they say	Why it is wrong	What to say instead
Single-row counter	"Just increment a counter in the database"	Row lock contention at 50K writes/sec causes cascading timeouts	"Shard the counter across N rows, each write picks a random shard"
Redis as primary counter	"Use Redis INCR, it handles 100K ops"	Redis is not durable by default, and a hot key still blocks the shard	"Use a durable store like Bigtable with sharded counters. Redis works as a read cache"
Ignoring decrements	"Just increment on new comments"	Comments get deleted, marked as spam, or moderated. Counter drifts	"Shards support negative values. Deletes decrement a random shard"
Over-engineering consistency	"Use a distributed transaction to keep the count exactly right"	Distributed transactions kill throughput at this scale	"Accept 5-second eventual consistency. Use read-your-own-writes for the commenter"
No rollup strategy	"Just sum all shards on every read"	Summing 64+ shards per read at 10M reads/sec overwhelms the store	"Periodic rollup consolidates shards into a snapshot. Reads are snapshot + active delta"

How I Would Communicate This in an Interview

Here is how I would actually say this:

Interview Cheat Sheet

Trigger: "Counter at scale" or "How would you count X" → say "sharded counters with periodic rollup."
Single row hotspot: "A single counter row becomes a write bottleneck due to row-level locking. The fix is to shard the counter across N rows."
Shard count guideline: "Start with 64 shards for hot resources, 4 for cold. Scale dynamically based on write velocity."
Rollup interval: "5-second rollup balances read freshness against rollup overhead. YouTube and Instagram use 1-10 second intervals."
Read path formula: "Displayed count = snapshot + SUM(active shards). Maximum staleness equals the rollup interval."
Decrements: "Deletes and spam removal decrement a random shard. Shards can go negative. The math works cleanly."
Read-your-own-writes: "Store a per-user session adjustment with TTL slightly longer than the rollup interval. Add it to the global count for that user only."
Counter drift: "If writes and increments are not atomically paired, a reconciliation job periodically compares COUNT(*) against the counter value and corrects drift."
Storage choice: "Bigtable or Spanner for durable sharded counters. Redis as a read-through cache for the snapshot value."
CDN caching: "Cache the count endpoint with a 2-5 second TTL at the CDN edge. Acceptable staleness avoids thundering herd on the counter store."

Test Your Understanding

Quick Recap

Single-row counters at high write throughput create lock contention that cascades into outages.
Sharded counters spread writes across N independent rows, dividing contention by N.
A periodic rollup job (every 5 seconds) consolidates shard values into a snapshot, keeping reads fast.
The displayed count is snapshot + SUM(active shards), which is at most one rollup interval stale.
Decrements (deletes, spam removal) work identically: write a negative value to a random shard.
Read-your-own-writes uses a per-session adjustment with TTL to give the commenter immediate feedback.
Dynamic shard allocation scales shard count up for viral videos and down for quiet ones.
Counter drift is corrected by periodic reconciliation jobs that compare the counter against actual row counts.

Distributed counters in Spanner explore the same sharded counter pattern in Google's globally distributed database, where TrueTime enables consistent rollup coordination.
CRDTs (Conflict-free Replicated Data Types) offer a theoretical foundation for counters that merge without coordination, which is the academic basis for the grow-only and positive-negative counter patterns used here.
Write-behind caching applies a similar "buffer writes, flush periodically" strategy to general-purpose cache updates, not just counters.
Event sourcing takes the counter problem further by storing every increment as an event and deriving the count from the event log, which gives you full auditability at the cost of storage.
Hot key mitigation addresses the broader problem of traffic concentration on a single key, which is exactly what sharded counters solve for the counter-specific case.