How video streaming delivers content to your screen

The Interview Question

Interviewer: "Your team is building a video platform that needs to serve millions of concurrent viewers across different devices and network conditions. Walk me through end to end what happens from the moment a user presses play until pixels appear on screen. How does the system adapt when their network drops from 20 Mbps to 2 Mbps mid-stream?"

This question tests whether you understand the full video delivery pipeline: encoding, segmentation, manifest files, CDN caching, adaptive bitrate switching, and buffer management. The interviewer wants to hear concrete protocol details (HLS, DASH), not just "we use a CDN."

What to Clarify Before Answering

You: "Before I trace the full pipeline, let me scope a few things..."

• "Are we talking about video-on-demand (pre-encoded files) or live streaming with real-time encoding?"
• "Should I cover the encoding and transcoding pipeline, or start from when the encoded segments are already stored?"
• "Do you want me to include DRM and content protection, or focus on the delivery mechanics?"
• "Should I address low-latency live streaming (LL-HLS, WebRTC) or standard latency delivery?"
• "Are we targeting browser-based players only, or also native mobile and smart TV apps?"

Why this matters: Video streaming has very different architectures depending on whether the content is live or pre-recorded, whether DRM is required, and what latency targets you need. Scoping prevents you from spending five minutes on encoding when the interviewer wants to hear about adaptive bitrate logic.

The 30-Second Answer

Video streaming works by encoding the source video into multiple quality levels (an "encoding ladder"), segmenting each quality into small 2-10 second chunks, and describing those chunks in a manifest file (m3u8 for HLS, mpd for DASH). When a user presses play, the player fetches the manifest, selects an initial bitrate based on estimated bandwidth, and starts downloading segments. A CDN caches these segments at edge locations close to viewers, reducing latency and origin load. The player continuously monitors download throughput and buffer health, switching to higher or lower bitrates using an adaptive bitrate (ABR) algorithm. The goal is to maximize quality while avoiding rebuffering, which is the single biggest factor in viewer experience.

The Architecture Overview

Looking at the diagram above, the pipeline splits into three domains. The ingest layer encodes and packages video into segments. The CDN layer caches and distributes those segments globally. The client player layer makes real-time decisions about which quality level to fetch next.

I find this architecture elegant because each layer operates independently. The encoder does not know anything about the viewer's network. The CDN does not know which bitrate the player will request. The player makes all quality decisions locally using its own bandwidth estimation and buffer state.

Video Encoding: Building the Quality Ladder

The encoding ladder is the foundation of adaptive streaming. For each piece of source content, the transcoder produces multiple renditions at different resolution and bitrate combinations.

A typical encoding ladder for a streaming service looks like this:

Codecs: The Compression War

H.264/AVC remains the universal baseline. Every device shipped in the last 15 years can decode it with hardware acceleration. The tradeoff is that it needs roughly 40% more bandwidth than newer codecs for the same visual quality.

H.265/HEVC cuts bitrate by 35-50% over H.264 at the same quality. The problem is licensing. HEVC has a fragmented patent pool that has historically made companies nervous about royalty costs. Despite this, it dominates 4K content delivery.

VP9 is Google's royalty-free answer to HEVC. YouTube uses VP9 extensively. It achieves similar compression efficiency to HEVC. Browser support is strong in Chrome and Firefox, but absent in Safari.

AV1 is the newest generation, developed by the Alliance for Open Media (Google, Netflix, Amazon, Apple, Microsoft). It achieves 30% better compression than HEVC with no royalties. The catch is that encoding is extremely slow (10-100x slower than H.264), making it practical only for VOD with offline encoding. Hardware decode support is arriving in devices from 2022 onward.

Segment Structure

Each rendition is split into fixed-duration segments (typically 2-6 seconds). Shorter segments enable faster bitrate switching but increase manifest size and HTTP request overhead. Longer segments are more efficient for CDN caching but make the player slower to adapt.

// Segment file naming convention
video/
  720p/
    segment_001.ts    // 2 seconds, ~750KB
    segment_002.ts
    segment_003.ts
    ...
  1080p/
    segment_001.ts    // 2 seconds, ~1.5MB
    segment_002.ts
    ...

I typically recommend 4-second segments for VOD content and 2-second segments for live streaming. This balances adaptation speed with network efficiency.

HLS vs DASH: Manifest Protocols

The manifest file is the control plane of video streaming. It tells the player what quality levels exist, where to find each segment, and how long each segment is.

HLS (HTTP Live Streaming)

HLS was created by Apple and uses .m3u8 playlist files. A master playlist points to variant playlists for each quality level:

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
360p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1280x720
720p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=6000000,RESOLUTION=1920x1080
1080p/playlist.m3u8

Each variant playlist lists the individual segments:

#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.000,
segment_000.ts
#EXTINF:4.000,
segment_001.ts
#EXTINF:4.000,
segment_002.ts

DASH (Dynamic Adaptive Streaming over HTTP)

DASH uses XML-based .mpd (Media Presentation Description) files. It is an international standard (ISO/IEC 23009-1) and is codec-agnostic, unlike HLS which historically required H.264.

<MPD type="static" mediaPresentationDuration="PT1H30M">
  <Period>
    <AdaptationSet mimeType="video/mp4" segmentAlignment="true">
      <Representation id="720p" bandwidth="3000000" width="1280" height="720">
        <SegmentTemplate media="720p/seg_$Number$.m4s" initialization="720p/init.mp4"
                         duration="4000" timescale="1000"/>
      </Representation>
      <Representation id="1080p" bandwidth="6000000" width="1920" height="1080">
        <SegmentTemplate media="1080p/seg_$Number$.m4s" initialization="1080p/init.mp4"
                         duration="4000" timescale="1000"/>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

Comparison

I recommend using CMAF (Common Media Application Format) segments with both HLS and DASH manifests. CMAF uses fragmented MP4 (fMP4) instead of MPEG-TS, which means you store segments once and serve them to both protocols.

Adaptive Bitrate: The Brain of the Player

The ABR algorithm is what makes streaming feel seamless. It runs entirely on the client side and makes per-segment decisions about which quality level to download next.

ABR Algorithm Families

There are three main approaches to ABR:

Throughput-based: Measure the download speed of the last few segments and pick the highest bitrate that fits within estimated bandwidth. Simple and responsive, but prone to oscillation when bandwidth fluctuates.

Buffer-based (BBA): Ignore bandwidth estimation entirely. Instead, map the current buffer level directly to a bitrate. If the buffer is nearly full (30+ seconds), pick the highest quality. If the buffer is draining (under 10 seconds), drop to the lowest. Netflix popularized this approach in their BBA paper.

Hybrid: Combine throughput estimation with buffer level. Most production players use this approach. When the buffer is healthy, use throughput to maximize quality. When the buffer is low, be conservative regardless of throughput.

// Simplified hybrid ABR algorithm
function selectNextBitrate(buffer, throughput, renditions) {
  if (buffer < CRITICAL_THRESHOLD) {     // < 5 seconds
    return renditions[0]                  // Emergency: lowest quality
  }
  
  if (buffer < LOW_THRESHOLD) {           // < 10 seconds
    safeRate = throughput * 0.6           // Conservative: 60% of estimated BW
    return highestBelow(renditions, safeRate)
  }
  
  if (buffer > HIGH_THRESHOLD) {          // > 25 seconds
    safeRate = throughput * 0.9           // Aggressive: 90% of estimated BW
    return highestBelow(renditions, safeRate)
  }
  
  // Steady state: 80% of estimated bandwidth
  safeRate = throughput * 0.8
  return highestBelow(renditions, safeRate)
}

CDN Delivery: Getting Segments Close to Viewers

A video platform without a CDN is like a restaurant with one kitchen serving the entire country. CDNs solve the latency and capacity problem by caching content at edge locations close to viewers.

Cache Architecture

Video segments are ideal CDN content: they are immutable (a segment never changes once encoded), uniformly sized, and accessed in predictable patterns (segment N is always accessed after segment N-1).

Cache Key Design

The cache key for a video segment typically includes the content ID, quality level, and segment number:

cache_key = /{content_id}/{quality}/{segment_number}.m4s
// Example: /movie-12345/1080p/seg_00042.m4s

This structure means that popular segments (the first few segments of a trending show) achieve near-100% cache hit rates at the edge. Less-watched content or segments deep into a video may need to pull from the origin shield.

CDN Performance Numbers

Buffer Management: Preventing Rebuffering

The player's buffer is a queue of downloaded, decoded-ready segments. Buffer management is the difference between a smooth experience and the spinning wheel that makes users close the tab.

Buffer States

The target buffer length is typically 15-30 seconds for VOD. Live streaming uses shorter buffers (3-8 seconds) to keep latency low, at the cost of higher rebuffer risk.

Startup Optimization

The time from pressing play to the first frame appearing is called "time to first byte" or more precisely "join time." Netflix targets under 2 seconds. The strategy is:

1. Start with the lowest quality rendition (fast to download)
2. Fill a minimum buffer threshold (usually 2-4 seconds)
3. Begin playback immediately
4. Ramp up quality while playback continues

This means the first few seconds are always lower quality, but the user sees video immediately instead of staring at a loading spinner.

Live Streaming vs VOD: Key Differences

Live streaming introduces constraints that VOD does not have. The content does not exist yet when the viewer starts watching, so the entire encoding-to-playback pipeline must run in real-time.

Low-Latency Live Streaming

Standard HLS/DASH live streaming has 15-30 seconds of latency from the real-world event to the viewer's screen. This is fine for watching a movie premiere, but unacceptable for live sports betting or interactive events.

LL-HLS (Low-Latency HLS) reduces latency to 2-4 seconds by using partial segments and blocking playlist reloads. Instead of waiting for a full 6-second segment to be ready, the server pushes partial segments (200ms-1s chunks) and the player fetches them as they become available.

WebRTC pushes latency below 1 second by bypassing HTTP entirely and using UDP-based peer-to-peer or server-to-client connections. The tradeoff is that WebRTC does not scale naturally to millions of viewers (no CDN caching), so it requires a Selective Forwarding Unit (SFU) architecture.

DRM: Protecting Content

Digital Rights Management prevents unauthorized copying and distribution of premium content. Every major streaming service uses DRM, and it is a requirement for licensing content from studios.

The three major DRM systems are:

CENC (Common Encryption) allows encrypting content once and using different DRM license servers for different platforms. The encrypted segments are identical; only the license acquisition differs per DRM system.

The flow works like this: the player encounters encrypted segments, sends a license request to the DRM license server, receives decryption keys, and decrypts segments in a hardware-protected pipeline (Trusted Execution Environment) that prevents the decrypted video from being accessible to the application or operating system.

What Happens When Things Break

Performance Characteristics

How This Compares to Alternatives

I reach for HLS/DASH for any content delivery at scale (VOD or live). I use WebRTC only when sub-second latency is a hard requirement (live auctions, video conferencing, collaborative editing). RTMP is a legacy protocol; I would only encounter it as an ingest protocol from encoders to the packaging service, never for viewer-facing delivery.

Interview Cheat Sheet

• When asked about video delivery: "Video streaming works by encoding content into multiple quality levels, segmenting each into 2-6 second chunks described by a manifest file (m3u8 for HLS, mpd for DASH), and caching those segments on CDN edges close to viewers."
• When asked about adaptive bitrate: "The player runs an ABR algorithm that monitors estimated throughput and buffer health to select the best quality for each segment. Modern players use a hybrid of bandwidth estimation and buffer-level gating to minimize rebuffering while maximizing quality."
• When asked about HLS vs DASH: "They solve the same problem with different manifest formats. Most production systems serve both from the same CMAF segments. HLS is required for Apple devices, DASH is the open standard used everywhere else."
• When asked about live vs VOD: "Live streaming requires real-time encoding, shorter segment durations, smaller player buffers (3-8s vs 15-30s), and has inherent latency (5-30s standard, 2-4s with LL-HLS). VOD is encoded offline and fully CDN-cached."
• When asked about CDN for video: "Video segments are perfect CDN content: immutable, uniformly sized, accessed sequentially. Popular content achieves 95-99% cache hit rates at the edge. An origin shield layer absorbs 90%+ of cache misses before they reach storage."
• When asked about rebuffering: "Rebuffering is the number one viewer experience killer. It is prevented by maintaining a forward buffer of 15-30 seconds, using conservative ABR that values stability over maximum quality, and ensuring the lowest quality rendition works on the slowest expected connection."
• When asked about DRM: "CENC encrypts content once. Widevine (Chrome/Android), FairPlay (Safari/iOS), and PlayReady (Edge/Windows) each license differently but decrypt the same encrypted segments. Hardware-backed DRM (L1) is required for HD and 4K content from studios."
• When asked about encoding costs: "H.264 is cheap and universal. H.265 saves 40% bandwidth but has licensing complexity. AV1 saves another 30% but is 10-100x slower to encode, making it practical only for VOD with offline encoding. Netflix and YouTube are migrating to AV1 for high-traffic content."

Test Your Understanding

Quick Recap

1. Video streaming encodes source content into multiple quality renditions forming an "encoding ladder" with different resolution and bitrate combinations.
2. Each rendition is split into 2-6 second segments described by a manifest file (m3u8 for HLS, mpd for DASH) that tells the player where to find each chunk.
3. CDNs cache these immutable segments at edge locations close to viewers, achieving 95-99% cache hit rates for popular content.
4. The player's ABR algorithm continuously monitors throughput and buffer health to select the optimal quality for each segment, balancing visual quality against rebuffer risk.
5. Live streaming adds real-time encoding constraints, smaller buffers (3-8s vs 15-30s), and inherent latency (5-30s standard, 2-4s with LL-HLS, sub-second with WebRTC).
6. DRM systems (Widevine, FairPlay, PlayReady) protect content using CENC common encryption, with the DRM security level determining the maximum allowed resolution per device.
7. Modern codecs (H.265, VP9, AV1) offer 30-50% bandwidth savings over H.264 but come with tradeoffs in encoding cost, device support, and licensing complexity.
8. The number one viewer experience metric is rebuffering rate, which is minimized by conservative ABR algorithms, adequate buffer size, and ensuring the lowest quality rendition works on the slowest expected connection.

Related Concepts

• CDN and edge caching: Video segments are the ideal CDN workload. Understanding CDN cache hierarchies (edge, shield, origin) is essential for video platform design.
• HTTP/2 and HTTP/3: Modern streaming benefits from multiplexed connections and QUIC's zero-RTT handshakes, reducing segment fetch latency.
• Load balancing: Video platforms use DNS-based and anycast routing to direct viewers to the nearest CDN POP.
• Object storage (S3, GCS): Origin segments are stored in object storage systems designed for high durability and throughput.
• Rate limiting and throttling: Video platforms must handle thundering herd problems when millions of viewers start a live event simultaneously.

title: "How video streaming delivers adaptive bitrate" description: "How DASH and HLS segment videos into chunks, switch bitrates based on bandwidth estimation, and use CDN edge caching for scale." tags:

• "video"
• "streaming"
• "cdn"
• "how-things-work" difficulty: "medium" category: "situational/how-things-work" order: 26 publishedAt: "2026-04-12" relatedArticles: []

Stub

This article is planned but not yet written. See the instruction files for writing guidelines.