P2P File Sharing

What is a peer-to-peer file sharing system?

A P2P file sharing system distributes large files directly between participants, with no central server holding the content. Every downloader simultaneously becomes an uploader, so aggregate bandwidth scales with the number of participants instead of shrinking per-user as more people join. I treat this question as a trust-elimination problem first and a bandwidth problem second: once you internalize that no single node can be relied upon, every mechanism in the design follows naturally. The interesting engineering challenges are coordination without central authority (how do peers find each other?), incentive alignment (how do you stop users from downloading without sharing?), and data integrity (how do you trust data arriving from strangers?). This question tests DHT design, piece-selection and choking algorithms, content addressing, and Merkle tree integrity, making it one of the richest distributed-systems questions at the hard tier.

Functional Requirements

Core Requirements

1. A user can add a file to the network and receive a shareable identifier (a magnet link or torrent file) that others can use to download it.
2. Any peer can download a file using only its identifier, without contacting the original uploader.
3. Files remain downloadable as long as at least one other peer holds a complete copy.
4. Every downloaded piece is verified for integrity; corrupted or tampered pieces are rejected and re-fetched.

Below the Line (out of scope)

• User authentication or private sharing (all shared content is treated as public).
• Rate limiting, traffic shaping, or ISP-level optimization (e.g., BitTorrent's LT-seeding and LEDBAT).
• Streaming playback before a download completes.
• Legal enforcement or content takedown mechanisms.

Authentication is below the line because the protocol operates on content-addressed data: knowing the infohash of a torrent already proves you have the right to request it. Production systems like private trackers layer HTTP authentication on top of the tracker protocol, separate from the core P2P mechanism.

Streaming is below the line because piece selection changes dramatically for video (sequential instead of rarest-first). Adding streaming support means running a parallel sequential-priority policy on the top N pieces while the rest of the torrent uses rarest-first. The tradeoff is covered in the piece selection deep dive.

Non-Functional Requirements

Core Requirements

• Availability: A torrent remains downloadable as long as at least one seeder is online. Target: 99.9% torrent availability for any torrent with 3+ seeders. There is no single point of failure in the content delivery path.
• Throughput: A peer downloading a popular file should saturate its uplink by receiving pieces from multiple peers simultaneously. Target is full bandwidth utilization (a 100 Mbps client downloads at 100 Mbps).
• Scale: The system must support millions of active torrents and tens of millions of simultaneous peers. The tracker or DHT must handle a sustained announce rate of hundreds of thousands of requests per second.
• Integrity: A corrupted or maliciously altered piece must be detected and discarded before assembly. Target: zero corrupted bytes written to disk; SHA-1 per-piece verification (BitTorrent v1 baseline) completes under 5 ms per 256 KB piece; SHA-256 per-block verification (BitTorrent v2) completes under 1 ms per 16 KB block on commodity hardware.
• Latency (peer discovery): A new peer should find its first set of peers for a torrent within 5 seconds of announcing.

Below the Line

• Sub-second peer discovery (requires centralized index or pre-seeded local DHT nodes).
• Global deduplication across torrents with identical content but different infohashes.

Full bandwidth utilization across millions of peers means a central server model fails immediately. A tracker serving 10 million concurrent peers at 100 Mbps each would need to push 1 exabyte per second. The tracker's only job is coordination (returning peer lists); the data transfer happens directly between peers.

I'd call out this number on the whiteboard early because it shuts down any central-server proposal immediately. Once the interviewer sees the exabyte math, they understand why P2P is the only viable architecture.

Integrity at petabyte scale means we cannot rely on TLS or a trusted origin. Data arrives from anonymous strangers who may be misconfigured, malicious, or silently corrupting packets. Each piece must carry a self-certifying hash so any receiver can verify it independently.

Core Entities

• Torrent: The metadata bundle for a shared file, containing the infohash (SHA-1 of its info dictionary), file name, total size, piece length (fixed, typically 256 KB to 1 MB), and an ordered list of SHA-1 hashes for each piece.
• Piece: A fixed-size chunk of the file data, identified by its zero-based index within the torrent. Pieces are downloaded independently and verified against the torrent's piece hash list before assembly.
• Block: A sub-unit of a piece (typically 16 KB) used in the wire protocol. Peers request blocks, not full pieces. Once all blocks of a piece arrive, the piece is assembled and verified.
• Peer: A network participant identified by a 20-byte peer ID, IP address, and port. A peer is a leecher if its download is incomplete and a seeder if it holds the complete file.
• Swarm: The complete set of peers (seeders and leechers) currently sharing a particular torrent.
• Tracker: A centralized coordination server that maps an infohash to a list of peer IP/port pairs. Peers announce their presence to the tracker periodically.

The primary relationships: a Torrent has many Pieces. A Swarm belongs to one Torrent and contains many Peers. Each Peer maintains a bitfield (a bit vector, one bit per piece) indicating which pieces it currently holds.

Schema detail and DHT storage layout are deferred to the deep dives.

API Design

The system has two distinct protocol surfaces: the tracker HTTP protocol (used by peers to discover each other) and the peer wire protocol (used by peers to exchange pieces directly).

FR 1 and FR 2: Announce presence and discover peers (tracker)

GET /announce
  ?info_hash=<20-byte-urlencoded-SHA1>
  &peer_id=<20-byte-urlencoded-random>
  &port=6881
  &uploaded=0
  &downloaded=0
  &left=<bytes-remaining>
  &event=started | completed | stopped

Response (compact format):
{
  "interval": 1800,
  "peers": "<binary blob: 6 bytes per peer (4-byte IP + 2-byte port)>"
}

The tracker returns up to 50 peer addresses. The peer opens connections to those addresses and begins exchanging pieces. The interval field tells the peer how often to re-announce (default 30 minutes). Use event=completed when the download finishes to tell the tracker the peer has become a seeder.

Supplemental (FR 1): Scrape (current swarm stats)

GET /scrape?info_hash=<20-byte-urlencoded-SHA1>

Response:
{
  "files": {
    "<info_hash>": {
      "complete": 142,
      "incomplete": 37,
      "downloaded": 9821
    }
  }
}

Scrape lets a client check swarm health before committing to a download. A torrent with 0 seeders cannot be completed regardless of leecher count.

FR 3 and FR 4: Peer wire protocol (direct peer-to-peer)

The wire protocol is length-prefixed binary, not HTTP. Key message types:

// Peer wire protocol messages (simplified)

HANDSHAKE:
  <pstrlen=19><"BitTorrent protocol"><reserved 8 bytes>
  <info_hash 20 bytes><peer_id 20 bytes>

BITFIELD (after handshake):
  <length><0x05><bitfield>  // one bit per piece; 1 = peer has it

INTERESTED / NOT_INTERESTED:
  <length><0x02>  // "I want pieces you have"
  <length><0x03>  // "I no longer want pieces from you"

CHOKE / UNCHOKE:
  <length><0x00>  // "I will not upload to you"
  <length><0x01>  // "I will now upload to you"

REQUEST (download block from peer):
  <length><0x06><piece_index><block_offset><block_length>

PIECE (upload block to peer):
  <length><0x07><piece_index><block_offset><block_data>

HAVE (announce newly completed piece):
  <length><0x04><piece_index>

REST-style HTTP verbs do not apply to the peer wire protocol. The protocol is a persistent binary TCP connection, not stateless request-response. Authentication is out of scope; in a production private tracker, the torrent file embeds a per-user passkey in the announce URL.

High-Level Design

1. Adding a file to the network

A peer who wants to share a file creates a torrent descriptor locally before any network communication.

Components:

• Seeding client: Hashes the file into fixed-size pieces and computes SHA-1(piece bytes) for each piece. Assembles the info dictionary and computes infohash = SHA-1(bencoded info dict).
• Torrent file / magnet link: The serialized descriptor containing the infohash, tracker URL, piece hashes, and file metadata. This is what the uploader shares with others.
• Tracker: Receives the seeder's first announce and adds it to the swarm list for that infohash.

Request walkthrough:

1. Seeding client reads the file from disk and splits it into N pieces of fixed size (e.g., 256 KB each).
2. For each piece, the client computes SHA-1(piece bytes) and stores the 20-byte hash in an ordered list.
3. Client computes infohash = SHA-1(bencoded info dictionary containing piece hashes + file name + length).
4. Client writes a .torrent file containing infohash, tracker URL, piece hashes, and piece length.
5. Client announces to the tracker with event=started, left=0 (it has the complete file).
6. Client generates a magnet link: magnet:?xt=urn:btih:<infohash>&dn=<name>&tr=<tracker_url>.