How garbage collection works in modern runtimes

The Interview Question

Interviewer: "Your Java service has P99 latency spikes of 200ms every few minutes, but the median is 2ms. You suspect garbage collection pauses. Walk me through how the JVM's garbage collector works, why it causes pauses, and what options you have to reduce them."

This question tests whether you understand memory management beyond "GC frees unused objects." The interviewer wants to hear about generational collection, the tradeoff between throughput and pause time, concurrent vs stop-the-world phases, and how to choose the right collector for a given workload.

What to Clarify Before Answering

You: "Before I walk through the GC internals, let me scope the conversation..."

• "Are we focused on the JVM specifically, or should I compare GC strategies across runtimes (JVM, V8, Go)?"
• "Is the concern about pause duration (latency) or total GC overhead (throughput)?"
• "What JVM version and collector is the service currently using? G1 is the default since Java 9."
• "Should I cover GC tuning knobs, or just the algorithmic fundamentals?"

Why this matters: GC design choices vary dramatically across runtimes. The JVM has seven production collectors. Go has a fundamentally different design philosophy (no generational collection). V8 uses a two-generational approach optimized for short-lived web objects. Scoping the answer prevents you from spending ten minutes on a surface-level tour of everything.

The 30-Second Answer

Garbage collection automatically reclaims memory occupied by objects that are no longer reachable from any live reference (root). The most common algorithm is mark-and-sweep: walk from root references (stack variables, static fields, registers), mark every reachable object, then sweep unmarked objects. Most modern collectors are generational, splitting the heap into a young generation (where most objects die quickly) and an old generation (where long-lived objects accumulate). Young gen collections are fast and frequent; old gen collections are expensive and infrequent. The fundamental tradeoff is between throughput (minimizing total GC overhead) and latency (minimizing individual pause times). Modern collectors like ZGC, Shenandoah, and Go's concurrent GC use concurrent marking to run most GC work alongside application threads, reducing stop-the-world pauses to sub-millisecond durations.

In an interview, GC questions typically probe three things: do you understand the algorithmic tradeoffs (copying vs mark-sweep vs mark-compact), can you reason about runtime-specific behavior (why Go has no generational collection, why V8 uses semi-space copying), and can you diagnose GC-related production issues (long tail latency, memory leaks, OOM crashes). This article covers all three.

The Architecture Overview

The diagram shows the fundamental GC cycle: allocate in the young generation, trace from roots to find live objects, then reclaim dead objects through sweeping, compaction, or copying. Objects that survive multiple young gen collections get promoted to the old generation, where they are collected less frequently.

I will now walk through the foundational algorithms, then show how each major runtime implements them differently.

Reference Counting vs Tracing: The Two Families

Every garbage collector belongs to one of two families: reference counting or tracing. Understanding the distinction is essential because it determines the fundamental behavior of the system. Most production runtimes use tracing, but reference counting shows up in important contexts (Python, Swift, Rust's Rc<T>, Objective-C). Knowing why most runtimes chose tracing over reference counting is a strong interview signal.

Reference counting

Each object maintains a counter of how many references point to it. When a reference is created, the counter increments. When a reference is destroyed, the counter decrements. When the counter reaches zero, the object is immediately freed.

// Reference counting pseudocode
function assign_reference(target, object):
  if target.ref != null:
    target.ref.count -= 1
    if target.ref.count == 0:
      free(target.ref)        // Immediate reclamation
  target.ref = object
  object.count += 1

Advantages:

• Deterministic: memory is freed the instant it becomes unreachable
• No stop-the-world pauses
• Simple to implement for simple cases

Problems:

• Circular references: If object A references B and B references A, both have count >= 1 forever, even if nothing else references them. This is a memory leak.
• Counter overhead: Every assignment requires atomic increment/decrement (expensive on multi-core CPUs)
• Cascading frees: Freeing one object can trigger a chain of frees for all objects it references

Languages that use reference counting (Python, Swift, Objective-C, Rust's Rc<T>) must add cycle detection as a separate mechanism. Python runs a cycle detector periodically that uses a tracing algorithm on reference-counted objects.

Reference counting also has poor cache behavior because decrementing a counter requires loading the object header into cache, even if the object itself is not needed. In a tracing collector, dead objects are never touched at all. This is sometimes called the "pay for the dead" problem: reference counting pays a cost proportional to the number of objects freed, while tracing pays a cost proportional to the number of objects that survive.

Tracing (mark-and-sweep)

Tracing collectors do not track individual references. Instead, they periodically walk the entire object graph starting from root references and mark everything reachable. Anything not marked is garbage.

// Mark-and-sweep pseudocode
function collect():
  // Phase 1: Mark
  worklist = all_root_references()      // Stack, globals, registers
  while worklist is not empty:
    object = worklist.pop()
    if not object.marked:
      object.marked = true
      for ref in object.references:
        worklist.push(ref)

  // Phase 2: Sweep
  for object in all_heap_objects():
    if not object.marked:
      free(object)                       // Reclaim garbage
    else:
      object.marked = false              // Reset for next cycle

The Generational Hypothesis: Why Young and Old Matter

The single most important insight in garbage collection is the generational hypothesis: most objects die young.

Empirical measurements across decades of production workloads show that 80-98% of objects become unreachable within microseconds of allocation. A function's local variables, temporary strings, iterator objects, lambda captures: they all die when the function returns.

This insight leads to generational collection: split the heap into two (or more) regions based on object age, and collect the young region frequently and cheaply.

Why young gen collection is fast

The young generation uses a copying collector. It divides its space into Eden (where all new objects are allocated) and two Survivor spaces. During collection:

1. Scan only the young generation (small, ~10-50MB)
2. Copy all live objects from Eden and the active Survivor space into the other Survivor space
3. The entire Eden space is now empty, ready for new allocations via bump pointer

The key insight: since 90%+ of young gen objects are dead, the collector copies very few objects (only the survivors). It does not need to visit or free the dead ones. The entire Eden space is reclaimed in one shot by resetting the allocation pointer.

Bump-pointer allocation is extremely fast. It is literally just incrementing a pointer by the object size. This is as fast as stack allocation and orders of magnitude faster than malloc/free in C, which must search a free list. The speed of young gen allocation is one of the reasons managed languages achieve high performance despite GC overhead.

The write barrier problem

If the young gen collector only scans young gen roots, it would miss references from old gen objects pointing to young gen objects. For example, if an old HashMap gains a new entry pointing to a young String, the young gen collector must know about that reference.

This is solved by write barriers and card tables:

// Write barrier pseudocode (executed on every reference store)
function store_reference(source_object, field, target_object):
  field = target_object                   // The actual store
  if is_old_gen(source_object) and is_young_gen(target_object):
    mark_card_dirty(source_object)        // Flag for young GC to scan

The card table divides old gen memory into small regions (cards, typically 512 bytes each). When a write barrier detects a cross-generational reference, it marks the corresponding card as "dirty." During young gen collection, the collector scans only dirty cards in old gen to find additional roots.

JVM Collectors: From Serial to ZGC

The JVM has the richest ecosystem of garbage collectors. Each one makes different tradeoffs between throughput, latency, and memory overhead.

The collector evolution

G1 Garbage Collector (the default)

G1 (Garbage-First) divides the heap into regions (typically 1-32MB each) rather than fixed young/old spaces. Each region is classified as Eden, Survivor, Old, or Humongous (for objects > 50% of region size).

G1's key innovation is the region-based approach. Instead of collecting the entire old generation, G1 identifies which regions have the most garbage (hence "Garbage-First") and evacuates only those regions. This keeps pause times predictable because you can control how many regions to evacuate per pause.

The -XX:MaxGCPauseMillis=200 flag sets a soft pause target. G1 adjusts how much work it does per collection to try to meet this target.

ZGC: Sub-millisecond pauses

ZGC achieves sub-millisecond pauses on heaps up to 16TB by doing almost all work concurrently. It uses colored pointers (metadata bits stored in unused pointer bits on 64-bit systems) to track object state without stop-the-world marking.

ZGC uses four metadata bits in each object pointer to encode the object's state:

• Marked0 / Marked1: Alternating mark bits for concurrent marking
• Remapped: Whether the pointer has been updated after relocation
• Finalizable: Whether the object is only reachable through a finalizer

This approach is called a "load barrier" because the metadata check happens when a pointer is loaded (read), not when it is stored (written). Every time your code reads an object reference, ZGC's load barrier checks whether the pointer needs updating.

The only stop-the-world phases in ZGC are:

1. Root scanning (scan thread stacks, ~microseconds)
2. Relocation start (prepare to move objects, ~microseconds)

Everything else (marking, reference processing, relocation) runs concurrently with application threads. When a thread accesses an object that has been relocated, the load barrier transparently updates the reference to the new location. This is called a "self-healing" barrier because after the first access, subsequent accesses to the same reference go directly to the new location.

V8 Garbage Collection: The Orinoco Pipeline

V8 (the JavaScript engine in Chrome and Node.js) uses a two-generational collector optimized for the unique characteristics of JavaScript: massive allocation rates and very short object lifetimes.

Young generation: the Scavenger

V8's young generation is a semi-space copying collector divided into two equal halves: "from-space" and "to-space." All allocations go into from-space via bump-pointer allocation (extremely fast, just incrementing a pointer).

When from-space fills up (typically 1-8MB):

1. Stop the world (very briefly, 1-10ms)
2. Scan roots and copy live objects from from-space to to-space
3. Objects that survived two scavenges are promoted to old generation
4. Flip: from-space becomes to-space and vice versa
5. The old from-space is now entirely empty

Since JavaScript creates enormous numbers of temporary objects (closures, string concatenations, array intermediates), the scavenger runs hundreds of times per second in busy applications, each time reclaiming most of the young generation.

V8 also uses parallel scavenging on multi-core systems: multiple GC threads cooperate to copy live objects from from-space to to-space simultaneously. This reduces young gen pause times on machines with 4+ cores. The scavenger also runs in a partially concurrent mode where some object copying starts before the full STW pause.

Old generation: Mark-Compact

V8's old generation uses a concurrent mark-compact collector with three phases:

1. Concurrent marking: A background thread walks the object graph and marks reachable objects. Application threads continue running. When they create new references, a write barrier records them for the marker to process.
2. Atomic pause: A brief stop-the-world pause to finalize marking (process remaining write barrier entries, scan roots that changed since marking started). Typically < 5ms.
3. Concurrent compaction/sweeping: Dead objects are freed and live objects are compacted to reduce fragmentation. This runs on background threads.

V8 also employs incremental marking for situations where concurrent marking cannot keep up. Instead of marking all objects in one burst, V8 interleaves small marking steps (~1ms each) with application execution. This prevents any single marking phase from causing a noticeable jank.

Idle-time GC scheduling

V8 takes advantage of browser idle periods (between frames, during network waits) to perform GC work. The requestIdleCallback API exposes these idle periods to JavaScript, but V8 uses them internally for GC as well. When the main thread is idle for more than a few milliseconds, V8 opportunistically runs marking or sweeping steps.

This is why a Node.js server under constant load has different GC characteristics than a browser tab that alternates between activity and idle periods. In Node.js, V8 rarely gets idle time, so GC work competes directly with request processing.

Go GC: Concurrent Tricolor Marking

Go takes a radically different approach to garbage collection. It has a non-generational, concurrent, tricolor mark-and-sweep collector. The design philosophy is simple: short pauses are more important than throughput, and simplicity is more important than sophistication.

Tricolor marking

Go's collector classifies objects into three colors during the marking phase. Think of it like a wavefront sweeping through the object graph:

• White: Not yet visited (potentially garbage). All objects start as white.
• Gray: Visited but references not yet scanned (in the worklist). These are the "frontier" of the mark wave.
• Black: Visited and all references scanned (definitely alive). These are behind the wavefront.

The algorithm starts by coloring all root-reachable objects gray. Then it repeatedly picks a gray object, scans its references (coloring any white referents gray), and colors the original object black. When no gray objects remain, all white objects are unreachable and can be freed.

// Go's tricolor marking algorithm
function concurrent_mark():
  // Initial: all objects white
  // Move roots to gray
  for root in all_roots():
    root.color = GRAY
    worklist.push(root)

  // Process gray objects (runs concurrently with mutator)
  while worklist is not empty:
    object = worklist.pop()
    for ref in object.references:
      if ref.color == WHITE:
        ref.color = GRAY
        worklist.push(ref)
    object.color = BLACK

  // After marking: white objects are garbage
  sweep_white_objects()
```is invariant is violated, the collector would miss a live object (the white one) and incorrectly free it. If the application (the "mutator") creates such a reference while the collector is running, the write barrier must intervene to restore the invariant

The critical invariant is the **tricolor invariant**: a black object must never point directly to a white object. If the application (the "mutator") creates such a reference while the collector is running, the write barrier must intervene:

// Go's write barrier (simplified Dijkstra-style) function write_barrier(slot, new_value): if gc_is_running and new_value.color == WHITE: new_value.color = GRAY // Shade the target gray worklist.push(new_value) // Add to marking worklist *slot = new_value // Perform the actual write

### Go's GC phases

```mermaid
flowchart TD
  subgraph GoGC["🔧 Go GC Cycle"]
    STW1["🔴 STW: Mark Start\n~10-30 microseconds\nEnable write barriers\nEnqueue root set"]
    ConcMark["🔍 Concurrent Mark\n1-100ms (concurrent)\nTricolor marking\nGC + app threads run together"]
    MarkAssist["⚙️ Mark Assist\nApp threads help mark\nWhen allocation outpaces GC"]
    STW2["🔴 STW: Mark Termination\n~10-30 microseconds\nDrain final worklist\nDisable write barriers"]
    ConcSweep["🗑️ Concurrent Sweep\nBackground goroutine\nFree unmarked objects\nReturn spans to heap"]
    Idle["✅ Idle\nWaiting for heap growth\nGOGC threshold: 2x live set"]
  end

  STW1 -->|"Start marking"| ConcMark
  ConcMark -->|"App allocates fast"| MarkAssist
  MarkAssist -->|"Help complete marking"| ConcMark
  ConcMark -->|"Marking complete"| STW2
  STW2 -->|"Begin sweep"| ConcSweep
  ConcSweep -->|"All swept"| Idle
  Idle -->|"Heap grows past threshold"| STW1

One important aspect of Go's design is mark assist. If the application allocates faster than the GC can mark, allocating goroutines are forced to do marking work before they can proceed with their allocation. This acts as natural backpressure: the faster you allocate, the more GC work you do, which slows your allocation rate. This prevents the heap from growing unboundedly during collection.

The GOGC setting controls the ratio of new allocations to the existing live set before triggering a collection. At GOGC=100 (default), GC triggers when newly allocated memory equals the live set. Setting GOGC=50 triggers twice as often (lower memory, more CPU overhead). Setting GOGC=200 triggers half as often (higher memory, less CPU overhead).

Since Go 1.19, you can also set GOMEMLIMIT to define a soft memory cap. When heap usage approaches the limit, Go triggers GC more aggressively, even ignoring GOGC, to stay under the limit. This is the recommended approach for containerized deployments where your memory budget is fixed by cgroup limits.

GC Tuning: The Practical Guide

What Happens When Things Break

Production GC problems almost always manifest as latency spikes, memory growth, or outright crashes. Here are the most common failure modes and how to handle them.

Latency spikes from full GC

The most common GC problem in production is a periodic P99 latency spike caused by old generation collection. The application runs fine for minutes, then hits a 200ms+ pause when old gen fills up. Request queues build up during the pause, causing a cascade of slow responses.

Out-of-memory despite available heap

This happens with non-compacting collectors (CMS, Go) when the heap is fragmented. Free memory exists in small scattered chunks, but no single contiguous block is large enough for the requested allocation. The collector cannot compact, so it fails even though total free memory exceeds the allocation size.

Common failure modes

A few things to note about these numbers. ZGC's sub-millisecond pauses are consistent across heap sizes from 100MB to 16TB. G1's mixed collection pauses scale with the number of regions evacuated. Go's STW pauses are independent of heap and live set size because they only scan thread stacks and enable/disable write barriers.

The "throughput overhead" column measures the percentage of CPU time spent on GC work. This is CPU you cannot use for application processing. For a latency-sensitive service, 15% throughput overhead is an acceptable price for sub-millisecond pauses. For a batch job, even 3% overhead adds up over hours of processing. | JVM | ZGC | < 1ms | < 1ms | 5-15% | ~15% (colored pointers) | | JVM | Shenandoah | 1-10ms | 1-10ms | 5-15% | ~10% (forwarding pointers) | | V8 | Orinoco | 1-10ms | 5-50ms | 5-10% | ~20% (semi-space) | | Go | Concurrent | N/A (no gen) | ~10-30 microseconds STW | 10-25% | ~2x live set |

How This Compares to Alternatives

I reach for the JVM with ZGC for latency-sensitive services that need large heaps and predictable tail latency. Go is the right choice when you want simple deployment, fast startup, and are willing to accept higher CPU overhead via card tables. Concurrent collectors use them to maintain the tricolor invariant during concurrent marking."

• When asked about G1 vs ZGC: "G1 is the default, balancing throughput and latency with 50-200ms pauses. ZGC sacrifices 5-15% throughput for sub-millisecond pauses regardless of heap size. Use G1 for general workloads, ZGC when your P99 SLO requires single-digit millisecond latency."

• When asked about Go's GC: "Go uses a non-generational concurrent mark-and-sweep with tricolor marking. STW pauses are under 100 microseconds. The tradeoff is higher CPU overhead (10-25%) because the GC runs concurrently and competes with application threads. Tune with GOGC and GOMEMLIMIT."

• When asked about memory leaks in GC languages: "GC prevents dangling pointers but not memory leaks. A leak happens when objects are retained longer than needed: unbounded caches, event listeners never removed, ThreadLocal values in pooled threads. Diagnose with heap dump analysis (JVM) or pprof (Go) or heap snapshots (V8)."

• When asked about GC tuning: "Start with the default collector and default settings. Measure P99 latency, GC pause duration, and throughput under realistic load. Only tune if SLOs are not met. The most impactful tunings are: choosing the right collector, then heap sizing, then advanced flags."

• When asked about allocation rate: "Allocation rate is the key driver of GC frequency. Reducing allocation rate (reuse objects, avoid boxing primitives, pool buffers, use StringBuilder instead of string concatenation) is more impactful than any GC tuning flag."

• When asked about reference counting vs tracing: "Reference counting frees objects immediately but cannot handle circular references and has per-assignment overhead. Tracing (mark-and-sweep) handles cycles naturally and pays cost proportional to live objects, not total allocations. Most production runtimes chose tracing."

• When asked about reference counting vs tracing: "Reference counting frees objects immediately when their count hits zero, giving deterministic destruction. But it cannot handle cycles without a separate cycle detector, and the per-assignment overhead of atomic increments is significant in multithreaded code. CPython uses refcounting plus a cycle collector. Most server runtimes use tracing collectors because they amortize cost over bulk collections."

• When asked about safepoints: "A safepoint is a point in the code where the runtime can safely pause a thread for GC. The JVM inserts safepoint polls at method returns, loop backedges, and allocation sites. A long-running counted loop without safepoint polls can delay the entire GC, stalling all other threads. Recent JDK versions added loop-strip-mining to mitigate this."

• When asked about V8's GC: "V8 uses a two-generational collector. The young gen scavenger is a semi-space copying collector that exploits the fact that 90%+ of JS objects die immediately. The old gen

• When asked about reference counting vs tracing: "Reference counting frees objects immediately when their count hits zero, giving deterministic destruction. But it cannot handle cycles without a separate cycle detector, and the per-assignment overhead is significant. CPython uses refcounting plus a cycle collector. Most server runtimes use tracing collectors because they amortize cost over collections."

• When asked about safepoints: "A safepoint is a point in the code where the runtime can safely pause a thread for GC. The JVM inserts safepoint polls at method returns, loop backedges, and allocation sites. A long-running loop without safepoint polls can delay GC for the entire application. This is a known JVM issue that has been addressed with counted loop safepoints in recent JDK versions." uses concurrent mark-compact. V8 also schedules GC during idle periods between frames."

• When asked about reference counting vs tracing: "Reference counting frees objects immediately but cannot handle circular references and has per-assignment overhead. Tracing (mark-and-sweep) handles cycles naturally and pays cost proportional to live objects, not allocations. Most runtimes use tracing because it is faster for typical workloadsck old-to-young references. Concurrent collectors use them to maintain the tricolor invariant during concurrent marking."

• When asked about G1 vs ZGC: "G1 is the default, balancing throughput and latency with 50-200ms pauses. ZGC sacrifices 5-15% throughput for sub-millisecond pauses regardless of heap size. Use G1 for general workloads, ZGC for latency-sensitive services."

• When asked about Go's GC: "Go uses a non-generational concurrent mark-and-sweep with tricolor marking. STW pauses are under 100 microseconds. The tradeoff is higher CPU overhead (10-25%) because the GC runs concurrently and competes with application threads."

• When asked about memory leaks in GC languages: "GC prevents dangling pointers but not memory leaks. A leak happens when objects are retained longer than needed: unbounded caches, event listeners never removed, ThreadLocal values in pooled threads. Diagnose with heap dump analysis or allocation profiling."

• When asked about GC tuning: "Start with the default collector and default settings. Measure P99 latency, GC pause duration, and throughput. Only tune if SLOs are not met. The most impactful tuning is choosing the right collector, then heap sizing, then advanced flags."

• When asked about allocation rate: "Allocation rate is the key driver of GC frequency. Reducing allocation rate (reuse objects, avoid boxing, pool buffers) is more impactful than any GC tuning flag."

• When asked about reference counting vs tracing: "Reference counting frees objects immediately when their count hits zero, giving deterministic destruction. But it cannot handle cycles without a separate cycle detector, and the per-assignment overhead of atomic increments is significant in multithreaded code. CPython uses refcounting plus a cycle collector. Most server runtimes use tracing collectors because they amortize cost over bulk collections."

• When asked about safepoints: "A safepoint is a point in the code where the runtime can safely pause a thread for GC. The JVM inserts safepoint polls at method returns, loop backedges, and allocation sites. A long-running counted loop without safepoint polls can delay the entire GC, stalling all other threads. Recent JDK versions added loop-strip-mining to mitigate this."

Test Your Understanding

Quick Recap

1. Garbage collectors automatically reclaim memory by identifying objects unreachable from root references (stack frames, static fields, thread locals).
2. Reference counting frees objects immediately when their reference count hits zero, but cannot handle circular references without a separate cycle detector.
3. Tracing collectors (mark-and-sweep) walk the object graph from roots, and their cost is proportional to live objects, not total allocations.
4. Generational collection exploits the fact that 90%+ of objects die young by collecting the young generation frequently and cheaply.
5. Write barriers track cross-generational references so the young gen collector does not miss old-to-young pointers.
6. The JVM offers collectors ranging from Serial (simple, high pauses) to ZGC (sub-millisecond pauses, concurrent), with G1 as the balanced default.
7. Go uses non-generational concurrent tricolor marking with sub-100-microsecond STW pauses, trading throughput for latency.
8. Allocation rate is the primary driver of GC frequency, and reducing allocations is more impactful than tuning collector flags.

Related Concepts

• Memory allocation strategies: Stack vs heap allocation, bump-pointer allocators, TLAB (Thread Local Allocation Buffer) design, and how escape analysis moves heap allocations to the stack when objects do not outlive their creating method.
• Memory models and concurrency: How write barriers interact with the Java Memory Model and Go's happens-before guarantees, and why GC safepoints matter for thread coordination in concurrent collectors.
• JIT compilation and inlining: How the JIT compiler's inlining decisions directly affect escape analysis. Inlined methods expose their allocations to the caller's scope, enabling the JIT to prove that objects do not escape and can be stack-allocated or scalar-replaced.
• Off-heap memory management: When GC overhead is unacceptable (large caches, memory-mapped files, direct byte buffers), applications manage memory outside the GC heap. Java's ByteBuffer.allocateDirect() and Go's cgo allocations bypass the collector entirely.
• Weak references and finalization: WeakReference, SoftReference, PhantomReference (Java), and runtime.SetFinalizer (Go) let applications interact with the GC lifecycle. Understanding reference strengths is critical for building GC-friendly caches and resource cleanup patterns.
• Container memory limits and GC: In Kubernetes and Docker environments, the GC must respect container memory limits (-XX:MaxRAMPercentage in Java, GOMEMLIMIT in Go). Misconfiguration leads to OOM kills even when the GC could have collected garbage in time.
• How HashMap works under the hood: Hash table implementations create many short-lived entry objects during rehashing, directly affected by GC behavior and young generation sizing decisions.
• How Prometheus works: Monitoring GC metrics (pause duration, allocation rate, promotion rate, heap occupancy) is essential for production GC tuning across JVM and Go runtimes.