Event loop and I/O multiplexing

The problem

Your Node.js API server is handling 10,000 concurrent HTTP connections. Each connection is waiting for a database query that takes 20 ms. A thread-per-connection model would need 10,000 threads. At 1 MB of stack space per thread, that is 10 GB of memory just for thread stacks, before any application code or data. On Linux, the practical thread limit per process is around 10,000-30,000 depending on stack size, but at that count, context switching overhead consumes most CPU cycles switching between threads rather than doing useful work.

The thread-per-connection model scales to hundreds of concurrent connections. It breaks at thousands.

Node.js serves all 10,000 connections on a single thread without blocking. The secret is that 9,999 of them are waiting for I/O at any given moment, and I/O waiting does not require a thread to sit idle. This is the problem the event loop solves.

What the event loop is

The event loop is a runtime pattern where a single thread continuously polls for events (I/O readiness, timers firing, signals), executes the callback for each ready event, and then polls again. It is paired with I/O multiplexing, a kernel mechanism that lets a single thread simultaneously monitor thousands of file descriptors for readiness without blocking.

Think of a waiter in a restaurant with 50 tables. The waiter does not stand at each table watching it until the customer decides what to order (thread per connection). Instead, the waiter circulates. When a customer raises their hand (I/O becomes ready), the waiter walks over and acts (executes the callback). One person serves 50 tables not by dividing into 50 people but by responding to signals rather than waiting at stations.

How the event loop works

The event loop runs as a while (true) loop. Each iteration is called a "tick." During each tick, it calls the OS's I/O multiplexing function (epoll_wait on Linux, kqueue on macOS) to ask: which of the file descriptors I am watching became ready?

Step by step through a single incoming HTTP request:

1. The OS receives a TCP SYN packet from a client. It completes the handshake and signals readiness on the listening socket's file descriptor.
2. The event loop is in the Poll phase, blocked inside epoll_wait(). The kernel wakes the process because the listening socket is ready.
3. The event loop calls accept() on the listening socket, getting a new file descriptor for this connection. It registers this new FD with epoll to watch for incoming data.
4. The client sends the HTTP request. epoll signals the connection FD is readable.
5. The event loop reads the HTTP bytes, parses the request, and calls the route handler.
6. The route handler starts a database query. The database driver opens a socket to PostgreSQL, sends the query, and immediately returns a Promise. No thread blocking.
7. The event loop continues to the next ready event. It processes other connections while the database query is in flight.
8. Twenty milliseconds later, PostgreSQL sends the response. The database socket FD becomes readable. epoll signals it.
9. The event loop reads the response, resolves the Promise, and runs the .then() callback (in the microtask queue, before the next I/O callback).
10. The route handler sends the HTTP response. The connection is returned to the pool or closed.

// Pseudocode: Node.js event loop (libuv simplified)

while (loop.is_alive()):
    // Phase 1: Timers
    now = current_time()
    for timer in timer_queue:
        if timer.expires_at <= now:
            run_callback(timer.callback)
            run_all_microtasks()

    // Phase 2: I/O callbacks
    for callback in pending_io_callbacks:
        run_callback(callback)
        run_all_microtasks()

    // Phase 3: Poll — wait for I/O
    timeout = time_until_next_timer()
    ready_fds = epoll_wait(watched_fds, timeout=timeout)
    for fd in ready_fds:
        run_callback(fd.callback)
        run_all_microtasks()

    // Phase 4: Check (setImmediate)
    for callback in check_queue:
        run_callback(callback)
        run_all_microtasks()

    // Phase 5: Close callbacks
    for callback in close_queue:
        run_callback(callback)
        run_all_microtasks()

The diagram below shows the data structures libuv maintains internally and how the microtask queue weaves between each phase:

spawnSync d2 ENOENT

epoll_wait() and I/O multiplexing

The heart of the event loop is the kernel's I/O multiplexing mechanism. epoll (Linux) and kqueue (macOS/BSD) allow a single thread to monitor thousands of file descriptors simultaneously.

// Simplified: registering a socket with epoll
int epfd = epoll_create1(0);

struct epoll_event ev;
ev.events = EPOLLIN;         // notify when data is available to read
ev.data.fd = client_socket;
epoll_ctl(epfd, EPOLL_CTL_ADD, client_socket, &ev);

// Later: block until any registered FD is ready
struct epoll_event events[MAX_EVENTS];
int nready = epoll_wait(epfd, events, MAX_EVENTS, timeout_ms);
// Returns the number of FDs that are ready
// events[] contains which FDs are ready and why
for (int i = 0; i < nready; i++) {
    handle_ready(events[i].data.fd);
}

The key property: epoll_wait() can monitor 100,000 FDs simultaneously. It returns only the ones that are actually ready. It does not return all 100,000 and ask you to check each one. This is "edge-triggered" notification: the kernel tells you only when state changes.

Without epoll, you would use select() or poll(), both of which require passing the entire list of watched FDs on every call. At 10,000 FDs, select/poll becomes O(N) per call. epoll is O(1) for ready FD delivery regardless of how many FDs are registered.

The microtask queue and Promise execution order

A critical detail: Promises (and process.nextTick) do not execute in the main event loop phases. They run in the microtask queue, which is drained completely after every phase transition.

// What order does this print?
setTimeout(() => console.log('timeout'), 0);

Promise.resolve().then(() => console.log('promise'));

process.nextTick(() => console.log('nextTick'));

console.log('synchronous');

// Output:
// synchronous       – runs immediately (current call stack)
// nextTick          – runs before other microtasks
// promise           – runs before any I/O or timer callbacks
// timeout           – runs in the Timers phase of the next tick

The execution order rule: synchronous code first, then process.nextTick, then resolved Promises, then I/O callbacks and timers. This order matters for correctness in complex async chains.

What blocks the event loop

The event loop processes one callback at a time. A callback cannot be preempted by another. If a callback takes 500 ms to run, no other callback runs during those 500 ms. This is the critical failure mode of the single-threaded model.

The rule: anything that is purely CPU-bound runs in the event loop thread itself. Any CPU-bound operation taking more than 5-10 ms will cause perceptible latency increase for all concurrent requests.

Production usage

Limitations and when NOT to use it

• CPU-bound workloads block all concurrent requests. A single bcrypt.hashSync() or large JSON.parse() lasting 100 ms blocks every other request queued behind it. Node.js is the wrong runtime for CPU-intensive work unless it is offloaded to worker threads.
• A single uncaught exception or infinite loop takes down the entire process. In a multi-threaded server, one thread crashing affects only that thread. In Node.js, one callback throwing uncaught kills the entire server.
• The libuv thread pool (default 4 threads) can become a bottleneck. Async file I/O and async crypto operations use this pool. With 100 concurrent file operations, 96 of them queue waiting for a pool slot. Increase the pool size with UV_THREADPOOL_SIZE=16 for I/O-heavy workloads.
• process.nextTick starvation is real. Recursive process.nextTick calls can starve all other callbacks: process.nextTick is drained completely before any I/O callback runs. A recursive nextTick loop never yields and effectively blocks the event loop.
• Not suitable for stateful in-process parallelism. If your service needs to run multiple CPU-heavy computations concurrently (video transcoding, ML inference), node worker threads add complexity and the V8 memory overhead is high. Python with processes or Go with goroutines are more ergonomic for CPU parallelism.
• Memory leaks are harder to detect. In a long-running event loop, closures that accidentally capture large objects prevent GC. In threaded models, thread termination forces stack cleanup. In the event loop model, leaked closures accumulate until the process restarts.

When to choose the event loop model

Interview cheat sheet

• When asked how Node.js handles 10,000 concurrent connections on one thread: State the answer clearly: I/O waiting does not require a thread. The event loop uses epoll to monitor thousands of sockets simultaneously. When data arrives, the callback runs. Between arrivals, the thread is free to handle other ready events.
• When asked what epoll does: It lets a single thread monitor thousands of file descriptors for readiness via a kernel-managed interest list. epoll_wait() returns only the FDs that are actually ready, making it O(1) for ready delivery regardless of total monitored count.
• When asked what blocks the event loop: CPU-bound work. Synchronous file reads, JSON.parse() on large payloads, crypto.pbkdf2Sync(), large sorts, and any pure computation. I/O (network, file reads with async API) does not block.
• When asked about the microtask queue: Promises and process.nextTick run in the microtask queue, which is drained completely between each phase of the event loop. They run before any I/O callback or timer, even one already queued.
• When asked about the Node.js event loop phases: Timers (setTimeout/setInterval), I/O callbacks, Poll (epoll_wait blocks here), Check (setImmediate), Close callbacks. Microtasks drain between each phase.
• When asked when NOT to use Node.js: When the workload is CPU-bound. Image processing, video transcoding, ML inference, bcrypt on a high-QPS auth service. These should be offloaded to a worker pool or implemented in a language with better CPU parallelism (Go, Rust, Python with multiprocessing).
• When asked about nginx vs Node.js architecture: nginx uses one worker process per CPU core, each with its own event loop. Node.js (single process) uses one event loop. Both use epoll. The difference: nginx spawns multiple processes to use all CPUs; Node.js uses cluster module or pm2 to do the same.
• When asked how Redis uses the event loop: Redis processes all commands in a single-threaded event loop, giving it predictable behavior with no lock contention on data structures. Network I/O in Redis 6.0+ is multi-threaded, but command execution (reading and writing data) is still single-threaded.

Quick recap

1. The event loop solves the thread-per-connection scalability problem: a single thread monitors thousands of sockets via epoll and runs a callback only when a socket is actually ready, eliminating idle waiting threads.
2. epoll_wait() is the kernel system call at the core of the event loop; it returns only the file descriptors that are ready, making I/O multiplexing O(1) for ready delivery regardless of total monitored count.
3. Node.js's event loop phases run in order (timers, I/O callbacks, poll, check, close), with the microtask queue drained completely between every phase; Promises and process.nextTick always execute before the next I/O or timer callback.
4. CPU-bound operations (synchronous file reads, large JSON parses, password hashing with sync API) block the event loop thread and delay every other concurrent request for the duration of the computation.
5. The single-threaded model is a failure when workloads are CPU-bound rather than I/O-bound; the mitigation is worker threads, multiple processes via cluster, or a runtime with native CPU parallelism.
6. Use async APIs exclusively, keep any synchronous processing under 1-2 ms, profile for CPU-bound hotspots with --cpu-prof, and use cluster or pm2 to run one event loop per CPU core in production.

Related concepts

• Scalability — The event loop's I/O multiplexing enables one process to handle thousands of concurrent connections, which directly affects how you scale Node.js services horizontally vs vertically.
• Networking — The event loop interacts directly with kernel networking primitives (epoll, TCP socket buffers, accept queues); understanding these explains why the event loop model is efficient for network I/O specifically.
• Microservices — Microservices built in Node.js rely on the event loop model for performance; knowing when this model breaks down (CPU-bound work) informs which services should be built in Node.js vs other runtimes.