Event loop and I/O multiplexing

The problem

Your Node.js API server is handling 10,000 concurrent HTTP connections. Each connection is waiting for a database query that takes 20 ms. A thread-per-connection model would need 10,000 threads. At 1 MB of stack space per thread, that is 10 GB of memory just for thread stacks, before any application code or data. On Linux, the practical thread limit per process is around 10,000-30,000 depending on stack size, but at that count, context switching overhead consumes most CPU cycles switching between threads rather than doing useful work.

The thread-per-connection model scales to hundreds of concurrent connections. It breaks at thousands.

Node.js serves all 10,000 connections on a single thread without blocking. The secret is that 9,999 of them are waiting for I/O at any given moment, and I/O waiting does not require a thread to sit idle. This is the problem the event loop solves.

What the event loop is

The event loop is a runtime pattern where a single thread continuously polls for events (I/O readiness, timers firing, signals), executes the callback for each ready event, and then polls again. It is paired with I/O multiplexing, a kernel mechanism that lets a single thread simultaneously monitor thousands of file descriptors for readiness without blocking.

Think of a waiter in a restaurant with 50 tables. The waiter does not stand at each table watching it until the customer decides what to order (thread per connection). Instead, the waiter circulates. When a customer raises their hand (I/O becomes ready), the waiter walks over and acts (executes the callback). One person serves 50 tables not by dividing into 50 people but by responding to signals rather than waiting at stations.

How the event loop works

The event loop runs as a while (true) loop. Each iteration is called a "tick." During each tick, it calls the OS's I/O multiplexing function (epoll_wait on Linux, kqueue on macOS) to ask: which of the file descriptors I am watching became ready?

Step by step through a single incoming HTTP request:

1. The OS receives a TCP SYN packet from a client. It completes the handshake and signals readiness on the listening socket's file descriptor.
2. The event loop is in the Poll phase, blocked inside epoll_wait(). The kernel wakes the process because the listening socket is ready.
3. The event loop calls accept() on the listening socket, getting a new file descriptor for this connection. It registers this new FD with epoll to watch for incoming data.
4. The client sends the HTTP request. epoll signals the connection FD is readable.
5. The event loop reads the HTTP bytes, parses the request, and calls the route handler.
6. The route handler starts a database query. The database driver opens a socket to PostgreSQL, sends the query, and immediately returns a Promise. No thread blocking.
7. The event loop continues to the next ready event. It processes other connections while the database query is in flight.
8. Twenty milliseconds later, PostgreSQL sends the response. The database socket FD becomes readable. epoll signals it.
9. The event loop reads the response, resolves the Promise, and runs the .then() callback (in the microtask queue, before the next I/O callback).
10. The route handler sends the HTTP response. The connection is returned to the pool or closed.

// Pseudocode: Node.js event loop (libuv simplified)

while (loop.is_alive()):
    // Phase 1: Timers
    now = current_time()
    for timer in timer_queue:
        if timer.expires_at <= now:
            run_callback(timer.callback)
            run_all_microtasks()

    // Phase 2: I/O callbacks
    for callback in pending_io_callbacks:
        run_callback(callback)
        run_all_microtasks()

    // Phase 3: Poll — wait for I/O
    timeout = time_until_next_timer()
    ready_fds = epoll_wait(watched_fds, timeout=timeout)
    for fd in ready_fds:
        run_callback(fd.callback)
        run_all_microtasks()

    // Phase 4: Check (setImmediate)
    for callback in check_queue:
        run_callback(callback)
        run_all_microtasks()

    // Phase 5: Close callbacks
    for callback in close_queue:
        run_callback(callback)
        run_all_microtasks()

The diagram below shows the data structures libuv maintains internally and how the microtask queue weaves between each phase:

spawnSync d2 ENOENT

epoll_wait() and I/O multiplexing

The heart of the event loop is the kernel's I/O multiplexing mechanism. epoll (Linux) and kqueue (macOS/BSD) allow a single thread to monitor thousands of file descriptors simultaneously.

// Simplified: registering a socket with epoll
int epfd = epoll_create1(0);

struct epoll_event ev;
ev.events = EPOLLIN;         // notify when data is available to read
ev.data.fd = client_socket;
epoll_ctl(epfd, EPOLL_CTL_ADD, client_socket, &ev);

// Later: block until any registered FD is ready
struct epoll_event events[MAX_EVENTS];
int nready = epoll_wait(epfd, events, MAX_EVENTS, timeout_ms);
// Returns the number of FDs that are ready
// events[] contains which FDs are ready and why
for (int i = 0; i < nready; i++) {
    handle_ready(events[i].data.fd);
}

The key property: epoll_wait() can monitor 100,000 FDs simultaneously. It returns only the ones that are actually ready. It does not return all 100,000 and ask you to check each one. This is "edge-triggered" notification: the kernel tells you only when state changes.

Without epoll, you would use select() or poll(), both of which require passing the entire list of watched FDs on every call. At 10,000 FDs, select/poll becomes O(N) per call. epoll is O(1) for ready FD delivery regardless of how many FDs are registered.

The microtask queue and Promise execution order

A critical detail: Promises (and process.nextTick) do not execute in the main event loop phases. They run in the microtask queue, which is drained completely after every phase transition.