How gRPC works under the hood

The Interview Question

Interviewer: "Your team is migrating inter-service communication from REST over HTTP/1.1 to gRPC. A colleague asks why you cannot just use JSON over HTTP/2 instead and skip the Protocol Buffers complexity. Walk me through what gRPC actually does at the wire level that makes it faster than REST, and where the performance gains actually come from."

This question separates candidates who think gRPC is "just faster JSON" from those who understand the layered architecture: Protocol Buffers for serialization, HTTP/2 for transport multiplexing, and the framework-level features (deadlines, interceptors, streaming) that make it a complete RPC system. The interviewer is testing whether you can quantify the gains and explain when they matter.

What to Clarify Before Answering

You: "Before I dive in, let me clarify the scope..."

• "Are we talking about unary RPCs (request-response) or do we need streaming (server, client, or bidirectional)?"
• "What is the message size? gRPC's serialization advantage is most dramatic for small, frequent messages. For large payloads, the serialization overhead is a smaller fraction of total latency."
• "Are the services in the same datacenter or across regions? gRPC's multiplexing advantage shines most on high-latency links."
• "Do we need browser client support? gRPC-Web has different constraints than native gRPC."
• "What language ecosystems are involved? gRPC's codegen quality varies significantly across languages."

Why this matters: gRPC is not universally better than REST. Its advantages (binary serialization, multiplexing, streaming) matter in specific contexts. A candidate who asks these questions demonstrates they understand the tradeoffs rather than just parroting "gRPC is faster."

The 30-Second Answer

gRPC is a high-performance RPC framework built on two foundations: Protocol Buffers for binary serialization and HTTP/2 for transport. When a client calls a gRPC method, the request message is serialized into a compact binary format using protobuf (5-10x smaller than JSON), sent over an HTTP/2 stream with header compression (HPACK), and deserialized on the server. HTTP/2 allows multiplexing hundreds of concurrent RPCs over a single TCP connection, eliminating the head-of-line blocking that HTTP/1.1 suffers from. gRPC adds four RPC patterns on top: unary (single request/response), server streaming, client streaming, and bidirectional streaming. The framework also provides built-in deadline propagation (timeouts cascade across service chains), interceptors (middleware for auth, logging, retry), and client-side load balancing with pluggable strategies.

The Architecture Overview

The architecture has three layers that I will walk through in detail. The client layer includes generated stubs (typed method calls that handle serialization), a channel (manages connections and load balancing), and interceptors (middleware chain). The transport layer is HTTP/2 with multiplexed streams, one per RPC call. The server layer mirrors the client with interceptors, a request router that dispatches to your service implementation, and protobuf deserialization.

I find this layered design elegant because each layer is replaceable. You can swap protobuf for FlatBuffers. You can run gRPC over alternative transports (in-process, Unix domain sockets). The interceptor chain is identical on client and server, making cross-cutting concerns symmetric.

The Channel Abstraction

The gRPC channel is a critical concept. It represents a virtual connection to a server endpoint and manages the underlying HTTP/2 connections, DNS resolution, load balancing, and reconnection logic. When you create a channel to myservice.example.com:443, the channel:

1. Resolves the hostname to IP addresses (via DNS or a custom resolver)
2. Creates HTTP/2 connections (called "subchannels") to one or more resolved addresses
3. Applies the load balancing policy to distribute RPCs across subchannels
4. Monitors connection health via keepalive pings
5. Reconnects automatically when connections drop

Channels are expensive to create (TLS handshake, HTTP/2 negotiation) and cheap to use. You should create one channel per service and share it across all goroutines/threads. Creating a new channel for each RPC is a common performance mistake that wastes connection setup time and prevents multiplexing benefits.

Protocol Buffers: The Serialization Layer

Protocol Buffers (protobuf) is the serialization format that gives gRPC its speed advantage over JSON. Understanding the binary encoding explains why protobuf messages are 5-10x smaller and 10-100x faster to parse.

A protobuf message is defined in a .proto file:

message User {
  int32 id = 1;
  string name = 2;
  string email = 3;
  repeated string roles = 4;
}

The field numbers (1, 2, 3, 4) are the critical design choice. Unlike JSON, where field names are transmitted as strings on every message, protobuf transmits only the field number (1 byte for fields 1-15) and the value. The field name is never sent over the wire. It exists only in the .proto schema and the generated code.

The binary encoding uses a tag-length-value (TLV) format:

Field 1 (id=42):     [0x08] [0x2A]           = 2 bytes
Field 2 (name="Jo"): [0x12] [0x02] [0x4A6F]  = 4 bytes

Compare the same data in JSON: {"id":42,"name":"Jo"} = 20 bytes. For a message with 20 fields, this difference compounds dramatically.

Varint Encoding: How Protobuf Compresses Integers

One of protobuf's cleverest optimizations is varint encoding for integers. Small numbers use fewer bytes:

Since most real-world integer fields contain small values (user IDs under 1 million, counts under 1,000, enum values under 50), varint encoding saves significant space across millions of messages. Negative numbers are encoded as 10-byte unsigned varints, which is inefficient. If you expect negative values, use sint32/sint64 which ZigZag-encode negatives into small positive values first.

Default Values and Wire Efficiency

In proto3, fields with default values (0 for integers, empty string for strings, false for booleans) are not serialized at all. This is a significant space optimization for sparse messages. If a User message has 20 fields but only 5 are set, only 5 fields appear on the wire. JSON, by contrast, typically includes all fields with null or empty values.

Schema Evolution Rules

Protobuf's compatibility model is strict but powerful:

• Safe changes: Add new fields (with new field numbers), remove fields (but reserve the number), rename fields (only the number matters on the wire), change int32 to int64 (wire-compatible).
• Breaking changes: Change a field number, change a field type to an incompatible type (string to int32), change repeated to singular (or vice versa).

HTTP/2 Transport: Multiplexing and Header Compression

gRPC chose HTTP/2 as its transport for three specific features: stream multiplexing, header compression, and flow control. Understanding how gRPC maps RPCs to HTTP/2 streams is essential for debugging performance issues.

How HTTP/2 Multiplexing Works

In HTTP/1.1, each request-response pair occupies an entire TCP connection for its duration. Sending 100 concurrent requests requires 100 TCP connections (or serial pipelining, which nobody uses because of head-of-line blocking). Each connection costs a TCP handshake (1 RTT), a TLS handshake (1-2 RTTs), and kernel memory for the socket buffer.

HTTP/2 multiplexes many logical streams over a single TCP connection. Each gRPC call opens a new stream (identified by a stream ID), sends request headers and the protobuf body as DATA frames, and receives response headers and the response body on the same stream. Streams are fully independent: a slow RPC on stream 5 does not block a fast RPC on stream 7.

HPACK Header Compression

gRPC sends metadata (method name, content-type, authority, deadlines, auth tokens) as HTTP/2 headers. HPACK compresses these headers using a dynamic table that both client and server maintain. After the first request, common headers like content-type: application/grpc are encoded as a single byte index into the table. For a service making 10,000 RPCs per second, this saves megabytes of bandwidth.

The dynamic table works as follows: the first RPC sends full header values and both sides add them to a table indexed by position. The second RPC sends only the table index (1-2 bytes) instead of the full header string (20-100 bytes). The table has a configurable maximum size (default 4,096 bytes), and entries are evicted FIFO when the table is full. For gRPC, where the same method names and content types repeat on every call, HPACK typically achieves 90%+ compression after the first few RPCs.

Flow Control: Preventing Producer-Consumer Imbalance

HTTP/2 provides two levels of flow control: connection-level and stream-level. Each receiver (client or server) advertises a window (default 64 KB) that tells the sender how much data it can send before waiting. As the receiver processes data, it sends WINDOW_UPDATE frames to expand the window.

For gRPC, flow control is critical in streaming scenarios. If a server-streaming RPC produces messages faster than the client can consume them, flow control prevents the server from overwhelming the client's buffer. The server's send call will block (or return a backpressure signal, depending on the language) when the window is exhausted. This is a feature, not a bug: it prevents unbounded memory growth.

I have seen teams disable flow control (by setting enormous windows) to "fix" perceived throughput issues. This is dangerous: it shifts the bottleneck from the transport layer to application memory, causing OOM kills under load. The correct fix is to increase consumer throughput, not disable the safety valve.

gRPC to HTTP/2 Mapping

Every gRPC call maps to HTTP/2 as follows:

The Four RPC Patterns

gRPC supports four communication patterns, each mapped to HTTP/2 streams differently. Understanding these patterns is essential because choosing the wrong one is a common design mistake.

Unary RPC (Request-Response)

The simplest pattern. Client sends one message, server returns one message. This is identical to a REST API call but with protobuf serialization and HTTP/2 transport.

rpc GetUser(GetUserRequest) returns (User);

Use unary RPCs for: CRUD operations, authentication, any request that expects a single response.

Server Streaming RPC

Client sends one request, server returns a stream of messages. The server sends messages as they become available and closes the stream when done.

rpc ListTransactions(ListRequest) returns (stream Transaction);

Use server streaming for: paginated results pushed incrementally, live updates (stock prices, notifications), large result sets where you want the client to start processing before the server finishes.

Client Streaming RPC

Client sends a stream of messages, server returns one response after receiving all messages.

rpc UploadChunks(stream FileChunk) returns (UploadResult);

Use client streaming for: file uploads, aggregation (send many data points, get a summary), bulk inserts.

Bidirectional Streaming RPC

Both client and server send streams of messages independently. Either side can send at any time, and the streams are fully independent (the server does not need to wait for the client to finish or alternate turns).

rpc Chat(stream ChatMessage) returns (stream ChatMessage);

Use bidirectional streaming for: chat/messaging, real-time collaboration, interactive CLI tools, any protocol where both sides produce data asynchronously.

How Streaming Maps to HTTP/2 Frames

Under the hood, each streaming message is sent as an HTTP/2 DATA frame with a 5-byte gRPC frame header: 1 byte compression flag + 4 bytes message length. The stream stays open (no END_STREAM flag) until the sender closes its half.

For server streaming, the client sends HEADERS + DATA + END_STREAM (closing its send direction), and the server sends multiple DATA frames followed by HEADERS (trailers) + END_STREAM. For bidirectional streaming, neither side sends END_STREAM until it is done, so both directions remain open simultaneously.

This mapping is important for debugging: if you see RST_STREAM frames in a network capture, it means one side cancelled the stream (often a deadline or client cancellation). If you see GOAWAY frames, the entire connection is being shut down gracefully (server restart, load balancer drain).

Deadline Propagation and Cancellation

This is the feature I consider most underappreciated in gRPC. Deadline propagation solves a problem that REST APIs handle poorly: cascading timeouts across a chain of services.

In a microservices architecture, Service A calls Service B, which calls Service C. With REST, each service sets its own independent timeout. If A sets a 5-second timeout and B takes 3 seconds before calling C, C gets the full timeout configured by B (say, 5 more seconds). The total chain can take 8 seconds, well beyond A's expectation. Meanwhile, A has already timed out and returned an error to the user, but B and C keep working on a request nobody will read.

gRPC solves this with deadline propagation. When A sets a 5-second deadline, that deadline is embedded in the gRPC metadata (as the grpc-timeout header). When B calls C, the gRPC framework automatically adjusts the deadline: if 3 seconds have elapsed, C gets a 2-second deadline. If C cannot complete in 2 seconds, it gets a DEADLINE_EXCEEDED error immediately.

Cancellation propagation works similarly. When a client cancels a request (user navigated away, circuit breaker tripped), gRPC sends an RST_STREAM frame on the HTTP/2 stream. The server receives this as a cancellation signal and can stop processing immediately. In a chain of services, the cancellation cascades: A cancels, B receives the cancellation and cancels its call to C, C stops processing. No wasted work.

Implementing Deadline Propagation in Practice

In Go, deadlines flow through the context.Context:

// Service A sets a 5-second deadline
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
resp, err := serviceB.DoWork(ctx, request)

When Service B calls Service C using the same context, gRPC automatically calculates the remaining deadline and sends it in the grpc-timeout header. No manual arithmetic required.

In Java, the pattern uses CallOptions:

// Service A sets deadline
stub.withDeadlineAfter(5, TimeUnit.SECONDS).doWork(request);

The critical detail: if your server handler spawns background goroutines or threads, those must also respect the context/deadline. A common bug is starting a database query in a goroutine that does not inherit the RPC context, so the query continues after the RPC is cancelled, wasting database resources.

gRPC Status Codes

gRPC defines 17 standard status codes that are more granular than HTTP status codes:

The most important distinction: UNAVAILABLE (14) is retryable, while INTERNAL (13) is not. If your server encounters a transient error (connection reset, brief timeout), return UNAVAILABLE so the client's retry policy kicks in. If it is a genuine bug (nil pointer, logic error), return INTERNAL. I have seen teams return INTERNAL for everything, defeating their retry policies and causing unnecessary user-facing errors.

Load Balancing: Client-Side, Proxy, and Service Mesh

gRPC's load balancing story is more complex than REST because of HTTP/2's persistent, multiplexed connections. With REST over HTTP/1.1, a load balancer sees a new TCP connection per request and can distribute them round-robin. With gRPC, all requests flow over one (or a few) long-lived connections, so a layer-4 load balancer just sends everything to one server.

Client-Side Load Balancing

gRPC has built-in client-side load balancing with pluggable policies:

• pick_first (default): Connect to the first resolved address and send all RPCs there. Failover to the next address only if the connection drops. This is fine for single-server setups but terrible for multi-server deployments.

• round_robin: Maintain connections to all resolved addresses and rotate RPCs across them. Simple and effective for small clusters. But it requires DNS or another mechanism to return all server addresses.

• xDS (External Discovery Service): A sophisticated protocol where a control plane (like Envoy, Istio, or Traffic Director) pushes routing configuration to gRPC clients. The client receives the server list, load balancing policy, health status, and even per-route configuration via the xDS protocol. This is what Google uses internally for all gRPC traffic.

Proxy-Based Load Balancing

For teams that cannot use client-side load balancing (legacy clients, polyglot environments), an L7 proxy like Envoy can terminate gRPC connections and distribute RPCs across backends. Envoy understands the gRPC protocol (HTTP/2 + gRPC framing) and can load-balance at the individual RPC level, not the connection level. This is the most common approach in Kubernetes environments.

Interceptors and Middleware

gRPC interceptors are the equivalent of HTTP middleware. They wrap every RPC call, running before and after the handler. The interceptor chain is identical on client and server, making cross-cutting concerns (auth, logging, metrics, retry) consistent.

An interceptor receives the RPC context (including metadata, deadline, and cancellation), the request, and a "next" function to invoke the next interceptor or the actual handler. This is the chain-of-responsibility pattern.

Common interceptor patterns:

How Interceptors Chain

A gRPC request passes through interceptors like layers of an onion. On the client side:

Your code → Auth interceptor → Metrics interceptor → Retry interceptor → Wire

On the server side:

Wire → Recovery interceptor → Auth interceptor → Logging interceptor → Your handler

Each interceptor can:

• Modify the request (add metadata, transform the message)
• Short-circuit the chain (return an error without calling the next interceptor, useful for auth rejection)
• Modify the response (transform the message, add trailing metadata)
• Record timing (wrap the call in a timer for metrics)

In Go, interceptors are functions passed to grpc.NewServer() or grpc.Dial(). In Java, they are classes implementing ServerInterceptor or ClientInterceptor. The go-grpc-middleware library provides production-ready interceptors for logging, recovery, auth, and retry that I recommend using instead of writing your own.

What Happens When Things Break

gRPC's failure modes are tightly coupled to HTTP/2's connection model. Understanding these failures is essential for building resilient services.

Connection Failures

When the underlying TCP connection drops (network partition, server crash), all active RPCs on that connection fail simultaneously with UNAVAILABLE. This is the downside of multiplexing: one connection failure affects all concurrent RPCs. gRPC's reconnection logic kicks in with exponential backoff (starting at 1 second, maxing at 120 seconds by default), but all in-flight RPCs are lost.

For comparison, with HTTP/1.1 (one connection per request), a connection failure affects exactly one request. With gRPC, it can affect hundreds. This is why keepalive configuration is critical: you want to detect dead connections quickly (short keepalive intervals) without overwhelming the network with pings.

Graceful Shutdown with GOAWAY

When a gRPC server needs to shut down (rolling deployment, scaling down), it sends an HTTP/2 GOAWAY frame. This tells clients: "I will finish processing your in-flight RPCs, but do not send new ones on this connection." Clients redirect new RPCs to other connections while waiting for existing ones to complete. This is how zero-downtime deployments work with gRPC.

The GOAWAY frame includes the last stream ID the server will process. Any RPC started after that stream ID is immediately rejected by the client without hitting the wire. The client transparently retries these on a new connection to a different server (if using load balancing).

Partial Failures in Streaming

Streaming RPCs have a unique failure mode: partial delivery. If a server-streaming RPC sends 500 messages and then the connection drops, the client has received some messages but not all. There is no built-in mechanism to resume from where it left off. Your application must implement its own checkpointing (e.g., "send me results starting from offset 500").

Performance Characteristics

How This Compares to Alternatives

I reach for gRPC when I control both client and server, need high-throughput low-latency communication, or need streaming. The schema-first approach with codegen saves enormous time on large teams where API contracts matter. I stick with REST/JSON for public APIs, browser-facing endpoints, and cases where human readability of the wire format aids debugging. I use GraphQL when the primary consumer is a frontend team that needs flexible querying of a complex data model.

When REST is Actually Better

I want to be clear about cases where gRPC is not the right choice:

• Public APIs: Your consumers use dozens of different languages and tools. REST is universally understood, and tools like curl, Postman, and browser dev tools work natively. gRPC requires codegen and a protobuf compiler, which raises the bar for API consumers.
• Low request volume: At 10 RPCs per second, the serialization difference between JSON and protobuf is negligible (microseconds). The operational complexity of maintaining .proto files, code generation pipelines, and gRPC-specific debugging tools is not justified.
• CRUD-heavy applications: REST's resource-oriented model (GET /users/123, POST /users) maps naturally to databases and caches. gRPC's method-oriented model (GetUser, CreateUser) requires more design effort for simple CRUD.
• Debugging in production: JSON payloads are human-readable in logs, network captures, and error messages. Protobuf payloads are binary blobs that require the .proto schema to decode. When you are debugging at 3 AM, human-readable wire formats save time.

When gRPC Wins Decisively

• High-throughput internal services: At 10,000+ RPCs per second, protobuf's 10x serialization speed and smaller message size translate to meaningful CPU and bandwidth savings.
• Polyglot microservices: gRPC's codegen produces typed clients in 10+ languages from a single .proto file. Every team gets a client that matches the server's API exactly, with no manual integration work.
• Streaming use cases: Real-time data feeds, log streaming, long-running computations with incremental results. gRPC's native streaming is far cleaner than bolting WebSocket or SSE onto REST.
• Service mesh environments: Envoy, Istio, and Linkerd have first-class gRPC support. Retry policies, circuit breakers, and traffic management work at the individual RPC level.

Interview Cheat Sheet

• When asked "why gRPC over REST": "Three specific gains: protobuf is 5-10x smaller and 10-100x faster to parse than JSON, HTTP/2 multiplexing eliminates head-of-line blocking and reduces connections from N to 1, and built-in streaming eliminates the need for WebSocket or SSE."

• When asked about HTTP/2: "gRPC maps each RPC to an HTTP/2 stream. Streams are multiplexed over a single TCP connection, so 100 concurrent RPCs share one connection. HPACK compresses headers, and flow control prevents fast producers from overwhelming slow consumers."

• When asked about protobuf compatibility: "Protobuf uses field numbers, not names, on the wire. You can add new fields (new numbers), remove fields (reserve the number), and rename fields freely. You must never reuse a field number or change a field's wire type."

• When asked about streaming: "gRPC supports four patterns: unary (request-response), server streaming (one request, many responses), client streaming (many requests, one response), and bidirectional streaming (both sides send independently). All map to HTTP/2 streams."

• When asked about deadlines: "gRPC propagates deadlines across service chains via the grpc-timeout header. If Service A sets a 5s deadline and Service B uses 2s, Service C automatically gets a 3s deadline. Cancellations propagate too, preventing wasted work on already-abandoned requests."

• When asked about load balancing: "HTTP/2 multiplexing breaks L4 load balancers because all RPCs go over one connection. Solutions: client-side load balancing (round_robin or xDS), L7 proxy (Envoy) that understands gRPC framing, or a service mesh that handles it transparently."

• When asked about error handling: "gRPC has 17 standard status codes (OK, CANCELLED, DEADLINE_EXCEEDED, NOT_FOUND, etc.) plus rich error details via the google.rpc.Status message. Always return appropriate status codes, not just INTERNAL for everything. UNAVAILABLE signals a retryable error, INTERNAL signals a bug."

• When asked about browser support: "Native gRPC requires HTTP/2 with trailers, which browsers do not expose via fetch/XHR. gRPC-Web is a variant that works over HTTP/1.1 using a proxy (like Envoy) that translates between gRPC-Web and native gRPC. The client uses a generated JS/TS stub."

• When asked about keepalive: "gRPC uses HTTP/2 PING frames for keepalive. Configure keepalive-time (how often to ping, default 2 hours, I use 30 seconds), keepalive-timeout (how long to wait for a pong, default 20 seconds), and permit-keepalive-without-calls (ping even when no active RPCs, essential for detecting dead connections early)."

• When asked about large messages: "The default max message size is 4 MB. If you need to send larger data, use streaming instead of increasing the limit. A client-streaming RPC can upload a 1 GB file as 4 MB chunks. This uses constant memory on both sides instead of buffering the entire file."

Test Your Understanding

Quick Recap

1. gRPC uses Protocol Buffers for binary serialization (5-10x smaller than JSON, 10-100x faster to parse) and HTTP/2 for multiplexed transport.
2. Protobuf achieves forward and backward compatibility through field numbers, not field names. Never reuse or change the type of a field number.
3. HTTP/2 multiplexes all RPCs over a single TCP connection using streams, eliminating per-request connection overhead and HTTP-level head-of-line blocking.
4. gRPC supports four RPC patterns: unary, server streaming, client streaming, and bidirectional streaming, all mapped to HTTP/2 streams.
5. Deadline propagation automatically adjusts timeouts across service chains, preventing wasted work. Always set deadlines on every RPC.
6. Client-side load balancing (round_robin, xDS) avoids the L4 load balancer trap where HTTP/2 multiplexing sends all traffic to one server.
7. Interceptors provide a symmetric middleware chain on client and server for auth, logging, metrics, tracing, and retry.
8. Use gRPC for internal service-to-service communication where you control both ends. Use REST/JSON for external and browser-facing APIs.

Related Concepts

• HTTP/2 protocol internals: The transport layer that powers gRPC, including stream multiplexing, HPACK header compression, and flow control.
• Protocol Buffers encoding: The binary serialization format including varint encoding, wire types, field numbers, and schema evolution rules.
• Service mesh (Istio/Envoy): The infrastructure layer that handles gRPC load balancing, mTLS, and observability transparently via sidecar proxies.
• API gateway design: How to expose gRPC services as REST endpoints using tools like grpc-gateway for external consumers.
• Distributed tracing (OpenTelemetry): How gRPC interceptors propagate trace context across service boundaries for end-to-end request visibility.