Storage types: block vs. file vs. object
When to use block storage, file storage, or object storage: performance characteristics, consistency models, cost, use cases for each, and how cloud services map to these types.
TL;DR
| Dimension | Block Storage | File Storage | Object Storage |
|---|---|---|---|
| Access model | Raw blocks, OS filesystem on top | Network filesystem (NFS/SMB), POSIX semantics | HTTP API (PUT/GET/DELETE), flat namespace |
| Latency | Sub-ms (NVMe: 100us, EBS gp3: 1-5ms) | 1-10ms (network + filesystem overhead) | 10-100ms (HTTP first byte) |
| Best for | Databases, boot volumes, low-latency random I/O | Shared access across servers, legacy apps, media processing | Images, videos, backups, data lakes, static assets |
| Cost (AWS) | $0.08-0.125/GB/mo (gp3/io2) | $0.30/GB/mo (EFS Standard) | $0.023/GB/mo (S3 Standard) |
| Scalability | Fixed size, attached to instance | Elastic, shared mount | Virtually unlimited (exabyte-scale) |
Default answer: Object storage for anything immutable or rarely updated (media, backups, static assets). Block storage for databases and applications requiring low-latency random I/O. File storage only when multiple servers need POSIX-compatible shared access.
The Framing
Your team launches a new service. The database runs on an EC2 instance with a 500 GB EBS volume. User-uploaded images go into the same EBS volume. Shared configuration files that all servers need live in Redis serialized as a blob.
Six months later: the EBS volume is 80% full because images take up 450 GB. Scaling the database to a larger instance means detaching and reattaching the volume (downtime). Configuration changes require manually updating Redis. And the bill for 500 GB of io2 storage ($62.50/month) is 13x what the same images would cost in S3 ($4.60/month per 200 GB).
Every storage type exists because the others can't do something it can. Block storage gives you sub-millisecond random I/O for databases. Object storage gives you infinite capacity for pennies per GB. File storage gives you shared POSIX access that legacy applications expect. Using the wrong type doesn't fail immediately. It fails at scale, at cost, or at operational complexity.
I've seen teams store everything on EBS and wonder why their AWS bill is astronomical. And I've seen teams try to run a database on S3 and wonder why latency is terrible. The rule is simple: match storage type to access pattern.
How Each Works
Block Storage: Raw Blocks, Maximum Performance
Block storage exposes fixed-size blocks (typically 512 bytes or 4 KB) to the operating system. The OS formats them with a filesystem (ext4, XFS, NTFS) and treats them as a local disk. Network-attached block storage (AWS EBS, GCP Persistent Disk, Azure Managed Disks) makes this model work over a network with near-local latency.
The key characteristic is random I/O at any byte offset. A database can read page 4,721 of a B-tree index, modify it, and write it back, all within microseconds for local NVMe or low single-digit milliseconds for network-attached volumes. This byte-addressable random access is what makes block storage essential for databases, and it's the one thing object and file storage fundamentally cannot provide.
# AWS EBS gp3 volume configuration
VolumeType: gp3
Size: 100 # GB
Iops: 3000 # Baseline (free), up to 16,000
Throughput: 125 # MB/s baseline, up to 1,000
Encrypted: true # AES-256 at rest
# AWS EBS io2 Block Express (high-performance)
VolumeType: io2
Size: 100
Iops: 64000 # Up to 256,000 for Block Express
Throughput: 4000 # MB/s
MultiAttachEnabled: false # true for shared-nothing clusters
Performance tiers matter. gp3 provides 3,000 baseline IOPS for free; io2 Block Express scales to 256,000 IOPS for latency-sensitive workloads. NVMe instance storage (i3, i4 instances) provides the lowest latency (~100 microseconds) but is ephemeral: data is lost when the instance stops.
RAID striping (RAID 0 across multiple EBS volumes) multiplies throughput linearly. Four gp3 volumes striped together provide 12,000 IOPS and 500 MB/s throughput. I use this pattern for write-heavy databases that need more throughput than a single volume provides.
Multi-Attach is available on io2 volumes, allowing a single volume to be attached to up to 16 Nitro-based instances simultaneously. This enables shared-nothing cluster architectures where multiple nodes access the same block device. However, the application must manage concurrent writes (the filesystem doesn't coordinate this for you). Oracle RAC and some custom distributed databases use this pattern.
EBS snapshots are incremental and stored in S3 behind the scenes. The first snapshot copies the full volume; subsequent snapshots copy only changed blocks. Snapshots can be copied across regions for disaster recovery. You can create a new volume from any snapshot in any availability zone, which makes migration and scaling straightforward. EBS also supports fast snapshot restore (FSR) which pre-warms the volume data, eliminating the performance penalty of reading from snapshot on first access. Without FSR, the first read of each block fetches from S3 (slow); with FSR, the volume performs at full speed immediately after creation.
Encryption at rest is available on all EBS volume types using AWS KMS keys. There's no performance penalty for encryption (it's handled by the Nitro controller hardware). For compliance requirements, encrypted volumes ensure data is protected even if the underlying hardware is accessed. Once you enable default encryption for a region, all new volumes are automatically encrypted.
Data Durability Comparison
Durability is how likely your data survives hardware failure. The three storage types have very different durability profiles:
- EBS (block): Durability is 99.999% (designed for 0.1-0.2% annual failure rate). Data is replicated within a single Availability Zone. If the AZ fails, the volume is lost unless you have snapshots (stored in S3, which has higher durability). For critical databases, combine EBS with daily snapshots and cross-region replication.
- S3 (object): Designed for 99.999999999% (11 nines) durability across 3+ AZs. AWS calculates this as less than 1 object lost per 10 million objects stored over 10,000 years. S3 is the most durable storage option available on any cloud provider.
- EFS (file): Data is replicated across multiple AZs within a region, providing similar durability to S3. EFS Standard stores data redundantly across 3+ AZs.
- Instance Store (NVMe): Zero durability guarantee. Data is lost on instance stop, termination, or hardware failure. Use only for temporary scratch data that can be regenerated.
The durability hierarchy: S3 (11 nines) > EFS (multi-AZ) > EBS (single-AZ) > Instance Store (ephemeral). For any data that matters, either use S3/EFS or back up EBS with snapshots.
| EBS Volume Type | IOPS (max) | Throughput (max) | Latency | Cost (200 GB/mo) | Best For |
|---|---|---|---|---|---|
| gp3 | 16,000 | 1,000 MB/s | 1-5ms | $16 + IOPS | General workloads |
| io2 Block Express | 256,000 | 4,000 MB/s | sub-ms | $25 + $0.065/IOPS | High-perf databases |
| st1 (HDD) | 500 | 500 MB/s | 5-10ms | $9 | Sequential throughput |
| NVMe instance | 400,000+ | 7,000+ MB/s | ~100us | Included with instance | Temporary scratch |
File Storage: Shared Access with POSIX Semantics
File storage provides a network filesystem that multiple servers mount simultaneously. All servers see the same files and directories with POSIX semantics (read, write, seek, lock). NFS (Network File System) for Linux and SMB/CIFS for Windows are the primary protocols.
AWS EFS (Elastic File System) is the canonical cloud example. It auto-scales capacity (no pre-provisioning), supports thousands of concurrent connections, and charges per GB actually used ($0.30/GB/month for Standard, $0.016/GB/month for Infrequent Access).
# Mount EFS on multiple EC2 instances
sudo mount -t nfs4 \
-o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 \
fs-0123456789abcdef0.efs.us-east-1.amazonaws.com:/ /mnt/shared
# All instances now see the same filesystem
ls /mnt/shared/config/
# app-config.yaml feature-flags.json tls-certs/
# FSx for Lustre: high-throughput parallel filesystem
# Up to 1,000 GB/s aggregate for HPC / ML training
FSx variants serve specialized needs. FSx for Lustre provides parallel filesystem performance (hundreds of GB/s throughput) for machine learning training and high-performance computing. FSx for Windows File Server provides SMB-compatible storage. FSx for NetApp ONTAP provides enterprise NAS features.
The trade-off: file storage is expensive per GB compared to object storage (13x the cost of S3 Standard) and slower than block for random I/O. Use it when you genuinely need multiple servers writing to the same filesystem simultaneously, not as general-purpose storage.
EFS performance has two dimensions: throughput mode and performance mode. Bursting throughput scales with stored size (50 KB/s per GB stored). Provisioned throughput lets you decouple throughput from storage size. For small filesystems (under 1 TB) with high throughput needs, provisioned throughput avoids the bursting bottleneck. Max I/O performance mode handles highly parallelized workloads at the cost of slightly higher per-operation latency.
NFS performance limitations are the main reason to avoid file storage when possible. NFS adds 1-10ms of latency per operation compared to local disk, and metadata-heavy operations (listing directories with thousands of files, checking file permissions) are significantly slower than on local block storage. Applications that open and close many small files rapidly (build systems, package managers) perform poorly on NFS. The rule: if your application's I/O pattern involves many small random operations, block storage is the right choice even if you need shared access (use a database or object store instead).
Object Storage: Infinite Capacity at Pennies per GB
Object storage uses a flat namespace where each object is identified by a key (string) and stored as an immutable blob. There's no filesystem hierarchy (the "folders" in S3 are just key prefixes). Objects are accessed via HTTP API: PUT to write, GET to read, DELETE to remove.
import boto3
s3 = boto3.client('s3')
# Upload an image
s3.put_object(
Bucket='my-app-images',
Key='users/456/avatar.jpg',
Body=image_bytes,
ContentType='image/jpeg',
ServerSideEncryption='AES256'
)
# Generate a pre-signed URL (temporary access, no auth needed)
url = s3.generate_presigned_url(
'get_object',
Params={'Bucket': 'my-app-images', 'Key': 'users/456/avatar.jpg'},
ExpiresIn=3600 # 1 hour
)
# Lifecycle policy: move to Glacier after 90 days
lifecycle_config = {
'Rules': [{
'ID': 'archive-old-images',
'Status': 'Enabled',
'Transitions': [
{'Days': 90, 'StorageClass': 'GLACIER'},
{'Days': 365, 'StorageClass': 'DEEP_ARCHIVE'}
]
}]
}
S3 provides strong read-after-write consistency (since December 2020). Throughput scales per-prefix: 5,500 PUT/COPY/POST/DELETE requests per second and 3,500 GET/HEAD requests per second per prefix. Distributing objects across prefixes scales linearly.
Storage classes provide cost tiers. S3 Standard ($0.023/GB/mo) for frequently accessed data. S3 Infrequent Access ($0.0125/GB/mo) for data accessed less than once a month. S3 Glacier Instant Retrieval ($0.004/GB/mo) for archival with millisecond access. S3 Glacier Deep Archive ($0.00099/GB/mo) for compliance archives rarely accessed. Lifecycle policies move objects between tiers automatically.
Versioning keeps every version of every object, enabling rollback. Server-side encryption (SSE-S3, SSE-KMS, SSE-C) encrypts at rest. Cross-region replication copies objects to another region for disaster recovery. Object Lock provides WORM (Write Once Read Many) compliance for regulatory requirements that mandate immutable storage.
Event notifications trigger Lambda functions, SQS messages, or SNS notifications when objects are created, modified, or deleted. This enables reactive architectures: a user uploads an image to S3, which triggers a Lambda that creates thumbnails in multiple sizes and writes them back to S3. The entire pipeline runs without any server management.
# S3 event notification configuration (Terraform)
resource "aws_s3_bucket_notification" "upload_trigger" {
bucket = aws_s3_bucket.uploads.id
lambda_function {
lambda_function_arn = aws_lambda_function.thumbnail_generator.arn
events = ["s3:ObjectCreated:*"]
filter_prefix = "uploads/"
filter_suffix = ".jpg"
}
}
S3 Transfer Acceleration uses CloudFront edge locations to speed up uploads from distant clients. Instead of uploading directly to the S3 bucket's region, the client uploads to the nearest edge location, which routes the data to S3 over AWS's optimized backbone network. This can improve upload speed by 50-500% for cross-continental transfers.
S3 Access Points simplify access management for shared buckets. Instead of managing complex bucket policies with conditions for different applications, each application gets its own access point with its own permissions. A data lake bucket might have separate access points for the analytics team (read-only), the ETL pipeline (read-write to specific prefixes), and the ML team (read-only with VPC restriction). Each access point has its own DNS name and IAM policy, which is cleaner than a single bucket policy with 50 condition blocks.
S3 Batch Operations process millions of objects in bulk: copy across buckets, invoke Lambda on each object, change storage class, or restore from Glacier. For large-scale migrations or transformations, Batch Operations handles retry, throttling, and progress tracking automatically rather than requiring custom scripts.
Object metadata is stored alongside the data. System metadata (Content-Type, Content-Length, Last-Modified) is set automatically. User metadata (up to 2 KB of custom key-value pairs) is set by the application. This metadata is returned with every GET request header and can be used for application logic without reading the object body. For example, storing x-amz-meta-upload-source: mobile on every object lets you filter analytics by upload origin without scanning object contents.
Multipart uploads handle large files efficiently. For objects over 100 MB, the SDK splits the upload into parts (default 8 MB each), uploads them in parallel, and assembles them server-side. A 5 GB video upload becomes 640 parallel 8 MB transfers, completing in a fraction of the time a single-stream upload would take. If one part fails, only that part is retried. The maximum object size is 5 TB.
S3 Select and Glacier Select let you query objects using SQL expressions without downloading the entire object. For a 1 GB CSV log file, you can SELECT * WHERE status = 'ERROR' and transfer only the matching rows. This reduces data transfer costs and latency for analytics on raw S3 data.
-- S3 Select: query a CSV object without full download
SELECT s.request_id, s.timestamp, s.error_message
FROM s3object s
WHERE s.status = 'ERROR'
AND CAST(s.timestamp AS TIMESTAMP) > TIMESTAMP '2026-03-01'
Head-to-Head Comparison
| Dimension | Block Storage | File Storage | Object Storage | Verdict |
|---|---|---|---|---|
| Latency | Sub-ms (NVMe: 100us, EBS: 1-5ms) | 1-10ms (network + FS overhead) | 10-100ms (HTTP first byte) | Block for latency-critical |
| Throughput | Up to 4,000 MB/s (io2 Block Express) | Up to 1,000 GB/s (FSx Lustre) | 100+ Gbps aggregate (multi-prefix) | File (Lustre) for parallel workloads |
| IOPS | Up to 256K (io2 Block Express) | Thousands (EFS elastic) | 5,500 PUT + 3,500 GET per prefix | Block for random I/O |
| Capacity | Up to 64 TiB per volume | Elastic, petabyte-scale | Virtually unlimited (exabyte) | Object for scale |
| Cost per GB/mo | $0.08 (gp3) to $0.125 (io2) | $0.30 (EFS Standard) | $0.023 (S3) to $0.004 (Glacier) | Object, 5-15x cheaper |
| Access model | Byte-level random read/write | Files with POSIX (read, write, seek, lock) | Whole-object via HTTP (PUT/GET/DELETE) | Depends on workload |
| Shared access | Single instance (Multi-Attach is limited) | Multiple servers simultaneously | Any client with HTTP access | File for shared mounts |
| Durability | 99.999% (single AZ) | 99.999999999% (EFS, 11 nines) | 99.999999999% (S3, 11 nines) | Object and file most durable |
| Update model | Modify any byte in place | Modify bytes within files | Replace entire object (immutable) | Block most flexible |
| Versioning | Snapshots (manual/scheduled) | None built-in (backup separately) | S3 versioning (every PUT creates new version) | Object has built-in versioning |
| Best for | Databases, boot volumes | Shared filesystem, legacy apps | Media, backups, data lakes | Each has its niche |
The fundamental tension: block storage gives you the lowest latency and most flexible access at the highest cost per GB. Object storage gives you the lowest cost and largest scale with the slowest, most limited access model. File storage fills the middle ground when you must share a filesystem across servers.
When Block Storage Wins
Block storage is the right choice when your application requires low-latency, random byte-level I/O.
Database data files. PostgreSQL, MySQL, MongoDB, and every other database store their data files on a block device. The database engine needs to read arbitrary 8 KB pages, write WAL entries sequentially, and fsync to guarantee durability. Object storage's HTTP API and 10-100ms latency make this impossible. File storage works technically but adds unnecessary latency and cost.
Boot volumes and OS disks. Every EC2 instance needs a root volume with an operating system. This must be block storage. EBS gp3 is the standard choice (3,000 IOPS baseline included in the volume cost).
High-IOPS transactional workloads. Payment processing, real-time bidding, and high-frequency trading require sub-millisecond I/O. io2 Block Express at 256,000 IOPS handles workloads that no other storage type can touch.
RAID striping for aggregated throughput. If a single volume can't provide enough throughput, stripe 4-8 EBS volumes with software RAID 0 for linear scaling. A database that needs 2,000 MB/s write throughput can get it from eight gp3 volumes striped together.
Temporary high-performance scratch space. NVMe instance storage (i3, i4 instances) provides local SSD with ~100-microsecond latency. For temporary workloads (sorting, rendering, Kafka page cache), this is 10x faster than network-attached EBS and free with the instance.
Database write-ahead logs (WAL). WAL writes are sequential and latency-sensitive. A separate EBS volume dedicated to WAL (with higher IOPS than the data volume) is a common optimization for write-heavy databases. PostgreSQL's wal_level, MySQL's innodb_log_file_size, and SQL Server's transaction log all benefit from dedicated block storage with provisioned IOPS.
High-performance computing (HPC) scratch. Molecular simulations, weather modeling, and financial Monte Carlo simulations generate terabytes of intermediate data. NVMe instance storage or io2 Block Express provides the IOPS and throughput these workloads demand. The data is ephemeral (regeneratable), so the lack of durability on instance storage is acceptable.
When File Storage Wins
File storage fills a niche: shared POSIX-compatible filesystem across multiple servers. It's rarely the first choice, but sometimes the only choice.
Legacy applications that expect a local filesystem. Older CMS platforms, WordPress multisite installations, and applications that use fopen()/fread()/fwrite() for shared state need a mounted filesystem. Refactoring them to use S3 APIs is often impractical.
Shared configuration and certificates. When 50 servers need the same TLS certificates, feature flags, or configuration files, an NFS mount is simpler than distributing files through a deployment pipeline. EFS provides this with zero pre-provisioning.
Media processing pipelines with shared workspace. Video transcoding, image processing, or ML training where multiple worker nodes read source files, write intermediate results, and produce outputs in a shared directory structure. FSx for Lustre provides the throughput for this (up to 1,000 GB/s aggregate).
Container shared volumes. ECS and EKS tasks that need shared persistent storage across containers. EFS integrates natively with both. While container-native storage (EBS CSI driver) works for single-container volumes, shared-across-containers requires EFS or a similar shared filesystem.
ML training with shared datasets. Training jobs across multiple GPU instances need to read the same dataset simultaneously. FSx for Lustre backed by S3 provides the bandwidth (hundreds of GB/s aggregate) needed for large-scale training. Each GPU instance mounts the same Lustre filesystem, reading training batches in parallel without competing for S3 API throughput limits.
Home directories in enterprise environments. When hundreds of developers need persistent home directories that survive instance termination, EFS provides elastic shared storage. Each developer's workspace is a directory on the shared mount, accessible from any instance they connect to.
WordPress and CMS deployments. Multi-server WordPress installations need shared access to the wp-content/uploads directory. All web servers must see the same uploaded media files. EFS provides this with no application changes: mount EFS at /var/www/html/wp-content/uploads on every web server. This is one of the most common legitimate EFS use cases.
CI/CD build caches. Build agents that share a dependency cache (npm modules, Maven artifacts, Docker layers) benefit from EFS. Instead of each agent downloading dependencies from the internet, they read from a shared cache directory. The cache fills on first access and subsequent builds are faster. EFS Intelligent-Tiering keeps frequently accessed cache files in Standard tier and moves stale files to IA automatically.
Video surveillance and media ingest. On-premises NAS (Network Attached Storage) devices expose file storage for camera systems that write continuous video streams. Cloud equivalents use EFS or FSx for real-time file ingestion before processing and archival to S3. The file storage layer buffers incoming streams while processing workers read, transcode, and push final output to object storage for long-term retention.
When Object Storage Wins
Object storage is the right choice for most data that doesn't need sub-10ms latency or byte-level updates.
User-generated content. Photos, videos, documents, avatar images. S3 handles billions of objects with 11 nines of durability (99.999999999%). Pre-signed URLs provide time-limited access without exposing bucket credentials. CDN integration (CloudFront) caches objects at edge locations globally.
Data lakes and analytics. Raw event data, log files, Parquet/ORC columnar files for Spark/Athena/BigQuery. S3 is the standard data lake storage layer. At $0.023/GB/month, storing 10 TB of raw analytics data costs $230/month. The same data on EBS would cost $800-1,250/month.
Backups and disaster recovery. Database snapshots, application backups, point-in-time recovery files. S3 Glacier ($0.004/GB/month) and Deep Archive ($0.00099/GB/month) provide archival storage at minimal cost. A 1 TB database backup costs $1/month in Glacier Deep Archive.
Static website assets. CSS, JavaScript bundles, images, and fonts served directly from S3 or via CloudFront. S3 static website hosting eliminates the need for a web server for static content.
ML model artifacts. Model checkpoints, training datasets, and inference artifacts. S3 provides the capacity for multi-terabyte model files and integrates natively with SageMaker, Bedrock, and other ML services.
Log aggregation and compliance. Application logs, audit trails, and access logs flow to S3 via Kinesis Firehose or Fluentd. S3's durability guarantees (11 nines) exceed regulatory requirements. Object Lock (WORM mode) prevents deletion or modification for compliance retention periods. S3 is the standard destination for centralized log storage.
Database backups. pg_dump outputs, mysqldump files, and SQL Server .bak files go to S3. Automated backup pipelines (AWS Backup, custom scripts) upload nightly, and lifecycle policies move old backups to Glacier automatically. A 1 TB daily database backup retained for 30 days costs $23/TB/month in S3 Standard; moving backups older than 7 days to Glacier drops effective cost to under $5/month.
Lifecycle management for cost optimization. S3 lifecycle policies automatically move objects through storage tiers: Standard (first 30 days) to Infrequent Access (days 31-90) to Glacier (day 91+) to Deep Archive (day 365+). I've seen teams cut storage costs by 60-70% by implementing lifecycle policies on their largest buckets.
S3 Intelligent-Tiering automates tier selection without lifecycle rules. Objects are moved between frequent and infrequent access tiers based on actual access patterns. For data with unpredictable access patterns, Intelligent-Tiering ($0.0025 per 1,000 objects monitoring fee) eliminates the guesswork. The monitoring fee is negligible for large objects but adds up for millions of small files.
The Nuance
Most real systems use all three storage types. The question is not "which storage type" but "which storage type for which data."
A typical web application architecture:
- Block: PostgreSQL database files on EBS gp3 (low-latency random I/O)
- Object: User uploads, static assets, and backups in S3 (cheap, durable, scalable)
- File: Shared configuration or certificates on EFS (multi-server POSIX access)
The common mistake is using one storage type for everything. Block storage for images is expensive and doesn't scale. Object storage for database files is impossibly slow. File storage for everything is expensive and adds latency unnecessarily.
Another common anti-pattern: storing large binary files (images, PDFs, videos) as database BLOBs. This bloats the database, slows backups, increases replication lag, and consumes database connections for file serving. The fix is simple: store the file in S3, store only the S3 key in the database. Reads go directly to S3 or CloudFront, completely bypassing the database.
For interview purposes, mentioning the BLOB anti-pattern unprompted signals real-world experience. Most candidates store everything in the database or everything in S3 without explaining why. The nuanced answer is always "metadata in the database, binary data in object storage, with pre-signed URLs for direct client access."
A subtler mistake: using EFS for data that only needs to be shared at deploy time, not at runtime. If all servers need the same ML model file but only load it at startup, download it from S3 during boot instead of paying for EFS. EFS is for data that changes during runtime and must be immediately visible to all servers.
Snapshot and Backup Strategy
EBS snapshots are incremental: the first snapshot copies the entire volume, subsequent snapshots copy only changed blocks. A 500 GB volume where 5 GB changes daily costs roughly $2.50/day for the daily snapshot (only 5 GB of new data). Cross-region snapshot copies provide disaster recovery. AWS Backup automates snapshot lifecycle: create daily, retain 7 daily + 4 weekly + 12 monthly.
S3 versioning serves a similar purpose for object storage: every PUT creates a new version. Combined with MFA Delete (requires multi-factor authentication to permanently delete), S3 versioning provides protection against accidental deletion and ransomware. For compliance, S3 Object Lock (WORM mode) prevents any deletion or modification for a configurable retention period.
EFS doesn't support snapshots natively, but AWS Backup can create EFS backups. These are full filesystem-level backups stored in an S3-backed vault. For point-in-time recovery, EFS backups are more limited than EBS snapshots (you restore the entire filesystem, not individual files).
Cost Comparison: 1 TB of Data
| Storage Type | Service | Monthly Cost | Notes |
|---|---|---|---|
| Block (gp3) | EBS | $80 | Fixed provisioned size |
| Block (io2, 10K IOPS) | EBS | $80 + $650 = $730 | Per-IOPS charges dominate |
| File (Standard) | EFS | $307 | Pay per GB used |
| File (IA) | EFS | $16 + request fees | Per-request charges add up |
| Object (Standard) | S3 | $23 | Most cost-effective |
| Object (Glacier IR) | S3 | $4 | Archival with ms retrieval |
| Object (Deep Archive) | S3 | $1 | Compliance, rare access |
The 15x cost difference between EFS Standard ($307) and S3 Standard ($23) per TB is the most commonly overlooked storage cost optimization. For data that doesn't need shared POSIX access, S3 is almost always the right answer.
Another cost optimization that teams miss: EBS volume right-sizing. CloudWatch metrics (VolumeReadBytes, VolumeWriteBytes, VolumeReadOps, VolumeWriteOps) reveal actual usage patterns. Many teams provision io2 at 30,000 IOPS but actually use only 3,000. Switching to gp3 saves thousands of dollars per month. AWS Compute Optimizer and third-party tools like CloudHealth automatically flag over-provisioned volumes.
When to Combine Storage Types
The most mature architectures use all three storage types deliberately:
- Web application: EBS gp3 for PostgreSQL (50 GB, $4/mo), S3 for user uploads and static assets (200 GB, $4.60/mo), EFS only if multiple servers need shared config files.
- ML pipeline: EBS gp3 for the orchestration database, FSx for Lustre for shared training data (high throughput), S3 for raw datasets and model artifacts (long-term storage).
- Media platform: EBS io2 for the metadata database (high IOPS for search), S3 for all media files (videos, images, thumbnails), CloudFront for global delivery, S3 Glacier for archived content.
- Analytics platform: EBS gp3 for operational databases, S3 for the data lake (Parquet/ORC files), no EFS needed (analytics workers read from S3 directly).
The pattern is consistent: block storage for stateful, latency-sensitive workloads (databases). Object storage for everything else. File storage only when shared POSIX access is genuinely required.
Don't overthink the storage decision in interviews. Draw the three-layer diagram (block for DB, object for assets, file for shared access) and explain the cost and access pattern reasoning. If the interviewer asks about a specific storage choice ("where would you store images?"), the answer is almost always "S3 with CloudFront" with a brief explanation of pre-signed URLs and lifecycle policies. If they ask about the database storage, the answer is "EBS gp3 unless we need more than 16,000 IOPS, then io2." These answers, backed by actual dollar amounts, demonstrate practical infrastructure knowledge that most candidates lack.
Common Storage Anti-Patterns to Avoid
- Storing binary files in the database. Use S3 for images, PDFs, and videos. Store only the S3 key in the database.
- Using io2 when gp3 is sufficient. Check CloudWatch IOPS metrics before upgrading. gp3 handles up to 16,000 IOPS at a fraction of io2 cost.
- Ignoring S3 lifecycle policies. Data that isn't accessed doesn't need to stay in S3 Standard. Lifecycle policies to IA and Glacier cut costs by 60-70%.
- Using EFS as a general file store. EFS is 13x more expensive than S3 per GB. Use it only for shared POSIX filesystem requirements.
- No encryption. Enable default encryption for all EBS volumes and S3 buckets. There's no performance penalty and it's required for compliance.
- Not using multipart uploads for large files. Files over 100 MB should use multipart upload for reliability and parallelism. The AWS SDK handles this automatically with
TransferManager. - Ignoring S3 request costs. S3 charges per request ($0.005 per 1,000 PUTs, $0.0004 per 1,000 GETs). Applications that make millions of small-object requests accumulate significant request charges. Batch small objects together or use S3 Select to reduce request volume.
Storage is one of the highest-impact cost optimization areas in any cloud architecture. Getting it right saves thousands of dollars per month at moderate scale and millions at hyperscale.
The single most important takeaway: match storage type to access pattern. Databases need block storage for low-latency random I/O. Media and backups need object storage for cost-efficient bulk storage. Shared configuration and legacy workloads need file storage for POSIX compatibility. When in doubt, default to object storage unless you have a specific latency or filesystem requirement that forces you elsewhere.
Cloud Provider Equivalents
| Category | AWS | GCP | Azure |
|---|---|---|---|
| Block (general) | EBS gp3 | Persistent Disk (pd-balanced) | Premium SSD v2 |
| Block (high IOPS) | EBS io2 Block Express | Persistent Disk (pd-extreme) | Ultra Disk |
| Block (local NVMe) | Instance Store (i3/i4) | Local SSD | Temp Disk (NVMe) |
| File (NFS) | EFS | Filestore | Azure Files (NFS) |
| File (HPC) | FSx for Lustre | Filestore (HPC) | Azure Managed Lustre |
| Object | S3 | Cloud Storage (GCS) | Blob Storage |
| Object (archive) | S3 Glacier/Deep Archive | Archive Storage | Cool/Archive Blob |
The concepts are identical across providers. The specific service names and pricing differ, but if you understand block/file/object on AWS, you understand it everywhere. Interviews rarely ask about GCP or Azure equivalents specifically, but mentioning that the concepts are cloud-agnostic demonstrates maturity.
Interview tip: sketch the three-layer storage architecture
When storage comes up in system design, draw the three-layer diagram: block for the database tier, object for media and assets, file only if you have a specific shared-access requirement. This shows cost awareness and access-pattern thinking. Most candidates default to "put it in the database" or "put it in S3" without distinguishing between the two.
Real-World Examples
Netflix: Uses all three storage types. Block storage (NVMe instance storage) runs their micro-caching layer on Open Connect appliances (custom CDN servers). Each appliance has 36 SSDs with 100+ TB of popular content cached locally, delivering 90 Gbps per server. Object storage (S3) stores the master video library (petabytes of encoded video at multiple bitrates, ~15,000 titles in ~10 different resolutions each). EFS stores shared configuration files across thousands of EC2 instances running the control plane. Their Open Connect CDN delivers over 15% of global internet traffic during peak hours.
Dropbox: Migrated from S3 to their own object storage system (Magic Pocket) to reduce costs. At 500+ PB of user data, S3 costs were estimated at $75M+ per year. They built a custom object storage layer on raw block devices across multiple data centers, achieving 3x cost reduction. The system uses erasure coding (not replication) for durability, storing data across 6 data centers with the ability to survive 2 simultaneous data center failures. This is an extreme case: most companies should use S3 rather than build their own object store.
Airbnb: Stores listing photos and user content in S3 with CloudFront for global delivery. At 7M+ active listings with an average of 20 photos each (~140M images), S3 handles the scale at a fraction of what EBS would cost. Their search service runs on EC2 instances with EBS io2 volumes for Elasticsearch indices (100K+ IOPS for search workloads). Shared ML model artifacts (5-10 GB each) are stored on EFS so all inference servers access the same trained models simultaneously without the cold-start latency of downloading from S3 on each instance boot.
Stripe: Uses a combination of block and object storage for their payment processing infrastructure. Database-level operations (transaction recording, ledger updates) run on high-IOPS block storage for the sub-millisecond latency required for payment authorization. Audit logs, receipts, and compliance documents flow to S3 for long-term retention with Object Lock for regulatory compliance. Their approach demonstrates the principle that different data within the same application has fundamentally different storage requirements.
Pinterest: Manages petabytes of image data in S3 with a custom metadata service that tracks every pin's image across multiple resolutions and formats. Original uploads go to S3 Standard, generated thumbnails and responsive sizes are stored alongside them. Their CDN cache hit rate exceeds 95%, meaning S3 origin handles only a fraction of the actual image traffic. For their search and recommendation infrastructure, they use NVMe instance storage for high-performance Elasticsearch clusters where the data can be rebuilt from S3 if an instance fails.
Spotify: Uses GCS (Google Cloud Storage, GCP's equivalent of S3) for storing audio files, podcast episodes, and user-generated playlists. Their audio delivery pipeline serves 600+ million users across 180+ markets. Block storage (Persistent Disks) runs Bigtable and custom databases for user session state and real-time recommendation serving, where latency requirements demand local-disk-like performance. Their architecture reinforces the same pattern: object storage for static content at scale, block storage for low-latency stateful services.
How This Shows Up in Interviews
Storage type selection appears in nearly every system design interview, usually when the candidate says "we need to store images" or "we need a database."
What they're testing: Do you understand that different data has different access patterns, and that storage types exist to match those patterns? Can you reason about cost and performance trade-offs?
Depth expected at senior level:
- Know the three types and when each applies (block for DB, object for media, file for shared)
- Understand S3 storage classes and lifecycle policies for cost optimization
- Know EBS volume types and when to use gp3 vs io2 vs NVMe instance storage
- Explain pre-signed URLs for client-direct uploads to S3
- Articulate cost differences with actual numbers ($0.023 vs $0.08 vs $0.30 per GB/month)
- Understand S3's prefix-based throughput limits and how CloudFront mitigates them
- Know when EFS is justified vs when S3 with application-level loading is better
- Discuss durability differences: S3's 11 nines vs EBS's single-AZ replication
| Interviewer asks | Strong answer |
|---|---|
| "Where would you store user-uploaded images?" | "S3 with CloudFront in front. Pre-signed PUT URLs for direct client upload. Lifecycle policy to move to IA after 30 days, Glacier after 90. Never on EBS; that's 5x the cost and doesn't scale." |
| "What storage for the database?" | "EBS gp3 for most workloads (3,000 baseline IOPS included). If we need 10K+ IOPS consistently, provision io2 with the exact IOPS needed. Not S3; the latency is too high for random I/O." |
| "Multiple servers need to read the same config files." | "EFS with NFS mount. All instances mount the same filesystem, changes are visible immediately. For small config that changes rarely, S3 with a pull-on-startup pattern also works and is cheaper." |
| "How do you handle a 50 TB data lake?" | "S3 with Parquet files partitioned by date. Athena or Spark queries directly on S3. At $0.023/GB, 50 TB costs about $1,150/month. The same data on EBS would cost $4,000-$6,250/month." |
| "What about S3 eventual consistency?" | "S3 has been strongly consistent since December 2020. All operations have strong read-after-write consistency at no extra cost. This is a common misconception from outdated documentation." |
| "How would you handle EBS snapshot management?" | "Automated daily snapshots with AWS Backup. EBS snapshots are incremental (only changed blocks), so a 500 GB volume's daily snapshot costs a fraction of the volume. Keep 7 daily, 4 weekly, 12 monthly. Test restores quarterly. For cross-region DR, enable snapshot copy to a second region." |
| "What's the difference between throughput and IOPS?" | "IOPS measures operations per second (how many reads/writes). Throughput measures bytes per second (how fast data moves). A database doing many small 8 KB reads needs high IOPS. A video encoder reading large sequential files needs high throughput. gp3 gives both independently: 3,000 IOPS and 125 MB/s baseline." |
Gotcha: S3 is strongly consistent now
Many candidates still say "S3 is eventually consistent." This was true before December 2020. Since then, all S3 operations have strong read-after-write consistency at no extra cost. Saying otherwise in an interview reveals outdated knowledge.
Quick Recap
- Block storage provides raw byte-level access with sub-millisecond latency. Use it for databases, boot volumes, and any workload requiring low-latency random I/O. It's the most expensive per GB ($0.08-0.125/GB/month) and typically attached to a single instance.
- File storage provides a shared POSIX filesystem over the network (NFS/SMB). Use it only when multiple servers must read and write the same files simultaneously. It's niche and expensive ($0.30/GB/month on EFS).
- Object storage provides unlimited capacity via HTTP API at the lowest cost ($0.023/GB/month for S3 Standard). Use it for images, videos, backups, data lakes, and static assets. Objects are immutable (full replacement on update).
- Most real systems use all three: block for the database, object for media and assets, file for shared configuration. Drawing this three-layer architecture in interviews demonstrates cost awareness and access-pattern thinking.
- S3 lifecycle policies (Standard to IA to Glacier to Deep Archive) cut storage costs by 60-70% for data with declining access frequency. This is the single highest-impact cost optimization for most AWS bills.
- S3 is strongly consistent since December 2020. Pre-signed URLs enable direct client uploads without server involvement. CloudFront in front of S3 handles virtually unlimited read throughput.
Related Trade-offs
- Databases for how the database layer uses block storage underneath
- Caching for the in-memory layer that sits above all three storage types
- Scalability for how storage choices affect horizontal and vertical scaling
- CDN for how CloudFront and edge caching layer on top of object storage
- SQL vs. NoSQL for database selection trade-offs that determine block storage requirements