#🧱 Building Block: Amazon S3 (Object Storage)

#Overview

Amazon S3 (Simple Storage Service) is a scalable object storage service that provides 11 nines (99.999999999%) of durability. It is the de facto standard for storing unstructured data — files, images, backups, and data lake content.

#Key Concepts

• Objects: Data stored as key-value pairs (key = path, value = blob up to 5TB).
• Buckets: Top-level containers that hold objects; globally unique names.
• Storage Classes: Standard, Infrequent Access (IA), Glacier, Intelligent-Tiering — trade access speed for cost.
• Pre-Signed URLs: Temporary, authenticated URLs for secure direct uploads/downloads without proxying through your server.
• Lifecycle Policies: Automatically transition objects between storage classes or delete after a period.
• Versioning: Keep multiple versions of an object for rollback and audit trails.
• Multipart Upload: Upload large files in parallel chunks for reliability and speed.

#When to Use

• Media/Blob Storage: Images, videos, audio, documents (any file type).
• Data Lake / Analytics: Raw data storage for batch processing (Spark, Athena queries).
• Backups & Archives: Database dumps, log archives with Glacier for low-cost cold storage.
• Static Website Hosting: Serve HTML/CSS/JS directly from S3 + CloudFront CDN.
• Immutable Content: Paste content, user-generated uploads that don't change after creation.

#When NOT to Use

• Frequently updated structured data (use a database).
• Low-latency random access (use Redis or a database).
• File system semantics needed (use EFS/NFS).

#Trade-offs

• Pros: Virtually unlimited scale, 11 nines durability, low cost ($0.023/GB/month), excellent CDN integration, no capacity planning.
• Cons: Higher latency than in-memory stores (~50-100ms), eventual consistency for overwrites (though read-after-write consistent for new PUTs), egress costs add up at scale.

#Used In This Repo

• E-Commerce (#11) — Product image storage (1.25PB)
• Pastebin (#25) — Paste content blob storage (18TB/year)
• Web Crawler (#12) — Raw web page archive (6PB over 5 years)