System Design: Concepts & Patterns

A COMPLETE, STANDALONE GUIDE  ·  THE FULL FIELD, ONE PAGE, TWELVE PARTS
System design is really just one repeated exercise: you're given a problem that's too big for one machine, one team, or one moment in time to handle safely, and you have to decide - deliberately, out loud, defensibly - which trade-offs you're making to solve it anyway. Every part below is a different lens on that same exercise: how to estimate the problem's real size, how to spread load without losing correctness, how to survive failure instead of pretending it won't happen, and how to talk about all of it like someone who's actually built these systems, not just read about them.
Part 1 of 12

Fundamentals of System Design

Before any diagram gets drawn, system design is a discipline of naming your constraints honestly and picking trade-offs on purpose.
1.1 What Is System Design, and What Are Its Goals?
Not "draw boxes and arrows" - deciding how a system behaves under real, imperfect conditions
  • System design is the practice of structuring software so it meets its requirements at the scale, reliability, and cost it actually needs to - not just "does it work on my machine"
  • The goals are almost always the same short list: it should scale as load grows, stay available when parts fail, remain maintainable as people and code change, and do all that within a cost someone's actually willing to pay
  • Those goals conflict with each other constantly - which is the entire reason this is a skill, not a checklist
1.2 Non-Functional Requirements & Trade-offs
What the system must be, not what it must do
Scalability
Handles more load by adding resources, not by rewriting everything.
Availability
Stays reachable through failure - measured in "nines," not good intentions.
Consistency
Every reader sees the same answer - or agrees, on purpose, not to for a while.
Latency
How long a user actually waits - often the metric that decides everything else.
⚠️ Common Misconception: "We should maximize all of these." You can't - every trade-off in this entire guide is one of these NFRs being deliberately favored over another. The functional requirements say what the system does; these decide how well it survives doing it.
1.3 Back-of-the-Envelope Estimation
Rough numbers, done fast, so a design is grounded before it's drawn
📐 Worked example: sizing a URL shortener
Say 100 million new short URLs are created a month, and reads outnumber writes 100:1. Writes/sec ≈ 100,000,000 / (30 × 86,400) ≈ ~40/sec - trivial. Reads/sec ≈ 40 × 100 = ~4,000/sec - the number that actually decides whether you need caching and read replicas. Storage: 100M rows/month × 500 bytes/row ≈ 50GB/month, or ~600GB/year - small enough that storage was never the constraint; read throughput was.
🌍 Why this matters: the numbers above tell you, before a single box is drawn, that this system lives or dies on caching reads - not on database write capacity, replication strategy, or storage cost. Estimation isn't decoration; it's what tells you which part of the design actually deserves your attention.
1.4 Design Process & Key Principles
A reliable order to think in - not a script to recite
Clarify & Scope
Estimate Scale
High-Level Design
Deep Dive
Name Trade-offs
🎯 Interview Perspective: nobody expects the "correct" architecture from memory - they're watching whether you can move through these five steps out loud, and whether you know which trade-off you just made and why, at every single one.
Part 2 of 12

Scalability & Performance Basics

The core toolkit for handling more without everything getting slower or falling over.
2.1 Vertical vs Horizontal Scaling
Make one machine bigger, or use more of them
Vertical (scale up)
  • More CPU, RAM, faster disk - same machine
  • Simple - no distributed-systems problems appear
  • Has a hard ceiling, and that machine is a single point of failure
VS
Horizontal (scale out)
  • More machines, splitting the load between them
  • No real ceiling - and no single point of failure
  • Now you need coordination: load balancing, replication, all of Part 6
2.2 Load Balancing
Pick a healthy server, and stop picking the ones that stopped being healthy (looping)
A load balancer's whole job in two parts: distribute requests across identical servers (round robin, least connections, weighted), and continuously health-check them so a dead server gets pulled out of rotation automatically - before it ever costs a user a failed request.
2.3 Caching - Client, Server, and CDN
The first request pays full price. Every one after that shouldn't (looping)
Client Cache
Lives in the browser - zero network trip at all when it hits.
Server / App Cache
Redis or similar, shared across app instances - saves the database.
CDN / Edge Cache
A copy sitting geographically close to the user, for content that barely changes.
2.4 Database Indexing & Performance Metrics
The difference between reading one page and reading every page
Indexing, in one line: a sorted side-structure that turns "scan every row" into "jump straight to it" - at the cost of extra storage and slightly slower writes, since the index has to update too.
The three numbers that matter: QPS (queries handled per second), latency (how long one request takes - watch p99, not average), throughput (total useful work completed per second). Optimizing one can quietly hurt another.
Part 3 of 12

Databases Deep Dive

Where most of a system's hardest, least-reversible decisions actually live.
3.1 SQL vs NoSQL, ACID vs BASE
Two related but separate choices - schema shape, and consistency philosophy
SQL
  • Fixed schema, relationships enforced by the database itself
  • Strong consistency guarantees by default (ACID)
  • Harder to horizontally scale writes
VS
NoSQL
  • Flexible or schema-less - the application enforces structure instead
  • Usually favors availability and partition tolerance over strict consistency (BASE)
  • Built from the ground up to scale out horizontally
🧾 ACID (Atomicity, Consistency, Isolation, Durability) promises a transaction happens completely or not at all. BASE (Basically Available, Soft state, Eventual consistency) trades that guarantee for scale - the data will become consistent, just not necessarily this instant.
3.2 Replication, Sharding & Consistent Hashing
Copy data for safety, split it for scale, and route to it without a giant lookup table (looping)
  • Replication: the same data, copied to multiple machines - a replica dying doesn't lose data, and reads can be spread across copies
  • Sharding: different data lives on different machines, split by some key - no one machine holds it all
  • Consistent hashing: the trick that lets you add or remove a shard without reshuffling nearly everything else - each key maps to a point on a ring, not a fixed machine index
3.3 CAP Theorem & PACELC
When the network splits, you have exactly one honest choice left to make (looping)
  • CAP: during a network Partition, you must choose between Consistency (refuse the request rather than risk a wrong answer) and Availability (answer anyway, possibly with stale data) - you cannot have both
  • PACELC extends it: even with no partition happening, there's still a trade-off between Latency and Consistency - waiting for every replica to agree costs time, always
  • Denormalization: deliberately duplicating data to avoid a costly join - trading storage and write complexity for read speed, the same trade-off spirit as caching
Part 4 of 12

Caching Strategies

Caching basics from Part 2, made precise: exactly where a write goes, and exactly how a stale copy gets fixed.
4.1 Cache Patterns - Aside, Through, Write
Three different answers to "where does a write actually go, in what order?" (looping)
Cache-Aside
App checks cache, misses, reads DB itself, then fills the cache. Most common, most explicit.
Write-Through
Every write updates cache and DB together. Safer, slightly slower per write.
Write-Back
Write hits the cache first, DB catches up later. Fast, risky if the cache dies first.
4.2 Invalidation, Redis, Cache vs DB, and CDN Caching
Keeping a copy honest is the actual hard part
  • Invalidation strategies: let it expire (TTL), delete/update it the instant the source changes (write-invalidate), or broadcast a "this changed" event to every cache holding a copy (event-driven)
  • Redis - an in-memory store that's usually the actual cache: sub-millisecond reads, rich data structures (not just key-value), optional persistence so a restart isn't a full cold cache
  • Cache vs DB: a cache is fast and disposable - losing it should degrade performance, never lose data. If losing the cache would lose real data, it isn't a cache, it's an undocumented database
  • CDN caching: driven by HTTP headers (Cache-Control, ETag) set by the origin - the CDN itself doesn't know or decide what's cacheable, your server does
⚠️ Common Misconception: "More caching is always better." A cache that's rarely re-read has all the invalidation cost and none of the benefit - caching is a trade you make deliberately, not a default you apply everywhere.
Part 5 of 12

Networking & Communication

Every system design conversation eventually has to admit: the network in between is slower, less reliable, and more expensive than people assume.
5.1 The OSI Model & TCP/IP, Briefly
Seven conceptual layers; in practice, you mostly live at two of them
  • The OSI model splits networking into layers - Physical, Data Link, Network, Transport, Session, Presentation, Application - each only responsible for its own job
  • TCP (Transport layer) gives you a reliable, ordered stream - it handshakes, retransmits lost packets, and guarantees order, at the cost of setup overhead
  • IP (Network layer) is just addressing and routing - get this packet from A to B, no guarantee it arrives, or arrives once
  • HTTP/HTTPS (Application layer) is what most system design conversations actually operate at - everything below it is usually "handled," until it isn't
5.2 HTTP/HTTPS, DNS, and Proxies
Forward proxy hides the client. Reverse proxy hides the server (looping)
  • HTTPS is just HTTP over TLS - the same request/response model, encrypted in transit
  • DNS, in one line: translates a name into an IP through a cached, hierarchical lookup (root → TLD → authoritative) - the full mechanics are worth their own deep dive, since DNS problems are almost always caching problems in disguise
  • Forward proxy: sits in front of clients, hiding who's asking - corporate networks, VPNs
  • Reverse proxy: sits in front of servers, hiding how many/which ones exist - this is what a load balancer usually actually is
5.3 Keep-Alive, Timeouts, Retries & API Gateways
The unglamorous settings that decide whether a slow dependency becomes an outage
  • Keep-Alive: reuse one TCP connection for multiple requests instead of a fresh handshake every time - a real, easy latency win
  • Timeouts: a call that never gives up is a call that can hang a thread, a connection pool, and eventually the whole service - always bound how long you'll wait
  • Retries: genuinely helpful for transient failures, genuinely dangerous without a limit and backoff (Part 8) - a naive retry storm can be what actually takes a struggling service down
  • API Gateway: a single front door doing auth, rate limiting, and routing to the right service - so no individual service has to reimplement any of it
Part 6 of 12

Distributed Systems Concepts

Once a system is more than one machine, "agreement" stops being free - these are the mechanics of getting it anyway.
6.1 Consistency Models & Idempotency
What "correct" means when copies of the same data can disagree
  • Strong consistency: every read sees the latest write, everywhere, immediately - expensive, and directly in tension with availability (Part 3's CAP)
  • Eventual consistency: reads might briefly return stale data, but every replica converges to the same value if writes stop - cheaper, and enough for most things that aren't a bank balance
  • Idempotency: an operation that produces the same result no matter how many times it's retried. "Set balance to $100" is idempotent; "add $50" is not - and retries (Part 8) are only safe on operations that are
6.2 Leader Election & Clock Synchronization
Someone has to be in charge, and everyone has to agree on "now" - neither is automatic (looping)
A leader dies, the remaining nodes notice missed heartbeats, one becomes a candidate and requests votes - a majority makes it the new leader. Clock synchronization matters because machine clocks genuinely drift apart over time; protocols like NTP correct for it, but distributed systems that need strict ordering often avoid trusting wall-clock time at all, using logical clocks instead.
6.3 Quorum & Gossip Protocols
Agreement without asking everyone, and information that spreads like a rumor (looping)
  • Quorum: a majority of nodes (say 3 of 5) is enough to confirm a decision - any two majorities are mathematically guaranteed to overlap by at least one node, which is what makes it safe
  • Gossip protocol: instead of one broadcaster telling everyone, each node periodically tells a few random peers what it knows - information spreads exponentially, with no single point that has to reach everyone directly
Part 7 of 12

Messaging & Asynchronous Systems

Two services that shouldn't have to wait on each other, and a system that reacts to what happened instead of being told what to do.
7.1 Message Queues & Publish-Subscribe
A queue shares work. Pub/Sub broadcasts a fact (looping)
  • Message queue (RabbitMQ-style): each message goes to exactly one consumer - multiple consumers share the workload, they don't each get a copy
  • Kafka-style systems act more like a durable, ordered log - consumers track their own read position and can replay history, rather than the broker deleting a message once it's read
  • Pub/Sub: every subscriber gets its own full copy of the same message - a publisher never even knows how many are listening
  • Event-driven architecture is what a whole system looks like once services talk this way by default: react to OrderPlaced, not a direct call demanding ChargeCard
7.2 Stream Processing & Delivery Semantics
Same lost acknowledgement, three different outcomes (looping)
At-Most-Once
Fire and forget. Fast, but a drop means the message is just gone.
At-Least-Once
Retry until acknowledged. Nothing's lost, but a late ack can create a duplicate.
Exactly-Once
The goal - usually built as at-least-once delivery plus idempotent processing that can't be fooled twice.
🌊 Stream processing basics: once events never stop arriving, you process each one as it comes rather than waiting for a full batch - often aggregating over a bounded window of time, since "the total, forever" isn't a useful number for a stream with no end.
Part 8 of 12

Reliability & Fault Tolerance

Failure isn't the exception you design around - it's a certainty you design for. This is the toolkit for surviving it gracefully.
8.1 Redundancy, Failover & Exponential Backoff
A retry that fires instantly, over and over, isn't resilience - it's a self-inflicted second outage (looping)
  • Redundancy: more than one copy of anything load-bearing - a single instance of anything is a single point of failure
  • Failover: when the primary dies, a standby takes over automatically - the whole reason redundancy has to be paired with detection, not just extra copies sitting idle
  • Exponential backoff: each retry waits longer than the last (1s, 2s, 4s, 8s...) - giving a struggling service room to recover instead of a thundering herd of instant retries making it worse
  • Jitter: add randomness to each wait time, so a thousand clients that failed at the same instant don't all retry at the same instant too
8.2 Circuit Breaker Pattern
Stop calling a dependency that's already drowning - for its sake and yours (looping)
Three states, exactly like a household circuit breaker: Closed (normal, requests flow through), Open (too many recent failures - stop calling immediately, fail fast instead of waiting on a timeout), Half-Open (after a cooldown, let one test request through - succeed and go back to Closed, fail and go back to Open). Without this, one slow dependency can back up every caller's threads waiting on it.
🎯 Interview Perspective: "Why not just use a short timeout instead of a circuit breaker?" A timeout still pays the cost of attempting every single call before failing. A circuit breaker in the Open state fails immediately, with zero cost, for every call until the cooldown ends - the two solve different parts of the same problem.
8.3 Bulkhead Pattern, Graceful Degradation & Monitoring
One compartment flooding shouldn't sink the whole ship (looping)
  • Bulkhead: isolate resources (thread pools, connections) per dependency, so one slow/overloaded dependency can't exhaust the resources every other call needs too
  • Graceful degradation: when a non-critical piece fails, serve a reduced experience instead of a total failure - show the page without recommendations rather than no page at all
  • Monitoring & alerting: none of the above helps if nobody's watching the signals that would tell you it's happening - alert on user-facing symptoms (error rate, latency), not just internal causes (CPU), to avoid alert fatigue
Part 9 of 12

Security in System Design

Every mechanism above assumes good-faith traffic. This part is what happens when it isn't.
9.1 Authentication vs Authorization, OAuth & JWT
"Who are you" and "what can you touch" are two separate checks, every single time (looping)
  • Authentication: proves identity - logging in. Authorization: governs permission - can this identified user touch this specific resource, checked on every request, not just once
  • OAuth: lets a user grant a third-party app limited access without ever handing over their password
  • JWT: a signed, self-contained token - the signature proves it wasn't tampered with, but the payload is only base64-encoded, not encrypted, so it's never actually secret
9.2 Data Encryption, Rate Limiting & Secure Design
Protecting data in flight and at rest, and protecting the system itself from being overwhelmed or tricked
  • Encryption in transit (TLS/HTTPS) protects data crossing the network. Encryption at rest protects a stolen disk or leaked backup - neither substitutes for actual access control
  • Rate limiting & throttling: a hard ceiling (reject with 429) versus deliberately slowing requests down - both exist to protect finite resources from abuse or accidental overload
  • Secure design best practices, compressed: least privilege, defense in depth, validate everything server-side, never trust the client, and assume any single control will eventually fail
⚠️ Common Misconception: "We hide the button in the UI, so it's secure." Hiding a control isn't authorization - if the API behind it doesn't independently check the same permission, anyone with dev tools can call it directly.
Part 10 of 12

Design Patterns & Best Practices

The shapes systems and code keep taking, over and over, because they solve real recurring problems.
10.1 Microservices vs Monolith, SOA, Serverless & Monorepo
Where the code that makes up your system actually lives, and how it's deployed
Monolith
  • One deployable unit - simple to develop and deploy early on
  • Scaling means scaling the whole thing, even for one hot piece
VS
Microservices
  • Independent services, each deployable and scalable on its own
  • Real operational cost: network calls, distributed debugging, all of Part 6
SOA
Microservices' predecessor - coarser-grained services, often sharing a central integration bus.
Serverless
You ship functions; the platform handles scaling and servers entirely. Great for spiky, event-driven work.
Monorepo
Many services or apps, one shared repository - easier cross-cutting changes, at the cost of tooling complexity at scale.
10.2 Classic Design Patterns
Four shapes that show up constantly, at every level from a single class to a whole architecture
Proxy
Adapter
Factory
Observer
  • Proxy: stands in for a real object, controlling access to it - a reverse proxy (Part 5) is this exact pattern at the network level
  • Adapter: translates one interface into another a caller already expects - how you integrate something that wasn't built for your system
  • Factory: centralizes the logic for creating an object, so callers don't need to know which concrete type they're getting
  • Observer: subscribers register interest and get notified on a change - the same shape as Pub/Sub (Part 7), just at the code level instead of the network level
10.3 API Design Best Practices & Versioning
The contract other people's code depends on - change it carelessly, and you break them silently
  • Consistent naming, predictable status codes, and pagination on anything that returns a list - the basics that make an API usable without reading source code
  • Versioning: a breaking change gets a new version (/v2/), not a silent mutation of /v1/ - existing consumers should never wake up to a different contract
  • Schema versioning matters just as much internally - a message format or database schema change has to stay compatible with whatever's still reading the old shape, especially mid-deploy (topic 22 of the main series)
Part 11 of 12

System Design Case Studies

Every part above is a tool. This is how the tools actually get picked up and used on a real problem.
11.1 High-Level Design vs Low-Level Design
Two different zoom levels, and two different audiences
HLD - High-Level Design
  • The boxes-and-arrows view: services, databases, queues, how they connect
  • Answers "what are the major pieces, and why"
VS
LLD - Low-Level Design
  • Class structures, function signatures, database schemas, API contracts
  • Answers "how does this one piece actually get built"
🗺️ An HLD is a city map - which neighborhoods exist and how traffic moves between them. An LLD is the floor plan of one specific building in that city.
11.2 The Step-by-Step Approach to Any System
Analyzing a real system and reviewing an existing design use the exact same muscle
  • Designing new: clarify requirements → estimate scale (Part 1) → sketch the HLD → deep-dive the riskiest piece → name the trade-offs out loud
  • Analyzing an existing real system: work backwards - what constraint likely forced this specific choice? A CDN in front usually means read-heavy, geographically spread traffic; a queue in the middle usually means a producer that can't afford to wait on a slow consumer
  • Reviewing someone else's design: find the single point of failure first, then ask what happens at 10x the stated scale - most real weaknesses hide in exactly those two questions
🎯 Interview Perspective: reviewing a design out loud - "here's what I'd question, here's what breaks at scale" - demonstrates the exact same judgment as designing one from scratch, and it's a skill worth practicing on real systems you didn't build.
Part 12 of 12

Practice, Iterate & Level Up

None of the previous eleven parts stick from reading them once. This part is the only one that's actually about you, not the systems.
12.1 Habits That Actually Compound
This is advice, not a mechanism - no diagram substitutes for actually doing it
✔ Solve one system design problem regularly - consistency beats intensity
✔ Discuss and review designs with other people, not just yourself
✔ Study real-world architectures deliberately - most public postmortems and engineering blogs are free case studies
✔ Stay current - the trade-offs in this guide shift as new tools change what's cheap and what's expensive
✔ Teach someone else - explaining a trade-off out loud is the fastest way to find the part you only thought you understood
✔ Build something real - a toy project that hits actual scale problems teaches more than another diagram ever will
🏁 Twelve parts, one underlying skill: naming a constraint honestly, and picking a trade-off on purpose. Everything in this guide is just that skill, applied to a different corner of the problem.