Stateless vs Stateful

SYSTEM DESIGN SERIES  ·  TOPIC 05 OF 23+  ·  part of the 04 → 05 → 06 chain
1. Stateless - No Memory Between Requests
Each request carries everything the server needs to handle it
  • Server holds no memory of you between requests
  • Every request carries all context it needs (token, ID, payload)
  • Any instance can handle any request - that's the whole point
  • Kill a node mid-traffic? Nothing lost, requests just reroute
  • Example: a REST API reading/writing a database per request
2. Stateful - Memory That Matters
Some server holds context a specific client depends on
  • Server holds data in memory or on local disk between requests
  • That node - not "a" node - must serve this client next
  • Examples: WebSocket / game servers, local session cache, the DB itself
  • Losing that node can mean losing that state entirely
  • Needs sticky routing, replication, or persistence to survive
3. Side by Side
Stateless
  • Horizontal scaling is trivial
  • Node failure = a non-event, just retry elsewhere
  • Simpler ops, but a round trip to fetch state each time
VS
Stateful
  • Fast local access - no round trip needed
  • Node failure can mean real data loss
  • Needs sticky routing or replication to survive
🏨 Analogy: Stateless is a hotel chain - any location, any night, same experience, because your booking lives in a central system. Stateful is renting a room and leaving your things in it - you must go back to that exact room.
4. Where Did the State Go?
"Stateless" doesn't mean the state vanished
Sessions → a shared store (Redis, DB) instead of local memory. Files → object storage (S3) instead of local disk. The state still exists - it just isn't stuck to one app server anymore. You've relocated the hard problem, not deleted it.
5. The Sticky Session Trap
A common workaround that quietly reintroduces the problem
  • Load balancer pins a client to one server (cookie / IP hash)
  • Looks stateless in the code, behaves stateful in practice
  • Uneven load - "busy" clients overload their pinned server
  • Lose that server and their session is still gone
📌 Sticky sessions are a workaround, not a fix - they reintroduce the exact fragility statelessness was meant to remove.
6. Examples in the Wild
REST API Server
Stateless
CDN / Static Edge
Stateless
Database
Stateful
WebSocket / Game Server
Stateful
Local In-Memory Cache
Stateful (the sneaky one)
Push state to the edges keep the middle tier stateless
The "sneaky" one matters most. A local in-memory cache feels harmless until you scale to 5 instances and each one has a different, silently stale answer.
7. Designing For It
Keep stateless when:
✔ It's your API / web tier
✔ You want simple horizontal scaling
✔ Nodes should be disposable, replaceable
✔ One instance dying should be a non-event
State Has to Live Somewhere - Choose Where, Deliberately
Accept stateful when:
✔ It's the datastore itself - something must hold the truth
✔ Local access beats easy scaling (e.g. game servers)
✔ You've deliberately added replication to survive node loss
✔ The live connection itself is the feature (WebSockets)
💡 Stateless services are exactly what makes Load Balancers (topic 06) simple - any node can take any request. Stateful ones need something smarter, like Consistent Hashing (topic 15).