4.2 — System Design for Non-Engineers

Scaling Mental Model

Systems don’t stay small. Understanding the concerns at each scale level lets you design ahead of the wall instead of hitting it.

ONE USER              MANY USERS             MASSIVE SCALE
─────────────────────────────────────────────────────────
Everything on         Need to think about:    Need to think about:
one computer.         - Multiple servers       - Load balancing
Simple.               - Database scaling       - Caching
                      - API rate limits        - CDNs
                      - Cost management        - Redundancy
                                               - Monitoring

Key Concepts

Term	Definition	Analogy
Scalability	A system’s ability to handle growth — more users, more data, more requests	A restaurant that can open more tables vs. one that’s always full
Load balancer	Distributes incoming requests across multiple servers so no single server is overwhelmed	A hostess seating guests across multiple dining rooms
Cache	Storing frequently accessed data in fast, temporary storage so you don’t recalculate or re-fetch it every time	Keeping frequently used files on your desk instead of walking to the filing cabinet
CDN (Content Delivery Network)	Servers distributed worldwide that store copies of your content close to users	Instead of one warehouse, having distribution centers in every major city
Rate limit	Maximum number of requests allowed in a time period	An API that allows 100 calls per minute — hit the limit and you wait
Webhook	A URL that receives automatic notifications when something happens in another system	A doorbell — when someone arrives, it notifies you automatically
Queue	A list of tasks waiting to be processed, handled in order	A line at the DMV — first in, first out
Middleware	Software that sits between two systems and processes requests as they pass through	A security checkpoint between you and the building — checks your ID, maybe modifies your badge, then lets you through
Container	A packaged unit of software that includes everything needed to run — code, runtime, dependencies. Runs the same everywhere.	A shipping container — standardized box that works on any ship, truck, or crane
Docker	The most common tool for creating and running containers	The company that standardized shipping containers
CI/CD	Automated pipeline that tests and deploys code when you push changes	An assembly line that automatically inspects and ships every product

These concepts connect directly to the Infrastructure Layer of the full stack. When you collaborate with engineers, these are the terms they use to talk about why something is slow, expensive, or unreliable.

Next: 4.3 — Memory Architecture | Phase overview: Phase 4