System Design Numbers & Patterns

Latency numbers (order of magnitude)

Operation	Time
L1 cache reference	1 ns
Branch mispredict	3 ns
L2 cache reference	4 ns
Mutex lock/unlock	17 ns
Main memory reference	100 ns
Compress 1 KB (Snappy)	2 us
Send 1 KB over 1 Gbps	10 us
Read 4 KB random SSD	16-100 us
Read 1 MB sequentially from memory	3-50 us
Round trip within a datacenter	0.5 ms
Redis/Memcached GET (same DC)	0.2-1 ms
Read 1 MB sequentially from SSD	50-200 us (about 1 ms in older tables)
HDD seek	3-10 ms
Read 1 MB sequentially from HDD	5-20 ms
Indexed DB query (warm)	1-10 ms
Round trip CA to Netherlands	150 ms

• Rules of thumb: memory is ~100x faster than SSD, SSD ~100x faster than a network hop across regions; avoid cross-region sync calls on the hot path.
• Sequential beats random; batch and compress before sending over WAN.

Capacity estimation

Quantity	Value
Seconds per day	86,400 (~1e5)
Seconds per month	~2.6e6
1M requests/day	~12 RPS
100M requests/day	~1,200 RPS
Peak factor	2-5x average
Scale	1 KB x 1M records = 1 GB; 1 GB x 1M = 1 PB
Powers	2^10 = 1 KB, 2^20 = 1 MB, 2^30 = 1 GB, 2^40 = 1 TB
Availability	99.9% = 8.8 h/yr (43 min/month); 99.99% = 52 min/yr; 99.999% = 5 min/yr

QPS            = DAU * actions_per_user_per_day / 86,400
peak QPS       = QPS * peak_factor (2-5)
storage/year   = writes_per_day * bytes_per_record * 365 * replication_factor
bandwidth      = QPS * avg_response_size
cache memory   = hot_fraction (20%) * daily_read_data
servers        = peak_QPS / QPS_per_server (add headroom ~30-40%)
DB connections = threads that need to hit the DB at once (not total clients)
Little's law   = concurrency = throughput * latency

• Typical single-node: relational DB 1-10k simple QPS (more with replicas), Redis ~100k ops/s, Kafka partition ~10-50 MB/s, app server 1-5k RPS for light handlers.
• Always state assumptions; round aggressively (1 day = 1e5 s).

CAP and consistency

• CAP: under a network partition you choose Consistency (reject/err) or Availability (serve possibly stale). Without partitions you trade latency vs consistency (PACELC).
• CP examples: ZooKeeper, etcd, HBase, MongoDB (default majority writes), Spanner. AP examples: Cassandra (tunable), DynamoDB (eventually consistent reads default), Riak.

Model	Guarantee
Strong / linearizable	every read sees the latest completed write; single logical copy
Sequential	all see the same order, not necessarily real time
Causal	causally related writes seen in order
Read-your-writes / monotonic reads	session-level guarantees
Eventual	replicas converge if writes stop

• Quorum: N replicas, W write acks, R read acks; R + W > N gives overlap (strong-ish reads). Typical N=3, W=2, R=2.
• Conflict resolution: last-write-wins (clock skew risk), vector clocks, CRDTs, app merge.
• Consensus: Raft/Paxos (leader election, log replication; majority quorum; tolerates f failures with 2f+1 nodes).

Caching patterns

Pattern	Flow	Trade-off
Cache-aside (lazy)	app reads cache; on miss reads DB and fills cache	most common; stale until TTL/invalidation; first request slow
Read-through	cache loads from DB itself	simpler app; provider support needed
Write-through	write to cache and DB synchronously	consistent, slower writes
Write-behind (back)	write to cache, flush to DB async	fast writes, loss risk on cache failure
Write-around	write to DB only, cache fills on read	avoids cache pollution; first-read miss
Refresh-ahead	refresh popular keys before expiry	complexity

• Invalidation: TTL (+ jitter to avoid synchronized expiry), delete on write (prefer delete over update to reduce races), event-driven (CDC).
• Cache stampede (thundering herd): single-flight/lock per key, probabilistic early expiry, stale-while-revalidate.
• Penetration (non-existent keys): cache negative results, Bloom filter. Avalanche: many keys expiring at once; add TTL jitter, replicas.
• Eviction: LRU, LFU, TTL, FIFO. Redis maxmemory-policy allkeys-lru.
• Layers: browser, CDN, API gateway, in-process (Caffeine), distributed (Redis). Spring: @Cacheable, @CacheEvict, @CachePut.
• Cache invalidation hard parts: cache and DB can disagree; read-modify-write races; hot keys (replicate or local-cache).

Load balancing

Aspect	Options
Layer	L4 (TCP/UDP, fast, no HTTP awareness; NLB, HAProxy tcp) vs L7 (HTTP routing, TLS termination, headers; ALB, Nginx, Envoy)
Algorithm	round robin, weighted RR, least connections, least response time, IP hash, consistent hashing (cache/shard affinity), power of two choices
Health	active checks (probes) + passive (error rates), outlier ejection
Stickiness	cookie/IP affinity; prefer stateless services + shared session store (Redis/JWT)
Global	DNS (GeoDNS), anycast, GSLB, CDN

• Consistent hashing: adding/removing a node moves ~1/N keys; virtual nodes balance load.
• Keep LB redundant (active-passive with VIP or active-active).
• Timeouts, retries (with budget and idempotency) and connection draining on deploy.

Database scaling

Technique	Use
Indexing, query tuning, pooling	first step; cheap
Vertical scaling	simple; ceiling and single point of failure
Read replicas	scale reads; replication lag means stale reads; route read-your-writes to primary
Partitioning (single DB)	split large tables by range/hash/list for pruning and maintenance
Sharding	horizontal split by shard key across nodes; cross-shard joins/transactions are costly
Denormalization / materialized views	trade writes and storage for read speed
CQRS	separate write model and read model(s)
Archival / TTL	keep hot data small

• Shard key: high cardinality, even distribution, matches access pattern; avoid hot shards (monotonic ids, celebrity keys). Resharding: consistent hashing or directory service.
• Replication: sync (durable, slower), semi-sync, async (fast, can lose data on failover).
• SQL vs NoSQL: relational for transactions/joins; key-value (DynamoDB/Redis) for simple access at scale; document (MongoDB) for flexible schema; wide-column (Cassandra) for write-heavy time series; search (Elasticsearch) for text; graph for relationships.
• ID generation: DB sequence, UUIDv4/v7, Snowflake (timestamp + machine + sequence, time-sortable).

Messaging semantics

Semantics	Meaning	How
At-most-once	may lose, never duplicate	ack before processing / no retry
At-least-once	never lose, may duplicate	ack after processing + retry; make consumers idempotent
Exactly-once	effect applied once	at-least-once + idempotency key / dedupe table; Kafka idempotent producer + transactions (within Kafka)

Broker	Model	Notes
Kafka	partitioned log, consumer groups, offsets, retention	ordering per partition only; parallelism = partitions; replay
RabbitMQ	exchanges to queues, ack/nack, routing	per-message routing, TTL, DLX; messages removed once acked
SQS	managed queue, visibility timeout	standard (at-least-once, unordered) vs FIFO
Pub/Sub, SNS	fan-out topics	combine with queues per subscriber

• Choose a partition key (e.g. orderId) to keep related events ordered.
• Dead-letter queue for poison messages; retry with exponential backoff.
• Transactional outbox: write the event to an outbox table in the same DB transaction, a relay (or CDC/Debezium) publishes it; solves dual-write.
• Saga: sequence of local transactions with compensating actions (choreography via events or orchestration); instead of 2PC.
• Backpressure: bounded queues, consumer lag alerts, rate limiting.

Resilience patterns

Pattern	What it does	Spring/Java tool
Timeout	bound every remote call; never infinite	client config, TimeLimiter
Retry + backoff + jitter	handle transient errors; only for idempotent calls	Resilience4j @Retry, Spring Retry
Circuit breaker	closed -> open (fail fast) -> half-open probe	Resilience4j @CircuitBreaker
Bulkhead	isolate resources per dependency (thread/semaphore pools)	Resilience4j @Bulkhead
Rate limiting	protect from overload: token bucket, leaky bucket, sliding window	Bucket4j, Resilience4j, gateway
Fallback / degrade	cached or default response	fallbackMethod
Idempotency key	safe retries of POSTs	unique request id stored with result
Load shedding	drop low-priority work under overload	gateway, queue limits
Health checks + graceful shutdown	route away from bad nodes, drain in-flight	probes, server.shutdown=graceful

• Retry storms: cap attempts, use retry budget, add jitter; retry only at one layer.
• Redundancy: multi-AZ, replicas, no single points of failure; RTO/RPO define recovery targets.
• Observability: logs, metrics (RED: rate, errors, duration; USE: utilization, saturation, errors), traces, SLO/error budgets.

Four-step answer framework

1. Requirements and scope (3-5 min): functional use cases (what is in/out), non-functional (scale, latency, availability, consistency, durability), read/write ratio. Do the estimation: QPS, storage, bandwidth.
2. High-level design (10 min): API (REST/gRPC, key endpoints), data model and store choice, boxes and arrows (client, LB, services, cache, DB, queue, CDN), one end-to-end flow for read and write.
3. Deep dive (15-20 min): the 1-2 hardest parts: schema and indexes, sharding key, cache strategy, consistency, ID generation, fan-out, ranking, idempotency, concurrency (locking, optimistic versions).
4. Bottlenecks and wrap-up (5 min): single points of failure, hot keys, failure modes, monitoring and alerting, security, cost, evolution; state trade-offs and what you would do next.

• Say trade-offs out loud ("X gives A at the cost of B, I choose X because the requirement is A").
• Start simple (monolith + DB + cache), scale only where the numbers demand it.