Fundamentals & Estimation

Separate functional requirements (what the system does) from non-functional ones (scale, latency, availability, consistency, durability), then fix scope and numbers before drawing boxes.

• Functional: core use cases only (e.g. "post a tweet, read timeline"), explicitly list what is out of scope.
• Non-functional: DAU, read/write ratio, latency target (p99), availability target, consistency need, data retention, geography.
• Then: estimate, API, data model, high-level design, deep dive on bottlenecks.

Convert users into requests per second, multiply by object size for bandwidth and by retention for storage, and keep powers of ten.

• 1 day = ~86,400 s (round to 100k). 1M requests/day = ~12 QPS; 100M/day = ~1,200 QPS.
• Peak = 2-5x average.
• Example: 10M DAU, 5 writes/day, 1 KB each: 50M writes/day = ~580 QPS avg, 50 GB/day, ~18 TB/year (before replication x3).
• Bandwidth = QPS x payload size (1,000 QPS x 100 KB = 100 MB/s = 800 Mbps).

Memory is nanoseconds, SSD microseconds, network round trips milliseconds; the ratios drive design decisions.

• Implications: avoid chatty cross-region calls, batch network I/O, cache hot data in memory, prefer sequential over random I/O.

Availability = uptime / total time. Components in series multiply (worse); redundant components in parallel combine as 1 - (1-a)^n (better).

• Serial: three 99.9% services in a chain = 0.999^3 = 99.7%.
• Parallel: two independent 99% replicas = 1 - 0.01^2 = 99.99%.
• Parallel math assumes independent failures; shared dependencies (same AZ, same DB, same deploy) break it.

An SLI is a measured indicator, an SLO is the internal target for it, an SLA is a contractual promise with consequences; the error budget is 1 - SLO.

• SLI example: proportion of requests with status < 500 and latency < 300 ms.
• SLO example: 99.9% over a rolling 30 days. Budget: 0.1% = ~43 minutes of badness per 30 days.
• SLA is looser than the SLO (e.g. 99.5%) so you breach internally before breaching contractually; breach triggers credits.
• Budget burn policy: when exhausted, freeze risky releases and prioritize reliability work.

Averages hide slow outliers; a user request often fans out to many backends, so the probability of hitting at least one slow call grows quickly.

• If a single call exceeds p99 with probability 1%, a request fanning out to 100 calls is slow with probability 1 - 0.99^100 = ~63%.
• Mitigations: hedged requests, timeouts and budgets, load shedding, smaller fan-out, caching, avoiding GC and queue buildup.
• Percentiles cannot be averaged across hosts; aggregate histograms instead.

During a network partition a distributed system must choose between consistency (linearizability) and availability (every non-failing node answers); partitions are not optional, so the real choice is CP vs AP only while partitioned.

• CP: refuse or block requests on the minority side (etcd, ZooKeeper, HBase, Spanner-like).
• AP: serve possibly stale or divergent data, reconcile later (Cassandra, DynamoDB eventual mode).
• When there is no partition you can have both C and A; CAP says nothing about latency.
• "C" here is linearizability, not the "C" in ACID.

If Partitioned choose Availability or Consistency, Else (normal operation) choose Latency or Consistency. It captures that replication always trades latency for consistency even without failures.

• Synchronous replication to a quorum costs round-trip latency on every write (EC); async replication is fast but may read stale (EL).

Linearizability (single up-to-date copy illusion) > sequential > causal > read-your-writes / monotonic reads (session guarantees) > eventual.

• Linearizable: operations appear atomic in real-time order; needed for locks, leader election, unique constraints.
• Causal: causally related operations are seen in order everywhere; concurrent ones may differ.
• Session guarantees: read-your-writes, monotonic reads, monotonic writes, writes-follow-reads.
• Eventual: if writes stop, replicas converge; no ordering promise meanwhile.
• Serializability is a transaction isolation property (multi-object), distinct from linearizability (single-object recency); "strict serializability" is both.

Make sure the user's reads after a write either hit the leader or a follower known to have caught up.

• Route reads of recently written data (or the user's own profile) to the leader for N seconds after a write.
• Track the replication position (LSN / GTID / version) in the session or token and read only from replicas that have reached it.
• Sticky routing of a user to one replica gives monotonic reads (not read-your-writes).
• Cross-device is harder: the position must be stored server-side per user.

With N replicas, write to W and read from R; if R + W > N the read and write sets overlap, so a read sees the latest acknowledged write (absent concurrent-write and sloppy-quorum caveats).

• Overlap does not give linearizability on its own (concurrent writes, clock skew, sloppy quorums and hinted handoff weaken it).
• Conflict resolution: last-write-wins (timestamps), vector clocks, CRDTs.

Latency is time per operation, throughput is operations per unit time, bandwidth is the maximum data rate of the link. Throughput = concurrency / latency (Little's Law).

• Higher parallelism raises throughput without lowering latency, until a resource saturates.
• Batching improves throughput but increases per-item latency.
• A 1 Gbps link has high bandwidth but a transatlantic round trip still costs ~80 ms.

Availability is "can I use it now", reliability is "does it behave correctly over time", durability is "is stored data never lost", and fault tolerance is the ability to keep operating despite component failures.

• Durability is achieved via replication, write-ahead logs, fsync, cross-AZ/region copies, backups (S3 claims 11 nines of durability yet 99.99% availability).
• A system can be available but wrong (unreliable), or durable but unavailable.
• Fault (component deviates) vs failure (system violates spec): fault tolerance prevents faults from becoming failures.

Compute writes/s, reads/s using a read-heavy ratio, then storage over retention.

• Writes: 100M / (30 x 86,400) = ~40/s. At 100:1 read ratio: ~4,000 reads/s avg, ~10-20k peak.
• Record ~500 B (long URL 400 B + code + metadata): 100M x 500 B = 50 GB/month, ~3 TB over 5 years, x3 replication = ~9 TB.
• Key space: base62, 7 chars = 62^7 = ~3.5 trillion, ample for 6 billion (5 years).
• Read bandwidth: 4,000 x 500 B = 2 MB/s (trivial); cache the hottest 20% (80/20 rule) in memory.

You cannot exceed a hard dependency's availability, so make the dependency non-critical on the request path.

• Cache last-known-good responses, serve stale on failure.
• Queue and retry asynchronously (accept the request now, call the vendor later).
• Add a secondary provider with failover, circuit breaker and timeout.
• Define degraded mode explicitly (e.g. "payment pending") and have the SLO exclude features that are optional.
• Series math: 0.9999 x 0.999 = ~99.89%, so the hard dependency dominates.

Match consistency to the cost of being wrong: money requires strong consistency and transactions, a like count tolerates eventual consistency.

• Counters can be sharded (N sub-counters summed on read) or batched via a stream to avoid hot rows.

Identify the first bottleneck from the estimate, fix only that, and describe the next two so the design shows a growth path without over-engineering.

• Compute per-component limits: a single Postgres handles ~5-10k simple QPS; a Redis node ~100k ops/s; a Kafka partition ~10 MB/s typical.
• If peak QPS < single-node limits, prefer vertical scaling plus replicas and a cache.
• Introduce sharding, multi-region, CQRS only when a number justifies it.

Messages: 50M x 40 = 2B/day = ~23k/s average, ~70k/s peak (3x).

• Storage: 100 B average text + 100 B metadata = 200 B x 2B = 400 GB/day, ~146 TB/year, x3 replication = ~440 TB/year.
• Connections: assume 20% concurrently online = 10M WebSocket connections; at ~100k connections/gateway node (tuned) that is ~100 gateway nodes.
• Fan-out: 1:1 chat = 1 delivery; group chat of 100 multiplies by 100 so groups dominate egress.
• Hot recent messages in memory/Cassandra-like wide-row store keyed by (chat_id, time).

The time is network: DNS + TCP + TLS handshakes plus multiple round trips at ~200 ms each from India to Virginia dominate.

• Round trips: DNS (1) + TCP (1) + TLS 1.3 (1) + request (1) = ~4 x 200 ms = 800 ms before bytes, and each sequential API call adds another 200 ms.
• Fixes: CDN for static assets and edge TLS termination, connection reuse/HTTP/2-3, regional deployment, read replicas or edge cache close to users, reduce sequential calls (batch/GraphQL/BFF).
• Measure with real-user monitoring split by geography before changing anything.

Pick user-centric SLIs (success ratio, latency) at the edge, set an SLO with a budget, and alert on budget burn rate rather than raw thresholds.

• SLI availability: successful (non-5xx) checkout requests / valid requests. SLO 99.95% / 30 days.
• SLI latency: 95% of requests < 400 ms, 99% < 1 s.
• Multi-window burn-rate alerts: page when 2% budget burns in 1 hour (burn rate 14.4) confirmed by a short window; ticket for slow burns (6x over 6 hours).
• 4xx caused by clients are excluded from "bad".

This is replication lag violating read-your-writes; pick routing or consistency changes based on how many reads are affected.

• Read the user's own recent writes from the leader (time-window or LSN-based routing).
• Semi-synchronous replication to one follower (adds write latency, bounds lag).
• Return the updated object in the write response and have the UI use it (no re-read needed).
• Monitor lag and remove lagging replicas from the read pool automatically.

Cart: AP (always accept adds, merge divergent versions later, as in the Dynamo paper). Inventory reservation/payments: CP (reject rather than oversell).

• Cart merge: union of items (CRDT-like set); removed items may reappear, which is acceptable.
• Inventory: atomic decrement on a single authoritative shard; or reserve with TTL and confirm on payment; reconcile asynchronously.
• Overselling can sometimes be tolerated with compensation (apology + refund) when it is cheaper than unavailability.

Concurrent in-flight work = QPS x latency = 20,000 x 0.05 s = 1,000 busy CPU-seconds per second, so ~1,000 cores at 100% utilization.

• Target 50-60% utilization for headroom: ~2,000 cores = 125 nodes of 16 cores.
• Add N+1/N+2 (survive an AZ loss: size so remaining AZs carry full load, i.e. x1.5 across 3 AZs).
• If most time is I/O waiting, threads (or async) rather than cores bound concurrency; measure CPU time separately from wall time.

Make consistency a per-operation choice, not a global one: strong paths for invariants, relaxed paths for reads and aggregates.

• E.g. e-commerce: order placement and payment on strongly consistent store; product catalog and search from replicas/cache with TTL; recommendations eventual.
• Expose consistency in the API (e.g. DynamoDB ConsistentRead, Cassandra per-query CL, Spanner stale reads with bounded staleness).
• Document staleness bounds so the UI can set expectations.

Serial composition gives ~99.8%, below the 99.95% goal, so change the architecture rather than the numbers.

• Make one call asynchronous (queue it; user gets accepted-response), removing it from the synchronous availability product.
• Add redundancy within each service (multi-AZ, N+1) and verify failures are independent.
• Cache or precompute results from one service; degrade gracefully when it is down.
• Reduce the real failure rate: safer deploys (canary), timeouts, bulkheads; most outages come from changes.