Cloud Computing Fundamentals

They differ in how much of the stack the provider manages. IaaS gives virtual machines, disks and networks; PaaS gives a managed runtime for your code; SaaS gives a finished application; FaaS runs individual functions on demand.

• IaaS: EC2, EBS, VPC. You patch the OS and runtime.
• PaaS: Elastic Beanstalk, App Runner, RDS (managed platform for a database).
• SaaS: Gmail, Salesforce. You only configure and use it.
• FaaS: AWS Lambda. You supply a function; scaling, patching and capacity are the provider's job.

A region is a geographic area (for example ap-south-1) made of several isolated Availability Zones. An AZ is one or more data centers with independent power, cooling and networking, connected to its siblings by low-latency links.

• Deploy across at least two AZs for high availability inside a region.
• Regions are isolated from each other; data does not leave a region unless you replicate it.

Weigh data residency and compliance first, then latency to users, then service availability and price.

• Some services and instance types launch in a few regions only.
• Prices vary per region.
• Disaster recovery regions should be far enough to avoid shared regional events.

The provider is responsible for security "of" the cloud (hardware, hypervisor, facilities, managed-service internals); the customer is responsible for security "in" the cloud (data, IAM, network rules, OS patching on IaaS, application code).

• The split shifts with the service model: on EC2 you patch the OS; on Lambda AWS patches the runtime but you still own code, dependencies and permissions.
• Encryption configuration, public bucket exposure and leaked keys are customer-side failures.

IAM controls who (principal) can do what (action) on which resource under which conditions. Building blocks: users, groups, roles, and policies.

• A role has no long-term credentials; principals assume it and receive temporary credentials from STS.
• Policies are JSON documents attached to identities or resources.
• Explicit Deny always wins over Allow; the default is implicit deny.

Grant only the actions and resources a workload needs, for as long as it needs them. Start narrow and widen from evidence.

• Use specific actions and resource ARNs instead of *.
• Use IAM Access Analyzer and last-accessed data to prune unused permissions.
• Add conditions (source VPC, tags, MFA) and permission boundaries.

{
  "Effect": "Allow",
  "Action": ["s3:GetObject"],
  "Resource": "arn:aws:s3:::orders-exports/reports/*"
}

Use a role. Compute services (EC2 instance profile, ECS task role, Lambda execution role, EKS IRSA/Pod Identity) deliver short-lived credentials automatically, so there is nothing to store or rotate.

• Access keys are long-lived and leak through repos, logs and images.
• The AWS SDK default credentials provider chain picks up role credentials with no code change.

Identity-based policies attach to users and roles; resource-based policies attach to the resource (S3 bucket, SQS queue, KMS key). Within one account either one allowing is enough; across accounts both sides must allow.

• Evaluation: explicit deny anywhere wins, then SCPs and permission boundaries must allow, then an allow must exist.
• Session policies can only further restrict.

A VPC is your logically isolated virtual network with a CIDR range, spanning all AZs in a region. Inside it you create subnets (each in one AZ), route tables, gateways and security controls.

• Internet Gateway (IGW) for public internet access.
• NAT Gateway for outbound-only access from private subnets.
• VPC endpoints for private access to AWS services.

A subnet is public if its route table has a route 0.0.0.0/0 to an Internet Gateway; otherwise it is private. Instances also need a public or Elastic IP to be reachable from the internet.

• Typical layout: load balancers in public subnets; app and database tiers in private subnets across 2-3 AZs.
• Private instances reach the internet outbound through a NAT Gateway in a public subnet.

Security groups are stateful, attach to network interfaces, and only have allow rules. NACLs are stateless, attach to subnets, have numbered allow and deny rules evaluated in order.

• Stateful: return traffic is automatically allowed.
• NACLs need explicit ephemeral-port rules for responses.
• Security groups can reference other security groups, which is the preferred way to model tiers.

They let resources reach AWS services without traversing the internet or a NAT Gateway. Gateway endpoints (S3, DynamoDB) are route-table entries and free; interface endpoints (PrivateLink) are ENIs with private IPs and are billed hourly plus per GB.

• Reduces NAT data-processing cost and exposure.
• Endpoint policies can restrict which buckets or actions are reachable.

VPC peering is a point-to-point, non-transitive link. Transit Gateway is a hub for many VPCs and on-premises links. Site-to-Site VPN runs over the internet; Direct Connect is a dedicated private circuit.

• Peering requires non-overlapping CIDRs.
• PrivateLink exposes a single service instead of the whole network.

Application Load Balancer (L7, HTTP/HTTPS, path and host routing), Network Load Balancer (L4, TCP/UDP/TLS, very high throughput, static IPs), Gateway Load Balancer (L3 for virtual appliances).

• ALB features: path/host/header routing, WebSocket, gRPC, OIDC auth, WAF integration.
• NLB preserves source IP and handles millions of requests per second with low latency.

A target group holds targets (instances, IPs, Lambdas). The LB probes each target on a path and port; targets failing a threshold of consecutive checks are removed from rotation until they pass again.

• Make the health endpoint cheap and reflect the ability to serve (readiness), not full dependency health.
• Tune interval, thresholds and deregistration delay (connection draining).

With it enabled, each LB node distributes requests across targets in all AZs; disabled, each node only sends to targets in its own AZ.

• Always on for ALB; off by default for NLB (cross-AZ data charges apply when enabled).
• Without it, uneven target counts per AZ cause uneven load.

Autoscaling adjusts capacity to demand. EC2 Auto Scaling groups support target tracking (keep a metric near a value), step scaling, scheduled scaling and predictive scaling.

• Target tracking on average CPU or RequestCountPerTarget is the simplest start.
• Set min, desired and max; spread across AZs.
• Cooldowns and warm-up prevent flapping.

Vertical scaling makes one machine bigger; horizontal scaling adds more machines. Horizontal needs stateless services and gives fault tolerance; vertical is simple but capped and usually needs a restart.

Object (S3): flat key-addressed blobs over HTTP, virtually unlimited, eventually durable at 11 nines. Block (EBS): raw volumes attached to one instance, low latency, for databases and boot disks. File (EFS/FSx): shared POSIX or SMB file systems mounted by many hosts.

S3 offers strong read-after-write consistency for all operations since December 2020. Standard stores data across at least three AZs with 99.999999999% durability.

• Classes: Standard, Intelligent-Tiering, Standard-IA, One Zone-IA, Glacier Instant/Flexible/Deep Archive.
• Lifecycle rules transition or expire objects automatically.
• Versioning and Object Lock protect against deletes and overwrites.

Enable Block Public Access at account level, use bucket policies with least privilege, enforce TLS, encrypt with SSE-S3 or SSE-KMS, and enable access logging and versioning.

• Share content with presigned URLs or CloudFront Origin Access Control instead of making objects public.
• Use VPC endpoint policies to constrain access paths.

gp3 is the general default with baseline 3,000 IOPS and 125 MB/s adjustable independent of size; io2 is for latency-sensitive databases with provisioned IOPS; st1/sc1 are HDD for throughput and cold data.

• EBS is AZ-bound; snapshots go to S3 and can restore in another AZ or region.

A managed database (RDS, Aurora, DynamoDB) automates provisioning, patching, backups, failover and monitoring. You still own schema design, indexing, queries, capacity choices and access control.

• RDS supports PostgreSQL, MySQL, MariaDB, Oracle, SQL Server; Aurora is a cloud-native MySQL/PostgreSQL-compatible engine.
• Limitations: no OS access, restricted superuser, constrained parameters.

Multi-AZ is for availability: a synchronous standby in another AZ with automatic failover, not readable (in the classic single-standby setup). Read replicas are for read scaling: asynchronous, readable, can be cross-region, and promotable manually.

• Failover typically takes 60-120 seconds and the endpoint DNS flips.
• Replica lag means stale reads.

Aurora separates compute from a distributed storage layer that keeps six copies across three AZs, auto-grows to 128 TiB, and supports up to 15 low-lag replicas that share the same storage.

• Faster failover (typically under 30 seconds) as replicas share storage.
• Aurora Serverless v2 scales compute in fine increments.
• Aurora Global Database replicates across regions with typically sub-second lag.

Pick DynamoDB for known access patterns, massive scale, single-digit-millisecond latency and minimal operations. Pick relational for ad hoc queries, joins and complex transactions.

• Design keys from access patterns; a poor partition key causes hot partitions.
• On-demand mode bills per request; provisioned mode with autoscaling is cheaper for steady load.
• Global tables give multi-region active-active replication.

SQS is a queue: one consumer group pulls and deletes each message. SNS is pub/sub push to many subscribers. Kinesis/MSK are ordered, replayable streams where consumers track their position.

• Fan-out pattern: SNS topic to several SQS queues.
• Streams support replay and multiple independent readers; queues do not retain after delete.

When a consumer receives a message it becomes invisible for the visibility timeout. If not deleted before it expires, the message reappears. After maxReceiveCount failed receives it moves to the DLQ.

• Set the timeout above the processing time (Lambda: at least 6x function timeout).
• Alarm on DLQ depth; redrive after fixing the bug.

Serverless means no server management, automatic scaling (including to zero) and pay-per-use pricing. Trade-offs: cold starts, execution limits (Lambda 15 minutes), vendor coupling, harder local debugging, and cost surprises at steady high volume.

A cold start is the extra latency when Lambda must create a new execution environment: download code, start the runtime, and run initialization before the handler. Later invocations reuse the warm environment.

• Phases: environment creation, runtime init, your static/init code, then the handler.
• Cold start impact is rare in steady traffic but visible after idle periods or scale-out bursts.

JVM startup, class loading, JIT warm-up and framework initialization (reflection, classpath scanning, DI) dominate. Typical Spring Boot cold starts are several seconds.

• Use lightweight frameworks or functional approaches; avoid classpath scanning.
• Shrink the jar; lazy-initialize SDK clients but build them outside the handler.
• Use SnapStart, provisioned concurrency, or GraalVM native image.
• More memory means more CPU and faster init.
• Prefer UrlConnection/CRT HTTP clients over heavy defaults.

SnapStart takes an encrypted snapshot of the initialized execution environment (memory and disk state) when you publish a version, and resumes new environments from that snapshot instead of initializing from scratch. It supports Java 11+ managed runtimes and cuts cold starts often by up to 10x.

• Works on published versions or aliases, not $LATEST.
• Uniqueness issues: random seeds, unique IDs, and network connections created during init may be duplicated or stale after restore.
• CRaC runtime hooks (beforeCheckpoint, afterRestore) let you refresh state.

Provisioned concurrency keeps N environments initialized and warm (costs money continuously, removes cold starts for that capacity). Reserved concurrency carves out a guaranteed and maximum concurrency for a function from the account pool (free).

• Reserved acts as both a guarantee and a throttle cap, useful to protect a downstream database.
• Account default concurrency limit is 1,000 per region (can be raised).

Each concurrent request gets its own environment; Lambda adds environments quickly (burst of up to 1,000 per 10 seconds per function) until the account or reserved limit.

• Limits: 15-minute timeout, 10 GB memory, 6 MB sync payload, 250 MB unzipped package (10 GB for container images).
• Throttled sync calls return 429; async ones are retried by Lambda.

Twelve-factor is a methodology for portable, scalable apps. Key factors: one codebase per app, explicit dependencies, config in the environment, backing services as attached resources, strict build/release/run separation, stateless processes, port binding, concurrency via processes, disposability, dev/prod parity, logs as event streams, admin tasks as one-off processes.

Instances are created, replaced and terminated at any time, so no instance may hold unique state. Put sessions in Redis/ElastiCache or use signed tokens, files in S3, data in a managed database, and work items in queues.

VMs virtualize hardware and each run a full guest OS under a hypervisor; containers share the host kernel and isolate processes with namespaces and cgroups. Containers start in seconds, are lighter and denser; VMs give stronger isolation.

• Firecracker microVMs (Lambda, Fargate) combine VM isolation with container-like startup.
• Containers must match the host kernel family (Linux containers need a Linux kernel).

ECS and EKS are orchestrators (AWS-native and managed Kubernetes). EC2 and Fargate are the compute they run on: EC2 means you manage the nodes, Fargate is serverless per-task compute.

• Fargate: no node patching, per-vCPU/GB-second billing, slower scaling than warm nodes, no privileged containers or host access.
• EKS adds portability and the Kubernetes ecosystem at the cost of operational complexity and a control-plane fee.

Modern JVMs (Java 10+, 8u191+) are container-aware and size heap from the cgroup limit. Set -XX:MaxRAMPercentage (for example 70-75) to leave room for metaspace, thread stacks and native memory.

• Memory limit must cover heap plus non-heap; otherwise the OOM killer terminates the container (exit code 137).
• CPU limits below 1 core make the JVM pick the serial GC and 1 compiler thread.
• Use layered or jlink-trimmed images and graceful shutdown to handle SIGTERM.

RTO (Recovery Time Objective) is the maximum acceptable time to restore service. RPO (Recovery Point Objective) is the maximum acceptable data loss measured in time.

• RPO of 5 minutes means backups or replication must be no more than 5 minutes behind.
• Tighter targets cost more.

From cheapest and slowest to most expensive and fastest: backup and restore (hours), pilot light (core data replicated, compute off, tens of minutes), warm standby (scaled-down full stack running, minutes), multi-site active/active (near zero).

HA means the system keeps serving despite component failure, achieved by removing single points of failure through redundancy across AZs. Availability = uptime / total time; 99.9% allows about 43 minutes of downtime a month, 99.99% about 4.3 minutes.

• Serial dependencies multiply availability: two 99.9% services in series give about 99.8%.
• Parallel redundancy raises it.

For regional-outage tolerance beyond what AZs give, strict RTO/RPO, data-residency partitioning, or global low latency. It adds major cost and complexity: data replication, conflict resolution, routing and operations.

• Patterns: active/passive with Route 53 failover; active/active with latency routing and global tables or Aurora Global Database.
• Active/active needs a strategy for write conflicts (last-writer-wins, regional ownership of data).

Route 53 supports simple, weighted, latency-based, geolocation, geoproximity, failover and multivalue routing, driven by optional health checks.

• DNS-based failover is bounded by TTL and client caching.
• Alias records point to AWS resources at the zone apex with no extra query charge.

A CDN caches content at edge locations close to users, cutting latency and origin load. CloudFront fronts S3, ALB or custom origins, terminates TLS at the edge and integrates with WAF and Shield.

• Control caching with Cache-Control headers and cache policies; invalidate sparingly.
• Versioned file names avoid invalidation.

Metrics (aggregated numbers over time), logs (discrete events) and traces (a request's path across services). Together they answer what is wrong, why, and where.

• AWS: CloudWatch Metrics/Logs/Alarms, X-Ray or OpenTelemetry (ADOT), and CloudTrail for API audit.
• Emit structured JSON logs with a correlation or trace ID.

Alert on symptoms that affect users, not every cause. The four golden signals: latency, traffic, errors and saturation.

• Alert on SLO burn rate rather than a raw CPU threshold.
• Use percentiles (p95, p99), not averages, for latency.
• Every alert needs an owner and a runbook.

Keep config outside the artifact (environment variables, SSM Parameter Store, AppConfig) and secrets in a secret store (AWS Secrets Manager) encrypted by KMS, fetched at runtime via IAM roles.

• Secrets Manager supports automatic rotation (RDS integration); Parameter Store is cheaper and fine for plain config and SecureString.
• Never commit secrets, bake them into images, or print them in logs.

KMS manages encryption keys in HSM-backed storage. Envelope encryption generates a data key per object, encrypts the data locally with it, then encrypts the data key with a KMS key and stores both together.

• Avoids sending large data to KMS (which caps direct encrypt at 4 KB).
• Key policies and grants control use; every use is logged in CloudTrail.

IaC defines infrastructure in versioned declarative files (CloudFormation, CDK, Terraform) so environments are reproducible, reviewable and auditable.

• Prevents configuration drift and manual console changes.
• State handling: Terraform state must be stored remotely with locking; CloudFormation tracks state in stacks.

Make cost visible, remove waste, right-size, then commit. Tag resources, use Cost Explorer and budgets, delete idle resources, pick right instance sizes, then buy Savings Plans or Reserved Instances for stable baseline.

• Spot instances give up to ~90% discount for interruptible work.
• Use S3 lifecycle and Intelligent-Tiering, gp3 volumes, Graviton (arm64) instances for better price-performance.
• Reduce data transfer: VPC endpoints, same-AZ traffic, CDN.

Cover the steady baseline with Savings Plans or Reserved Instances, handle variable load on-demand, and run fault-tolerant batch or stateless workers on Spot.

• Spot gets a two-minute interruption notice; handle it with checkpointing and diversified instance pools.
• Compute Savings Plans apply across EC2, Fargate and Lambda.

Data transfer (egress to internet, cross-AZ, cross-region), NAT Gateway per-GB processing, unattached EBS volumes and old snapshots, idle load balancers, over-verbose CloudWatch logs, and API request charges.

AWS's guidance organized in six pillars: operational excellence, security, reliability, performance efficiency, cost optimization and sustainability, used for design reviews and trade-off discussions.

API Gateway is a managed front door that handles routing, auth (IAM, Cognito, JWT, Lambda authorizers), throttling, request validation and caching for REST, HTTP and WebSocket APIs.

• HTTP APIs are cheaper and faster than REST APIs but have fewer features.
• Synchronous integration timeout is 29 seconds by default for REST APIs.

Synchronous calls couple caller availability and latency to the callee; asynchronous messaging buffers work in a queue, absorbs spikes and lets each side scale and fail independently.

• Needs idempotent consumers, retry with backoff and DLQs.
• Trade-off: eventual consistency and harder end-to-end tracing.

Cloud calls fail transiently. Use bounded timeouts, retry only idempotent operations with exponential backoff plus jitter, and trip a circuit breaker to stop hammering a failing dependency.

• Retries multiply load (retry storms); cap attempts and budget retries.
• AWS SDK v2 has built-in retry policies with backoff.

Rolling (replace instances gradually), blue/green (a full new environment, switch traffic), canary (a small percentage first, then ramp), and feature flags to decouple deploy from release.

• Blue/green gives instant rollback but doubles capacity temporarily.
• Database changes must be backward compatible (expand/contract) so old and new code coexist.

Scalability is the ability to handle more load by adding resources; elasticity is doing so automatically and also releasing them when demand drops, so you pay only for what you use.

Multi-tenancy serves many customers on shared infrastructure. Models: silo (account or stack per tenant), pool (shared resources with a tenant ID), or bridge (mix).

• Pool is cheapest but needs strict row-level access control and noisy-neighbor controls such as per-tenant throttling and quotas.
• Silo gives stronger blast-radius and compliance isolation at higher cost.

An account is the strongest isolation boundary for permissions, quotas and billing. Organizations groups accounts, gives consolidated billing, and applies Service Control Policies as guardrails.

• Typical split: prod, non-prod, shared services, logging/security.
• Limits and service quotas are per account, which also contains blast radius.

If the service runs in at least two AZs behind a load balancer, health checks drain the failed AZ's targets and traffic shifts to survivors; the Auto Scaling group launches replacements in healthy AZs. A single-AZ deployment goes down.

• Check: capacity headroom (survivors must absorb the extra load, so run N+1 AZ capacity), RDS Multi-AZ failover, single-AZ resources (NAT Gateway, EBS volumes, One Zone caches).
• Do not fail over manually before verifying health checks and metrics.

The database was deployed Single-AZ, or Multi-AZ failover is delayed or blocked. Single-AZ RDS has no standby, so recovery needs a restore from snapshot in another AZ.

• Multi-AZ failover takes a minute or two and the app must reconnect; stale pooled connections and cached DNS prolong the outage.
• Mitigate with Multi-AZ enabled, connection validation and retry in the pool, and a short DNS TTL in the JVM.

Use Cost Explorer grouped by service, then by usage type, account, region and tag, comparing day over day to find the spike. Then drill down in CloudTrail and resource listings to identify who created what.

• Common causes: leaked access keys mining crypto in an unused region, NAT/data transfer from a loop, runaway Lambda recursion, unbounded logs, forgotten large instances or snapshots.
• Contain first: disable the keys, stop resources; then fix and add guardrails.

An infinite trigger loop: each output object fires a new invocation which writes another object. Concurrency and cost grow until throttled.

• Fix: write to a different bucket or prefix and use prefix/suffix filters on the notification.
• Set reserved concurrency to cap blast radius; Lambda also detects some recursive SQS/SNS/Lambda loops and stops them.
• Add a billing alarm and anomaly detection.

Deactivate the key immediately (not just delete, so you can audit), review CloudTrail for its activity, revoke active sessions via a deny policy keyed on aws:TokenIssueTime, remove unauthorized resources, then rotate and move the workload to roles.

• Removing the commit from git history is not enough; assume it was scraped.
• Enable secret scanning and pre-commit hooks.

Symptoms: latency spikes without matching load, high CPU steal time, or throttled I/O. Check %steal in top, burstable credit balance, and EBS queue length.

• Fixes: dedicated hosts or the largest size of a family, Nitro-based non-burstable types, provisioned IOPS (io2/gp3), container CPU and memory requests and limits.
• For multi-tenant apps, apply per-tenant rate limits and bulkheads.

Burstable instances earn CPU credits while below baseline and spend them when above. When credits run out, the CPU is throttled to the baseline (for example 20% of a vCPU on a t3.medium in standard mode).

• Check the CPUCreditBalance metric.
• Fix: switch to M/C/R families, enable unlimited mode (extra charge), or right-size.

Reactive scaling takes minutes (metric lag, instance boot, JVM warm-up). Use scheduled or predictive scaling before the known event, keep headroom, and absorb bursts with queues, caching and rate limiting.

• Use warm pools, smaller faster-booting images, or containers.
• Scale on a leading indicator (request rate, queue depth) rather than CPU.
• Contact AWS for extreme known spikes and check service quotas.

Common causes: health check grace period shorter than app startup time, so instances are marked unhealthy and killed before ready; scale-in and scale-out thresholds too close; or a crashing app.

• Set the grace period above JVM and Spring startup time, use target tracking, and separate cooldowns.
• Check the ALB health check path, port and security group reachability.

502 means the target sent an invalid response or closed the connection; 504 means the target did not respond within the ALB idle timeout (default 60 seconds).

• Classic 502 cause: the backend's keep-alive timeout is shorter than the ALB idle timeout, so the backend closes a reused connection. Set the app keep-alive higher than the ALB idle timeout (for example Tomcat keepAliveTimeout above 60 s).
• 504: slow queries or downstream calls; add timeouts and check target response time.
• Review ALB access logs and target_status_code.

Typical causes: the target's security group does not allow inbound from the load balancer's security group on the health check port; wrong health check path or port; app returns 401/302 on the path; or the app has not finished starting.

• Fix by referencing the ALB security group as the source.
• Make the health path unauthenticated and return 200.

Verify the route table: a 0.0.0.0/0 route to a NAT Gateway (which itself sits in a public subnet with an IGW route and an Elastic IP). Then check security group egress, NACLs (ephemeral ports), DNS resolution and, for S3, a gateway endpoint plus endpoint and bucket policies.

• Use VPC Reachability Analyzer and flow logs to see where packets drop.

A VPC-attached Lambda has only private IPs, so it needs a route through a NAT Gateway in a public subnet to reach the internet. Placing it in a public subnet does not help since its ENI has no public IP.

• For AWS services use VPC endpoints to avoid NAT cost.
• Only attach a Lambda to a VPC if it needs VPC resources like RDS.

Measure first (init versus handler duration in the REPORT line), then apply the cheapest wins.

• 1. Enable SnapStart on a published version/alias; typically cuts to a few hundred ms.
• 2. Raise memory (CPU scales with it) and use arm64.
• 3. Trim dependencies, lazy beans, avoid classpath scanning, or move to Spring Cloud Function, Micronaut or Quarkus.
• 4. Provisioned concurrency for latency-critical paths.
• 5. GraalVM native image for the best cold start with build complexity.

Random or ID-generator state was initialized during init and captured in the snapshot, so restored environments start from the same state.

• Generate randomness and unique IDs inside the handler, or reseed in a CRaC afterRestore hook.
• Re-create network connections and refresh credentials or time-bound caches after restore.

Each concurrent environment opens its own connection, so a burst of 500 invocations exhausts max_connections and new connections hang or fail.

• Use RDS Proxy to pool connections, cap reserved concurrency, keep pool size at 1 per environment, and reuse connections across invocations.
• Alternatives: DynamoDB, or a queue in front to throttle consumers.

No. SQS Standard is at-least-once. Duplicates occur on visibility timeout expiry, partial batch failures or retries, so consumers must be idempotent.

• Use a deduplication store (DynamoDB conditional write, idempotency key) or naturally idempotent operations (upsert).
• Report partial batch failures with batchItemFailures so successful messages are not re-run.

Configure a dead-letter queue with a maxReceiveCount (for example 3-5) so repeated failures move aside, alarm on DLQ depth, inspect and redrive after a fix.

• On FIFO queues a stuck message blocks its message group, so a DLQ is essential.
• Distinguish transient errors (retry with backoff) from permanent ones (send to DLQ immediately).

Scale consumers on ApproximateNumberOfMessagesVisible (or backlog per instance), increase batch size, optimize processing, and for downstream limits apply backpressure or rate limiting rather than more consumers.

• Track age of the oldest message as the key SLO signal.
• If ordering is required per key, FIFO message groups allow parallelism across groups only.

Pick Kinesis (or Kafka/MSK) when you need ordered replay, multiple independent consumers of the same data, or time-window processing. Pick SQS for simple work distribution with per-message acknowledgement and easy scaling.

• Kinesis ordering is per shard; hot partition keys limit throughput (1 MB/s or 1,000 records/s per shard write).
• A failing record blocks the shard unless handled with retries, bisect on error or skip.

Exactly-once delivery is not realistic; build exactly-once effect with at-least-once delivery plus idempotent processing. Store an idempotency key with the result in the same transaction as the side effect.

• Use unique constraints or a conditional write (attribute_not_exists) keyed by message or business ID.
• For cross-service flows use the outbox pattern and sagas.

Find the top queries with Performance Insights and fix indexes first. Then offload reads to replicas (reader endpoint), add a cache (ElastiCache) for hot data, and only then scale the instance up.

• Replica reads are eventually consistent; route read-your-writes paths to the primary.
• Pool connections with RDS Proxy.

Use Aurora Global Database (replication lag typically under 1 second) with a warm standby stack in the secondary region. On disaster, detach and promote the secondary cluster, then shift Route 53 to the secondary region.

• Pre-provision compute, secrets, KMS keys and config in both regions; Secrets Manager supports multi-region replication.
• Automate and rehearse the runbook; measure real RTO.

Enable S3 versioning (and MFA Delete or Object Lock for compliance), restrict s3:DeleteObject* permissions, and replicate to another account or region.

• With versioning, a delete adds a delete marker; remove the marker to restore.
• Lifecycle rules should expire noncurrent versions after a retention period to control cost.

Use expand/contract: first deploy a backward-compatible schema (add a nullable column), deploy code that writes both, backfill, switch reads, and only later drop the old column. Old and new app versions must both work against the intermediate schema.

• Blue/green and rolling deployments always have a window where both versions run.

Exit 137 is SIGKILL (128 + 9), most often the kernel OOM killer after the container exceeded its memory limit. In Java the cause is usually heap plus metaspace, threads and direct buffers exceeding the limit, not a Java OutOfMemoryError.

• Set -XX:MaxRAMPercentage=70, cap -XX:MaxMetaspaceSize and -Xss if needed, and raise the limit or reduce load.
• Enable -XX:+HeapDumpOnOutOfMemoryError to a mounted path; check OOMKilled in task or pod status.

Old instances receive SIGTERM and exit before finishing requests, or the load balancer keeps sending traffic while they stop. Deregister first, then drain, then stop.

• Enable Spring Boot server.shutdown=graceful and a spring.lifecycle.timeout-per-shutdown-phase.
• Set the ALB deregistration delay and the ECS stopTimeout (or Kubernetes terminationGracePeriodSeconds plus a preStop sleep) to cover the drain.
• Ensure PID 1 receives signals (exec form entrypoint).

Configuration drift: manual console changes, per-environment config baked into builds, unpinned dependencies or mutable latest image tags.

• Build once, promote the same immutable artifact (digest-pinned image) through environments with only injected config differing.
• Provision everything with IaC and detect drift.

The app read the secret once at startup, so after rotation its credentials became invalid. Or the rotation was not two-phase and the old password stopped working while consumers still used it.

• Fetch with a short TTL cache and refetch on auth failure.
• Use alternating-user rotation (Secrets Manager multi-user strategy) so old and new credentials overlap.
• Test rotation in staging.

Use federated SSO with a time-boxed role (IAM Identity Center permission set), read-only by default, with MFA, an approval workflow and CloudTrail logging. Prefer session-based access (SSM Session Manager) over opening SSH ports.

• No shared accounts or long-lived keys.
• Break-glass roles with alerts when used.

An explicit deny in any applicable policy wins. In the same account, an allow in either the identity or the resource policy is sufficient when there is no deny.

Check for explicit denies in the bucket policy, SCPs, permission boundaries or session policies; the object's KMS key policy (the role also needs kms:Decrypt); the correct resource ARN (bucket/* versus bucket); VPC endpoint policy; and the object owner for cross-account writes.

• Use the IAM policy simulator and CloudTrail error details.

Find the top talkers with VPC Flow Logs, then add S3 and DynamoDB gateway endpoints (free), interface endpoints for heavy AWS services (ECR, CloudWatch, STS), and keep inter-service traffic inside the VPC.

• Pull container images via ECR endpoints; cache dependencies.
• A NAT per AZ avoids cross-AZ charges but costs hourly each; weigh volume.

Cross-AZ traffic costs about $0.01/GB each direction. Reduce chattiness (batching, compression), co-locate heavily communicating services per AZ with zone-aware routing (topology-aware routing in Kubernetes, ALB zonal affinity), and cache.

• Do not drop to a single AZ to save money; you lose resilience.

Schedule shutdowns outside working hours, use smaller instance sizes, Spot, and ephemeral per-PR environments torn down automatically; apply TTL tags with cleanup jobs and budgets per account.

For steady, sustained load containers are usually cheaper and avoid cold starts and Lambda concurrency limits. Lambda wins for spiky, low-volume or idle-heavy workloads and for minimal operations.

• Rough check: Lambda cost scales with requests x duration x memory; at constant 200 rps with 100 ms and 512 MB, a few Fargate tasks cost less.
• Consider team skills, latency requirements and deployment model.

Lambda caps at 15 minutes and 10 GB memory. Break work into chunks via SQS or Step Functions, stream the file rather than loading it, or move to Fargate/AWS Batch for long jobs.

• Step Functions orchestrates fan-out with the Map state.
• Use S3 byte-range reads and increase /tmp ephemeral storage (up to 10 GB) if required.

First cache static and cacheable API responses at the edge with CloudFront. For dynamic traffic, use CloudFront or Global Accelerator for backbone routing, then deploy a regional stack closer to users with latency-based routing.

• The hard part is data: read replicas or global tables near users, with a single-writer or conflict strategy for writes.

Both regions accept writes to the same record and replicate asynchronously; DynamoDB global tables resolve conflicts by last-writer-wins on timestamp, so one write silently overwrites the other.

• Mitigate: route each entity to a home region (partition ownership), use CRDTs or merge logic, or make writes append-only/commutative.
• Aurora Global Database is single-writer by design.

Replication was asynchronous, so writes committed in the failed region but not yet replicated were lost; this is the actual RPO, which is nonzero for cross-region async replication.

• Reconcile later from the primary's logs or event stream if recoverable, and revisit the RPO decision.
• Synchronous cross-region replication is slow and rarely used.

Ingestion is the main cost. Lower log levels in production (no DEBUG), drop noisy health-check logs, sample verbose events, set retention periods, and archive to S3 via subscription or Firehose for long-term storage.

• Use the Infrequent Access log class for rarely queried logs.
• Metric filters and Embedded Metric Format avoid logging just to count.

Distributed tracing: propagate a trace context (W3C traceparent) across calls and export spans with OpenTelemetry (ADOT collector to X-Ray or another backend). Put the trace ID in logs for correlation.

• Spring Boot 3 integrates Micrometer Tracing and the OpenTelemetry bridge.
• Look at span waterfalls for the slowest hop (often a DB or an external call).

Alert fatigue: replace cause-based thresholds (CPU > 80%) with symptom-based SLO alerts (error rate, latency burn rate), add runbooks, tune or delete noisy alarms, and route non-urgent signals to tickets.

Low CPU with high latency points to waiting: connection pool exhaustion, thread pool saturation, GC pauses, slow downstream calls or EBS/network throttling.

• Check JVM metrics (GC pause, heap, thread states), Hikari active/pending, downstream latencies, and instance network/EBS limits (EBSIOBalance%, network allowance exceeded).
• Take thread dumps and enable GC logging.

Sessions were kept in instance memory. Externalize to ElastiCache (Redis) via Spring Session, or use stateless JWT, and enable connection draining so in-flight requests finish.

Rehost (lift and shift) onto EC2 is the fastest; replatform (move DB to RDS, files to S3, add ASG) gives early gains with modest change; refactor into cloud-native services has the highest payoff and risk. Choose by deadline and value, and iterate (the "6 Rs": rehost, replatform, repurchase, refactor, retire, retain).

Failover swaps DNS to the standby. The JVM may cache the old IP, the pool holds dead connections, and no socket or connect timeouts mean threads block until TCP timeouts.

• Set networkaddress.cache.ttl to a low value (such as 5-30), configure Hikari connectionTimeout, validationTimeout and maxLifetime, set driver socketTimeout and use the AWS Advanced JDBC Wrapper's failover plugin to react faster.
• Retry idempotent transactions.

S3 supports at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix, and scales by prefix. Spread keys across more prefixes (a hash or date-sharded prefix), retry with exponential backoff, and batch small objects.

Apply per-tenant rate limits and quotas at the gateway (API Gateway usage plans or custom token buckets), bulkhead resources (separate thread pools or queues per tier), shard heavy tenants to dedicated cells, and monitor per-tenant metrics.

• Use SQS fair queuing or per-tenant message groups to stop one tenant starving others.

Split the service into independent, identical cells (full stacks with their own data) serving a subset of customers, with a thin router in front. A failure or bad deploy affects only one cell, limiting blast radius, and capacity grows by adding cells.

• Deploy changes to one cell first as a natural canary.
• Cost: operational overhead and rebalancing tenants between cells.

Use OIDC federation with roles trusting only specific repositories and branches via sub claim conditions, scope permissions to the deployment, separate roles per environment, require approvals for production, and prevent pull requests from forks accessing deploy roles.

Encrypt at rest with KMS (customer-managed keys for control and audit) and in transit with TLS; enable CloudTrail (data events for S3 where needed) in all regions with log file validation, send logs to a locked-down separate account, and use AWS Config and Security Hub for continuous compliance.

Quantify instead of refusing: compare the extra cost of a second AZ (a second small instance set, cross-AZ transfer, Multi-AZ DB at about 2x DB cost) with the revenue and reputation loss of downtime against the SLO. Most production systems justify multi-AZ; non-critical internal tools may not.

Production scale hit service quotas or API rate limits (for example EC2 API calls, KMS requests per second, Lambda concurrency, or SES sending limits) that staging never reached.

• Monitor Service Quotas, request increases ahead of launches, cache KMS data keys, and use SDK retries with backoff and jitter.
• Load test at production-like scale.