AWS

A VPC (Virtual Private Cloud) is your logically isolated slice of the AWS network — you pick a private CIDR block (e.g. 10.0.0.0/16, up to 65,536 IPs) and everything you launch lives inside it.

A subnet is a subdivision of that CIDR pinned to one Availability Zone (e.g. 10.0.1.0/24 in us-east-1a). Subnets are the unit of placement and routing:

• Each subnet has one route table — that's what decides where its traffic can go.
• AWS reserves 5 IPs per subnet (network, router, DNS, future, broadcast), so a /24 gives 251 usable.

What interviewers listen for: you know a VPC is regional, a subnet is zonal, and that routing — not naming — defines behavior.

The classic end-to-end question. Hit each layer:

1. DNS — browser/OS cache → recursive resolver → root → TLD → authoritative (Route 53) returns the ALB/CloudFront IP. TTL controls caching.
2. TCP handshake — SYN / SYN-ACK / ACK to port 443.
3. TLS handshake — server cert (ACM), cipher negotiation, session keys. TLS 1.3 does this in 1-RTT.
4. HTTP request — hits CloudFront (edge) or ALB, which terminates TLS, evaluates listener rules, and forwards to a healthy target.
5. App tier — service processes, hits cache (ElastiCache) then DB (RDS/DynamoDB).
6. Response renders; browser fetches static assets from CDN.

Senior signal: mention where latency accumulates (DNS ~20-120ms cold, TLS 1-2 RTT, cross-AZ hops ~1-2ms each) and where you'd put caching.

Nothing on the subnet itself — it's purely the route table.

A subnet is public when its route table has a route 0.0.0.0/0 → Internet Gateway. That's it.

For an instance in that subnet to be reachable it also needs:

• A public IP or Elastic IP attached
• Security group / NACL rules allowing the traffic

A 'private' subnet routes 0.0.0.0/0 to a NAT Gateway (outbound-only) or has no default route at all (fully isolated — common for DB tiers).

Trap answer to avoid: 'a public subnet is one with public IPs.' Auto-assign public IP is a convenience setting, not the definition.

Two senior points:

1. Deploy one NAT GW per AZ — routing cross-AZ to a single NAT GW adds cost and makes that AZ a failure domain for the whole VPC's egress.
2. NAT data-processing charges are a classic hidden cost — S3/DynamoDB traffic through NAT should go through gateway VPC endpoints instead (free).

Security Groups — instance/ENI level, stateful (return traffic auto-allowed), allow rules only, all rules evaluated. Your primary tool: 99% of controls live here.

NACLs — subnet level, stateless (must allow return traffic + ephemeral ports 1024-65535 explicitly), allow and deny rules, evaluated in numbered order.

When NACLs earn their keep:

• Blocking a specific malicious CIDR fast (SGs can't deny)
• Coarse subnet-tier guardrails (e.g. DB subnets never accept traffic from public subnets)
• Compliance frameworks that demand subnet-level controls

Best practice: reference SGs by SG ID, not CIDR — app-sg allows 443 from alb-sg survives IP churn and reads as intent.

IAM answers who can do what on which resource.

• User — long-lived identity with credentials. Modern stance: near-zero IAM users; humans federate via Identity Center (SSO).
• Role — identity with no credentials, assumed via STS for temporary creds (15min-12hr). Used by EC2 instance profiles, Lambda execution roles, cross-account access, IRSA on EKS.
• Policy — JSON document: Effect / Action / Resource / Condition. Managed vs inline; identity-based vs resource-based.

Evaluation logic: explicit Deny always wins → then any Allow → default deny. With SCPs and permission boundaries, the effective permission is the intersection of all layers.

Senior signal: 'roles everywhere, no static keys, least privilege enforced by conditions (aws:SourceVpc, aws:PrincipalOrgID).'

• Standard — hot data, 11 nines durability, ~$0.023/GB-mo.
• Intelligent-Tiering — unknown/shifting access patterns; auto-moves objects between tiers for a tiny monitoring fee. Default choice for most data lakes.
• Standard-IA / One Zone-IA — infrequent access, ~40-50% cheaper, but retrieval fee ($0.01/GB) + 30-day minimum. One Zone = single AZ, use only for re-creatable data.
• Glacier Instant / Flexible / Deep Archive — archives: instant ms access, minutes-hours, or 12-48hr at ~$0.00099/GB-mo respectively. 90-180 day minimums.

The interview trap: IA classes cost more than Standard if data is accessed frequently — retrieval fees dominate. Always cite lifecycle policies driven by S3 Storage Class Analysis, not guesses.

Decision by workload shape, not fashion:

• Lambda — event-driven, spiky, short (<15 min), stateless. Pay per ms; scales to zero. Wrong for steady high-throughput (cost crossover vs Fargate at sustained load) or long connections.
• Containers (ECS/EKS) — microservices, steady traffic, teams shipping many services, need for portability/sidecars. ECS Fargate = lowest ops burden; EKS when you need the K8s ecosystem or run it already.
• EC2 — full OS control, GPUs, licensed software, extreme network/disk tuning, or lift-and-shift legacy.

Cost anchor: a service at constant 50% CPU is usually cheapest on containers with Savings Plans; a cron that runs 30s/hour is Lambda territory.

Senior signal: 'I choose the highest abstraction the workload tolerates — less undifferentiated ops.'

• Region — a geographic cluster (e.g. us-east-1, ap-south-1), fully independent: separate control planes, IAM is global but most services are regional.
• AZ — one or more physically separate datacenters within a region: independent power, cooling, network; connected by <2ms, high-bandwidth private fiber. A region has 3+ AZs (us-east-1 has 6).
• Local Zones / Edge locations — extensions for latency-sensitive workloads / CloudFront POPs (600+).

Design rules that follow:

• Multi-AZ = your default for high availability (survives datacenter failure).
• Multi-region = for disaster recovery and data-residency, at meaningful complexity cost.
• Cross-AZ data transfer costs $0.01/GB each way — architecture decisions have a bill attached.

Note: AZ letters are shuffled per account (us-east-1a ≠ same DC across accounts) — use AZ IDs (use1-az1) for coordination.

Layer 4 (NLB) routes on TCP/UDP — IP + port only. It can't see HTTP.

• Millions of req/s, ultra-low latency (~100µs), static IPs / EIP support
• Preserves source IP by default
• Use for: TCP/UDP protocols, extreme throughput, PrivateLink (NLB is required), gaming, IoT

Layer 7 (ALB) terminates HTTP(S) and routes on content — path, host header, HTTP method, query string, headers.

• Path-based (/api/* → service A), host-based routing, WebSockets, gRPC, HTTP/2
• Native WAF integration, OIDC/Cognito authentication at the LB, fixed responses, weighted target groups (canary)
• Use for: web apps, microservices routing, anything HTTP

Rule of thumb: ALB unless you need raw TCP/UDP, static IPs, source-IP preservation at L4, or PrivateLink — then NLB. They also stack: NLB → ALB target is now supported for static-IP + L7 routing.

Three related but distinct things:

• Gateway endpoints — route-table entries for S3 and DynamoDB only. Free. No ENI. Traffic never leaves the AWS network. Always deploy these — they eliminate NAT data-processing charges for the two heaviest services.
• Interface endpoints — an ENI with a private IP in your subnet fronting an AWS service (or a partner service) via PrivateLink. ~$0.01/hr/AZ + $0.01/GB. Private DNS makes sqs.us-east-1.amazonaws.com resolve to the ENI.
• Endpoint services — the producer side of PrivateLink: you put your own service behind an NLB and expose it to other VPCs/accounts without peering, route overlap issues, or transitive access.

Design decision: consumer needs one AWS service privately → interface endpoint. You're the provider selling/sharing a service across accounts → endpoint service. Full network-level connectivity between VPCs → peering/TGW instead.

Cost trap: dozens of interface endpoints across many VPCs adds up — centralize them in a shared-services VPC with Route 53 private hosted zones, resolve from spokes via TGW.

Options:

1. Site-to-Site VPN — managed IPsec, two tunnels (dual AWS endpoints) per connection, ~1.25 Gbps per tunnel ceiling. BGP or static. Can use ECMP over multiple tunnels on Transit Gateway to aggregate (up to ~50 Gbps).
2. Client VPN — managed OpenVPN for workforce access; per-connection + per-hour billing; auth via AD/SAML/certs.
3. Direct Connect (DX) — dedicated fiber, 1/10/100 Gbps, consistent latency, lower per-GB egress ($0.02 vs $0.09). Weeks to provision.
4. DX + VPN as backup — the standard enterprise pattern: BGP prefers DX, VPN over internet takes over on failure.
5. Accelerated VPN — Site-to-Site over Global Accelerator to reduce internet-path variance.

Decision logic: VPN = fast to stand up (hours), fine for <1 Gbps and tolerance for internet jitter. DX = predictable latency, heavy sustained transfer, compliance. Hybrid answer that scores points: 'two DX locations for resilience, or one DX + VPN failover with actively tested failover, all landing on Transit Gateway.'

Peering — 1:1, non-transitive, free data transfer within the same AZ ($0.01/GB cross-AZ). Fine for a handful of VPCs.

The math kills peering: full mesh of n VPCs needs n(n-1)/2 peerings — 10 VPCs = 45, 50 VPCs = 1,225. Each has its own route-table entries (limit ~125 peerings/VPC) and no transitive routing means no shared egress/inspection.

Transit Gateway — hub-and-spoke, up to 5,000 attachments, transitive routing, multiple route tables for segmentation (prod can't reach dev), inter-region peering, and it's the landing point for DX/VPN.

Cost: ~$0.05/hr per attachment + $0.02/GB processed — you pay for the simplicity.

My rule: ≤5 stable VPCs → peering. Anything resembling a platform (many accounts, shared services, central egress/inspection VPC with GWLB firewalls) → TGW. At very large scale add AWS Cloud WAN or TGW peering across regions, and use RAM to share the TGW across the org.

Follow-up they'll ask: 'two VPCs with overlapping CIDRs?' — peering/TGW both fail; answer is PrivateLink or NAT.

Situation: I owned a checkout service doing ~4K TPS with a 99.95% SLO — every minute down was ~$30K revenue.

Task: survive AZ loss with zero manual action, and have a credible region-loss story.

Action:

• Stateless tier: ASG/EKS across 3 AZs, ALB health checks tuned (5s interval, 2 unhealthy threshold) so bad nodes drain in ~15s. Capacity planned N+1 per AZ — losing one AZ leaves 66% capacity ≥ peak load.
• State: Aurora with a reader in each AZ (failover <30s), ElastiCache multi-AZ with cluster mode; idempotency keys on writes so retries are safe.
• Static stability: no control-plane dependency at failover time — capacity pre-provisioned, not 'launch instances when AZ fails' (everyone tries that simultaneously).
• Multi-region (pilot-light): async Aurora Global Database (RPO <1s typical), Route 53 failover records with health checks, IaC to scale the warm region; quarterly game-day failovers.

Result: survived a real us-east-1 AZ impairment with zero customer impact; measured region failover at 12 minutes RTO vs 15-minute target.

Key line: multi-AZ is for availability, multi-region is for disaster — and an untested failover plan is a rumor, not a plan.

Situation: inherited an AWS estate at ~$460K/month growing 8% MoM with flat traffic — classic drift.

Task: cut ≥25% in two quarters without impacting SLOs, and build controls so it doesn't regrow.

Action (in order of leverage):

1. Visibility first — tagging standard enforced via SCP + Config; Cost Explorer/CUR into dashboards per team; anomaly alerts.
2. Waste (weeks 1-4): orphaned EBS volumes/snapshots, idle load balancers, unattached EIPs, over-provisioned gp2→gp3 (20% cheaper + decoupled IOPS), dev environments auto-stopped nights/weekends (~65% of their hours).
3. Rightsizing: Compute Optimizer + memory metrics; downsized ~40% of instances one size (typical util was 12-18% CPU).
4. Commitments: after rightsizing stabilized, Compute Savings Plans covering 70% of steady-state (66% discount vs on-demand); Spot for CI and batch (90% off).
5. Architecture: S3 Intelligent-Tiering + lifecycle to Glacier; gateway VPC endpoints killed $9K/mo of NAT processing; consolidated 14 NAT gateways; cross-AZ chatter reduced with AZ-aware routing.
6. Guardrails: budgets per account with alerts, cost review in sprint rituals, unit metric (cost per 1K requests) on every service dashboard.

Result: -31% ($143K/mo) in 5 months; unit cost down 38%; growth curve flattened to traffic-correlated.

Order matters: rightsize before committing — Savings Plans lock in your waste.

Layered model — effective permissions are the intersection:

1. SCPs (org level) — outer guardrails: deny leaving allowed regions, deny root usage, deny disabling CloudTrail/GuardDuty, deny IAM user creation. SCPs never grant.
2. Identity Center — humans federate from IdP; permission sets materialize as roles in each account. No IAM users, no long-lived keys.
3. Cross-account roles for workloads — hub account principals assume spoke-account roles via sts:AssumeRole; trust policies pin aws:PrincipalOrgID + ExternalId for third parties.
4. Permission boundaries — for delegated admin: platform team lets app teams create roles, but boundary policy caps what those roles can ever do (can't escalate past the boundary).

Role chaining caveats: chained sessions cap at 1 hour regardless of role max; session tags don't propagate by default (need sts:TagSession); each hop is a CloudTrail event — trace with sts:SourceIdentity.

Interview differentiator: mention IAM Access Analyzer for unintended external access, and that you treat unused permissions as a metric (last-accessed data driving quarterly least-privilege reviews).

Situation: security scanner flagged a bucket with customer exports exposed via a bucket policy added during a debugging session.

Task: contain in minutes, assess blast radius, then make the class of error impossible — not just this instance.

Action:

1. Contain (<15 min): enable bucket-level Block Public Access; verified with Access Analyzer. Didn't delete the policy yet — evidence.
2. Assess: S3 server access logs + CloudTrail data events → determine if any unauthenticated GetObject succeeded during exposure window; inventory affected objects; loop in legal/compliance for notification obligations.
3. Eradicate root cause: the debugging change bypassed review — direct console access in prod.
4. Prevent (the real answer):
   • Account-level Block Public Access in every account, org-wide
   • SCP denying s3:PutBucketPublicAccessBlock changes and public ACL/policy operations
   • Public content served only via CloudFront + OAC, never raw buckets
   • Default encryption (SSE-KMS) + aws:SecureTransport deny policy
   • Config rules + Access Analyzer continuous monitoring, auto-remediation Lambda
   • Console write access removed; changes only via reviewed IaC

Result: exposure window 40 minutes, logs showed zero external reads; org-wide guardrails shipped in a week; repeat findings dropped to zero across 50 accounts.

Point that scores: 'block public access at the account and org level — bucket-level settings are one typo from failure.'

Choose by access pattern, consistency needs, and ops appetite:

• RDS — standard engines (Postgres/MySQL/etc.) when you need engine-specific features, extensions, or lift-and-shift. Ceiling: vertical scaling + up to 15 read replicas; writes are single-node bound. Multi-AZ = availability, not scale.
• Aurora — same wire protocols, storage decoupled: 6-way replicated across 3 AZs, auto-grows to 128TB, replica lag 10-20ms, failover <30s, up to 15 readers off shared storage. Aurora Serverless v2 for spiky loads; Global Database for cross-region (<1s RPO). Still single-writer (writes bound by biggest instance) unless you accept multi-master trade-offs.
• DynamoDB — key-value/document at any scale: single-digit-ms at millions of RPS, no ops. The price: you must know your access patterns up front (single-table design), 400KB item limit, no ad-hoc joins; ACID transactions exist but cost 2x capacity.

My heuristic: relational model + moderate scale → Aurora Postgres. Known key-based patterns + massive/spiky scale or per-request billing → DynamoDB. Anything needing rich ad-hoc queries stays relational; don't force analytics into either — that's what the warehouse is for.

Follow-up to expect: 'writes exceed one Aurora writer?' — shard by tenant, offload with queues, or re-model hot paths onto DynamoDB.

Situation: flash-sale traffic drove ProvisionedThroughputExceededException spikes; one product's counter item took ~30% of all writes.

Task: stop the throttling without 10x-ing table capacity for one hot key (adaptive capacity caps a single partition at ~3,000 RCU / 1,000 WCU — table-level capacity can't fix a single-key ceiling).

Action:

1. Diagnose: CloudWatch Contributor Insights → top-N hottest keys; confirmed one PK dominated.
2. Write sharding: hot counter split across N suffixed keys (product#123#0..9), writes randomized, reads aggregate the shards. N sized to peak WCU / 1,000 with headroom.
3. Read path: DAX in front for read-heavy items (µs latency, absorbed 85% of reads); item TTLs sanity-checked.
4. Buffering: non-transactional writes (view counts) moved to Kinesis → aggregated batch writes every second — converted spiky writes into smooth ones.
5. Switched table to on-demand during sale events (handles sudden 2x of any previous peak instantly), reviewed back to provisioned + autoscaling for steady state.

Result: zero throttles at next event at 4x traffic; table cost up only 12%.

The senior insight: partition design is a data-modeling problem, not a capacity knob — high-cardinality partition keys with even access distribution, and design for your top-3 access patterns before writing a line of code.

Policies: simple, weighted (canary/gradual migration), latency-based (route to lowest-latency region), failover (primary/secondary with health checks), geolocation (compliance/localization by user location), geoproximity (bias dial), multivalue (up to 8 healthy records, poor-man's LB), IP-based (per-CIDR).

Active-passive design:

• Failover policy: primary record → region A ALB, secondary → region B.
• Health checks hit a /health endpoint that reflects deep health (DB reachable, dependencies OK) — not just a 200 from the LB. Calculated health checks aggregate child checks to avoid failing over on one flapping component.
• Low TTL (30-60s) on failover records — but know that some resolvers ignore TTLs, so DNS failover is minutes for full convergence, never seconds.
• Health check evaluates from multiple AWS regions (default 3-of-8 checkers must fail).

Gotchas that show seniority:

1. Failback control — automatic failback can flap; often you want manual failback after root-causing.
2. Standby must be continuously verified (synthetic traffic) or it won't work when needed.
3. For faster, TTL-independent failover consider Global Accelerator (anycast IPs, ~30s traffic shift, no DNS caching problem).
4. Route 53 ARC (Application Recovery Controller) for audited, one-button regional failover with readiness checks.

The settings that actually cause incidents:

• Deregistration delay (draining) — default 300s. During deploys a target is removed but in-flight requests get this long to finish. Too high = slow deploys; too low = 5xx on long requests. Match it to your p99 request duration + margin (e.g. 30s for a 5s-p99 API).
• Sticky sessions — ALB cookie (AWSALB) pins a client to a target. It fights autoscaling (new instances get no traffic until cookies expire) and breaks on target loss. Correct fix is externalizing session state (ElastiCache); stickiness is a legacy crutch.
• Cross-zone LB — always on for ALB (free). On NLB it's off by default and billed — leaving it off with uneven target counts per AZ causes hot instances.
• Idle timeout — default 60s. Must be shorter than your app's keep-alive timeout, or the ALB reuses a connection the app just closed → intermittent 502s. This one line explains most 'random 502' tickets.
• Health checks — healthy/unhealthy thresholds × interval = detection time. 30s interval × 3 = 90s of traffic to a dead node; tier-1 services want 5-10s intervals.
• Slow start — ramp traffic to new targets over N seconds; underused for JIT-warmed runtimes (JVM).

Also know: ALB scales itself — pre-warm via support for known thundering-herd events; 1024 max request header size can bite SSO-heavy apps.

Goals: latency, offload, and a single security choke point.

Architecture:

• CloudFront in front of everything — static (S3 + OAC, bucket never public) and dynamic/API traffic. Even uncacheable requests benefit: TLS terminates at 600+ POPs and rides the AWS backbone (persistent connections, less TCP/TLS setup) — typically 20-40% TTFB improvement for far users.
• Caching strategy: cache policies with explicit key design (headers/cookies/query strings minimized — every key dimension fragments hit ratio). Static: long TTL + versioned filenames. API GETs: short TTL (10-60s) + stale-while-revalidate absorbs thundering herds.
• WAF on the distribution: managed rule groups (Core, Known Bad Inputs, IP reputation) + rate-based rules per IP + bot control on login/checkout. Count mode first, then block — never blind-enable.
• Shield: Standard is free/automatic (L3/4). Shield Advanced when you need SRT access, cost-protection during attacks, and L7 auto-mitigation.
• Origin protection: origins accept traffic only from CloudFront — custom header verified at ALB, or the managed prefix list in SGs. Otherwise attackers bypass your entire edge.
• Functions: CloudFront Functions (µs, viewer events) for redirects/header rewrites; Lambda@Edge for auth-at-edge (JWT verification before the request crosses the ocean).

Metric that matters: cache hit ratio — every point of hit ratio is origin capacity and cost you don't pay.

Situation: cluster on VPC CNI in /24 subnets; pods stuck Pending with failed to assign an IP address, nodes at 40% CPU.

Why: with the AWS VPC CNI every pod gets a real VPC IP from the subnet. An m5.large caps at 29 pods (3 ENIs × 10 IPs − 1); more importantly three /24 subnets ≈ 750 usable IPs — a few hundred pods plus nodes/LBs exhausts them. CPU headroom is irrelevant; you're out of addresses.

Action:

1. Immediate: enabled prefix delegation (ENABLE_PREFIX_DELEGATION=true) — each ENI slot holds a /28 (16 IPs), raising max pods per m5.large to 110 and cutting EC2 API churn.
2. Structural: added secondary CIDR 100.64.0.0/16 (CG-NAT space, no overlap with corporate ranges) to the VPC with large dedicated pod subnets per AZ via custom networking (ENIConfig) — pods draw from the big range, nodes stay in routable space.
3. Hygiene: tuned WARM_PREFIX_TARGET to stop over-reserving; capacity alarm on subnet free-IP count; documented IP math in the platform runbook (pods × growth × 2, per AZ).

Result: scheduling failures gone; cluster headroom went from ~200 pods to ~15K without renumbering.

Senior point: EKS IP planning is a day-0 decision — retrofitting CIDRs on a live cluster is painful; plan /16-scale pod space or use IPv6 clusters for greenfield.

The real question is who operates the platform and what does the ecosystem cost you.

ECS (Fargate) case:

• Near-zero platform team — no control plane, no upgrade treadmill, no CNI/CSI/IRSA plumbing
• Deep, native AWS integration (ALB, IAM per task, CloudWatch) with far fewer moving parts
• Right when: AWS-only strategy, <100 services, no existing K8s expertise, platform team ≤2 people

EKS case:

• The K8s ecosystem is the product: Helm charts, operators, ArgoCD, Karpenter, service mesh, vendor charts ship K8s-first
• Portability/multi-cloud posture and hiring pool
• Right when: you already run K8s, need CRD-based platform abstractions, or vendor/OSS software assumes it

The honest cost: EKS is $73/mo/cluster but the real cost is a 2-4 person platform team — upgrades every ~10 months (forced), add-on lifecycle, security patching of the node/agent stack. That's $500K+/yr of engineering.

My call at 40 teams: if there's no existing K8s estate and no CRD-shaped requirements — ECS Fargate, spend the saved headcount on developer experience. If K8s is already in the building or the roadmap needs operators/mesh — EKS with a real platform team, one shared multi-tenant cluster pattern with namespace isolation, not 40 clusters.

What they're testing: whether you frame it as TCO + team topology, not a checkbox matrix.

Cold starts: new execution environment = runtime init + your init code. Typically 100-400ms (Node/Python), 1s+ for JVM, +2-4s historically for VPC (now ~hundreds of ms with Hyperplane ENIs). p50 is fine; the pain is p99 on spiky, latency-sensitive paths.

Mitigations in order of preference: trim deps & init code (lazy clients), right-size memory (CPU scales with it — 1769MB = 1 vCPU; more memory is often cheaper via shorter duration), provisioned concurrency for the latency-critical alias (pre-warmed environments), SnapStart for Java.

Concurrency: account soft limit ~1,000 concurrent (raisable to tens of thousands); reserved concurrency carves out capacity and caps a function (protects downstream DBs); burst ramps in 500-3,000/min region-dependent. Per-event-source math differs: SQS scales pollers, Kinesis = one concurrent per shard per parallelization factor.

Where Lambda is wrong:

• Steady high throughput — at sustained load, Fargate/EC2 is 3-10x cheaper (the crossover is roughly 'busy > ~30-40% of the time')
• Long-lived connections (WebSockets at scale), >15 min jobs, GPU work
• Heavy per-invoke init that can't amortize
• When 'function sprawl' replaces service boundaries — 400 functions with no ownership model is an operational anti-pattern

Senior signal: you talk about Lambda cost/latency envelopes and downstream-protection (concurrency as a bulkhead), not just 'serverless is cheap.'

By semantics:

• SQS — point-to-point work queue: one consumer group, each message processed once. Standard (at-least-once, best-effort order, ~unlimited TPS) vs FIFO (exactly-once dedup, ordered per group, 300-3,000 TPS). Retention up to 14 days. The default buffer between services.
• SNS — fan-out pub/sub, push-based: one event → many subscribers (SQS queues, Lambda, HTTP). No replay, no retention (dead-letter to SQS). SNS→SQS fan-out is the classic durable pattern.
• EventBridge — event bus with content-based routing: rules match on payload fields, 100+ SaaS/AWS sources, schema registry, archive + replay, scheduler, pipes. Choose for event-driven architectures between many services/accounts where routing logic lives in the bus, not in code. Watch: 4K TPS default soft limits, higher latency (0.5s) than SNS.
• Kinesis Data Streams — ordered, replayable stream: shard-based (1MB/s in, 2MB/s out per shard), multiple independent consumers re-reading the same data, retention to 365 days. For analytics pipelines, CDC, clickstreams — when order + replay + multiple readers matter. (Kafka/MSK when you need its ecosystem or >MB messages via tiering.)

Heuristics: decouple two services → SQS. Broadcast → SNS or EventBridge (routing complexity decides). Stream processing/replay → Kinesis.

The follow-up: idempotency — at-least-once delivery is universal here; consumers must dedupe (idempotency keys, conditional writes) or FIFO/exactly-once semantics must be paid for explicitly.

The spectrum (cost ↑, RTO/RPO ↓):

1. Backup & restore — RPO hours (last backup), RTO 4-24h+. Cheapest. For non-critical/internal systems.
2. Pilot light — data replicated continuously (Aurora Global, S3 CRR, DynamoDB global tables); core infra defined in IaC but scaled to zero/minimal. RPO seconds-minutes, RTO 30min-2h (scale-up + cutover).
3. Warm standby — full stack running scaled-down (say 10-20% capacity), continuously taking synthetic or shadow traffic. RPO seconds, RTO minutes.
4. Active-active — both regions serve production behind latency/weighted routing; failover = weight shift. RPO ~0, RTO seconds-minutes. Cost ~1.7-2x, plus the engineering cost: data conflict strategy, regional isolation, request routing affinity.

For payments: regulators and revenue math (say $50K/min) justify warm standby minimum, active-active for the authorization path. Design notes:

• Ledger on a strongly-consistent primary with async cross-region replica — know your RPO honestly (async = seconds of potential loss; reconcile via idempotent replay from a durable event log)
• Route 53 ARC for audited failover; static stability (standby capacity pre-provisioned)
• Game days quarterly — measured RTO, not aspirational

The trap: claiming active-active without a data-conflict answer. If two regions can write the same account balance, you've traded an availability problem for a correctness problem.

Why multi-account at all: accounts are the strongest isolation boundary AWS offers — blast radius, per-team billing, per-env service quotas.

OU structure:

Root
├── Security OU: log-archive, security-tooling (GuardDuty/SecHub delegated admin)
├── Infrastructure OU: network (TGW, DX, egress), shared-services
├── Workloads OU: prod / non-prod (per team or service)
├── Sandbox OU: developer experiments, budget-capped, wide-open regions denied
└── Suspended OU: quarantine (deny-all SCP)

SCP layers: root-user deny, region allowlist, deny CloudTrail/Config/GuardDuty tampering, deny IAM-user creation, prod OU: deny public S3 controls changes. SCPs are guardrails — permissions still come from IAM inside accounts.

Networking: hub-and-spoke — TGW in the network account shared via RAM; centralized egress VPC (NAT + inspection with GWLB firewalls); no VPC in workload accounts talks to the internet directly. IPAM allocates non-overlapping CIDRs per account automatically.

Account vending: Control Tower Account Factory (or AFT/org-formation in code) — a new account lands with baseline in <1h: CloudTrail→log-archive, Config, security-hub enrollment, standard VPC (or none), SSO permission sets, budget alarms, tags.

Identity: Identity Center, no IAM users anywhere, break-glass roles monitored.

Senior point: the landing zone is a product with a roadmap — teams onboard themselves via vending, and drift is detected (Config aggregator) rather than assumed away.

KMS mechanics you must be able to explain: KMS keys never leave HSMs and encrypt only 4KB directly — everything real uses envelope encryption: KMS generates a data key, you encrypt data with the data key locally, store the encrypted data key alongside the data, and call KMS only to decrypt the data key. This is what S3/EBS/RDS do under the hood. Key policies (resource-based) are the root of trust; grants for temporary service access; cross-account = key policy + IAM on both sides.

Secrets Manager vs Parameter Store:

• Parameter Store — free (standard tier), config + non-rotating secrets (SecureString), 4KB/8KB, no native rotation.
• Secrets Manager — $0.40/secret/mo + API cost, native rotation via Lambda (managed rotators for RDS/Redshift/DocDB), cross-region replication, resource policies for cross-account.
• Heuristic: config and static values → Parameter Store; anything with a lifecycle (DB creds, API keys) → Secrets Manager.

Zero-downtime rotation — the part that separates seniors:

1. Two-user (alternating) strategy for databases: rotate user B while app uses user A, swap on next fetch — no single moment where creds are invalid.
2. Apps must fetch at runtime with caching + TTL (client-side caching lib or sidecar), never bake secrets at deploy time; on auth failure, force-refresh once before alerting.
3. For EKS: IRSA + Secrets Store CSI driver / External Secrets Operator — pods get IAM identity, secrets sync as volumes/K8s secrets with rotation reflected.

Anti-patterns to name: secrets in env vars in plaintext task defs, one god-secret shared by 30 services, rotation that's configured but has never been exercised.

Three pillars, but opinionated:

• Metrics — RED per service (rate, errors, duration histograms) + USE for infra. CloudWatch EMF or Prometheus (AMP) via ADOT collectors. Every service ships a standard dashboard from a template — no bespoke snowflakes.
• Traces — OpenTelemetry SDKs → X-Ray or vendor. Propagate context through SQS/EventBridge (trace headers in message attributes) or your traces die at every queue. Tail-based sampling: keep 100% of errors + slow requests, 1-5% of happy path.
• Logs — structured JSON, one schema org-wide (trace_id, service, level). CloudWatch Logs with subscription filters → S3 for cheap retention; Logs Insights for query.

Alerting philosophy — SLO-based, not threshold-soup:

• Each service defines availability + latency SLOs; alerts fire on error-budget burn rate (fast burn: 14.4x over 1h; slow burn: 6x over 6h) — pages correlate with user pain, not CPU spikes.
• Everything else (disk, memory, queue depth) is a ticket, not a page. Target: <2 pages/on-call/week or you fix the alerts.

Cost control (observability is routinely 10-15% of the infra bill):

• Log levels enforced (INFO in prod, no debug firehoses); sampling on high-cardinality metrics; metric cardinality budgets per team (a single unbounded label like user_id can 100x a bill)
• Retention tiers: hot 2-4 weeks, archive to S3/Athena

Senior close: observability's success metric is MTTD/MTTR trend, not gigabytes collected.

Layered defense, outside-in:

1. Edge (CloudFront + WAF): rate-based rules (e.g. 2,000 req/5min per IP) kill dumb floods before they cost you compute; bot control on expensive endpoints; IP reputation lists.
2. API Gateway: account-level steady-state + burst limits (token bucket, default 10K rps/5K burst); usage plans + API keys for per-tenant quotas (10 rps free tier, 100 rps paid, monthly caps). Returns 429 with Retry-After. (Note: usage-plan throttling is best effort per node — treat as coarse.)
3. Application layer — the precise tier: token bucket per tenant in ElastiCache Redis (atomic Lua: refill + consume in one round trip, ~1ms). This is where fairness lives: per-tenant, per-endpoint costs (a search costs 10 tokens, a read costs 1).
4. Downstream protection: Lambda reserved concurrency / worker pool caps as bulkheads so even admitted traffic can't crush the database; SQS buffering for write paths — absorb the spike, process at sustainable rate.

Design details that score:

• Graceful degradation order: shed anonymous before authenticated, reads before writes, batch before interactive
• Standard response contract: 429 + Retry-After + rate-limit headers (X-RateLimit-Remaining) so clients can behave
• Client SDKs with exponential backoff + jitter — without jitter, synchronized retries recreate the spike
• Rate-limit metrics per tenant feed abuse detection and capacity planning

The trap question: 'why not just autoscale?' — you can't autoscale faster than a spike arrives, and unbounded scaling turns a DoS into a Denial-of-Wallet.

The bill nobody models up front. The big five:

1. NAT Gateway processing — $0.045/GB on top of transfer. S3 backups through NAT is the classic five-figure mistake → gateway VPC endpoints (free) for S3/DynamoDB; interface endpoints for chatty services (ECR, CloudWatch, STS) when volume justifies $0.01/GB.
2. Cross-AZ — $0.01/GB each direction ($0.02 total). Microservice meshes and Kafka replication generate terabytes of east-west chatter → topology-aware routing (K8s topologyAwareHints), AZ-affinity for consumer→broker (Kafka rack awareness / MSK fetch-from-closest-replica), co-locate chatty pairs.
3. Internet egress — $0.09/GB (first ~10TB tier) → serve through CloudFront (cheaper per GB, free origin→CloudFront transfer), compress (gzip/brotli — 70-90% on JSON), and question payloads (field selection, pagination).
4. Cross-region replication — $0.02/GB inter-region; DR and global tables are recurring line items — replicate what you need, not whole buckets by reflex.
5. Same-region but via public IPs — traffic between instances using public IPs pays internet rates even in-region; keep private paths private.

Process fix: Cost Explorer grouped by usage type + VPC Flow Logs to Athena to attribute flows to teams. Put data-transfer cost per service on dashboards; it's invisible until someone looks, and it's often 15-25% of the bill.

One-liner that lands: 'compute you can rightsize later; data-transfer patterns you have to architect — they're expensive to retrofit.'

Blue/green — two full environments; cut traffic over at once (or 90/10 then 100).

• Implementations: ECS + CodeDeploy (managed B/G with test-listener validation hooks), ALB weighted target groups, Route 53 weighted records (beware client DNS caching), Lambda alias shifting.
• Strengths: instant, clean rollback (flip back); whole-environment validation before real traffic; simple mental model.
• Weaknesses: 2x capacity during deploys; database schema is the hard part — schema must be forward/backward compatible (expand-migrate-contract) since both colors share the DB.

Canary — one environment, shift a slice (1% → 10% → 50% → 100%) with automated analysis gating each step.

• Implementations: ALB weighted TGs, App Mesh/service mesh weights, CodeDeploy canary configs, Argo Rollouts on EKS (analysis templates against CloudWatch/Prometheus).
• Strengths: real-traffic validation with bounded blast radius (1% of users, not 100%); catches issues synthetic tests miss.
• Weaknesses: needs real observability — canary without automated metric comparison (error rate, p99, business KPIs vs baseline) is theater; slow; version skew must be handled (APIs, cache/serialization compatibility).

When each is wrong: canary is wrong for tiny-traffic services (1% = 3 requests/hour proves nothing — use B/G) and for breaking schema changes. B/G is wrong when the failure mode only appears under sustained real load, and expensive for giant stateful fleets.

Non-negotiable either way: rollback is automated on SLO breach, not a human decision at 3am.

The performance model: S3 supports 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD per second per prefix, and scales automatically across prefixes (scaling can take minutes and briefly return 503s — retry with backoff). 'Prefix' is any key substring — you get parallelism by key design.

1M objects/hour ≈ 278 PUT/s average — one prefix handles it, but bursts and growth won't. Design:

1. Key naming: avoid monotonic keys (2026-07-09-12-00-01-... funnels writes to one hot prefix). Prefix with a hash shard: {0-f}{0-f}/date/... → 256 prefixes ≈ 896K PUT/s ceiling. If analytics needs date-ordering, keep date after the shard.
2. Multipart upload for anything >100MB (required >5GB): parts up to 10,000 × 5GB, uploaded in parallel — bandwidth scales with parallelism. Tune part size (~64-128MB) for throughput vs retry cost; abort incomplete multiparts via lifecycle rule (invisible storage cost otherwise).
3. Reads: byte-range GETs parallelize large-object reads; CloudFront/caching in front of hot objects; S3 Select is gone, so columnar formats + Athena for query-in-place.
4. Batch small objects: 1M tiny objects/hour is PUT-cost-dominated ($0.005/1K) — aggregate at the producer (Kinesis Firehose buffering into ~128MB Parquet files) → 100x fewer requests, better analytics scans.
5. Consistency: S3 is strongly consistent (since 2020) — list-after-write is reliable; but design event-driven (S3 → EventBridge) rather than LIST-polling, which is slow and costs at scale.

Senior close: at this volume, request cost and file-size distribution matter more than raw bandwidth — 'fewer, bigger, columnar' wins.

What breaks: overlapping CIDRs can't peer or attach to the same TGW route table cleanly; you end up NATing between your own VPCs, PrivateLink-ing everything, or renumbering live VPCs — renumbering means rebuilding (VPC CIDRs are immutable primary).

The plan:

1. Carve a dedicated supernet — e.g. 10.0.0.0/8 split by region: 10.0-31.x us-east-1, 10.32-63.x eu-west-1, etc. Regional blocks make TGW/firewall route summarization trivial (one route per region, not hundreds).
2. Size per account/VPC: standard workload VPC = /20 (4K IPs) with subnets: 3× /23 private (app), 3× /26 public (LBs/NAT only — public subnets should be tiny), 3× /24 data. Bigger for EKS-heavy accounts, or:
3. EKS pods on non-routable space — secondary CIDR from 100.64.0.0/10 (CG-NAT) for pod IPs via custom networking; pods rarely need corporate routability, and this stops K8s from eating your /8.
4. Reserve deliberately: keep ~30-40% of the supernet unallocated for acquisitions, new regions, and mistakes. Coordinate with corporate/on-prem ranges first — the collision that hurts most is with the datacenter you'll connect via DX.
5. Automate with IPAM — AWS VPC IPAM (or NetBox) as source of truth: pools per region/environment, auto-allocation in account vending, overlap detection, utilization alerts at 70%.

Senior line: 'IP planning is one of the few genuinely irreversible decisions in cloud — I spend real design time on it and automate allocation from day one so humans never hand out CIDRs from a spreadsheet.'

STS is the credential mint: AssumeRole, AssumeRoleWithWebIdentity, AssumeRoleWithSAML return temporary creds (access key + secret + session token) with 15min-12h lifetime. Everything modern is built on it.

The chain for each compute platform:

• EC2 instance profile: instance metadata service serves auto-rotating role creds; SDKs pick them up via the default provider chain. Enforce IMDSv2 (session-token hops=1) — IMDSv1 + SSRF is the canonical credential-theft path (Capital One).
• ECS task roles: per-task (not per-instance) roles via the task metadata endpoint — two services on one host get different identities.
• EKS — IRSA: cluster's OIDC provider is registered with IAM; a ServiceAccount annotation maps to a role; the pod gets a projected JWT which the SDK exchanges via AssumeRoleWithWebIdentity. Trust policy pins the OIDC issuer + sub (namespace:serviceaccount) — pod-level identity, no node-role sharing. (EKS Pod Identity is the newer, simpler alternative — same goal, less OIDC ceremony.)
• Outside AWS — IAM Roles Anywhere (X.509) or GitHub OIDC for CI: the CI job presents its OIDC token, trust policy pins repo/branch — no deploy keys in CI secrets.

Why no long-lived keys: they don't expire, they leak (repos, laptops, logs), they're unattributable when shared, and rotation is a manual promise. Temporary creds bound to a verifiable identity + CloudTrail sourceIdentity give you expiry, attribution, and revocation for free.

Enforcement: SCP denying iam:CreateAccessKey, credential reports audited, exceptions time-boxed with a written path to zero.

Kinesis Data Streams:

• Fully managed, no brokers to run; on-demand mode autoscales; pay per shard-hour + PUT payload
• Ceilings: 1MB/s in, 2MB/s out per shard (fan-out consumers get 2MB/s each with enhanced fan-out); 1MB record max; retention ≤365 days; resharding is an operation you manage (on-demand mode hides it)
• Ecosystem: native Lambda triggers, Firehose delivery to S3/warehouse, KCL checkpointing in DynamoDB

MSK (managed Kafka):

• The Kafka ecosystem is the point: Kafka Connect + Debezium for CDC, exactly-once semantics, transactions, compacted topics, ksqlDB/Flink, schema registry, MirrorMaker; huge hiring pool
• Still real ops even 'managed': broker sizing, partition strategy, rebalancing, version upgrades, storage scaling (MSK Serverless trades limits for less ops)
• Better for: high partition counts, >1MB messages, long-lived compacted state, multi-team platform semantics

For CDC + analytics specifically: CDC pipelines lean Kafka — Debezium is the de-facto CDC standard, compacted topics naturally model table state, and exactly-once into downstream stores matters for correctness. If the team is small and the need is 'events → S3/warehouse + some Lambda consumers', Kinesis + Firehose is dramatically less to operate and DMS can handle basic CDC.

My decision rule: no dedicated streaming-platform owner → Kinesis. CDC correctness requirements, Kafka-ecosystem tooling, or multi-team stream platform ambitions → MSK, with a named owning team and capacity/partition standards from day one.

Situation: datacenter lease expiring — a hard date; 300 VMs across ~60 applications, typical mix: 20% modern, 60% standard 3-tier, 20% legacy/unknown-owner.

Task: migrate within 12 months without business disruption, and avoid recreating the datacenter's problems at cloud prices.

Action:

1. Discover & assess (month 1-2): Application Discovery Service + interviews → dependency map (the real one, from network flows, not the wiki); classify each app by one of the 7 Rs — retire (we found 11% of VMs served nothing), retain (2 mainframe-adjacent), rehost, replatform, repurchase (2 apps → SaaS), refactor (only 3 apps where business case justified it during migration).
2. Foundation first (month 1-3, parallel): landing zone (accounts, SSO, TGW, DX), security baseline, migration factory tooling (MGN for rehost), and the wave plan — apps grouped by dependency cluster, not org chart.
3. Waves (month 3-11): wave 1 = low-risk internal apps to prove the factory (5 apps); then 2-week waves of 8-12 apps. Per app: replicate via MGN → test in isolated VPC → cutover window with DNS flip + rollback plan → 2-week hypercare. Databases: DMS with CDC for near-zero-downtime cutovers of the critical ones.
4. Quick replatforms only where cheap: Windows/SQL licensing right-sizing, load balancers → ALB, cron VMs → EventBridge+Lambda. Everything else: migrate first, modernize with data later.

Result: 289 VMs disposed (34 retired outright), lease exited on time, ~22% run-rate reduction from retirement + rightsizing alone — before any modernization.

Mistakes avoided (say these): refactor-everything-in-flight (schedule killer), migrating unknown dependencies alphabetically, lift-and-shift pricing surprise (commit Savings Plans only after waves stabilize), and skipping the retire conversation — the cheapest VM to migrate is the one you delete.

Situation: GuardDuty flags that credentials issued to an EC2 instance role are being used from an IP outside AWS — the signature of metadata-service credential theft (typically via SSRF in an app on that instance).

Task: contain fast without destroying evidence, scope the damage, close the hole.

Action:

1. Contain (first 15 min):
   • Revoke the stolen session: attach an inline deny-all with aws:TokenIssueTime condition to the role ('revoke active sessions') — instances pick up fresh creds, the attacker's stolen session dies
   • Isolate the instance: swap SG to a no-ingress/no-egress forensics SG, detach from ASG (don't terminate — evidence), snapshot EBS + capture memory if tooling exists
2. Scope (hours): CloudTrail for every call by that role's session: LookupEvents on the access key ID → what did they read/change? S3 data events, IAM changes (persistence attempts — new users/keys/roles?), any AssumeRole pivots. Athena over CloudTrail for the full window.
3. Eradicate root cause: app had an SSRF; instance allowed IMDSv1. Fixes: force IMDSv2 with hop limit 1 fleet-wide (SCP + launch-template enforcement), patch the SSRF, WAF rule for metadata-IP patterns as belt-and-braces.
4. Harden systemically: least-privilege pass on the role (it had s3:* — why?), GuardDuty auto-response (EventBridge → Lambda quarantine for this finding type), tabletop the scenario for other teams.

Result: stolen session revoked within 18 minutes of the finding; CloudTrail showed reads of one non-sensitive bucket, no persistence; IMDSv2 enforced across 900 instances that week.

What they're testing: do you know the specific mechanics — session revocation vs key rotation, IMDSv2's session-token defense, and CloudTrail as the scoping tool.

Stage 1 — 1K users (~10 rps): monolith on 2× small instances behind an ALB, RDS Postgres multi-AZ, S3+CloudFront for static. Boring is correct. Skip: microservices, K8s, caching layers — complexity you don't need yet. Do from day one: IaC, CI/CD, backups tested, structured logs, and stateless app tier (sessions external) — that's what makes every later stage possible.

Stage 2 — 10-50K (~100-500 rps): first real bottleneck is always the database read path. Add ElastiCache (cache-aside on hot reads, ~80%+ hit rate), read replicas for reporting, connection pooling (RDS Proxy). ASG on request-count-per-target. CDN caching for API GETs. Trigger metrics: DB CPU >60% sustained, p99 creeping past SLO.

Stage 3 — 100-300K (~1-3K rps): the monolith's deploy contention hurts before its runtime does — extract the 2-3 hottest/most-independently-changing domains into services (not 30 microservices). Async everything non-interactive: SQS between tiers, EventBridge for domain events. Writes start hurting → Aurora, consider DynamoDB for hot key-value paths (sessions, carts). Real observability (tracing across services) becomes mandatory, not nice-to-have.

Stage 4 — 1M (~10K rps, spiky): cell-based or at least AZ-static-stable architecture; shard the write path (by tenant/user hash) or move it to DynamoDB; search traffic → OpenSearch; heavy reads → CQRS-style read models. Load-test to 2x projected peak; game days; per-service SLOs with error budgets gating feature velocity.

The senior framing: each transition is triggered by a measured bottleneck, not anticipation — premature stage-4 architecture at stage-1 scale is how startups die of complexity. Cost per 1K requests should fall at every stage; if it rises, the architecture is wrong.

The pattern: all outbound internet traffic from 50+ workload VPCs routes via TGW to one egress VPC: TGW → GWLB (Gateway Load Balancer) fronting a firewall fleet (Palo Alto/Fortinet appliances or AWS Network Firewall) → NAT gateways → IGW. Return traffic follows the same path (GWLB's GENEVE encapsulation preserves flow symmetry).

Why centralize:

1. One inspection point — IDS/IPS, TLS inspection where mandated, FQDN/domain allowlisting for compliance (PCI/HIPAA egress-control requirements)
2. Fewer NAT gateways — 50 VPCs × 3 AZs = 150 NAT GWs (~$4.8K/mo idle) collapses to 3; at low per-VPC volume this pays for the TGW
3. Consistent policy — SCP denies IGW creation in workload accounts; exfiltration requires beating one audited chokepoint, not finding one forgotten VPC

The honest costs: TGW processing $0.02/GB on all egress + firewall appliance licensing/compute; added latency (~1-2ms); the egress VPC is now tier-0 infrastructure — its failure is everyone's failure, so 3-AZ firewall fleets, health-checked GWLB targets, and a tested bypass runbook are mandatory. High-volume flows (multi-TB S3 sync) should bypass via VPC endpoints — inspecting AWS-service traffic through the firewall is cost without benefit.

When it's overkill: <10 VPCs, no regulatory egress requirements, cloud-native workloads talking mostly to AWS services (endpoints cover it) — per-VPC NAT with flow logs + GuardDuty is simpler and cheaper.

Senior signal: you name the flow-symmetry problem GWLB solves, the endpoint-bypass optimization, and the fact that centralized egress creates a new tier-0 dependency you must engineer for.

The architecture: Aurora's storage is a purpose-built distributed layer — data in 10GB protection groups, 6 copies across 3 AZs, quorum writes (4/6) and reads (3/6). Compute nodes only ship redo log records to storage (no full-page writes, no checkpointing from compute); storage applies the log continuously. Crash recovery is near-instant because there's no redo replay on the writer.

What it buys:

• Failover <30s typically: replicas share the same storage volume — promotion needs no data catch-up (vs RDS standby replay)
• 15 read replicas at ~10-20ms lag (they read the shared volume; lag is cache-invalidation, not replication)
• Fast clones (copy-on-write) — production-size test databases in minutes at delta-only cost; Backtrack rewinds without restore
• Storage auto-grows to 128TB; I/O-Optimized tier flips the cost model for I/O-heavy workloads (no per-I/O charge)
• Global Database: storage-level cross-region replication, typically <1s lag, managed failover

Sharp edges to name:

1. Still single-writer — write throughput is bound by the largest instance class; horizontal write scale means sharding above Aurora or rethinking the hot path
2. Replica lag is low but not zero — read-after-write on replicas still needs session pinning or explicit consistency handling
3. Cost model: standard tier bills per million I/Os — an I/O-heavy workload can cost multiples of the instance price (measure, then pick I/O-Optimized)
4. Serverless v2 scales in ~0.5 ACU steps quickly, but cold minimum capacity and per-ACU pricing make 'serverless = cheap' false at steady load
5. Failover is fast but connections still break — apps need retry/reconnect logic and RDS Proxy smooths connection storms

Interview one-liner: 'Aurora moved the durability problem from the database process into a distributed log-structured storage service — that's why failover, clones, and replicas are cheap, and why writes still aren't horizontal.'

The pillars — operational excellence, security, reliability, performance efficiency, cost optimization, plus sustainability. The tool is a questionnaire; the value is the argument it forces.

How I run it so it isn't theater:

1. Scope one workload (not 'the platform'), with the team that owns it in the room — architect, senior eng, ops owner. 2×2-hour sessions beat an all-day slog.
2. Evidence, not vibes: every 'yes we do that' needs a pointer — the dashboard, the runbook, the last game-day report. 'We could' = 'we don't.'
3. Prioritize HRIs (high-risk issues): the output is a ranked backlog with owners and dates, reviewed monthly — not a PDF that dies in a drive.

What each pillar reliably surfaces in real orgs:

• Reliability: single-AZ NAT/database 'temporary' choices from 2 years ago; DR plans never executed; missing backpressure between services
• Security: IAM wildcards, secrets in env vars, no data-classification tiering (everything encrypted the same = nothing prioritized)
• Cost: unattributed shared accounts, no unit metrics, Savings-Plan coverage bought before rightsizing
• Ops: runbooks that don't match reality, alerting noise (>5 pages/night = trust erosion), no post-incident review discipline
• Performance: load tests that don't reflect real traffic mix; caches with unknown hit ratios

Cadence: annually per tier-1 workload, plus at major design changes. Track HRI burn-down as a platform KPI.

Senior framing: 'the review's real product is shared understanding of accepted risk — the business explicitly signing off on what we're not fixing is as valuable as the fixes.'