Compute Architecture

TL;DR

• Compute selection is a workload-shape problem: match duration, statefulness, and interruption tolerance to the right abstraction before picking a service by trend.
• Spot Instances are the highest-leverage, most underused cost lever for interruption-tolerant work — 60-90% savings versus On-Demand.
• ECS vs. EKS is an organizational-fit decision, not a technical-superiority one; picking EKS without the operational budget to run it creates incidents later.
• Fargate trades a per-vCPU/GB price premium for zero node management — worth it when platform engineering time is scarcer than budget.
• Lambda cold starts are fixable (memory tuning, package trimming, Provisioned Concurrency) and rarely justify abandoning the platform.
• Elastic Beanstalk is a fast on-ramp, not a long-term production platform — most workloads outgrow its abstraction and graduate to direct EC2/ECS/ALB management via IaC.

Core Concepts

EC2 with Launch Templates and Auto Scaling

EC2 remains the correct choice when host-level control is required: custom kernel parameters, specific instance families for GPU/memory-bound workloads, or licensing that mandates dedicated hosts. Launch Templates replace the deprecated Launch Configurations and support versioning, mixed instance policies, and both On-Demand and Spot allocation in one definition — version them in IaC for repeatable, auditable provisioning. Auto Scaling resilience depends on the scaling signal: target tracking against a custom CloudWatch metric that reflects actual user-facing load (queue depth, request latency) produces materially better outcomes than scaling on CPU utilization alone, which is a proxy that often lags or misrepresents real load.

Spot Instances

Spot Instances draw from unused EC2 capacity at up to a 90% discount versus On-Demand, in exchange for a 2-minute interruption notice when AWS reclaims the capacity. They fit any interruption-tolerant, checkpointable workload: batch processing, CI/CD runners, stateless web tiers behind fast health-check-driven replacement. Architecting for Spot means:

• Checkpointing state so a reclaimed instance can resume work rather than lose it.
• Using mixed instance policies across multiple instance types and Availability Zones — depending on a single instance type/AZ pool means a capacity shortage in that one pool takes down the entire fleet.
• Choosing a Capacity-Optimized allocation strategy over lowest-price, since lowest-price chases the cheapest pools, which are also the ones most likely to be reclaimed.

ECS vs. EKS as containers orchestration

Covered in depth in its own comparison table below — the short version is that this is a platform-ownership decision driven by team capability and ecosystem needs, not a universal ranking.

Fargate

Fargate removes node management from both ECS and EKS: no OS patching, no capacity planning, no bin-packing. The trade-off is a real price premium per vCPU/GB versus a well-utilized self-managed EC2 fleet. For teams without dedicated platform engineering headcount, that premium is frequently cheaper than the engineering time node lifecycle management (patch cadence, AMI rotation, capacity right-sizing) actually consumes.

Lambda

Lambda is the right default for event-driven, short-duration, bursty workloads. It is not a universal replacement for long-running services, and cold starts are a real, measurable latency cost on customer-facing paths. The effective mitigation order:

1. Right-size memory allocation — memory scales CPU proportionally in Lambda, and increasing it often reduces cold start duration more than expected, in addition to speeding up execution.
2. Trim the deployment package and dependency tree — smaller packages initialize faster.
3. Apply Provisioned Concurrency specifically to the functions on the latency-sensitive path — this keeps a pool of pre-initialized execution environments warm, eliminating cold start for those invocations at a standing cost.

A wholesale migration off Lambda is rarely the correct response to a cold-start complaint; it is a treatable symptom, not a structural limitation of the platform.

AWS Batch

AWS Batch is purpose-built for queued, scheduled, and massively parallel compute. It manages the Spot/On-Demand instance mix, job queue, and compute environment sizing automatically — materially less operational surface than hand-rolling equivalent orchestration on raw EC2 Auto Scaling groups.

Elastic Beanstalk

Elastic Beanstalk is a fast on-ramp for standard web applications — minimal setup to get something running. The abstraction that makes it fast to start becomes limiting once fine-grained control over underlying resources is needed. Most production architectures either graduate off Beanstalk entirely or manage the underlying EC2/ECS/ALB resources directly via IaC once requirements exceed what the platform anticipated.

ECS vs. EKS

EC2 vs. Fargate vs. Lambda vs. AWS Batch

Common Pitfalls

• Pitfall: Choosing EKS for perceived industry standardization. Why: The decision is treated as a technical-superiority contest rather than an organizational-capability match. Fix: Evaluate Kubernetes operational readiness (staffing, on-call experience, upgrade cadence) before committing, and default to ECS absent a concrete need for the CNCF ecosystem.
• Pitfall: Leaving interruption-tolerant batch workloads on On-Demand pricing indefinitely. Why: “We’ll add Spot later” notes get deprioritized once the workload is functionally stable. Fix: Treat Spot evaluation as a required step in the design review for any batch or stateless workload, not an optional follow-up.
• Pitfall: Treating Lambda cold starts as a reason to abandon serverless. Why: Cold start is diagnosed as a platform limitation rather than a tunable parameter. Fix: Exhaust memory tuning, package trimming, and targeted Provisioned Concurrency before considering a platform migration.
• Pitfall: Running Elastic Beanstalk well past the point where direct resource control is needed. Why: Teams fight the abstraction instead of graduating off it once requirements mature past what Beanstalk anticipated. Fix: Migrate to IaC-managed EC2/ECS/ALB resources once configuration needs exceed Beanstalk’s supported customization surface.
• Pitfall: Scaling Auto Scaling Groups purely on CPU utilization. Why: CPU is a proxy metric that can lag or misrepresent actual user-facing load (e.g., I/O-bound or queue-driven workloads). Fix: Use target tracking against a custom CloudWatch metric that reflects real demand.

Interview Questions

• “A workload’s compute bill is dominated by a nightly batch job. Walk me through your cost optimization approach.” — tests whether Spot is the first instinct and whether checkpointing for interruption tolerance is understood.
• “Defend a choice of ECS over EKS for a 15-person engineering org with no prior Kubernetes experience.” — tests organizational judgment, not just technical service knowledge.
• “A Lambda API has p99 latency complaints traced to cold starts. What do you try, in what order, and why?” — tests whether tuning is attempted before a platform migration is proposed.
• “When would host-level control justify choosing raw EC2 over Fargate or Lambda?” — tests whether the candidate can name concrete constraints (kernel tuning, GPU instance families, licensing) rather than defaulting to “it’s cheaper.”
• “Design a compute strategy for a workload that’s 80% steady-state and 20% unpredictable spikes.” — tests whether Reserved/Savings Plans for baseline plus Auto Scaling or Spot for burst is the reasoned answer.