DevOps & Platform Engineering

TL;DR

• CodePipeline/CodeBuild/CodeDeploy are AWS’s native CI/CD; Jenkins, GitHub Actions are legitimate alternatives based on actual pain, not platform preference.
• Deployment strategy choice determines blast radius: all-at-once with manual rollback is high-risk; canary/linear with automated rollback is high-leverage.
• CodeBuild jobs needing private on-prem access require explicit VPC configuration; default (non-VPC) CodeBuild cannot reach private networks.
• Internal Developer Platforms (IDPs) provide paved-road templates; without them, every team reinvents pipeline configuration, security scanning, and deployment strategy.
• Fragmented CI/CD tooling defers costs: security scanning and guardrails must be re-implemented per tool, and reusable infrastructure patterns become impossible.
• Pipeline IAM roles must follow least-privilege per stage; overly broad roles are a common security gap.

Core Concepts

CI/CD Tool Choice: Actual Pain vs. Platform Preference

The mistake is migrating CI/CD tools for reasons like “it’s AWS-native” without naming an actual problem with the current tool.

Questions to ask before migrating:

• What specific pain point is the current tool causing? (Plugin maintenance burden, scaling limits, self-managed infrastructure risk?)
• What concrete benefit does the new tool provide for that pain point?
• What is the migration risk and effort?

Example bad decision: Migrating from well-functioning Jenkins to CodePipeline “because CodePipeline is AWS-native.” If Jenkins is working fine, migration risk outweighs the benefit.

Example good decision: Jenkins is a plugin maintenance and scaling burden; migrating to CodePipeline (no plugins, AWS-managed scaling) solves a named problem.

AWS CodePipeline, CodeBuild, CodeDeploy

CodePipeline orchestrates a multi-stage deployment: source (CodeCommit, GitHub), build (CodeBuild), test, deploy (CodeDeploy), etc. Each stage is a distinct step; the pipeline coordinates transitions and error handling.

CodeBuild runs build/test commands in a managed, ephemeral container. No standing infrastructure to manage. Integrates with IAM for fine-grained permissions.

CodeDeploy handles the actual deployment to EC2, on-premises servers, ECS, Lambda, etc. Supports deployment strategies: all-at-once, rolling, blue/green, canary, linear.

Advantage: Deep IAM integration. Permissions are managed through the same IAM roles as the rest of your AWS infrastructure.

Deployment Strategies: All-at-Once vs. Canary vs. Linear

All-at-once: All traffic switches to the new version immediately. If the new version is bad, the entire customer base is affected until a human notices and rolls back. Typical detection time: 10+ minutes.

Canary: Route a small percentage (e.g., 5%) of traffic to the new version. Monitor for errors. If errors are high, automatically roll back. If clean, gradually shift remaining traffic. Detection time: <1 minute.

Linear: Shift traffic gradually (e.g., 10% every minute) to the new version. Monitor at each step. Automatic rollback if errors spike. Detection time: <1 minute.

Blast radius: All-at-once has maximum impact. Canary and linear are high-leverage risk reduction.

CodeBuild VPC Configuration

Default (non-VPC) CodeBuild: Runs in AWS-managed infrastructure. Cannot reach private networks (on-premises, VPC-hosted databases).

VPC-configured CodeBuild: Runs inside a specified VPC. Can reach on-premises networks via VPN/Direct Connect, and VPC-hosted resources.

Common failure: CodeBuild jobs fail mysteriously trying to pull from a private artifact repository. Root cause: VPC configuration was never set up. Fix: Configure CodeBuild to run in a VPC with access to the private repository network.

Internal Developer Platforms: Paved Roads

An IDP provides self-service, pre-configured templates that application teams use to scaffold new services:

• Repository template: Pre-configured GitHub/CodeCommit repo with standard directory structure.
• Pipeline template: Standard CodePipeline definition with security scanning, tests, and deployment strategy already configured.
• Infrastructure template: Approved IAC patterns (CloudFormation/Terraform/CDK) for common infrastructure (ALB, database, caching).
• Deployment template: Standard blue/green or canary strategy already in place.

Benefit: New services are consistent by default. Security scanning is standardized. Deployment risk is reduced.

Cost of not having an IDP: Every team reinvents pipeline configuration. Some teams do security scanning; some skip it. Some teams use canary deployments; some use all-at-once. Deployment rigor is wildly inconsistent. The platform team cannot standardize because there’s no common substrate.

Pipeline IAM Roles: Least Privilege per Stage

A pipeline with a single broad IAM role grants all stages the same permissions. If a build stage is compromised, it has access to deploy to production, change databases, delete S3 buckets, etc.

Better approach: Each stage has its own IAM role with minimal permissions. Build stage can only compile and push images to ECR. Deploy stage can only update ECS services. This compartmentalizes risk.

Example: Build stage compromise can only push a malicious image; it cannot deploy it or modify data. Deploy stage can only deploy approved images; it cannot modify the build process.

Comparison Table

Command/Configuration Reference

CodeBuild with VPC Access

Create a CodeBuild project with VPC configuration:

aws codebuild create-project \
  --name my-build \
  --service-role arn:aws:iam::123456789012:role/codebuild-role \
  --artifacts type=CODEPIPELINE \
  --environment type=LINUX_CONTAINER,image=aws/codebuild/standard:4.0,computeType=BUILD_GENERAL1_SMALL \
  --vpc-config subnetIds=subnet-12345678,securityGroupIds=sg-12345678

CodeDeploy with Canary Strategy

AppSpec configuration for canary deployment:

version: 0.0
Resources:
  - TargetService:
      Type: AWS::ECS::Service
      Properties:
        TaskDefinition: "my-task:1"
        LoadBalancerInfo:
          ContainerName: "my-container"
          ContainerPort: 8080
Hooks:
  - BeforeAllowTraffic: "pre-traffic-test-lambda"
  - AfterAllowTraffic: "post-traffic-test-lambda"

CodeDeploy deployment with canary:

aws deploy create-deployment \
  --application-name my-app \
  --deployment-group-name my-deployment-group \
  --deployment-config-name CodeDeployDefault.ECSCanary10Percent5Minutes \
  --s3-location s3://my-bucket/app.zip

Common Pitfalls

Interview Questions

• A platform team wants to migrate from Jenkins to CodePipeline. What questions do you ask before agreeing the migration is worth it? — Tests whether they interrogate actual problems vs. platform preference, and understand migration risk/benefit trade-offs.

• Design a deployment strategy for a customer-facing service deploying multiple times a day, with the goal of minimizing blast radius. — Tests whether canary/linear traffic-shifting with automated rollback is the instinct, and whether health checks and rollback triggers are included.

• Your CodeBuild job is failing when it tries to pull from a private artifact repository. Walk me through your debugging approach. — Tests understanding of VPC configuration as the prerequisite for CodeBuild to reach private networks.

• How would you pitch an Internal Developer Platform to engineering leadership? — Tests whether they can articulate the long-term consistency and security benefits, not just short-term velocity gains.