CI/CD Fundamentals

TL;DR

• CI = automatically build/test every change against mainline; Continuous Delivery = every passing change is packaged and ready to release (often behind a manual trigger); Continuous Deployment = every passing change releases to production automatically, no gate.
• Pipeline stage order should follow “fail fast”: cheap/fast checks (lint, unit tests) block first; slow/expensive checks (integration, e2e) run only after those pass.
• “Build once, deploy everywhere” — build exactly one artifact and promote it through environments; rebuilding per environment risks environment drift and non-reproducible bugs.
• Artifacts must be immutably versioned; overwriting a latest tag on every build silently destroys rollback capability.
• Manual approval gates should map to blast radius/risk, not be applied uniformly — automate low-risk environments, gate production.
• Biggest gotcha: “the pipeline passed” and “the code is ready to ship” are not equivalent claims — a green pipeline only proves the tests that exist all passed.

Core Concepts

CI/CD Concepts

Continuous Integration (CI): every change is automatically built and tested against the mainline branch as soon as it’s committed, surfacing integration problems within minutes rather than at a large, delayed merge.

Continuous Delivery (CD): extends CI — every change that passes the pipeline is automatically packaged into a deployable artifact and made ready to release, typically gated by a final manual trigger before production.

Continuous Deployment: extends Continuous Delivery one step further — every passing change is automatically released to production, with no manual gate at all.

The distinction between Delivery (“ready to release, human decides when”) and Deployment (“automatically released, no human in the loop”) is the detail most commonly confused when these terms are used loosely.

Pipeline Design

“Fail fast” is a structural ordering principle: place fast, cheap checks (static analysis, linting, unit tests) at the front of the pipeline so they block immediately on failure. Place slower, more expensive checks (integration tests, full end-to-end suites) later, so pipeline compute and developer wait time aren’t spent running expensive tests to rediscover a failure a cheap test would have caught in seconds. Running checks in the wrong order, or in parallel without regard to cost, wastes both time and shared test infrastructure.

Build Stages

A build stage should produce exactly one artifact (a container image, a compiled binary, a package) that is then promoted, unmodified, through every subsequent environment — dev, staging, production. This is the “build once, deploy everywhere” principle.

Rebuilding a fresh artifact separately per environment breaks this guarantee: dependency resolution can pick different transitive versions between builds, and a base image tag can resolve to a different underlying patch level at different build times. The result is that what passed staging is not provably the same bits running in production — a common, hard-to-diagnose source of “works in staging, breaks in prod” bugs with no code diff to explain them.

Testing

A pipeline’s test stage proves only what it was configured to check. A fully green pipeline confirms that the tests which exist all passed — it says nothing about test coverage gaps, whether the existing tests validate the correct behavior, or non-functional concerns (performance regressions, security posture) the pipeline was never configured to evaluate. Treating “pipeline passed” as synonymous with “ready to ship” is a category error: green is necessary, not sufficient, evidence of production readiness.

Artifacts

Artifacts should be immutable and uniquely versioned (a commit SHA, a semver tag, a build number) rather than overwritten. A pipeline that continually republishes to a latest tag/label loses the ability to redeploy “exactly what was running two versions ago,” because that specific artifact no longer exists — it was overwritten by every subsequent build. Reliable rollback depends on every previously deployed artifact remaining retrievable.

Release Pipelines

Manual approval gates should be placed according to actual risk and blast radius, not applied uniformly across every environment. Staging/ephemeral environments are low-risk and reversible with no customer impact, so automating deployment there maximizes feedback speed. Production deployments carry real business and customer risk; a deliberate human confirmation point — not necessarily slow, potentially a single click — can catch context a fully automated pipeline structurally cannot know about (e.g., an unrelated incident in progress that makes this deploy a bad idea right now).

CI vs. Continuous Delivery vs. Continuous Deployment

Push-Based vs. Pull-Based Pipelines

Command / Configuration Reference

# Example CI pipeline stage ordering (conceptual, fail-fast structure)
stages:
  - lint          # fast, cheap — fails immediately on style/static issues
  - unit_test      # fast — seconds to low minutes
  - build_artifact  # produce exactly one versioned, immutable artifact
  - integration_test # slower — runs only if unit tests passed
  - e2e_test        # slowest — runs only if integration tests passed
  - deploy_staging   # automated, no manual gate
  - manual_approval  # human gate — only before production
  - deploy_production

docker build -t myapp:${GIT_SHA} .        # build one immutable, uniquely tagged artifact
docker push registry/myapp:${GIT_SHA}      # push versioned artifact, not "latest"
docker pull registry/myapp:${GIT_SHA}       # deploy the exact same artifact to every environment
git tag v2.3.1 && git push origin v2.3.1    # tag the commit corresponding to a shipped release

Common Pitfalls

• Pitfall: Rebuilding a fresh artifact separately for each environment. Why: dependency resolution and base image patch levels can differ between builds, so “the same code” may not mean “the same bits.” Fix: build one artifact and promote it unmodified through every environment.
• Pitfall: Running slow integration/e2e tests before fast unit tests. Why: wastes compute and developer feedback time discovering a failure a cheap test would have caught first. Fix: order pipeline stages by cost/speed, fastest first, each gating the next.
• Pitfall: Overwriting a latest artifact tag on every build. Why: destroys the ability to redeploy a specific prior version, since it no longer exists. Fix: version every artifact immutably (commit SHA, semver, build number) and never overwrite a published artifact.
• Pitfall: Applying manual approval gates uniformly to every environment, including ephemeral/staging. Why: adds delay to low-risk, reversible deployments with no corresponding safety benefit. Fix: gate approval by actual blast radius — automate low-risk environments, gate production.
• Pitfall: Treating a green pipeline as proof the code is production-ready. Why: a passing pipeline only confirms existing tests passed; it says nothing about coverage gaps or unconfigured non-functional checks. Fix: track test coverage and non-functional checks explicitly, and treat “pipeline passed” and “ready to ship” as separate claims.

Interview Questions

• “Explain Continuous Integration, Continuous Delivery, and Continuous Deployment as three distinct things, with a concrete example of each.” — Tests precise understanding versus using the terms interchangeably.
• “A team rebuilds their Docker image separately for every environment. What’s wrong with this, and what’s the fix?” — Tests whether “build once, deploy everywhere” is understood as a concrete practice, not an abstract principle.
• “When would you insist on a manual approval gate, and when would you actively remove one?” — Tests whether gate placement is reasoned from actual risk rather than applied uniformly out of caution.
• “Why should fast unit tests run before slow integration tests in a pipeline?” — Tests understanding of fail-fast as a structural, not incidental, design choice.
• “What operational capability is lost if a pipeline overwrites the same artifact tag on every build?” — Tests whether the link between immutable versioning and rollback is understood.
• “What’s the difference between push-based and pull-based (GitOps) deployment models?” — Tests breadth beyond a single linear pipeline model.