ArgoCD

TL;DR

• ArgoCD continuously reconciles Kubernetes cluster state to match Git, replacing manual deployment scripts with declarative, auditable GitOps.
• Sync Waves and Hooks coordinate sequencing and prerequisites within a single sync; wave ordering only works reliably if downstream resources have real readiness probes configured.
• ApplicationSets with prune: true can cascade-delete entire environments from a single mistaken Git commit; this power requires structural safeguards.
• Multi-cluster setups concentrate a critical dependency on the central ArgoCD instance’s availability; treat its backup and recovery as a first-class operational priority.
• Polling (Git-watch) and webhooks are complementary, not competing; polling is the fallback that catches missed webhooks.

Core Concepts

Applications and Git reconciliation

An Application is ArgoCD’s core unit: a Git source (repo, path, revision) mapped to a destination cluster and namespace. ArgoCD continuously watches that Git path and reconciles the cluster’s actual state to match the declared desired state. This is the declarative core: Git is the single source of truth, ArgoCD is the agent that enforces it.

Sync Waves and readiness-gated ordering

Resources tagged with argocd.argoproj.io/sync-wave: N deploy in numeric order within a sync cycle (lower numbers first). ArgoCD waits for each wave to report healthy before proceeding to the next. Critical gotcha: a Deployment with no readiness probes reports “healthy” as soon as it’s progressing, not necessarily ready to serve traffic. Wave ordering only gates progression meaningfully when the resources being waited on have real health signals.

Hooks and lifecycle coordination

Hooks run at specific sync phases:

• PreSync — before main resources (database migrations, smoke tests)
• Sync — alongside main resources
• PostSync — after all resources are applied
• SyncFail — on failure paths

A PreSync hook runs and completes fully before the main Deployment ever applies. The hook-delete-policy: BeforeHookCreation is mechanically essential: Kubernetes forbids duplicate Job names, so without a delete policy, the previous completed migration Job blocks the next sync’s migration from being created.

apiVersion: batch/v1
kind: Job
metadata:
  name: db-migrate
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
spec:
  template:
    spec:
      containers:
      - name: migrate
        image: myapp:latest
        command: ["./migrate", "up"]

ApplicationSets: templating applications at scale

ApplicationSets generate multiple Applications from a single declaration using generators: list-based, cluster-based, Git-directory, or matrix combinations. A Git-directory generator with envs/dev, envs/staging, envs/prod auto-creates one Application per directory. Adding an environment is adding a directory. Operational risk: with prune: true, deleting a directory in Git deletes the Application and all its managed resources — a single mistaken commit can cascade-delete an entire environment’s production infrastructure with no gate.

Multi-cluster deployments

A central ArgoCD instance can manage many downstream clusters. Existing workloads on those clusters keep running if the central instance goes down (ArgoCD is not in the live traffic path), but no cluster can reconcile new Git changes, correct drift, or execute rollbacks until the central instance recovers. This concentrates a critical dependency: treat the central instance’s availability, backup, and recovery as a first-class operational priority, not an afterthought.

App of Apps: self-management risk

Using ArgoCD to manage ArgoCD’s own configuration is architecturally elegant but introduces a specific risk: a bad configuration change can break the sync mechanism itself, and Git-based recovery becomes unavailable exactly when needed. Every team using this pattern must document and test a break-glass procedure (direct kubectl access to fix ArgoCD directly) rather than trusting Git-based recovery will always work.

Polling vs. webhooks

ArgoCD polls Git every 3 minutes by default; webhooks trigger near-instant sync on push. Disabling polling in favor of webhooks-only removes a safety net: a missed or failed webhook leaves the cluster out of sync indefinitely if polling is disabled. Keeping polling enabled as a fallback means a missed webhook is caught within the next polling interval, not silently never syncing.

ArgoCD vs Flux vs Traditional CI/CD

Configuration Reference

# Basic Application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
spec:
  project: default
  source:
    repoURL: https://github.com/myorg/config
    path: app/
    targetRevision: main
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

# Sync Wave ordering with hooks
apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
    argocd.argoproj.io/sync-wave: "-1"

---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "1"

Common Pitfalls

• Pitfall: Sync waves appear correctly ordered but don’t actually gate progression. Why: Downstream Deployments have no readiness probes; ArgoCD considers them healthy as soon as they’re progressing. Fix: Configure real readiness probes on any resource whose health gates the next wave.

• Pitfall: A hook Job blocks the next sync because Kubernetes won’t allow a duplicate name. Why: The previous Job completed but wasn’t cleaned up; no delete policy was configured. Fix: Always add hook-delete-policy: BeforeHookCreation to any hook.

• Pitfall: A mistaken Git commit deletes a directory; entire environment’s production resources are pruned. Why: ApplicationSet with prune: true cascades the deletion to all managed resources. Fix: Require pull-request review before allowing directory deletions on ApplicationSet-managed paths; consider requiring human confirmation for prune operations.

• Pitfall: Central ArgoCD instance is down; no cluster can deploy or roll back. Why: Multi-cluster setup concentrates the control plane as a critical dependency. Fix: Treat central instance availability as a first-class operational concern; maintain runnable backups and break-glass procedures; consider multi-zone or multi-region HA for the control plane.

• Pitfall: Cluster silently drifts out of sync; nobody notices until it causes an incident. Why: Polling was disabled in favor of webhooks-only; a missed webhook went undetected. Fix: Keep polling enabled as a fallback; monitor for out-of-sync status programmatically.

Interview Questions

• Explain the mechanism by which a PreSync hook with the correct delete policy prevents a deployment blocker. — Tests understanding of hook lifecycle, the Job-naming constraint, and how delete policies solve it.

• A sync wave appears correctly ordered in the manifest, but the downstream service still fails because it can’t connect to the upstream one. What’s happening? — Tests whether readiness-probe gaps are recognized as the root cause.

• An ApplicationSet with prune: true deletes an environment from one mistaken Git commit. What structural safeguards should prevent this? — Tests whether the cascade-delete risk is understood and mitigation strategies are articulated.

• Your team runs ArgoCD itself via the app-of-apps pattern. ArgoCD breaks; you can’t fix it via Git. What’s your recovery plan? — Tests whether break-glass procedures and direct kubectl fallbacks are planned proactively.

• Explain why disabling polling in favor of webhooks-only is actually riskier than keeping both. — Tests understanding of the complementary safety-net relationship, not a redundant either/or.