40. Architecture Patterns
Architecture Patterns
TL;DR
- DevOps maturity requires CI/CD, GitOps, platform engineering, and DevSecOps integrated together, not CI/CD alone.
- Maturity models suggest next steps based on context (deployment frequency, risk tolerance, team size); maximum automation is not universally correct.
- Fragmented adoption of tools across teams multiplies operational surface area with no compounding shared value; shared conventions consolidate capability.
- Security integrated into golden paths (secure defaults in templates, policy enforcement in GitOps reconciliation) scales automatically; bolted-on gates depend on every team’s diligence.
- Demonstrated capability (tested rollback, tested security gate) is the real signal of maturity; sophisticated tooling never tested under failure is unproven.
- Biggest gotcha: mistaking strong CI/CD for complete DevOps maturity, or treating tool adoption as evidence of actual operational capability.
Core Concepts
CI/CD maturity vs. overall DevOps maturity
An organization with strong, fast CI/CD but no GitOps, platform engineering, or integrated DevSecOps has solved the most visible, immediately-rewarded part — but a mature practice integrates four interconnected disciplines: CI/CD (build and test), GitOps (deployment), platform engineering (developer experience), and DevSecOps (security at every stage). Strong CI/CD alone is necessary but insufficient.
Maturity models and context
The maturity spectrum from “manual deployment with checklist” to “fully automated GitOps-driven deployment with progressive delivery” isn’t a ladder every organization should climb to the top of. The right target level depends on actual deployment frequency, risk tolerance, team size, and regulatory constraints. A maturity model’s value is helping identify the next realistic step, not mandating maximum automation universally.
Fragmented vs. consolidated tooling
When GitOps, platform self-service, and DevSecOps are each independently adopted by different teams without shared conventions, the result is compounding organizational cost: different GitOps tools per team prevent knowledge transfer; different self-service patterns prevent building genuinely shared platform capability; the organization must maintain expertise across a wide, uncoordinated tooling surface instead of consolidating around shared choices that create compound value.
Security integration: golden path vs. final gate
Security enforced as a separate final pipeline gate depends on every team remembering to pass through it and depends on the gate staying comprehensive as deployment paths evolve. Security baked into the golden path itself — secure defaults in platform templates, policy enforcement as part of GitOps reconciliation — scales consistently across teams automatically, without depending on individual diligence.
Demonstrated vs. unproven capability
Tool adoption alone doesn’t prove underlying capability works under real conditions. A team with sophisticated GitOps and platform tooling that’s never tested rollback, or a DevSecOps pipeline that’s never blocked a real vulnerable deploy, has unproven maturity. A team that’s demonstrated a fast, safe recovery from an actual production incident has proven maturity.
DevOps Maturity Spectrum
| Aspect | Manual | Scripted CI/CD | Mature GitOps | Platform-driven |
|---|---|---|---|---|
| Deployment | Checklist-driven, manual | Automated pipeline | GitOps-reconciled | Platform templates, self-service |
| Rollback | Manual steps, scripted | Automated in pipeline | GitOps revert (commit history) | Automated via platform |
| Security | Manual review gates | Scanned in pipeline | Integrated in GitOps | Defaults in templates, enforced in reconciliation |
| Developer experience | Direct infra access | Pipeline approval steps | Self-service via GitOps | Platform abstraction layer |
| Blast radius of error | Broad (manual steps) | Limited (pipeline gates) | Narrow (committed diffs) | Minimal (platform guardrails) |
Configuration / Command Reference
# Example: Security integrated into GitOps (Kyverno + ArgoCD)
# 1. Platform team defines secure baseline in platform templates
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: secure-template
spec:
destinations:
- namespace: '*'
server: https://kubernetes.default.svc
sourceRepos:
- 'https://github.com/org/app-configs'
# 2. Policy enforced in reconciliation loop (Kyverno webhook)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-resource-limits
spec:
validationFailureAction: enforce # Blocks in GitOps loop
rules:
- name: check-limits
match:
resources:
kinds:
- Pod
validate:
message: "Resource limits required by platform security policy"
pattern:
spec:
containers:
- resources:
limits:
cpu: "?*"
memory: "?*"
# 3. Developer uses platform self-service (secure by default)
argocd app create myapp \
--repo https://github.com/org/app-configs \
--path myapp \
--dest-server https://kubernetes.default.svc \
--dest-namespace default
# Kyverno automatically enforces policy during ArgoCD reconciliation
# Testing maturity: simulate rollback scenario
argocd app rollback myapp <revision>
# Or: git revert <commit>; ArgoCD syncs automatically
# Testing security gate: attempt to deploy with missing limits
# Kyverno rejects; demonstrates gate actually works
Common Pitfalls
- Pitfall: Organization with impressive CI/CD but no coordinated GitOps, platform, or security strategy; mistaking pipeline speed for overall maturity. Why: CI/CD is most visible and immediately rewarded; holistic maturity is less obvious. Fix: Evaluate maturity across all four disciplines (CI/CD, GitOps, platform, DevSecOps) together.
- Pitfall: Every team independently adopting different GitOps, platform, and security tooling. Why: No organizational governance or consolidation effort. Fix: Establish shared conventions and consolidate tooling choices for compound organizational value.
- Pitfall: Security scanning as a separate final gate that teams can route around. Why: Bolted-on gate doesn’t scale; depends on every team’s diligence. Fix: Integrate security into golden paths and GitOps reconciliation loops.
- Pitfall: Organization confident in sophisticated tooling, discovering during a real incident that rollback/recovery was never tested and doesn’t actually work. Why: Tool adoption assumed to equal capability. Fix: Regularly test critical capabilities (rollback, security blocking) under real conditions, not just in isolated tests.
- Pitfall: Assuming maximum automation is universally correct for an organization. Why: Context (deployment frequency, team size, regulatory constraints) varies. Fix: Match target maturity level to actual organizational context and needs.
Interview Questions
- An organization has excellent CI/CD but no GitOps or platform engineering. How do you assess their actual DevOps maturity? — Tests whether CI/CD alone is recognized as necessary but insufficient.
- Different teams independently adopted different GitOps tools. What’s the real cost beyond inconsistency? — Tests understanding of the compounding cross-team cost and lost compound value.
- How do you verify claimed DevOps maturity beyond looking at the tooling list? — Tests whether demonstrated capability (tested rollback, tested security gate) is the core evaluation criterion.
- How would you design a platform to integrate security defaults rather than treating security as a final gate? — Tests understanding of golden-path security integration and scalability benefits.