Multi-Container Pods

TL;DR

• Sidecar: Auxiliary container extends app functionality (e.g., FluentBit log collector alongside app)
• Ambassador: Local proxy for outgoing connections (e.g., ProxySQL routing to active database primary)
• Adapter: Normalizes container output format (e.g., app metrics → Prometheus format)
• Init Containers run sequentially to completion before app containers start; K8s 1.28+ allows restartPolicy: Always for native sidecars
• Containers share network namespace (localhost accessible), storage volumes, and IPC; no resource isolation by default
• All containers in pod compete for pod’s cgroup allocation; leaky sidecar can OOM-kill main app without per-container limits

Core Concepts

Sidecar Pattern

Auxiliary container runs alongside main app, sharing pod’s lifecycle, storage, and network. Extends or enhances functionality without modifying app container.

• Example: FluentBit collecting app logs from shared /var/log volume, shipping to Elasticsearch
• Starts with app, dies with app

Ambassador Pattern

Local proxy container routes outgoing connections on behalf of main app. Hides external topology from application.

• Example: App connects to localhost:3306, Ambassador runs ProxySQL routing to active RDS primary (handles failover)
• Simplifies connection logic in app; topology changes are proxy’s responsibility

Adapter Pattern

Container normalizes/transforms outputs before exposing to cluster. Standardizes incompatible formats.

• Example: App writes custom metrics, Adapter converts to Prometheus format, exposes on port 9100
• Enables monitoring tool integration without modifying app

Init Containers

Run sequentially to completion before any app container starts. Common for:

• Waiting for databases to be ready
• Pulling config from Vault
• Running schema migrations
• Initializing shared volumes

K8s 1.28+ allows restartPolicy: Always on init containers, making them “native sidecars” that start first and run alongside app containers.

Shared Resources

All containers in pod share:

• Network namespace (one IP, shared port space)
• IPC namespace (shared memory)
• Storage volumes (mounted at different paths)
• Cgroups (CPU/memory allocation is per-pod, not per-container)

Comparison Table

Command & Configuration Reference

Multi-Container Pod YAML

apiVersion: v1
kind: Pod
metadata:
  name: app-with-sidecar
spec:
  volumes:
  - name: shared-logs
    emptyDir: {}

  initContainers:
  - name: init-setup
    image: busybox
    command: ["sh", "-c", "echo 'setup' > /shared/config"]
    volumeMounts:
    - name: shared-logs
      mountPath: /shared

  containers:
  # Main app container
  - name: app
    image: myapp:v1
    volumeMounts:
    - name: shared-logs
      mountPath: /logs
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 500m
        memory: 512Mi

  # Sidecar log collector
  - name: log-collector
    image: fluent-bit:latest
    volumeMounts:
    - name: shared-logs
      mountPath: /logs
    resources:
      requests:
        cpu: 50m
        memory: 64Mi
      limits:
        cpu: 200m
        memory: 256Mi

Container Management Commands

# View all containers in pod
kubectl get pod app-with-sidecar -o jsonpath='{.spec.containers[*].name}'

# Logs from specific container
kubectl logs app-with-sidecar -c app
kubectl logs app-with-sidecar -c log-collector -f

# Exec into specific container
kubectl exec -it app-with-sidecar -c app -- sh

# View init container logs
kubectl logs app-with-sidecar -c init-setup

Common Pitfalls

Pitfall: Shared cgroup Resource Contention Sidecar has memory leak; consumes entire pod’s memory allocation; main app gets OOM-killed.

• Why: All containers in pod share one cgroup; no per-container isolation. Leaky sidecar starves app container.
• Fix: Always define explicit resources.limits on every container (init and regular). Example: app limits 512Mi, sidecar limits 256Mi. Monitor container metrics.

Pitfall: Init Container Timeout Init container hangs waiting for external service; pod never becomes ready.

• Why: No built-in timeout; if init container blocks indefinitely, pod never reaches Running state.
• Fix: Implement timeout logic in init container itself (e.g., for i in {1..30}; do curl db:5432 && break; sleep 1; done). Set restartPolicy: Never or OnFailure.

Pitfall: Expecting Network Isolation Between Containers App container tries to connect to sidecar expecting it to be on different IP/port.

• Why: Containers share network namespace; both on same IP. No need for cross-network communication.
• Fix: Use localhost to reach sidecar, or shared volumes for IPC. Don’t expose sidecar externally via Service.

Interview Questions

Q — Describe the Sidecar, Ambassador, and Adapter patterns. When would you use each? Sidecar enhances app functionality (e.g., logging, metrics scraping) without modifying app code—runs alongside app sharing storage/network. Ambassador is a local proxy hiding external topology changes (e.g., database failover proxy)—app connects to localhost, ambassador handles complexity. Adapter transforms outputs to standard formats (e.g., custom metrics → Prometheus)—enables tool integration. Use Sidecar for cross-cutting concerns, Ambassador for dependency stability, Adapter for standards integration.

Q — How do containers in a pod communicate? What’s shared between them? Containers share network namespace (same IP, port space), IPC namespace (shared memory), and storage volumes. They reach each other via localhost. Can communicate via Unix sockets on shared volumes. No inter-container network isolation; they’re on the same “machine.”

Q — What happens if a sidecar container has a memory leak? How do you prevent OOM-killing the main app? All containers share the pod’s cgroup allocation. A leaky sidecar consumes memory unbounded, starving the main app. Kubelet’s OOM-killer may terminate any container when pod exceeds limits. Fix: Set explicit resources.limits on every container (don’t just set pod-level limits). Monitor container memory usage independently. Example: main app limits 512Mi, sidecar limits 256Mi—total 768Mi for the pod.