Kubernetes Cluster Setup

TL;DR

• Minikube runs single-node cluster in a VM; KIND runs multi-node cluster as Docker containers—KIND is faster for CI/CD
• kubeadm bootstraps production clusters: kubeadm init on control plane generates CA/certs and static pod manifests
• kubeadm join on workers uses secure tokens to authenticate and register kubelet with API Server
• kubeconfig file (~/.kube/config) contains cluster API endpoints, client certs/tokens, and context definitions
• Swap must be disabled—scheduler assumes strict resource bounds; swapping breaks memory accounting
• After kubeadm init, all nodes stay NotReady until CNI plugin (Calico, Cilium) is installed

Core Concepts

Local Development: Minikube vs. KIND

Developers need lightweight sandbox clusters. Two main tools:

Minikube: Runs a single-node cluster in a local VM (or Docker). Slower startup, heavier resource footprint. Best for general learning, testing Kubernetes add-ons.

KIND (Kubernetes in Docker): Multi-node clusters run as sibling Docker containers. Seconds to boot, minimal RAM. Ideal for CI/CD pipelines, testing Helm charts, integration testing without paying for VMs.

Production Setup with kubeadm

kubeadm bootstraps bare-metal or cloud VMs into a Kubernetes cluster:

1. kubeadm init (Control Plane): Validates system requirements (swap disabled), generates CA and certificates, writes static Pod manifests to /etc/kubernetes/manifests/, outputs join token for workers
2. kubeadm join (Worker Nodes): Authenticates with secure token, fetches cluster config, installs certificates, registers kubelet with API Server

Kubeconfig File

YAML file containing cluster connection details, client credentials, and context mappings. Default location: ~/.kube/config. Structure:

• clusters: API server URLs and CA certificates
• users: Client credentials (certificates, tokens)
• contexts: Cluster + user pairs for switching between deployments

Swap and Resource Accounting

Kubernetes scheduler places pods assuming memory allocations are strict and deterministic. If the kernel swaps memory to disk, resource accounting becomes inaccurate and performance degrades unpredictably. Disabling swap ensures pods remain bounded to physical memory.

Comparison Table

Command & Configuration Reference

Minikube

minikube start --nodes 3
minikube status
kubectl cluster-info --context minikube

KIND

# Create multi-node cluster from config
kind create cluster --config kind-config.yaml --name dev-cluster

# kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker

kubeadm (Production)

# On control plane node
kubeadm init --pod-network-cidr=10.244.0.0/16

# Outputs join command to stdout; save it
kubeadm token list  # View available tokens

# On each worker node
kubeadm join <API_SERVER_IP>:6443 --token <TOKEN> \
  --discovery-token-ca-cert-hash sha256:<HASH>

Cluster Verification

# View cluster info
kubectl cluster-info --context kind-dev-cluster

# List all nodes
kubectl get nodes

# Check control-plane component status (deprecated but informative)
kubectl get componentstatuses

# View all system pods (CoreDNS, kube-proxy, CNI, API server, etcd, etc.)
kubectl get pods -n kube-system

# Inspect kubeconfig
kubectl config view

Disable Swap (Required)

swapoff -a
# Make persistent by commenting swap line in /etc/fstab

Common Pitfalls

Pitfall: NotReady Nodes After kubeadm init After kubeadm init, all nodes show NotReady and CoreDNS pods stuck in Pending.

• Why: Kubernetes doesn’t auto-install a CNI (Network Plugin). Without CNI, pod-to-pod networking is broken.
• Fix: Install a CNI plugin immediately—kubectl apply -f https://docs.projectcalico.org/manifests/tigera-operator.yaml (Calico) or use Cilium, Weave, etc.

Pitfall: Swap Not Disabled Before Cluster Join Worker nodes fail to join or system becomes unpredictably slow after join.

• Why: kubelet’s cgroup memory limits don’t work correctly with swap; scheduler’s placement decisions become inaccurate.
• Fix: Run swapoff -a before kubeadm init and kubeadm join. Edit /etc/fstab to persist.

Pitfall: Invalid or Expired Join Token kubeadm join fails with auth error; worker cannot reach control plane.

• Why: Tokens expire after 24 hours by default. Network connectivity or firewall rules may block port 6443.
• Fix: Generate new token on control plane: kubeadm token create --print-join-command. Verify network connectivity to API server: nc -zv <API_IP> 6443.

Interview Questions

Q — Compare Minikube and KIND for local Kubernetes development. When would you use each? Minikube runs a single-node cluster in a local VM, slower to boot and heavier on resources, good for general learning. KIND runs multi-node clusters as Docker containers, boots in seconds, minimal overhead, ideal for CI/CD pipelines and integration testing. Use Minikube for exploring features; use KIND for testing multi-node scheduling, affinity rules, and Helm chart deployments in CI.

Q — Walk through the kubeadm cluster bootstrap process. What happens during kubeadm init and kubeadm join? kubeadm init validates system requirements, generates a CA and client certificates, writes static Pod manifests for control-plane components (API server, etcd, scheduler, controller manager) to /etc/kubernetes/manifests/, and outputs a join command with a secure token. kubeadm join on worker nodes authenticates using the token, fetches cluster certificates, and registers the kubelet with the API Server, establishing trust for the node to join the cluster.

Q — Why is swap disabled in Kubernetes, and what happens if you forget? The Kubernetes scheduler makes placement decisions assuming memory allocations are hard limits. Swap breaks this assumption—if memory pressure causes the kernel to swap to disk, performance degrades unpredictably and resource accounting becomes inaccurate. Nodes become slow or unresponsive, breaking the scheduler’s ability to guarantee pod performance. Always run swapoff -a before kubeadm init and persist in /etc/fstab.