Jenkins

TL;DR

• Jenkins is a self-hosted, plugin-extensible automation server; pipelines are defined as a Jenkinsfile (Groovy, Declarative or Scripted) checked into the application repository.
• Its biggest strength — the plugin ecosystem — is also its biggest operational risk: plugins have interdependencies, and an unpinned, untested update can break every pipeline that touches a shared dependency at once.
• Shared Libraries eliminate copy-pasted pipeline logic across repos; Kubernetes agents eliminate idle static-agent cost and peak-hour queueing.
• Credentials must go through the Credentials plugin — never hardcoded in the Jenkinsfile, since the file is versioned in Git and a hardcoded secret is permanently recoverable from history.
• Biggest trade-off vs. GitLab CI / GitHub Actions: maximum flexibility and plugin coverage, at the cost of running and maintaining the server, agents, and plugin set yourself.

Core Concepts

Installation and administration

Standing up Jenkins itself is straightforward; the operational discipline that matters long-term is plugin management. Plugins depend on specific versions of other plugins, and updating one can silently break others that share a dependency. The mature practice: pin plugin versions explicitly, test updates in a staging Jenkins instance before touching production, and use a plugin-installation-manager tool (e.g. jenkins-plugin-cli) to make the whole plugin set reproducible from a declared list rather than hand-managed through the UI.

Pipelines: Jenkinsfile as code

Pipeline logic (Declarative or Scripted syntax) should live in a Jenkinsfile in the application repository, never defined only through the Jenkins web UI. A repo-stored Jenkinsfile is version-controlled and reviewable via pull request, with a real audit trail of who changed the deployment logic and why. A UI-defined pipeline can be silently modified by anyone with Jenkins access, with no review and no diff — a governance gap in any environment that still has UI-defined jobs.

// Jenkinsfile — Kubernetes agent, staged pipeline
pipeline {
    agent {
        kubernetes {
            yaml '''
            spec:
              containers:
              - name: golang
                image: golang:1.22
                command: ['sleep', 'infinity']
            '''
        }
    }
    environment {
        DOCKER_REGISTRY = credentials('docker-registry')
    }
    stages {
        stage('Lint')  { steps { container('golang') { sh 'golangci-lint run ./...' } } }
        stage('Test')  { steps { container('golang') { sh 'go test -race -coverprofile=coverage.out ./...' } } }
        stage('Build & Push') {
            when { branch 'main' }
            steps {
                sh 'docker build -t myapp:${BUILD_NUMBER} .'
                sh 'docker push ${DOCKER_REGISTRY}/myapp:${BUILD_NUMBER}'
            }
        }
    }
}

Shared Libraries

Shared Libraries solve the duplication problem that appears once a team maintains more than a handful of repos: 40 Jenkinsfiles that each hand-implement the same lint-test-build-scan-push sequence are 40 copies of the same logic, and a change to the standard sequence means editing all 40. A Shared Library moves that common logic into a separate, versioned repository, imported with a single @Library('my-shared-lib') _ line at the top of each Jenkinsfile. A change to the shared logic propagates to every consumer (or every consumer pinned to a specific version) instead of drifting silently out of sync.

Agents: static vs. Kubernetes

Static, always-on agent nodes are the classic Jenkins scaling bottleneck: they sit idle most of the day (wasted compute cost) and become a hard queueing bottleneck during peak hours when demand exceeds the fixed pool. Kubernetes Agents, via the Kubernetes plugin, replace this with pods spun up on demand per build and terminated after — elastic capacity that scales with actual queue depth and costs nothing when idle.

Queueing and slowness are two separate problems with two separate fixes: Kubernetes agents fix queueing (elastic capacity), while parallelizing pipeline stages and caching dependencies fix slowness (actual build duration). A team with 45-minute builds that also queue badly during peak hours needs both fixes — applying only one leaves the other problem fully intact.

Credentials

Credentials must go through the Credentials plugin, never hardcoded as plain strings in a Jenkinsfile. The risk is concrete, not theoretical: a Jenkinsfile is versioned in Git, so a hardcoded password is permanently in commit history (recoverable even after later removal), visible to anyone with repo read access, and likely to leak into build console logs the moment the pipeline echoes an environment variable for debugging.

Declarative vs. Scripted Pipeline

Jenkins vs GitLab CI vs GitHub Actions

Command / Configuration Reference

// Reference credentials in a pipeline
environment {
    DOCKER_REGISTRY = credentials('docker-registry') // injects username/password or token as env vars
}

// Import a Shared Library
@Library('my-shared-lib') _        // pulls the default branch/version of the library
@Library('my-shared-lib@v2.1.0') _ // pin to a specific tag for reproducibility

// Parallel stages to reduce build duration
stage('Parallel checks') {
    parallel {
        stage('Lint') { steps { sh 'golangci-lint run ./...' } }
        stage('Unit') { steps { sh 'go test ./...' } }
    }
}

# Plugin management (jenkins-plugin-cli) for reproducible plugin installs
jenkins-plugin-cli --plugin-file plugins.txt

Common Pitfalls

• Pitfall: A plugin update breaks several unrelated pipelines. Why: Plugins have interdependencies on specific versions of other plugins, and updates are applied directly to production without staged validation. Fix: Pin plugin versions, test updates in a staging Jenkins instance first.
• Pitfall: Dozens of Jenkinsfiles with identical logic drift out of sync over time. Why: Pipeline logic was copy-pasted per repo instead of centralized. Fix: Extract a Shared Library and import it with @Library.
• Pitfall: Static agent pools sit idle most of the day while builds queue for an hour at peak. Why: A fixed-size agent pool cannot elastically match demand. Fix: Move to Kubernetes agents that scale pods with queue depth.
• Pitfall: A credential hardcoded in a Jenkinsfile is still recoverable months after being “removed.” Why: Git retains full history; deleting a line doesn’t delete the commit that added it. Fix: Use the Credentials plugin exclusively, and rotate any secret ever found in history.
• Pitfall: A UI-defined pipeline is modified with no review trail, causing an unexplained production deploy. Why: UI-defined jobs bypass version control and PR review entirely. Fix: Require every pipeline to be a Jenkinsfile stored in the application repository.

Interview Questions

• “Fifty developers, 45-minute average builds, queueing badly at peak hours. Walk me through your fix.” — Tests whether the candidate separates the queueing problem (elastic agents) from the speed problem (parallelization/caching).
• “Explain what a Jenkins Shared Library solves, with a concrete before/after.” — Tests whether the duplication problem and its fix are genuinely understood.
• “A hardcoded credential was found in a Jenkinsfile that’s been in Git for 8 months. What’s the full remediation, not just the immediate fix?” — Tests whether the candidate thinks beyond removing the line (rotation, history considerations, root-cause process fix).
• “When would you choose Scripted over Declarative Pipeline syntax?” — Tests understanding of the trade-off between structure/validation and raw flexibility.
• “Compare Jenkins to GitHub Actions for a team just starting out on GitHub.” — Tests awareness of operational overhead versus flexibility trade-offs across tools.