Code Quality

TL;DR

• Code coverage measures which lines/branches executed during tests, not whether those tests assert meaningful behavior — treating coverage as the goal itself (not a proxy) leads to gaming.
• Line coverage (a line executed at all) vs. branch coverage (both true/false paths of conditionals tested) can differ significantly; 100% line coverage doesn’t guarantee 100% branch coverage.
• SonarQube distinguishes code smells, bugs, and security vulnerabilities by severity; treating all three identically in a quality gate (zero-tolerance) leads to either tool bypass or alarm fatigue.
• “New code” quality gates enforce standards only on newly written/modified code, preventing debt accumulation without demanding infeasible retrofits of large legacy codebases.
• Comparing teams by aggregate quality score is misleading; the score reflects codebase age/complexity/risk more than actual engineering quality.
• Biggest gotcha: optimizing for a metric (coverage percentage, quality score) instead of what the metric was meant to proxy (genuinely tested, maintainable code).

Core Concepts

Code coverage: proxy vs. target

Code coverage measures which lines or branches executed during tests; it does not measure whether tests assert meaningful behavior. A hard gate requiring 90% coverage, treated as the goal itself rather than a proxy for “genuinely well-tested code,” predictably produces tests that execute code without verifying it — coverage climbs, actual test quality doesn’t. This is a well-known failure mode of treating a proxy metric as the target: the metric can be satisfied without the underlying goal (confidence the code works) being achieved.

Line coverage vs. branch coverage

Line/statement coverage counts a line as “covered” if it executed at all; it says nothing about whether every logical path through a conditional was exercised. A function with if/else where only the if branch is tested can show 100% line coverage while the else branch (where the bug might be) was never verified. Branch coverage specifically tracks whether both paths of each conditional were exercised. At the same reported percentage, branch coverage is a meaningfully stronger signal than line coverage, because it catches untested conditional paths.

SonarQube severity tiers

SonarQube categorizes findings into code smells (style/maintainability), bugs (functional correctness), and security vulnerabilities (security), with genuinely different severity and risk implications. Treating all three identically in a strict gate (“zero findings blocks merge”) backfires: it becomes impossible to satisfy (trivial style issues block) and gets bypassed, or teams ignore the tool (alert fatigue) and miss real vulnerabilities.

“New code” quality gates: pragmatic enforcement

The “new code” pattern enforces quality standards only on newly written or modified code, not retroactively on entire legacy codebases. Requiring a multi-year-old codebase to instantly meet a new quality bar is often infeasible and stalls adoption. Focusing enforcement on new/changed code prevents further debt while allowing incremental improvement of existing code — a pragmatic balance between preventing future debt and avoiding an impossible retrofit.

Context matters in team comparison

Comparing teams by a single aggregate quality score ignores context: codebase age, domain complexity, prior investment, inherent risk profile. This risks punishing teams working on legitimate older or harder code while rewarding teams on newer or simpler codebases for no real quality difference. The score measures context more than quality.

Meaningful assertions in tests

A test that executes code without asserting on the result contributes to coverage numbers but not to actual confidence the code works. “Meaningful assertion” means the test verifies expected behavior, not just that the code doesn’t crash.

Line Coverage vs. Branch Coverage: Key Differences

Configuration / Command Reference

# SonarQube: scan code and report findings
sonar-scanner \
  -Dsonar.projectKey=my-project \
  -Dsonar.sources=src \
  -Dsonar.host.url=http://sonarqube:9000 \
  -Dsonar.login=<token>

# Configure SonarQube quality gate with differentiated severity
# In SonarQube UI:
# Quality Gates → New → set conditions:
#  - Security Hotspot: > 0 → Fail
#  - Critical bugs: > 0 → Fail
#  - High severity code smells: > 5 → Fail (not zero-tolerance)
#  - Coverage: < 80% → Fail (context-based, not 90%)

# "New code" quality gate in sonar-project.properties
sonar.qualitygate.waitForQualityGate=true
sonar.newCodeDefinition=previous_version  # or specific date

# JaCoCo (code coverage for Java)
# Maven: add jacoco-maven-plugin to pom.xml
# Output: target/site/jacoco/index.html shows line and branch coverage

# Python coverage with branch tracking
coverage run --branch -m pytest tests/
coverage report -m  # shows line and branch coverage

Common Pitfalls

• Pitfall: Engineers write tests that execute code without meaningful assertions to hit a coverage target. Why: Coverage treated as the goal itself rather than a proxy for genuine test quality. Fix: Spot-check high-coverage tests to verify they assert meaningful behavior; treat coverage as a signal to investigate, not a target to optimize.
• Pitfall: A 100% line-covered function has an untested else branch containing the actual bug a customer reports. Why: Line coverage doesn’t require both conditional paths to be exercised; branch coverage does. Fix: For genuinely critical code paths, use branch coverage specifically, not just line coverage.
• Pitfall: A zero-tolerance SonarQube gate (block on any finding) is disabled within weeks because trivial style issues block every merge. Why: Undifferentiated severity treats minor code smells the same as security vulnerabilities. Fix: Configure differentiated thresholds (fail on security/critical, track but don’t block on style).
• Pitfall: A large legacy codebase’s SonarQube adoption stalls because the initial gate demands retroactive compliance from code nobody has budget to rewrite. Why: Infeasible all-or-nothing retrofit. Fix: Adopt a “new code” gate to prevent future debt while allowing incremental improvement.
• Pitfall: Teams compared and ranked by aggregate quality score, demoralizing the team working on the organization’s oldest, hardest codebase. Why: The score reflects context (age, complexity) more than quality. Fix: Account for codebase context when evaluating quality; avoid rankings based on a single score.

Interview Questions

• A team hits a 90% coverage requirement but bug reports haven’t decreased. What’s likely happening, and how would you investigate? — Tests whether coverage-gaming (tests with no meaningful assertions) is the correct hypothesis.
• Explain line coverage vs. branch coverage with a concrete example where they diverge. — Tests mechanical understanding of the distinction and why it matters for real bugs.
• How would you roll out a SonarQube quality gate on a 10-year-old, 2-million-line legacy codebase without stalling development? — Tests whether a new-code-focused, incremental gate is the pragmatic approach.
• A team’s quality score is lower than peer teams’ scores. What context would you want before concluding their code quality is worse? — Tests awareness that scores reflect context as much as actual quality.