On August 1, 2012, Knight Capital's trading system sent 4 million erroneous orders into the market in 45 minutes, costing the company $460 million. The root cause had been sitting in the codebase for years: dormant code that became dangerous only when a later change brought it back into play.
The incident shows why passing tests is not always enough. A code change can affect parts of the system that were never touched directly, while the existing test suite still stays green.
To find those gaps before release, teams need to examine what changed, trace affected dependencies, and verify whether existing tests validate the new behavior. This helps uncover untested paths, changed logic, and downstream coverage gaps that a passing test suite may miss.
The first step is knowing what kind of gap a code change can create, because not every gap shows up in the same place or needs the same kind of review.
To find test coverage gaps before release, analyze what changed, trace affected dependencies, and verify whether existing tests validate the impacted behavior. A passing test suite or high coverage percentage alone cannot confirm that all changed behavior is adequately tested.
Code changes create new test coverage gaps when they add an untested path, change behavior that existing tests do not fully check, or affect downstream code that is not re-tested. The test suite can still pass if its existing tests do not exercise or validate the behavior affected by the change.
Coverage changes as the code changes. Tests that covered the relevant behavior before an update may no longer cover every path introduced or affected by it.
For example, a refactor adds new branching logic inside an existing function. The current tests still call the function and pass, but they exercise only the original paths. The new branch remains untested even though the function itself appears covered.
Known coverage gaps are usually documented in a backlog ticket, test plan, or TODO comment. But some gaps introduced by code changes go unnoticed. In this article, we refer to these as shadow gaps. Unlike known gaps, they have no existing record, so teams need to actively look for them when reviewing code changes.
To develop this article, I spoke with experienced software testing practitioners and drew on their firsthand experiences, real testing incidents, and published engineering postmortems. I focused on cases where teams were confident in their existing tests, but a code change still exposed an untested path in production.
Across these cases, I identified three recurring patterns: newly introduced flows, changed logic that existing tests did not fully validate, and downstream dependencies affected by a change. These observations informed the three gap types explored throughout this article.
Code changes can introduce three types of test coverage gaps: new-flow gaps, changed-logic gaps, and dependency coverage gaps. Each requires a different approach to detection.
Each type needs a different way of catching it. A new-flow gap shows up in a diff. A changed-logic gap requires reading what the tests actually assert, not just whether they pass, because code coverage measures execution, not whether the behavior is actually validated. A dependency coverage gap requires knowing what calls into or relies on the changed code in the first place.
To identify potential test coverage gaps after a code change, start by reviewing the code diff, tracing affected dependencies, and checking historical failure patterns. Together, these signals help narrow down which paths may need additional testing, even when the existing test suite passes.
Start with the diff and identify the files, functions, or modules changed directly. Then trace beyond the edit to identify behavior that may have changed indirectly.
Identify what calls, imports, or relies on the changed code. A function can pass its own tests while still changing the behavior of a service or workflow downstream. Agentic AI testing can help teams investigate affected code paths and flag potential coverage gaps for further review.
Check whether the affected area has failed after similar changes before. For example, a payment module with several past incidents after schema changes warrants more scrutiny when its schema changes again than an isolated utility with no comparable failure history.
The table below maps common code-change signals to potential coverage gaps and shows what teams should check next.
|
Signal from your diff or call graph |
Likely gap type |
What to check next |
|
New file, function, or branch with no existing test |
New-flow gap |
Write a targeted test for the new path. |
|
Internal logic changed, but the same tests still pass |
Changed-logic gap |
Check whether the existing assertions validate the new behavior, not just whether the function runs. |
|
A shared module, utility, or internal API changed |
Dependency gap |
Trace its callers and check whether their tests exercise the affected behavior. |
|
The area has failed after similar changes before |
Elevated risk |
Review existing coverage and add targeted testing where the previous failure pattern could recur. |
|
Isolated, low-traffic code with no comparable failure history |
Lower risk |
Assess whether monitoring is sufficient if no new flow, changed logic, or dependency gap is present. |
* This is triage, not a requirement to test every affected line. The aim is to spend additional test coverage where the change creates meaningful risk.
To verify a test coverage gap, check whether the affected path runs during testing, whether the tests can detect incorrect behavior on that path, and whether the path is reachable in production. Coverage shows execution, but it does not confirm that the behavior is actually being validated.
- Check coverage across test layers. Run coverage for the affected path across unit, integration, and end-to-end tests. A path with zero unit-test hits may still be exercised elsewhere in the test suite.
- Validate the behavior, not just execution. A test can execute a branch without asserting the behavior that matters. Review the assertions or use mutation testing to check whether introducing a controlled fault causes the relevant tests to fail.
- Rule out unreachable code. An uncovered branch may sit behind a disabled feature flag, an impossible condition, or obsolete defensive logic. Check whether the path could become reachable in production before deciding how urgently it needs testing.
If the result is still unclear, write a targeted test that drives the suspected path directly. This separates a genuine test gap from code that is already validated elsewhere or cannot execute in practice.
Once a gap is confirmed, the next step is deciding how to address it within the development workflow, before the change reaches production.
Rather than adding a separate review stage, build test gap analysis into the existing CI/CD workflow:
Making these checks part of pull request reviews helps teams assess coverage as code changes, rather than relying only on results from previous test runs.
A shadow gap is a coverage hole created by a code change rather than a known, tracked gap. It doesn't appear on any backlog or dashboard, and it's usually only found after something breaks in production.
A known gap is documented somewhere: a ticket, a test plan note, or a TODO comment. A shadow gap has no record. It exists because a change altered behavior, added a new path, or shifted a dependency, and nobody wrote a test to cover it yet.
Yes. Coverage percentage measures whether a line was executed during a test run, not whether the test actually validates the current behavior of that line. A changed-logic gap can sit inside code with full line coverage.
No. It's a triage step that runs on top of existing testing practices, used to decide what needs new test coverage before a release and what can ship with monitoring instead of a full regression pass.
Test impact analysis identifies which existing tests need to be re-run after a change. Test gap analysis identifies where no test exists at all for the affected path. Impact analysis tells you what to rerun; gap analysis tells you what is missing.
Prioritize fixing it when the affected path handles critical flows like authentication, payments, or data integrity. Monitoring is a reasonable call for isolated, low-traffic paths where a failure is quickly reversible.
Test coverage should be reassessed when a code change alters behavior, introduces a new path, or affects downstream dependencies. A passing suite or existing coverage score cannot, on its own, confirm that every path affected by the latest change is still adequately tested.
For each meaningful change, teams can start with the diff, trace the affected dependencies, and verify coverage across the impacted paths. Higher-risk gaps can trigger targeted testing, while lower-risk changes can be evaluated against the team's monitoring and release controls.
As CI/CD pipelines automate more change detection and dependency mapping, coverage review can become more change-aware, too. The goal is to evaluate coverage against what each release can affect, rather than carrying forward confidence from the previous version of the code.
Want to automate test coverage as your code changes? Compare the best automation testing tools on G2, based on verified user reviews.