Using AI for test selection, often called test impact analysis, means analyzing which files or code paths a commit actually touched and running only the tests known to exercise that code, rather than running an entire suite on every change, which becomes increasingly valuable as a suite and codebase grow large enough that running everything slows down feedback loops. The underlying technique builds a dependency graph mapping test cases to the source files, classes, or screens they exercise, either through static analysis or by recording coverage during past runs, then intersects that graph with a commit's changed files to select a minimal relevant subset. Machine learning models can improve on pure static mapping by learning from historical data which tests have failed together with which kinds of changes in the past, catching indirect dependencies static analysis misses, such as a shared style resource affecting visual regression across unrelated screens. This approach trades some risk of missing an untested indirect dependency for substantially faster CI feedback, so most teams still run the full suite on a schedule or before release even while using selective testing on every commit. Nanobase AI, an NVIDIA Inception Program member, applies this kind of AI-driven prioritization inside its Mobile Test Lab to keep everyday CI runs fast while still validating full coverage before release.
The problem this solves, and the risk it introduces
Running an entire mobile test suite on every commit becomes a bottleneck once a codebase and suite both grow large enough that full execution takes longer than developers are willing to wait for feedback. Test impact analysis, or test selection, addresses this by running only the tests known to exercise the code a commit actually changed. The tradeoff is real: selective testing trades some risk of missing an untested indirect dependency for substantially faster CI feedback, which is why most teams keep a full-suite run on a schedule even while using selection on every commit.
Three techniques for building the selection
| Technique | How it works | Main weakness |
|---|---|---|
| Static call-graph analysis | Maps which tests touch which source files through code structure | Misses dynamic or reflection-based dependencies |
| Coverage-based mapping | Records which tests exercise which code during past runs | Requires an initial coverage-recording pass, drifts as code changes |
| ML-learned co-failure patterns | Learns from historical data which tests have failed together with certain changes | Needs a meaningful history of past failures to learn from |
Machine learning models can improve on pure static mapping by catching indirect dependencies static analysis misses, such as a shared style resource affecting visual regression across unrelated screens, since these relationships surface from historical failure patterns rather than explicit code references. None of these three techniques is strictly better than the others; most mature implementations combine static mapping as a floor with ML-learned patterns layered on top.
A safe rollout sequence
- Start with test impact analysis as an additive, informational signal only, running the full suite alongside it and comparing which tests the selection would have chosen against the full run's actual failures.
- Measure the miss rate over several weeks, meaning how often a relevant failing test would have been excluded by the selection.
- Move to selective execution on every commit only once the miss rate is acceptably low, while keeping a full-suite run on a nightly or pre-release schedule as a safety net.
- Re-evaluate the selection model periodically as the codebase and test suite evolve, since dependency mappings drift over time.
Running selection as an informational signal before trusting it operationally is what turns "we think this works" into a measured, defensible miss rate.
Why the full-suite safety net stays even after adoption
Selective testing is a speed optimization for everyday feedback loops, not a replacement for comprehensive validation before something ships. Keeping a full-suite run on a schedule or before release is what makes the risk of selective testing acceptable, since it catches whatever the selection missed before it reaches production.
Frequently asked questions
Does test impact analysis work well for a small test suite?
The speed benefit scales with suite size, so a small suite that already runs in a few minutes may not see enough benefit to justify the added complexity of maintaining a selection system.
How does this relate to parallelizing test execution?
They solve different problems and can be combined: test impact analysis reduces how many tests run, while parallel execution reduces how long the selected tests take to run. See parallel mobile testing and CI time for the parallelization side.
What happens when a test impact analysis system makes a wrong call?
It either excludes a test that should have run, risking a missed regression, or includes more tests than necessary, reducing but not eliminating the speed benefit; tracking the miss rate during rollout is how teams catch and correct this before fully trusting the system.
Can AI-driven test selection also help prioritize which failures to investigate first?
That is a related but separate use of AI in the pipeline, focused on triage rather than selection; see AI triage of failed mobile test runs for how that complementary technique works.
How Nanobase AI helps
Nanobase AI, an accepted member of the NVIDIA Inception Program, applies this kind of AI-driven test prioritization inside its Mobile Test Lab to keep everyday CI runs fast while still validating full coverage on a scheduled cadence before release. A full-suite run always stays on the schedule as the safety net, even after selective testing is trusted for everyday commits.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.