Reducing mobile test flakiness starts with replacing fixed sleep calls with explicit waits tied to real UI state, such as Espresso's idling resources or XCUITest's waitForExistence, so tests proceed only once the app is actually ready rather than after an arbitrary delay. Animations and transitions are a common culprit, so disabling system animations on the emulator or simulator and turning off unnecessary background sync and analytics during test runs removes a large source of timing variance. Network calls should be mocked or run against a stable staging environment rather than production, since real network latency is one of the least controllable variables in a CI run. Isolate test state between runs by resetting app data, clearing shared preferences or UserDefaults, and avoiding shared accounts or backend records that other parallel tests might mutate. Track flakiness with a quarantine mechanism that reruns failing tests a bounded number of times and flags intermittent failures for review rather than letting them silently pass or block every build. Nanobase AI tunes Mobile Test Lab pipelines to isolate and eliminate these timing and state issues so Android and iOS suites produce consistent, reliable results in CI/CD.
Flakiness is a process problem, not just a code problem
Individual fixes, an explicit wait here, a mocked network call there, only address symptoms if there is no process tracking which tests flake, how often, and who owns fixing them. Teams that get flakiness under control durably, rather than fixing the same three tests every quarter, build a lightweight governance loop around it: classify failures, quarantine repeat offenders automatically, and assign ownership rather than letting flaky tests silently erode trust in the whole suite. Without a tracking process, flaky tests get muted informally and coverage quietly degrades.
A flakiness taxonomy
| Category | Typical cause | Primary fix |
|---|---|---|
| Timing flakiness | Fixed sleeps instead of explicit waits | Replace with idling resources or waitForExistence |
| Animation flakiness | UI interacted with mid-transition | Disable system animations on emulator/simulator |
| Network flakiness | Real backend calls with variable latency | Mock the backend or use a stable staging environment |
| State flakiness | Shared accounts or unreset app data between runs | Isolate test data, reset state in setup/teardown |
| Environment flakiness | Under-resourced CI host, no hardware acceleration | Verify KVM/hardware acceleration is actually enabled |
| Order-dependency flakiness | Tests assume a prior test's side effects | Enforce test independence, run in random order in CI |
Classifying a failure by category before fixing it prevents applying the wrong remedy, like a sleep, to a state or environment problem.
A quarantine and ownership policy
- Any test failing intermittently (defined as passing on an immediate rerun with no code change) gets automatically flagged, not manually noticed.
- Flagged tests move to a quarantine suite that still runs and reports but does not block merges, with a visible dashboard of what's quarantined and since when.
- Each quarantined test gets an owner and a deadline, typically two sprints, to diagnose the root cause using the taxonomy above.
- Tests exceeding the deadline are escalated to a team lead rather than left indefinitely quarantined, since permanent quarantine is equivalent to deleting the test.
- Track quarantine count as a visible team metric so flakiness reduction has the same visibility as feature velocity.
A quarantine list with no owner and no deadline is just a slower way of deleting the test.
Why blanket retries hide the real problem
Automatically retrying every failed test two or three times before marking it a real failure is a common first response, and it does reduce noisy CI failures in the short term, but it also masks genuine intermittent bugs and lets flaky tests accumulate indefinitely since they never actually block anything. A bounded, visible retry policy combined with the quarantine tracking above is more sustainable than silent, unlimited retries, because it still surfaces the underlying flakiness for someone to eventually fix rather than papering over it forever. Unlimited silent retries and no process at all produce the same long-term outcome: nobody ever fixes the root cause.
Frequently asked questions
How much flakiness is normal in a mobile test suite?
There is no universal acceptable rate, but a useful operational target is treating any test that fails intermittently more than roughly once every ten runs as requiring investigation rather than tolerance, since flakiness compounds as suites grow and erodes trust faster than it appears to.
Should flaky tests block a release?
Generally no for tests already in quarantine with an active owner and deadline, but a newly introduced flaky test on a critical flow like checkout should block until diagnosed, since it may indicate a real, intermittent production bug rather than a test issue.
Does disabling animations affect what the test is actually validating?
Disabling system animations changes timing but not functional behavior, so it is safe for most functional UI tests. Keep a small separate suite with animations enabled if you specifically need to validate transition behavior or animation-triggered state changes.
Who should own fixing flaky tests, QA or engineering?
Whoever owns the code the test covers should own diagnosing flakiness in it, since root causes are often in the app's async handling or state management, not the test framework; a QA-only ownership model tends to produce workarounds rather than fixes.
How Nanobase AI helps
Nanobase AI tunes Mobile Test Lab pipelines around this kind of taxonomy and quarantine discipline, isolating timing, state, and environment issues so Android and iOS suites produce consistent results rather than noisy ones. We also build the dashboards and ownership workflow into your existing CI/CD pipeline so flakiness reduction is a tracked process, not a one-off cleanup. For Android-specific root causes, see our Espresso flakiness guide.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.