The right device count depends on your app's actual analytics on device and OS version distribution among real users, not a generic industry number, so the first step is pulling that data from your existing crash reporting or analytics tool before sizing anything. A reasonable starting approach covers the top handful of device models and manufacturers by usage share, generally including at least one recent flagship and one mid-range or budget device per platform, since performance and rendering differences show up most between those tiers rather than between similar flagship models. Screen size and density coverage matters more than raw device count for catching layout bugs, so include at least one small phone, one large phone, one tablet, and if relevant to your user base, one foldable, rather than several phones of similar size. For automated regression testing specifically, most of that coverage can run on emulators and simulators configured to match those same screen sizes and OS versions, reserving a small number of physical devices for the hardware-specific cases emulators cannot fully replicate. Revisit device coverage roughly twice a year as new OS versions and device categories gain meaningful market share. Nanobase AI, an NVIDIA Inception Program member, sizes Mobile Test Lab coverage against a client's actual user device data rather than a generic device list.
The matrix explodes before the device count does
A single app targeting two platforms, three OS versions each, four screen classes, and two network conditions already produces dozens of theoretical combinations before anyone adds a second locale. Buying enough physical hardware to cover that matrix directly is neither affordable nor necessary, because most of those combinations differ only in software-visible dimensions (screen size, density, OS version) that an emulator or simulator reproduces accurately. The right question is not "how many devices" but "how many of these combinations actually require physical silicon," which is a much smaller number.
A three-tier device model
Structuring coverage by test tier, rather than by a flat device list, keeps cost proportional to what each tier is actually trying to catch.
| Tier | Purpose | Typical mix | Run frequency |
|---|---|---|---|
| Smoke | Catch build-breaking regressions fast | 1-2 emulator/simulator profiles per platform | Every commit |
| Regression | Catch functional and layout regressions | 6-10 emulator/simulator profiles spanning screen sizes and OS versions | Every pull request or nightly |
| Release candidate | Catch hardware-specific and real-world issues | Small physical device set plus full emulator matrix | Before each release |
Each tier exists to answer a different question, so collapsing them into one flat device list defeats the point of tiering at all.
What actually forces a physical device
A handful of categories consistently need real hardware rather than a virtual profile: true camera sensor and ISP behavior, GPS signal characteristics in motion, thermal throttling under sustained load, carrier-specific radio and SMS behavior, and biometric sensor false-accept and false-reject rates. Everything outside those categories, including most layout, navigation, and functional logic bugs, is reliably caught on emulators and simulators configured to match real screen and OS profiles. If a bug class is not on this short list, buying a physical device to catch it is usually the wrong fix.
Sizing against your own usage data, not a generic list
Pull device and OS version distribution from your existing crash reporting or analytics tool before naming any specific model. A generic "top 10 devices" list from an industry report will not match your actual user base, and sizing against the wrong distribution wastes both hardware budget and CI time on device profiles few users actually run. Revisit this distribution roughly twice a year, since flagship refresh cycles and new OS releases shift it measurably over that timeframe. A device farm sized against someone else's user base is really sized against nothing at all.
Frequently asked questions
Do we need a physical device for every screen size we support?
No. Screen size and density differences are almost entirely reproducible on emulators and simulators using platform-provided device definitions. Reserve physical hardware for hardware-behavior testing, not layout testing, since layout bugs show up identically on a correctly configured virtual profile.
Should the release-candidate tier run on every release?
Yes, ideally, since it is the tier most likely to catch hardware-specific regressions that automated emulator suites cannot. If release frequency makes a full physical pass impractical, rotate a representative subset of the physical fleet across releases rather than skipping the tier entirely.
How does foldable and tablet support change device count?
It adds a device category, not just more units, since foldables introduce fold and unfold transitions that a flat-screen device does not have. See our guide to testing foldables, tablets, and screen sizes for how to handle that dimension.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, sizes Mobile Test Lab coverage against a client's actual device and OS analytics rather than a generic device list, splitting work across smoke, regression, and release-candidate tiers so physical hardware spend stays proportional to what emulators genuinely cannot cover. The result is a coverage plan sized to a client's real users, not a generic device catalog. This pairs with our broader AI Mobile Test Lab approach and our work on self-hosted device farm economics.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.