Native Android apps and web apps need to be tested on multiple phones and tablets before being placing into production. Period. Sure you can use the emulator to do some very basic UI sizing validation, but there is no substitute for testing on actual devices.
Typically, I have one phone with the latest version of the OS and the rest of my phones are WiFi-only and have no carrier agreement. Buying used devices with no cellular contract is one way to save money, and can help budget strapped shops to do more thorough cross-device quality assurance (QA) tests. Obviously there are too many different types of Android devices on the market for the typical shop to have all of them in house, but having a few different models and versions gives you a huge QA advantage.
Why not use the out-of-the-box emulator? I’m not saying you shouldn’t use the emulator at all. It can be a valuable tool for check UI differences across various screen sizes and resolutions. But, in my opinion that’s about it. Personally I don’t use the out-of-the-box Android emulator much at all for daily coding since I almost always have a phone handy. Here are a few things that an emulator cannot help you test:
- Exact look and feel
- Application load times
- Application performance
- UI performance (View transitions, orientation changes, scrolling)
- Browser differences (for web apps)
- Subtle differences in UI touch events
- Real-world GPS testing
- Battery usage/consumption
- Unique characteristics from the handset manufacturer and carrier.
My final beef with the Android emulator is it is awfully slow even on powerful laptops. It’s simply not a feasible test platform if you have to do multiple builds in a reasonable amount of time. Waiting for the emulator to load can be a painful experience. Granted, under some circumstances you can leave a single instance of the emulator running and push multiple builds to it, it can still seem sluggish. From a productivity standpoint, my experience has been its cumulatively much faster to simply push builds to a phone. And then, as mentioned above, reserve the emulator for specific instances of testing screen sizes and resolutions.
Recommended bare-bones set up. I don’t have a particular manufacturer recommendation because that’s a personal choice. My bare-bones recommendations are more focused on Android SDK version and having at least one phone with the major versions. Since Donut-based v1.6 phones now represent 0.2% or less of all phones I no longer recommend testing on it. Having a v2.x phone is a really good idea for general distribution apps as it’s a fact that roughly 50% of the phones in circulation are still running it:
- V2.x smartphone
- V3.x smartphone
- V4.x smartphone
- V4.x tablet
If you have experience with testing across different versions please post your comments!
References