2. Screen Size and Density
In comparison to the extremely controlled nature of iOS, the combination of screen sizes and screen densities in the Android universe add an extra challenge to app testing. Helpfully, and despite there being more than 200 distinct devices, Android classifies all official devices into one of four screen sizes and one of four screen density classes.
- Screen sizes: Small, Normal, Large, Extra Large
- Screen Density: Low dpi, Medium dpi, High dpi, Extra High dpi
Since testing on all devices is nearly impossible, these classifications will give testers and developers a good idea of how the app will appear on devices within the same screen size/density category. A recent breakdown of active devices looks like this:
|Low dpi||Medium dpi||High dpi||Extra High dpi|
As you can see, the most popular devices are squarely in the normal screen size range and generally sport either high screen density or medium density. This data is updated fairly frequently on the Android Developers website, so check it often as more devices hit the market.
3. Platform Versions
Android supports and tracks ten platforms/versions ranging from “Cupcake” 1.5 to “Ice Cream Sandwich” 4.0.4 (I am slightly outdated). Not all devices support all platform versions and new versions are not released to all handset makers at the same time. Instead, new releases are dripped out in increments, often leaving users eagerly waiting.
Despite being the most recent version, Ice Cream Sandwich is only the third most used platform and still has not been rolled out to all devices. “Gingerbread” 2.3.3 continues to dominate Android devices with 64% of use and “Froyo” 2.2 comes in second at 19%. Ice Cream Sandwich comes in third by only 1% (over “Éclair” 2.1).
By testing an app exclusively on the latest version you will isolate an extremely high number of users. Conversely, if you do not update an app to work consistently on newer releases you may lose current users as they upgrade. With so many platforms present on Android devices it’s important to test apps on at least the top three versions – if not the top four or five. To pinpoint the most popular versions visit Android Developers Platform Versions page, which tracks devices active in Android’s Google Play market.
WHAT TO TEST
As with all mobile app testing, common functions should be tested on as many devices as possible to ensure consistency. However, Android’s device and platform combinations present new challenges – a feature that functions perfectly on one device may cause a bug on another. In addition to normal testing considerations, there are a number of recurring issues that commonly crop up on a variety of Android devices. These issues are prevalent enough that they should be added to test cases on as many devices as possible.
These functions may seem like no-brainers when it comes to testing, but skipping one can spell doom for an app.
a. Registration & Login: Testing should make sure the registration and login processes are intuitive, easy to complete and functional from start to finish. It is important to have testers on a variety of devices complete the entire registration and log-in process to ensure everything runs smoothly and no error messages occur.
b. Menu Options: Often times, menu options can be difficult to access and decipher. Make sure that menu items like Help, About, etc. are easy to find and navigate. This is especially important to test on a variety of screen sizes and with real users, since fingers are much larger than a mouse on an emulator.
c. Keys: Any problems related to scrolling, text selection, the back button, etc. are bound to lead to trouble, so make sure your key functionality is clear and consistent. Also, be sure to check that the app will function consistently using both a physical keyboard and touchscreen.
d. Interruptions: How does the app behave when the device battery is at full strength, medium strength and low strength? What about if the user gets an incoming call or text? If there is another app running in the background? These are all real life scenarios that users are going to encounter. Don’t let them take down your app. (Crash logs are particularly helpful in diagnosis if other events are adversely effecting an app.)
e. Error Messages: Your error messages should be clear, concise and actionable. “Error 12a26q” may make sense to the developers, but it doesn’t help users know what went wrong or how to fix it. Make your error messages easily understandable and you’re a step ahead of virtually all mobile applications on the market today.
f. Landscape v. Portrait: An app’s functionality and usability should not be affected when changing from portrait to landscape mode. Test to be sure buttons, fields and menu options are easily accessible and functional in both situations.
g. Settings: Change a device’s settings and repeat necessary tests to ensure an end-user’s custom settings won’t affect the app’s performance.
Also be sure to check an app’s effect on battery life. Many users expect a phone’s battery to last an entire day, at minimum. And with phones performing more and more tasks, battery life is stretched ever thinner. If an app sucks more than its fair share of power it will be ditched by users.