.png){kind=icon}
Each time you develop a new mobile application it is important to verify it displays and works as designed on various types of devices and screen resolutions. Developing a mobile application for iOS and Android requires you to run your tests across multiple devices and OS configurations. Testing an app involves multiple challenges. The most common solutions for developers are to have their apps tested manually by a team of professionals or to use emulator and simulator programs. We looked at the the advantages and disadvantages of real devices testing vs emulators.
Testing on Emulators and Simulators
Often, people are confused about the difference between an emulator and a simulator. Emulators and simulators both allow you to run software in an environment however they do work differently. Emulators mimic both a mobile device’s software and hardware. On the other hand simulators only mimic the internal behavior of the device and not the hardware. One of the main downfalls of an emulator is that it’s typically slow. A simulator runs much faster but doesn’t provide as accurate results about how your app will work on a device.
Testing on an Android emulator does not require a complicated setup. Android actually offers an emulator download on their website: https://developer.android.com/studio/run/emulator.html
You are able to simulate many capabilities of various devices, however there are some hardware functionalities it does not support.
For iOS apps, Xcode 9 has a built-in simulator app. Xcode allows you to run multiple instances at once, and you can also use it to run automated tests. You can find a list of tips and a help page on Developer.apple.com.Simulators and emulators are useful In the early stages of development, when you are first testing your app features. However, there is a list of scenarios you won’t be able to test using a simulator or emulator:
1. Memory issues
2. Image colors
3. Push notifications
4. Connectivity testing
5. Geo-location
6. Speed and performance testing
7. Battery usage
8. Usability Testing
Real Devices Testing
Real devices testing gives you multiple advantages. You can interact with your application the same way an end user will. You can manually execute functional test scenarios and make sure all the features available in your mobile app work as expected on a physical device. You can also run some connectivity testing to make sure that your app responds well when you switch across different networks. And more generally you can see how your app responds in terms of performance when you test it.
Testing across different OS versions allows you to see if all the functionalities in your app are working correctly and you will be more likely to discover crashes or memory issues on a real device than you will on a simulator.
If you’d like to learn more about iOS and Android testing check out some of the recent blog posts we have shared.
While real devices testing is the best solution for uncovering bugs and getting accurate results, it means investing thousands of dollars in mobile devices. You also need to think about the time it takes to update devices, maintain a device library and remembering to charge devices regularly.
Over the years we have created a large device library and a team of in-house professional QA testers to address the two main issues of resources and devices. We’ve developed a system to track, maintain and test across hundred of devices. We are able to keep our device library updated all year round and eliminate that cost for our clients. If you are curious about our Test Lab visit our Lab page.
Using a device farm
Another solution for developers is to use a cloud-based solution such as a device farm. Some of these platforms connect your app to the crowd but you take a risk of exposing your designs and ideas to the world. And if you decide to use a cloud-based automation solution then in most cases you will need to have an automation engineer on your team to write and maintain the automated test.
Conclusion
As a testing partner of companies who build apps for millions of users across the globe we know and understand the importance of testing on real physical devices. Simulators and emulators are a good solution in the early stages of development but you cannot rely solely on these tools if you want to deliver a high quality application. If you decide to use a cloud-based testing solution review all the different options in detail prior to moving forward.