As your idea finally begins to materialise into the beautiful app you imagined, you begin (or become more hands-on in) testing it. Generally not overlooked, as repercussions otherwise are well-known and often narrated as cases of exemplified failure.
Yet, time instills the fear of being beaten and urges one to finish development quickly by rushing through its testing stage.
It goes without saying that a hasty testing process is much more likely to miss many obscure errors, bugs or loopholes, leading to the wide release of an unstable app! That would be much much worse than a delayed app.
Due to my vast experience in testing, I’ve seen developers employ different methods that, in their opinion, was the most time-efficient. Yet, none of those methods was perfect, and they often faced similar problems, resulting from common mistakes. That led me to recognize some common mistakes in mobile app testing.
Mistake #1: Testing before knowing the purpose
Let’s start with the biggest mistake developers tend to make, probably due to their misappropriation of the principles of MVP (to learn ASAP) to testing procedures.
As software developers and testers, you probably know the importance of learning everything and have read in detail about Eric Ries’ model of Build and learn cycle. Common inferences from that would be to test ASAP to increase your learning pace. However, the key here is “learning”, and you can’t learn unless you understand the App’s and its tests’ purpose well. Hence, before you start testing, know the definition of your purpose. Further, define success fluently to make better inferences.
I further put this as the first testing mistake because without knowing the purpose, it’s quite likely that you would end up designing tests that don’t quite evaluate your App impartially. Your biases could end up ruining a lot of your time! One simple way to overcome this bias would be through understanding the core assumptions and purpose and recognizing your biases.
Mistake #2: Testing Everything, without priorities
Imagine that to create the perfect and flawless App, you decide to test all possible scenarios, even those that differ from each other only minutely and painstakingly. This isn’t efficient at all. Further, in most cases, it’ll be impossible.
Instead, you should test for major scenarios that differ from each other, explore them through different features and test their integration with the App’s core functionality.
Further, you should prioritize the scenarios and tests in order to streamline your testing process. Streamlining is not the same as being heedless or negligent, however, so be careful. So you’re trying to be more efficient, not skipping on major tests to save time. To avoid this pitfall, try to narrow down changes in the app as you move forward and focus on those. Use Context-driven testing to plan your testing activities to give you a better focus.
Mistake #3: Web vs Mobile testing
Well, a mobile App sure is a type of software, which might lead many people (especially those used to Web App Development and Testing) to use the same principles of Web/Software Testing in Mobile Apps.
One must understand that the mobile app is completely different from a web app. Where Web or computer software is designed to fit a bigger display with a relatively fixed location, Mobile Apps are designed for “mobile” use; mobile users are almost always on the move, and your App Development and Testing process must keep that in mind.
For instance, testing for mobile applications must keep the following things in mind (which can be safely ignored for web):
- Range of mobile devices
- Different mobile operating systems
- Different versions and updates of iOS, Android, etc
- Different mobile carriers
- Memory and Battery usage for different mobile systems
Users could want to use their Mobile Apps while eating, during sports activities, while traveling, at work, etc. Further, a mobile app can take advantage of special hardware features present in mobiles. Keeping these and the different environments in mind will lead much more useful and holistic tests.
Mistake #4: Only focusing on the looks, or the UI
No one would deny the importance of UI for a mobile app. It’s the first things users see, and the first thing they feel when they use it after all. And as everyone knows first impression matters.
However, mobile apps are more than just a shiny UI. Mobile Apps use plenty of APIs, and you must make a note of checking each one out. How does the API deliver data on different requests? Is the API delivering the correct response to a request?
Besides the APIs, mobile apps also rely on many backend services. It is definitely worth checking out these backend functions too. How will the backend handle slow requests in cases of low bandwidth or network issues? Further, the network itself has different consequences for the Apps, hence Apps must be tested in various types of networks, with different bandwidth and capacities. How well do these functions integrate with hardware features?
Mistake #5: Having no methodology or plan
This relates to the first mistake I mentioned: not knowing the purpose! While you must always know the purpose in detail, you must also be aware (albeit vaguely in the initial stages) of the types of tests you’d need to do.
Plan for performance testing during the pre-production stage. Be able to define the level of quality assurance and performance criteria during the design stage itself. Establish the key performance indicators (KPI). Design your service level agreements before the production. Fix the methodology for testing and schedule all activities related to performance beforehand. Make sure your testing process is well planned and on schedule.
Further, when you test, you must be aware of the guidelines of different platforms. It doesn’t matter if you’re building the App for iOS, Android, or Windows Phone, each platform has its own set of guidelines that you should know. You should follow these guidelines to root common development mistake out. Integrate your plan with these guidelines whenever possible.
Mistake #6: Analysing each test in isolation
Trying to understand the results of a test without understanding the context or the implications and interferences of other tests/ scenarios in it would never give an accurate understanding.
As you start testing and move on from one test to another, you don’t abandon the first one’s implications and connections to the second test. Try to do regression testing.
Continuous regression testing is much more likely to assure the relevance of your tests, and quality of your applications.
Make use of data generation tools and include production data to give more realistic input to the tests.
Pay attention to each test and analyze them for anomalies and errors. You should also use monitoring tools to report on anomalies and correlate the statistics in various formats.
Mistake #7: Not testing in different versions of OS, phones etc
This type of testing involves only testing in one type of Android device or one iOS device. But we all know how fragmented the mobile market today is, in terms of hardware and software combinations. This type of testing saves money, but it doesn’t give a true measure of the App’s performance.
It’s very important to test on various mobile devices to have a higher coverage of devices: this means testing on various versions of iPhones, iPads, or Android devices. But, still, try to focus on the ones that are most typically used by your target audience.
To avoid that, do the following:
- Use the technologies that will allow you to distribute your testing into the cloud and give it a more real world experience.
- Avoid testing from a single location within closed firewalls.
- You can also use network emulation tools and create virtual user groups for more efficient performance testing.
Mistake #8: Network testing problems not checked
As mentioned in a few points above, mobile apps are used in various circumstances, on the move, with or without WiFi, and in different types of mobile networks.
You can only test for network issues when you go out in the field to test the Apps in a variety of scenarios and situations. Some bugs might only manifest in certain types of network conditions, but it’s nevertheless important to find and root them out.
Mistake #9: Performance issues not addressed
Try to run performance tests regularly throughout the development process and make sure you are going in the right direction.
Test every single unit, your infrastructure, and the database regularly. Don’t wait for an entire App to be completed before starting your performance testing.
This might also show itself in the form of ignorance of intermittent problems. Performance errors that occur infrequently and recover quickly are often easily and swiftly ignored. But, by not digging further, and not finding out the root cause of such problems, you might miss an opportunity to catch a systemic problem early on before it turns into a huge issue.
Going into traffic logs to debug such performance issues might seem like a chore, but in the long run, it’s worth doing.
Mistake #10: Syncing tests not done
In the age of cloud, App testing must involve how well it syncs across devices, how safe it is, and how long it takes.
Further, how easily can the App be updated? Have you tested the update and installation processes? Does it integrate well with cloud without erasing important data?
Syncing further brings the issue of security testing. Most of us will agree that security is a very important area for any type of software or App, but it’s not the easiest test to conduct.
If you don’t have a lot of experience in this type of testing, make sure you talk to experts and ask them in detail about security issues related to your App. There are various lists on the Internet, that offer an overview of common potential security issues in software. But don’t use them as a guidebook, instead, use them to question your app and its architecture to determine the potential risks. Don’t wait to address security issues until something particularly damaging has happened.
This was no means a definitive or an all-inclusive list of testing mistakes, for many unknown loopholes might still await an impatient or hasty developer.
Testing is done with several monotonous iterations of performing the same activities to ensure the desired result is achieved a significant number of times. Only then, would you be able to say that your objective of delivering a quality product is fulfilled.
The above list will be incredibly useful for a tester to avoid the common traps at all stages of testing, hence making his testing process and reports more significant and useful to a developer. In the end, it should lead to the production of a better App in less time.
Abhinav Prasoon is the Quality Lead at Alphalogic. People have described him as almost a perfectionist because his testing methods are impeccable. When he’s not busy with work, he is cherishing his love of chocolates, novels, and adventures.