20 PRACTICAL ADVICE TO HELP YOU BETTER TEST YOUR MOBILE APPS
Obey the principles without being bound by them.
—Bruce Lee
Failure comes only when we forget our ideals and objectives and principles.
—Jawaharlal Nehru
Nowadays, the very first lesson that is taught to every mobile tester is “mobile testers needs to reflect a different mindset and mobile testing requires a clearly distinct approach”. Beyond any doubt, whether you establish a solid mobile testing framework yourself or someone does it for you, it will bring many benefits. Don’t worry: I will skip explaining you all the benefits because I believe you have heard it millions of times and have no space for a new bunch of assumptions. So let me give you some of the most essential mobile testing advice and try to help you in defining your testing strategies based on those. You may adapt these tips to almost every mobile testing project you deal with. Do not hesitate to use them if you are under pressure because of an upper manager who has no idea about mobile testing.
1. Do early testing by getting involved early in the projects and try to do as much static testing as possible. Project managers have a tendency to reject one or two days of static testing because of the time it consumes. On the hand, the same project managers are willing to postpone the same project for twenty more days because dynamic testing is not properly finished due to unverified bugs that could have been identified with static testing!
2. Do not test everything. Some testers are heroic and say that they need to test everything or test the product 100 percent. Please calm these people down and tell them that 100 percent testing is impossible. Rather, you may propose risk-based testing, which is a far better and achievable approach.
3. Dedication matters, so let mobile testers test your apps. I find it difficult to understand the fact that business analysts get offended when you say you need a tester to test any product. But when the time comes to testing, they try to stay away from it and say that they have lots of other high-priority tasks and it would be better if they focus instead on their analysis/design assignments. Ultimately, do not believe in crocodile tears; employ your mobile testers.
4. Testers are happy to find bugs because they are paid for that. Do not assign a different meaning to their pessimistic, suspicious, and questioning characters. They find defects for living, not for getting developers fired.
5. Defects do not stay in public areas; instead they like to hang out in forgotten places and live their lives on the boundaries. If you expect to detect a running horse mistakenly put on the splash screen of a mobile banking application, do not wait in vain. People would detect the horse before it appears to you.
6. Found defects are an indication of the presence of many others. Never ever say, “I found three defects in module X, so let’s test other modules. We are safe in module X.” In such case, remember ant nests. When you look at the nest from the top you see only a few ants. But when you dig into the ground, you see hundreds of ants. Defects are like ants. Most of the time they are invisible, and they stay and act together.
7. Defects are not always program malfunctions; sometimes they can be the things that just bother you. When you are testing a mobile app, you should not only seek functional deficiencies; sometimes defects can be the annoying, silly things rather than program malfunctions. Some developers like to say, “This is not a defect; it is working.” Do not believe in such illusions. Report the bugs.
8. In the mobile world, you need to do usability and performance testing more than in any other category. Users are impatient, and they are not primarily focused on functionality. Rather, they are in search of simple and user-friendly products.
9. Do not try to automate everything; user behavior should also be given priority in most cases. In the mobile world, any test case can be automated, but if you are looking for real feedback, you need to do manual testing. Don’t get upset; this is not the end of the world, and no one is promoting you if you automate some nonsense test case.
10. Do not praise your testers by the number of test cases they prepare in a given period of time. Instead, you need to assess them by the defects per test cases ratio. In this case, you should remember that bigger is not always better (at least in testing, it is not!). While testers are preparing test cases, they generally race against time. Consequently, preparing defect detectable test cases is much better than preparing hundreds of passing (developer lover) test cases. Encourage testers to detect more defects with fewer test cases.
11. Try to include some negative testing scenarios in your test suites. Happy-path testing will not find you defects after some time.
12. While you are doing usability testing, you need to soliloquize the phrase “more is less.” If an app contains many fields, lots of unused/hidden functionalities, and numerous controls, there is a large probability that it will make its users unhappy. Mobile apps should not create any paradox of choice while they are being used by their users. By the way, I suggest you read the book The Paradox of Choice by Barry Schwartz. If you do, you will understand the “more is less” phenomenon better.
13. Do not forget to perform claim testing, an activity checking whether the app store claims (app descriptions) are fulfilled by the app itself. We know many apps that claim they save the world and will show you the heaven, but that in reality suck. As a mobile tester, you need to reveal this as well.
14. Be sure to stay away from “dark side” while you are testing. Remember the movie Star Wars: Episode III—Revenge of the Sith and the character Anakin Skywalker. He was born to be on the good side (Jedi force), but after he passed to the dark side, he became Darth Vader. If you are a mobile tester, you need to report the bugs (even though they are to be fixed in the afternoon release), announce valid metrics (despite that they are pointing you to some unattended test cases), and be truthful. If you falsify any data or do not report a defect because you are in love with a developer, you enter the dark side. Unfortunately rest will be easier. Before you know it, you will become the Darth Vader of your organization. As Master Yoda said, “Do not let your feelings towards developers cloud your judgment”. Be careful!
15. Regrettably, the mobile world is evolving a lot faster than mobile testers. This means that talented and skillful mobile testers are needed more than ever, and the gap between the mobile world and testers will be widened unless organizations train their testers and make the necessary investments. However, if you are a mobile tester and you are waiting for your company to arrange the necessary training for you and continuously invest in your development, you are in an imaginary world. No one will invest in you more than you. So wake up and try to create your own challenge. Buying a brand-new smartphone is not always better than paying for mobile testing training. You need to do both.
16. Be sure that you are checking your test objectives and their alignment with the business requirements before you start any test activity.
17. Try to be dynamic and prepare yourself for time pressures, frequent changes, and budget limitations.
18. Adapt your test methodology to your SDLC approach. Testing is a purely context-driven activity, so it makes a significant difference if you are doing agile, waterfall, spiral, or v-model development.
19. A defect-free app does not mean that you are successful. The main success factor of any mobile app is the user satisfaction it creates.
20. And finally, there is the illusion of 100 percent testing. In no circumstance ever say, “we have tested the app 100 percent.” This is not possible! Even though you have tested a mobile app for years, no one can be sure that the product is tested 100 percent. During the training that I deliver, people sometimes object to this principle and say, “We can be sure that all the requirements are covered by our test cases. Current tools and technologies allow us to achieve 100 percent requirement coverage.” My answer is simple; 100 percent requirement coverage does not mean that the product is tested 100 percent. This is because the requirements are still being written by humans, and no one can be 100 percent sure that these artifacts are covering all the functional and nonfunctional aspects of any mobile product. So once you say, “All the requirements are covered,” it does not mean that 100 percent of the product is tested. At least wise people will distinguish the difference.
I think that the foregoing principles and strategies are important in mobile testing. Whether you agree 100 percent or not, a majority of them can help you in establishing your ideas, organizations, and processes, and they can assist you in making better decisions.
As Victor Hugo once said, “Change your opinions, keep to your principles; change your leaves, keep intact your roots.”