USER ACCEPTANCE TESTING HAS MANY MEANINGS

If you are irritated by every rub, how will you be polished?

—Mevlana C. Rumi

As is evident from its name, a user acceptance test (UAT) is an evaluation of a software product carried out by its target users. In order to see that the software product is functioning as required, user acceptance tests follow patterns for validating the user requirements especially.

After this formal definition, I want to expand on UAT’s meaning by adding more perspectives. Is it an activity of finding bugs, or is it an activity of gaining confidence that the product meets business needs?

From an IT test department’s perspective, it is an additional test phase for finding bugs that were missed in the earlier test levels. On the other hand, if you ask other IT departments (also the business departments), they would say that UAT is an activity of gaining confidence. Both views seem logical, but test departments need to know that UAT is not specifically done for correcting their deficiencies. A test department’s main goal should be that all the test levels before UAT are to be carried out with perfection, and after system testing these people should feel ready to go live without UAT.

The main purpose of a UAT should be having users’ real feedback and vision about a software product and allowing them to interact with the system before it goes live. Uncovering the unidentified defects from previous test activities is a nice-to-have feature of a UAT. That’s why we ask users to prepare their own user test scenarios and to not repeat the ones that were carried out before by the IT test department.

This is very important because if you rerun system tests in UAT, there is no point at which the users are involved. If these people reflect their daily habits and points of view on the test scenarios, then we can talk about a valuable UAT. Otherwise, UAT will be an just extension of system testing.

When we investigate the users’ behaviors who are participating in the UATs, it’s another story. We see several patterns. Some users attend UATs only to recommend new features, others attend to just manipulate/change how a system is designed, and others attend for nothing but to follow the test cases distributed by the IT test departments. All these user types are common in any UAT, and IT test departments need to be careful about selecting, handling, guiding, and monitoring these people. I accept that IT test departments’ responsibilities in a UAT can be seen as supportive responsibilities rather than primary responsibilities, but still we are the ones who should take care of this activity and finally receive the UAT sign-offs. This way we can roll out the software product and hand over our responsibilities to operations departments.

Before I forget, another special user type you may observe in a UAT is the emotional user. These users are attending UATs for “liking” the products instead of “accepting” them. Since liking is more intuitive and feeling based, it harder to “like” anything than “accept” it. If any product meets your basic needs, you may accept to use it, but this doesn’t mean that you will like it. Liking is an emotion that is harder to achieve and easier to lose. So if you have emotional users, explain to them the real meaning of user “acceptance” tests and guide them to be more process-driven, analytical, neutral, and objective in their test activities.

You should always remember that UAT users generally do not have any software background or solid technological knowledge. They are not 100 percent booked for UATs, and they have their ongoing business responsibilities. Testing is not their main responsibility, and they can be unaware of any testing techniques and tools or any defect-tracking software. And finally, they are outsiders! They are not directly reporting to you (to testers), and they represent the customer side.

Under these circumstances, you might as well prepare yourself for the UAT battle. Get ready for facing different perspectives, different attitudes, different targets, different motivations, different skills, and different outcomes.

You may answer the following questions first:

  • Who will attend the UAT?
  • What are their backgrounds/skills?
  • What is the scope of the UAT?
  • Which mobile platforms will be tested?
  • Which mobile devices will be tested?
  • How will the test devices be managed?
  • What deliverables are expected to be produced?
  • What is the entry and exit criteria of the test?
  • What kind of test data will be used and how?
  • Which tools will be used for test management and defect tracking?
  • How will the users be trained for using the tools?
  • Who is the decision maker who will give the sign-off?
  • Who will coordinate the UAT members?
  • How will the UAT reporting be done? Who will do it?
  • What metrics will be included in the test report? How are they collected?
  • What is a defect? What is a CR (change request)?
  • Who will prepare the test cases (IT testers or business users)?