BUY TOOLS IF YOU REALLY NEED THEM
Simplicity is the ultimate sophistication.
-Leonardo da Vinci
When asked, ‘How do you write?’ I invariably answer, ‘one word at a time.’
-Stephen King
What worries you, masters you
-John Locke
You Wear a Crown but You’re No King
-A song by Blessthefall
A big problem in any organization, and a very easy way of squandering money, is buying software that will never be used.
I feel a need to write about this subject because I have seen millions of dollars of investments that were thrown away on software that remained untouched or utilized only minimally. Maybe this seems unreasonable to you at first glance, but I assure you that many test organizations are keen on buying software without considering the necessity and return. They just buy the software for the sake of telling people, “Yes, we have the necessary software for this activity.”
When we narrow the subject to software testing, everything is more or less the same. Organizations invest heavily in buying any kind of testing software, but they generally don’t care or know enough about their utilization. I remember having three test management tools at the same time and still using Excel sheets for writing and maintaining the test cases. What a waste!
Vendors, on the other hand, know very well how to attract people. They prepare perfect presentations about their products, create virtual environments for making proof of concepts (PoCs), and try to make you feel comfortable and impressed. Surely this comfort and optimistic ambiance lasts until just after the tools are purchased!
You generally notice major deficiencies and observe issues related to your own environment (internal network, firewall, operating system, servers, and so on) just after you intend to use them. But why? Everything was really perfect in the PoC, what has happened?
The answer is very straightforward: you relied on a simple virtual experience and did not try the tool in your own environment. That’s it! Below are tips for an effective PoC; I think all are known but not strictly followed.
- Treat the tool selection process seriously,
- Identify and quantify your problem,
- Clarify your needs and document tool requirements,
- Identify requirements for coaching in use of tools
- Select a pilot project and include all the necessary stakeholders,
- Try to experience as many tools as possible—one tool per issue is never enough
- Try the tools in your own environment (network, firewall, OS, DBs, applications, etc.)
- Check the integration between tools (e.g., Test Management Tool and Defect Tracking Tool)
- Be careful about buying bundle products merely because bundles seem to be cheaper options; most of the time they boost the amount of shelfware in your organization
- Check non-recurring costs (knowhow transfer, PoC process, initial purchasing costs, adaptation and required development/customization costs)
- Check the recurring costs (maintenance costs, support fees and cost of ownership)—a tool can be cheap or even almost free for buying but can be really expensive for maintaining
- Check open-source alternatives and complements—you don’t have to pay for everything
- Do not buy more licenses than needed—be a little bit of a skinflint
- Be patient about the return; you need to commit and sacrifice for the perfect utilization—things never happen by themselves
- Do not buy many tools at a time; take your time and proceed step by step—remember, accuracy and perfection require dedication
There are many others that can be added to the above list, yet one last thing I want to say is that if you don’t really need a tool for realizing any specific software testing activity, then stay away from the tool (e.g., be ad hoc or toolless). Buying a fancy tool will never make you go one step further unless you need it. Instead it may result in the opposite by bringing inefficiencies and cost overruns.
Now, please let me address the correct timing of selecting tools. All the aforementioned actions will surely assist you in selecting the right tool, provided you do all your selections at the right time.
Let me make it more clear by analyzing the dilemma: tool-driven processes vs. process-driven tools. I have sadly experienced many organizations (including the ones I worked for) selecting their tools before defining their processes. This is not a good approach, since you lose control over the tools by letting them design your processes and drive your daily operational activities. These are just tools; they are not your strategies, plans, or organizational goals, so don’t let them take control. Instead, you make your tools help you in achieving your goals. This is feasible only with correct timing. Before setting a common language within your test department and defining your test processes (e.g., policies, strategies, basis, life cycle, deliverables, and so on), you should not talk about any tools or even PoCs.
Below is the pyramid of test governance. You can take it as a guideline for yourself and go up step by step. When you are finished with training, processes, organization, and techniques, the last building block shall be the tools.
