This is a deceptively simple question because expensive means different things to different people. For a project manager, it tends to be that testing mostly is done at the wrong time, at the end of a project when budgets are depleted and fixing errors is time-consuming. For the call centre, it is the number of support calls from frustrated users.

In other words, do you and your stakeholders have the same definition of what expensive is?

My real point is that the concept of value is multi-dimensional. Thus, in order to determine whether testing is expensive, we need to consider all the dimensions. I think there are five key dimensions, which we can call the five Ws:

• What in general makes testing expensive?
• When is it expensive?
• Where is it expensive?
• Who makes it expensive?
• Why do we even require testing in the first place?

I am no expert on how to run a profitable business and satisfy all the stakeholders involved financially, but I have to admit that, as a quality assurance professional, I am surprised to find that those five Ws are not asked. It is as if business schools only teach people how to be successful when things are going well, and not to assess and mitigate risks. Maybe it’s easier and cheaper just to say “sorry”. I suppose I shouldn’t complain because this is probably why people like me have a career in testing and quality assurance in the first place!

To get an idea what I am ranting about, let’s look at a typical process which is generally ignored during planning because it is very unlikely that there will be sufficient control over information to measure effectively the tasks performed. The process I am referring to is the defect (bug) life cycle. How much does it really cost to fix a bug found during acceptance testing or in production?

Looking at the rough sample diagram above we can see about nine possible stages to a bug life cycle. On the right are lists of typical tasks performed during each stage. If you measure the effort spent, and the cost of the people involved you might be surprised with the numbers. Assume 25 bugs out of a total of 250 bugs need to be fixed by the end of a two week period. Will you be confident to release to production without measuring the current quality through testing?

Many times we are being challenged and struggle to show quality because it might only be visible in getting to production. Over the course of development, developers will be less likely to test all of the things that can go wrong. I think it is muscle memory by constantly focusing only on getting the code to work. This explains the theory behind the graph that is part of all test related training - bugs become exponentially more expensive to manage during the SDLC.

“How much is that bug worth to the people that matter?”

The irony is that little attention is assumed to guarantee quality according to the context of the system - until something goes wrong and it negatively affects someone. During crunch periods, there tends to be an increase in the total number of testers involved on the project. Sometimes the testers with the wrong skill sets are pulled in just to get numbers up and to add more eyes to the system. However, is it sustainable, effective, efficient, or repeatable? At times, you do require the rare skills of test specialists or a tester with a broad range of skills, but the contracting fees cannot fit in your already limited budget.

Many times I run into situations where people were employed for their personality and not for their skills to make the project a success. Add to that the costs for performance testing tools and environments. Security testing is not cheap either. There are design risks if globalisation is not considered and can result in a big rework. There is also that dreadful maintenance risk to keep the system running for the next 3 to 5 years. How much is red tape crippling the efficiency and effectiveness of testing in some businesses?

Remember too that some people using the product will exploit your system. If you are still not convinced then have a read on sites like CNET.com and search for the term “glitch”.

“Could it be an intentional blindness forced onto decision makers by society to ignore the negativity of asking “what if” questions?”

Obviously there will be solutions to many challenges, at a price of course. However, changing cultures and processes over a short period seldom go according to plan. No matter how much money you throw at problems, they cannot get solved in the desired timeframes.

I believe testing can become cheaper or at the very least become better. Bring the minds of the right people together from the beginning to ensure quality throughout the SDLC with the required levels of testing. Involve all the stakeholders, including the clients, to collaborate, set realistic goals and take responsibility for the quality.

Respect the role and responsibilities of each other in the team but also understand and accept the team’s capabilities.