Estimation for Software Testing

Do you estimate Testing effort as X% of Coding effort?

Scene 1:

The customer has provided adequate documentation on the requirements for the software to be developed and now requests for a quote. Pre-sales consultants, with a developer background, spend just enough time in analyzing the requirements and have discussions with the developers/architects/project managers on how much effort it would take for the development (analysis, design, coding, project management) in the specific architecture/technology. They would go through a series of refinements with various stakeholders before it is being projected to the customer. However, when it comes to the point of estimating the testing effort, the oft-heard response is: “Oh! Don’t you worry; it is just x% of the coding effort. That has worked for us…..”

Well, that’s a scene I have witnessed many a times.

Now, let’s cut to Scene 2:

The customer has provided adequate documentation on the requirements for the software to be developed and now requests for a quote. The twist in the tale here is that a tester has also been invited to the pre-sales discussion where he/she is also involved to understand the requirements and provide the testing estimate. (Well, that makes for a better scenario than the previous one because, thankfully, they have at least understood the need for a tester to provide the testing estimate). So, the tester goes about his/her job and tries to ask many questions and get clarification on the various aspects that need to be tested, the various OS, browsers and devices compatibility. Then the tester comes up with his/her estimate on the testing effort. Zoom in to the pre-sales consultant who sports a shocking look on his/her face looking at the estimated figures and says: “This estimate cannot be right. The testing effort is more than the coding effort”.

Cut again!!

Now, we know what the problem is. Many people assume that the testing effort is only x% of coding effort and hence can never be more than it.

To all who have been living with such myths, I would like to present to you all that comes under the purview of an estimate for testing software.

  • We, as testers, need to verify not just the functionality of the application, but also other quality aspects (the “ity”s) of the software such as usability, reliability, scalability, security etc…
  • When testing a software that needs to adhere to standards and guidelines with respect to the User Interface, industry specified statutory and regulatory requirements, the effort that goes into it also needs to be accounted for during testing.
  • It would take close to “y” amount of time to develop software that is going to be compatible in one operating system. However, if the software is to be compatible in multiple OS then the time taken to test its compatibility would be ‘multiple * y’.
  • While trying to do such compatibility testing and testing each build, there is a lot of effort that goes into environment creation and cleanup even before we actually begin to test. Even if we try to use VMs there is quite an amount of effort that is required which needs to be accounted for.
  • Testing estimate should take into account at least 2 cycles of functional testing along with the effort taken for testing the other aspects of software and then one final round of release cycle testing. After we have completed 2 rounds of functional testing, if there are critical bugs found during the 2nd cycle of testing, then would we say that the product is ready to go through the release testing? Never! If we want to deliver a quality product, we need to go through several cycles of testing until the product is free of critical errors. Hence, the estimation is bound to vary if the number of cycles of functional testing spikes over 2. (This definitely ups the pressure on the development team).
  • We can never do away with the fact that there is going to be a lot of other learning as we go about exploring the software. It only makes sense to make for the effort for exploratory testing, too.

In spite of all this, there are going to be a lot of questions coming from customers on why we need to have effort estimated for all the above? Well, the toughest task is convincing them on this and bursting the bubble of myth they are living in!


Tags: Testing
previous post: Near Field Communication (NFC) next post: What’s the Mantra for detecting your security threats?