Software Testing Estimation Techniques

Posted in (Software Testing & QA)

Software testing estimation techniques are as old as the binary digits in the history of digital computers. Software testing is one of the important stages during software development life cycle (SDLC) to check and control the quality of the application.

Today, over 30% to 70% of a project’s resources are dedicated towards testing. Still, testing is often considered as an underestimated part of software development. Moreover, academic research in testing is low compared with the importance of the subject in real-life software projects.

Software testing estimation techniques involves experimentally and systematically checking the correctness of software. It is performed by applying test experiments to a software system, by making observations during the execution of the tests and by subsequently assigning a verdict about the correct functioning of the system. The correctness criterion against which the system is to be tested should be given in some kind of system specification.

The specification prescribes what the system has to do and what not, and, consequently, constitutes the basis for any testing activity. Despite its importance, testing is often an under-exposed phase in the software development process. Software testing turns out to be expensive, difficult and problematic while, on the other hand, research and development in testing have been rather immature.

Software Testing Best Practices

In order to implement testing best practices for a software development or mobile app development project, one needs to analyze the risks and complexities about the project by estimating the testing efforts. Software testing estimation techniques play a very important role in building credibility before initiating any software or mobile app testing project. To achieve bug free code for your software and mobile applications, software testing estimating techniques should be implemented by your team.

Today, any software application developed is unique in its own domain. So it becomes difficult and inconvenient to estimate software testing accurately at the very first attempt. Even if the project scope is understood and requirements are clear, it is still arduous to estimate a complex system that is going to be built with its unique set of requirements.

Software Testing Estimation Techniques

One of the most important factors while estimating testing efforts is the hands-on experience on varied projects for the software test life cycle. No longer can one just take a guessing approach about the number of days for any task or working on the old time formula of one third of the development effort.

Although this method is not based on any scientific principle or technique, it is one of the most widely used estimation technique by companies offering software-testing services. Unfortunately the development versus testing effort method has given many failures in software projects in the past, thereby compromising the software or mobile apps on quality.

In the recent years there have been many techniques that have been developed for estimating the software testing timeframe. These techniques are: 3-Point Software Testing Estimation Technique, Use-Case Point Method and Wide Band Delphi Method.

3-Point Software Testing Estimation Technique

3-Point Software Testing Estimation Technique is based on statistical methods in which each testing task is broken down into sub tasks and then three types of estimation are done on each tasks.

The formula used by this technique is:
Test Estimate = P + (4*N) + E / 6
Where,

  • P = Positive Scenarios
  • N = Negative Scenarios
  • E = Exceptional Scenarios

Standard deviation for the technique is calculated as,

  • Standard Deviation (SD) = (N – E)/6

Use – Case Point Method

Use-Case Point Method is based on the use cases, where we calculate the unadjusted actor weights and unadjusted use case weights to determine the software testing estimation.

The formula used for this technique is:

  • Unadjusted actor weights = Total no. of actors (positive, negative and exceptional)
  • Unadjusted use case weight = Total no. of use cases
  • Unadjusted use case point = Unadjusted actor weights + Unadjusted use case weight
  • Adjusted use case point = Unadjusted use case point * [0.65+ (0.01 * 50]
  • Total Effort = Adjusted use case point * 2

Wideband Delphi

In Wideband Delphi Method, work structure is broken down for each task and is distributed to a team comprising of 3-7 members for re-estimating the task. The final estimate is the result of the summarized estimates based on the team consensus. This method speaks more on experience rather than any statistical formula. The Wideband Delphi testing estimation technique logically estimates the group iteration efforts required in a visual manner for the testing team. This test was coined by Barry Boehm and is widely accepted software testing estimation technique to solve complex problems.

How to evaluate a software testing estimation technique?

For any software testing estimation technique, it is highly recommended that the following factors should be taken into account:

  1. Domain knowledge and core requirements
  2. Risks and complexity of the application
  3. Team knowledge on the subject/skills
  4. Historical data for the previous estimation for improvement and accuracy
  5. Estimation should include buffer time
  6. Bug cycles for the project
  7. Resources availability

However these techniques just provide the means for estimating but rely heavily on the team productivity variations, individual skills, complexity of the unknown factors like system environment and downtime.

Despite advances in formal methods and verification techniques, a system developed still needs to be tested before it is implemented in real-life. Testing remains a truly effective means to assure the quality of a software system of non-trivial complexity, as well as one of the most intricate and least understood areas in software engineering. Testing, is an important research area within computer science is likely to become even more important in the future.

Rishabh Software leverages the best practices for software testing and mobile app testing, helping clients globally to achieve their business goals. Our team of software and mobile developers solves complex test cases and simplifies the testing efforts. Contact us today or call us on 1-877-RISHABH (1-877-747-4224) to know more on implementing the right software testing estimation technique through our Application testing services customized just for your application.

We also recommend:

  1. Software Testing Governance
  2. Software Regression Testing Best Practices
  3. Custom Software Development Services
  4. Build applications faster by outsourcing software development
  5. Healthcare Software Development
Enjoyed this article! Share it with others.

Comments

    chandu
    May 31, 2011

    Are the estimates what we get by the above techniques is in hours?
    Pl clarify me…

    Thanks in Advance

      M Jha
      May 31, 2011

      Hi Chandu,

      All the above techniques provide the estimates in hours which can be later attributed to person days and person man-months.

      I hope I’m able to solve your query.
      regards,
      M Jha.

    sivakumar kalidoss
    November 9, 2011

    in the formula Test Estimate = P + (4*N) + E / 6

    i need clarification.
    is this applies to a single feature. For eg. i have a login screen with user id/pwd button
    do i need to apply this formula just for validating user id field.

    if so i have 1 positive, 4 negative scearnio.
    is that right?

    in the formula it is divided by 6. why it is divided by 6. do you have any specific reason. please clarify.

    Sivakuamr Muthiah
    November 10, 2011

    Hi Siva,
    Hope below link will clarify your doubt

    http://www.visionarytools.com/decision-making/3-point-estimating.htm

    M Jha
    November 14, 2011

    Hi Sivakumar K,

    The techniques above specified can be applied to a single feature or mulitple features. None of the above mentioned techniques is used for finding out the negative scenarios or performing any validations.

    The above techniques are used for getting the estimated effort for a certain defined tasks which is to be anlsysed for preparation and execution of test cases. The above formula is derived from PERT analysis and is little bit tweaked in terms of test cases, for getting the effort for set of test cases classified in positive(P), negative(N) and exceptional(E) test cases. Hence the effort is calculated as P+(4*N)+E/6. Values 4 and 6 are the constant and can be interpreted for the difficulty or the complexity of the test cases and can also be taken as optimistic, pessimistic and most likely classification based on the PERT analysis.

    For example,

    1. Say in a project you have total 705 test cases out of which 400 positive, 180 negative and 125 exceptional set of test cases then the effort calculated will be roughly estimated at 1140.833 with standard deviation of 9.16

    2. In another example, say if you have total 7 test cases out of which 4 are positive, 2 negative, and 1exceptional, then the effort will be 12 with standard deviation of 0.16

    I hope this solves your query.

    Thanks & regards,
    M Jha.

    M Jha
    November 14, 2011

    Hi Sivakumar M,

    Thanks for the link that provides detailed information on PERT analysis and the 3 point estimation technique. However the 3-point mentioned in the above article is little bit tweaked for 3-point, as the software testing already deals into positive, negative and exceptional scenarios and then deriving the final effort.

    Thanks & regards,
    M Jha

    Venu
    January 4, 2012

    Hi Jha,
    Can you please give one example for Use – Case Point Method estimation techinque like as 3point estimaiton as above

    Many thanks,
    Venu R

    M Jha
    January 5, 2012

    Hi Venu,

    Consider a project having 10 use cases for which there will be minimum 10 test cases. So as per Use Case Point method, we can derive the efforts as follows:
    Say for 5 positive test cases it takes 5 minutes to execute, 3 negative test cases it takes 10 minutes and 2 exceptional test cases as 15 minutes respectively which can vary as per complexity of the test case execution then,
    Unadjusted Actor Weights = 85 (summation of +ve, -ve and exceptional test cases with actor weight which is in minutes)
    Unadjusted Use Case Weight = 10 (total no. of test cases or use cases)
    Unadjusted Use Case Points = 85+10 = 95
    Adjusted Use Case Point = 95* [0.65 + (0.01 * 50)] = 109.45, where 0.65 is the constant factor for technical complexity and (0.01*50 = 0.5) is the environmental constant for the test case execution in worst case scenario)
    Total or Final Effort = 109.45 * 2 = 218.5
    I hope this will help in understanding the final effort calculation for getting the test execution effort.

    Normally we use UCP to calculate the project efforts but in this way you can calculate the efforts for the test efforts needed for the project.

    Thanks & regards,
    M Jha.

    bindu
    April 12, 2012

    Appreciate Jha for a detailed explanation and example for UCP. It was really useful.

    Deepika
    August 14, 2012

    Hi, how do you know how many test cases will be needed when the requirements are not clear? How do you provide high level estimation at the initial stage?

    M Jha
    August 16, 2012

    Hi Deepika,

    Very valid question. A lot of software test professionals are into this dilemma when the requirements are not clear and majority of the projects have a scope creep conundrum. From my perspective when the requirements are not clear then in that case, Three – Point estimation technique is the best way to derive the approximate no. of test cases. Three point estimation facilitates that for every requirement there will be one positive test case and for every positive test case there will be 4 negative test cases. By using the aforementioned formula you can derive the efforts for high level estimation. All the above estimation techniques if used correctly will provide you a tolerance of 5-10% variation or deviation against high level requirements estimated efforts.

    Thanks & regards,
    M Jha.

    basu
    October 3, 2012

    HI Jha,

    How can u say “one positive test case and for every positive test case there will be 4 negative test cases”. Can you please give some example.

      Ajit
      October 5, 2012

      Hi Basu, thanks for pointing that out. On behalf of M. Jha, please see the response as below:

      When there is an uncertainty in requirements then Three – Point estimation technique is the best way to derive the approximate no. of test cases. Here as per the technique “for every positive test case there will be 4 negative test cases”; but practically it may happen for some positive test cases there can be 4 negative test cases. And that’s why we are expecting tolerance of 5-10% variation or deviation against high level requirements estimated efforts.

      Hope it helps solve your query. Do let us know if any doubt still exists.
      Thanks once again.

    Rohit
    December 29, 2012

    Does this take into account complexity of the test case. What if my positive test case is more complex and will take more time to designe

    Rohit
    December 29, 2012

    Let me paraphrase my question. I am talking about the 3 point estimation technique, what will I do if I encounter a complex test case that has dependencies on other modules/3rd party softwares. Will this be treated as exceptional test case?
    If not what is treated as an exceptional test case?

      Ajit
      January 1, 2013

      Hi Rohit,

      Thanks for your time. Please see below the response to your query.

      The complexity of test case has to be taken into consideration for positive, negative and exceptional scenarios. As you have rightly said that the complexity increases with 3rd party software integration. This integration complexity can have impact on positive and/or negative and/or exceptional test case.

      An exceptional test case would cover special cases or exceptional conditions similar to some of the alternative branches of a use case.

      Please let us know, if we’re able to solve your query.
      Thanks

    Khaja Mohideen
    April 1, 2013

    Pls explain the test effort estimation methods widely used with an real time example

      Kaushal Shah
      April 7, 2013

      Thank you for your time. A brief on the widely used estimation methods/techniques are already mentioned in the article. We will be doing a separate article on the same topic in the near future which will cover a real time example.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>