Tecnologias De Informacion

End-user (contracted) interest in outsourcing testing services has grown consistently, though moderately, in the past few years. Sourcing professionals can use the steps in this research to define essential metrics to control the performance of testing services providers.

Key Findings
■

Outsourcing testing services is not yet a mature discipline. Organizations have limited internalbaseline metrics regarding quality and efficiency of their testing capabilities. The most prevalent outsourcing deals include staff augmentation and execution support for specific types of testing. The focus is on skills and applying standards and automation. Most testing services providers don't formalize efficiency improvements through metrics as part of the original contract. Providers indicate thatthe data is not available upfront to make valid improvement statements that can be used as an incentive in a contractual agreement. When outsourcing testing services, as a result of a lack of end-to-end coverage and insight in capabilities, metrics are either lacking, or focused on delivery on time and in budget — that is, proof of the estimation of work related to testing effort.

■

■

■Recommendations
Organizations looking to outsource testing services should:
■

Invest in analyzing their testing capabilities and process and automation maturity to define more effectively improvements that can and should be realized through the outsourced services. Understand that testing is required to verify and validate that a business objective is being realized through a project ora development effort, throughout the life cycle, not only at a user acceptance phase. Therefore, clear functional and nonfunctional acceptance criteria should be defined for each phase of the life cycle.

■

■

Define metrics that minimize the delay in defect detection and resolution (such as defect leakage or defect reinjection) as the cost of resolution increases per phase. This meansstarting at the requirements gathering phase, and not unit testing, to detect defects. Define metrics that focus on continuous improvement, as service providers continue to invest in automation and internal process efficiency (such as resolution or test case productivity).

■

Table of ContentsAnalysis..................................................................................................................................................3 1.0 Clarify the Focus of the Agreement.............................................................................................4 2.0 Clearly Define Metrics.................................................................................................................5 3.0 Use Defect, Requirements and Test Case Metrics Across theDevelopment Life Cycle...............9 3.1 Defect Leakage to Production..............................................................................................9 3.2 Defect Leakage to Next Phase or Defects Resolved per Phase...........................................12 3.3 Defect Resolution Improvement..........................................................................................16 3.4Zero Critical Defects in Production......................................................................................19 3.5 Defect Rejection Rate.........................................................................................................21 3.6 Defect Reinjection Rate......................................................................................................23 3.7 TestCase Execution Ratio..................................................................................................25 3.8 Test Case Efficiency Improvement......................................................................................27 3.9 Test Case Productivity Improvement...................................................................................29 3.10 Requirements Stability...