Pymes

Published in: Crosstalk, September 1996

Developing a Testing Maturity Model, Part II
Ilene Burnstein, Taratip Suwannasart and C.R.Carlson
This article expands the level structure of the Testing Maturity Model (TMM). The maturity goals, subgoals, and the set of activities, tasks, and responsibilities associated with each level are shown. These are necessary to apply the TMM to test processassessment and improvement. Also discussed are automated tool support for test process maturity in the form of a Testers' Workbench and the nature of the relationship between the Capability Maturity Model and the Testing Maturity Model. This section describes behavioral characteristics of organizations at the five Testing maturity Model (TMM) levels. We give an informal view of each level and aperspective on maturity growth and maturity deficiencies as an organization progresses through the TMM hierarchy. Behavioral Characteristics of the TMM Levels Level 1 - Initial: Testing is a chaotic process; it is ill-defined and not distinguished from debugging. Tests are developed in an ad hoc way after coding is done. Testing and debugging are interleaved to get the bugs out of the software. Theobjective of testing is to show that the software works [1]. Software products are released without quality assurance. There is a lack of resources, tools, and properly trained staff. This type of organization would be on Level 1 of the Capability Maturity Model (CMM) developed by the Software Engineering Institute. There are no maturity goals at this level. Level 2 - Phase Definition: Testing isseparated from debugging and is defined as a phase that follows coding. It is a planned activity; however, test planning at Level 2 may occur after coding for reasons related to the immaturity of the test process. For example, at Level 2 there is the perception that all testing is execution-based and dependent on the code, and therefore it should be planned only when the code is complete. Theprimary goal of testing at this level of maturity is to show that the software meets its specifications [2]. Basic testing techniques and methods are in place. Many quality problems at this TMM level occur because test planning occurs late in the software lifecycle. In addition, defects propagate into the code from the requirements and design phases, as there are no review programs that address thisimportant issue. Post-code, execution-based testing is still considered the primary testing activity. Level 3 - Integration: Testing is no longer a phase that follows coding; it is integrated into the entire software lifecycle. Organizations can build on the test planning skills they have acquired at Level 2. Unlike Level 2, planning for testing at TMM Level 3 begins at the requirements phase andcontinues throughout the lifecycle supported by a version of the V-model [3]. Test objectives are established with respect to the requirements based on user and client needs and are used for test case design and success criteria. There is a test organization, and testing is recognized as a professional activity. There is a technical training organization with a testing focus. Basic tools support keytesting activities. Although organizations at this level begin to realize the important role of reviews in quality control, there is no formal review program, and reviews do not yet take place across the lifecycle. A test measurement program has not yet been established to qualify process and product attributes. Level 4 - Management and Measurement: Testing is a measured and quantified process.Reviews at all phases of the development process are now recognized as testing and quality control activities. Software products are tested for quality attributes such as reliability, usability, and maintainability. Test cases from all projects are collected and recorded in a test case database to test case reuse and regression testing. Defects are logged and given a severity level. Deficiencies...