Estudiante

Applying the ISO 9126 Quality Model to Test Specifications — Exemplified for TTCN-3 Test Specifications
Benjamin Zeiss∗ ‡, Diana Vega∗∗ ‡‡, Ina Schieferdecker‡‡, Helmut Neukirchen‡, Jens Grabowski‡
‡

Software Engineering for Distributed Systems Group, Institute for Informatics, University of G¨ ttingen, Lotzestr. 16–18, 37083 G¨ ttingen, Germany. o o{zeiss,neukirchen,grabowski}@cs.uni-goettingen.de ETS, Technical University Berlin, Franklinstr. 28–29, 10587 Berlin, Germany. {vega,ina}@cs.tu-berlin.de

‡‡

Abstract: Quality models are needed to evaluate and set goals for the quality of a software product. The international ISO/IEC standard 9126 defines a general quality model for software products. Software is developed in different domains and the usage of the ISO/IEC quality modelrequires an instantiation for each concrete domain. One special domain is the development and maintenance of test specifications. Test specifications for testing, e.g. the Internet Protocol version 6 (IPv6) or the Session Initiation Protocol (SIP), reach sizes of more than 40.000 lines of test code. Such large test specifications require strict quality assurance. In this paper, we present anadaptation of the ISO/IEC 9126 quality model to test specifications and show its instantiation for test specifications written in the Testing and Test Control Notation (TTCN-3). Example measurements of the standardised SIP test suite demonstrate the applicability of our approach.

1

Introduction

Test specifications developed today by industry and standardisation are usually voluminous and are, in ourexperience, often regarded as complex. Such statements are based on subjective opinions about few quality aspects that stand out, but neither is the term “complexity” clearly defined nor is it evident how it relates to the quality of a test specification. In the context of software engineering, metrics are a common means to quantify quality aspects of software. Metrics are classified into those thatconcern products, processes, and resources. While metrics support the measurement of quality aspects, they do not provide an answer what constitutes quality in software. To gain reasonable statements from metrics, a quality model is required which defines distinct characteristics and corresponding subcharacteristics that relate to software quality. ISO/IEC 9126 [ISO04] is a standard describingsuch a model for software products.
∗ Supported ∗∗ Supported

by a PhD scholarship from Siemens AG, Corporate Technology. by Alfried Krupp von Bohlen und Halbach-Stiftung.

Quality aspects of test specifications are somehow related to the characteristics stated in ISO/IEC 9126. However, a more elaborated analysis on the peculiarities and differences between the quality aspects that constitutetest specifications and software has not been made yet. Still, quality aspects concerning test specifications and test implementations are constantly subject of discussions and various test metrics have been developed measuring single aspects only [Sne04, VS06, ZNG+ 06a, ZNG+ 06b]. Thus, a more general view on the different quality aspects of test specifications is needed. The contribution of thispaper is a quality model for test specifications which is derived from the ISO/IEC 9126 quality model. For concrete investigations, we selected the Testing and Test Control Notation (TTCN-3) [ETS05a] which is standardised by the European Telecommunications Standards Institute (ETSI). TTCN-3 is a language for test specification and implementation, i.e. it supports abstract test specifications which can becompiled and executed if additional implementation components (such as an SUT adapter) are provided. This paper is structured as follows: Section 2 introduces ISO/IEC 9126. Subsequently, our quality model for test specifications is presented and discussed in Section 3. Section 4 describes an instantiation of our quality model for TTCN-3. An application of our TTCN-3 specific model to different...