Antivirus software testing for the new millenium

Antivirus Software Testing for the New Millenium

Abstract: The nature of technology is changing rapidly; likewise, the nature of viral threats to the data dependent upon the technology is evolving. Thus, the technologies we rely upon to provide protection from these threats must adapt. In the last twelve months, several anti-virus software vendors have announced exciting new technologies whichclaim to provide “faster, better, cheaper” response to computer virus incidents within organizations. However, there is currently little guidance regarding the best way to evaluate the efficacy of such claims. Faster than what? Better than what? Less costly compared to what? Clearly, there can only be one technology which is “faster, better, most cost efficient" than all of the others, yet if theadvertising claims are to be believed, all products are not merely created equal, they are all created superlative! In this paper, the requirements for these next generation anti-virus systems will be examined. There will be a discussion of reviewing strategies that can help to determine to what extent those requirements have been met. To this end, the problem will be approached from a functionalperspective, not gearing the test design to particular implementations. In this way, an array of tests will be created which are not vendor or product specific, but which can and should be employed industry-wide.

Authors:

Sarah Gordon (sgordon@format.com) IBM Thomas J. Watson Research Center, U.S.A, Fraser Howard (fph@format.com) Virus Bulletin, U.K.

Point of Contact:

Sarah Gordonsgordon@format.com sgordon@dockmaster.ncsc.mil

Keywords: computer virus, anti-virus product testing, anti-virus product certification, testing methodology, testing criteria, functional requirements.

Antivirus Software Testing for the Year 2000 and Beyond
Sarah Gordon (sgordon@format.com) Fraser Howard (fph@format.com)

Introduction

In the last twelve months, several anti-virus softwarevendors have announced exciting new technologies which claim to provide “faster, better, cheaper” response to computer virus incidents within organizations [Anyware, 2000; NAI, 2000; PC-Cillin, 2000; Symantec, 2000a; Symantec, 2000b; Thunderbyte, 2000; Trend, 2000]. However, there is currently little guidance regarding the best way to evaluate the efficacy of such claims. Faster than what? Betterthan what? Less costly compared with what? Clearly, there can only be one technology which is “faster, better, most cost efficient" than all of the others, yet if the advertising claims are to be believed, all products are not merely created equal, they are all created superlative! In this paper, the requirements for these next generation anti-virus systems will be examined. There will be adiscussion of reviewing strategies that can help to determine to what extent those requirements have been met. To this end, the problem will be approached from a functional perspective, not gearing the test design to particular implementations. In this way, an array of tests will be created which are not vendor or product specific, but which can and should be employed industry-wide.
The State of theNation: Anti-virus testing in the 90’s

Antivirus product testing has improved greatly since the simple zoo scanning offered in the first published reviews. Many, if not most, of the technical and administrative problems documented in [Gordon, 1993; Laine, 1993; Tanner, 1993; Gordon, 1995; Gordon & Ford, 1995; Gordon & Ford, 1996; Gordon, 1997] have been resolved. Today’s tests provide a solid,albeit not perfect, measure of product capabilities.

As tests have become more complex, several bodies have emerged as leaders and innovators in this area. Some of the more widely-accepted tests1 within the industry are outlined briefly below: (i) ICSA Certification The International Computer Security Association (ICSA) has been performing tests of antivirus software since 1992; many popular...