J Forensic Sci, September 2010, Vol. 55, No. 5 doi: 10.1111/j.1556-4029.2010.01426.x Available online at: interscience.wiley.com
Simon J. Walsh,1 Ph.D.; James M. Curran,2 Ph.D.; and John S. Buckleton,3 Ph.D.
Modeling Forensic DNA Database Performance*
ABSTRACT: Over the past decade or more, DNA databases have been a focal point of development for the forensicfield. Using this approach, forensic and law enforcement agencies have aided millions of investigations, many of which would remain unsolved but for the intelligence links provided from DNA database comparison. However, despite their widespread use and increasingly broad legislative and operational reach, there has been limited overarching performance modeling or reflection on drivers of operationalor financial efficiency. This study derives an inferential model for DNA database performance using data from major national DNA database programs. Parameters that optimize desirable database outputs (matches) are isolated and discussed, as is an approach for maximizing financial efficiency and minimizing ethical impact. This research takes important steps toward identifying measures of performancefor forensic DNA database operations. KEYWORDS: forensic science, DNA database, Combined Offender DNA Index System, U.K. NDNAD, Canada, New Zealand, intelligence, performance management
Forensic DNA databases have altered the landscape of the criminal justice system (CJS) and reshaped the field of forensic science. They have provided new challenges to the mechanisms by which forensic evidencecan be utilized and have brought pressures and increased responsibility upon those who administer their use. There has been widespread review and commentary regarding the legal and socio-political basis of DNA databases, but there remains a lack of meaningful empirical assessment of database performance and effectiveness. To assess the database performance we need to define and decide how we mightmeasure the success of forensic DNA databases. To do this effectively is a complex undertaking that requires the coalescence of a range of experimental methodologies across numerous domains of society. Most jurisdictions do not monitor the performance of their databases beyond reporting a one-dimensional index of output relating to the number or proportion of hits. We believe that this is a majoromission. To date, there has been little attention given to the assessment of database performance—due mainly to the fact that through their entire history, there has been minimal demand for such evaluation. Forensic DNA databases have always provided results, many of which involve spectacular and unprecedented contributions to the most serious of cases. These results have taken little strategicthought to achieve, and to some degree this will always be the case. However, an era is arriving where more profound
1 Forensic & Data Centres, Australian Federal Police, Canberra, ACT, 2601, Australia. 2 Department of Statistics, University of Auckland, Private Bag 92019, Auckland 1142, New Zealand. 3 ESR Ltd., Private Bag 92-021, Auckland, New Zealand. *Research emerges from the Ph.D. researchof Simon J. Walsh undertaken at the University of Technology, Sydney (UTS). The project was supported by a UTS Industry Link Grant. Received 29 Sept. 2008; and in revised form 8 July 2009; accepted 29 July 2009.
performance management and consideration of database effectiveness will be essential to ensure an ongoing contribution and to manage future challenges. As a database ages a number ofchanges occur, some of which are detrimental. Consider the crime sample database. These are samples from crime scenes and may be from the true offender or may be from an irrelevant source—such as the home owner of a house that has been burgled. Scenes of crime officers (SOCOs) seek to keep the fraction of relevant samples high. However, certain types of sample, such as cigarette butts, and...