A skeptic's guide to computer models

A Skeptic’s Guide to Computer Models
by John D. Sterman

This article was written by Dr. John D. Sterman, Director of the MIT System Dynamics Group and Professor of Management Science at the Sloan School of Management, Massachusetts Institute of Technology, 50 Memorial Drive, Cambridge, MA 02139, USA; email: jsterman@mit.edu. Copyright © John D. Sterman, 1988, 1991. All rights reserved. Thispaper is reprinted from Sterman, J. D. (1991). A Skeptic’s Guide to Computer Models. In Barney, G. O. et al. (eds.), Managing a Nation: The Microcomputer Software Catalog. Boulder, CO: Westview Press, 209229. An earlier version of this paper also appeared in Foresight and National Decisions: The Horseman and the Bureaucrat (Grant 1988).

D-4101-1

A SKEPTIC’S GUIDE TO COMPUTER MODELS

TheInevitability of Using Models......................................................3 Mental and Computer Models...........................................................4 The Importance of Purpose..............................................................5 Two Kinds of Models: Optimization Versus Simulation and Econometrics.......6Optimization......................................................................6 Limitations of Optimization............................................6 When To Use Optimization............................................9 Simulation ........................................................................10 Limitations of Simulation..............................................11Econometrics.....................................................................14 Limitations of Econometric Modeling................................16 Checklist for the Model Consumer .....................................................21 Conclusions ...............................................................................22 References.................................................................................23

2

D-4101-1

A SKEPTIC’S GUIDE TOCOMPUTER MODELS

A Skeptic’s Guide to Computer Models...
But Mousie, thou art no they lane In proving foresight may be vain; The best-laid schemes o’ mice an’ men Gang aft a-gley, An lea’e us nought but grief an’ pain, For promis’d joy! Robert Burns, “To a Mouse”

The Inevitability of Using Models Computer modeling of social and economic systems is only about three decades old. Yet in that time,computer models have been used to analyze everything from inventory management in corporations to the performance of national economies, from the optimal distribution of fire stations in New York City to the interplay of global population, resources, food, and pollution. Certain computer models, such as The Limits to Growth (Meadows et al. 1972), have been front page news. In the US, some havebeen the subject of numerous congressional hearings and have influenced the fate of legislation. Computer modeling has become an important industry, generating hundreds of millions of dollars of revenues annually. As computers have become faster, cheaper, and more widely available, computer models have become commonplace in forecasting and public policy analysis, especially in economics, energy andresources, demographics, and other crucial areas. As computers continue to proliferate, more and more policy debates—both in government and the private sector—will involve the results of models. Though not all of us are going to be model builders, we all are becoming model consumers, regardless of whether we know it (or like it). The ability to understand and evaluate computer models is fastbecoming a prerequisite for the policy maker, legislator, lobbyist, and citizen alike.

During our lives, each of us will be faced with the result of models and will have to make judgments about their relevance and validity. Most people, unfortunately, cannot make these decisions in an intelligent and informed manner, since for them computer models are black boxes: devices that operate in completely...