The Neglected Art of Bounding Analysis Environmental Science & Technology April 1, 2001 / Volume 35 , Issue 7 / pp 162 A — 164 A. M. GRANGER MORGAN
Are the answers provided by today's detailed risk analyses reasonable? Is valued insight being overlooked as a result of analysts' focus on the intimate details of environmental problems? If so, what can we do about this? Environmental risk analysis has fallen into a standard front-to-back mode of operation: Estimate the magnitude and pattern of releases of the pollutants of concern; model their transport and transformation through the environment; estimate the location and physiological state of people, animals, and plants and the exposures they will receive; apply dose-response functions; and estimate the resulting impacts. All of this makes perfect sense if the relevant science is pretty well known and good data are available on factors such as the behaviors of the populations at risk. However, in practice, the science is often highly uncertain. The release rates may not be known with precision. There is often great uncertainty about transport and transformation processes. Good measurements, or model estimates, of exposure are frequently lacking. There may be fundamental uncertainties about the analytical form of the dose-response functions, and even when there are not, there may be uncertainty about the specific coefficient values that define that function. We often have only a rough idea of where people (or other organisms) are, what they are doing, or what their physiological state is. What to do? The conventional answer has been to plow on–do the best one can by adding uncertainty analysis to the standard front-to-back mode of operation. Develop probabilistic models. Use available data to describe uncertainty and variability. And if that is insufficient, as it usually is, elicit expert judgments in the form of subjective probability distributions. Insert those distributions into the models. Perform stochastic simulation or some other form of uncertainty analysis. Report results as probability distributions, or perhaps in summary form as best estimates (e.g., as means) with associated uncertainty bounds. Today's approach represents a big improvement over the typical analysis of 25 years ago, which ignored uncertainty, used single-value "best estimates", and turned out a single number whose meaning was completely unclear. However, with current interest in applying ever more complex forms of uncertainty analysis to the standard front-to-back mode of risk assessment, we have begun to lose sight of the fact that there are frequently other methods that offer important insight in the face of large uncertainty. These are the methods of "bounding analysis". Conventional front-to-back analysis buries itself in the details of the problem in a head-on brute-force analytical attack. Bounding analysis (1, 2) steps outside the details of the problem and asks, "Can we use our broad general knowledge about the world in order to set some bounds on where the answer might lie?" It is time for environmental risk analysts to become more skilled in, and start making greater use of, methods of bounding analysis. How might they do this? For example, in
greater use of, methods of bounding analysis. How might they do this? For example, in estimating the impact on lung cancer of an airborne pollutant, call it TZX, the analyst could start with the total known incidence of lung cancer. Available data and expert judgment could be used to probabilistically attribute a portion of the observed cancers to known causes such as smoking and radon. Then an approximate list of all other causal factors that might play a role in the remaining unattributed cases could be constructed and arguments could be made about the relative plausibility of each and about the completeness of the list. Although it certainly would not be possible to obtain a precise estimate of the number of lung cancers caused by TZX in this way, it might be possible to set a range that bounds that number and to make arguments about the precision with which that number can currently be estimated using conventional front-to-back analysis. Material and energy balance methods can sometimes help to bound an answer. We know that in our normal world, both mass and energy are conserved. By keeping track of the mass and energy that flows into and out of a process or environmental system, we can often set bounds on answers more readily than may be possible with conventional brute force front-to-back methods. Correlation among variables is one factor that can reduce uncertainty in the results of probabilistic analysis. A bound can sometimes be set on the potential importance of unknown correlations if one first does the uncertainty analysis treating all the variables as perfectly independent, then does the analysis treating all variables as perfectly correlated, and compares the two results. Don't misunderstand. This is not a call to abandon the methods of probabilistic risk assessment that we have developed so laboriously over the past quarter century. It is a call for risk analysts to think more carefully about the context in which they apply those methods, and to supplement the insights drawn from such analysis with simpler order-ofmagnitude and other arguments constructed from our general knowledge of the world in which specific problems are contained. Simple bounding analysis can be a powerful tool for insight. It is a tool that the environmental risk analysis community needs to become more adept at using. Acknowledgment Preparation of this piece was supported by Carnegie Mellon's Center for the Study and Improvement of Regulation.
References 1. Cohen, J. E. How Many People Can the Earth Support?; W. W. Norton: New York, 1995. 2. Harte, J. Consider a Spherical Cow: A Course in Environmental Problem Solving; University Science Books: Sausalito, CA, 1988.
M. Granger Morgan is head of the Department of Engineering and Public Policy at Carnegie Mellon University where he is also the Lord Chaired Professor of Engineering and holds academic appointments in the Department of Electrical and Computer Engineering and in the H. John Heinz III School of Public Policy and Management.
