![[Skip Navigation]](/images/pixel.gif){kind=tracking}
|
Restoration Economics
Risk and Uncertainty in Environmental Restoration ProgramsEnvironmental restoration projects are challenging because they are often one-of-a-kind. Project planners may not have much previous information on which to base cost estimates or probabilities of project success. The variables that are not known in advance by the planner involve uncertainty and risk and are generally referred to as uncertainties. Examples are the cost of materials and the benefits of using one restoration technique over another with regard to project performance. The project planner’s primary role regarding uncertainty and risk is to identify the parts of the project that have the greatest potential to cause concern (e.g., cost overruns, failure to achieve goals and objectives) and describe them clearly so that decisions can be made with knowledge of the degree of reliability of the information (Water Resources Council 1983). Project planners use several ways to describe and distinguish between what is risky and what is uncertain. Unfortunately, the terminology used and the specific definitions of the words are not always clear-cut. For example, Frank Knight wrote what is considered a landmark book in 1921 on economic risk and uncertainty. His definitions of these words are still used today by many economists. Knight said that risk is present if you can assign a probability to future events, but uncertainty is present if the likelihood of future events is indefinite or incalculable (Knight 1921). By his definition, risk could have either a positive or a negative outcome. However, the connotation of risk for many people is the chance that something bad will happen. Charles Yoe (1996) uses this intuitive definition when he describes risk and uncertainty analysis in the context of restoration projects implemented by the US Army Corps of Engineers (Corps). Because his definitions have particular relevance to environmental restoration, we will adopt his definitions of risk and uncertainty in this paper. They state that
The formal manner by which the uncertainties and risks of a project are analyzed is often called risk analysis, though there are many variations of this term and definition. The Corps separates a risk analysis into three components: risk assessment, risk management, and risk communication (Yoe 2001). Risk assessment describes the risks and their associated uncertainties, and answers questions such as What can go wrong?, How can it happen?, How likely is it?, and How bad can it be? (Yoe 2001). Risk management outlines the purpose and goals of the risk assessment and then uses the assessment results to decide what to do about the risks. Risk communication is the exchange of information among the interested parties (e.g., decision makers, the public, risk assessors). Although the component descriptions help us understand the formal process of analyzing risk and uncertainty, in practice, the three components often occur simultaneously and are sometimes difficult to distinguish (Yoe 2001). A risk assessment can provide decision makers with crucial information about things such as
Before beginning the risk assessment, the planner should ensure that there is no uncertainty about the purpose and objectives of the restoration project. Problems that motivated the study should be identified, and should include a clear cause-and-effect linkage that provides opportunities for improving conditions. Planners should also identify other factors that need to be considered for the project (e.g., constraints or limitations).
With these planning objectives and considerations acting as a mission statement for the project, all of the project plans and variations of plans can then be compared and selected based on their contributions to the planning objectives. The formation of clear, concise, descriptive project statements can go a long way toward reducing uncertainty. When the problems are uncertain, so are the solutions. The best place to start the risk assessment is to acknowledge that uncertainty exists and plan to incorporate it into the analysis from the start. The level of detail involved in assessing risk depends on the importance, size, complexity, and budget of the project. The methods and the models used to assess risk are generally acceptable as long as the answers obtained are adequate for decision-making. In general, when a project is planned properly, there will be a broad range of plans available for analysis. They often include scenarios such as not doing the project at all (a no-action alternative), and several alternative ways of doing the project (e.g., using different kinds or amounts of materials, doing restoration work all in one season or over several years). Future conditions are then established for each of the plans, often using a computer model. This simulation requires both data and judgment. Uncertainties that might need to be incorporated include project design, costs, and performance. Generally, estimates are based on a planner’s experience, but because environmental restoration projects are often unique, the planner may have little experience relevant to the particular project. For example, there is no sure way to know whether modifying the landscape will really result in improvements. Some planners may have developed similar projects and can use their success to place reasonable expectations on the probability of success of the current project. However, when such information is not available, the planner can employ other techniques, such as sensitivity analyses (described later), to test project performance. After the future conditions of all the plans are evaluated, planners can compare the expected results and select a plan for implementation. Once the planner has admitted risk and uncertainty are associated with an environmental restoration program, a variety of tools are available to help planners make informed decisions. The tools (many of them are computer programs) provide a rational and systematic means of assessing the unknown variables important to a project. Computer models have been designed to address the complexity of ecosystems by simplifying important elements that need to be understood to make informed decisions. The models focus on what is important to the problem at hand, rather than on numerous specific details. The models come in many forms, but many require the planner to go through several steps to get the most relevant answers: 1) Identify the variables of interestWhen we consider what we know for certain, it does not take long to realize there is relatively little we can be certain about. Fortunately, not all the uncertainties we face are equally important. Although not absolutely certain, many things are effectively certain (such as the sun rising tomorrow), whereas other things are relatively trivial and can be ignored. The same concepts hold true for risks and uncertainties in environmental restoration planning. Some uncertainties are more important than others. For example, the availability and cost of a large-diameter culvert pipe may not change much in a year’s time, but the amount of rainfall and associated stream discharge during a given season could vary widely and have an enormous impact on whether fish are able to navigate the culvert. Between the extremes of “effectively certain” and “trivial” lie the uncertainties that need to be addressed. To decide which uncertainties may need to be analyzed in a formal manner, planners must consider whether an uncertain variable could have a significant effect on the decision being made. If it could, it is important (Yoe 1996). 2) List what is known about the variables of interest, both quantitative (numeric information) and qualitative (non-numeric information)Once the important variables are identified, information is gathered in sufficient detail adequately to define existing conditions at the restoration site. This information is also used to define the most likely future condition of the site if restoration is not conducted. The data collected should be relevant to the restoration project and have bearing on the problems, opportunities, objectives, plans, or their effects. The more relevant data are available, the less uncertainty there will be in the project outcome. Subjective data are usually best provided as an interval estimate. Planners need to be honest about what information is known. For example, although it is tempting to say that suitable eelgrass substrate covers 65 percent of the proposed restoration site, it may be far more honest to say that with 95 percent confidence, suitable substrate covers between 50 percent and 70 percent of the site. 3) Identify the types of uncertaintyUncertainty is attributable to one of two sources. Inherent variability is the ordinary variability in a system, such as the randomness of natural processes. This type of variability includes such things as stream flow, which is considered random in time, or the success rate of plants purchased to revegetate a project site (Yoe 2001). Knowledge uncertainty is a lack of understanding of events or processes or a lack of data from which to draw inferences. The uncertainty is reduced with further information. Many other terms are used to describe risk and uncertainty, but these other definitions can be categorized into one of these two sources. However, it is important that planners and decision makers understand from which source the uncertainty originates. For example, if it is determined that there is a 10 percent risk that an ecosystem restoration project will fail to meet a performance target, and that this failure is due to inherent variability, then it may mean the project, even if perfectly executed, would fail to meet its objectives one year in ten. However, if the failure is due to knowledge uncertainty, then there may be a 10 percent chance the project will always fail to meet the target. The distinctions need to be made and communicated clearly to those involved in the decision-making process. Overall, knowledge uncertainty can be reduced, but inherent variability cannot. Thus, resources should be allocated mainly to reducing the uncertainties that are knowledge-based (Yoe 2001). There are specific ways to deal with different kinds of uncertainty. Some kinds of knowledge uncertainty may be because of a lack of information or skill (e.g., an inability to list the species dependent on a given kelp bed). The uncertainty can be reduced by obtaining the knowledge through education, training, or talking to experts. Those things that are currently unknown may require more research and/or time. Some things are fundamentally unknowable (e.g., value-based questions about the “right” thing to do, or philosophical questions). Other knowledge uncertainties include things that are random or unpredictable, such as the next year a hurricane will hit the project site. Because there is no solution to these questions, the situations are treated as part of inherent variability and often represented with probabilities. Quantity uncertainty is often a major source of uncertainty in restoration planning. For example, we may need to know the current resident population of deer mice in the study area. Although there is a correct single number, we will likely never know the true population with certainty. We use numbers in many of our estimates that assume measurements have been made. In many cases we have made measurements or know the cost of an item, but still have uncertainty. For instance, you cannot determine an exact mean stream temperature even after recording 100 stream temperatures, or know for certain the cost of planting 150 meters of willows along a streambank next May even if you know it would cost $1000 to plant today. Models often use mathematics to simplify reality and represent complex situations and systems, with the purpose of representing a functioning system. They can be used to predict everything from the impacts of climate change (Willows and Connell 2003) to the number of bald eagles nesting in the state in 10 years under various scenarios. Examples of model uncertainties include how well the models represent complex ecosystem relationships and how reasonable the assumptions are that underlie the model. 4) Identify the approaches used to address identified uncertainties (use a reference, such as Yoe 1996) and then perform the risk analysis A reference can provide guidance on specific ways to address particular kinds of uncertainties. Although we can group the uncertainties into two categories (i.e., inherent variability and knowledge uncertainty), many subsets of uncertainty types, such as the quantity or model uncertainties described above, must also be considered. The uncertainties can be dealt with in a number of ways ranging from using professional judgment or consulting experts, to performing a sensitivity analysis, to representing uncertainty with probabilities based on a distribution of the expected values of the uncertain variables. Again, the level of detail in the analysis is dependent on the level of detail required to make a decision. Because they are often used to perform risk analysis, sensitivity analysis and computer simulation models are briefly described below. Sensitivity analysisA sensitivity analysis measures the impact on the expected outcome of an analysis of changing the value of one or more important, uncertain variables ( Marshall 1999). For example, a computer model can be used to estimate the total cost of a 20-day project if required costs are known. Total costs can then be calculated based on different time variables, such as an 18-day or a 25-day project. Planners systematically try different values for variables considered to be uncertain, most often to reflect optimistic or pessimistic scenarios to establish boundaries for the expectations of some part of the project. By systematically changing the assumptions and examining their impacts on the outcome of the project, a planner is able to make better decisions about whether to proceed as planned, or to try to resolve some of the uncertainties before going ahead. Many sensitivity analyses involve the creation of alternative scenarios for groups of uncertain variables considered critical to the analysis, that together might characterize themes, such as “no action,” “aggressive,” “optimistic,” “pessimistic,” “low cost,” “high cost,” and others. The value of a sensitivity analysis is to consider the optimistic, expected, and pessimistic scenarios for a project, with reasonable values substituted for the uncertain ones. A sensitivity analysis does not estimate the likelihood of each scenario occurring, so it gives results different from those of analyses based on probabilities (Yoe 2001). Incorporating Probabilities into Risk AnalysisAnother way to quantify the risks of a restoration project is to use probability and statistics. Probability is the likelihood of something happening (e.g., the likelihood that water flows will average 55 cubic feet per second). Statistics are mathematical ways to summarize and describe large amounts of uncertain information (Males 2002). When a range of possibilities exists for an important variable that the planner must consider, a number of generic mathematical formulas, such as uniform, triangular, and normal, can be used to describe the distribution of possibilities. For example, it might be estimated that between 800 and 1100 cubic yards (cy) of dredge material might be required to fill the hole left behind after a construction barge ran aground. Furthermore, costs could be estimated at not less than $95/cy, but as much as $200/cy. The best-case scenario involves small quantities and low costs; the worst case is the opposite. These best and worst cases result in costs of $76,000 and $220,000. Although the costs are bracketed, there is no indication of how likely either of these scenarios will be. To incorporate probability, one or more of the variables (e.g., quantity, cost) must be replaced with distributions. Distributions define the likelihood of various values throughout a range. If all values are equally likely (e.g., if it is just as likely for the project to require 100 cy of material as 1100 cy of material, then there is a uniform distribution) (Figure 2).
If something more is known about the variables, then a different distribution can be used. For example, if 1000 cy is the most likely quantity of dredge material that will be used in the project, then a minimum, maximum, and a most likely value can be identified. This describes a simple, triangular distribution (Figure 3).
The final example of a distribution, which is useful for representing many real-world processes, is the normal distribution (Figure 4). This distribution pattern follows a bell-shaped curve. The bell-shaped curve has values that are concentrated in the center and that decrease on either side. This means that the there is less of a tendency to come up with unusually extreme (i.e., very high or very low) values, compared to some other distributions. Because the bell-shaped curve is symmetric, the probability of deviations from the mean value is comparable in either direction.
Commercial risk analysis computer software is available that will calculate the outcome of hundreds or thousands of possible scenarios and then study the results. Many of these use what are known as Monte Carlo simulation modeling techniques. The computer programs can run the simulation, substituting random sample values for the uncertain variables (e.g. quantity and cost) that it generates based on their likelihood under a specific distribution (Males 2002). Each new calculation is called an iteration of the model, and a simulation is a collection of many iterations (Yoe 2001). For example, assume a computer model is used that employs the triangular distribution in Figure 3 as the basis for obtaining random numbers for the amount of dredge material that will be used to restore a site. After hundreds of iterations one would expect that most iterations substituted a number near 1000 cy for the quantity, but that several iterations substituted numbers closer to the low or high end of the range (i.e., closer to 800 cy or 1100 cy). After a sample simulation of 10,000 iterations, a result might indicate that there is a 90 percent chance that the costs will be between $93,790 and $173,770. The simulation also may provide insight into which uncertain variables are most significant in determining an overall outcome. Available resources can then be applied toward refining the estimates for those variables to reduce the risk and the uncertainty further. Once the risk analysis is complete, it is good practice to have it peer reviewed as a means to check the analysis for integrity and credibility before a final decision is reached. The assessment of the contributions of each of the various plans to meeting objectives may provide the most objective means of comparison (Yoe 1996). A good risk analysis will help to prevent bad investments or ensure that good ones are not overlooked. For example, suppose two plans have similar outputs, but one plan is estimated to cost slightly more. Under some cost-effectiveness guidelines, the more expensive project could be dropped from further consideration. A risk analysis, however, might indicate that under some scenarios, the more expensive plan would have a greater potential of producing the desired ecological output than would the less expensive plan. In this case, it would be remiss not to consider it as a viable option (Males 2002). These examples are a few ways in which to project future conditions if some of the important variables are uncertain or carry potential risk. The results of the analysis provide the planner with a means to compare a variety of plans or results under potential conditions while knowing the inherent risks associated with each one. Full documentation throughout the risk analysis is essential. It is important to describe the initial uncertainty, the steps taken to reduce the uncertainty, and what uncertainty remains. The hazards, probabilities, and consequences of risky situations are equally important to disclose. After the restoration project is completed, monitoring can provide information on the performance of the project over time. The process of adaptive management is extremely useful in this regard. At regular intervals, the success of a restoration project (based on the goals and objectives specified during project planning) is measured or assessed using the results of the monitoring program. Careful evaluation of credible evidence from monitoring may be followed by adjustments to the program that are likely to improve overall project success. For example, projects have been modified when they appear to be failing, when technical knowledge improves, or when social preferences change. As a restored system evolves, such as a formerly diked area that has been breached and opened to tidal inundation, it is typically subject to stresses that can significantly alter the course and progress of the project (Thom 1997). After the restorative action is taken, the system may exist in a number of states over time. In the first year following restoration, the plants and animals one would expect to find in a natural salt marsh may not be present. Over a number of years, however, plant species tolerant of high salinities would be expected to colonize the restored area. A potential setback could be partial colonization of the restored area by a non-native, invasive plant species. The results from the monitoring program could be used to determine the rate and extent of the nuisance species invasion. Current information on nuisance species, control methods, and policies of local, state, and federal agencies on control of nuisance species, would help project managers decide whether any action to eradicate the invasive plant should be taken.
The process of project planning, restoration (action), monitoring, evaluation, and adjustment can be repeated as many times as necessary to keep the project on track toward meeting all of its objectives (Figure 5). Careful tracking and dissemination of this information will help others learn from the project’s successes and failures, ultimately reducing uncertainties associated with similar projects in the future (Noble and others 2000). In summary, uncertainty and risk exist for all environmental restoration projects. It is important clearly to state the project goals and objectives and to determine the most important sources of variability. Once the important variables have been identified, the types of uncertainty can be determined and suitable approaches for addressing the uncertainties identified. This process may be as simple as using best professional judgment or as complex as performing a computer simulation based on probability distributions. By clearly communicating the types and assessments of risk and uncertainty to those involved in the decision-making process, the “best” plan can be selected for implementation. Regardless of the method of risk analysis used, the project objectives should be matched with monitoring methods that allow the success of the project to be measured or assessed at regular intervals. Applying the continual evaluation process of adaptive management leads to cost-effective, successful restoration projects. References*Some of the documents below are in Adobe portable document format (PDF) and requires Adobe Acrobat Reader. Knight, F.H. 1921. Risk, Uncertainty, and Profit. Houghton Mifflin. Boston , MA . Available from http://www.econlib.org/library/Knight/knRUP1.html; accessed 20 January 2004 ; Internet. Males, R.M. 2002. Beyond expected value: Making decisions under risk and uncertainty. IWR Report 02-R-4. Institute of Water Resources, U.S. Army Corps of Engineers. Alexandria, VA. Marshall, H.E. 1999. "Sensitivity Analysis" in Technology Management Handbook. Dorf, R.C. ed. Press. Boca Raton, FL. Thom, R.M. 1997. "System-development matrix for adaptive management of coastal ecosystem restoration projects." Ecological Engineering. Volume 8. Pages 219 to 232. Water Resources Council. 1983. Economic and environmental principles and guidelines for water and related land resources implementation studies. Government Printing Office. Washington, D.C. http://www.iwr.usace.army.mil/iwr/pdf/p&g.pdf Accessed December 3, 2003 . Yoe, C.E. 1996. An introduction to risk and uncertainty in the evaluation of environmental investments. IWR Report 96-R-8, prepared for the U.S. Army Corps of Engineers, Institute for Water Resources. Alexandria, VA. Yoe, C.E. 2001. Ecosystem restoration cost risk assessment. IWR Report 02-R-1, prepared for the U.S. Army Corps of Engineers, Institute for Water Resources. Alexandria, VA. Willows, R.I., and R.K. Connell (eds.) 2003. Climate adaptation: Risk, uncertainty and decision-making. UKCIP Technical Report. UKCIP. Oxford, UK. |
NOAA Coastal Services Center
2234 South Hobson Avenue • Charleston, SC 29405-2413
843-740-1200 (phone) • 843-740-1224 (fax)
Directions to the NOAA Coastal Services Center
United States Department of Commerce
National Oceanic and Atmospheric Administration
National Ocean Serivice
Privacy Policy
Link Policy
Contact the NOAA Coastal Services Center
Updated on July 24, 2007