Home » Articles » Volume 37 » Issue 4 » Science, Risk, and Risk Assessment and Their Role(s) Supporting Environmental Risk Management

 
 

Science, Risk, and Risk Assessment and Their Role(s) Supporting Environmental Risk Management

 

View PDF

Abstract

The U.S. Environmental Protection Agency (EPA) fulfills its mission of protecting public health and the environment by, among other things, developing and enforcing regulations that implement environmental laws enacted by Congress. Ensuring that its regulations have a sound analytical foundation reduces both controversy and, to some extent, court challenges, and increases the likelihood of compliance by the regulated community, which is key to achieving real environmental improvement. The environment, risk, and environmental risk are case- or site-specific and too complex to capture fully. EPA uses risk assessment as a key source of scientific information along with other relevant information (e.g., costs) for making good, sound decisions about managing risks to human health and the environment. Risk assessment is a necessary tool used to inform decisions where direct measurements are not possible. While risk assessment involves science and is a scientific activity, it is best described as “trans-scientific”; normative elements and judgment are inherent. EPA has instituted numerous processes and systems to make risk assessments tractable and feasible, while ensuring their overall quality. This Article reviews risk assessment and its role in risk management decisions, with emphasis on science and policy influences on procedures for conducting such assessments and making such decisions, and vice versa.

I. Quality Decisions Needed to Meet EPA’s Mission of Environmental Protection

The mission of the U.S. Environmental Protection Agency (EPA) is to protect human health and to safeguard the natural environment-air, water, and land-upon which life depends. EPA fulfills this mission by, among other things, developing and enforcing regulations that implement environmental laws enacted by Congress. Inferred is the fact that successful environmental problem-solving must encompass not only “what must be done” (setting a standard or risk management objective), but equally “how it shall be accomplished” (implementation and enforcement). Determining environmental standards, policies, guidelines, regulations, and actions requires making decisions considering this full spectrum. The quality of any decision and resulting action “determines how well environmental programs actually work and the extent to which they achieve health and environmental goals.”[1]There are numerous factors which impact the quality and success of any decision or action, and many of these are competing or contradictory forces, such as:

Legally defensible <<<<<<<< >>>>>>>>> Clear and Concise
Comprehensive <<<<<<<< >>>>>>>>> Simple
Flexible <<<<<<<< >>>>>>>>> Easy to implement
Sound analysis <<<<<<<< >>>>>>>>> Timely and inexpensive

The quality and acceptability of any decision is one that effectively balances these factors.

As described in Risk Assessment Principles and Practices:

Determining environmental standards, policies, guidelines, regulations, and actions requires making decisions which are often contentious. Setting an environmental standard that is too lax may threaten public health or the environment, while a standard that is unnecessarily stringent may impose a significant marginal economic cost for small marginal gain. Environmental decisions are often time-sensitive, for example when public health is known or suspected to be at risk. The decisions must frequently be made with incomplete or imperfect information and many times under the additional pressure of heightened public scrutiny and concern. And, once made, the decisions are often challenged in court and subject to high levels of public and scientific scrutiny.[2]

As a result, such contentious decisions must be based on the current state of knowledge-certainty is not required, and appropriate means must be used. Rational support for answers to key questions and an estimate of confidence in the decision must be provided.

There are often conflicting interests bearing on environmental decisions, and as a result, it is well recognized that it is important (and in some cases even mandated) to consider a broad range of factors when making decisions about risk management including:

·     Risk-nature, magnitude, severity, and likelihood of adverse outcomes/effects

·     Economic factors-costs and benefits of risks and risk mitigation alternatives

·     Laws and legal decisions-framework that prohibits or requires some actions

·     Social factors-attributes of individuals or populations that may affect their susceptibility to risks from a particular stressor

·     Technological factors-feasibility, impact, and range of risk management options

II. Role of Science

EPA’s credibility depends on the science and analysis underlying its decisions. Ensuring that its regulations have a sound analytical foundation reduces controversy and, to some extent, court challenges. It also increases the likelihood of compliance by the regulated community, which is key to achieving real environmental improvement[3]and public confidence.

As noted above, science is one factor of many which contribute to a decision. In reality, most decisions are based on a balance of factors, and in some cases may even be influenced or determined solely by factors other than science, such as political or economic considerations.[4]Allowing decisions to be influenced or directed by factors other than science is, of course, legitimate. However, there is a tendency for decision makers to look to science to support or justify their positions (“the science made me do it”).[5]Science is often viewed as objective, value-neutral, and concrete, and many feel that if science can support a decision, it will be non-controversial and acceptable to the broadest range of stakeholders. Highlighting the fact that a decision was based on factors other than science illuminates the underlying values supporting the decision. As a result, this may provide a motivation to influence or undermine the science to provide a science-based rationale for the decision.[6]

The distinction between science-informed decisions and science-based decisions is critical. Rarely is there sufficient scientific certainty which unequivocally points to or supports a specific decision or action. Quality and defensible decisions are those which are rational, where science is used to inform, and where the limits of science are made clear. Decisions presented as science-based are often supported by rationale alone and are subject to criticism and challenge. To maintain the quality of environmental decisions, the EPA must ensure a science-informed focus that relies on high quality scientific information. The rest of this Article will provide an overview of how EPA approaches this.

III. Risk Assessment as a Key Source of Scientific Information

Risk assessment is a key source of scientific information for making good, sound decisions about managing risks to human health and the environment.

EPA conducts risk assessment to provide the best possible scientific characterization of risks based on a rigorous analysis of available information and knowledge-that is, a description of the nature and magnitude of the risk, an interpretation of the adversity of the risk, a summary of the confidence or reliability of the information available to describe the risk, areas where information is uncertain or lacking completely, and documentation of all of the evidence supporting the characterization of the risk . . . . Risk assessment, therefore, informs decision makers about the science implications of the risk in question . . . . The primary purpose of a risk assessment is not to make or recommend any particular decisions; rather, it gives the risk manager information to consider along with other pertinent information.[7]

To better understand the role of science in EPA, as manifested through risk assessment, this section focuses on defining risk and risk assessment, the role of policy and judgment, and procedures or practices the EPA uses to provide the highest quality information to decision makers.

A. What is Risk?

Risk is an inherent consequence of life that is not possible to altogether avoid or eliminate.[8]Risk is a very elusive and often contentious concept, but in general, it is a concept that denotes a potential negative impact or outcome that may arise from some present or future process.[9]The definitions of risk depend on specific contexts or applications, and there are multiple dimensions to risk, ranging from the tangible and quantitative to the psychological and emotional. Kaplan and Garrick describe risk as consisting of three components: 1) outcome(s), 2) likelihood, and 3) severity.[10]Most importantly, risk is the complete range of possible outcomes, their severity, and their likelihood, and not just the actual outcomes which have occurred or will actually occur.[11]Another critical dimension of risk is uncertainty-the fact that outcomes are uncertain makes them risk, and unavoidable.[12]These concepts are illustrated in the following quote:

Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns-the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.[13]

Each of the italicized elements represents an aspect of risk.

B. What is Environmental Risk?

If risk generally is the potential negative outcomes arising from current or future processes, what does environmental risk mean? The environment is a complex and interconnected web of organisms, systems, and processes at various levels of organization. Environmental risks can be described as results from the stressors (natural or anthropogenic) which negatively impact these organisms, systems, and processes.[14]Any perturbation (stressor) that impacts this complex web of interdependent systems can have a multitude of potential outcomes. Figure 1 presents a conceptual model of the theoretical pathways and routes of exposure between stressors (and sources of stressors) and effects (endpoints) for human and ecological receptors. Further complicating the assessment of environmental risks is the fact that not only can there be a wide range of impacts associated with stressors in the environment, but environmental releases, fate and transport (environmental concentrations), exposures (contact), and biological responses will tend to vary both spatially and temporally.[15]

While the phrasing is different in various statutes, and while different judicial interpretations certainly influence how these mandates are put into practice, EPA offices are faced with qualitatively similar mandates that point

to at least some level of comparability between assessments.[16]One way to state a general risk goal under these diverse mandates is to protect an appropriate fraction of the population from exposures that produce unacceptable risk (of adverse effects), and to do so with some appropriate degree of confidence.[17]

Figure 1: Generic Conceptual Model

The fundamental policy questions that should be answered to determine any risk-based goal are:

1.      Who are we protecting?-e.g., sensitive individuals, individuals, populations, communities

2.      What are we protecting them from?-e.g., adverse effects within a continuum of responses, perturbation of compensatory processes, clinical signs of disease

3.      What constitutes an appropriate degree of confidence in that protection?

C. The Need for Risk Assessment

Given the complexity of the environment, risk, and environmental risks, it is not feasible to measure or assess environmental risks fully. The natural environment is a complex web of systems at various levels of organization. The makeup of the environment and the mix of stressors (in both composition and magnitude) vary dramatically over time and space, and the environment is comprised of interconnected systems which are not distinct. As a result, it is not feasible to directly isolate or measure all impacts with respect to any past or ongoing stressors, or to predict future conditions.

Any full analysis of risks must consider the entirety of the system. Thus, risk analysis must be interdisciplinary, drawing on diverse fields such as biology, toxicology, ecology, engineering, geology, statistics, and even social sciences, to create a rational framework for evaluating environmental hazards. A significant consequence of risk analysis’ broad scope is that “the most powerful method of science-experimental observation-is inapplicable to the estimation of overall risk in exactly those instances where public policy most demands assessment of risk.”[18]

Risk assessment is used to address problems which cannot be directly measured and has one purpose: to support the decision-making process. While risk assessment involves science and is a scientific activity, it is not science per se.[19]Risk assessment focuses on “questions which can be asked within the framework of science, but which are beyond the capacity of science to answer” through focused observational experimentation and is thus “trans-scientific.”[20]

D. How to Make Risk Assessment Tractable and Feasible?

The environment, risk, and environmental risk are site- and case-specific and too complex to capture fully. Thus, for a risk assessment to be tractable, the policy questions driving the analysis must be limited to analytically manageable problems. Any thorough assessment of the environment and environmental risk would be extremely resource intensive, and mostly infeasible, especially given the number of decisions and assessments facing EPA. Therefore, there is a push toward reducing the resource demands of risk assessment without compromising the overall quality of the assessment. As a result, in the general sense, the Agency has pursued the use of defaults, guidelines, and shared tools.

Default assumptions are used to address inherent uncertainties and data gaps. A default assumption is “the option chosen on the basis of risk assessment policy that appears to be the best choice in the absence of data to the contrary.”[21]While often controversial in how they are developed and applied, the scientific community has long supported EPA’s use of defaults as a reasonable way to deal with uncertainty.[22]

Guidelines are developed to help guide EPA scientists assess risks and to inform decision makers and the general public about its procedures. Specific guidelines cover various aspects of risk assessment, such as carcinogenicity, exposure, neurotoxicity, and ecology.[23]The first set of EPA guidelines was published in 1986,[24]and they are revised or new ones developed as experience and scientific understanding evolve. For example, the EPA published guidelines for carcinogen risk assessment in 2005.[25]Tools set forth in the guidelines consist of models, databases, and analytical frameworks to support or guide specific analyses.

While such approaches certainly promote efficiency and consistency, some view this enforced procedural consistency as a substitute “for serious evaluation of the consistency and overall desirability of the uncertain range of risk outcomes.”[26]It should also be noted that these approaches each represent a simplification of the assessment. Choices are made to simplify the problem either in reducing the scope and/or the degree to which natural processes are fully captured. An additional approach EPA pursues in making risk assessments tractable and feasible is the iterative approach to risk assessment. The iterative approach begins the risk assessment with a simple screening analysis and moves as needed to a more detailed resource-intensive analysis.[27]Risk assessments can be very complex and resource intensive, and may vary in terms of complexity and rigor. This is ultimately a balance of cost (resources) and uncertainty. Generally, the more complete or rigorous the assessment, the lower the uncertainty but the greater the cost, and vice versa (i.e., the lower the rigor, the lower the cost, but the greater the uncertainty). The rigor in specific risk assessments is often geared toward the decision and the needs of the decision maker. The iterative approach to risk assessment accounts for these concerns-balancing the resource constraints versus improving the scientific basis decisions are based upon-by increasing complexity of the assessment only as needed.[28]

E. Judgment, Normative Values, and Policy Choices are Unavoidable

While many view science as objective and value-neutral, judgment is in fact inherent and unavoidable both in conducting experimental observations and in the interpretation of those observations. For example, judgment is critical and unavoidable in stating a hypothesis, study design, sampling strategy, and statistical analysis. “Implicit in scientific inference is the role of professional judgment.”[29]This is especially true-even exacerbated-in risk assessment, which essentially consists of integrating vast amounts of data and evidence across multiple disciplines that are not always subject to verification through empirical observation.

Risk assessments are intended to support decisions which are influenced by statute, values, and policy; therefore, to effectively support decision makers, assessments must address or account for these factors. Any decision regarding the scope of an assessment, its underlying assumptions and analytical approaches, and how uncertainty is handled are all informed by policy, judgment, and statute. Each of these often has associated normative foundations which are then integrated into the assessment and in the end are indistinguishable from the pure science.[30]For example, consistent with its mission, the EPA errs on the side of protecting public health when assessing risks.[31]Some suggest that risk assessments should be value-free or objective, without normative foundations.[32]However, it should be noted that such an approach is itself value-laden. For example, focusing an assessment on the “best” (e.g., maximum likelihood) estimate of an average person allows greater risk and harm to more susceptible persons and fails to consider risks which have not yet been conclusively proven.

As noted above, given the inherent nature of uncertainty and data gaps, it has become accepted practice to use defaults to address such deficiencies.[33]The use of default options has long been supported,[34]though the principles for choosing defaults go beyond science and inevitably involve policy choices.[35]These defaults can be incorporated into or serve as the foundation for general EPA guidelines (e.g., linear low-dose extrapolation) or specific assessments (e.g., exposure or parameter values). In general, the EPA uses defaults that are conservative; conservative defaults, although scientifically plausible given existing uncertainty, are more likely to result in overestimating rather than underestimating risk.[36]The EPA believes this conservative bias is aligned with its mission to protect public health and safeguard the environment.[37]

The use of conservative defaults has long been the basis for criticism levied at EPA risk assessment practices.[38]These critics contend “that EPA has so overemphasized conservatism that most risk estimates are alarmingly false, meaningless, and unscientific.”[39]The EPA responds that “the use of default assumptions does not render the process or results non-scientific.”[40]Risk assessments inevitably involve varying degrees of scientific uncertainty. Therefore, while science remains a necessary element of risk management decisions, it may be insufficient in the regulatory process. As uncertainty increases, it is unavoidable that risk management decisions become more conservative in nature: this is not necessarily unscientific.[41]Others have suggested that the claims of these critics of conservatism tend to be more reflexive, undocumented by evidence, and exaggerated than EPA’s risk estimates.[42]“[S]ome of the intensity marking this debate is due to a variety of misimpressions about what conservatism is and what its ramifications are.”[43]Regardless of one’s position, judgment and values are inherent and unavoidable in risk assessment. Therefore, the EPA must strive to be transparent.[44]

F. Statutory Influences in Risk Assessment

As noted in the Staff Paper Risk Assessment Principles and Practices, despite standardized guidelines and methodologies, apparent inconsistencies in risk assessment practices across the Agency may stem from differences found in the statutory language.[45]Such language on risk and protection varies, opening the way for subtle differences in interpretation. For example, a single EPA entity, the Office of Air and Radiation, administers statutes with varying mandates:[46]

Protect public health with “an adequate margin of safety”[47]

“[P]rovide an ample margin of safety to protect public health . . . or to prevent . . . an adverse environmental effect”[48]

“[P]rotect the public welfare from any known or anticipated adverse effects” [49]

“[Not] cause or contribute to an unreasonable risk to public health, welfare, or safety”[50]

“[P]rotect sensitive and critically sensitive aquatic and terrestrial resources”[51]

“[R]educe overall risks to human health and the environment”[52]

“[M]itigate . . . environmental and health risks”[53]

In addition to the variability of the language found in the various statutes EPA administers, the purpose and scope of risk assessments can differ. As a result, it is not surprising that differences in terms can lead to subtle and maybe significant differences in risk assessment practices across the EPA.

G. Limits of Risk Assessment

Risk assessment represents the key source of scientific information used in making decisions about managing risks to human health and the environment. Risk assessment is a necessary tool used to inform decisions when direct measurements are not possible.[54]However, one must be aware of the nature of risk assessment so as not to place undue and unreasonable demands upon it. While risk assessment is a key source of scientific information, it is not a science.[55]Risk assessments are only an approximate, incomplete, and unverifiable description of reality. While risk estimates are not absolute predictors of risk or descriptions of truth, they can provide useful information to decision makers. After all, “all models are wrong but some are useful.”[56]

Uncertainty is inherent and unavoidable in risk assessment, but this does not impugn the integrity and utility of assessment results. While uncertainty greatly impacts the ability to develop environmental policy and regulation, it does not necessarily prevent action. Risk management is characterized by decisions and actions in the face of uncertainty, and public health and environmental protection in general requires such action. The courts have long deferred to agency discretion in responding to uncertainty, so the presence of uncertainty does not challenge specific decisions, rules, or regulations.[57]

IV. Ensuring Quality Science to Support EPA Decisions

Given all of the limitations of risk assessment-the need to balance resources with detailed analysis, implicit normative aspects, uncertainty, and the context-specific nature of these assessments-how can the EPA ensure its assessments are of the highest quality? The EPA has an ongoing commitment to ensure the quality of information used to support Agency decisions and actions.[58]Such efforts are achieved through existing policies, systems, and programs which are broadly discussed in EPA’s Information Quality Guidelines.[59]The quality of science and analysis that underlie EPA regulations is vital to the credibility of EPA decisions and, ultimately, its effectiveness in protecting human health and the environment.[60]In 2001, EPA Administrator Whitman directed the agency to convene a task force to reexamine its “regulatory development process and identify ways to strengthen and improve the quality of supporting scientific, economic, and policy analysis.”[61]In general, the task force found the existing system for developing regulations to be “well designed, but that certain areas needed improvement.”[62]The task force recommended that the EPA reaffirm the Action Development Process and the need for consistent use of Analytic Blueprints and agency-wide involvement in the Options Selection process. [63]

A. Action Development Process (ADP)[64]

The EPA publishes hundreds of actions a year that define the technical and operational details of environmental programs. Some actions are fairly narrow and routine, while others may be broad and complex, but all must be of consistently high quality. The ADP is a method for producing quality actions. It ensures that the agency uses quality information to support its actions and that scientific, economic, and policy issues are adequately addressed at the proper stages in action development. The EPA’s ADP guidance identifies common characteristics of quality actions:[65]

·     Achieve environmental objectives cost-effectively;

·     Are consistent with legal requirements, executive orders, directives, Agency guidance, and national policies;

·     Reflect EPA-wide involvement when necessary, in particular, involvement of offices with cross-cutting responsibilities;

·     Reflect appropriate consideration of the views outside EPA;

·     Consider multimedia effects;

·     Consider pollution prevention principles and innovative alternatives during the inves­tigative and development process;

·     Are based on sound economic, scientific, legal, policy, and technical analyses;

·     Can be efficiently implemented and effectively enforced;

·     Are clear, concise and written in plain language; and

·     Are timely.

At each step in the ADP, the soundness of any scientific analysis or information is reviewed and assessed.[66]Resources are not always available to fully meet all of the characteristics of quality actions.[67]As a result, decisions are often made to focus an assessment to ensure that it is practical, feasible, and useful.[68]Science plays a prominent role at various points in the process, including in early management guidance and during formation of the analytical blueprint.[69]

B. General Assessment Factors[70]

The EPA relies on information integrated across a broad range of disciplines to develop an overall assessment.[71]This “‘weight-of-evidence’ approach considers all relevant information in an integrative assessment that takes into account the kinds of evidence available, the quality and quantity of the evidence, the strengths and limitations associated with each type of evidence, and explains how the various types of evidence fit together.”[72]How a body of evidence is integrated “depend[s] on the type of decision or action being undertaken.”[73]While there may be specific guidance on how the weight-of-evidence is approached for specific applications such as carcinogenicity, there is also a need for more general guidance, which is provided by the Summary of General Assessment Factors:[74]

·     SoundnessThe extent to which the scientific and technical procedures, measures, methods, or models employed to generate the information are reasonable for, and consistent with, the intended application.

·     Applicability and UtilityThe extent to which the information is relevant for the Agency’s intended use.

·     Clarity and CompletenessThe degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations, and analyses employed to generate the information are documented.

·     Uncertainty and VariabilityThe extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods, or models are evaluated and characterized.

·     Evaluation and ReviewThe extent of independent verification, validation and peer review of the information or of the procedures, measures, methods, or models.[75]

The Science Policy Council describes how the factors are used by the EPA:

These assessment factors reflect the most salient features of EPA’s existing information quality policies and guidelines. Whether the information consists of scientific theories, computer codes for modeling environmental systems, environmental monitoring data, economic analyses, social survey or demographic data, chemical toxicity testing, environmental fate and transport predictions or a human health risk assessment, EPA generally evaluates information by weighing considerations that fit within these five assessment factors. Thus, these factors encompass considerations that are weighed in the process of evaluating the quality and relevance of information. The appropriate level of quality for any particular information product is necessarily related to how and in what context the information is to be used.[76]

C. Peer Review[77]

Strong, independent science is of paramount importance to the quality and credibility of EPA policies and actions. One important way to ensure decisions are based on defensible science is to have an open and transparent peer review process. Consistent Agency-wide implementation of peer review has been an EPA priority for many years.[78]The EPA issued its first Peer Review Policy in 1993 and its first Peer Review Handbook in 1998. The Handbook was updated in 2000 based on feedback from EPA’s science community and again in 2006 in response to the Office of Management and Budget’s 2004 “Final Information Quality Bulletin for Peer Review.”[79]This policy encourages and expects peer review of all scientific and technical information that is intended to inform or support EPA decisions.[80]Depending on the particular product, peer review may be accomplished internally, using independent EPA experts, or externally, using independent outside experts.[81]The policy also notes that for influential scientific information, including highly influential scientific assessments, peer review is expected, and that external peer review is the preferred approach.[82]

V. How Can It Go Wrong?

The EPA has developed numerous systems, procedures, and processes to ensure that the highest quality science is used to support its decisions.[83]Despite these steps, there are often concerns, real or perceived, that science is improperly used or characterized to support specific decisions. The following are potential situations that are susceptible to or could lead to improper use or characterization of science:

Project scoping (early guidance)-As risk assessments are intended to meet specific purposes or to support specific decisions, it is critical that proper input and direction be obtained from the decision maker as to what is important to the decision, to help identify specific issues and determine the technical approach. However, the potential exists to use this phase to define a desired outcome rather than identify issues and the technical approach as intended. Also the potential exists to selectively define the problem to meet a predetermined outcome or action.

Demand for certainty-Risk assessment uses a systematic approach to estimating or characterizing an (albeit incomplete) “illustration of the world.” Uncertainty, defined as a lack of precise knowledge as to what the truth is (qualitative or quantitative), is inherent in this illustration.[84]As such, it cannot demand the certainty and completeness of science.

Targeting of specific uncertain elements-Risk assessments are very complex and require the integration of information across a broad range of disciplines, and often consist of numerous components, each with models and multiple parameters. Uncertainty is inherent and unavoidable in the overall assessment as well as in each component. The uncertainties for any component, model, or parameter can be focused on to impugn the integrity of the overall assessment or to initiate a detailed and protracted debate. Such efforts may focus on (albeit uncertain) elements which may have little or no impact on the overall assessment or the decision.

Limited resources-Risk assessments are highly complex, and thorough assessments are resource intensive; it simply is not possible to conduct such an assessment for every application. As described above, the EPA attempts to balance the resource demands with the needs of the decision to ensure the assessment is reasonable and appropriate for a specific decision which may impact the overall uncertainty in any assessment.[85]Furthermore, it is not uncommon for review of risk assessments by outside parties to consume more resources in reviewing those assessments than were expended in the conduct of the assessment in the first place. As a result, specific uncertainties can be highlighted that lead to demand for certainty or targeting of specific uncertain elements.

Process / administrative requirements-There are numerous administrative demands and burdens outlining the conduct and use of risk assessments. Many of these (e.g., OMB Circular A-4 requirement for full regulatory analysis, such as a benefit-cost analysis[86]) may pose undue or unreasonable demands on specific assessments which are either not possible given the current state of knowledge, or infeasible given finite resources. As a result, it is not possible to meet all requirements to their fullest extent for all assessments. While not fully meeting all requirements does not necessarily impugn the overall integrity or utility of a particular assessment, it does make these assessments susceptible to challenge. Specific controversial assessments may be challenged or targeted through selective enforcement of requirements.

Who decides on quality of science and when?-The ADP delineates who evaluates the quality of science and when this evaluation takes place,[87]though it is less clear who or when this is done in external review of EPA products. OMB has a clear role in reviewing EPA actions and supporting analysis,[88]though there are several avenues where such evaluation may occur. Review of EPA regulatory packages is the most prominent,[89]however, additional review may occur via other authorities, such as OMB’s general Information Quality Guidelines,[90]those specific to Peer Review,[91]and most recently their Good Guidance Practice.[92]Given the ambiguity of many provisions or definitions, they have wide flexibility to identify specific products at varying steps in the process. Additionally, much of their review may not be publicly transparent.

Differentiating science from policy-As described above, risk assessments are intended to support decision making, not purport “truth,” and they are considered a “trans-science” activity. As a result, judgment and normative aspects are inherent in any risk assessment, and what is pure science is not easily distinguishable. Being relatively indistinguishable from policy, it makes risk assessment susceptible to challenge under the rubric of policy and law, and the processes for resolving such matters are more subjective than typical rules of science. As a result, it may be difficult to challenge the use of science as rationale (supporting a presupposed position) as opposed to the preferred approach of using science in a rational process for decisions.

VI. What Can the EPA Do To Maintain and Improve the Role of Science?

Environmental decision making and risk assessment are not wholly scientific but rather utilize and rely on science for their foundations. These activities can best be described as a weighing of evidence provided by all available scientific data, and other relevant information and considerations.[93]Since risk assessment and environmental decision making are complex and rely on multiple disciplines, the weight-of-evidence requires the combined input of relevant disciplines such as toxicology, biology, chemistry, epidemiology, statistics, and engineering. While views on the “state of the world” may change significantly when other data are brought into consideration, the overall decision is based on the totality of the evidence; no single study, whether positive or negative, drives the overall weight-of-evidence judgment.[94]Judgment on the weight-of-evidence involves consideration of the quality and adequacy of the available data and consistency across lines of evidence.[95]

Weight-of-evidence is a commonly accepted but elusive concept. It is a common term in scientific and policy-making literature, but its definition is unclear, defies categorization, and is variably applied. Clearer and more transparent definitions of what is meant by “weight-of-evidence” and how it is used will enhance the quality of risk assessments used to protect and improve public health and the environment.[96]A formal definition can lead to a more formal description of the process as well, and clearly delineate scientific evidence and how it is used. Crawford-Brown has proposed such a framework, defining the concept of “sound science” as a dialogical process rooted in rational exploration of the evidence.[97]This focus on rationality, if formalized, could greatly reduce the desire or ability to use science as a rationale, thereby maintaining its integrity and minimizing the potential, perceived or real, to manipulate science.

Should such a framework be agreed upon and defined, the next step would be to formalize the process. With formal schemes in place, there is a basis upon which to evaluate science, and more importantly evaluate how science is used. Finally, transparency is the foundation of a democratic society, and it is critical to ensure quality decisions and public confidence. Related to transparency is accountability to the process and the decisions.

A final point is the acknowledgment that uncertainty is inherent and unavoidable, and that decisions are made in light of such uncertainty. As a result, we cannot expect perfect decisions but rather hope for the “best” decisions under the current state of knowledge. The courts have continuously deferred to agencies and provide them the flexibility to act appropriately given attendant uncertainties.[98]If we had complete information or “truth” about the state of our world, we would make an optimal choice which maximizes the benefits of a risk reduction or control mitigation measure relative to the cost of that measure. However, given the inherent uncertainty, a chance that the wrong decision will be made is unavoidable. Such an acknowledgment on the part of the decision makers, and especially the public, may minimize the need for definitive answers and the misuse or mischaracterization of science to support any specific decision. While this will not alleviate the pressures to hide political decisions as scientific, it will hopefully be a major step forward.


* U.S. Environmental Protection Agency, National Health and Environmental Effects Research Laboratory, Research Triangle Park, North Carolina. This Article was part of an invited presentation at Law, Science, and the Environment Forum: A Meeting of the Minds, at Lewis and Clark Law School, April 19-20, 2007. The views expressed in this Article are solely those of the author, and do not necessarily reflect the views of the Environmental Protection Agency.

[1] EPA, Task Force Report on Improving EPA Regulations 1 (2001) [hereinafter EPA, Task Force Report].

[2] Office of the Sci. Advisor, EPA, Risk Assessment Principles and Practices, EPA/100/B-04/001, at 3 (2004) [hereinafter EPA, Risk Assessment Principles and Practices].

[3] EPA, Task Force Report,supra note 1, at 2.

[4] Holly Doremus, Using Science in a Political World: The Importance of Transparency in Natural Resource Regulation, in Rescuing Science from Politics 143, 143 (Wendy Wagner & Rena Steinzor eds., 2006).

[5] Wendy Wagner & Rena Steinzor, Conclusion, in Rescuing Science from Politics,supra note 4, at 295.

[6] EPA, Risk Assessment Principles and Practices,supra note 2, at 4-9.

[7]Id. at 3.

[8]See, e.g., A. Hawthorne Criddle, Evaluation of Risks,in Risk Management 19, 19 (H. Wayne Snider ed., 1964).

[9]See Stanley Kaplan & B. John Garrick, On the Quantitative Definition of Risk, in 1 Risk Analysis 11, 12 (1981) (“The notion of risk . . . involves both uncertainty and some kind of loss or damage that might be received.”).

[10]Id. at 13.

[11]See id. at 13-14 (explaining how the likelihood and measure of damages of a list of risks, ordered by increasing severity of damages, may be used to derive first a discrete “staircase function” risk curve, then the continuous “smoothed” risk curve that the discrete curve is presumed to approximate; and with respect to such curves, asserting that “[i]t takes a whole family of curves to fully communicate the idea of risk.”).

[12]See id. at 14-16 (imagining a criticism that “[a] risk analysis is essentially a listing of scenarios. In reality, the list is infinite . . . . I’m not worried about the scenarios you have identified, but about those you haven’t thought of,” then attempting to account for that criticism through the derivation of a risk curve).

[13] Donald Rumsfeld, Dep’t of Def., News Briefing, News Transcript, Feb. 12, 2002, available at http://www.defenselink.mil/transcripts/transcript.aspx?transcriptid=2636 (emphasis added).

[14] EPA, Guidelines for Ecological Risk Assessment 1 (1998), available at http://oaspub.epa.gov/eims/eimscomm.getfile?p_download_id=36512.

[15]Id. at 87-88.

[16]See, e.g., Notice of availability of final Guidelines for Ecological Risk, 63 Fed. Reg. 26,846, 26,846 (May 14, 1998).

[17]Id. at 26,854.

[18] Alvin M. Weinberg, Reflections on Risk Assessment, in 1 Risk Analysis 5, 5 (1981).

[19] Robert H. Cumming, Is Risk Assessment a Science?, in 1 Risk Analysis 1, 1 (1981).

[20]Id. at 1-2.

[21] Nat’l Research Council, Risk Assessment in the Federal Government: Managing the Process 63 (1983).

[22]See Nat’l Research Council, Science and Judgment in Risk Assessment 174-86 (1994) [hereinafter Nat’l Research Council, Science and Judgment] (describing both the concerns and the value the scientific community sees in EPA’s use of defaults).

[23] Nat’l Ctr. For Envt’l Assessment, EPA, Human Health Guidelines, http://cfpub.epa.gov/
ncea/cfm/nceaguid_human.cfm (last visited Nov. 18, 2007); Nat’l Ctr. For Envt’l Assessment, EPA, Ecological Guidelines, http://cfpub.epa.gov/ncea/cfm/nceaguid_ecological.cfm (last visited Nov. 18, 2007).

[24] EPA, Guidelines For Carcinogen Risk Assessment, EPA/630/R-00/004 (1986), available at http://cfpub.epa.gov/ncea/cfm/nceaguid_human.cfm.

[25] EPA, Guidelines for Carcinogen risk assessment, EPA/630/P-03/001F (2005), available at http://oaspub.epa.gov/eims/omm.getfile?p-download_id=434774 (follow hyperlink “Guidelines for Carcinogen Risk Assessment 2005”).

[26] Dale Hattis & Elizabeth L. Anderson, What Should Be the Implications of Uncertainty, Variability, and Inherent ”Biases”/”Conservatism” for Risk Management Decision-Making?, 19 Risk Analysis 95, 97 (1999).

[27] Nat’l Research Council, Science and Judgment,supra note 22, at 84.

[28]Id.

[29] Carl F. Cranor, Toxic Torts: Science, Law and the Possibility of Justice 142 (2006).

[30]See EPA, Risk Assessment Principles and Practices,supra note 2, at 12-13.

[31]Id. at 11.

[32]Cf. U.S. Office of Mgmt. & Budget, Proposed Risk Assessment Bulletin 14 (2006), available at http://www.whitehouse.gov/omb/inforeg/proposed_risk_assessment_bulletin_
010906.pdf. After comment and peer review, the Office of Management and Budget (OMB) did not adopt a final version of the Risk Assessment Bulletin. Instead a memorandum was released highlighting some of the general principles of the Proposed Risk Assessment. Memorandum from the Office of Mgmt. & Budget and Office of Sci. & Tech. Policy to the Heads of Executive Depatrments and Agencies 2 (Sept. 19, 2007), available at http://www.whitehouse.gov/
omb/memoranda/fy2007/m07-24.pdf.

[33]See supra note 22 and accompanying text.

[34]Id.

[35] Nat’l Research Council, Science and Judgment,supra note 22, at 7.

[36]Id.

[37] EPA, Risk Assessment Principles and Practices,supra note 2, at 11.

[38]See, e.g., Adam M. Finkel, Is Risk Assessment Really Too Conservative?: Revising the Revisionists, 14 Colum. J. Envtl. L. 427, 428 (1989) (noting that “[t]he denunciation of QRA [Quantitative Risk Assessment] as an ideologically-motivated exercise in exaggeration has cropped up from time to time, beginning even before United States federal agencies started to codify their QRA procedures”).

[39] EPA, Risk Assessment Principles and Practices,supra note 2, at 17.

[40]Id.

[41] M.D. Rogers, The Precautionary Principle, a View from Europe, Risk Policy Rep., at 41, 42 (2000).

[42]See, e.g., Adam M. Finkel, Is Risk Assessment Really Too Conservative?: Revising the Revisionists, 14 Colum. J. Envtl. L. 427, 432 (1989).

[43] Nat’l Research Council, Science and Judgment,supra note 22, at 602.

[44] EPA, Risk Assessment Principles and Practices,supra note 2, at 22-23.

[45]Id. at 14.

[46]Id. at 15-16.

[47] Clean Air Act, 42 U.S.C. § 7409(b)(1) (2000).

[48]Id. § 7412(f)(2) (2000).

[49]Id. § 7409(b)(2) (2000).

[50]Id. § 7521(a)(4)(A) (2000).

[51]Id. § 7651 note (2000) (Acid Deposition Standards).

[52]Id. § 7671k(a) (2000).

[53]Id. §7401 note (2000) (Radon Gas and Indoor Air Quality Research).

[54]See Cumming,supra note 19, at 1 (noting that, unlike traditional science, risk assessment “must deal with questions as they arise without regard to . . . the quality and completeness of data that are obtainable or at hand”).

[55]Id.

[56] George E.P. Box, Robustness in the Strategy of Scientific ModelBuilding,in Robustness in Statistics 201, 202 (R.L. Launer & G.N. Wilkinson eds., 1979).

[57]E.g., Chevron U.S.A. Inc. v. Natural Res. Def. Council, Inc., 427 U.S. 837, 844-45 (1983) (describing the “well settled principle” of deference to agency interpretations when the meaning or reach of a statute requires reconciling conflicting policies).

[58]See Office of Envtl. Info., EPA, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency 3-4 (2002).

[59]Id. at 15-16.

[60] EPA, Task Force Report,supra note 1, at 1.

[61]Id.

[62]Id. at 3.

[63]Id. at 3, 7.

[64] Office of Policy, Economics, and Innovation, EPA, EPA’s Action Development Process: Guidance for EPA Staff on Developing Quality Actions (2004) [hereinafter EPA’s Action Development Process].

[65]Id. at 6-7.

[66]Id.at 67-69.

[67]Id. at 7.

[68]Id.

[69]Id. The purpose of early guidance from senior management is to establish priorities and expectations early and to identify major policy, economic, and scientific issues. Id. at 26-27. The purpose of the analytic blueprint is to identify information and scientific analyses that will help inform the decision, to develop resources and a schedule for completing work, to address statutory and executive orders, and to serve as an up-front agreement on a technical approach. Id. at 28.

[70] Science Policy Council, EPA, A Summary of General Assessment Factors For Evaluating the Quality of Scientific and Technical Information, EPA100/B-03/001 (2003).

[71]Id. at 2.

[72]Id.

[73]Id.

[74]Id. at 4.

[75]Id.

[76]Id.

[77] Science Policy Council, EPA, Peer Review Handbook, EPA100/B-06/002 (3rd ed. 2006).

[78]Id. at 11.

[79]Id.

[80]Id.

[81]Id.at 12.

[82]Id. at 30, 44.

[83]See Office of Envtl. Info., EPA,supra note 58, at 22.

[84]See id. at 49.

[85]See id. at 24.

[86]See generally, U.S. Office of Mgmt. & Budget, Circular A-4 (2003), available at http://www.whitehouse.gov/omb/circulars/a004/a-4.pdf (outlining key elements of a regulatory analysis, including analytical approaches such as detailed benefit-cost analyses to help maximize risk reduction for a given level of resource expenditure).

[87]See generally EPA’s Action Development Process,supra note 64, at 67-69.

[88]See Exec. Order No. 12866, 50 Fed. Reg. 51735 (Sept. 30, 1993).

[89]See Gen. Accounting Office, Rulemaking OMB’s Role in Reviews of Agencies’ Draft Rules and the Transparency of Those Reviews, GAO-03-929, at 3 (2003), available at http://www.gao.gov/new.items/d03929.pdf.

[90]See generally Office of Mgmt. & Budget, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies (2001), available at http://www.whitehouse.gov/omb/fedreg/final_information_
quality_guidelines.html (stating OMB and agency responsibilities for maintaining the quality and integrity of information distributed by federal agencies).

[91]See generally Office of Mgmt. & Budget, Executive Office of the President, Final Information Quality Bulletin for Peer Review (2004), available at http://www.whitehouse.gov/omb/memoranda/fy2005/m05-03.pdf (stating that all important scientific information shall be peer reviewed before dissemination by the federal government, and establishing general peer review criteria).

[92]See generally Memorandum from Joshua B. Bolten, Director, Office of Mgmt. & Budget 2 (Dec. 16, 2004) available at http://www.whitehouse.gov/omb/memoranda/fy2007/m07-07.pdf (stating the OMB has responsibility to oversee and coordinate regulatory policy for the Administration, and setting forth policies for issuing guidance documents that generally further the goal of increasing transparency in agency guidance practices).

[93] Douglas L. Weed, Weight of Evidence: A Review of Concept and Methods, 25 Risk Analysis 1545, 1546 (2005).

[94]Id. at 1549.

[95] Douglas Crawford-Brown, The Concept of ‘Sound Science’ in Risk Management Decisions, 7 Risk Management 7, 14 (2005).

[96] Weed,supra note 93, at 1552.

[97] Crawford-Brown,supra note 95, at 9.

[98]See supra note 57 and accompanying text.

 

Print this pageEmail this to someoneTweet about this on TwitterShare on Facebook

Comments are closed

Sorry, but you cannot leave a comment for this post.