Overview

Eliminating alternatives, or refutation, is a powerful approach to evaluating information. The ability to refute all but one alternative is a strong standard of proof for causality, and it is easily understood and widely practiced. It is the basic technique of literature's most famous master of inference, Sherlock Holmes.

When you have eliminated the impossible, whatever remains, however improbable, must be the truth. - Sir Arthur Conan Doyle, Sign of Four (1890).

Refutation is also an effective way of reducing the number of alternatives to be considered before seeking additional evidence.

It is a particularly good option for Stressor Identification (SI) when the set of alternatives is limited, and when disproof does not rely on statistics (see CADDIS Volume 4 page on Intrepreting Statistics).

Specifically, if the SI is conducted to support a permitting action, logical elimination of the permitted source as a potential cause of the observed injury is a sufficient causal analysis. Because of the complexity associated with ecological systems and multiple stressors, many of you will not have the evidence necessary to confidently eliminate causes. Your evaluations will rely on comparing the strength of evidence for the different candidates.

Refutation as a method for establishing causality has strong roots in the philosophy of science. Popper, Platt, and other conventional philosophers of science have argued that it is logically impossible to prove a hypothesized relationship, but it is possible to disprove hypotheses (e.g., Platt 1964, Popper 1968).

Thus, if a set of possible causes has been identified, once all but one alternative have been disproven and eliminated, the remaining hypothesis must be true. For example, if a body of water is found to be acidic, it is possible to establish the cause as atmospheric acid deposition by eliminating acid mine drainage, geologic sulphate, and biogenic acids as causes (Thornton et al. 1994).

Eliminating alternatives has three major limitations:

• It may not be possible to identify a complete set of candidates. Due to limited knowledge, it may not be possible to identify a complete set of candidates. Also, the array of possible causes is potentially infinite, as there is no clear boundary between plausible and absurd hypothetical causes (Susser 1986b, 1988).
• It is difficult to reliably generate unambiguous test results. The process of elimination is limited by the difficulty of performing reliable tests and obtaining unambiguous results in ecological studies. If all but one cause is rejected on uncertain grounds, it is difficult to accept the remaining candidate cause with confidence.
• One cause may mask another sufficient cause. Elimination of causes should be done with particular care when multiple sufficient causes may be operating. The evidence for one cause may be so strong that it masks the effects of another sufficient cause and appears to be the sole cause. In addition, the temporal sequence of cause and effect may appear to be wrong when one sufficient cause precedes another. For example, toxic chemicals in an industrial effluent may impair a biological community. If the stream is subsequently channelized, the effects of degrading riffle habitat would be obscured by the industrial effluent. The degraded riffles would have been sufficient to degrade biological communities within a pristine stream and therefore should be retained as a candidate cause. Similar issues are also relevant to spatial sequences such as those occurring in streams or rivers.

Most often the objective of SI is to identify all sufficient causes (for example, when the goal is to remediate or restore a waterbody). In these cases, you should perform the elimination step iteratively. That is, each cause eliminated during the first round should be reevaluated to determine whether its effects may have been masked by another cause. If so, the candidate cause should be retained.

In extreme cases, the masked secondary causes will remain unidentified, because the primary causes are so conspicuous. For example, if channelization has eliminated nearly all fish, it may not be apparent that episodic pesticide runoff would affect sensitive species. Such eclipsed secondary causes will become apparent only after the primary causes have been remediated.

References

• Platt JR (1964) Strong inference. Science 146:347-353.
• Popper KR (1968) The Logic of Scientific Discovery. Harper and Row, New York NY.
• Susser M (1986b) The logic of Sir Karl Popper and the practice of epidemiology. American Journal of Epidemiology 124:711-718.
• Susser M (1988) Falsification, verification and causal inference in epidemiology: reconsideration in light of Sir Karl Popper's philosophy. Pp. 33-58 in: Rothman KJ (Eds). Causal Inference. Epidemiology Resources Inc., Chestnut Hill MA.
• U.S.EPA (1994) Environmental Monitoring and Assessment Program (EMAP) Assessment Framework. U.S. Environmental Protection Agency, Research Triangle Park NC. EPA/620/R-94/016.

Top of Page