Evidentiary Value

Edited by Peter Gärdenfors and Bengt Hansson

Library of Theoria no 15, CWK Gleerup, Lund, 1983

Contents:

 

The Evidentiary Value Model: Too Short an Introduction

(this introduction was written 2004 for this site)

The evidentiary value model (EVM) was developed in the early 1970s in papers by the philosophers Martin Edman and Sören Halldén, who were themselves influenced by the work of the Swedish lawyer Per Olof Ekelöf.

See:

Martin Edman, “Adding independent pieces of evidence”, in Modality, Morality and Other Problems of Sense and Nonsense, edited by B. Hansson, Lund 1973, 180-8;
Sören Halldén, “Indiciemekanismer”, Tidskrift for Rettsvitenskap, 86 (1973), 55-64;
Per Olof Ekelöf, Rättegång IV, Stockholm 1963 (first edition)/ 1982 (fifth edition).

The theory is discussed in Evidentiary Value: Philosophical, Judicial and Psychological Aspects of a Theory, Library of Theoria, Lund 1983, edited by P. Gärdenfors, B. Hansson and N.-E. Sahlin.

The present text is based on my “How to be 100% certain 99.5% of the time”,Journal of Philosophy, 83 (1986), 91-111; and on my and Wlodek Rabinowicz’s “The Evidentiary Value Model”, in Handbook of Defeasible Reasoning and Uncertain Management Systems, Vol. 1, edited by D. M. Gabbay and Ph. Smets, Kluwer, Dordrecht 1997, 247-65. These papers contain proofs, discussion of epistemological questions (comparing EVM with reliabilist theories of knowledge) and comparisons with other theories (Bayesianism and Glen Sahfer’s theory). The second also provides an extensive list of primary and secondary references.

—

The central concept of EVM is that of evidentiary mechanisms, reliable processes of a kind, the links between hypothesis and evidence. The idea is that these mechanisms convey evidentiary value. Evidentiary value can therefore be assigned only by way of considering such mechanisms.

Problems of evidentiary evaluation have a common logical structure. First, anevidentiary theme (a hypothesis, denoted H) is the focus of our attention. This is the claim for which a case must be made. Second, pieces of evidence either confirm or disconfirm the evidentiary theme. A piece of evidence will be called an evidentiary fact (and denoted e).

In addition to these two concepts, shared by most theories of evidence, the evidentiary value model introduces the notion of evidentiary mechanisms(denoted either A[H/e], or, when there is no danger of confusion, simply A). The original idea was to interpret such a mechanism as a causal link between an evidentiary theme and an evidentiary fact. But there are other types of mechanism — for example, inferential processes. All forms of what might be called “sound reasoning” can and should be counted as reliable processes, or reliable mechanisms.

One way to think of the evidentiary mechanism is to interpret it as a link between an evidentiary theme and an evidentiary fact which, if present, can be said to “prove” the theme, given the evidentiary fact. Think of this mechanism (denoted M) as a triple consisting of an evidentiary theme, an evidentiary fact and an event such that, if we know that this event has occurred, and we have received the evidentiary fact, we have a proof of the hypothesis.

P(A/e) is the probability that the mechanism has worked given the evidence. The evidentiary value model now claims that, in evaluating the hypothesis, the value P(A/e) is more important than P(H/e). So rather than assessing the probability of the hypothesis given the evidence, we should assess the probability that the evidence proves the hypothesis. What is important is the probability that the mechanism has worked given the evidentiary fact, i.e. that the links between the evidentiary theme and the evidentiary fact have been present. (There may, of course, be several different mechanisms connecting e to H, but this possibility causes no insurmountable problems.)

If the sensory mechanisms of a witness have worked, a judge can expect a trustworthy report of what happened at the scene of the crime. But if these mechanisms did not work, the witness may or may not report a correct version of what actually took place. A supposed witness in a court case who does not have any relevant information may nevertheless correctly testify to the guilt of the defendant. Assume that a judge is in the business of trying to reach an opinion as to whether the evidence gives knowledge of the hypothesis under consideration, rather than merely trying to form a belief as to whether the hypothesis is true. He or she is then trying to ascertain, on the basis of his evidence, the probability of the existence of a reliable link between the hypothesis and the evidentiary fact — trying to ascertain howprobable it is that the evidentiary mechanism has worked, given the evidence at hand.

The evidence thus has value only insofar as we, given the evidence, believe the mechanism to be working (for example, in that the witness has registered what actually happened). Clearly, P(H/e) = P(A/e). For those states where the mechanism works, and e obtains, H holds. For those states where the mechanism does not work, however, we know that H may still hold. Or, the inequality holds because, whenever the mechanism is working, the hypothesis is true, given e. If there is a “proof” of the theme, we know that it is true; but for all we know, if such a proof cannot be found the theme may still be true. Thus, according to EVM, the true evidentiary value is the probability that the mechanism has worked given the evidence. The evidentiary mechanisms are the conveyers of evidentiary value. That is:

EV(H,e) = P(A[H,e]/e)

Here “[H,e]” tells us that A links H and e; but in most cases we do not need to be that precise.

Note that the argument above is based on the assumption that Ae (A and e) implies H, or, at least, that P(H/Ae) = 1. This is a simplifying assumption. We suppose that an evidentiary mechanism, when it works, is 100% reliable. (But it still may work only 99.5% of the time!) Excluded from consideration are “chancy” mechanisms which, even when they are working, do not always deliver right answers. For such mechanisms, P(Ale) could be higher than P(H/e). This may, for example, be the case if P(H/~Ae) is very low, P(A/e) is very high, and the probability P(H/Ae) is not exceedingly high.

—

The idea of an evidentiary mechanism can be made more precise. The first person to examine evidentiary mechanisms was Martin Edman (see the paper mentioned above).

An evidentiary mechanism may or may not work. If it works, the evidence and the fact that the mechanism worked, eA, prove the evidentiary fact, H. But if it does not work, ~A, e~A does not tell us much about H. We thus note that e = eA V e~A. Assume that eA ⇒ HA — an assumption stronger than that so far made. Then e = HA V e~A.

It can reasonably be assumed that the evidentiary theme does not influence the working of the mechanism. In probabilistic terms, one would say that Aand ~A are independent of H. This assumption, together with the one made above, implies that

P(A/e) = P(H)P(A)/(P(H)P(A) + P(e~A)).

We also note that evidentiary value will fall in the closed interval between 0 (when P(H) = 0) and P(A)/(P(A) + P(e~A)) (when P(H) = 1).

Edman proves a most useful theorem. Assume that F is an event such that P(eF) is not 0; and assume that HF and A, and F and e~A are independent; then

(*) P(A/eF) ≥(≤) P(A/e) if and only if P(H/F) ≥(≤) P(H), provided that P(Ale) < 1

This theorem provides an interpretation of the rules suggested by EVM.

As Edman emphasizes, it is important to note that F is assumed to be what he calls an external event, i.e. is not allowed to influence the working of the mechanism. If, for example, the mechanism does not work, F should make it neither more nor less likely that the mechanism yields a given value. Nor should F increase the reliability of a working mechanism.

—

In a complete theory of evidence, rules governing the combination of different evidentiary assessments are needed. If there are two pieces of concurring evidence, e and f, both of which point to the truth of an evidentiary theme, H, via two independent evidentiary mechanisms, A and B, the relevant measure is P(A or B/ef). Given the appropriate independence conditions it can be shown that

(S) P(A or B/ef) ≥ P(A/e) + P(B/f) – P(A/e)P(B/f)

The independence conditions assumed in the derivation of this rule are

P(A/e) = P(A/ef~B)

and

P(A/ef) ≥ P(A/e)

The first condition says that the evidentiary value of one piece of evidence is not altered if another evidentiary fact turns up that is spurious in the sense that its evidentiary mechanism malfunctions. If there are no links between the evidentiary theme and the evidentiary fact, combined values are uninteresting. The second condition says that the combination of two pieces of concurring evidence will not decrease the probability that the first mechanism is working.

In a case where there are two pieces of conflicting evidence pointing to contradictory hypotheses H and G, we have:

(M) P(A/ef) = (P(A/e) – P(A/e)P(B/f))/(1 – P(A/e)P(B/f)).

The independence conditions needed are

P(A/e) = P(A/ef~B),

and

P(A/efB) = 0

If the mechanism for one piece of evidence has not worked, the evidentiary value of the other piece is not changed, just as in the concurring case. And, if we know that one of the mechanisms has worked, then the probability that the other mechanism has also worked is zero, because H and G conflict, P(H/eA) = P(G/fB) = 1.

—

In the S-rule there is an inequality sign, but in the M-rule we have an equality. Why this asymmetry?

First, the S-rule: What we want to do is to show that the crucial assumption, P(A/ef) ≥ P(A/e), is reasonable given our understanding of what an evidentiary mechanism is.

We have two independent evidentiary mechanisms yielding concurring pieces of evidence. We know that both P(H/e) and P(H/f) are greater than or equal to P(H). Edman’s theorem (*) tells us that P(A/ef) is greater than or equal to P(A/e) — again, so long as f is an external event with respect to the first mechanism (P(A/f) = P(A)).

According to Edman, the intuitive explanation of this result is that the evidentiary fact f raises the probability of the theme, which results in an increased evidentiary value (P(A/ef)), which in turn explains the S-rule’s inequality sign. Given that we have the inequality sign in the independence condition P(A/ef) ≥ P(A/e), the inequality reappears in the S-rule itself.

EVM looks at independent evidentiary mechanisms that yield concurring pieces of evidence, not at independent and concurring pieces of evidence. If we assume that the evidentiary facts are independent, that e and f are independent, things are very different. But the two pieces of evidence cannot be independent when they support, or are connected to, the same evidentiary theme.

Second, the M-rule: With two conflicting evidentiary mechanisms, we are instead focusing on the probability that one of the mechanisms has worked given the combined evidence — that is, on P(A/ef).
P(A/ef) = P(A/efB)P(B/ef) + P(A/ef~B)(1-P(B/ef). Under the assumption that
P(A/efB) = 0, P(A/ef) = P(A/ef ~ B)(1 – P(B/ef). By the independence assumption, P(A/ef) = P(A/e)(1 – P(B/ef)) and P(B/ef) = P(B/e )(1- P(A/ef)). And it directly follows that (M).

—

The Evidentiary Value Model has some very attractive properties:

(i) Evidentiary value, as defined by EVM, is (potentially) one-sided. According to EVM it may happen that EV(~H, e) = 0, even though EV(H, e) < 1. To calculate that EV(~H, e) it is not enough to know that EV(H, e). In this respect, evidentiary values behave like Bernoullian pure probabilities.

In Ars conjectandi, Jacques Bernoulli makes a distinction between two types of evidence: pure and mixed. Mixed evidence supports both the hypothesis H and its negation ~H. When we assess the strength of evidence in terms of probability, this means that a certain part of the total probability indicates Hwhile another part indicates ~H. Since mixed evidence points both to H and~H, the resulting probabilities sum to 1. (Frequential information is a good example of mixed evidence.) We might know that n% of people in a given population have property B and that the remaining 100-minus-n% of people do not have it.
Pure evidence, on the other hand, can support H, support ~H or be entirely neutral. In this case we have sub-additivity (over H and ~H), and the resulting probabilities cannot be understood as probabilities in the usual way — for example, as Bayesian probabilities. The inequality arises from that portion of the evidence which is neutral. The Bernoullian probabilities are not really inconsistent with our traditional concept of probability, since in the case of pure evidence that only supports H, we may identify the Bernoullian probability for the hypothesis given the evidence with P(H/e) = EV(H, e) = P(A[H,e]/e) and the Bernoullian probability for the negation of the hypothesis given the evidence with EV(~H,e) = P(A[~H,e]/e) = 0.

One-sidedness is thus an old idea. It dates back at least to Bernoulli, and it was present in the concept of evidentiary value first introduced in the works of Ekelöf, Edman and Halldén.

(ii) Shafer’s theory, the traditional and Bayesian theory, and EVM make different independence assumptions, and it can be argued that the assumptions made in the first two of these theories are too strong. EVM looks at independent evidentiary mechanisms that yield concurring pieces of evidence, not at independent and concurring pieces of evidence. For EVM it is important that the “internal” mechanisms of the measuring instruments are stochastically independent, nothing stronger than that. The two pairs of probability P(A/ef), P(B/ef) and P(A/e), P(B/f) should not be identified or confused. It is the former pair that are appropriate probabilities for computing combined values.

(iii) Frank Ramsey tells us that a belief is knowledge if it is true, certain and obtained by a reliable process — that is, if it obtained by a reliable process and always lead to success. Ramsey’s theory has been much discussed and has developed in different directions. An important question is what we mean by “a reliable process”. EVM helps us define, understand and develop the concept, strengthening externalist theories of knowledge. At the same time, EVM obtains support from the success of theories of knowledge of that type.

(iv) It is obvious that EVM manages risk better than some of its competitors. We know that P(H/e) ≥ EV(H,e) = P(A[H,e]/e). Focusing on the causal link between the evidentiary theme and the evidentiary fact guarantees that any evidentiary value will not be too high. More important, this focus prevents us from counting immaterial evidence in favour of the hypothesis. In a court of law nothing could be more important.

(v) Note that the choice of theory of evidence is not unproblematic. When forensic evidence is reported it is often supplemented by a (written) report of trustworthiness. These reports deploy the categories “conclusive evidence that…”; “strong support for a view that…”; “support for a view that…”; “moderate support for a view that…”, and so on. They are meant to reflect (Bayesian) probabilities of the form P(H/e). But EVM tells us that what is important from the judge’s viewpoint is the probability that the forensic instruments, the forensic mechanisms, have worked. It is thus obvious that this type of (graded) scale does not reflect true evidentiary value, i.e. EV(H,e), and that it can lead to an assessment in which the probability that the hypothesis is true is too high.

An alternative scalar approach would be to ask whether “It is conclusive that the hypothesis caused the evidence” (or “it is beyond question that the evidentiary mechanism has worked”); or whether “there is strong support that the hypothesis caused the evidence” ( or “that the evidentiary mechanism has worked is strongly supported”), and so on. This is grade scale focuses on the evidentiary value, on the working of the underlying evidentiary mechanism.

We know that the words we use to present information frame our probability judgements. Grade scales have to be designed with great care. For example, they should be constructed in such a way that they do not subvert the intentions of the legal system or that there is a risk of unwarranted support for guilty verdicts for defendants.

Finally, and while we are on the subject, the Swedish system of law ought to (but currently does not) use the latter type of “evidentiary” scale. The Swedish system is greatly influenced by the work and thought of Ekelöf. It is not a traditional common law system, and evidentiary values are an inseparable part of it. A Bayesian grade report does not accord with its underlying objectives, and thus for several reasons results in bad risk management. But within the framework of other types of legal system as well, it is far better to give evidentiary value reports than “pure” probability estimates (bearing in mind that not all probabilistic and frequentistic information has evidentiary values or can be reported as such).

Copyright © 2004 by
Nils-Eric Sahlin
University of Lund
nils-eric.sahlin@telia.se