Commentary: Likelihood Ratio as Weight of Forensic Evidence: A Closer Look

A commentary on
Likelihood Ratio as Weight of Forensic Evidence: A Closer Look

by Lund, S. P., and Iyer, H. (2017). J. Res. Natl. Inst. Stand. Technol. 122:27. doi: 10.6028/jres.122.027

A recent article (Lund and Iyer, 2017) provides, in the words of its title, a closer look at the likelihood ratio as the weight of forensic evidence. This note comments critically on two aspects of the article.

The first aspect concerns two related statements. In the abstract the statement is made that “[W]e find the likelihood ratio paradigm to be unsupported by arguments of Bayesian decision theory, which applies only to personal decision making and not to the transport of information from an expert to a separate decision maker.” The idea presented in this statement of lack of support for the likelihood ratio as a means of transport of information is repeated in the conclusion where it is stated that “… we hope the forensic science community comes to view the LR as one possible, not normative or necessarily optimum, tool for communicating to DMs (decision makers)” (Lund and Iyer's emphasis). Despite this opinion of these authors, it was shown many years ago by I.J.Good in two brief notes in the Journal of Statistical Computation and Simulation (Good, 1989a,b) repeated in Good (1991) and in Aitken and Taroni (2004) that, with some very reasonable assumptions, the assessment of uncertainty inherent in the evaluation of evidence leads inevitably to the likelihood ratio as the only way in which this can be done.

In order to show that the likelihood ratio is the only way to evaluate evidence, it is necessary to introduce some mathematical notation. This is a device to ease the presentation of the argument. The argument could be made verbally but would be lengthy and more difficult to follow. Consider evidence E which it is desired to evaluate in the context of two mutually exclusive propositions H_p and H_d. Denote the value of the evidence by V. Of course, this statement makes the implicit assumption that evidence has a value that can be measured. The value will depend on background information I. Four and only four factors have been introduced, E, H_p, H_d and I. Thus, V is a function of these four factors, V = f(E, H_p, H_d, I). There is uncertainty about E, so it should be analyzed probabilistically. Use of the argument of conditional probability leads to f(E ∣ H_p, H_d, I)f(H_p, H_d, I), rather than forms such as f(H_p ∣ H_d, E, I) or variants of it. The expression f(H_p, H_d, I) does not involve the evidence, which reduces considerations further to f(E ∣ H_p, H_d, I). Propositions H_p and H_d are mutually exclusive so if E is to be a function of both H_p and H_d then f(E ∣ H_p, H_d, I) is a combination of two functions, one that involves H_p and not H_d and one that involves H_d and not H_p. Value may thus be expressed as a function of the probabilities of E given H_p (and I) and of E given H_d (and I). Again, this makes implicit assumptions, namely that there is a probability that can be associated with evidence and that is dependent on a proposition and background information. For ease of notation explicit mention of I will be omitted from notation in what follows.

Let x = Pr(E ∣ H_p) and y = Pr(E ∣ H_d). The assumption that V is a function only of these probabilities can be represented mathematically as

$V=f(x,y)$

for some function f.

Now, consider another piece of evidence T which is irrelevant to E, to H_p and to H_d. Irrelevance is taken in the probabilistic context to be equivalent to independence so that T may be taken to be independent of E, of H_p and of H_d. It is then permissible for Pr(T) to be given notation which does not refer to any of E, H_p or H_d. Thus, let Pr(T) be denoted by θ. Then

$\begin{array}{l}\mathrm{Pr}(E,T\mid H_p)\\ =\mathrm{Pr}(E\mid H_p)\mathrm{Pr}(T\mid H_p)\text{ by the independence of }E\text{ and }T\\ =\mathrm{Pr}(E\mid H_p)\mathrm{Pr}(T)\text{ by the independence of }T\text{ and }{H}_p\\ =x\theta .\end{array}$

Similarly,

$Pr(E,T\mid H_d)=y\theta .$

The value of (E, T) is f(θx, θy) by the definition of f. However, evidence T is irrelevant and has no effect on the value of evidence E. Thus, the value of the combined evidence (E, T), f(θx, θy), is equal to the value V of E, f(x, y), and

$\begin{array}{l}V=f(x,y)=f(\theta x,\theta y)\end{array}$

for all θ in the interval [0,1] of possible values of Pr(T).

The only class of functions of (x, y) for which this can be said to be the case is the class which are functions of x/y or

$Pr(E\mid H_p)/Pr(E\mid H_d)$

which is the likelihood ratio. Hence the value V of evidence has to be a function of the likelihood ratio. Lund and Iyer wish the forensic community to view the likelihood ratio as one possible tool for communication with decision makers. We hope that we have shown here through the argument of Good that it is the only logically admissible form of evaluation. Incidentally, note that no recourse has been made to arguments of Bayesian decision theory. The support of these arguments for the likelihood ratio paradigm, as suggested in the abstract, is not necessary.

The second aspect is minor and concerns a definition. The concept of weight of evidence is an old idea. The term weight of evidence for the logarithm of the likelihood ratio was given by Charles Sanders Peirce (Peirce, 1878). It is not the likelihood ratio that should be referred to as the weight of evidence as is done in the title of the article. It is better to refer to the likelihood ratio as the value of the evidence and its logarithm as the weight of the evidence. The logarithm of the likelihood ratio has the pleasingly intuitive operation of additivity when converting the logarithm of the prior odds in favor of a proposition to the logarithm of the posterior odds in favor of the proposition.

$\begin{array}{rc}log\left\{\frac{Pr(H_p\mid E)}{Pr(H_d\mid E)}\right\}=log\left\{\frac{Pr(E\mid H_p)}{Pr(E\mid H_d)}\right\}+log\left\{\frac{Pr(H_p)}{Pr(H_d)}\right\}.& (1)\end{array}$

When considering the scales of justice it is the logarithm of the probabilities of the evidence given each of the two competing propositions that should be put in the scales, not the probabilities. Equation (1) can be rewritten as

$\begin{array}{l}log\left\{Pr(H_p\mid E)\right\}-log\left\{Pr(H_d\mid E)\right\}=\\ log\left\{Pr(E\mid H_p)\right\}-log\left\{Pr(E\mid H_d)\right\}+log\left\{Pr(H_p)\right\}-log\left\{Pr(H_d)\right\}\\ =\left[log\left\{Pr(E\mid H_p)\right\}+log\left\{Pr(H_p)\right\}\right]-\left[log\left\{Pr(E\mid H_d)\right\}+log\left\{Pr(H_d)\right\}\right]\end{array}$

Expressions to the left of the negative sign in the last line are associated with one pan in the scales, expressions to the right with the other pan. Thus log(Pr(E ∣ H_p)) is added to the prior log probability for H_p in one scale and log(Pr(E ∣ H_d)) is added to the prior log probability for H_d in the other scale. The difference in the sums of the two pairs of log probabilities is a more intuitive characteristic of the evidence to which the term weight may be applied than the ratio of the probabilities of the evidence given the respective propositions.

Author Contributions

CA drafted this commentary which results from equal and direct intellectual contributions of all listed authors.

Funding

The authors gratefully acknowledge the support of Leverhulme Trust through the Emeritus Award EM-2016-027 (CA), the Swiss National Science Foundation through grant No. BSSGI0_155809 and the University of Lausanne (AB).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Aitken, C. G. G., and Taroni, F. (2004). Statistics and the Evaluation of Evidence for Forensic Scientists, 2nd Edn. Chichester: John Wiley and Sons Ltd.

Google Scholar

Good, I. J. (1989a). C312: yet another argument for the explication of weight of evidence. J. Stat. Comput. Simul. 31, 58–59. doi: 10.1080/00949658908811115

CrossRef Full Text | Google Scholar

Good, I. J. (1989b). C319: weight of evidence and a compelling metaprinciple. J. Stat. Comput. Simul. 31, 121–123. doi: 10.1080/00949658908811131

CrossRef Full Text | Google Scholar

Good, I. J. (1991). “Weight of evidence and the Bayesian likelihood ratio” in The Use of Statistics in Forensic Science, eds C. G. G. Aitken and D. A. Stoney (Chichester: Ellis Horwood), 85–106.

Google Scholar

Lund, S. P., and Iyer, H. (2017). Likelihood ratio as weight of forensic evi-dence: a closer look. J. Res. Natl. Inst. Stand. Technol. 122:27. doi: 10.6028/jres.122.027

CrossRef Full Text | Google Scholar

Peirce, C. S. (1878). “The probability of induction,” in The World of Mathematics, 1956, Vol. 2, ed J. R. Newman (New York, NY: Simon Schuster), 1341–1354.