Since the concept of positive predictive value (PPV) and negative predictive value (NPV) seems to frustrate some vivisection activists, I thought this article might be helpful for them. An Italian research team published a study in the December issue of the journal Hepatology reporting that a scoring system for Wilson’s disease has a 93% positive and 92% negative predictive value in children. Equally important, the study revealed that the penicillamine challenge test (PCT) was not accurate. The researchers studied 40 children ages 1-21 years. The scoring system was evaluated against molecular analysis and copper content tests. An early diagnosis of Wilson’s disease is vital if the children are to avoid liver and neurological problems. This is how science is done when the results actually matter.
The use of PPV and NPV in order to decide if a test or treatment or system is worth using in medical science or in other situations is an everyday phenomenon. Without them, physicians and scientists would still be using treatments or tests that were actually causing more harm than good. Evaluations like PPV and NPV must be used in place of intuition or bias. Physicians and scientists alike have biases and even prejudices. (Bias being an expected result when not enough evidence is present to actually justify the expectation, while prejudice is expecting a specific result despite evidence to the contrary. An example of bias would be the fact that I like treatment A over treatment B despite the fact that no studies have been done comparing the two. An example of prejudice would be expecting animal models to predict human response to drugs and disease. There is no way around bias in science but prejudice usually exists in association with money or ego.)
Science is not an intuitive process. Science does not follow common sense or intuition. Intuitively we would stand on the surface of the earth and say the sun rotates around us. Intuitively, we would think that if dogs have a liver and if humans have a liver then we should be able to study one and learn about drug response in the other. Both intuitions are false.
Albert Einstein said common sense is the collection of prejudices acquired by age eighteen. Common sense is great if the question revolves around what our evolutionary ancestors encountered in their daily lives. However, it fails at most everything else. Common sense tells us the earth if flat, that the sun circles the earth, and that if a drug was safe in monkeys it should be safe for humans. Brian Greene wrote in The Fabric of the Cosmos:
The overarching lesson that has emerged from scientific inquiry over the last century is that human experience is often a misleading guide to the true natureof reality . . . Modern science has spearheaded one assault after another on evidence gathered from our rudimentary perceptions, showing that they often yield a clouded conception of the world we inhabit. (Greene 2004)
Along the same lines, Levitt and Dubner wrote in Freakonomics:
It was John Kenneth Galbraith, the hyperliterate economic sage, who coined the phrase "conventional wisdom." He did not consider it a compliment. "We associate truth with convenience," he wrote, "with what most closely accords with self-interest and personal well-being or promises best to avoid awkward effort or unwelcome dislocation of life. We also find highly acceptable what contribute most to self-esteem." Economic and social behaviors, Galbraith continued, "are complex, and to comprehend their character is mentally tiring. Therefore we adhere, as though to a raft, to those ideas which represent our understanding." (Levitt and Dubner 2006)
Science takes us beyond intuition, conventional wisdom, and common sense and it should take us beyond bias and prejudice. In honest, truth-seeking people, it does.
Everyone needs to remember the above in part because just being smart and well educated is not enough. Those things cannot replace science and the scientific method. For example, history is replete with examples of very smart people making mistakes. Louis Agassiz denied Darwin’s theory of evolution and said in 1867: “I trust to outlive this mania [(Pigliucci 2002) p13].” Many smart people even today doubt evolution. The great geologist Charles Lyell also denied evolution thinking instead that there were many centers of creation where new species appeared as needed [Ibid.]. Lord Kelvin thought the sun had not been around long enough for evolution to be the modus operandi of the forms we have today [(Pigliucci 2002) p21].
Lord Kelvin (truly one of the brightest people ever to live) also thought that the study of physics had almost yielded essentially all truths around 1900. He said at the end of the 19th century that there were only two dark clouds obscuring the horizon of the Newtonian landscape and they would soon evaporate and all of physics would be known. Pure hubris! One cloud was the problem of black body radiation and the other the Michelson-Morley experiment. The problems concern the mechanical explanations of heat and light. The question why do hot things give off the colors they do led to the field of quantum mechanics. The answer to the Michelson-Morley experiments included the field of relativity.
Descartes thought animals were not sentient. Astronomer Simon Newcomb published a paper explaining why airplanes would never fly. His analysis was perfect except for the lift effect of airfoil. He published his analysis two months before the Wright brothers flew. When Louis Leakey stated he wanted to go to Africa to search for “early man” everyone told him he was wasting his time as it was already known that early man would be found in Asia [(Milner 2009) p96] (Leakey and Lewin 1977). Baron Georges Cuvier did not believe in evolution stating: “Fossil man does not exist.” [(Milner 2009) p101]
For approximately two decades [Hugo] de Vries's theory eclipsed Darwin's, and during this period most biologists were confident that de Vries had laid Darwin to rest. Erik Nordenskiöld's History of Biology, written in the early 1920s, claimed that "Darwin's theory has long been rejected in its most vital points." And as late as 1932, Clarence Ayres, in his biography of Thomas H. Huxley, said: "All of Darwin's 'particular views' have gone down wind: variation, survival of the fittest, natural selection, sexual selection, and all the rest. Darwin is very nearly, if not quite, as outmoded as Lamarck." [(Milner 2009) P134]
One of the most famous phrases in biology, the “Central dogma of DNA” was a mistake. Watson, the co-discoverer of the double helix and inventor of the phrase thought dogma meant hypothesis. [(Milner 2009) p 74]. French anatomist Georges Cuvier stated: “Fossil man does not exist” [(Milner 2009) p 316]. Alfred Russel Wallace wrote in his book The Wonderful Century (1898):
In 1840 a good geologist confirmed these discoveries [that man-made stone tools had been found in sealed layers with the bones of extinct elephants and other Ice Age mammals], and sent an account of them to the Geological Society of London, but the paper was rejected as being too improbable for publication! All these discoveries were laughed at or explained away. . . . These, combined with numerous other cases of the denial of facts on a priori grounds, have led me to the conclusion that, whenever the scientific men of any age disbelieve other men's careful observations without inquiry, the scientific men are always wrong.[(Milner 2009) p 345.]
Sadly, Alfred Russel Wallace, co-discover with Darwin of evolution by natural selection, opposed smallpox vaccines. [(Milner 2009) p 348.]
So we need science and objective scales in order to separate truth from fiction. PPV and NPV allow us to do this for many things including answering the question as to whether animal models are predictive.
Litchfield (Litchfield 1962) studied rats, dogs, and humans in order to evaluate responses to 6 drugs. Only side effects that could be studied in animals were calculated. The results are as follows:
39 physical signs in 3 species (234 possible responses)
Toxic effects found in man53
Toxic effects found in man only 23
Toxic effects also found in man18
Toxic effects not found in man 19
Toxic effects also found in man 29
Toxic effects not found in man 24
19 false positives
35 false negatives
Sn = 18/(18+35) = 34%
PPV = 18/(18+19) = 49%
24 false positives
24 false negatives
Sn = 29/(29+24) = 55%
PPV = 29/(29+24) = 55%.
Suter reviewed the effects of 6 drugs in humans and animals. Animals and humans shared 22 side effects (true positives). Animals incorrectly identified 48 side effects that did not in fact occur in humans (false positives). The animals incorrectly missed 20 side effects that did occur in humans (false negatives). This results in a sensitivity (Sn) = 22/(22+20) =52% and a PPV = 22/(22+48) = 31%. (Suter 1990)
Two reviews addressed those drug cases where the clinical toxicity was so severe as to lead to withdrawal from marketing in the approximate period 1960-1990. In 4 of 24 cases there was animal data that correlated with the human and in the other report, only 6 of 114 clinical toxicities had animal correlates. (Heywood 1990) (Spriet-Pourra, Auriche, and (Eds) 1994) The PPV for these two are obviously very low. (See Animal Models in Light of Evolutionfor more.)
Science is important! Paul Thagard said:
. . . society faces the twin problems of lack of public concern with the important advancement of science, and the lack of public concern with the important ethical issues now arising in science and technology, for example we around the topic of genetic engineering. One reason for the dual lack of concern is the wide popularity of pseudoscience and the occult among the general public. Elucidation of how science differs from pseudoscience is the philosophical side of an attempt to overcome public neglect of genuine science. (Thagard 1998)
(I would add that one reason the public is dissuaded from scientific arguments is the fact that science is more difficult to comprehend that watching sports, sit-coms, and reality-based shows on TV. Our educational system has failed to educate many and hence they are not prepared to use science or think critically or to appreciate the importance of thinking critically and using science.)
Alas, another reason science is important is that scientists themselves ignore genuine science when their ego or livelihood is challenged by the results from science. Results like those that show animal models cannot predict human response to drugs and disease.
Greene, Brian. 2004. The Fabric of the Cosmos: Alfred Knoph.
Heywood, R. 1990. Clinical Toxicity--Could it have been predicted? Post-marketing experience. In Animal Toxicity Studies: Their Relevance for Man, edited by CE Lumley and S. Walker. Lancaster: Quay.
Leakey, Richard, and Roger Lewin. 1977. Origins: EP Dutton.
Levitt, SD, and SJ Dubner. 2006. Freakonomics: Morrow.
Litchfield, J. T., Jr. 1962. Symposium on clinical drug evaluation and human pharmacology. XVI. Evaluation of the safety of new drugs by means of tests in animals. Clin Pharmacol Ther 3:665-72.
Milner, Richard. 2009. Darwin's Universe: Evolution from A to Z: University of California Press.
Pigliucci, Massimo. 2002. Denying Evolution: Sinauer.
Spriet-Pourra, C, M Auriche, and (Eds). 1994. SCRIP Reports: PJB.
Suter, KE. 1990. What can be learned from case studies? The company approach. In Animal Toxicity Studies: Their Relevance for Man, edited by C. Lumley and S. Walker. Lancaster: Quay.
Thagard, Paul. 1998. Why astrology is not science. In Philosophy of Science: The Central Issues, edited by M. Curd and J. A. Cover: Norton.