Skip to main content

Liars and Statistics. Part I

On January 23, 2013, Professor Robin Lovell-Badge, head of the division of Stem Cell Biology and Developmental Genetics at the Medical Research Council National Institute for Medical Research in London, wrote a blog titled: Nine out of ten Statistics are taken out of Context.

Mark Twain once opined: “A lie can travel halfway around the world while the truth is putting on its shoes.” Similarly, it takes but a sentence to tell a lie, but it may take a volume to refute the lie. Most of my blogs are longer than the average blog, in part, because I am taking it upon myself to correct lies and other misinformation. I try to assume the burden of proof even though the claimant actually has the burden of proof on him. As I am merely refuting the claims, I could simply say: “You have not offered evidence” and be done with it. But such is not consistent with the real world. Furthermore, I play by the rules, meaning that I use references and critical thinking as opposed to merely making claims and then acting indignant if anyone questions the claims that were offered without proof. Neither do I use the Gish Gallop: raising so many points that no one can respond to all of them because of time and space issues.

I have divided this blog into three parts as the refutation I am offering goes into concepts and facts that are complicated. Be that as it may, I have still not addressed everything in his essay. Regardless, sorry about the length! (All quotes that have emphasis in bold are my additions.)

Tom Holder posted this introduction for the Lovell-Badge essay: “This is a very clear and thorough debunking of a common animal rights myth where they suggest that because nine out of ten drugs that pass animal tests still fail to be approved, that animal tests must not work.” With that claim in mind, let‘s see what scientists in and out of Pharma have to say about whether they attribute the failures to animal models. In parts II and III, I will analyze the essay by Lovell-Badge.

Enna and Williams, in 2009, state:

Success in federally funded drug discovery initiatives has had a checkered history. As one example, while the 1971 National Cancer Act gave the National Cancer Institute a charter to cure cancer, the incidence of this disease in the United States remains the highest in the world, with a death rate that has remained unchanged for over 50 years (193.9 per 100,000 in 1950 vs. 193.4 per 100,000 in 2002). This lack of progress is both surprising and disappointing given the billions of dollars spent over the past 40 years on improving treatment options, reducing cancer-related behaviors, such as smoking, and increasing efforts in early detection (Aggarwal, Danda, Shan Gupta, & Gehlot, 2009). Many are now coming to the realization that, as in other therapeutic areas, the greatest limitation for identifying new drugs for treating cancer are the deficiencies in the animal models used for testing NCEs [new chemical entities, also referred to as new molecular entities or NMEs] (Aggarwal et al., 2009) . . .

A major hurdle in the translational medicine undertaking is the fact that most preclinical animal models of disease generally lack predictive value with respect to the human condition under study. Indeed, the false positives that result from the present generation of animal assays are a major cause of NCE attrition in the clinic either because of lack of efficacy or the appearance of unacceptable side effects that were not detected preclinically [in animals]. While there are notable, albeit retrospective, exceptions (Zambrowicz & Sands, 2003), this weakness in the conventional drug discovery process has not been resolved with the use of transgenic animals which themselves contribute additional confounds that further complicate data interpretation. [1]

Schreiber et al., in 2010, state:

The ability of recombinant DNA to provide nearly unlimited access to human proteins resulted in a second approach that is also common today—target-based drug discovery. Here, therapeutic targets are selected using insights gained most often from biochemistry, cell biology and model organisms. Small molecules are identified that modulate the targets (often by small-molecule screening) followed by optimization and clinical testing. Although this is a robust process, the common failure of candidate drugs in late-stage clinical testing, owing to unforeseen toxicity or lack of efficacy, reveals limits in our ability to select targets using surrogates of human physiology, such as in vitro assays and animal models. [2]

Markou, Chiamulera, Geyer, Tricklebank (of Eli Lilly), and Steckler (of Johnson and Johnson) state in 2009:

Despite great advances in basic neuroscience knowledge, the improved understanding of brain functioning has not yet led to the introduction of truly novel pharmacological approaches to the treatment of central nervous system disorders. This situation has been partly attributed to the difficulty of predicting efficacy in patients based on results from preclinical studies. . . . Few would dispute the need to move away from the concept of modeling CNS diseases in their entirety using animals. However, the current emphasis on specific dimensions of psychopathology that can be objectively assessed in both clinical populations and animal models has not yet provided concrete examples of successful preclinical-clinical translation in CNS drug discovery. . . . Since the founding of the American College of Neuropsychopharmacology (ACNP) in December 1961, there have been tremendous advances in neuroscience knowledge that have greatly improved our understanding of brain functioning in normal and diseased individuals. Unfortunately, however, these scientific advancements have not yet led to the introduction of truly novel pharmacological approaches to the treatment of central nervous system (CNS) disorders in general, and psychiatric disorders in particular (Hyman and Fenton, 2003; Fenton et al., 2003; Pangalos et al., 2007). . . . [3]

Neuzil et al., states in 2012:

Animal testing is not ideal either, as the predictive value of such tests is limited owing to metabolic differences between humans and animals, and many ethical issues are raised by the testing.[4]

Björquist et al., in Drug Discovery World 2007:

Furthermore, the compound attrition rate is negatively affected by the inability to predict toxicity and efficacy in humans. These shortcomings are in turn caused by the use of experimental pre-clinical model systems that have a limited human clinical relevance . . .  Animal models are today important tools to detect adverse effects of compounds but are costly and their clinical relevance is widely debated. In fact, animal models are about 50% effective in predicting human toxicity to the liver, heart and during development.[5]

Sharp and Langer write in 2011:

The next challenge for biomedical research will be to solve problems of highly complex and integrated biological systems within the human body. Predictive models of these systems in either normal or disease states are beyond the capability of current knowledge and technology.[6]

Zhang et al., state in 2010:

 [The publication of the report Toxicity Testing in the 21st Century: A Vision and a Strategy by the National Research Council of the National Academies of Science (NAS)] is a long-due response to the call by many for alternatives to the currently standard, whole-animal-based methodologies, which are inefficient, costly, and have had only limited success in making informative connections to human health risk associated with environmental chemical exposures.[7]

Elias Zerhouni, former director of NIH and current head of R&D at Sanofi was quoted in the June 25, 2012 issue of Forbes as saying: “R&D in pharma has been isolating itself for 20 years, thinking that animal models would be enough and highly predictive, and I think I want to just bring back the discipline of outstanding translational science, which means understand the disease in humans before I even touch a patient.”

Raven wrote in 2012: “ ‘The mouse models really don't reflect the human condition,’ says Shaw Warren, an infectious disease specialist at the Massachusetts General Hospital in Boston. ‘Clearly, current animal models seem to be incapable of predicting results in human trials of new agents,’ says Mitchell Fink, a surgeon at the University of California–Los Angeles.” [8]

Mullane and Williams [9] state in 2012: “The difficulties in predicting drug efficacy from preclinical models have been of concern for more than two decades . . . Thus, novel findings apparently related to the systems and targets involved in disease causality; the delineation of the efficacy, selectivity and safety of NCEs; and the predictive relevance of biomarkers and animal model data to the human disease state, even when there is evidence for target engagement in humans, all frequently fail to enhance the success rate for new drug applications (NDAs).” They continue stating that one reason for the problems Pharma is facing is: “(i) An over-reliance on animal models of diseases that are poorly validated in the manner they are applied.”

Clearly, scientists, not just animal advocates, do link the failure rate of new drugs to animal models. This is mainly due to the inability of animal models to predict efficacy and safety—the very things they are supposed to predict. While there are many other problems with Pharma, reliance on the animal model is well recognized and discussed. Peruse just about issue of a drug development journal and you will find an article discussing the problems with animal models and why early human testing is the key to solving the pipeline problem as well as the efficacy and safety problems.

I have stated many times that Pharma needs better tools. Animal models are not predictive for human response to drugs and disease. In vitro cell lines and in silico are, for the most part, not predictive either in terms of what Pharma is currently lacking. When seeking to analyze and solve the problem, however, the question that must be addressed is: “What are the tests being used for?” Some in vitro tests are of predictive value just as some in silico tests are. But animal models are primarily used to predict human response in terms of efficacy and safety and they fail utterly in this. When Holder claims that the following essay by Lovell-Badge negates this principle he is at odd with the scientists who are doing or who have done drug development. He is also at odds with the empirical evidence that reveals animal models are in fact not predictive for efficacy and safety.

On to the Lovell-Badge essay.


1.         Enna, SJ, M Williams (2009) Defining the role of pharmacology in the emerging world of translational research. Advances in pharmacology 57:1-30. 10.1016/S1054-3589(08)57001-3.

2.         Schreiber, SL, AF Shamji, PA Clemons et al. (2010) Towards patient-based cancer therapeutics. Nat Biotech 28:904-906.

3.         Markou, A, C Chiamulera, MA Geyer et al. (2009) Removing obstacles in neuroscience drug discovery: the future path for animal models. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology 34:74-89. 10.1038/npp.2008.173. 2651739.

4.         Neuzil, P, S Giselbrecht, K Lange et al. (2012) Revisiting lab-on-a-chip technology for drug discovery. Nature Reviews. Drug Discovery 11:620-632. 10.1038/nrd3799.

5.         Björquist, P, P Sartipy, R Strehl et al. (2007) Human ES cell derived functional cells as tools in drug discovery. Drug Discovery World:17-24.

6.         Sharp, PA, R Langer (2011) Promoting Convergence in Biomedical Science. Science 333:527. 10.1126/science.1205008.

7.         Zhang, Q, S Bhattacharya, ME Andersen et al. (2010) Computational systems biology and dose-response modeling in relation to new directions in toxicity testing. Journal of toxicology and environmental health. Part B, Critical reviews 13:253-276. 10.1080/10937404.2010.483943.

8.         Raven, K (2012) Rodent models of sepsis found shockingly lacking. Nat Med 18:998-998.

9.         Mullane, K, M Williams (2012) Translational semantics and infrastructure: another search for the emperor’s new clothes? Drug Discovery Today 17:459-468.


Popular Video