Several blogs have been written about drug testing, prediction, and why animal testing is necessary. I want to take a moment here and address one aspect of the “animal testing for drug safety” issue. Shanks and I go into much more detail in Animal Models in Light of Evolutionand I have mentioned almost all of this before in this blog, so this will be somewhat brief.
In the biological science, especially drug testing, scientists use the simple statistics I have referred to many times: sensitivity, specificity, positive and negative predictive values. This is how scientists can determine if a test modality per se is predictive for what will happen in humans. The studies that have been conducted on animals for toxicology and so forth reveal PPV and NPV that fall far below what is considered predictive in medical science. (SeeAnimal Models in Light of Evolution.)
A problem that some seem to be having is the notion that a test or modality as a whole, is something that can be predictive sometimes but not predictive at other times. Let’s take the chest x-ray for an example. (The following is for illustration purposes only and should not be taken as representative of current medical practice.)
Let’s assume a patient has a cough that produces blood. His physician orders a chest x-ray that reveals a crab-looking lesion in the left upper lobe. The question we now ask is how good of a predictor is a chest x-ray for diagnosing cancer. If we study the medical records of 5000 patients who have had a chest x-ray that showed a crab-like lesion and who then had a lung biopsy (the gold standard) we might find that 4500 of the biopsies revealed a cancer diagnosis while 500 revealed normal lung tissue.
Let’s also assume we evaluated the records of 5000 patients with a normal chest x-rays and for some reason they all decided to have a lung biopsy also. (Not something anyone would do.) Out of those 5000 patients, 10 turned out to have cancer diagnosed by the biopsy. We now do the calculations for positive predictive value (PPV) and negative predictive value (NPV) and come up with a PPV=0.9 and a NPV=0.998 (out of 1.0). Good numbers. The chest x-ray is predictive when used to diagnose lung cancer in our scenario. But now lets change the numbers.
Same 10,000 patients but instead of 4500 positive for cancer when the x-ray reveals a crab-looking lesion, we only see 3000 cancers on biopsy. And instead of 10 cancers when the chest x-ray was negative we see 1500. Now we have a PPV 0.6 and a NPV of 0.7. That would not be considered predictive for medical care. (For more, see Wikipedia or a textbook.)
In the first example the chest x-ray was predictive for lung cancer. In the second it was not. In the second example, the chest x-ray was not “predictive some of the time” and “not predictive at other times.” In the second example, the modality known as a chest x-ray was simply not predictive. To put it in the vernacular, it did not work! Whether a test for toxicity or drug efficacy, or a research method for determining the pathophysiology of HIV is predictive or not depends on the numbers. This is not confined to medical science. Whether a drug sniffing dog can find the cocaine (is predictive for cocaine the suitcase), also can be judged based on these simple calculations.
Neither animal nor human testing is 100% predictive (has a PPV and NPV of 1.0) for all humans. Human clinical trials are much better than animals testing however. That is why they are done. Claims have been made the equivalent of which is: “Method A is not perfect and method B is not perfect therefore both are equal.” The fallacy here is obvious. Method A might be ten times better than method B but still neither would be perfect. Nevertheless, method A is clearly superior.
There are other issues with the way drugs are developed and for the most part the pharmaceutical industry acknowledges the shortcomings, as I have pointed out in previous blogs. This blog is not meant to be an explanation of the drug development process as a whole. But if anyone thinks a modality can be predictive for drug A but not drug B that is the same as saying my horoscope predicted my future yesterday but failed today and therefore astrology is a predictive modality. That is not the way science works.
I understand why nonscientists have a problem with this. PPV and NPV are not intuitive ways of approaching the world. What I fail to understand is why competent scientists cannot read the same papers I read and come to the conclusion that animals cannot predict human response. Perhaps they can and do and that is why they do not provide references to their opinions in science blogs and refuse to debate the issue with me in peer review scientific journals. Much easier to take on acupuncture.