by Steven Hatch
Intriguing research? Quite possibly. Should patients who have cancer call their oncologists to inquire about experimental venom treatments? No, they shouldn’t. Do stories like this with a catchy title encourage those kinds of calls? They do, and, depending on the size of the story and the fantastical nature of the claims, those kinds of calls can occasionally grind a clinic to a halt as the physicians scramble to read about the research so as to give a knowledgeable response. Sometimes, as with this “Venom Cures Cancer” story, there isn’t any scientific study for a doctor to read about because this is preliminary research being presented at a scientific conference and hasn’t yet been published in a peer-reviewed journal.
Second, if it is clinical research, understand the kind of research that’s being done. News articles that don’t describe the basic design of the study aren’t worth reading. This book didn’t go into much detail about the different types of studies, except for emphasizing the difference between studies that identify correlations and those that directly test the effects of a variable (like a drug or vaccine trial). Studies that look at correlation should be viewed with a certain level of healthy skepticism. This is the central problem with the many epidemiologic studies that are fodder for news media: reports about fiber consumption, the link between using electronic devices and various diseases, the environmental causes of autism—these and other studies are all done largely by looking at correlations.
That’s not to imply that people should simply dismiss all studies that look at correlations: healthy skepticism is not the same thing as abject disbelief. Still, it’s worth keeping in mind the story of hormone replacement therapy, where the correlation data indicating it was beneficial was strong, but, when it was put directly to the test, hormone therapy failed. So if there is a clinical question that can be studied directly in a trial (usually, but not always, this applies to drugs), a well-designed trial is always superior.
It’s also useful to know whether investigators are studying weak or strong effects, and whether the studies looking at those effects are appropriately large or small. There’s always some study looking at the effects of, say, coffee consumption on any number of health questions, such as whether it increases the risk for heart attacks, whether consumption during pregnancy results in low birth weight, of whether it’s protective against Alzheimer’s disease. Whatever effects routine coffee consumption may have in these situations, they are likely to have a small impact, so studies that assert that something like coffee is linked with certain outcomes, yet enroll a relatively small number of people, should be regarded with caution, and are probably not worth changing your coffee habits immediately.
Finally, it’s critically important to understand the size of the benefit or harmful effect. Some clinical studies may have findings that meet statistical significance but may nevertheless not be “significant” in anything other than statistics. If a paper or a study merely says that there is some benefit, a good question to ask is, How big is the benefit? If the answer is slow in coming, or never does come, then it could very well be making mountains out of molehills.
For example, male circumcision of infants is sometimes justified in medical terms by studies indicating it may lower one’s risk of penile cancer later—much later—in life. Supposing these studies are accurate and well designed, circumcision would lower the risk of developing a cancer that, in industrialized nations, affects less than one man per 100,000. To put it differently, this research shows that circumcision lowers a man’s risk of getting penile cancer from almost nothing to half of almost nothing. There is just not very much medical benefit to be had, at least in terms of cancer prevention in nations with a high standard of living.* (I hope I am clear here—especially as a member in reasonably good standing of a culture that still practices male circumcision—that I am not opposed to the practice but am using the arguments about its medical benefits to make a point about statistical significance.)
The situation is different in less industrialized nations. Circumcision at any age in sub-Saharan Africa is extremely likely to reduce one’s risk of acquiring, and possibly passing along, HIV infection. The data on circumcision lowering the risk of penile cancer is also more robust in Africa, where penile cancer is significantly more common than in Europe or North America.
Similarly, a recent set of guidelines has come out about screening smokers for lung cancer with CT scans. The guidelines were based on a large trial comparing smokers who got an annual scan versus those who got an annual chest X-ray. The trial showed that there was a lifesaving benefit to the scans: if you just had an annual chest X-ray, you were more likely to die of lung cancer than if you got an annual CT.
At this point, if I have done a decent job of explaining overdiagnosis, positive predictive value, and uncertainty, your sensors should be on high alert wondering just how much more likely death will befall a person who gets annual X-rays instead of CT scans. The answer, as you may not be totally surprised, is not much, although the benefits can be framed in very different ways. The CT scans resulted in a 20 percent reduction in mortality over five years’ time. Whether that sounds like a big benefit I will leave for you to judge, but not before I translate that same benefit into a different form. The 20 percent reduction means that, for every one thousand people, three extra lives will be saved during a five-year span.
That statistic, too, might prompt some wondering about the potential harms of CT scanning: Is there any downside to annual screening that might negate this fairly modest benefit? One answer we know in theory (meaning that it has not been studied extensively at a clinical level) is that CTs pack a lot of radiation. That is, CTs themselves may increase one’s risk of lung cancer, and presumably the risks are going to be higher among younger patients because the radiation has more of an opportunity to create aberrant cells that lead to cancer over decades.
There is at least one other well-understood harm from these CTs, and this, too, should sound familiar, for the chance that just one CT will result in a false positive is a staggering 25 percent. After three years, the false-positive rate is nearly 40 percent. Mind you, although a false-positive mammogram may be associated with a more intense form of anxiety, suffering, and loss than a false-positive CT suggestive of lung cancer, a breast biopsy is for the most part a procedure that does not carry significant risk of physical harm. The same cannot be said of lung biopsies no matter how they are done, and moreover their accuracy is much lower than that of breast biopsies.
Lung CT scans for smokers is a different blizzard with its own set of snowballs, but the principles of the storm remain the same, which seems an appropriate place to end.
Bibliography
INTRODUCTION
Glare, P., et al. A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ 2003; 327:195–201.
Kondziolka, D., et al. The accuracy of predicting survival in individual patients with cancer. J Neurosurg 2014; 120:24–30.
Pollan, Michael. In defense of food: An eater’s manifesto. New York: Penguin Press, 2008.
Porter, Roy. The greatest benefit to mankind: A medical history of humanity. 1st American ed. New York: W. W. Norton, 1998.
Silver, Nate. The signal and the noise: Why so many predictions fail—but others don’t. New York: Penguin Press, 2012.
Taleb, Nicholas Nassim. The black swan: The impact of the highly improbable. New York: Random House, 2007.
CHAPTER ONE: PRIMUN NON NOCERE
Albertsen P., et al. Prostate cancer and the Will Rogers phenomenon. J Natl Cancer Inst 2005; 97(17):1248–1253.
American Cancer Society. Cancer facts & figures 2013. Atlanta: American Cancer Society, 2013.
Arkes, H., and Gaissmaier, W. Psychological research and the prostate-cancer screening controversy. Psychological Science 2012; 23:547–553.
Berliner, Uri. Tired of gloom and doom? Here’s the best good news of 2013. Morning Edition, National Public Radio, December 24, 2013.
Berner
, E., and Graber, M. Overconfidence as a cause of diagnostic error in medicine. Am J Med 2008; 121(5A):S2-S23.
Black, William. Overdiagnosis: An underrecognized cause of confusion and harm in cancer screening. J Natl Cancer Inst 2010; 92(16):1280–1282.
Carter, H. B., et al. Early detection of prostate cancer: AUA guideline. American Urological Association, 2013.
Harach, H., et al. Occult papillary carcinoma of the thyroid: A “normal” finding in Finland; A systematic autopsy study. Cancer 1985; 56(3): 531–538.
Haselton, M., and Buss, D. Biases in social judgment: Design flaws or design features? In Responding to the social world: Implicit and explicit processes in social judgments and decisions, ed. J. P. Forgas, K. D. Williams, and W. von Hippel, pp. 23–43. Cambridge: Cambridge University Press, 2013.
Heath, Iona. Overdiagnosis: When good intentions meet vested interests. BMJ 2013; 347:f6361–6363.
Ilic, D., et al. Screening for prostate cancer. Cochrane Database of Syst Rev 2013, Issue 1.
Kahneman, Daniel. Thinking fast and slow. New York: Farrar, Straus, and Giroux, 2011.
Monynihan, R., et al. Preventing overdiagnosis: How to stop harming the healthy. BMJ 2012; 344:e3502–e3508.
Moyer, V. A., and the US Preventive Services Task Force. Screening for prostate cancer: US Preventive Services Task Force recommendation statement. Ann Intern Med 2012; 157:120–134.
Rosenhan, D. L. On being sane in insane places. Science 1973; 179 (4070):250–258.
Soos, G., et al. The prevalence of prostate carcinoma and its precursor in Hungary: An autopsy study. Eur Urol 2005; 48(5):739–744.
Spitzer, Robert. On pseudoscience in science, logic in remission, and psychiatric diagnosis: A critique of Rosenhan’s “On being sane in insane places.” J of Abnorm Psychol 1975: 84(5): 442–452.
Spitzer, R., et al. The scientific credibility of Lauren Slater’s pseudopatient diagnosis study. J Nerv Ment Dis 2005; 193:734–739.
Welch, H. Gilbert. Overdiagnosed: Making people sick in the pursuit of health. Boston: Beacon Press, 2011.
Welch, H. G., and Black, W. Overdiagnosis in cancer. J Natl Cancer Inst 2010; 102:605–613.
Wiener, R. S., et al. When a test is too good: How CT pulmonary angiograms find pulmonary emboli that do not need to be found. BMJ 2013; 347:f3368-f3375.
CHAPTER TWO: VIGNETTE
Mlodinow, Leonard. The drunkard’s walk: How randomness rules our lives. New York: Pantheon Books, 2008.
CHAPTER THREE: SNOWBALL IN A BLIZZARD
American Cancer Society. Breast cancer facts & figures 2011–12. Atlanta: American Cancer Society, 2012.
Bassett, L., and Gold, R. The evolution of mammography. AJR 1988; 150; 493–498.
Caplan, Arthur. “Mammogram advice accurate but not ‘right.’” NBCNews.com, November 19, 2009. http://www.nbcnews.com/id/34040273/ns/health-cancer/t/mammogram-advice-accurate-not-right/#.VZhGllLbJMs, retrieved July 2015.
Elmore, J., et al. Screening mammograms by community radiologists: Variability in false-positive rates. J Natl Cancer Inst 2002; 94(18): 1373–1380.
Feld, Ellen. A doctor discovers a patient in the mirror. New York Times, July 19, 2010.
———. Finding little comfort in the statistics of survival. New York Times, October 17, 2011.
Fletcher, Suzanne. Whither scientific deliberation in health policy recommendations?—Alice in the Wonderland of breast cancer screening. N Engl J Med 1997; 336:1180–1183.
Gøtzsche, P. C., and Nielsen, M. Screening for breast cancer with mammography. Cochrane Database of Syst Rev 2011, Issue 4.
Jørgensen, K., and Gøtzsche, P. C. Overdiagnosis in publicly organised mammography screening programmes: Systematic review of incidence trends. BMJ 2009; 339:b2587-b2595.
Kerlikowske, Karla. Screening mammography in women less than age 50 years. Curr Opin Obstet Gynecol 2012; 24:38–43.
Kolata, Gina. Mammogram debate took group by surprise. New York Times, November 20, 2009.
———. Vast study casts doubt on value of mammograms. New York Times, February 11, 2014.
Lobo, Roger. Where are we 10 years after the Women’s Health Initiative? J Clin Endocrinol Metab 2013; 98:1771–1780.
Major cancer agencies respond to USPSTF’s new mammography guidelines. Oncology, December 15, 2009. http://www.cancernetwork.com/oncology-journal/major-cancer-agencies-respond-uspstf%E2%80%99s-new-mammography-guidelines, retrieved July 2015.
Mandelblatt, J., et al. for the Breast Cancer Working Group of the Cancer Intervention and Surveillance Modeling Network (CISNET). Effects of mammography screening under different screening schedules: Model estimates of potential benefits and harms. Ann Intern Med 2009; 151:738–747.
Miller, A., et al. Twenty-five year follow-up for breast cancer incidence and mortality of the Canadian National Breast Screening Study: Randomised screening trial. BMJ 2014; 348:g366–g376.
Nyström, L., et al. Long-term effects of mammography screening: Updated overview of the Swedish randomised trials. Lancet 2002; 359:909–919.
Pace, L., et al. Trends in mammography screening rates after publication of the 2009 US Preventive Services Task Force recommendations. Cancer 2013; 119:2518–2523.
Rabin, Roni Caryn. New guidelines on breast cancer draw opposition. New York Times, November 16, 2009.
Saul, Stephanie. Prone to error: Earliest steps to find cancer. New York Times, July 19, 2010.
Shapiro, Sam. Evidence on screening for breast cancer from a randomized trial. Cancer 1977; 39:2772–2782.
Susan G. Komen for the Cure. Facts for life: Breast cancer detection. http://ww5.komen.org/uploadedFiles/_Komen/Content/About_Breast_Cancer/Tools_and_Resources/Fact_Sheets_and_Breast_Self_Awareness_Cards/BreastCancerDetection.pdf. Retrieved July 2014; updated version retrieved July 2015.
US Preventive Services Task Force. Screening for breast cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2009; 151:716–726.
Zahl, P. H., et al. Incidence of breast cancer in Norway and Sweden during introduction of nationwide screening: Prospective cohort study. BMJ 2004; 328:921–924.
CHAPTER FOUR: THE PRESSURES OF MANAGING PRESSURE
Gibbons, G., et al. Refocusing the agenda on cardiovascular guidelines: An announcement from the National Heart, Lung, and Blood Institute. Circulation 2013; 128(15):1713–1715.
Husten, Larry. The heart guidelines are dead: Long live the heart guidelines. Forbes, August 9, 2013.
———. Minority report: Five guideline authors reject change in blood pressure goal. Forbes, January 13, 2014.
James, P., et al. Evidence-based guideline for the management of high blood pressure in adults: Report from the panel members appointed to the Eighth Joint National Committee (JNC8). JAMA 2014; 311(5):507–520.
Moser, M., et al. Report of the Joint National Committee on the detection, evaluation, and treatment of high blood pressure: A cooperative study. JAMA 1977; 237(3):255–261.
Myers, M. and Tobe, S. A. Canadian perspective on the Eighth Joint National Committee (JNC8) hypertension guidelines. J Clin Hypertens 2014; 16(4):246–248.
Navar-Boggan, A., et al. Proportion of US adults potentially affected by the 2014 hypertension guideline. JAMA 2014; 311(14):1424–1429.
Peterson, E., et al. Recommendations for treating hypertension: What are the right goals and purposes? JAMA 2014; 311(5):474–476.
SHEP Cooperative Research Group. Prevention of stroke by anti-hypertensive drug treatment in older persons with isolated systolic hypertension: Final results of the Systolic Hypertension in the Elderly Program (SHEP). JAMA 1991; 265:3255–3264.
Wright, J., et al. Evidence supporting a systolic blood pressure goal of less than 150 mm Hg in patients aged 60 years or older: The minority view. Ann Int Med 2014; 160(7):499–503.
CHAPTER FIVE: LYME’S FALSE PROPHETS
Asher, Tammy. Unprecedented antitrust investigation into the Lyme disease treatment guidelines development process. Gonzaga Law Rev 2011;
46:117–145.
Blumenthal, Richard. Attorney general’s investigation reveals flawed Lyme disease guideline process, IDSA agrees to reassess guidelines, install independent arbiter. Office of the Attorney General of the State of Connecticut, press release, May 1, 2008.
Brody, Jane. A threat in a grassy stroll. New York Times, July 15, 2008. cave76. Stricker’s alleged scientific misconduct. Lymenet Europe, January 27, 2008. http://www.lymeneteurope.org/forum/viewtopic.php?f=7&t=627&sid=f5c5d5cbd75745be96847bcd6158ce80&start=10.
Centers for Disease Control and Prevention. Other types of laboratory testing. http://www.cdc.gov/lyme/diagnosistesting/labtest/otherlab/index.html, accessed July 2015.
Chang, Kenneth. Paleontology and creationism meet but don’t mesh. New York Times, June 29, 2009.
Cressey, Daniel. XMRV paper withdrawn. Nature (blog), December 22, 2011.
Feder, H., et al. A critical appraisal of chronic Lyme disease. N Engl J Med 2007; 357:1422–1430.
International Lyme and Associated Diseases Society (ILADS). Basic information about Lyme disease. http://www.ilads.org/lyme/about-lyme.php, retrieved July 2015.
Klein, Jerome. Danger ahead: Politics intrude in Infectious Diseases Society of America guideline for Lyme disease. Clin Inf Dis 2008; 47:1197–1199.
Klempner, M., et al. Two controlled trials of antibiotic treatment in patients with persistent symptoms and a history of Lyme disease. N Engl J Med 2001; 345:85–92.
Landers, Susan. Lyme disease debate provokes treatment divide, legal action. American Medical News, December 25, 2006.
Lantos, P., et al. Final report of the Lyme Disease Review Panel of the Infectious Diseases Society of America (IDSA). Clin Inf Dis 2010; 51(1):1–5.