Every now and then I like to remind our medical students of a few salient points when deciding whether a patient needs a certain drug, or whether their medication needs changed at all. These are in no particular order; they are not meant to be comprehensive, nor are they meant to replace the Ten Commandments of Prescribing that I've mused on previously. However, they may help in that decision-making process when you're weighing up the findings of studies.
- Statistical significance is not the same as biological significance.
Just because you see a really impressive p value, this does not mean that prescribing drug A will be much more effective than drug B (or even nothing at all). You need to ask yourself: what is the size of the actual effect that is being measured? Is this a clinically relevant distinction - i.e. will your patient be much better off on drug A than they would be otherwise?
- Relative risk is clinically meaningless - you need absolute risks.So what if drug A has twice the risk of complication X than drug B? If the complication only arises in 1% of patients on A, that means that only 2% of patients get the complication on B. That's still not a lot. Furthermore, for rare complications, the numbers are typically very low indeed, so calculating this relative risk is difficult, and the confidence intervals tend to be large.
- Trial results are very different from real world results.
Drugs are trialled on relatively standardised patients being monitored in standardised ways, and the subjects and researchers are typically super-motivated to make sure they're taking the drug properly and doing everything by the book. So even if the superiority of drug A over drug B is truly demonstrated in a trial in a certain group, it is very rare that the same degree of superiority persists in the Real World, which is where your patients live.
- Trials rarely include elderly patients on polypharmacy with multiple co-morbidities.
This is important, because very often these are precisely the people in whom you will be prescribing drug A or B. It's related to the point above - patients are complex. If you are going to start a patient on a drug, be aware that it is very likely that they would not have made the entry criteria for the study that assessed its efficacy in the first place, so exercise appropriate scepticism that you're doing any damn good at all.
- A percentage improvement in a surrogate marker is not the same as effect on underlying disease.
Perhaps one of the most important points. All young doctors have it drummed into them: treat the patient, not the blood result. Lowering someone's cholesterol from (say) 6.0 to 4.5mmol/l is of no value whatsoever unless you can show that you have reduced their risk of heart attack (and not increased their risk of other problems). It does not matter what the rates of heart attack are in the population for patients with cholesterols of 6.0 vs 4.5 - the relevant measure is: what is the rate of heart attacks in patients whose cholesterol has been lowered from 6.0 to 4.5. These are not the same thing.
- Differences between treatment and control groups in trials do not equate to the probability of improvement if you treat your individual patient.
This is a slightly more complex one. Say you have a drug A which has been shown in a trial to be effective against The Purple Heebie-Jeebies (PHJ). In a big trial the group of patients on A showed improvement in 50% whereas the control group had a 25% rate of improvement. In walks wee Mrs Miggins, and you confidently diagnose PHJ. Now you might think that if you start her on drug A she will have a 50% chance of getting better, whereas if you didn't, her chances are only 25%. Sounds good. However, let's assume that the trial data are correct, and she fits the bill. You have 4 Mrs Migginses, and you start them all on drug A; 2 of them get better, whereas only one would have got better otherwise. But you have treated FOUR patients in order to get a clinical improvement in one. Suddenly this doesn't look so impressive, even if the data are correct. This concept is the "Number Needed to Treat" (NNT), and it is perhaps the hardest yet most important concepts in all of therapeutics.
- Numerical difference between groups is not the same as individual improvement score.
If 50% of patients on drug A show an improvement, whereas 25% of controls show improvement, that is very much not the same as saying that the patients on the drug do twice as well as the patients in the control group. This sounds obvious, but all it means is that a greater number of patients show improvement, not that all the patients show a greater improvement. Subtle, but important.
So there you go - just a few little items among many to get us thinking about whether we are doing the right thing with Mrs Miggins when her Purple Heebie-Jeebies start flaring up. Please add your thoughts to the comments below. Am I being unfair? Go to it.