The anatomical and pathophysiological mechanisms of disease, though important to understand, are not the evidence that underpins the validity of medical treatment. Medicine is essentially an observational science and clinical trials endeavour to determine significant differences between the natural history of disease and the effect of treatment. Some basic understanding of statistics is needed - especially when assessing risk factor modification.
Significance
A result is statistically significant when the 'p' value is less than 0.05. This arbitrarily chosen value means that there is a 95% likelihood that an observation is not due to chance. The p value is a measure of the reliability of an observation, but it does not quantify any effect.
The word 'significant' is frequently used inconsistently. A statistically significant result from a trial is sometimes erroneously interpreted as having a high clinical significance.
Reporting risk reductions
Trials look at the incidence of outcomes with and without intervention. Absolute risk reduction is the difference between the outcome in the control group and the outcome in the intervention group in a specified time period.
The relative risk reduction is the absolute risk reduction as a proportion of the baseline rate. A relative risk reduction often seems impressive, but it may only represent a small difference. For example, if the event rate is 0.2% in the control group and 0.1% in the intervention group the relative risk reduction is 50%, but the absolute risk reduction is only 0.1%.
One must always know whether a quoted risk change is relative or absolute. Benefits of treatment are often presented in relative terms, but harms and adverse effects are usually presented in absolute terms (Table 1).
Table 1
|
Absolute and relative risk
|
|
Event rate control
|
Event rate intervention
|
Relative risk reduction
|
Absolute risk reduction
|
Number needed to treat
|
p value
|
|
20%
|
|
10%
|
|
50%
|
|
10%
|
|
10
|
|
<0.05
|
4%
|
|
2%
|
|
50%
|
|
2%
|
|
50
|
|
<0.05
|
0.2%
|
|
0.1%
|
|
50%
|
|
0.1%
|
|
1000
|
|
<0.05
|
|
The p value measures the reliability of the observation, not the quantum of effect. If the effect is small, a small p value can still be achieved with a large sample size.
|
Number needed to treat or harm
The number needed to treat is the number of patients who must be treated for a period of time to prevent one having the outcome of interest. It is the inverse of the absolute risk reduction (1/ARR). For example, if the absolute risk reduction after five years is 2%, then the number needed to treat is 50 (1/0.02). Fifty people need to be treated for five years to prevent one adverse outcome. This means that the outcome of interest will be unchanged for the 49 other people who took the treatment for five years. Some of these 49 people may come to harm as a result of adverse effects of treatment.
The number needed to harm is a less frequently published number. It is essentially the inverse of the absolute rate of adverse effects. Over 10 years, if 4% of women suffer venous thromboembolism while on hormone replacement therapy and 2% without hormone replacement therapy, the absolute harm rate of the therapy is 2% and the number needed to harm is 50. That is, for every 50 women treated one will develop a thrombosis that would not have otherwise occurred.2
Outcome
Trial end points are varied and one must have a clear understanding of the outcomes measured. Death, disability and morbidity are clinical end points, while others such as blood pressure, cholesterol or bone density are surrogate or intermediate markers. Surrogate end points may have merit as indicators of potential benefit, but they rely on other evidence providing a causal link to clinical outcomes. In the end all interventions must be justifiable by an improvement in patient well-being, that is, by clinical end points.