Are we there yet? - Travel along the information highway seeking evidence-based medicine

Introduction

To practise evidence-based medicine doctors must have access to the evidence when they need it. The vast increase in healthcare information makes it difficult to do this.

The National Health and Medical Research Council (NHMRC) classifies the strength of evidence into four levels. These levels reflect the research methods used in clinical trials (Table 1). Ideally all treatment decisions would be based on Level I, the highest level of evidence. Failing this, lower levels of evidence need to be used to guide the decisions.

To help people access the higher levels of evidence the Cochrane Collaboration is collating all randomised-controlled trials and systematically reviewing the results. The Cochrane Collaboration makes this information available electronically through its subscription databases, and via State government initiatives such as the Clinical Information Access Program of New South Wales, and Victoria's Clinicians Health Channel.

It is impossible to keep up with all the developments in medicine. Inevitably patients will present clinical problems to which their doctors do not know the solutions. Advice can be sought from colleagues, but it must be remembered that the opinion of respected authorities, based on clinical experience, descriptive studies or reports of expert committees is considered to be the lowest form of evidence.

Case - to immunise or not to immunise?

A fit, 35-year-old woman who is 30 weeks pregnant consults you in mid-winter because she wants advice about influenza vaccination. She has heard that last year a woman died of `flu' late in pregnancy. Indeed, just recently one of her friends had `the flu', was off work for a week, and even some weeks later her friend has not regained full strength. Your patient does not wish to lose time from work with the flu as she has a large information technology consulting project to finish before she delivers; she plans to work up until the end of the 37th week of pregnancy. She thinks it might be a good idea to be immunised, but does not want to harm her unborn child. One of her friends has given her the Consumer Medicine Information part of an influenza vaccine package insert (Fig. 1).

Action

Next steps

After reading the Consumer Medicine Information leaflet it appears there are no definite contraindications to the influenza vaccine. Although there is a general note of caution relevant to pregnancy, there is no recommendation for or against immunisation of healthy pregnant women. Your well-educated patient reiterates her desire to be protected from influenza but also her concern for the well-being of her unborn child. She seeks your advice.

To attempt to address her question, you and the patient read the medical part of the information in the package insert. This information is also silent on pregnancy as an indication for use. The only note regarding pregnancy states that there is no convincing evidence of risk to the fetus from immunisation of pregnant women using inactivated virus vaccines, bacterial vaccines or toxoids.

Doctor's literature search

To obtain evidence of the pros and cons of influenza vaccination in late pregnancy an electronic search was performed (Table 2).

Surprisingly, no authoritative guidelines were found, nor was Level I evidence available for this simple, widespread procedure. The papers found by the Medline search were evaluated on-screen as follows:

• the titles were reviewed and the papers were judged as relevant, possibly relevant or not relevant
• the abstracts, if present, of the papers judged by their titles to be relevant or possibly relevant, were reviewed and judged as possibly providing relevant information or not
• the abstracts that remained after this selection process were judged as helping to make a decision or providing no help at all.

This approach yielded 50 papers. A review of their titles suggested 11 were relevant and 22 possibly relevant. Of these 33 papers, five abstracts appeared able to inform the decision, but only three provided possibly useful information.

The first paper describes a study in which 189 women who were immunised just before or during pregnancy were compared with a control group of 517 women. There was no association between immunisation and maternal, perinatal or infant complications or outcomes. No teratogenicity was seen. This small sample lacks the statistical power to detect even relatively frequent events, thus it provides only weak evidence. The unstructured abstract does not allow readers to judge whether this was a randomised trial, nor even whether the women received the immunisation inadvertently or deliberately. If the vaccine was given deliberately an institutional ethics review board presumably approved the practice, which would suggest the practice was thought to be safe, but this is by no means clear.

The second paper describes the influenza vaccine for 1978-79. It states that pregnant women do not appear to have any special risk from influenza vaccination; physicians evaluating them should use the same criteria applied to other persons. The third paper's abstract provides the same advice. These two papers appear to be quoting the same primary source, but the basis for their advice is not clear.

This search took 37 minutes via a high-speed university internet access portal. Searchers who do not have reliable high-speed internet access would be expected to take longer, as might inexperienced searchers. The search yielded little evidence upon which to base a decision of whether or not to give influenza immunisation.

Librarian's literature search

An experienced medical librarian was told of the case and independently performed an electronic search of Medline and the Cumulated Index of Nursing and Allied Health Literature (CINAHL). Her search strategy is shown in Figure 2. She also sought information in the Cochrane Collaboration and Aust Health databases.

The results of the librarian's search were evaluated using the same method as for the doctor's literature search. The titles of the 45 articles found suggested that nine articles were relevant and 30 possibly relevant. Of these, 22 had abstracts, but only five appeared able to inform the decision. Three of these papers provided directly useful information while the rest provided indirect evidence.

The strongest recommendation to use influenza vaccination in pregnancy contains no information with which to judge the basis for this advice. At best this could be Level IV evidence, at worst uninformed opinion.

Another paper describing a study that aimed to test whether maternal immunisation could improve passive antibody protection in young infants reported that women in the last trimester of pregnancy were given trivalent inactivated influenza virus vaccine. Similar evidence is provided by another paper reporting a study of 448 eligible pregnant women who were offered the influenza vaccine at routine prenatal visits. These papers infer Level IV evidence, but the validity of the studies cannot be adequately judged.

Supporting evidence comes from another paper that discusses possible approaches to a flu pandemic. In the abstract the authors state `Pregnant women should probably be vaccinated'. This is Level IV evidence at best.

The final article of the five selected reports a study of hospitalisations and deaths from selected acute cardiac or respiratory conditions in pregnant women during influenza seasons. In a nested case-control study, 4369 women enrolled in a Medicaid program with a first study event during an influenza season were compared with 21 845 controls. In comparison with postpartum women, the odds ratios associated with study events increased from 1.44 (95% confidence interval (CI) 0.97-2.15) for women at 14-20 weeks gestation to 4.67 (95% CI 3.42-6.39) for those at 37-42 weeks. Women in their third trimester without other identified risk factors for influenza morbidity had an event rate of 21.7 per 10 000 women-months during an influenza season. Approximately half of this morbidity, 10.5 (95% CI 6.7-14.3) events per 10 000 women-months, was attributable to influenza. Influenza-attributable risks in comparable non-pregnant and postpartum women were 1.91 (95% CI 1.51-2.31) and 1.16 (95% CI 0.09-2.42) per 10 000 women-months, respectively. The data suggest that, out of every 10 000 women in their third trimester without other identified risk factors who experience an average influenza season of 2.5 months, 25 will be hospitalised with influenza-related morbidity. This is not an article about treatment, but does describe the magnitude of the influenza problem in pregnant women.

Comment

The paucity of information yielded by the electronic searches is disappointing. The lack of authoritative, up to date, immunisation guidelines was surprising. None of the possibly relevant papers could be accessed in full-text format. The limited evidence that was found needs to be interpreted with caution. Ideally, when assessing individual studies one should obtain the full texts of the papers to critically appraise their methods so that one can judge the validity of the studies and the applicability of the results to one's patients. At present only a few journals (such as Australian Prescriber) allow electronic access to their full text without prior subscription. In most instances clinicians are unable to access the papers they need to appraise.

The electronic searches conducted independently by the doctor and by the experienced medical librarian found different information. Each search took approximately 35 minutes. If critical appraisal of the full text of the articles had been possible, it would have added even more time to the process required for the practice of evidence-based medicine.

Conclusion

The road to evidence-based medicine is long, and we are but part way along it. Nonetheless, in the same way that modern transport has shrunk physical distances, it seems likely that information technology will continue to make accessible health-related information that previously was not accessible.

What are practitioners to do? To stretch the analogy further still, intrepid explorers will continue to take paths into the unknown and will through their trailblazing make information more accessible to the less adventurous. The intrepid explorers may be members of the Cochrane Collaboration or members of special societies or other organisations that take it upon themselves to produce evidence-based practice guidelines. Some individual clinicians who make the extraordinary effort of seeking out the best available evidence when they need it might also be among these explorers. Economic and other pressures dictate that not everyone can be an explorer. For the moment, in many areas there is no evidence and if there is, many doctors do not have the skills or time to find and appraise it.

Postscript

The doctor contacted an expert by e-mail for advice, and received the following reply.

`In previous years the flu vaccine has not been recommended for pregnant women. This year, the NHMRC has recommended it for all pregnant women. The reason for the change was the result of a case where a pregnant woman got influenza and actually ended up dying from it; the vaccine would have prevented her death. There is no evidence that the vaccine does any harm to the mother or the baby.' (Personal communication, Associate Professor Philip Hegarty, Faculty of Health and Behavioural Sciences, Deakin University, 2000)

The Consumer Medicine Information 2001 now recommends influenza vaccination for pregnant women who are in an at-risk group.

E-mail: [email protected]