Observational studies and their utility for practice

Randomised controlled clinical trials are the best source of evidence for assessing the efficacy of drugs. Observational studies provide critical descriptive data and information on long-term efficacy and safety that clinical trials cannot provide, at generally much less expense.

Observational studies include case reports and case series, ecological studies, cross-sectional studies, case-control studies and cohort studies. New and ongoing developments in data and analytical technology, such as data linkage and propensity score matching, offer a promising future for observational studies. However, no study design or statistical method can account for confounders and bias in the way that randomised controlled trials can.

Clinical registries are gaining importance as a method to monitor and improve the quality of care in Australia. Although registries are a form of cohort study, clinical trials can be incorporated into them to exploit the routine follow-up of patients to capture relevant outcomes.

Observational studies involve the study of participants without any forced change to their circumstances, that is, without any intervention.¹ Although the participants’ behaviour may change under observation, the intent of observational studies is to investigate the ‘natural’ state of risk factors, diseases or outcomes. For drug therapy, a group of people taking the drug can be compared to people not taking the drug.

The main types of observational studies used in health research, their purpose and main strengths and limitations are shown in the Table.^2-8 Their purpose may be descriptive, analytical or both.

• Descriptive studies are primarily designed to describe the characteristics of a studied population.
  
• Analytical studies seek to address cause-and-effect questions. 
  

Case reports and case series are strictly speaking not studies. However, they serve a useful role in describing new or notable events in detail. These events often warrant further formal investigation. Examples include reports of unexpected benefits or adverse events, such as a case report describing the use of high-dose quetiapine in treatment-resistant schizophrenia after intolerance to clozapine developed⁹ and a case report of a medication error involving lookalike packaging.¹⁰

Ecological studies are based on analysis of aggregated data at group levels (for example populations), and do not involve data on individuals. These data can be analysed descriptively, but not definitively for causation. Typical examples include studies that examine patterns of drug use over time. One example is the comparison of the use of non-steroidal anti-inflammatory drugs and COX-2 inhibitors in Australia and Canada.¹¹ Sometimes ecological studies describe associations between drugs and outcomes, such as changes in the rates of upper gastrointestinal haemorrhage after the introduction of COX-2 inhibitors.¹² However, because individual-level data are not presented, causality is at best only implied in ecological studies. The 'ecological fallacy' refers to the error of assuming that associations observed in ecological studies are causal when they are not.

Cross-sectional studies collect data at a single point in time for each single individual, but the actual data collection may take place over a period of time or on more than one occasion. There is no longitudinal follow-up of individuals. Cross-sectional studies represent the archetypal descriptive study.¹ Typically, they provide a profile of a population of interest, which may be broad, like the Australian Health Survey undertaken intermittently by the Australian Bureau of Statistics,¹³ or focused on specific populations, such as older Australians.¹⁴

Case-control studies focus on determining risk factors for an outcome of interest (such as a disease or a drug’s adverse effect) that has already occurred.⁵

First, two groups of participants are assembled:

• those who already have the outcome (cases)
  
• those who do not have the outcome (controls), who are often matched to the cases to make them similar and reduce bias.
  

Second, data on previous exposure to selected risk factors are collected and compared to see if these risk factors are more (or less) common among cases versus controls. Case-control studies are useful for studying the risk factors of rare outcomes, as there is no need to wait for these to occur. Multiple risk factors can be studied, but each case-control study can involve only one outcome.⁵ One example explored the relationship between the use of antiplatelet and anticoagulant drugs (risk factor) and the risk of hospitalisation for bleeding (outcome) in older people with a history of stroke.¹⁵ Another case-control study explored the risk factors for the development of flucloxacillin-associated jaundice (outcome).¹⁶

Cohort studies compare outcomes between or among subgroups of participants defined on the basis of whether or not they are exposed to a particular risk or protective factor (defined as an exposure). They provide information on how these exposures are associated with changes in the risk of particular downstream outcomes. Compared to case-control studies, cohort studies take individuals with exposures and look for outcomes, rather than taking those with outcomes and looking for exposures. Cohort studies are longitudinal, that is they involve follow-up of a cohort of participants over time. This follow-up can be prospective or retrospective. Retrospective cohort studies are those for which follow-up has already occurred. They are typically used to estimate the incidence of outcomes of interest, including the adverse effects of drugs.

Cohort studies provide a higher level of evidence of causality than case-control studies because temporality (the explicit time relationship between exposures and outcomes) is preserved. They also have the advantage of not being limited to a single outcome of interest. Their main disadvantage, compared to case-control studies, has been that longitudinal data are more expensive and time-consuming to collect. However, with the availability of electronic data, it has become easier to collect longitudinal data.

One prospective cohort study explored the relationship between the continuous use of antipsychotic drugs (exposure) and mortality (outcome) and hospitalisation (outcome) in older people.¹⁷ In another older cohort, a retrospective study was used to explore the relationship between long-term treatment adherence (exposure) and hospital readmission (outcome).¹⁸

Compared to randomised controlled trials, observational studies are relatively quick, inexpensive and easy to undertake. Observational studies can be much larger than randomised controlled trials so they can explore a rare outcome. They can be undertaken when a randomised controlled trial would be unethical. However, observational studies cannot control for bias and confounding to the extent that clinical trials can. Randomisation in clinical trials remains the best way to control for confounding by ensuring that potential confounders (such as age, sex and comorbidities) are evenly matched between the groups being compared. In observational studies, adjustment for potential confounders can be undertaken, but only for a limited number of confounders, and only those that are known. Randomisation in clinical trials also minimises selection bias, while blinding (masking) controls for information bias. Hence, for questions regarding drug efficacy, randomised controlled trials provide the most robust evidence.

New methods of analysis and advances in technology are changing the way observational studies are performed.

Clinical registries

Clinical registries are essentially cohort studies, and are gaining importance as a method to monitor and improve the quality of care.¹⁹ These registries systematically collect a uniform longitudinal dataset to evaluate specific outcomes for a population that is identified by a specific disease, condition or exposure. This allows for the identification of variations in clinical practice²⁰ and benchmarking across practitioners or institutions. These data can then be used to develop initiatives to improve evidence-based care and patient outcomes.²¹

An example of a clinical registry in Australia is the Australian Rheumatology Association Database,²² which collects data on the biologic disease-modifying antirheumatic drugs used for inflammatory arthritis. Clinical data from treating specialists are combined with patient-reported quality of life data and linked to national databases such as Medicare and the National Death Index. This registry has provided insight into the safety and efficacy of drugs and their effect on quality of life. It was used by the Pharmaceutical Benefits Advisory Committee to assess cost-effectiveness of these drugs.²³

Another example is the Haemostasis Registry. It was used to determine the thromboembolic adverse effects of off-label use of recombinant factor VII.²⁴

Clinical registries can also be used to undertake clinical trials which are nested within the registry architecture. Patients within a registry are randomised to interventions and comparators of interest. Their outcome data are then collected as part of the routine operation of the registry. The key advantages are convenience, reduced costs and greater representativeness of registry populations as opposed to those of traditional clinical trials.

One of the first registry-based trials was nested within the SWEDEHEART registry.²⁵ This prospectively examined manual aspiration of thrombus at the time of percutaneous coronary intervention in over 7000 patients.²⁶ The primary endpoint of all-cause mortality was ascertained through linkage to another Swedish registry. The cost of the trial was estimated to be US$400 000, which was a fraction of the many millions that a randomised controlled trial would have cost.

Propensity score matching

Even without randomising people within cohorts, methods have emerged in recent years that allow for less biased comparisons of two or more subgroups. Propensity score matching is a way to assemble two or more groups for comparison so that they appear like they had been randomised to an intervention or a comparator.²⁷ In short, the method involves logistic regression analyses to determine the likelihood (propensity) of each person within a cohort being on the intervention, and then matching people who were on the intervention to those who were not on the basis of propensity scores. Outcomes are then compared between the groups. Propensity score analysis of a large cohort of patients with relapsing remitting multiple sclerosis found that natalizumab was superior to interferon beta and glatiramer acetate in terms of improved outcomes.²⁸

Data technology

Increasing sophistication in techniques for data collection will lead to ongoing improvements in the capacity to undertake observational studies (and also clinical trials). Data linkage already offers a convenient way to capture outcomes, including retrospectively. However, ethical considerations must be taken into account, such as the possibility that informed consent might be required before linking data. Machine learning will soon allow for easy analyses of unstructured text (such as free text entries in an electronic prescription).²⁹ Patient-reported outcome measures are important and in future will be greatly facilitated by standardised, secure hardware and software platforms that allow for their capture, processing and analyses.

While clinical trials remain the best source of evidence regarding the efficacy of drugs, observational studies provide critical descriptive data. Observational studies can also provide information on long-term efficacy and safety that is usually lacking in clinical trials. New and ongoing developments in data and analytical technology offer a promising future for observational studies in pharmaceutical research.

Julia Gilmartin-Thomas is a Dementia research development fellow with the National Health and Medical Research Council (NHMRC) - Australian Research Council (ARC). Ingrid Hopper is supported by an NHMRC Early Career Fellowship.