Observational Databases and Registries

Deborah P. Lubeck, Ph.D., Vice President, Health Economics and Outcomes Research, Lifecycle Sciences Group, ICON Clinical Research

Introduction

The outcomes and costs associated with medical innovations are critical issues for society. New therapies are required to be efficacious, safe, and effective in usual practice. While randomized clinical trials are the standard for safety and efficacy of medication or device, results may not reflect treatment effectiveness in the community. The evidence gap between clinical trials and studies of actual effectiveness has been a focus of health services research for more than 30 years with renewed interest in this topic given recent decisions by the US government.^1-6

Key elements of clinical trials, such as narrow selection criteria, reduced sample sizes and restrictions on concomitant medications limit the data available for decision-making. Other sources of variability including practice and patient heterogeneity, comorbid illness and imperfect adherence to treatment regimens limit the generalizability of trial results for community practices. For these reasons, randomized clinical trials do not always provide practitioners with the information needed to determine if the intervention will work in clinical practice.^{4, 7}

There are alternatives to the conventional randomized clinical trial that yield results generalizable to clinical practice, while providing rigorous measurement of outcomes. These include pragmatic clinical studies that randomize patients to comparative therapies in usual care settings, retrospective cohort studies, and non-randomized prospective observational studies.^{4, 8, 9} These studies measure multiple outcomes, capture the impact of long-term illness, evaluate the role of concomitant disease or treatment play in long term effectiveness, and have been shown to have similar results to clinical trials in some cases.^10-12

What are Observational Databases?

Observational, prospective studies, also called registries, document the experience of persons after the identification of a specific event, such as a diagnosis, clinical milestone, or initiation of medical or surgical treatment. All participants who have the disease of interest are enrolled in the study irrespective of the presence or absence of other health conditions, past treatment or compliance with treatment. Sequential measurement of clinical and patient-reported outcomes, obtained at regular intervals in a uniform manner, is an essential component of these studies. Disease registries are useful in evaluating a breadth of data in a timely fashion, including patterns of comorbidities, disease course, physician diagnosis and treatment patterns, patient reported outcomes, and resource utilization in community settings. Registries which focus on a specific drug are often conducted to evaluate long-term safety.

Table 1

What are the Objectives of Observational Databases?

There may be multiple objectives for observational databases and each guide the study design and patient population. One goal may be to accumulate and document a large heterogeneous patient experience. Registries provide access to large numbers of patients cared for by community practitioners. Clinical data, including use of concomitant medications or time to treatment failure, adverse events, survival, resource utilization (often requested for reimbursement approval¹³) workforce participation, and health-related quality of life may all be collected over time, irrespective of whether or not the patient remains on therapy. These data can then be used in analyses of different patient subgroups to identify non-responders or responders, and patients at greater risk for disease progression.

Another goal of observational studies is to identify and prioritize the key issues for comparative effectiveness research, including aiding in development of clinical trials to address gaps in clinical knowledge, head-to-head treatment comparisons, to study the natural history of a disease subsequent to medical or surgical intervention, and to evaluate how existing therapies interact with new products, procedures or clinical practices. A special focus is to identify rare and time-delayed treatment or disease-related events that may not be captured in clinical trials due to small study populations, shortened length of follow-up, or the limited use of concomitant medications.

Lastly, observational databases may be designed to obtain information on practice patterns over time, by type of provider, and geographic location. Randomized clinical trials are conducted under well-defined protocols with formal evaluation of treatment compliance. In usual clinical practice, new technologies are introduced into practice slowly and treatment compliance may vary, thus producing outcomes different from those observed in the trial. Registry data provide information on how treatments are introduced into clinical practice, how these new treatments influence practice, and how these practices compare against recommended treatment guidelines.

When are Observational Studies Useful?

Observational study designs provide a body of evidence that complement data collected in randomized clinical trials. There are conditions and environments where disease registries are especially useful. For example, over the past decade diagnosis and treatment of localized prostate cancer has evolved. Earlier diagnosis, associated with prostate specific antigen screening, is associated with the increased use of local treatment (radical prostatectomy, external-beam radiation, and interstitial radiotherapy).^14-16 However, these treatments may negatively affect patient quality of life, resulting in impotence and incontinence. Though there is improved survival the potential for reduced quality of life is a significant factor in treatment decisions.¹⁷

There are also important events, such as rare side effects or long-term treatment effects, that are difficult to capture in a clinical trial due to limited sample size or shortened follow-up. Also, persons excluded from clinical trials (e.g. on concomitant medications; with comorbidities; of specific ages) are often the most intense users of health care resources and have more quality of life impairment. Registries may be the only approach for capturing their long-term clinical outcomes or rare events.^18,19

Table 2

Implementation

Appropriate implementation of disease registries is critical to adequate and appropriate data collection, management and analysis. Registries typically employ methods different from those in randomized clinical trials. A Scientific Advisory Board of clinical experts, site participants, and technical representatives is crucial for developing appropriate data collection forms, setting research priorities, and annual review of forms (to keep abreast of clinical practice). Study physicians and nurses must be committed to continuing recruitment of patients and to data entry.

While there are usually no mandated visits in registries, standardized data collection forms and rigorous quality assurance protocols are essential. Data may be collected electronically to facilitate quality assurance and data reporting. Electronic data collection has many advantages: simultaneous quality assurance, routine computation of calculated variables, and timely graphic summaries available to the physician for patient care or benchmarking. Simplicity and brevity are necessary to ensure prospective and complete data collection in longitudinal observational studies.

Observational databases are an ideal venue for capturing patient reported information, including quality of life, satisfaction with care, resource utilization, and disease and treatment symptoms, even though this may require additional effort. Successful longitudinal databases have provided rigorous studies of patient outcome based on descriptors and interventions provided by patient questionnaires, and do not rely solely on the medical history.

Data Analysis

The intention of many large, observational studies is to go beyond descriptive data to draw causal inference about treatment impact and efficacy. But observational data also introduce their own biases. These data are usually based on individuals who elect to join in the registry, with experiences that occur prior to the start of data collection and which may not be completely documented. Registries may also have characteristics that unexpectedly influence outcomes. For example, persons who participate in longitudinal databases may be better educated, more likely to be retired, and more likely to be female. All of these characteristics may affect treatment and disease outcomes.

There are many techniques for analyzing observational data. These include mixed models, random effects model, proportional hazards regression, generalized estimating equations, and nonparametric approaches.^{20, 21} Larger sample sizes allow for inclusion of critical covariates in the analyses; an approach not often used in clinical trials where evaluable patients or intent-totreat patients are compared. Approaches for addressing selection bias in analyses include propensity scores and instrumental variables.^{22, 23} Using these approaches patients may be matched on clinical or demographic characteristics for a more stand cohort comparison. Since the type of errors present in observational studies vary widely, it is critical to involve a statistician familiar with issues of selection bias, censoring, and missing data in discussions of analytic design and methods.

Summary

There are many types of research topics that may be addressed with registry data dependent on the heterogeneity of patients and clinical sites, length of follow-up and completeness of data collection.^24-30 Examples include:

Identify rare adverse events or events associated with concomitant medications
Evaluate outcomes associated with disease and treatment (burden of illness, cost-effectiveness, measurement of incremental benefit)
Impact of comorbidity on treatment and clinical outcomes
Comparative effectiveness of provider type, setting of care, and different medical and surgical interventions
Evaluation of changes in clinical practice (including adherence to clinical guidelines)

Scientifically rigorous registries require large samples sizes and standardized data collection, and may require longer follow-up to evaluate critical outcomes. All of these factors need to be considered when designing a disease registry.

References:

Please see references online at: www.iconplc.com/insight