Secondary use of routinely collected patient data in a clinical trial: An evaluation of the effects on patient recruitment and data acquisition

https://doi.org/10.1016/j.ijmedinf.2012.11.008Get rights and content

Abstract

Purpose

Clinical trials are time-consuming and require constant focus on data quality. Finding sufficient time for a trial is a challenging task for involved physicians, especially when it is conducted in parallel to patient care. From the point of view of medical informatics, the growing amount of electronically available patient data allows to support two key activities: the recruitment of patients into the study and the documentation of trial data.

Methods

The project was carried out at one site of a European multicenter study. The study protocol required eligibility assessment for 510 patients in one week and the documentation of 46–186 data elements per patient. A database query based on routine data from patient care was set up to identify eligible patients and its results were compared to those of manual recruitment. Additionally, routine data was used to pre-populate the paper-based case report forms and the time necessary to fill in the remaining data elements was compared to completely manual data collection.

Results

Even though manual recruitment of 327 patients already achieved high sensitivity (88%) and specificity (87%), the subsequent electronic report helped to include 42 (14%) additional patients and identified 21 (7%) patients, who were incorrectly included. Pre-populating the case report forms decreased the time required for documentation from a median of 255 to 30 s.

Conclusions

Reuse of routine data can help to improve the quality of patient recruitment and may reduce the time needed for data acquisition. These benefits can exceed the efforts required for development and implementation of the corresponding electronic support systems.

Highlights

► Database queries can show significantly higher sensitivity and specificity than manual recruitment. ► Pre-population of case report forms can significantly accelerate data acquisition. ► Time savings through electronic support of patient tracking seemed considerable.

Introduction

Running clinical studies is time-consuming and requires constant focus on data quality. The lack of time physicians are able to dedicate to non-care related activities make it difficult to conduct studies in parallel to patient care. This is especially true for large studies that depend on the input of multiple investigators with varying personal interest in the study. From the point of view of medical informatics, the growing amount of electronically available patient data enables the reuse of the electronic medical record for clinical research e.g. to support the two key activities in such studies: the recruitment of patients and the documentation of study data [1].

The general idea for routine data based recruitment support is to encode the eligibility criteria of a study and thus enable a computer to regularly compare them with the electronic profile of all patients within the hospital or outpatient clinic. Both, very simple algorithms based on manual translation of eligibility criteria to SQL scripts [2], [3] and more sophisticated approaches like semantic web techniques [4], probability calculation [5], [6] and natural language processing [7], [8], [9] have been applied to compare eligibility criteria with routinely documented patient data. Integration into the physician's workflow ranges from invocation of passive screening lists [10] to active systems, which notify a predefined person about new matches using pagers, emails and pop ups [11], [12]. While a retrospective review by Cuggia et al. [13] identified 28 different recruitment support systems described in the literature until 2008, only one of these compared the quality of the system's propositions against those of manual recruiters: Based on patient data from 3 years, Fink et al. [14] retrospectively evaluated the eligibility of 261 patients for 14 cancer trials and compared their results with the actual trial inclusions achieved by clinicians in that time period. They report a potential increase in the number of trial participants of 250%. A more recent study by Weng et al. [15] compared an SQL-based screening system for post-Acute Coronary Syndrome with two simpler methods. The specificity of the screening system was 19% compared to 8% and 9% for the latter two. Despite its low specificity, the screening tool contributed to an increase of enrolled patients of 66%.

The rationale for routine data based data acquisition support is the assumption, that some of the trial data is already available as part of the routine clinical documentation. This assumption was recently confirmed by El Fadly et al. [16] who found 13.4% of 232 data elements required for a trial on hypertension to be readily available from the electronic health record (EHR). Direct transferral of this data from the EHR into the study database would eliminate the need for redundant documentation of similar data and might thus decrease the amount of time an investigator needs to dedicate to each trial. The reuse of EHR data for data acquisition is becoming increasingly popular as Dean et al. [17] states in a review for the case of outcomes research. Though we were unable to find a similar review for clinical trials, individual reports do exist, like the point-of-care clinical trial (POCCT) on insulin therapy for diabetes [18] and the interventional STARBRITE trial on advanced heart failure [19]. The risk of poor data quality is however much more intensely discussed than the potential benefits [20], [21]. In fact, we could not find any publication that measured time-savings achieved through EHR-based data acquisition in a running trial.

We thus believe that there is a significant lack of reports on the application of secondary use in running clinical trials and its benefits in quantifiable measures. The objective of this research was to quantify the benefit of both single source measures by (1) comparing the sensitivity and specificity of manual and electronically supported patient recruitment and (2) comparing documentation times needed for manual and semi-automatic data acquisition.

Section snippets

Organisational setting

Erlangen University Hospital is a tertiary care centre with over 1300 beds situated in southern Germany. In 2011 there were over 59,000 inpatients and nearly 411,000 outpatients. The hospital offers 14 surgical and 10 non-surgical disciplines that request for anaesthesia services in 50 operating rooms localized in one central and 7 decentralized operating theatres. In a typical one week period between 500 and 600 surgeries are pursued. Erlangen University Hospital has one interdisciplinary

Effectiveness of patient recruitment

During the recruitment phase, 510 patients had a surgery with a participating anaesthetist at Erlangen University Hospital. At least one CRF was documented for 327 of these patients directly in the operating room. The DWH query identified 343 different patients as eligible for the study. Comparison of both lists showed 42 patients identified by the DWH report but not included by anaesthetists. 21 of the patients who were recruited manually by the anaesthetists did not show up on the DWH report.

Discussion

In our project two different scenarios for reusing routine data in order to support study conduct have been demonstrated within a prospective cohort study and effects of the IT interventions have been evaluated. First, a database query for eligible patients was compared to manual recruitment by anaesthetists. The result shows sensitivity (88%) and specificity (87%) of manual patient selection by medical staff to be already high. Both sensitivity (99%) and specificity (100%) of the DWH query

Conclusion

We quantitatively proved in a real, running study, that the number of enrolled patients can be increased by a DWH-supported recruitment process. Furthermore, we demonstrated, that time savings in study data acquisition can be achieved by automated reuse of data from the ICU's electronic medical record. The effects are encouraging for the application of similar methodologies to other clinical trials in the intensive care setting.

Authors’ contributions

Felix Köpcke analysed and interpreted the data, and has written the manuscript. Stefan Kraus, Axel Scholler and Thomas Ganslandt made substantial contributions to the acquisition of data. Stefan Kraus developed the PDMS-generated Word-template, whereas Thomas Ganslandt developed the data warehouse report. Carla Nau and Hans-Ulrich Prokosch involved themselves in conception and design of the study. Jürgen Schüttler, Hans-Ulrich Prokosch and Thomas Ganslandt reviewed the manuscript. Jürgen

Conflicts of interest

There is no conflict of interest.

Summary points

What was already known on the topic

  • The time needed to re-enter already existing data into study documents and insufficient recruitment of study participants impede the conduction of clinical trials.

  • Electronic systems are regularly proposed to help with both issues, though their actual benefit has not been sufficiently quantified.

What this study added to our knowledge

  • Provided the eligibility criteria match the contents of the electronic medical

Acknowledgement

This project was supported by grant 01EZ0941B from the German Federal Ministry of Education and Institutional funding of Erlangen University Hospital and Erlangen University, Germany.

References (26)

  • C. Papaconstantinou et al.

    An expert system for assigning patients into clinical trials based on Bayesian networks

    J. Med. Syst.

    (1998)
  • N. Ash et al.

    Finding appropriate clinical trials: evaluating encoded eligibility criteria with incomplete data

  • L. Li et al.

    Comparing ICD9-encoded diagnoses and NLP-processed discharge summaries for clinical trials pre-screening: a case study

  • Cited by (28)

    • Connecting healthcare and clinical research: Workflow optimizations through seamless integration of EHR, pseudonymization services and EDC systems

      2018, International Journal of Medical Informatics
      Citation Excerpt :

      Clinical trials usually consist of several stages in which routinely collected data is eligible to support e.g. trial feasibility, patient recruitment or the execution of trial documentation. Recent research has shown that data elements required for clinical trials overlap with available ones in the EHR [13–17]. Therefore, different projects have developed technical infrastructures that address single and multicenter approaches for a broad legislation compliant, syntactical and semantical interoperable solution to accompany the heterogeneous landscape of EHR systems and organizational burdens [18–20].

    • Feasibility of automatic evaluation of clinical rules in general practice

      2017, International Journal of Medical Informatics
      Citation Excerpt :

      Re-use of clinical databases is widely accepted in several research domains such as clinical epidemiology and decision analysis among others [14]. Re-use is often motivated by the fact that primary data collection for quality assurance is expensive [15]. Re-use of data nevertheless has its weaknesses such as incomplete data, inappropriate terminology for the secondary application and undocumented observations.

    • Real-time enrollment dashboard for multisite clinical trials

      2015, Contemporary Clinical Trials Communications
    • Does single-source create an added value? Evaluating the impact of introducing x4T into the clinical routine on workflow modifications, data quality and cost-benefit

      2014, International Journal of Medical Informatics
      Citation Excerpt :

      In this regard, the so-called single-source approach attempts to capture data once and reuses it again for different purposes. Theoretical frameworks [4,5] have been developed for the secondary use as well as concrete implementations within EHR systems [6–10] and data warehouses [11–13]. However, the secondary use is not unrestrictedly possible, because EHR data is not always available in a structured manner as needed to facilitate the research process [14].

    View all citing articles on Scopus
    1

    Tel.: +49 9131 85 46474.

    2

    Tel.: +49 9131 85 33680.

    3

    Tel.: +49 9131 85 33103.

    4

    Tel.: +49 9131 85 33676.

    5

    Tel.: +49 9131 85 26721.

    6

    Tel.: +49 9131 85 26753.

    View full text