FormalPara Take-home message

There are significant center-specific differences in outcomes after out-of-hospital cardiac arrest after adjusting for patient-specific factors. These differences are partially explained by in-hospital treatment decisions.

Introduction

Functional outcomes of patients who survive out-of-hospital cardiac arrest (OHCA) and receive in-hospital care are determined in part by their underlying health status and arrest-specific factors, but many aspects of medical care after cardiac arrest may influence outcomes as well [1,2,3,4,5]. Reporting of cardiac arrest outcomes specific to individual hospitals is increasing, but at this time there is no risk-adjustment standard to benchmark hospital performance [6]. Post-resuscitation care varies widely between centers, including many practices associated with outcome such as targeted temperature management (TTM), utilization of coronary angiography and percutaneous coronary intervention (PCI), mechanical circulatory support, glucose control, oxygenation and ventilation practices, blood pressure management, sedation regimes, and prognostication practices [4, 5, 7,8,9]. Some of these management strategies align with the volume of patients cared for at a given center [9,10,11]. Given these inconsistencies and the medical complexity of these high-risk patients and the need for urgent triage, an improved understanding of which in-hospital treatment options and interventional strategies may affect outcomes is needed. The ability to risk-adjust overall outcome by center is an important first step, enabling identification of modifiable differences between management strategies.

Patient- and arrest-specific risk factors for poor outcome following cardiac arrest have been described [12,13,14,15,16,17], but few studies have reported center outcomes adjusted for risk using patient-level data [7, 8] or identified in-hospital factors that may explain variation between centers. We sought to develop a risk-adjustment model to evaluate between-center effects of functional outcome at hospital discharge in patients with OHCA who received TTM as an initial step to identify potential in-hospital treatment variation that might explain such differences. We also explored variation in various treatment modalities and diagnostic tests that may potentially explain some of the differences in outcomes between high- and low-performing centers [18].

Methods

Data source

The International Cardiac Arrest Registry (INTCAR) is a multicenter, international database of US and European centers including both in-hospital and out-of-hospital cardiac arrest patients. The registry began enrolling patients in 2006 and as of November 2017 it included 6010 patients from 42 hospitals. Centers enrolled consecutive adult patients admitted to an intensive care unit after cardiac arrest. Management of patients was at the discretion of the treating center, according to local best practices. Centers participated in the registry on a voluntary basis, there was no reimbursement for enrolling patients, and all had institutional review board approval at their center. We included patients from INTCAR with OHCA enrolled between the years 2006 and 2017, and excluded centers that enrolled less than 25 patients. INTCAR consists of two sequential and non-overlapping iterations: a 1.0 data set (years of 2006–2011) and a 2.0 data set (years 2011–2017); we combined these data sets and included variables found in both. INTCAR data encompassed the Utstein data points [19] as well as many in-hospital variables related to post-resuscitation care [14]. Although centers enrolled consecutive adult patients with both in-hospital (IHCA) and out-of-hospital cardiac arrest (OHCA), only patients with OHCA were included in this analysis.

Outcome

The primary outcome was the Cerebral Performance Category (CPC) score at hospital discharge. Consistent with previous reporting in large clinical trials, CPC was dichotomized into good outcome (normal to moderate cerebral disability: CPC 1–2) and poor outcome (severe cerebral disability to brain death: CPC 3–5) [2, 20, 21]. The time point of hospital discharge was chosen because longer-term outcome is influenced by factors other than hospital care, including post-discharge services, insurance status, and various comorbidities, which were not recorded in the registry [22,23,24]. Secondary outcome was delayed CPC which is typically determined at 6 months either by review of medical records or a telephone call.

Predictors

Candidate variables from both the 1.0 and 2.0 database included age, sex, pre-arrest CPC, past medical history [composite endpoint of chronic obstructive pulmonary disease (COPD), coronary artery disease (CAD), arrhythmia, congestive heart failure (CHF), hypertension, chronic kidney disease, liver disease, obesity, malignancy, renal disease, non-insulin-dependent diabetes mellitus (NIDDM), insulin-dependent diabetes mellitus (IDDM)], initial rhythm (shockable versus non-shockable), time to return of spontaneous circulation (ROSC) (including both no-flow and low-flow time), bystander cardiopulmonary resuscitation (CPR), witnessed arrest, and defibrillation.

In-hospital factors

In-hospital factors available between the two data sets included several temperature-related events, including target temperature (32–34 °C, 35–36 °C, 37 °C or greater), time to initiation of target temperature, and post-temperature management fever. The utilization of cardiac interventions and hemodynamic support were analyzed, including cardiac catheterization, percutaneous intervention, and coronary artery bypass grafting occurring while the patient remained unconscious, while they were awake, or not performed (analyzed for all patients, patients with known ST-elevation myocardial infarction on ECG, and those with a shockable rhythm), thrombolysis, and intra-aortic balloon pump use. Utilization of diagnostic tests used to guide neurologic care and prognostication were evaluated, including use of electroencephalogram (EEG), continuous electroencephalogram (cEEG), magnetic resonance imaging (MRI), computed tomography (CT), and somatosensory evoked potential (SSEP). Early withdrawal of life support was evaluated as patients who had both withdrawal of life-sustaining therapies and an ICU length of stay of 3 days or less.

Missing data

The effect of missing data was assessed with each explanatory and outcome variable. The distribution of the model variables was compared between patients with complete and incomplete data to verify that the population of patients with missing data was similar.

Statistical analysis

Continuous variables were assessed for linearity of response on outcome and categorized if needed because of nonlinearity. The relationship of candidate variables with outcome was initially assessed in a univariate manner using logistic regression; these were retained in the model if the p value was less than 0.20. The decision was made a priori to force three selected variables into the model (time to ROSC, age, initial shockable rhythm), regardless of statistical significance, on the basis of prior evidence suggesting significant prognostic value [25,26,27,28]. A hierarchical logistic regression model for good outcome as a function of patient demographic and clinical variables was created with a random center-specific effect. Performance was assessed using area under the receiver operator characteristic (ROC) curve and likelihood ratio tests to predict good outcome. The model was then used to calculate risk-standardized good functional outcome rates based on “Method 3” described in a comparison of national risk adjustment [29]. This was done by first finding the predicted outcome of each patient within each center (predicted outcome) and then measuring the expected rate of outcome at each facility, given the predicted probability for outcome for patients at that center (expected outcome). The risk-adjusted ratio was calculated as the registry average outcome multiplied by the ratio of observed and expected outcomes. This approach allows for control of clustering among the 25 centers by calculating a center-specific intercept within the model. Risk-standardized mortality rates were then calculated as the observed rate divided by the expected rate at each center where the expected rate is the predicted rate from the hierarchical logistic regression model substituting a null center effect. Thus the risk-standardized rate using this approach allows for adjustment based on patient mix for each center and simultaneously allows for shrinkage due to center clustering. This methodology of risk adjustment increases content validity compared to classic logistic regression-based modeling and has higher convergent validity compared to shrinkage estimator-based risk adjustment [29]. The analysis was repeated using the subgroup of patient who met Utstein comparator criteria (shockable rhythm, received bystander CPR, and arrest was witnessed).

We then evaluated high- and low-performing centers defined as risk-standardized ratio confidence intervals significantly above or below the registry average, respectively. These were pooled into high and low groups and in-center resource utilization was compared. Factors found to be statistically significant between high- and low-performing groups were then added into the full model and evaluated for improvement in model performance using ROC curves and evaluation of Akaike information criterion.

Results

Patient population

The INTCAR data included 6010 patients from 42 centers and 4544 patients had OHCA. A total of 3855 patients from 25 centers had complete data and enrolled at least 25 patients (Table 1). The average age of this study population was 61 years (± 15 years), 31% were female, 53% of patients had an initial shockable rhythm, the average time to ROSC was 26 (± 18) min, and 34% achieved good functional outcome at hospital discharge. Influence of individual components of past medical history are shown in Supplementary Table 1.

Table 1 Patient characteristics
Full size table

Missing data

There were 420 patients with incomplete data. The variable most missing was time to ROSC, absent in 6% of cases. The second most often missing variable was outcome at hospital discharge, absent in 1.6% of cases. A sensitivity analysis was performed to compare patients with and without missing data, which revealed similar age (60 ± 17 years vs. 61 ± 15 years, p = 0.09) and time to ROSC (26 ± 24 min vs. 26 ± 18 min, p = 0.76). There was a difference in incidence of initial shockable rhythm (46% vs. 53%, p = 0.007) for missing and nonmissing data, respectively. Within the group of patients with missing data, 253 (60%) were missing the variable for ROSC and 288 (67%) were unwitnessed. The outcome of good CPC at hospital discharge was 30% for patients with missing data and 34% for patients without missing data (p = 0.21). Multiple imputation was performed with similar results in the multivariable model, with the exception of age (Supplementary Table 2).

Model development

Univariate and multivariate analyses of all candidate variables are shown in Table 2. Linearity with outcome was assessed for age and time to ROSC. The relationship between time to ROSC and the primary outcome was found to be nonlinear and was therefore categorized by 5-min intervals and referenced to the largest subgroup (15–20 min). Age, sex, number of medical diagnoses, initial rhythm, time to ROSC, witnessed arrest, bystander CPR, and defibrillation were found to be statistically significant predictors of outcome and were retained in the model.

Table 2 Full univariate and multivariate model for outcome of dichotomized hospital discharge CPC
Full size table

Outcome by center after risk adjustment

The unadjusted frequency of a good functional outcome at hospital discharge ranged from 11% to 63% with a center-mean of 39% among the 25 centers. The risk-standardized outcome rate ranged from 20% (CI 12–27%) to 50% (CI 39–61%). The distribution following adjustment is shown in Fig. 1. When limited to centers with confidence intervals that did not overlap the registry average, four high-performing centers were significantly above average, with a range of 40–50% of risk-adjusted good outcome by center, and an average for the group of 44%. Similarly, five low-performing centers were significantly below the average, with a range of 20–27% risk-adjusted good outcome by center and an average for the group of 24% (Fig. 1). Observed, predicted, and expected rates of good outcome at hospital discharge and values of risk-adjusted ratio are shown in Supplementary Table 3.

Fig. 1
figure 1

Good outcome at hospital discharge by center after risk adjustment

Full size image

For the secondary outcome of delayed CPC at the registry average of 6 months, the unadjusted frequency of good functional outcome at an average of 6 months ranged from 0% to 54% with a center-mean of 35% among the 25 centers. The risk-standardized outcome rate ranged from 0% (CI 0–13%) to 54% (CI 42–65%). Following adjustment, four centers performed significantly better than the registry average and five centers performed significantly worse than the registry average (Fig. 2). Observed, predicted, and expected rates of good outcome at 6 months and values of risk-adjusted ratio are shown in Supplementary Table 4.

Fig. 2
figure 2

Good outcome at follow-up by center after risk adjustment

Full size image

Characteristics of high- versus low-performing centers

Patient characteristics and in-hospital resource utilization were compared between high- and low-performing centers (Table 3). Treatment variables that were significantly different between high- and low-performing centers included time to start of target temperature, TTM target goal, use of cardiac catheterization and PCI while patients were unconscious (for patients with STEMI, shockable rhythm, and all patients), and use of thrombolysis. Of patients with a poor outcome, the use of prognostication variables differed with the use of continuous EEG, MRI, and SSEP (Supplementary Table 5). Withdrawal of life-sustaining therapies within the first 3 days was more common in the high-performing centers (194 patients (23%) versus 181 patients (15%), p < 0.001). There was no difference between high- and low-performing centers in the incidence of fever in the first 72 h, diagnosis of pneumonia, use of intra-aortic balloon pump, or treatment by coronary artery bypass grafting (CABG). Adding these significant variables to the model resulted in only a modest improvement AUC (0.84–0.89) and a lower Akaike information criterion, suggesting these treatment factors only modestly improve model performance (Fig. 2).

Table 3 Characteristics of four high-performing centers and five low-performing centers
Full size table

Utstein comparator subgroup

A total of 1296 patients (25% of cohort) met criteria of having a shockable rhythm, receiving bystander CPR, and having a witnessed arrest. The incidence of good outcome was higher than in the full cohort, at 57.3%. High- and low-performing centers were similar (in the low-performing group, one was no longer in that group and another was included that otherwise would not have been and in the high-performing group, one hospital was not included in the Utstein group).

Discussion

In a large international registry of patients treated with TTM after OHCA, profound differences in center-specific rates of good functional outcomes were observed and persisted after adjustment for the major patient-specific factors known to be associated with outcome. The four high-performing centers reported a greater use of temperature management goal of 33 °C, faster time to initiation of target temperature, and higher rates early cardiac catheterization and PCI (prior to awakening). There was also a higher utilization of continuous EEG and SSEP than the low-performing centers. These post-resuscitation processes of care include treatments that could influence outcome, such as TTM performance and PCI as well as diagnostic modalities (EEG, SSEP) that may be markers for other elements of care such as a more nuanced approach to neurologic prognostication that incorporates multimodal diagnostics [30, 31]. The center-specific differences in outcome were not fully explained by the treatment factors we evaluated, suggesting others contribute, possibly including rewarming rate, hemodynamic management, oxygenation and ventilation parameters, and glucose management, all of which have been shown to be independently associated with outcome in prior studies [5, 32], but were not available in our data. Moreover, direct prognostication data related to withdrawal of life-sustaining therapy were also not available and may greatly impact outcomes [33,34,35,36]. We did find that there was a difference in withdrawal of life-sustaining therapy and ICU length of stay of 3 days or less, suggesting that early prognostication practices differed between high- and low-performing centers. Center volume, although associated with outcome in the univariate model, was not associated with outcome in the multivariable model and is inconsistent with other publications of high volume centers having more favorable overall outcomes [9, 10]. The reason for this may be due to lack of statistical power to detect this effect. Our data suggest that there are center effects influencing OHCA outcomes. Reporting such severity-adjusted data may ultimately help identify key features of high-quality post-resuscitation care, and define standards for assessing hospital performance.

Our data agree with previous studies showing differences in cardiac arrest outcomes by center after various types of adjustment. Merchant et al. evaluated 135,896 in-hospital arrests from the American Heart Association’s Get With the Guidelines-Resuscitation Registry adjusted for 36 predictors of outcome and found that adjusted in-hospital survival rates ranged from 12.4% to 22.7% at different centers; however, they included patients that did not achieve ROSC and others treated without TTM [7]. Carr et al. [8] evaluated a multicenter clinical registry of ICU patients and found that in-hospital mortality ranged from 46% to 68% between the 39 centers after adjusting for age, severity of illness, and ventilation status. Our demonstration of variability in outcomes between centers after adjusting for the case mix is consistent with these findings in a different population. Our methodology of risk adjustment decreased the likelihood of overestimating center differences, which is a frequent error in random center effects modeling [29, 37].

The variations we observed in risk-standardized outcomes suggest that center-specific characteristics, either in terms of resources, protocols, or practices, may directly affect functional outcomes after cardiac arrest. These variations in outcome represent an opportunity to identify which treatment factors, from the many identified as candidates, most affect outcome. We identified several that appear to be important: time to initiation target temperature, early cardiac catheterization, and early PCI. Unfortunately, limitations in the data set precluded analysis of hemodynamic management, ventilation and oxygenation parameters, glucose control, or how prognostication testing was interpreted including withdrawal of life-sustaining therapies policy. Other post-resuscitation treatments such as sedation and shivering management [1] and seizure management [38] have also been shown to vary by center and may contribute to outcomes. Our study did not have patient-level data for specific aspects of some of these treatments including sedation and shivering data, seizure management, and how prognostic testing was interpreted. Evaluating these factors in future studies may further improve our model. The differences in outcome associated with use of prognostication tests is likely more complex than the mere presence or absence of these tools; they could be a marker for neurologist or neurointensivist involvement, and could relate to which patients receive that testing and how the data are used to guide care, such as the early withdrawal of life support. Similar challenges have been identified in other multicenter practice studies in other disease states [39], where an in-depth communication and quality improvement effort was initiated with an improvement in outcomes [39]. This could be used as a platform for process-improvement in centers that provide post-resuscitation care.

Study strengths and limitations

The strengths of this study include the benefits of a large international data set, which allows comparisons between centers. We also used a method of risk adjustment that captures and corrects for differences in center size as recommended by Centers for Medicare and Medicaid Services [29]. INTCAR also allowed us to evaluate some in-hospital factors to help understand some of the clinical differences between high- and low-performing centers.

This study should be interpreted within the context of several limitations. Although data dictionaries were developed to reduce variability in data entry and the registry guidelines were to enroll consecutive patients, sites were responsible for internally monitoring the quality of their data entry. We also found that patients with nonshockable rhythm were more likely to be unwitnessed. We believe this explains why there are fewer missing data among patients with shockable rhythm. This did not appear to cluster at any particular hospital. Analysis after multiple imputation showed similar odds ratios, with the exception of age, which was significant in nonimputed data and nonsignificant in the imputed data set. Limiting our analysis to data points that were concordant between the 1.0 and 2.0 data restricted our analysis and there were some variables that were not available in both data sets that may have been useful, including etiology of arrest. The ability to further understand differences in care between high- and low-performing centers would benefit from in-depth interviews and a review of full protocols and adherence to those protocols to identify themes that may explain the variability in outcomes. Also, because centers participated in the registry at different time points, we were not able to evaluate patient volume, which has been associated with improved outcomes [10, 11, 30]. Lastly, hospital discharge CPC was the outcome of interest rather than 6 month CPC. Since longer-term outcome is influenced by other factors including post-discharge services, insurance status, and other comorbidities, we felt that restricting the outcome to hospital discharge was the most appropriate for addressing our research question.

This is the first study of its kind that introduces an accessible risk-adjustment model for comparing center performance based on patient-level data for patients admitted with OHCA. Outcome differences for these patients are not solely explained by differences in patient case mix, but also represent variations in patient care, which are often unmeasured. The next steps of comparing processes across centers would be to attempt to uncover root causes of systematic differences among centers including sedation, shivering management, metabolic management, applications prognostic tests, hemodynamic and ventilator targets, and seizure management. Nonetheless, it is of interest that in an era where some are now questioning the utility of post cardiac arrest use of therapeutic hypothermia and early coronary angiography, these results from a large post cardiac arrest registry affirm their value in high-performing centers [40].

Conclusions

Considerable variability persists between centers in functional outcome among patients after OHCA at hospital discharge despite adjustment for baseline risk. High-performing centers more frequently have a faster time to target temperature, provide cardiac catheterization and PCI prior to awakening, and are more likely to utilize continuous EEG and SSEP compared to low-performing centers, but these differences only partially explain the differences in outcomes noted. This model provides an opportunity to explore difference in care delivery and potentially improve processes of care. Additional work is needed to establish normative standards for good outcomes after resuscitation from out-of-hospital cardiac arrest based on risk adjustment, and to fairly assess hospital performance and investigate the specific features of post-resuscitation care that directly influence patient outcomes.