Article Text

Predicting 90-day modified Rankin Scale score with discharge information in acute ischaemic stroke patients following treatment
  1. Andrew K ElHabr1,
  2. Jeffrey M Katz2,3,
  3. Jason Wang4,5,
  4. Mehrad Bastani5,
  5. Gabriela Martinez6,
  6. Michele Gribko7,
  7. Danny R Hughes8,9 and
  8. Pina Sanelli3,5
  1. 1Department of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA
  2. 2Department of Neurology, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Manhasset, New York, USA
  3. 3Department of Radiology, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, USA
  4. 4Department of Medicine, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, USA
  5. 5Center for Health Innovations and Outcomes Research, Feinstein Institute for Medical Research, Manhasset, New York, USA
  6. 6Siemens Healthineers, Malvern, Pennsylvania, USA
  7. 7Department of Neurology, North Shore University Hospital at Manhasset, Manhasset, New York, USA
  8. 8School of Economics, Georgia Institute of Technology, Atlanta, Georgia, USA
  9. 9Department of Radiology and Imaging Sciences, Emory University, Atlanta, Georgia, USA
  1. Correspondence to Andrew K ElHabr; andrewelhabr{at}gatech.edu

Abstract

Objectives To understand variability in modified Rankin Scale scores from discharge to 90 days in acute ischaemic stroke patients following treatment, and examine prediction of 90-day modified Rankin Scale score by using discharge modified Rankin Scale and discharge disposition.

Materials and methods Retrospective analysis of acute ischaemic stroke patients following treatment was performed from January 2016 to March 2020. Data collection included demographic and clinical characteristics and outcomes data (modified Rankin Scale score at discharge, 30 days and 90 days and discharge disposition). Pearson’s χ2 test assessed statistical differences in distribution of modified Rankin Scale scores at discharge, 30 days and 90 days. The predictive power of discharge modified Rankin Scale score and disposition quantified the association with 90-day outcome.

Results A total of 280 acute ischaemic stroke patients (65.4% aged ≥65 years, 47.1% female, 60.7% white) were included in the analysis. The modified Rankin Scale score significantly changed between 30 and 90 days from discharge (p<0.001) after remaining stable from discharge to 30 days (p=0.665). The positive and negative predictive values of an unfavourable long-term outcome for discharge modified Rankin Scale scores of 3–5 were 67.7% (95% CI 60.4% to 75.0%) and 82.0% (95% CI 75.1% to 88.8%), and for non-home discharge disposition were 72.4% (95% CI 64.5% to 80.2%) and 74.5% (95% CI 67.8% to 81.3%), respectively.

Conclusions Discharge modified Rankin Scale score and non-home discharge disposition are good individual predictors of 90-day modified Rankin Scale score for ischaemic stroke patients following treatment.

  • stroke

Data availability statement

The data that support the findings of this study are available from the corresponding author on reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Introduction

Stroke is the primary cause of adult-onset disability and a leading cause of death in the USA.1 Understanding the degree of long-term disability in stroke survivors is important for rehabilitation planning, estimating the cost efficiency of stroke care and determining the efficacy of acute stroke interventions and management.2–4 The modified Rankin Scale (mRS) score is the standard disability outcome measure used in both stroke patient care and clinical trials and has been found to be especially valuable when evaluated at 90 days after stroke onset.5–7

Because of the practical difficulty obtaining reliable follow-up information from stroke survivors, mRS scores are not commonly assessed after patient discharge in clinical settings. Even in research studies, long-term follow-up is often challenging and incomplete. Therefore, discharge mRS scores are often employed as a surrogate for long-term outcomes.8 One study of 261 patients in a statewide hospital-based acute stroke registry found discharge mRS to be a strong independent predictor of 90-day mRS scores.9 Another study found that discharge mRS correlates strongly with clinical scores predicting long-term adverse outcomes.10 However, both concluded that discharge mRS score is not a sufficient proxy of 90-day mRS score based on their calculations of Cohen’s weighted kappa statistics between 0.21 and 0.40, a range which has been suggested to indicate only fair agreement.11 Nevertheless, there is some evidence that 30-day mRS scores may be a good proxy for 90-day mRS score.12

Furthermore, others have used discharge disposition, which is readily available in administrative claims databases, as the primary poststroke outcome.13–16 While previous studies examining the association between discharge disposition and 90-day mRS indicate that discharge disposition is a good predictor of 90-day mRS scores, these were conducted within clinical trials.17–19 There is no research assessing the association between discharge disposition and 90-day mRS in the clinical practice setting, which is necessary to ensure findings from clinical trials are generalisable to real-world populations.

The purpose of this study was to better understand the variability of individual mRS scores from stroke hospitalisation discharge to 90 days and to assess prediction of 90-day mRS scores using discharge mRS scores and discharge disposition in a non-clinical trial cohort.

Materials and methods

Study population

A consecutive series of acute ischaemic stroke patients treated at a single comprehensive stroke centre from January 2016 to March 2020 were identified from the Get-With-The-Guidelines Stroke (GWTG-Stroke) registry. Patients meeting the following inclusion criteria were studied: aged ≥18 years; discharge diagnosis of acute ischaemic stroke; arrival to the comprehensive stroke centre within 24 hours of last known well time; and treatment with either intravenous tissue-type plasminogen activator (IV-tPA) or endovascular therapy (EVT) or both. Patients were excluded if they expired during hospitalisation; were not discharged to home, inpatient rehabilitation facility (IRF), or skilled nursing facility (SNF); or had incomplete mRS scores from the 30-day and/or 90-day follow-up periods. We only included treated patients in our analysis because this group primarily had complete follow-up mRS scores available in the GWTG-Stroke registry. The mRS assessments were performed by neurology residents who are certified in mRS scoring.

Data collection

Data were collected and managed using Research Electronic Data Capture.20 Individual patient-level demographic and clinical data was collected: age (<65, ≥65), biological sex, race (white, black, other), admission National Institutes of Health Stroke Scale (NIHSS) score (0–9, 10–19, 20–42), transfer status from primary stroke centre, last known well time to arrival (0–6 hours, 6–24 hours), stroke treatment type (IV-tPA, EVT, both), length of stay (LOS) (≤7 days, >7 days), and stroke risk factors (atrial fibrillation, coronary artery disease, diabetes mellitus, dyslipidaemia, hypertension, obesity, previous stroke and smoking history). Consistent with previous studies, discharge disposition was categorised as discharged to either home, IRF or SNF.17–19 The mRS score was adapted from the original Rankin Scale score to include levels 0 and 6, thus categorising the magnitude of disability into one of 7 mutually exclusive categories: 0 (no symptoms), 1 (no significant disability), 2 (slight disability), 3 (moderate disability), 4 (moderately severe disability), 5 (severe disability), 6 (expired).21 22

Statistical analysis

Group comparisons were conducted using Pearson’s chi-squared test for categorical variables. The mRS scores at discharge, 30 days and 90 days were categorised as: good outcome (mRS 0–2), poor outcome (mRS 3–5) and expired (mRS 6). An unfavourable outcome (mRS score of 3–6 at 90 days) was selected as the outcome of interest in the prediction modelling. Pearson’s χ2 test was performed among all combinations of the distributions of mRS scores at discharge, 30 days and 90 days. Multivariate logistic regression was used to estimate the association between all model variables and 90-day mRS (dependent variable). The least absolute shrinkage and selection operator (LASSO) logistic regression with 10-fold cross-validation and bootstrapped resamples was used to select the most important predictors of 90-day mRS. LASSO predictors were selected using the one-standard-error rule with receiver operating characteristic area under the curve as the performance metric.23 Positive and negative predictive values (PPV and NPV) and positive and negative likelihood ratios (PLR and NLR) were computed. The PPV represents the proportion of patients that experienced an unfavourable outcome according to the variable of interest, while the NPV represents the proportion of patients that experienced a favourable outcome when the variable of interest is negated. The PLR measures how likely the variable of interest predicts an unfavourable outcome, while the NLR measures how likely the negated variable of interest predicts a favourable outcome. We used a weighted generalised score statistic to compare PPVs and NPVs.24 Statistical significance was assessed as p-values less than 0.05. All analyses were performed using R version 4.0.0 (R Foundation for Statistical Computing, Vienna, Austria), and Sankey diagrams were generated using the ggalluvial package V.0.11.3.25 26

Results

A total of 816 patients with acute ischaemic stroke who received treatment were identified from the GWTG-Stroke registry during the study period. Figure 1 presents the flow diagram for patient inclusion in this study. Of the 816 ischaemic stroke patients with a recorded discharge mRS score, 10.7% (87/816) expired during the hospitalisation and were excluded from the analyses. Of the remaining stroke survivors, 3.8% (28/729) were excluded due to leaving against medical advice or were discharged to hospice or some other ambiguously labelled facility. In this exclusion group, only 25.0% (7/28) of patients had complete follow-up information—all of whom expired by 90 days. Of the remaining patients, 60.1% (421/701) had incomplete follow-up information after discharge, with 37.1% (156/421) missing both 30-day and 90-day mRS score, 61.3% (258/421) missing only 90-day mRS score, and 1.7% (7/421) missing only 30-day mRS score. The final study cohort was comprised of 280 treated acute ischaemic stroke survivors having complete mRS scores recorded at discharge, 30-day and 90-day follow-up periods.

Figure 1

Flow diagram of patient inclusion in the study cohort. mRS, modified Rankin Scale.

Table 1 outlines the demographic and clinical characteristics of the study cohort overall and stratified by discharge disposition. Overall, patients were 65.4% (183/280) aged ≥65 years (mean of 69.8 years with range of 30–96 years), 47.1% (132/280) female and 60.7% (170/280) white with 88.6% (248/280). Online supplemental table 1 details the differences in demographic and clinical characteristics and discharge mRS scores between patients with and without complete follow-up. There was no statistically significant difference in the distribution of discharge mRS scores (p=0.556) nor discharge disposition (p=0.683) in the study cohort with complete follow-up data compared with patients excluded from this study because of incomplete follow-up data.

Supplemental material

Table 1

Demographic and clinical characteristics and outcomes of stroke survivors who received treatment stratified by discharge disposition

Among all patients, 43.6% (122/280) were discharged home, 41.4% (116/280) to IRF, and 15.0% (42/280) to SNF. When compared with those discharged to IRF or SNF, those who were discharged home had the highest proportion of admission NIHSS scores between 0 and 9 (67.2% (82/122), p<0.001), LOS≤7 days (91.0% (111/122), p<0.001) and IV-tPA (91.8% (112/122), p=0.002). Those discharged to IRF, compared with the other two groups, had significantly more male patients (65.5% (76/116), p<0.001), transfer patients from a primary stroke centre (60.3% (70/116), p=0.007) and EVT patients (56.9% (66/116), p<0.001). Those discharged to SNF had significantly higher proportion of patients≥65 years (88.1% (37/42), p=0.003) and the highest proportions of stroke risk factors, including atrial fibrillation (42.9% (18/42), p<0.001) and previous stroke (19.0% (8/42), p=0.038), with the lowest proportion of patients arriving within 6 hours of last known well time (76.2% (32/42), p=0.023). There were no significant differences among the discharge disposition groups according to race, coronary artery disease, diabetes mellitus, dyslipidaemia, hypertension, obesity or smoking.

Table 1 also shows the discharge, 30-day and 90-day mRS scores of stroke survivors for the study cohort and stratified by discharge disposition. For each time period, patients discharged to home had the highest percentage of mRS scores 0, 1 and 2, IRF had the highest percentage of patients with mRS score 3, and SNF had the highest percentage of patients with mRS scores 4, 5 and 6 (p<0.001), with the exception that the mRS scores of 2 and 4 at 90 days being highest proportionally for those discharged to IRF.

Table 2 compares the distribution of mRS scores at different intervals (discharge, 30-day and 90-day follow-up) for the study cohort and stratified by discharge disposition. The distribution of discharge and 30-day mRS scores were not significantly difference for the overall cohort (p=0.665) or when stratified by discharge disposition (Home (p=1); IRF (p=1); SNF (p=0.637)). However, there were significant differences in the distribution of mRS scores when comparing 90-day mRS to either discharge or 30-day mRS for the overall cohort and when stratified by discharge dispositions of home (discharge vs 90 days (p<0.001); 30 days vs 90 days (p<0.001)) and SNF (discharge vs 90 days (p<0.001), 30 days vs 90 days (p=0.026)). The significant variability in mRS scores over time are more explicitly seen when the paths of mRS scores are drawn out from discharge to 30 days to 90 days with a Sankey diagram (figure 2A). Despite the lack of statistically significant difference between discharge and 90-day mRS score when stratified for patients discharged to IRF (p=0.149), 13.8% (16/116) of patients worsened from the mRS 0–2 group at discharge to the mRS 3–6 group at 90 days and 21.6% (25/116) of patients improved from the mRS 3–5 group at discharge to the mRS 0–2 group at 90 days (figure 2B).

Figure 2

(A) Sankey diagram demonstrating the variability in mRS scores from discharge to 30 days to 90 days for the overall stroke survivor cohort (n=280). (B) Sankey diagram demonstrating the variability in the mRS scores from discharge to 30 days to 90 days for stroke survivors stratified by discharge disposition. The mRS scores are categorised as good outcome (mRS 0–2), poor outcome (mRS 3–5) and death (mRS 6). IRF, inpatient rehabilitation facility; mRS, modified Rankin Scale; SNF, skilled nursing facility.

Table 2

Comparison of mRS scores between interval time periods stratified by discharge disposition

The multivariate logistic regression results are reported in table 3. Age ≥65 years (OR: 2.27, p=0.044), males (OR: 0.36, p=0.009), diabetes mellitus (OR: 3.35, p=0.010), discharge mRS of 3–5 (OR: 3.13, p=0.005), discharge disposition of home (OR: 0.32, p=0.011) and LOS>7 days (OR: 3.72, p=0.002) were found to be significantly associated with mRS scores of 3–6 at 90 days. When performing variable selection with LASSO, only age ≥65 years (OR: 2.88, p=0.002), discharge mRS of 3–5 (OR: 2.76, p=0.005), discharge disposition (home (OR: 0.44, p=0.037), SNF (OR: 3.62, p=0.005)), and LOS>7 days (OR: 2.99, p=0.002) were selected.

Table 3

Logistic regression analyses predicting 90-day mRS score for the two model versions using all variables and selected variables

Table 4 displays the predictive values for the discharge variables selected by the LASSO model and the overall predictions made by this model for observing mRS scores of 3–6 at 90 days. Because the discharge disposition of home is more closely correlated with a favourable outcome than an unfavourable one, we used the combination of the two non-home sites, that is, IRF/SNF, as a single discharge disposition. Of the patients that the selected variables model predicted to have an unfavorable outcome (90-day mRS 3-6), 74.4% (95% CI 67.0% to 81.8%) are predicted correctly (PPV); and of the patients that this model predicted to have a favourable outcome (90-day mRS 0–2), 79.6% (95% CI 73.1% to 86.1%) are predicted correctly (NPV). This PPV is significantly greater than the PPV of discharge disposition IRF/SNF (p=0.002), but not significantly different than that of discharge mRS 3–5 (p=0.318). Similarly, the model’s NPV is significantly greater than the NPV of discharge mRS 3–5 (p=0.045), but not significantly different than that of discharge disposition IRF/SNF (p=0.191). The PPV, NPV, PLR and NLR values for all discharge variables are in online supplemental table 2.

Table 4

Positive and negative predictive values for selected discharge variables to predict unfavourable outcome (mRS 3–6) at the 90-day follow-up period

Discussion

This study demonstrates the use of employing both discharge mRS and discharge disposition for predicting 90-day mRS scores, the standard measure of long-term functional outcomes in acute ischaemic stroke patients. It is important to understand the predictive value of these variables because both discharge mRS and discharge disposition are readily available in clinical and administrative datasets enabling widespread study of stroke outcomes following treatment outside the context of clinical trials in real-world practice.

The first part of our analysis showed that although there were minimal changes in patient mRS scores between discharge and 30 days, there were considerable differences between 30 and 90 days. This finding aligns with the observation that there is only modest agreement between discharge and 90-day mRS in clinical settings.9 10 Even though the changes in mRS scores between 30 days and 90 days are statistically significant (table 2), we find that figure 2 is even more informative on this matter. When we study these numbers at the individual patient level, we find that 98.6% (276/280) of patients maintained the exact same functional status between discharge and 30 days, while this was only true for 31.8% (89/280) of patients between 30 and 90 days. Moreover, the functional status of 33.2% (93/280) and 35.0% (98/280) of patients improved and worsened, respectively, between 30 and 90 days. This observation is upheld to some extent when the mRS scores are stratified by discharge disposition. Since most improvement from rehabilitation has been reported within the first 90 days after stroke, we would not expect significant changes observed in mRS scores between discharge and 90 days to continue.27 28 Because there was almost no difference between mRS scores at discharge and 30 days in our study, and discharge scores are more likely to exist in comparable studies, we used discharge mRS scores for the rest of the analysis.

Although the previous observations indicate that there is a descriptive statistical difference between mRS scores at discharge and 90 days, there may be a use for the intermediate outcomes of discharge disposition and discharge mRS to predict 90-day mRS score. Thus, we investigated discharge disposition and mRS score, both individually, and in combination with additional factors to predict 90-day mRS score. First, instead of using all possible variables to build a final predictive model of 90-day mRS score, we used LASSO to select a subset of these variables with the idea that LASSO will select discharge disposition and discharge mRS if they are deemed to have better predictive capabilities than other available variables. In our analysis, discharge disposition and discharge mRS were selected along with age and LOS.

In table 4, we found that the selected variable model significantly outperformed discharge disposition and discharge mRS in one of the two predictive metrics (PPV or NPV), but not both. Specifically, we found no significant difference between the model and discharge disposition when predicting a favourable 90-day mRS score, and no significant difference between the model and discharge mRS when predicting an unfavourable 90-day mRS score. These results provide more evidence that discharge disposition and discharge mRS are good individual predictors of 90-day mRS score, even outside the context of clinical trials.17–19 Given that we observed almost no difference between discharge and 30-day mRS in our study, our finding that discharge mRS is a good individual predictor of 90-day mRS is in accordance with a previously published study of 5997 stroke patients aggregated from numerous clinical stroke trials between 1998 and 2006 which found that the 30-day mRS alone reliably predicted the 90-day mRS.12 We also found that supplementing discharge information with patient age and LOS can significantly enhance the prediction of 90-day mRS scores, similar to what has been previously suggested in clinical trials.18 19

When comparing the predictive metrics of discharge disposition for 90-day mRS score with similar studies using data gathered for clinical trials that found discharge disposition to be a strong independent predictor of 90-day mRS, our findings were generally similar. Our estimated 95% CIs for the PLRs and NLRs for each discharge disposition of predicting an unfavourable 90-day mRS (online supplemental table 2) overlapped with those of a recently published study analysing clinical trial data.19 We make similar observations when comparing our study to a separate study, except that the PLR for IRF was significantly higher and NLR for IRF was significantly lower in their study when compared with ours, based on non-overlapping confidence intervals of these PLR and NLR estimates.17 This indicates that the patients in our study were more likely to have a favourable mRS score at 90 days when discharged to IRF. Although it is difficult to tell whether this finding is a result of varying patient cohorts, hospital discharge decisions or rehabilitation results, it serves as further evidence of the importance of evaluating the predictive value of discharge disposition and discharge mRS specifically within clinical practice.

There were a few notable limitations to this study. First, the reasons for the lack of follow-up for excluded patients were unknown. This potential bias can work both ways. For those with a favourable mRS score at discharge, it may be assumed that they were still doing well months later. Conversely, it is possible that a patient with a favourable outcome at discharge regressed quickly and expired, and it was difficult to find a relative to contact for follow-up, or the patient switched healthcare providers. Although there were a few significant differences in the presenting characteristics between patients with and without follow-up, we highlight that there were no significant differences in the discharge disposition nor discharge mRS (online supplemental table 1), which supports the generalisability of our results to a population of treated ischaemic stroke patients in clinical practice. The only significant differences between these two groups were the proportion of atrial fibrillation, transfer status, and the specific type of treatment utilisation (IV-tPA, EVT and both). We repeated the analysis to include patients without follow-up while using discharge mRS as the outcome, and none of the aforementioned variables were found to be significantly associated with discharge mRS.

Another limitation is our small sample size. Our concern about the available sample size is one reason why we chose to use the LASSO variable selection procedure, so as to increase the power of our model. Also, last known well time to arrival and transfer status were not used in the main logistic regression analysis because there were too many missing observations. This time window is important for treatment, and a larger and more complete sample size may be helpful in further evaluating the predictive value of treatment for 90-day mRS.

In conclusion, we found that discharge mRS scores remained stable for the first month after discharge, regardless of the discharge location. However, between 30 days and 90 days postdischarge, mRS scores changed significantly in 2/3 of patients (approximately 1/3 improved, 1/3 deteriorated and 1/3 unchanged). Also, the discharge mRS score and discharge disposition of home by themselves do not perform significantly worse than a model combining this information along with age and LOS for predicting 90-day mRS in acute ischaemic stroke patients following treatment. In particular, discharge mRS 3–5 predicted unfavourable outcomes (90-day mRS 3–6) while discharge disposition to home predicted favourable outcomes (90-day mRS 0–2) similar to the model. Thus, if a proxy for 90-day mRS must be used, it may be sufficient to use one of these two discharge variables by themselves. Although generalising these results to only the treated population may seem limiting, we know that the number of ischaemic stroke patients arriving within the treatment window for IV-tPA has increased over time and that approximately three-fourths of these patients are able to receive treatment.29 Also, with extension of the EVT treatment eligibility time window up to 24 hours, the number of hospitals performing EVT has increased over time, even in lower annual EVT volume hospitals.30 In the future, similar analyses may be performed with administrative claims data to determine if an improved model can be developed with more observations and data sourced from multiple locations. If an improved model is developed with this type of data, we may better understand the impact of stroke intervention and resource utilisation on long-term functional outcomes with a much larger cohort than has previously been studied.

Data availability statement

The data that support the findings of this study are available from the corresponding author on reasonable request.

Ethics statements

Ethics approval

Feinstein Institutes for Medical Research (Northwell Health) Institutional Review Board number HS16-0480 approval was obtained with a waiver of consent due to the retrospective nature of the study.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors AKE, DRH and PS contributed to the study conception and design. AKE performed material preparation, data collection and analyses. AKE wrote the initial draft of the manuscript, and all authors commented on previous versions of the manuscript with considerable feedback coming from JMK, DRH and PS. All authors read and approved the final manuscript.

  • Funding This work was supported by National Institutes of Health grant number R56NS114275.

  • Competing interests GM is a Siemens Healthineers employee and shareholder.

  • Provenance and peer review Not commissioned; internally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.