Inflated Estimates of Proportional Recovery From Stroke: The Dangers of Mathematical Coupling and Compression to Ceiling

Abstract

The proportional recovery rule states that most survivors recover a fixed proportion (≈70%) of lost function after stroke. A strong (negative) correlation between the initial score and subsequent change (outcome minus initial; ie, recovery) is interpreted as empirical support for the proportional recovery rule. However, this rule has recently been critiqued, with a central observation being that the correlation of initial scores with change over time is confounded in the situations in which it is typically assessed. This critique has prompted reassessments of patients' behavioral trajectory following stroke in 2 prominent papers. The first of these, by van der Vliet et al presented an impressive modeling of upper limb deficits following stroke, which avoided the confounded correlation of initial scores with change. The second by Kundert et al reassessed the value of the proportional recovery rule, as classically formulated as the correlation between initial scores and change. They argued that while effective prediction of recovery trajectories of individual patients is not supported by the available evidence, group-level inferences about the existence of proportional recovery are reliable. In this article, we respond to the van der Vliet and Kundert papers by distilling the essence of the argument for why the classic assessment of proportional recovery is confounded. In this respect, we reemphasize the role of mathematical coupling and compression to ceiling in the confounded nature of the correlation of initial scores with change. We further argue that this confound will be present for both individual-level and group-level inference. We then focus on the difficulties that can arise from ceiling effects, even when initial scores are not being correlated with change/recovery. We conclude by emphasizing the need for new techniques to analyze recovery after stroke that are not confounded in the ways highlighted here.

Keywords: biomarkers; prognosis; recovery of function; statistics; stroke; upper extremity.

Fig 1

Illustration of compression enhanced coupling: typical pattern of recovery data, reproduced three times, each panel highlighting a different patient (1, 2 & 3 in panels A, B & C respectively). Data was generated from the unconstrained standard-form regression model in Bonkhoff et al (2020) (Slope = 1.24, intercept = 17, i.e. Y = 1.24.X + 17). Outcome scores were generated from 400 randomly sampled initial scores, with Gaussian noise (SD 10) added, and values above 66 set to zero. Fugl-Meyer scores initially (X) and at outcome (Y) are shown in each panel, with non-fitters (0 to 10) removed, and the room available (i.e. potential) for recovery highlighted on the right side of each patient plot. The critical correlation that is tested is r(X,(Y-X)), with extreme negative values indicating proportional recovery. A key property that has to hold for r(X,(Y-X)) to be negative is that smaller values of X (lower on axis) should correspond to larger values of Y-X. The panels here illustrate why that cannot fail but be true, assuming the data has two properties: a) a ceiling and b) a general trend to get better. Both of these are true in this figure and invariably. Simply stated, if X is small, as it is for patient 1 (i.e. x1), there is room for recovery (i.e. y1-x1 can be large), but if X is large, as it is for patient 3 (i.e. x3), there is very little room for recovery (i.e. y3-x3 must be small). That is, ceiling prevents patients with high initial-scores from being able to demonstrate their improvement: even if they have actually recovered capacities that place them well above the “Normal” level under Fugl-Meyer, their measured recovery will be bounded by ceiling, and thus small. In this way, a strong negative correlation would be observed, see panel D, and a proportional recovery pattern would be identified. This correlation would be due to compression to ceiling and may have little, if any, relationship to the true underlying recovery pattern in the data.

Fig 2

proportional recovery versus ceiling, as presented in Kundert et al (2019) . Performance is measured using the Fugl-Meyer Upper Extremity scale. To stay consistent with Kundert et al (2019) , data in both panels is plotted differently from elsewhere in this article. X-axes show initial impairment, i.e. maximum minus performance-score (66-X), while Y-axes show change, i.e. recovery (Y-X). In this form, ceiling is the filled black diagonal line and lines of best fit are in dashed black. [A] Canonical proportional recovery pattern simulated by Kundert et al (2019) . [B] Observed pattern of human data from Winters et al (2015) , showing the compression to ceiling that can generate spurious evidence for proportional recovery. Data points in bottom right corner (overlaid by dashed shape with grey fill) are non-fitters and are not relevant to the current discussion. Panel [A] is adapted from figure 1 panel a of Kundert et al (2019) , with the permission of the publisher, copyright © 2019, Sage. Panel [B] is adapted from figure 2 of Winters et al (2015) , with the permission of the publisher, copyright © 2015, Sage.

Fig 3

Ceiling effects in van der Vliet et al (2020) : Fig 1(A) (our panel [a]) and Fig 2(H,I) (our panels [b,c]) from van der Vliet et al (2020) . [a] Representation of data, with five groups identified by Bayesian mixture modelling, shown in different colours, and curves showing model fits to exponentials. Patients’ performance is measured on the FM-UE scale. These five groups were combined into three, which are indicated by brackets and named Good, Moderate and Poor (note, colour coding changes from original groups). [b,c] Accuracy of fits. Critically, the Good group shows a substantial ceiling effect, while Moderate does not. Consistent with this, posterior predictive value (panel [b]) and misses (panel [c]) are close to perfect for Good, but much less for Moderate. Panel [a] is adapted from figure 1 panel A of van der Vliet et al (2020) and panels [b & c] are adapted from figure 2 panels [H & I] of van der Vliet et al (2020) , with the permission of the publisher, copyright © 2020, Wiley.