Tag: external validation

Posted on

A clinical prediction model to assess surgical outcome in patients with cervical spondylotic myelopathy

A clinical prediction model to assess surgical outcome in patients with cervical spondylotic myelopathy

The Spine Journal 15 (2015) 388–397

Clinical prediction rules are valuable tools in a surgical setting but should not be used to guide clinical practice until validated in other populations.

PURPOSE: The objective of this study was to validate a clinical prediction rule developed to determine surgical outcome in patients with cervical spondylotic myelopathy (CSM). The study will also identify key clinical predictors of outcome at a global level.

STUDY DESIGN/SETTING: This is a prospective multicenter cohort study.

PATIENT SAMPLE: Two-hundred seventy-eight and 479 surgical CSM patients enrolled in the AOSpine CSM—North American (CSM-NA) and CSM—International (CSM-I) studies, respectively.

OUTCOME MEASURES: The outcome measure was a Modified Japanese Orthopedic Association (mJOA) Scale.

METHODS: A clinical prediction model was built using data from 272 patients enrolled in the CSM-NA study. Bootstrapping was used for internal validation. The original model was externally validated using data on 471 patients participating in the CSM-I study. The predictive performance of the model was evaluated, including its discrimination, measured by area under the receiver operating curve (AUC), and calibration, assessed by calibration slope, observed: expected ratios, and Hosmer-Lemeshow goodness-of-fit test.

RESULTS: The modified original model consisted of six covariates: age (odds ratio [OR], 0.96), duration of symptoms (0.76), baseline severity score (1.21), psychiatric comorbidities (0.44), impairment of gait (2.48), and smoking status (0.50). The AUC for the original model was 0.77 (95% confidence interval [CI]: 0.71, 0.82) and across the bootstrap replicates was 0.77 (95% CI: 0.76, 0.77), reflecting good discrimination and internal validity. The model tested on the CSM-I dataset yielded an AUC of 0.74 (95% CI: 0.69, 0.79), a calibration slope of 0.75, and an insignificant Hosmer-Lemeshow test. The ORs generated for baseline mJOA (OR, 1.26), impairment of gait (2.67), age (0.97), and smoking (0.55) were very similar to the original values of 1.28, 2.39, and 0.97, respectively. Duration of symptoms (OR, 0.94) had a significantly different odds ratio than in the original model, but the direction of its relationship with outcome was the same. Psychiatric comorbidities was not a significant predictor at an international level, likely because of underreporting: only six patients outside of North American centers were diagnosed with depression or bipolar.

CONCLUSIONS: The parameter estimates generated from the original analysis were internally valid. The original model was also externally valid. The most significant global predictors of surgical outcome were baseline myelopathy severity, age, smoking status and impaired gait.

Posted on

Validation of a prognostic score for early mortality in severe head injury cases

Brain_trauma_CT

J Neurosurg 121:1314–1322, 2014

Traumatic brain injury (TBI) represents a large health and economic burden. Because of the inability of previous randomized controlled trials (RCTs) on TBI to demonstrate the expected benefit of reducing unfavorable outcomes, the IMPACT (International Mission on Prognosis and Analysis of Clinical Trials in TBI) and CRASH (Corticosteroid Randomisation After Significant Head Injury) studies provided new methods for performing prognostic studies of TBI. This study aimed to develop and externally validate a prognostic model for early death (within 48 hours). The secondary aim was to identify patients who were more likely to succumb to an early death to limit their inclusion in RCTs and to improve the efficiency of RCTs.

Methods. The derivation cohort was recruited at 1 center, Hospital 12 de Octubre, Madrid (1990–2003, 925 patients). The validation cohort was recruited in 2004–2006 from 7 study centers (374 patients). The eligible patients had suffered closed severe TBIs. The study outcome was early death (within 48 hours post-TBI). The predictors were selected using logistic regression modeling with bootstrapping techniques, and a penalized reduction was used. A risk score was developed based on the regression coefficients of the variables included in the final model.

Results. In the validation set, the final model showed a predictive ability of 50% (Nagelkerke R2), with an area under the receiver operating characteristic curve of 89% and an acceptable calibration (goodness-of-fit test, p = 0.32). The final model included 7 variables, and it was used to develop a risk score with a range from 0 to 20 points. Age provided 0, 1, 2, or 3 points depending on the age group; motor score provided 0 points, 2 (untestable), or 3 (no response); pupillary reactivity, 0, 2 (1 pupil reacted), or 6 (no pupil reacted); shock, 0 (no) or 2 (yes); subarachnoid hemorrhage, 0 or 1 (severe deposit); cisternal status, 0 or 3 (compressed/absent); and epidural hematoma, 0 (yes) or 2 (no). Based on the risk of early death estimated with the model, 4 risk of early death groups were established: low risk, sum score 0–3 (< 1% predicted mortality); moderate risk, sum score 4–8 (predicted mortality between 1% and 10%); high risk, sum score 9–12 (probability of early death between 10% and 50%); and very high risk, sum score 13–20 (early mortality probability > 50%). This score could be used for selecting patients for clinical studies. For example, if patients with very high risk scores were excluded from our study sample, the patients included (eligibility score < 13) would represent 80% of the original sample and only 23% of the patients who died early.

Conclusions. The combination of Glasgow Coma Scale score, CT scanning results, and secondary insult data into a prognostic score improved the prediction of early death and the classification of TBI patients.

Proudly powered by WordPress