Automation of Cardiac VASQIP Postoperative Outcome Variables using Supervised Machine Learning Methods

Introduction: The Veterans Affairs Surgical Quality Improvement Program (VASQIP) Cardiac dataset has 187 variables, including twenty-two 30-day outcome variables, extracted by surgical quality nurses (SQNs) for all cardiac surgeries. The outcomes variables are used in risk-adjusted outcomes models for quality monitoring and improvement, as well as research. Automating the extraction of these variables could improve efficiency and timeliness.

Methods: We applied supervised machine learning methods using diagnosis, procedure, and medication codes within 30 days following surgery, and not preexisting, from the VA database as predictors. We evaluated these predictors using two decision-tree algorithms, random forest and gradient-boosted trees. We performed hyperparameters optimization using a grid-search to identify the highest F1 score for predicting SQN extracted outcome values.

Results: Postoperative atrial fibrillation (F1: 0.70), reoperation for bleeding (F1: 0.62), and ventilator use for greater than 48 hours (F1: 0.65) had the best performance. Cardiac arrest requiring CPR (F1: 0.46), tracheostomy (F1: 0.53), and renal failure (F1: 0.48) had fair performance. Other outcome variables considered including stroke, coma lasting longer than 24 hours, endocarditis, and more had poor performance.

Conclusion: Supervised machine learning methods can be used to assign the same outcome values provided by SQNs for only some VASQIP surgical outcomes with good performance. Even in cases where diagnosis definitions are similar to outcome definitions such as for Atrial Fibrillation, disagreements on diagnoses are frequent. For both quality monitoring and research, it is important to understand what is driving incongruence between the structured codes and SQN judgements.