Exactech Study Shows Fairness, Accuracy of Predict+ for Shoulder Surgery

Exactech has reported a new study that shows its Predict+ machine learning clinical support tool creates fair, accurate outcomes predictions for shoulder arthroplasty patients of different ethnicity, sex, and age.
 
The study was recently published in the Journal of Shoulder and Elbow Surgery and is the first of its kind in orthopedics, the company said.
 
The study quantified the accuracy of Predict+ when predicting outcomes for anatomic and reverse total shoulder replacement in over 8,000 patients, stratified by patient subgroups of age, gender, and ethnicity. The authors reported they could identify which patient groups get fair predictions and which received unfair predictions.
 
The study reported Predict+ was deemed fair for 98.6% of regression predictions, 99.4% of substantial clinical benefit (SCB) classification predictions, and 100% of minimal clinically important difference (MCID) classification predictions.
 
“This new machine learning study is important because it proposes a new statistical test methodology to evaluate the fairness of AI predictions,” Chris Roche, Exactech’s sr. VP of extremities, told the press. “And because it establishes the clinically relevant criteria that determine acceptable differences in prediction accuracy between shoulder arthroplasty patients of different demographic status. Perhaps the biggest barrier to adoption of machine learning-based clinical decision support tools is the perception that they are not fair and accurate for all patients. This new clinical outcome study is the first of its kind in the orthopaedic literature to evaluate if an AI prediction is fair.”
 
“Bias in AI clinical predictions can adversely impact decision making,” added Vikas Kumar, Exactech’s VP of machine learning. “Biased or unfair predictions are more likely to occur in patient groups that are underrepresented in the training data. As a result, it is critical to train and evaluate machine learning prediction tools against diverse datasets that are representative of all potential patients. These results were only possible because of the large volume of high-quality clinical data Exactech has collected with the Equinoxe shoulder system. Using our proposed machine learning testing methodology, we aim to provide the statistical determination of what makes a fair or unfair prediction for all machine learning clinical decision support tools.”
 
Predict+, part of the company’s Active Intelligence smart solutions portfolio, was released in November 2020.