Patient-generated health data may enhance the accuracy and yield of referrals for many medical conditions. However, for chronic rhinosinusitis, traditional statistical methods have failed to establish a clear relationship between objective and subjective measures.
Accordingly, the diagnosis of chronic rhinosinusitis has traditionally relied on objective measures of disease burden, including computed tomography (CT) or endoscopic findings. Moreover, the lack of a clear relationship between objective markers and subjective patient-reported outcomes has limited the use of patient-generated health data for the diagnosis of chronic rhinosinusitis.
During the 2024 American Rhinologic Society (ARS) meeting at the COSM held on May 15–16, 2024 in Chicago, Arun Raghavan, MD from the University of Washington described a novel approach that relies on a machine learning model using pre-visit, patient-generated health data to predict objective measures of sinonasal inflammation detected on CT.
The study included 543 unique patients, who were evaluated at a tertiary care rhinology clinic and subsequently underwent CT imaging studies. Among the patients, whose mean age was 48 years, 52% were male, and 52% had a primary diagnosis of chronic rhinosinusitis. Prior to an in-person evaluation, patient-generated health data were collected via a novel, web-based platform. The collected patient-generated health data included relevant patient history, cardinal symptoms, and Sino-Nasal Outcome Test-22 (SNOT-22) scores. Expert reviewers manually assigned Lund-Mackay scores (LMS) to the CT scans, with LMS ³ 5 considered to be cases positive for sinonasal inflammation.
Three different machine learning models (Random Forest Classifier, XGBoost, and deep neural network) were considered for this analysis and were evaluated for their overall accuracy, sensitivity, specificity, and area under the curve. The area under the curve was regarded as a measure of the model’s ability to discriminate between two predicted classes: the presence or lack of sinonasal inflammation on CT. The XGBoost, which is a tabular dataset model, achieved the best performance with specificity of 91.9% and accuracy of 74.5%.
Next, the researchers decided to perform a sub-analysis, looking at the diagnostic criteria for chronic rhinosinusitis. They considered as positive chronic rhinosinusitis cases those that had more than two cardinal symptoms of the disease plus inflammation signs on CT above the previously defined threshold. Again, the investigators found that the XGBoost model performed best, with specificity of 97.7% and an improvement of the prediction power over that for sinonasal inflammation alone on practically all metrics.
The data demonstrated that machine learning, in combination with pre-evaluation, patient-generated health data, can be used to predict CT findings of sinonasal inflammation. Dr. Raghavan emphasized that, to his knowledge, “this is the first demonstration of a prediction of objective CT findings from pre-evaluation, patient-generated health data alone; especially on the chronic rhinosinusitis diagnostic criteria, where machine learning models achieved very high specificity, suggesting that they may be good rule-in tests.”
Dr. Raghavan also discussed the limitations of the study. In particular, the relatively lower sensitivity of the machine learning model using patient-generated health data may not be useful in borderline cases, where there is a practical need to rule out chronic rhinosinusitis.
The findings of this study may have implications for the clinical practice, as they suggest a machine learning model using pre-visit, patient-generated health data has the potential to serve as a screening rule-in test for CT-positive chronic rhinosinusitis. Moreover, they may have an impact on subspecialty referrals and the cost-effective ordering of CT scans, especially when deciding in which patients there is a high likelihood to detect positive CT scan results. However, further investigations on larger cohorts are needed to confirm and expand these findings.