Implementing AI for Fracture Detection: Accuracy and Impact Assessment

This study aims to assess the impact of AI analysis tools on the radiologist's workflow in the
context of radiographic imaging used to rule out traumatic bone lesions. The primary objectives include evaluating the accuracy and usability of these tools in fracture detection and understanding their influence on the radiologist's workflow. The findings can serve as guidance on how to evaluate, select, and implement AI tools for medical imaging.

1. IMPLEMENTINGAI FOR
FRACTUREDETECTION
ACCURACYANDIMPACTASSESSMENT
A.DUMITRESCU C. BUITRAGOTELLEZ
SOLOTHURNERSPITÄLERAG, SWITZERLAND

2. A.I. «HOTTEST»TOPIC IN RADIOLOGY
> 100 MEDICAL AI COMPANIES
> 500 IMAGING FDA-APPROVED ALGORITHMS
RESEARCH FOCUSES ON:
-NEW APPLICATIONS IN MEDICALIMAGING
-ASSESSMENTS OF ACCURACY ANDRELIABILITY
-IMPACT ON RADIOLOGY WORKFLOW

3. 1. ACCURACY ASSESSMENT
3 HOSPITALS
925 RADIOGRAPHICSTUDIES
2 AI TOOLS
2. IMPACT ASSESSMENT
12 RADIOLOGISTS
14 ITEM QUANTITATIVE QUESTIONNAIRE
STUDY DESIGN

4. ASSESSMENT OF AI TOOLS
SPECIFICITY> SENSITIVITY
GOODPOSITIVE PREDICTIVEVALUE
HELPFULTOCONFIRMFINDINGS
SENSITIVITYCOULDBENEFITFROMFUTUREIMPROVEMENTS

5. EXAMPLES:
NON-DISPLACEDDISTALFIBULAFRACTURE

6. CONFIRMEDBYCT

7. ?TOEFRACTURE

8. NON-DISPLACEDFRACTUREATTHEGREATTOE

9. ?TOEFRACTURE

10. ?TOEFRACTURE(MAGNIFIED)

11. NON-DISPLACEDFRACTUREATTHEFIFTHTOE

12. FALSENEGATIVE EXAMPLE:
FAILURETODETECTSCAPHOIDFRACTURE

13. FALSEPOSITIVE / DOUBT:

14. FALSEPOSITIVE / DOUBT:

15. TYPICALPROBLEMS:
METAL

16. TYPICALPROBLEMS:
CHRONIC CHANGES

17. IMPACT ASSESSMENT
•WEBFORMQUESTIONNAIREFORPARTICIPATINGRADIOLOGISTS
•QUANTITATIVEASSESSMENTUSINGLIKERTSCALEFROM1 TO6
•PEARSON'SCORRELATIONMATRIXTOIDENTIFYRELATIONSHIPS
•ANOVAANDKRUSKAL-WALLISTESTFORDIFFERENCESINRESPONSES

18. AI ASSESSMENT QUESTIONNAIRE

19. QUESTIONNAIRE EVALUATION
Mean questionnaire response values for all radiologists
and for each subspecialisation

20. QUESTIONNAIRE EVALUATION
•Overall usefulnesswas ratedhigh, aswellasperceived
increasein diagnosticreliability
•Clarity and accessibility of AI results were rated very high,
indicating good design of the AI’s output templates.
•Trust in the AI’s diagnostic accuracy scored lower, suggesting
a degree of caution when relying upon AI results.
•Scores varied across subspecialisations, with general
radiologists rating the AI’s usefulness higher

21. CORRELATIONMATRIX

22. CORRELATIONANALYSIS
•Radiologist’syearsof experiencecorrelateinverselywith
perceiveddiagnosticreliability.
•Clarityandaccessibilityof resultscorrelatehighlywiththe
desiretocontinueusingAI.
•Radiologistswhoreportedimproveddiagnosticaccuracyare
likelytocontinueusingthesetoolsin thefuture.
•Radiologists who want to continue using AI tools also value
ongoing assessment and improvement.

23. CONCLUSIONS
•BothAI toolsdemonstratedgoodspecificityandpositive
predictivevalue.
•AI generallyperceivedas helpful and easy to use.
•Clarity and accessibility of results in the PACS environment is
a key factor in the desire to continue using AI tools.
•More experienced radiologists tend to be more skeptical,
and might benefit from targeted training.

24. THANK YOU !
Questionsorcomments?
[email protected]