Reem Agbareia

The people who follow the emerging evidence on large language models (LLMs) in clinical medicine will notice something unusual. A systematic review of 30 studies found diagnostic accuracy ranging from 25% to 98%, a nearly fourfold spread across similar models on similar tasks1. Minor changes in prompt wording and format can shift model performance substantially, even when the underlying clinical task is unchanged2.

INTRODUCTION Vasculitides are systemic inflammatory and immune-mediated disorders affecting blood vessels that pose many diagnostic challenges due to their varied presentation patterns.1, 2 Early diagnosis and treatment are critical for improving patient outcomes.3 Artificial intelligence (AI) has emerged as a potential tool to address these challenges in vasculitis care. Vasculitides can be classified by the size of the blood vessels they affect.

Large language models (LLMs) are increasingly used in medical diagnostics.1 However, their performance in rheumatology remains understudied.2 This gap is particularly pronounced in multimodal capabilities, where visual components play a crucial role in diagnosis.3 We evaluated 2 state-of-the-art LLMs, GPT-4o (OpenAI) and Claude Sonnet 3.5 (Anthropic), on their ability to diagnose rheumatologic and immunologic conditions using multimodal inputs.