In a significant experiment, a US medical school used Open AI's Chat-GPT 4 to see if it can make accurate diagnoses in challenging medical cases.
Physician-researchers at Beth Israel Deaconess Medical Center (BIDMC) in Boston found that Chat-GPT 4 selected the correct diagnosis nearly 40 per cent of the time.
Chat-GPT 4 also provided the correct diagnosis in its list of potential diagnoses in two-thirds of challenging cases, revealed the findings, published in JAMA.
"Recent advances in artificial intelligence have led to generative AI models that are capable of detailed text-based responses that score highly in standardised medical examinations," said Adam Rodman, co-director of the Innovations in Media and Education Delivery (iMED) Initiative at BIDMC.
"We wanted to know if such a generative model could 'think' like a doctor, so we asked one to solve standardised complex diagnostic cases used for educational purposes. It did really, really well," said Rodman, also an instructor in medicine at Harvard Medical School.
To assess the chatbot's diagnostic skills, Rodman and colleagues used clinicopathological case conferences (CPCs), a series of complex and challenging patient cases including relevant clinical and laboratory data, imaging studies, and histopathological findings published in the New England Journal of Medicine for educational purposes.