A Comparative Analysis of ChatGPT and Google’s AI’s “Bard” in Medicine

Ethan Waisberg; Joshua Ong; Mouayar Masalkhi; Nasif Zaman; Pritul Sarker; Andrew G. Lee; Alireza Tavakkoli

doi:10.61838/kman.najm.1.2.5

Authors

Ethan Waisberg Department of Ophthalmology, University of Cambridge, Cambridge, United Kingdom Author https://orcid.org/0000-0001-8999-0212
Joshua Ong Department of Ophthalmology and Visual Sciences, University of Michigan Kellogg Eye Center, Ann Arbor, MI, United States Author https://orcid.org/0000-0002-6750-6036
Mouayar Masalkhi University College Dublin School of Medicine, Belfield, Dublin, Ireland Author
Nasif Zaman Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, Nevada, United States Author https://orcid.org/0000-0003-0120-0939
Pritul Sarker Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, Nevada, United States Author https://orcid.org/0000-0002-6290-5484
Andrew G. Lee Center for Space Medicine, Baylor College of Medicine, Houston, Texas, United States Author https://orcid.org/0000-0002-2473-299X
Alireza Tavakkoli Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, Nevada, United States Author

DOI:

https://doi.org/10.61838/kman.najm.1.2.5

Keywords:

generative artificial intelligence, large language model, GPT-3.5, LLM, transformer network

Abstract

Background: Bard AI, an AI chatbot developed by Google, emerged as a response to the success of OpenAI's ChatGPT. Bard utilizes natural language processing and machine learning techniques to emulate human-like dialogue.

Objectives: In this paper, we wanted to compare the Bard’s performance to that of ChatGPT at various medical and surgical related tasks.

Methods: The responses generated were then examined by three doctors based at three different institutions to compare the performance of each AI chatbot for each specific prompt.

Results: Bard had the ability to generate a discharge summary, summarize medical literature, and recommend relevant medical guidelines. However, Bard’s generated responses were not always clinically appropriate and contained both minor and major errors. Bard and ChatGPT will likely be followed by even more capable AI systems.

Conclusions: As these new tools are released it is important that they be viewed cautiously, ensuring that patient safety remains the fundamental priority.