{"id":161,"date":"2023-04-29T10:29:46","date_gmt":"2023-04-29T14:29:46","guid":{"rendered":"https:\/\/tecpsych.com\/techpsych\/?p=161"},"modified":"2023-04-29T10:29:47","modified_gmt":"2023-04-29T14:29:47","slug":"jama-study-comparing-physician-and-artificial-intelligence-chatbot-responses-to-patient-questions","status":"publish","type":"post","link":"https:\/\/tecpsych.com\/techpsych\/jama-study-comparing-physician-and-artificial-intelligence-chatbot-responses-to-patient-questions\/","title":{"rendered":"JAMA Study Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions"},"content":{"rendered":"
\n
\"\"<\/figure>\n<\/div>\n\n\n

A cross-sectional study, a public and nonidentifiable database of questions from a public social media forum (Reddit\u2019s r\/AskDocs) was used to randomly draw 195 exchanges from October 2022 where a verified physician responded to a public question. Chatbot responses were generated by entering the original question into a fresh session (without prior questions having been asked in the session) in December, 2022. The original question along with anonymized and randomly ordered physician and chatbot responses were evaluated in triplicate by a team of licensed health care professionals. Evaluators chose \u201cwhich response was better\u201d and judged both \u201cthe quality of information provided\u201d (very poor<\/em>, poor<\/em>, acceptable<\/em>, good<\/em>, or very good<\/em>) and \u201cthe empathy or bedside manner provided\u201d (not empathetic<\/em>, slightly empathetic<\/em>, moderately empathetic<\/em>, empathetic<\/em>, and very empathetic<\/em>). Mean outcomes were ordered on a 1 to 5 scale and compared between chatbot and physicians.<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Results<\/strong><\/p>\n\n\n\n

The chatbot responses were preferred over physician responses and rated significantly higher for both quality and empathy. Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit. Randomized trials could assess further if using AI assistants might improve responses, lower clinician burnout, and improve patient outcomes.<\/p>\n\n\n\n

Limitations <\/strong><\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

The main study limitation was the use of the online forum question and answer exchanges. Such messages may not reflect typical patient-physician questions. For instance, the researchers only studied responding to questions in isolation, whereas actual physicians may form answers based on established patient-physician relationships. It is not known to what extent clinician responses incorporate this level of personalization, nor did the authors evaluate the chatbot\u2019s ability to provide similar details extracted from the electronic health record. Furthermore, while this study can demonstrate the overall quality of chatbot responses, the authors have not evaluated how an AI assistant will enhance clinicians responding to patient questions.<\/p>\n\n\n\n

Key Points from the Study<\/strong><\/p>\n\n\n\n

Question<\/strong>  Can an artificial intelligence chatbot assistant, provide responses to patient questions that are of comparable quality and empathy to those written by physicians?<\/p>\n\n\n\n

Findings<\/strong>  In this cross-sectional study of 195 randomly drawn patient questions from a social media forum, a team of licensed health care professionals compared physician\u2019s and chatbot\u2019s responses to patient\u2019s questions asked publicly on a public social media forum. The chatbot responses were preferred over physician responses and rated significantly higher for both quality and empathy.<\/p>\n\n\n\n

Meaning<\/strong>  These results suggest that artificial intelligence assistants may be able to aid in drafting responses to patient questions.<\/p>\n\n\n\n

[Link to Journal article] JAMA Intern Med. Published online April 28, 2023.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

A cross-sectional study, a public and nonidentifiable database of questions from a public social media forum (Reddit\u2019s r\/AskDocs) was used to randomly draw 195 exchanges from October 2022 where a verified physician responded to a public question. Chatbot responses were generated by entering the original question into a fresh session (without prior questions having been … Continue reading JAMA Study Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","_eb_attr":"","om_disable_all_campaigns":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[5],"tags":[26],"class_list":["post-161","post","type-post","status-publish","format-standard","hentry","category-blog","tag-ai-chatbot-vs-clinicians"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/posts\/161","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/comments?post=161"}],"version-history":[{"count":1,"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/posts\/161\/revisions"}],"predecessor-version":[{"id":165,"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/posts\/161\/revisions\/165"}],"wp:attachment":[{"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/media?parent=161"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/categories?post=161"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tecpsych.com\/techpsych\/wp-json\/wp\/v2\/tags?post=161"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}