Monthly Archives: April 2023

JAMA Study Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions

Blog

A cross-sectional study, a public and nonidentifiable database of questions from a public social media forum (Reddit’s r/AskDocs) was used to randomly draw 195 exchanges from October 2022 where a verified physician responded to a public question. Chatbot responses were generated by entering the original question into a fresh session (without prior questions having been asked in the session) in December, 2022. The original question along with anonymized and randomly ordered physician and chatbot responses were evaluated in triplicate by a team of licensed health care professionals. Evaluators chose “which response was better” and judged both “the quality of information provided” (very poor, poor, acceptable, good, or very good) and “the empathy or bedside manner provided” (not empathetic, slightly empathetic, moderately empathetic, empathetic, and very empathetic). Mean outcomes were ordered on a 1 to 5 scale and compared between chatbot and physicians.

Results

The chatbot responses were preferred over physician responses and rated significantly higher for both quality and empathy. Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit. Randomized trials could assess further if using AI assistants might improve responses, lower clinician burnout, and improve patient outcomes.

Limitations

The main study limitation was the use of the online forum question and answer exchanges. Such messages may not reflect typical patient-physician questions. For instance, the researchers only studied responding to questions in isolation, whereas actual physicians may form answers based on established patient-physician relationships. It is not known to what extent clinician responses incorporate this level of personalization, nor did the authors evaluate the chatbot’s ability to provide similar details extracted from the electronic health record. Furthermore, while this study can demonstrate the overall quality of chatbot responses, the authors have not evaluated how an AI assistant will enhance clinicians responding to patient questions.

Key Points from the Study

Question  Can an artificial intelligence chatbot assistant, provide responses to patient questions that are of comparable quality and empathy to those written by physicians?

Findings  In this cross-sectional study of 195 randomly drawn patient questions from a social media forum, a team of licensed health care professionals compared physician’s and chatbot’s responses to patient’s questions asked publicly on a public social media forum. The chatbot responses were preferred over physician responses and rated significantly higher for both quality and empathy.

Meaning  These results suggest that artificial intelligence assistants may be able to aid in drafting responses to patient questions.

[Link to Journal article] JAMA Intern Med. Published online April 28, 2023.

Mental Health App to Manage Distress

Blog, News, , ,

A new mental health smartphone App has been developed to help people regulate their emotions in healthy ways.

Rutgers researcher, Edward Selby, PhD, Director of Clinical Training for the Rutgers University Clinical Psychology Program has taken a different approach to a mental health app. The app prompts users to consider their mental health at different times throughout the day, increasing awareness of unique personal experiences. Progress and improvement can be viewed over time to help users identify and better understand how their emotions change and the triggers that may cause those changes.

According to Dr Selby, “The better we can understand the underlying causes and dynamics that result in the problems we define as ‘mental illness,’ the better we can design, tailor and adapt treatments to help improve those underlying problems. Selby’s research shows people often react emotionally to stressful situations in ways that make the situations worse. When this happens, people are at higher risk for harmful behaviors, such as substance use, binge eating, hostility and self-injury.

According to the National Alliance on Mental Illness, more than 20 percent of adults in the United States experienced mental illness in 2020. There is increasing evidence that technology may be used to address mental health care beyond the conventional office setting. These approaches, including smartphone treatment apps, may also help reach patients in need of mental health services who lack access to care.

The name of the app is “Storm/breaker” and it’s goal is to “help people naturally and automatically understand dynamic emotional, cognitive, interpersonal and behavioral experiences occurring in their lives. As understanding of these processes grows, they will spontaneously begin to make more healthy and adaptive responses to upsetting situations that arise.”

Storm/breaker app is designed to help users in a number of ways, including:

  • People can learn to understand their unique emotional, psychological and behavioral patterns which Selby said is essential to making positive changes in one’s life. 
  • People can begin to make changes to improve their emotional experiences that may help to defuse upsetting situations and avoid problematic behaviors.
  • The app’s customizable clinical toolkit will allow people to link to other smartphone apps that may help further manage their stress, including entertainment apps to distract, relaxation apps and productivity apps.

Selby said while other apps attempt to convert typical in-person therapy into a smartphone experience, Storm/breaker is a standalone intervention designed specifically to harness the advantages of daily smartphone use. 

Selby will discuss his research on mental health in an episode of the PBS series Healthy Minds with Dr. Jeffrey Borenstein in May 2023 during Mental Health Awareness Month.
The app was programmed in collaboration with Michigan Software Labs.


Source: https://ifh.rutgers.edu/news/

New Brain Health and Fitness Program Helps Adults with Mild Cognitive and Mental Health Issues

Blog,

Managing a digital world can be an ongoing challenge for people facing aging, cognitive decline, mental health issues or other concerns. Tasks many people take for granted, like shopping online, ordering a food delivery, getting money from an ATM or buying a mass transit card, can stretch their technical skills.

To help people better manage technology, and to improve their cognitive function, the Miller School of Medicine Department of Psychiatry and Behavioral Sciences recently established the Brain Health and Fitness Program, which augments patient care with computerized cognitive and functional skills training.

According to program director Philip Harvey, Ph.D., Leonard M. Miller Professor of Psychiatry and Behavioral Sciences, vice chair for research, chief of the Division of Psychology. “There are many people who have trouble learning new technologies, and we want to help them get a better handle on that. That includes adults with serious mental illness, as well as older people who may have mild cognitive impairments, or even healthy adults who simply want to acquire new skills. Increasing cognitive performance makes it a lot easier to learn new skills, regardless of people’s current situation.”

Custom Designed Skills Assessment and Training

Designed to last three to six months, this fee-for-service program begins with an individualized assessment of each patient’s condition, the psychiatric services they are receiving and any medications that they have been prescribed.
Then a customized program is developed for the patient. The team helps patients access the program’s evidence-based software modules, which run on PCs, Macs, or tablets. Patients then self-administer the training at home, community centers or other preferred venues.

The cloud-based software, called Functional Skills Assessment and Training (FUNSAT), was designed by Dr. Harvey and Sara Czaja, Ph.D., professor of gerontology at Weill Cornell College of Medicine and professor emeritus at the Miller School. It teaches people how to perform important tasks, such as online shopping, operating ticketing kiosks and withdrawing money from an ATM. They can also learn medication organization and adherence, a crucial task for patients receiving integrated pharmacological augmentation and brain fitness training.

The Goals of the Program

  1. Improvements in critical cognitive skills such as processing speed, concentration, attention and short-term memory
  2. Associated improvements in functional skills, quality of life, confidence and self-efficacy
    • Occupational and academic performance
    • Everyday living skills
    • Health related self care
  3. Alleviation of caregiver burden in cases where in cases where functional improvements have led to caregiver distress

FUNSAT Program

FUNSAT is simple to complete at home. Patients train for around two hours a week, for at least 15 minutes per session. The program staff monitors their progress online and sends encouraging messages, if necessary.

Through the software, participants learn by doing.
“In our most recent studies, we’ve shown that when people improve in the training, they actually start doing these things in the real world,” said Dr. Harvey. “FUNSAT improves their ability to perform certain tasks, as well as boosting cognition particularly in concert with cognitive training. Not to mention, the practice training gives them confidence to go out and actually do these activities.”

Enhancing Skills in Everyday Technology

Though the Brain Health and Fitness Program is currently based in South Florida, the software’s cloud configuration could make it available to virtually anyone with a good Internet connection. Dr. Harvey and colleagues have worked with a number of facilities to implement FUNSAT, including the Los Angeles County Department of Mental Health, the New York State Office of Mental Health, the Manhattan Psychiatric Center and aging centers throughout the country. The online training helps patients tune their skills before going back into the world.


Though not covered by insurance, the program is rapidly gaining popularity, as it provides a unique opportunity to improve people’s quality of life.
“We have found that two-thirds of the people doing the training make tremendous progress,” said Dr. Harvey. “It helps them improve their skill levels and learn to use everyday technologies that had been giving them trouble. It’s a great way to enhance their well-being.”