International

Intelligent Patient Assessment & Monitoring System: Developing an Emotion Recognition Algorithm for Vocal Cues

Wednesday, November 1, 2023

6:10 PM – 6:16 PM

Location: Station 12

Presenting Author(s)

Kalia Lofitou, MSc (she/her/hers)

PhD Student
Center for Applied Neuroscience, University of Cyprus
Limassol, Limassol, Cyprus
TT

Theocharis Theocharides

Associate Professor and Center Director
KIOS Research and Innovation Centre of Excellence, Cyprus
Fofi Constantinidou, PhD, CCC-SLP, CBIS, FACRM (she/her/hers)

Professor and Center Director
University of Cyprus
Nicosia, Nicosia, Cyprus
Eva Pettemeridou, epette01@ucy.ac.cy (she/her/hers)

Post-doctoral fellow
CAN-KIOS, UCY
Nicosia, Nicosia, Cyprus

Research Objectives: This project will develop an intelligent Patient Assessment & Monitoring System (iPAMS) consisting of different sensor networks and smart software algorithms with the ultimate goal to monitor and assess patients with TBI in the rehabilitation setting.

Design: Cross-sectional study.

Setting: The study took place at the Centre for Applied Neuroscience, University of Cyprus.

Participants: The algorithm developed was tested on a set of data collected from 24 neurotypical participants (male = 8; female = 16), with an age range of 18-51 (M = 27.08; SD = 7.34). The data set of the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) was used to create the algorithm of the Online Speech Emotion Recognition using Python 3.7.6. The code of speech expressions included happy, sad, angry, fearful, disgust, surprise, and neutral. The gender differentiation was also inserted to detect the different emotions between men and women. The output is a real-time speech emotion recognition.

Interventions: Not applicable.

Main Outcome Measures: A sophisticated online algorithm has been developed to detect and classify vocal signals into different emotions.

Results: The speech recognition algorithm developed and implemented was shown to effectively detect and classify all targeted emotions, in men and women. Most importantly, this speech recognition technique can be applied in real-time emotion detection.

Conclusions: These findings suggest that real-time voice processing algorithms could contribute towards the understanding of the emotional state of patients, more efficiently, prior to the presence of any disruptive behavior. The idea behind this is that this software algorithm could specifically identify a potential behavioral alert through a personalized patient profile and notify the rehabilitation facility and the patient with TBI with real-time feedback.

Author(s) Disclosures: None disclosed.

Learning Objectives:

Understand that machine learning algorithms can analyze audio signals, such as speech patterns, tone, and prosody, to detect and classify emotions expressed by patients during therapy sessions or interactions with healthcare professionals.
Understand that real-time feedback from audio machine learning techniques can provide caregivers with insights and guidance on how to respond effectively to patient emotions and prevent disruptive behaviors.
Recognize the relevance and effectiveness of algorithms regarding the installation of at-home devices aiming at the optimal independence of patients at home.