Deep Learning-Enabled Vision-based ADL Object Localization for Assistive Technology

Research Objectives: This research aims to develop and evaluate a vision-based approach for localizing objects used in activities of daily living (ADL). Specifically, the study aims to investigate the accuracy and precision of object localization using a pre-trained deep learning model and RealSense depth camera and to develop a mapping function that can align the camera's depth data with the pre-trained model's object detection results. The proposed approach is evaluated on various ADL objects in different real-world settings.

Design: Experimental Study.

Setting: In the Biorobotics Lab, University of Wisconsin-Milwaukee..

Participants: 3 researchers (healthy individuals) with a mean age of 27.3 years old.

Interventions: The proposed system localizes different ADL objects in 3D cartesian coordinates to assist the assistive robot in various real-world settings. Participants were requested to complete an evaluation survey following the completion of the study to ensure the system's usability for real-world application.

Main Outcome Measures: The main outcome measures for this study are accuracy and precision of object localization, computation time, and user feedback on usability and effectiveness.

Results: The study's results indicate that the proposed vision-based approach for ADL object localization is effective and accurate. Object localization accuracy reached up to 92%, with an average distance error of 5 cm from the ground truth location. The computation time was found to be reasonable, with an average processing time of 15 frames per second. User feedback was positive, with the system's usability and effectiveness.

Conclusions: This study presents a vision-based approach for ADL object localization, using a deep learning model, depth camera, and mapping function. The approach was found to be accurate, with high object localization accuracy and reasonable computation time. Future work will include this localization approach to manipulate an assistive robot semi-autonomously using robot vision. Overall, the system demonstrates the potential of the proposed approach in real-world applications for assistive technologies and robotics settings.

Author(s) Disclosures: The authors have no conflicts of interest to declare.