Deep-cARe System: Projection-based Augmented Reality with Deep Learning for Elderly Healthcare in Indoor

Developing innovative and pervasive smart technologies that provide medical support and improve
the welfare of the elderly has become increasingly important as populations age. Elderly people
frequently experience incidents of discomfort in their daily lives, including the deterioration of cognitive
and memory abilities. To provide auxiliary functions and ensure the safety of the elderly in daily living
situations, we propose a projection-based augmented reality (PAR) system equipped with a deeplearning
module. In this study, we propose three-dimensional space reconstruction of a pervasive
PAR space for the elderly. We propose the application of a deep-learning module to lay the
foundation for contextual awareness. Performance experiments were conducted for grafting the deeplearning
framework (pose estimation, face recognition, and object detection) onto the PAR technology
through the proposed hardware for verification of execution possibility, real-time execution, and
applicability. In addition, several user interfaces (UI) were implemented according to the scenarios
regardless of distance between users and the proposed system. In this study, we developed a
bidirectional PAR system that provides the relevant information by understanding the user
environment and action intentions instead of a unidirectional PAR system for simple information
provision. We present a discussion of the possibility of care systems for the elderly through the fusion
of PAR and deep-learning frameworks.