Multilateral Multimodal Human-Robot Collaboration for Robotic Nursing Assistance: Prototype System and Preliminary User Studies
Abstract: Robotic systems have been extensively employed in hospitals and nursing facilities to execute diverse assistance tasks. With the rising demand, there is a crucial need for an adaptable and efficient framework that extends beyond unilateral human-robot communication.
We introduce an innovative robotic nursing assistance system with multilateral multimodal human-robot (MMHR) collaboration, enabling nursing robots to be assisted by remote and on-site operators. Through our augmented reality (AR) interfaces, the remote and local operators can monitor the robots’ operations, issue task and action commands, and facilitate collaborative assistance and information exchange via AR cues or verbal communication. Our preliminary user study evaluated the usability of the prototype system and validated the efficacy of our MMHR collaboration in a representative nursing assistance task scenario. The results show significant improvements in overall task efficiency for the remote operator and reveal human strategies and rationales in the spontaneous multilateral human-robot collaboration.