The ICORR 2017 submission ‘Megane Pro: myo-electricity, visual and gaze tracking data acquisitions to improve hand prostethics’ by Francesca Giordaniello et al. was selected for the Rehabweek Best poster completion.
The IEEE International Conference on Rehabilitation Robotics (ICORR) 2017 will take place from July 17-20 in London, UK.
During the past 60 years scientific research proposed many techniques to the control of robotic hand prostheses with surface electromyography (sEMG). Few of them have been implemented in commercial systems also due to robustness that may be improved with multimodal data. In this paper we introduce the first acquisition setup, acquisition protocol and dataset including surface electromyography (sEMG), eye tracking and computer vision to study robotic hand control. A data analysis on data from healthy controls gives a first ideas of the capabilities and constraints of the acquisition procedure that is will be applied to amputees in a next step. Different data sources are not fused together in the analysis. Nevertheless, the results support the use of the proposed multimodal data acquisition approach for prosthesis control. The sEMG movement classification results confirm that it is possible to classify several grasps with sEMG alone. sEMG can detect the grasp type and also small differences in the grasped object (accuracy: 95%). The simultaneous recording of eye tracking and scene camera data shows that these sensors allow performing object detection for grasp selection and that several neurocognitive parameters need to be taken into account for this. In conclusion, this work on intact subjects presents an innovative acquisition setup and protocol. The first results in terms of data analysis are promising and set the basis for future work on amputees, aiming to improve the robustness of prostheses with multimodal data.