In the Bionics group, in the framework of rehabilitation and pain therapy, we are always interested in finding new ways to detect the intentions of an amputee, that is, we want to reconstruct the intended configuration of the missing hand and wrist, and we have information the stump only at our disposal.
How to compactly, cheaply and non-invasively to do it? So far the only really successful attempt is represented by surface electromyography. But clearly, the muscle movements cause deformations in the surface of the forearm, as you press with your fingers.
What if we tried to reconstruct the missing limb by looking at the forearm, while applying those forces?
We have already checked that, with a fixed camera and forearm, it is possible to reconstruct the intended finger forces using fiducial markers. Now we want to free the subject from the setup and mount the camera on top of the screen, and go beyond that!
Your task will be that of reconstructing the hand and wrist configuration by using the images from the camera. We don't know as yet what kind of markers and/or image processing technique will be required; we already have a set of machine learning algorithms in place that enforce this schema starting from electromyographic data, ultrasound images and tactile traces; but as well, your task will also comprise finding the right method for this job.
Offline tests on healthy subjects will be performed as we go about. An online implementation and tests on amputees are later possible, if the approach is proved to be feasible. We also have a state of the art Virtual Reality system available, consisting of a HTC Vive. This allows to test the setup also in Virtual Reality.
Six months minimum, starting as soon as possible.
C. Nissler, N. Mouriki, C. Castellini, V. Belagiannis and N. Navab, "OMG: Introducing optical myography as a new human machine interface for hand amputees,"
2015 IEEE International Conference on Rehabilitation Robotics (ICORR), Singapore, 2015, pp. 937-942.
C. Nissler, N. Mouriki, C. Castellini, "Optical Myography: Detecting Finger Movements by Looking at the Forearm. Frontiers in Neurorobotics." 2016;10:3.