The amazing manipulation capabilities that illusionists develop in years of training show us clearly the versatility of the human hand. But even everyday tasks like picking a coin from a wallet are – from a robotics point of view – utterly impressive. We used modern medical imaging technologies to investigate the movement of the human hand skeleton, in order to learn from nature for the improvement of our robots.
In cooperation with Rechts der Isar hospital, Munich, we took a large series (~50 images) of magnetic resonance (MRI) images of a live and healthy human hand in different postures. MRI allows three-dimensional views of the inside of the human body. The method works by measuring the response of hydrogen atoms inside the body to magnetic stimulation and has – unlike CT imaging – no known side effects.
Even an everyday task like picking a coin from a wallet must seem like a magic trick to today's robots.
To derive a kinematic model from the MRI images, we conducted the following steps:
Pose estimation of a bone. The points are extracted from the MRI images. The bone shown in blue on the left is taken from one MRI image and the one in red on the right from another. The pose estimation algorithm determines the movement that is necessary to match the blue and red points. The blue points on the right show the result of the pose estimation.
Resulting hand model with 24 degrees of freedom. The index finger metacarpal bone (marked by a black square) is taken as the base of the model. In joints with two axes, the first axis is shown red and the second one green.
Segmentation: Highlight the data that belongs to each individual bone.
Pose estimation: Determine the position and orientation("pose") of each bone with respect to a reference pose.
Identification of joint axes: Numerically determine the position and orientation of joint axes that optimally incorporate the measured bone poses, using different joint models (one or two axes, non-intersecting or intersecting axes).
Build the hand model: Select joint types that appropriately fulfil the compromise between accuracy and complexity, and combine the joints to socalled kinematic chains.
For the pose estimation we used an algorithm that the robot Justin uses to identify the location of objects on a table. (The task is similar: Matching three-dimensional point clouds.)
The resulting hand model is shown on the picture next to the text. The base of the model is the index finger metacarpal bone ("palm bone"), marked by a black square. From there, the kinematic chains extend, indicated by black lines. A kinematic chain is a series of joints, where the position of the last link (in this case the fingertip) depends on the joint angles of all joints in the chain.
The first joint of the thumb is modeled by two non-intersecting axes of rotation, connected by a thick line. The second joint of the thumb also exhibits significant side ward movement and is therefore also modeled by two joint axes, in this case intersecting ones.
The four fingers all have one axis of rotation that allows for a side ward movement and three axes for bending and stretching. The arching of the palm takes place around three axes pointing roughly in the direction of the long axes of the palm bones.
We took MRI series of two more subjects in order to see which features of the hand model apply to "the" human hand in general and which features differ between individual persons. These MRI data are currently being processed.
Apart from kinematics, other aspects of the human hand are also important for its fine manipulation abilities, for example touch sensing, motion planning and motion control.
