On board the ISS, NASA astronaut Frank Rubio is in control of the robotics team on "Earthly Mars" at DLR in Oberpfaffenhofen. As part of the first surface avatar experiments, humans and robots had the task of retrieving a scientific instrument - here a seismograph - from the lander and placing it in the mission area.
Credit: DLR (CC BY-NC-ND 3.0).
The Columbus module of the International Space Station ISS houses the Robot Command Terminal for the surface avatar experiments. In addition to a laptop with a graphical user interface, the commander has two input stations: The joystick sigma (left) can be used to intuitively control the movement and viewing directions of the various robots. And with the interaction device (right), NASA astronaut Frank Rubio can control the robots' arms and, for example, move the arm of DLR robot Rollin' Justin as if it were his own. Because the control is governed by seven degrees of freedom, the movements feel very natural. Thanks to built-in force feedback, the human operator also feels what the robot "feels" by sensing the corresponding resistance when the robot touches or grips something.
A Mars environment has been created for the Surface Avatar experiments at the German Space Operations Center at DLR in Oberpfaffenhofen. The series of experiments is led by the DLR Institute of Robotics and Mechatronics and is being conducted in collaboration with the European Space Agency (ESA). The aim is to demonstrate the use of intelligent robots together with astronauts and to further develop the technologies in order to use them for astronautical missions to the Moon and Mars in the future.
ESA's Interact Rover drives on four wheels and has two movable robot arms. One arm is equipped with a gripper to collect rock samples.
DLR's humanoid robot Rollin' Justin successfully delivered the rock sample from the rover to the lander. Now DLR's robotic lander LAMA could bring the valuable cargo back to Earth. With this Mars simulation, the first experiments of Surface Avatar have been successfully completed. At least two additional series of experiments are planned at intervals of around six months, they will be increasingly extensive and complex.
Frank Rubio supported the first Surface Avatar experiments as part of the long-duration Expedition 69, which will run from March 2023 to September 2023. This is the US astronaut's second mission on board the International Space Station (ISS)
Researchers at the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt; DLR) are developing key technologies such as telepresence robotics for the exploration of space, including the Moon and Mars. This will allow robots on a distant planet to carry out tasks commanded by a human from a spacecraft in orbit. The 'Surface Avatar' mission team at DLR in Oberpfaffenhofen has now demonstrated how this could work. From the International Space Station (ISS), a single person, NASA astronaut Frank Rubio, simultaneously controlled several robots on Earth, allowing them to act partially or fully autonomously as required. This human-robot team collaboration is a first and the successful start of a new series of ISS experiments. The Surface Avatar project is led by the DLR Institute of Robotics and Mechatronics and is being carried out in collaboration with the European Space Agency (ESA).
"For us, it is very important to focus on the aspect of human-robot collaboration in order to provide the astronauts with optimal support during crewed spaceflight. To this end, we developed the technologies for collaborative robots some years ago, and these are now widely used in terrestrial applications. With the latest breakthroughs in AI, robots are becoming so versatile and intelligent that they can easily be used by non-roboticists," explains Alin Albu-Schäffer, Director of the DLR Institute of Robotics and Mechatronics.
The robotics team is pursuing two goals with the experiments. On the one hand, they want to demonstrate how different robots can work together to complete complex tasks for space missions. On the other hand, the team is investigating the variations in how the robots can be telecommanded in order to use them as intelligent co-workers at any time exactly as the situation requires.
Work from manual to fully automated
For the current experiment, a Martian landscape was constructed at the German Space Operation Center in Oberpfaffenhofen. In this scenario, three robots were to carry out initial work on the planet's surface in advance of the arrival of humans. Frank Rubio commanded the robots from the Columbus module of the ISS and was able to carry out all the tasks during the two-hour test period. With the help of DLR's humanoid robot Rollin' Justin, he unloaded the lander and installed a seismic sensor. The astronaut used ESA's Interact Rover to monitor the terrain and had the DLR Landing and Mobility Test Facility (LAMA) to support the scientific activities.
Rubio tested a controller with scalable autonomy, which is a completely new technology. In other words, he could determine the extent to which a robot should perform an action autonomously. At the push of a button, he could have a robot perform a task completely autonomously . But the astronaut could also take over the robot as an avatar and carry out individual steps as if with his own hand. For this purpose, he had the Robot Command Terminal (RCT) at his disposal, which combined three control elements. On a screen, he could see what the individual robot saw at any time, control its movements with a joystick and feel what the robot 'felt' thanks to an interactive device with force feedback. For example, when Frank Rubio as Rollin' Justin placed a seismometer on the planet’s surface, the astronaut felt the resistance of the instrument in his hand.
Cooperation on the ground and in space
The RCT is intuitive to use, so Rubio quickly got used to the tele-operation. As a result, he was able to carry out most of the tasks without requiring any assistance. The robotics team monitored the experiment from the Mars laboratory in Oberpfaffenhofen and was in radio contact with the astronaut.
"We are excited to be a step closer to giving astronauts and experts on Earth a wide range of possibilities to command and manage teams of different robots in space. We will be able to use our robots on the surface as physical avatars and intelligent co-workers to execute ever more complex tasks", says Principal Investigator Neal Lii from the DLR Institute of Robotics and Mechatronics.
"This human-robot collaboration paves the way for future missions and permanent outposts on the Moon and beyond," adds ESA Project Leader Thomas Krüger from the ESA Human Robot Interaction Laboratory.
To enable the project team to concentrate on the technical aspects of their work, they were supported by their colleagues at the Columbus Control Centre. The operations team took care of the practical requirements and made sure that Surface Avatar and the other activities on board the ISS were coordinated.
In the future - the Moon and Mars
Robots can be used wherever it is too dangerous for humans. Exploration missions in space are therefore inconceivable without robotic support. For future missions, the interaction between humans and robots will play an even more important role. From the Surface Avatar experiments, the researchers are gaining fundamental data on this – for example, how latency affects control, how microgravity influences an astronaut's perception and what challenges arise during operation. For the upcoming Lunar Gateway and other missions to the Moon or Mars, Surface Avatar will thus decisively advance telerobotics.