BLOG: 25 Years Robot Archive in Neustrelitz
Anniversaries briefly interrupt the flow of our fast-paced lives and clear the way for a look back to the past as well as forward to the future. After last year’s major event celebrating 40 Years DFD, in Neustrelitz at least, the 25th anniversary of our robot archive is on the agenda. It is perhaps worth glancing back to that time of ferment when, among other changes, the integration of the Neustrelitz ground station as part of DFD was well advanced, but by no means complete. Combining two until then autonomously-functioning entities involves practical matters such as information channels and administrative changes like cost accounts, but it must also take place in parallel in the minds of all those involved.
At the time, space activities was undergoing a structural transition from mission-specific data management facilities and archives to multi-mission systems. The time of simply supplying evidence that “we can do that” had come to an end for future missions. Factors of cost and sustainability were becoming ever more dominant. This development accompanied a technology change from only manually-handled storage (for example on HDDT=High Density Digital Tape) to tape cassette storage media used by robot-operated tape libraries – these technologies were passing through an especially impressive growth phase.
Such a growth phase is a kind of Cambrian Explosion. One is confronted with a proliferation of proprietary attempts at solutions for storage media and formats, recording methods, tape libraries, and the specific software required for making the named hardware components usable for an efficient archiving application. Whatever combination one would have selected would in any case be new and untested. Any decision for a particular solution could turn out to be a dead end in just a few years. Of course, our colleagues in Oberpfaffenhofen had some preliminary experience because of missions that were either ongoing (e.g, ERS-1) or planned (like ENVISAT), but they had meanwhile encountered a number of teething problems connected with the newly introduced technologies and their linkage. Then in Neustrelitz the archive for the PRIRODA-MOMS-2P mission had to begin operation within a short time. In the course of working on fulfilling the existing requirements, the teams at both locations began to consolidate. In 1995 a joint steering committee was established and it was decided to have archives at both locations and to initially divide up the work with short- and mid-term mission archives at both locations and a long-term archive in Oberpfaffenhofen. Except for differences in storage capacity, the archives were to have the same technical architecture in order to exploit synergetic effects to the extent possible, such as
- Increasing the pool of experience,
- Streamlining purchases (replacement parts, media, servicing),
- Distributing operations over the two physical locations, and
- Having the potential for redundancy by taking both locations into account.
This and the subsequent work was the responsibility of teams under Willi Wildegger and Hans-Jürgen Wolf (“the two Ws“) at their respective locations. Learning from the software problems encountered in Oberpfaffenhofen, the jointly favoured HSM software (SAM-FS) was subjected to thorough joint testing before purchasing was initiated. Without these tests DFD would not only have been the discoverer of SAM-FS for the European market, but also its first operational user (the first user was Berlin Technical University in October 1995).
So the archive in Neustrelitz started routine operation in May 1996 (formal approval of the robot came on March 5, 1996), making DLR the second user in Europe.
The technical parameters of the first development stage of the NZ archives (76 GB RAID-System, 5.6 TB total capacity) would surely be worth only an amused smile today, since current tape cassettes are available with a native capacity of 18 TB. For comparison, the current capacity in Neustrelitz is today ca. 30 PB (in Oberpfaffenhofen 50 PB). Also, the mentioned technologies, especially in the tape and tape-library areas, have undergone a rigorous process of consolidation, which only a few providers survived.
But structurally, the archive hardly differs from systems now commonly in use. A few “by-products”, especially the storage conferences and the close cooperation between the two spatially-separate DFD departments, have survived to the present time.
What does a look into the future suggest? Tor sure, know-how relating to the process of archiving huge amounts of data and to applications for sovereign remote sensing data will endure. But perhaps the casually-formulated prediction of a cloud-based “archive from a wall socket” will become a reality already in the near future. And then maybe “two Ws” will again be needed to successfully take this step for DFD.