AI in Space – Embed2Scale at the EOC
Artificial intelligence brings onboard satellite data analysis to the level of human observation. Instead of data, information can be sent directly to the ground. For example: Rather than downloading and analyzing area-wide, high-resolution satellite images with the help of a ground station network and human inspection, in the near future, only the coordinates of forest fires actually detected will be sent directly to the firefighting teams in the field.
To bring this vision of AI analysis to life on resource-constraint satellite hardware, the data must be compressed into "embeddings". The European project "Embed2Scale" (E2S) is actively working on such “intelligent” embeddings for Earth observation and climate model data. Embeddings are translations of complex data into vectors of a low-dimensional vector space. In these vector spaces, similarity searches or classifications, for example, can be carried out more easily and quickly.
The Embed2Scale consortium project met at the EOC in early July to discuss the status of their research. Also present: advisory board members and satellite companies ICEYE and Orora Technologies. The Finnish company ICEYE operates the world's largest fleet of SAR (Synthetic Aperture Radar) satellites. Thanks to their robustness to atmospheric conditions (clouds, lighting, etc.), these satellites are suitable for many operational applications, such as ship detection, the detection of oil slicks or the classification of sea ice. The satellites of the Munich-based start-up company OroraTech provide high-resolution thermal images of the Earth in near real-time to detect forest fires at an early stage. Both, OroraTech and ICEYE, are very interested in satellite AI, as it enables new, delay-free real-time services that can be implemented directly on the satellite. But it is not only satellites that benefit from embeddings: Strong compression, up to a thousand-fold, should enable resource-saving real-time analysis of data across distributed data centers in the future.
The translation of data into embeddings is achieved through neural compression using large foundation models such as TerraMind, currently one of the most powerful open-source models of its kind. EOC employees are in close touch with researchers from IBM Research Europe in order to utilize TerraMind. This deep neural network architecture was trained with data from satellite sensors, data on geomorphology, and surface properties of the Earth.
Embed2Scale has published the "NeuCo-Bench" software to quantify the quality of the generated embeddings for use cases such as maritime monitoring, biomass estimation, cloud characterization, and agricultural monitoring. Earth2Vec ultimately emerged from this benchmarking efforts – a user-driven initiative in which companies, government institutions, and research organizations join forces to advance neural compression for Earth observation applications in practice.
Embed2Scale (E2S) is funded under the European Union's Horizon Europe program by the European Union Space Program Agency (EUSPA), the state of Switzerland and the United Kingdom. Under the coordination of Martel Innovate, the following partners form the consortium: DLR, IBM Research Europe, the EU organization SatCen, the Helmholtz center Forschungszentrum Jülich, the Slovenian company Sinergise (now part of the US company Planet), the Spanish satellite operator hisdeSAT, the University of Zurich, the University of Münster, and the University of Oxford.