Geospatial Foundation Models for EO

Foundation Models (FMs) represent one of the latest leap forward in AI, following the era of Deep Learning. Trained on vast amounts of unlabeled data through self-supervised learning, these models capture rich patterns that can be applied to a wide array of downstream tasks, even with limited or no additional training data. This paradigm holds particular promise for Earth Observation (EO) by enabling breakthroughs in analytical and predictive capabilities.

In EO, FMs can significantly enhance applications such as geospatial semantic segmentation, crop type mapping, etc. By pretraining on large-scale datasets, they deliver better downstream performance. Their latent space representations and embeddings also enable powerful insights while reducing the need for extensive labeled data, a critical advantage in remote sensing, where labeling is often expensive and time-consuming.

Despite these benefits, integrating FMs into EO workflows poses distinct challenges. EO data often spans multiple modalities, resolutions, and spectral bands, requiring specialized adaptation and careful model updating. Moreover, FMs demand significant computational resources and optimized training strategies, particularly when handling enormous, continuously growing geospatial datasets. Evaluating and benchmarking FMs for these specialized applications further complicates their deployment, as existing benchmarks may be limited in scope.

This session begins with a lecture on why High-Performance Computing (HPC) is not just a convenience but a structural requirement for building foundation models that generalize across geographies, sensors, and downstream tasks. Training deep learning models on planetary-scale Earth observation data demands massively parallel processing, fast interconnects, and large-scale storage that turn months of computation into days.

The session then introduces TerraMind, a generative multimodal foundation model for EO developed in the ESA-funded FAST-EO project. TerraMind introduces the concept of Thinking in Modalities, enabling coherent reasoning across heterogeneous data sources, and achieves state-of-the-art results on established community benchmarks. Using the TerraTorch toolkit, participants will use their generative capabilities and fine-tune TerraMind for a real-world downstream task in an interactive, hands-on session.

Finally, the session covers how the emergence of Geospatial FMs has shifted remote sensing analysis from imagery to embeddings, learned representations of EO data. These present a low barrier to EO analysis, providing features that can be used across various downstream tasks. In an interactive hands-on session, participants will learn how to deploy them for crop yield prediction, benchmarking embeddings against state-of-the-art methods.

Agenda

‍

Block 1: Introduction

• Is High-Performance Computing (HPC) Needed? Training deep learning models on planetary-scale Earth observation data

Blocks 2 and 3: TerraMind on downstream tasks

• Hands-on TerraMind: multimodal outputs and fine-tuning for a real-world downstream task with TerraTorch

Block 4: Embeddings for crop yield prediction

• Hands-on: retrieving embeddings, generating them from satellite imagery, and deploying them for crop yield prediction