DaFab’s Data Management with DASI
Workflows processing Earth Observation (EO) data have a problem – the body of available EO data is vast. And growing rapidly. Within the DaFab EU project, AI-driven workflows must process massive quantities of EO data, made available by the Copernicus project, in an efficient and reliable manner. This presents a range of problems, including locating the relevant data, decoupling relatively fast and scalable compute tasks from slower data transfers, storing the data in a way that the workflows can use it, and managing the lifetime of any temporary copies required. This is where DASI (the Data Access and Storage Interface) plays a critical role. It provides the smart bridge between storage systems and compute environments. DASI’s semantically driven data management design helps build intelligent, scalable, and optimized AI workflows in the DaFab project.
Summer School: AI for Earth Observation and Scalable Data Management
Held in Ljubljana from 22nd to 25th of September 2025, this intensive summer school offers a unique opportunity to delve into the cutting-edge intersection of Artificial Intelligence (AI) and Earth Observation (EO), combined with the essential skills for managing large-scale data and workflows in modern computing environments. Participants will gain theoretical knowledge and practical experience in applying AI techniques to analyze EO data, optimizing AI performance, managing complex workflows with Kubernetes, and handling massive datasets.
High Performance Kubernetes (HPK)
Bridging Cloud-Native Workflows and HPC for DaFab
One of the main goals of the DaFab project is to enable seamless, multisite scientific workflows—without the need to physically move data between sites. This is crucial in environments where data transfer is restricted by bandwidth, administrative domains, or security policies. One of the main DaFab’s goals is to orchestrate distributed workflows across multiple HPC and cloud sites, letting data stay in place while computation moves to where it’s needed.