FAIRiCUBE - final project review presentation

FAIRICUBE F.A.I.R. INFORMATION CUBES FINAL REVIEW MEETING, THE HAGUE STEFAN JETSCHNY & FAIRICUBE PARTNERS, 15.10.2025 FAIRICUBE REVIEW MEETING AGENDA 2 14:30 -14:45 Project intro and objectives, achievements Stefan Jetschny , NILU (WP1) 14:45 -15:15 UC overview, UC1 & UC3 deeper dive Maria Ricci , S4E and Martin Kapun , NHM (WP2 & WP3) 15:15 -15:35 FAIRiCUBE Hub services Christian Schiller, EOX, Dimitar Misev, CU (WP4&WP5) 15:35 -15:55 Meta data, validation, AI ethics Kathi Schleidt, EPSIT and Stefan Jetschny , NILU (WP1, WP7) 15:55 - 16:15 Impact and persistence Stefan Jetschny, Giacomo Martirano , EPSIT(WP6) FAIRICUBE INTRODUCTION 3 Stefan Jetschny, Kathi Schleidt FAIRiCUBE introduction and objectives - Mission & Objectives - Consortium Focus 1 : Use cases Focus 2 : FAIRiCUBE Hub services Focus 3 : Input for Green deal data spaces enable players from beyond classic Earth Observation (EO) domains to provide, access, process, and share gridded data and algorithms in a FAIR and TRUSTable manner (SO1) making Earth Observation (EO) data more accessible, interoperable, and impactful (SO1) creating FAIRiCUBE HUB, a crosscutting platform and framework for data ingestion, provision, analysis, processing, and dissemination (SO1, SO2, SO3) Building on existing technology and services Exploit and leverage machine learning techniques (SO2) Pilot studies to integrate ML capabilities into FAIRiCUBE Hub services (Lab) Demonstrate and validate FAIRiCUBE services as iterative co-creation process with Use Cases (SO3, SO4) Real world problems in close collaboration with stakeholders Provide persistent input for the design and implementation of the green deal data space (SO5) OBJECTIVES 4 enable players from beyond classic Earth Observation (EO) domains to provide, access, process, and share gridded data and algorithms in a FAIR and TRUSTable manner (SO1) making Earth Observation (EO) data more accessible, interoperable, and impactful (SO1) creating FAIRiCUBE HUB, a crosscutting platform and framework for data ingestion, provision, analysis, processing, and dissemination (SO1, SO2, SO3) Building on existing technology and services Exploit and leverage machine learning techniques (SO2) Pilot studies to integrate ML capabilities into FAIRiCUBE Hub services (Lab) Demonstrate and validate FAIRiCUBE services as iterative co-creation process with Use Cases (SO3, SO4) Real world problems in close collaboration with stakeholders Provide persistent input for the design and implementation of the green deal data space (SO5) OBJECTIVES 5 1) WP2 and WP3 UC1 and UC3 2) WP4 and WP5 FAIRiCUBE Hub 3) WP6 and WP7 CONSORTIUM Research institutes NIL - NILU climate and environmental research institute, Norway WER - Wageningen university and research, Netherlands NHM - Natural History Museum Vienna, Austria Environmental SME’s S4E - space4environment, Luxembourg 4SF - 4sfera, Spain EPS - Epsilon, Italia Infrastructure service providers EOX - EOX, Austria CUB - Constructor University Bremen, Germany, supported by rasdaman GmbH domain specialist/use case owner, geospatial data specialist, infrastructure specialists 6 THIS IS FAIRICUBE 7 2022 2025 We brought together technical specialists and researchers from various domains. We start understanding each other and contribute to FAIRiCUBE Hub! 60 on Teams 40 on GitHub 270 on LinkedIn 400 on Website FAIRICUBE USE CASES 8 Stefan Jetschny, Jaume Targa UC1 : Urban adaptation to climate change (urban focus) UC2: Agriculture and biodiversity nexus (regional focus) UC3: Environmental adaptation genomics in drosophila (regional focus) UC4: Spatial and temporal assessment of neighborhood building stock (urban focus) UC5: Biodiversity occurrence Cubes (regional to European focus) FAIRICUBE USE CASE 1 „URBAN ADAPTATION TO CLIMATE CHANGE” 9 Maria Ricci, Manuel Löhnertz, Marco Cattaneo, Andrea Peters Can the integration and ML-based analysis of currently available biodiversity, agriculture, environmental, and remote sensing data provide comprehensive, verifiable, and actionable insights for different regions? Can data cube functionality and ML help in finding causal relationships between effects of farm level measures, indicators of physical conditions, and direct measures of biodiversity? Can the insights obtained in the study region be extended to other regions by reusing learned patterns applying transfer learning? RESEARCH QUESTIONS Two study areas in Netherlands: Arable land Agricultural grasslands Species abundance - proportional number of observations (represented by points or polygons) per grid cell Species distribution Maximum entropy model - MaxEnt Evaluates parameters such as the “area under curve” value and the omission rate Calculates metrics such as individual variable contribution and partial dependence scores/plots Species richness – biodiversity indicator based on representative farmland bird species distribution Causal modeling - Causal Graph Analysis, to proof the expected causal correlations between agricultural activities and biodiversity Crop rotation and species richness in arable land Grass mowing intensity and species richness in grasslands METHODS 17 Study areas within Netherlands Data storage Rasdaman data catalog S3 storage on EOXHub Data extraction services WCPS Python Client used to extract data from the Rasdaman servers Data extraction Basic queries – spatial/temporal subset, coordinate system transformation, NDVI calculation Processing environment - EOXHub Established JupyterLab custom kernels High performance computation - EOXHub Species distribution modelling required higher RAM memory FAIRICUBE SERVICES 18 Insight into the principles of underlying data cube architectures - in the context of integrating and analysing multidimensional environmental datasets within unified platforms Used approaches for data interoperability, accessibility, and computational efficiency for large-scale spatiotemporal analyses Comparative experience of two distinct technological data cube providers - Rasdaman and EOX Central comprehensive meta data collection at FAIRiCUBE STAC Catalog Incompatibility of UC source vector data formats with the n-dimensional coverage model employed by data cube infrastructures Data access mechanisms of several sources, which involved complicated and non-standardized protocols. These were not directly compatible with the automated ingestion processes and thus required preliminary data acquisition and preprocessing workflows. More advanced EO workflows such as the generation of multi-seasonal, cloud-free mosaics were not available as standardized procedures and had to be developed independently BENEFITS / DRAWBACKS OF FAIRICUBE SERVICES 19 Achievements Ability to confirm causal relationship between the agricultural activity and biodiversity indicator - specifically on the case of crop rotation index and species richness estimated effect suggests that higher crop rotation scores are associated with increased biodiversity confirmed using a refutation analysis, which showed negligible variation when the treatment variable was replaced with random inputs, supporting the validity of the findings Developed methodology for transforming raw bird species observation data into scientifically meaningful estimates of biodiversity in agricultural landscapes In overall, established consistent data-driven assessment which supports the role of biodiversity indicators in ecological inference and decision-making SUMMARY OF ACHIEVEMENTS/RESULTS AND IMPACT FOR OUR DOMAIN 20 The developed methods offers a valuable framework for evaluating the ecological impacts of agricultural policies – such as the EU Green Deal – and for guiding the development of targeted, biodiversity-friendly farming measures using quantifiable indicators. UC provides a foundation for developing policy and decision support tools that help farmers adopt practices aligned with biodiversity improvement goals. SUMMARY OF ACHIEVEMENTS/RESULTS AND IMPACT FOR OUR DOMAIN 21 FAIRICUBE USE CASE 3 „ENVIRONMENTAL ADAPTATION GENOMICS IN DROSOPHILA” 22 Martin Kapun, Sonja Steindl DROSOPHILA AS AN ECO-EVO MODEL 23 Approximately 1,600 species From local food specialists to globally distributed generalists Ecology and environmental selective forces that determine adaptation largely unknown Genes underlying environmental adaptation often unknown https://tumbleweed.com.au/blogs/tumbleweed-community/fruit-flies-ascreatures-of-the-compost-1?srsltid=AfmBOoo6GZ1fkjbed9qv5WygkeLN2ihqXrQhwfsP0SKalpwyv2y9KAW Pal Mahadevan, et al. (2024) iScience 27 RESEARCH QUESTIONS 24 1. Continental scale •Which environmental factors have the strongest influence on adaptation in D. melanogaster. •Which genomic regions or genes in D. melanogaster show signatures of adaptation to specific environmental conditions? 2. Local (urban) scale •How does the species richness and composition change in urban and rural habitats? •Which environmental factors influence the occurrence of certain species? Vienna 1.ADAPTIVE EVOLUTION IN EUROPE 25 Methods Genomic Datasets oPan-European fly collections & genome sequencing o>300 Populations-samples through time and space oPublished in MBE (IF: 10.7) Imputation of missing genomic datasets oML methods; Collaboration with UC4 oPaper in IEEE Xplore; 2nd paper just accepted Environmental datasets oQuerycube online tool to slice E/O data from datacubes stored at Rasdaman oClimate Data (ERA5) through Copernicus API oInterpolated climate data from WorldClim II FAIRICUBE USE CASE 5 „VALIDATION OF PHYTOSOCIOLOGICAL METHODS THROUGH OCCURRENCE CUBES” 32 Susanna Ioni, Kryštof Chytrý, Heimo Rainer UC5-RESEARCH OBJECTIVES 33 This Use Case investigates: • Which environmental factors shape plant distribution in European habitats (study case habitat S22) • Whether a GBIF-EO data method based is comparable or diverge from EUNIS habitat predictions UC5 challenges habitat distribution methods used in EUNIS prediction maps by combining species occurrence data from GBIF (vegetation surveys, museum collections, citizen science) with environmental variables (EO) through a machine learning approach. UC5-METHODS 34 Species data: • GBIF occurrences (incl. synonyms) of diagnostic species Habitat S22 • Filtered for valid coordinates with uncertainty (<500 m) Environmental data (EO): elevation, slope, TWI, HLI, aridity index, temperature, precipitation Data processing: • Pseudo-absences with random buffer method (1 km radius) • Built spatial cubes at resolutions 1 km Europe and 100 m Alps Modeling: Ensemble built on individual models (GLM, GAM, RF), evaluated with TSS scores Validation: Compared predictions with EVA plots Comparison: Assessed predicted distributions against the official EUNIS probability map of S22 UC5-WORKFLOW 35 FAIRICUBE SERVICE 36 Fairicube Service was helpful for: • Testing our pipeline in JupyterLab • Documenting pipeline to share our use case with the community • Publish and document our datasets, metadta and produced scripts ACHIEVEMENTS & RESULTS 37 Documentation: User Friendly Script (R script in GitHub) Datasets (output) Digital Library Outreach: •Conference SPNHC-TDWG Japan 2024 •Conference GDDS 2025 •Conference Living Planet 2025 •Extended Abstract (soon published in BISS) •Poster (Zenodo) 37 CONTINUATION / EXPLOITATION OF UC RESULTS 38 • Business plan developed by ACT students, Wageningen University (2024) → Valuable learning opportunity for all parties involved • No current plans to apply UC results beyond NHMW →Processing pipeline to be utilized within the Botany Department and other biological disciplines of the Museum FAIRICUBE HUB SERVICES ADVANCES AND ENHANCEMENTS 39 Christian Schiller, Dimitar Mišev ⚫Evolution of services −FAIRiCUBE Lab −Data Catalog and Browser −Evolving the STAC Standard and FAIRiCUBE Catalog Editor −rasdaman Workspace −rasdaman Dashboard ⚫Commercial outlook −Extended operation −Customers and supported Initiatives FAIRiCUBE is extensively dependent on its back-ends, powered by : EOxHub Workspaces – a cloud platform where use case teams can host, process, and analyse EO data collaboratively and efficiently. rasdaman Workspaces – a Domain-independent Array DBMS for flexible raster data management and analytics EOX and CU/rasdaman represented as SMEs in FAIRiCUBE continue the commercial exploitation with results from FAIRiCUBE. FAIRiCUBE Hub is a crosscutting platform and framework for data ingestion, provision, analysis, processing, and dissemination persistent demonstrator (for at least a year) blueprint and live example FAIRICUBE HUB SERVICES 40 Services & Apps rasdaman & OGC ⚫WM&, WMTS ⚫WCS, WCPS ⚫OAPI Cov. ⚫openEO ⚫xarray ⚫s3 ⚫etc. etc. SentinelHub ⚫Process API ⚫Batch API ⚫Stats API ⚫openEO ⚫WMS EDC GeoDB ⚫PostgREST API ⚫openEO (TBD) ⚫MLflow ⚫TensorBoard ⚫DVC ⚫rasdaman AI model mgr ML/AI Externals, e.g. Euro Data Cube and EarthServer Datacube Federation Data/Resource/ Code requests FAIRiCUBE Lab - EOxHub Workspace Worker Plane Control Plane ⚫Configuration Mgmt. ⚫GitHub Identity Provider ⚫JupyterLab Profiles ⚫Keycloak ⚫Shared Jupyter Notebooks ⚫Shared Conda Environments ⚫Shared Secrets ⚫Shared Object Storage ⚫Workload Orchestration ⚫Apps ⚫JupyterLab IDE ⚫User Code Execution ⚫pygeoapi - Headless Execution ⚫GitHub code synchronisation ⚫Prometheus ⚫Grafana-Loki ⚫Docker Container Object Storage FAIRiCUBEs Models FAIRiCUBE - rasdaman Workspace Multi-parallel, distributed rasdaman server ⚫Federated AI-Cubes ⚫Visual & zero-coding: ChatCUBE, dashboard, openEO, WCS client, … ⚫Coding: Jupyter R+Python, OWSlib, ... ⚫Intelligent automatic import ⚫Pixel-level security rasdaman database Models FAIRiCUBEs 3rd Party Code (UDF) External archive integration GeoTIFF, NetCDF, … Knowledge Base Services ⚫Django, PostgreSQL ⚫GitHub, ReadTheDocs GPT4-0 ChromaDB Digital Library Query Tool Catalog ⚫GitHub Pull Requests ⚫GitHub Actions (CI/CD) ⚫GitHub Pages ⚫STAC-FastAPI ⚫STAC Browser FAIRICUBE HUB 41 ⚫To better meet the needs of FAIRiCUBE’s use cases, the catalog also introduces its own definitions as a STAC Extension. ⚫This makes it possible to describe and discover EO data in ways that are both standards-compliant and tailored to FAIRiCUBE’s innovative work. −Created new extension → " region extension" −Extending existing extensions ⚫Adding description to " contacts extension" ⚫adding units to datacube extension" − "FAIRiCUBE extension" → tailored to fit project specific definitions EVOLVING THE STAC STANDARD 48 ⚫Integrated access to the STAC-based Catalog-Editor ⚫Adapted to the latest FAIRiCUBE proposed STAC extensions and enhancements ⚫A tool to supply metadata in the correct structure and syntax FAIRICUBE STAC CATALOG EDITOR 49 RASDAMAN WORKSPACE openEO 50 GitHub user account integration Calendar support in datacube queries ⚫E.g. aggregate data per day / month / year Management of coverage metadata, thumbnails, catalog information All datacubes published in FAIRiCUBE catalog with full metadata 47 local datacubes; totaling 6.5 TB 112 federated datacubes; totaling 5.8 PB (Sentinel 1, 2, 3, 5p, ERA5) Python UDFs for AI/ML use cases IMPLEMENTATION PROGRESS 51 LGN datacube - Land Use Database of the Netherlands Toolbox (left) −Catalog −WCPS editor −Predefined views Temporal navigator (bottom) Globe base layer (background) DASHBOARD INTERFACE 52 “Highlight built-up areas with elevation between 0 and 15 meter and population density of less than 100 in red, and areas with elevation below 0 in blue.” −Combine datacubes in different CRS and resolution −EU_DEM: elevation datacube from EarthServer −european_settlement_map_2015: local datacube Tapping into global location-transparent EarthServer Datacube Federation DATACUBE FEDERATION 53 FAIRICUBE – EVOLUTION More seamless access to EO data, applications, and tools Improved usability for researchers and developers A stronger foundation for FAIRiCUBE’s mission to make EO data Findable, Accessible, Interoperable, and Reusable (FAIR) Easier user management via Admin console Self registration with separate approval step, additionally integrated into a "Slack" workflow … and many more Back-end enhancements 54 FAIRiCUBE LAB will still be available for 12 months after the project end FAIRiCUBE: Demonstration of results, notebooks and workflows The underlying EOxHub services have become a centrally important enabler and will sustainably support various initiatives: EuroDataCube: a marketplace for satellite data, Jupyter notebooks and insights on demand EarthCODE: open-access platform that promotes FAIR Open Science principles and collaboration in Earth System Science research DeepESDL: ESA's Deep Earth System Data Laboratory PolarTEP: Polar Thematic Exploitation Platform GTIF: Austrian Demonstrator GTIF, Cerulean GTIF, Baltic GTIF DESIDE: Supporting policy and decision-makers in decreasing pollution and optimising routes in polar regions OUTLOOK 55 ML inference on datacubes Model management + model fencing Continued in EU FAIRgeo Location-transparent federation Continued in EU SkyFed: cloud/edge integration User-friendly point data extraction querycube.nilu.no Rasdaman innovations used worldwide by industry, agencies, research Example: EarthServer datacube federation OUTLOOK: RASDAMAN 56 Input for Green deal data spaces 57 Kathi Schleidt, Stefan Jetschny •Join session after FAIRiCUBE review •Meta data •Validation framework •AI ethics consideration FAIRICUBE website FAIRiCUBE Hub FAIRiCUBE Digital Library Use Case Contributions Data and Models on GitHub Zenodo FAIRiCUBE resources will remain available! 66 Project website General information News UC introduction UC scrollytellings Reports Documentation 67 FAIRiCUBE Hub Access point to services Validation Documentation links Search and Querry Information Catalogs Documentation 68 Digital Library Overview Methods Data Processes Solutions Link to resources Documentation Documentation 69 Link to resources GitHub Zenodo Data Catalog Scrollytelling Publications as F.A.I.R. as possible! FAIRiCUBE Knowledge Base Services Upscaling and business planning 71 City data analysis toolbox (UC1) Bio-Agri-Why (UC2) QueryCube (UC3) UC4 methodology Habitat prediction mapping (UC5) FAIRiCUBE KERs (Key Exploitable Results) FAIRiCUBE HUB UPSCALING and business planning 72 Methodology under test, final product/service to be identified Prototype of product/service under production Prototype of service/product ready Product/service early adopted Product/service widely adopted The KERs roadmap 73 Time T1: project end UC KER status at the end of the project City data analysis toolbox QueryCube Bio-Agri-Why UC4 methodology Habitat prediction mapping Product/service early adopted Product/service widely adopted Prototype of service/product ready Prototype of product/service under production Methodology under test, final product/service to be identified Upscaling timeline legend Different timeline for each KER Product/service early adopted Product/service widely adopted Prototype of service/product ready Prototype of product/service under production Product/service early adopted Product/service widely adopted Prototype of service/product ready Prototype of product/service under production Product/service early adopted Product/service widely adopted Prototype of service/product ready Prototype of product/service under production Product/service early adopted Product/service widely adopted Product/service early adopted Product/service widely adopted FAIRiCUBE Hub Product/service widely adopted Product/service early adopted Upscaling and business planning 74 Upscaling plan FAIRiCUBE Hub Business Model UCi KER Business Model UCj KER Business Model Full BP for UC5 KER!! Upscaling and business planning SO1: Provide an environment enabling stakeholders to access and analyse data utilizing ML SO2: Enable processing pipeline techniques leveraging ML for datacubes SO3: Enable direct utilization of data-derived information for policy SO4: Assure that FAIRiCUBE meets the needs of real-world stakeholders SO5: Assure widespread uptake and persistence of the FAIRiCUBE HUB OBJECTIVES 81 COMMUNICATION CHANNEL KPIS STATUS EU POLICY ~ 3 events 4 events: • EIONET 11-2023 • Biodiversity sessions of the joint workshop (ESA and DG-RTD) • Cluster event in Brussels 6-2025 • GDDS event in Vienna 6-2025 DOMAIN SPECIFIC EVENTS ~ 5 events 19 events NEWSLETTER ~ 6 newsletters 6 newsletters published WEBSITE ~ 2K views - > changed to 1K ~1K page views SOCIAL MEDIA ~ 1K followers -> changed to 500 ~260 followers on LinkedIn, 99 post during project PRESS RELEASES AND ARTICLES ~ 5 articles 7 articles published in national, regional, European online media INNOVATION WORKSHOPS ~ 2 workshops GDDS event 2 days TRAININGS AND WORKSHOPS ~ 3 training days 7 webinars, 7 workshops, Geo-scripting course, ACT training SCIENTIFIC PAPERS ~ 3 papers 3 papers in scientific journals; 4 scientific papers in prep; 3 conference papers; 5 posters Dissemination documentation