13.2 Data Management Plan (DMP)

Data Management Plan D eliverable 13.2 Authors : Markus Donat (BSC) Pierre Antoine Bretonniere (BSC) 2 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). Document information Grant agreement 101137656 Project title Towards an Integrated Capability to Explain and Predict Regional Climate Changes Project acronym EXPECT Project start date 01/04/2024 Related work package WP13 Related task(s) T13.1/T13.4 Lead organisation BSC Authors Markus Donat /Pierre Antoinne Bretonniere Submission date 30/09/2024 Dissemination level PU History Date Submitted by Reviewed by Notes 16/09/2024 Markus Donat Marsia Ellina (BSC) Feedback on ethics and GDPR compliance 27/09/2024 Markus Donat Marsia Ellina (BSC) First complete version after internal iterations Please cite this report as: Donat, M., Bretonniere, PA., (2024), Data Management Plan, D13.2 of the EXPECT project Disclaimer: Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Table of content 3 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). About 4 Executive summary 4 1.Data Summary 5 2.FAIR data 8 3.Other research outputs 9 4. Allocation fo resources 9 5. Data security 10 6. Ethics 10 List of tables Table 1-Information on new and re-used data _____________________________________________ 6 4 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). About The climate system is changing rapidly and some regions have seen increases in extremes beyond what is expected from climate model simulations. To support targeted climate adaptation strategies, EXPECT will enable trustworthy assessments and predictions of regional climate change including extremes by developing a prototype operational capability for integrated attribution and prediction of climate. This ambitious goal is closely aligned with the WCRP Lighthouse Activity on Explaining and Predicting Earth System Change. EXPECT will identify and quantify the mechanisms by which physical processes govern regional climatic changes, including extremes, on inter-annual to multi-decadal time scales. It will do so by exploiting newly available climate simulations and Earth Observations (EOs), and by combining machine learning (ML) with physical methods. The research will target fundamental knowledge gaps related to atmospheric circulation and land-atmosphere interactions, which represent major limitations in current climate predictions and projections, and in particular in understanding changes in European summer extremes. To underpin the research, and benefitting the wider research community, EXPECT will develop tools to efficiently analyze a variety of large data sets in combination that are hosted in different repositories across institutions. This will facilitate the exploitation of recent investments into high-resolution climate models and E O data. EXPECT will further build data science capacity for the scientifically robust, efficient and reproducible analysis of the massive data assets, including novel ML approaches, and provide training for the climate science community and the next generation of researchers in particular. EXPECT will thus deliver significant scientific and technological advances for society and the climate science community that will last well beyond the project, in support of WCRP’s strategic objectives. Executive summary The EXPECT Data Management Plan (DMP) describes the collection, use, processing and storage of any data related to the project. The DMP (D13.2) is a living document that will be updated regularly in order to include changes regarding new data, new agreements between the partners, the need for exploitation of the results or the sharing of data with third parties. The updated versions of the DMP will be delivered in M24 (D14.2) and the final version in M40 (D15.2). The DMP is also consistent with the rights of the Granting Authority reflected in the Grant Agreement (Art.16) regarding “the right to use non-sensitive information relating to the action and materials and documents received from the beneficiaries (notably summaries for publication, deliverables, as well as any other material, such as pictures or audio-visual material, in paper or electronic form) for policy, information communication, dissemination and publicity purposes, during the action or afterwards”. 5 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). 1. Data Summary The EXPECT project has a strong focus on exploiting existing numerical data, or data that are being produced in other projects, and on developing methods to generate new knowledge from these data. These data are (or will be) largely available in publicly accessible archives such as ESGF, ESA CCI, Copernicus C3S, or the DestinE data lake. To a limited extent we will also produce new data, which will be stored at different institutional archives and made accessible as far as possible. During the first two years of the project, we primarily use existing climate simulation data from the Coupled Model INtercomparison Project phase 6 (CMIP6), the Large Ensemble Single Forcing Model Intercomparison Project (LESFMIP), reanalysis data such as ERA5, and earth observations. These data are being provided via the ESGF, Copernicus Climate Change Service and ESA. In the framework of the activities of Theme 4, DKRZ and BADC will provide access to a large part of data from CMIP6 and other netcdf data that the users have shown interest for. The exact needs in terms of datasets, volumes and accesses are being gathered through a survey sent to the whole consortium. The results will be reported and analyzed in the milestone 7.1 due at Month 12 and will be used as input for the next review of the data management plan. Preliminary results show that the datasets used are mainly ERA5 reanalysis, seasonal forecast from ECMWF, and CMIP6 data. Most of these datasets are under Creative Commons Attribution 4.0 International License (CC BY 4.0; https://creativecommons.org/licenses/) for ESGF and an open license (https://cds.climate.copernicus.eu/api/v2/terms/static/licence-to-usecopernicus-products.pdf) for the ERA5 and seasonal forecast data. The table below summarises the characteristics of the data mentioned in the proposal of the project. This will evolve based on the results of the survey. 6 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). Table 1-Information on new and re-used data OriginProject Type Source Existi ng New-Expected availability Licences ERA 5 netcdf/ grib Climate Data Store yes https://cds.climate.copernicus.eu/a pi/v2/terms/static/licence-to-usecopernicus-products.pdf ERA 6 netcdf/ grib Climate Data Store no end of 2024 CAMS netcdf/ grib Climate Data Store yes https://ads.atmosphere.copernicus. eu/api/v2/terms/static/licence-touse-copernicus-products.pdf CMIP6 netcdf CMOR ESGF yes CC BY 4.0; https://creativecommons.org/licens es/ LESFMIP netcdf CMOR ESGF yes CC BY 4.0; https://creativecommons.org/licens es/ NextGEM S grib and netcdf Connection to the Levante supercomputer / small subsets published at WDCC yes CC BY 4.0; https://creativecommons.org/licens es/ EERIE netcdf CMOR ESGF no CC BY 4.0; https://creativecommons.org/licens es/ Destinati on Earth grib DestinE Core Service Platform (DESP) no Theoretically in October 2024 but date still not official Unknown OptimES M netcdf CMOR ESGF no CC BY 4.0; https://creativecommons.org/licens es/ ESA CCI netcdf https://climate.esa.int/ yes “ Licensing conditions follow the principles of openness and transparency wherever applicable. All ECV data products resulting from the project are freely accessible.” LSA SAF EUMETSA T netcdf lsa-saf.eumetsat.int yes The use of these products is granted to every interested user, free of charge. If you wish to use these products, EUMETSAT's copyright credit must be shown by displaying the words "copyright (year) EUMETSAT" on each of the products used. 7 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). Despite the focus of EXPECT on exploiting existing data, some research tasks involve the generation of new data. These are listed below, together with their intended storage location, and indicating the responsible project partner: - Multi-annual initialized predictions with individual forcing (ECMWF): a subset of the data will be made available at the ECMWF Web API (https://www.ecmwf.int/en/computing/software/ecmwf-web-api), and the full set of output variables will be stored at ECMWF’s MARS archive - Datasets generated or enhanced by novel AI applications (DKRZ): will be made available in the World Data Center for Climate (WDCC), a certified long-term archive hosted at DKRZ (which includes the assignment of persistent identifiers (e.g. DOIs). - JULES simulations with different soil hydraulic parameter sets (UREAD): will be made available via the UKRI-CEDA archive (CC BY 4.0; https://creativecommons.org/licenses/ ) - New updated forcing data (IAA-CSIC, GEOMAR – will be replaced by new partner Univ. of Heidelberg, amendment in progress) will be made available via ESGF / input4MIPs (https://aims2.llnl.gov/search?project=input4MIPs) Apart from the numerical data, the project deliverables will be stored in the project’s wiki which will be used as an internal repository. The EXPECT wiki is managed from a private server in BSC and it is used exclusively by the project’s scientific and administrative team. Username and password is generated automatically and sent to each user. Moreover, all the project’s deliverables have a public dissemination level (PU), which means that once approved by the granting authority, they will be fully open and accessible to the wider scientific community via the CORDIS interface. 8 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). 2.FAIR data EXPECT will develop infrastructure for an integrated analysis of a variety of large datasets stored in different structures and repositories. This will deliver increased coverage of datasets visible to and accessible via web processing services and put in place systems to support distributed data analytics across more types and formats of data than currently possible, and do so by developing appropriate standards, and implementing them in production software. Specifically, we will increase the efficiency of analysing model outputs by enabling distributed compute workflows across distributed HPC centres (initially DKRZ, BSC and CINECA). Moreover, EXPECT will build data science capacity and enable the wider climate science community to easily reproduce the project’s data science developments and use the project outputs as a basis for further developments. This will be achieved through committed efforts to share code and ready-to-use data, and the organisation of, and contribution to, specific training events. The FAIRness of the data in the project but not generated by the project itself is already ensured in all the datasets. ESGF data has persistent identifiers (PIDs and DOIs) and is based on a community vocabulary and strict metadata conventions (CMOR) stored in the netcdf format. ERA5 and seasonal forecast, as part of the climate data store also have DOIs. The format is GRIB and has also controlled vocabularies and strict metadata requirements, ensuring its FAIRness. Work on data FAIRness will be conducted in WP 8 task 8.2. In this task, we will enhance existing FAIR data checkers developed within the EOSC infrastructure to give an objective score to the data we produce, stage and redistribute in this project. Additionally, this task will connect the data of the project to the OpenAIRE graph to increase its “findability” and overall FAIRness. We will also assign PIDs to datasets generated during the project or used in it, adding them to the catalogues. 9 Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor CINEA can be held responsible for them. The UK participation in the project is funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee, and Canadian participation in the EXPECT project draws upon research supported by the Government of Canada’s New Frontiers in Research Fund (NFRF). 3.Other research outputs A dedicated ‘data science platform’, will be created and integrated with the project website (D12.2) to promote open science and ensure reproducibility and build data science capacities in the wider community. The platform will host a database of standardized benchmarks, evaluation protocols, source codes, and pre-trained machine learning models. These components will provide a centralized location for researchers and practitioners to access and replicate the benchmark datasets and machine learning models developed within the project. Scientific articles will be published in open-access (OA) journals (favouring the “golden” OA option wherever possible to comply with Horizon Europe funding rules). Manuscripts will be uploaded to preprint servers, to make the results available to the wider public as early as possible. We will also favour publication in journals that have implemented open and transparent peer-review practices, such as the journals published by the European Geosciences Union / Copernicus (e.g. Earth System Dynamics, Weather and Climate Dynamics, Hydrology and Earth System Sciences) and the Open Research Europe publishing platform of the European Commission. The code corresponding to the activities of Theme 4 will be version-controlled and stored in the GitLab instance of the BSC in a public repository (https://earth.bsc.es/gitlab/external/expect_theme4 ) that was created in August 2024 where users of the project can register on demand. 4. Allocation fo resources The data management is managed by the partners of Theme 4. There is one responsible for each institution at which data will be stored. BSC: Pierre-Antoine Bretonnière DKRZ: Stephan Kindermann