D1.2 – Equify Data Architecture

1 | P a g e Establishing a Framework for Quantifiable Evidence and Impact of Ecosystem ChangeThroughout the Lifecycle of UK Floating Offshore Wind Farms (EQUIFy) WP1 – Co-Development & Co-Delivery D1.2 – Equify Data Architecture Author: Thomas Mansfield ([email protected]) Version: 1.0 Date: 1st October 2025 DOI: 10.5281/zenodo.17178362 EQUIfy is part of the ECOFlow program, funded by UKRI (NE/Z504099/1) and The Crown Estate. 2 | P a g e Executive Summary This document is the first issue or a living document that sets out the initial design and vision for the EQUIFy Data Architecture, a key deliverable within the ECOFlow program. Its primary aim is to provide a central, authoritative resource for identifying all datasets generated throughout the project, ensuring that each dataset is clearly documented with licensing and sharing guidance. By doing so, the document supports all project partners in understanding and making full use of the data sharing infrastructure available to them. The architecture is built on a federated model, allowing data creators to retain full control over their datasets by hosting them on their own infrastructure. Rather than centralizing data storage, the system enables partners to register their datasets with the project, making them discoverable and accessible through a central ERDDAP data portal managed by Plymouth Marine Laboratory (PML). This approach ensures that data ownership, access policies, and updates remain with the data providers, while still enabling seamless discovery, access, and integration for all project partners. ERDDAP acts as the central access point, offering both user-friendly tools and robust APIs for programmatic access. It allows users to search, subset, and combine datasets, and to download data in a variety of formats, with licensing and citation guidance provided for each dataset. Where necessary, authentication can be configured to control access to sensitive data. The architecture also aligns metadata with external standards, such as those from the NERC Vocabulary Services, supporting interoperability and advancing the FAIR data principles. Importantly, this document is intended to serve as a practical guide for all project partners, helping them to understand, access, and utilize the data sharing infrastructure. There is also a potential opportunity to expand this federated architecture to support the wider ECOFlow program, should that be required in the future. Finally, the project is committed to continuous improvement. Lessons learned throughout the implementation and operation of the data architecture will be collected and incorporated into future updates, ensuring that the system evolves to meet the needs of all stakeholders and maximizes the impact of the project’s data resources. This living document will be updated throughout the EQUIFY project to capture the evolving needs of the EQUIFy data community. A final issue is expected to be issued in late 2028. 3 | P a g e Contents Executive Summary ................................................................................................................................ 2 Contents .................................................................................................................................................. 3 1 Introduction ...................................................................................................................................... 4 1.1 Scope ...................................................................................................................................... 4 1.2 EQUIFy Data Management and Sharing Considerations ....................................................... 4 1.2.1 Data Management, Use and Sharing ................................................................................ 4 1.2.2 EQUIFy Project Data Sharing License .............................................................................. 4 1.2.3 Expected EQUIFY Data .................................................................................................... 4 1.2.4 Data Retention and Disposal ........................................................................................... 10 2 EcoFlow Data Architecture Design ................................................................................................ 11 2.1 High level needs for the EQUIFy data architecture............................................................... 11 2.2 Objectives .............................................................................................................................. 11 2.3 Architecture Design; A federated approach .......................................................................... 12 2.3.1 Architecture Design – In-situ data federate ..................................................................... 12 2.3.2 Architecture Design – Remote sensing and model data federate ................................... 14 2.3.3 Architecture Design – Socio-economic data federate ..................................................... 14 2.4 Opportunities for expansion beyond the current project scope ............................................ 15 3 Implementation of the data architecture - A worked example ...................................................... 16 4 Future plans and next steps .......................................................................................................... 18 5 References ..................................................................................................................................... 19 Annex A: EQUIFY data architecture; Opportunities for a multi-institute data staging area ............ 20 4 | P a g e 1 Introduction 1.1 Scope This document forms the first of two planned deliveries of the EQUIFy Data Architecture. The document describes the planned EQUIFy data architecture, providing a guidebook to project members and external stakeholders on managing data related to the project throughout their lifecycle. Particular emphasis is placed on describing the responsibilities, processes and tools available to the EQUIFy team to support the creation of rich metadata and enable the controlled sharing of emerging datasets with project partners and external stakeholders. This document is intended to compliment and support the EQUIFy Data Management and Sharing Plan, a document that is managed by the British Oceanographic Data Centre (BODC) in close collaboration with the EQUIFy team, to summarise staff responsibilities, data collection policies, data quality and arrangements for long-term management of data beyond the project. The EQUIFy data architecture plan aligns with PML’s Scientific Data Management Policy [1] and UKRI’s Natural Environment Research Council’s data sharing policy [2]. An updated version of this document will be issued once the majority of the EQUIFy data collection activities have been completed (M44), providing an implementation record for the data architecture described in this document, along with a series of lessons learned. 1.2 EQUIFy Data Management and Sharing Considerations Data management and sharing considerations are structured across the data lifecycle. 1.2.1 Data Management, Use and Sharing Initially, PML’s ERDDAP [3] service will enable project partners and external users to access locally stored data in near real time with additional data brokers provided where required. Subsequently, quality checked and processed data sets will be archived at the relevant data centres, providing a persistent findable, accessible, interoperable and reusable (FAIR) [4] data resource. 1.2.2 EQUIFy Project Data Sharing License To support the timely sharing of data within the EQUIFy project, the following project licence text has been developed, providing two project specific licences that can be utilised by data owners [Option 1] Permission is granted to EQUIFy project partner institutes and interested parties to use this dataset for research and analytical purposes only. Redistribution, republication, or any form of data sharing with unauthorized third parties is strictly prohibited without prior written consent from the data owner. All users must acknowledge the source of the dataset in any publications or outputs. [Option 2] Permission is granted to EQUIFy project partner institutes to use this dataset for research and analytical purposes only. Redistribution, republication, or any form of data sharing with unauthorized third parties is strictly prohibited without prior written consent from the data owner. All users must acknowledge the source of the dataset in any publications or outputs. 1.2.3 Expected EQUIFY Data Table 1 contains an overview of the data expected t be created or modified by the EQUIFy project, with key the key attributes listed that will allow the data to be correctly managed, distributed and projected throughout the subsequent stages of the project life cycle. 5 | P a g e Table 1 - Expected EQUIFy Datasets Dataset Description Dataset Contact Name(s) Estimated Data Volume Data Format Licence Intellectual Property Rights (IPR) Contact Data Archiving Centre DOI Required FLOW engineering model data for momentum and TKE parameterisations T. Nishino ≈ 10 GB TBC (nonproprietary formats preferred) • Within 6 months of data generation: Project licence (1.2.2 [Option 1]) • After 6 months of data generation: OGL [5] University of Oxford Oxford University Research Archive (ORA) Yes Celtic Sea ocean and ecosystem model (FVCOM-ERSEM) outputs J. Clark ≈ 150 GB (volume may increase depending on the number and length of scenarios) .nc • Within 6 months of data generation: Project licence (1.2.2 [Option 1) • After 6 months of data generation: OGL [5] Plymouth Marine Laboratory (PML) CEDA (Archival managed by BODC) Yes Plankton Imaging (IFCB and Pi-10 data) J. Clark ≈ 200 GB .csv and .png OGL [5] PML BODC (TBC) Yes USV NRT ecosystem monitoring data – CTD, Chla, met-underway J. Wihsgott ≈ 500 MB .nc • Within 6 months of data generation: Project licence (1.2.2 [Option 1) • After 6 months of data generation: OGL PML BODC Yes USV delayed mode ecosystem monitoring data – CTD, Chl-a, metunderway J. Wihsgott ≈ 500 MB .nc • Within 6 months of data generation: Project licence (1.2.2 [Option 1) PML BODC Yes 6 | P a g e Dataset Description Dataset Contact Name(s) Estimated Data Volume Data Format Licence Intellectual Property Rights (IPR) Contact Data Archiving Centre DOI Required • After 6 months of data generation: OGL [5] USV ecosystem monitoring data – Fishery Acoustics B. Willamson ≈ 100’s GB (Raw data) ≈ 100 MB (data products) TBC (nonproprietary formats preferred) • During the project: Project licence (1.2.2 [Option 2) • At project completion: Project licence (1.2.2 [Option 2) • 2 years after project completion: OGL University of the Highlands and Islands (UHI) None for raw data. (Note: There is a lack of standardisation and these are large datasets). Data products will be archived alongside publications. No (Except for data products that support publication) NRT glider data: T,S, DO, Chl, CDOM, BBP MARS ≈ MB .nc .tbd, .sbd, .cac OGL PML BODC No Recovery Glider Data – (i.e. the data from the glider SD card when the glider is recovered MARS ≈ MB & GB for turbulence data .ebd, .dbd., cac.. PD0 • During the project: Project licence (1.2.2 [Option 1) At project completion: Project licence (1.2.2 [Option 1) • 2 years after project completion: OGL PML BODC No Delayed Mode Glider Data: TS. DO, Chl, DOM, BPP, TKE dissipation J. Wihsgott ≈ 5 GB .nc & nonnonproprietary format • During the project: Project licence (1.2.2 [Option 1) At project completion: Project licence (1.2.2 [Option 1) PML BODC Yes 7 | P a g e Dataset Description Dataset Contact Name(s) Estimated Data Volume Data Format Licence Intellectual Property Rights (IPR) Contact Data Archiving Centre DOI Required • 2 years after project completion: OGL WCO L4 datasets (Inc. CTD, ocean currents, meteorological data and weekly biogeochemical sampling) T. Mansfield MBs .csv OGL [5] PML BODC Yes (DOI assigned to annual additions) eDNA Data J. Robidart MBs .csv • Within 6 months of data generation: Project licence (1.2.2 [Option 1) • After 6 months of data generation: OGL [5] National Oceanography Centre (NOC) National Center for Biotechnology Information (NCBI) and the Ocean Biodiversity Information System (OBIS) (TBC) Yes, is data archived within OBIS Regional Environmental Prediction (REP) model outputs S. Berthou ≈ 100 TB total (Approx 250 GB used by the project) .nc • Within 6 months of data generation: Project licence (1.2.2 [Option 1) • After 6 months of data generation: OGL [5] MetOffice MetOffice DAC No Moored instrument data – Fishery Echosounder B. Williamson ≈ 1 TB TBC (nonproprietary formats preferred) • During the project: Project licence (1.2.2 [Option 2) • At project completion: Project licence (1.2.2 [Option 2) UHI None for raw data. (Note: There is a lack of standardisation No (Except for data products that support publication) 8 | P a g e Dataset Description Dataset Contact Name(s) Estimated Data Volume Data Format Licence Intellectual Property Rights (IPR) Contact Data Archiving Centre DOI Required • 2 years after project completion: OGL and these are large datasets). Data products will be archived alongside publications. Moored instrument data – Current velocities (Raw) J. Wihsgott and B. Williamson ≈ 2 GB or more TBC (nonproprietary formats preferred) • During the project: Licence [Option 2] • At project completion: Licence [Option 2] • 2 years after project completion: OGL Marine Directorate (MD) / UHI BODC (submitted with processed data) No Moored instrument data – Current velocities (Processed) J. Wihsgott ≈ 500 MB .nc • During the project: Project licence (1.2.2 [Option 1) • At project completion: Project licence (1.2.2 [Option 1) • 2 years after project completion: OGL PML BODC Yes Moored instrument data – CTD J. Wihsgott ≈ 500 MB .nc • During the project: Project licence (1.2.2 [Option 1) • At project completion: Project licence (1.2.2 [Option 1) MD / UHI BODC Yes 9 | P a g e Dataset Description Dataset Contact Name(s) Estimated Data Volume Data Format Licence Intellectual Property Rights (IPR) Contact Data Archiving Centre DOI Required • 2 years after project completion: OGL Moored instrument data – Chl-A J. Wihsgott ≈ 500 MB .nc • During the project: Project licence (1.2.2 [Option 1) • At project completion: Project licence (1.2.2 [Option 1) • 2 years after project completion: OGL MD / UHI BODC Yes Moored instrument data – Oxygen J. Wihsgott ≈ 500 MB .nc • During the project: Project licence (1.2.2 [Option 1) • At project completion: Project licence (1.2.2 [Option 1) • 2 years after project completion: OGL MD / UHI BODC Yes Moored instrument data – Cetacean click detector B. Williamson ≈ 1 TB TBC (nonproprietary formats preferred) • During the project: Project licence (1.2.2 [Option 2) • At project completion: Project licence (1.2.2 [Option 2) • 2 years after project completion: OGL UHI None (No known data archiving centre for passive acoustic data). Data products will be made available alongside publications where required. No 16 | P a g e 3 Implementation of the data architecture - A worked example At the time of writing this report, the fist EQUIFy data collection activity has recently drawn to a close. The collection of initial datasets provides an opportunity to provide a worked example of the planned dataset, providing an tangible example of how data management throughout the EQUIFy project will be managed. The data collection that will be described in detail is the deployment of an Acoustic Doppler Current Profiler (ADCP) sensor deployed from 14th August 2025. Upon recovery of the sensor, the raw data file was transferred onto servers within PML and processed to provide a useable dataset. These data have been linked to an ERDDAP, with metadata developed in collaboration with the responsible scientist and the EQUIFy data architect. These data, with accompanying metadata, are now available to be shared throughout the EQUIfy consortium. An overview of this initial data pipeline is shown in Figure 5 and Figure 6 which provide an abstracted view of the data pipeline that can be compared to the data architecture designs and a set of screenshots that depict the specific implementation respectively. Figure 5 - An abstracted view of the ADCP data pipeline 17 | P a g e Figure 6 - Screenshots of the data pipleine implementation. As an example of the services provided to EQUIFy data users, the ADCP data can be viewed at the QR code displayed in , with the discovery metadata available from the QR code provided in Figure 7 and Figure 8. Figure 7 - QR code providing example ADCP data from the EQUIFy project Figure 8 - QR code providing discovery metadata for the example ADCP data. 18 | P a g e 4 Future plans and next steps This report represents the first of two planned document releases for the project. It outlines the design and implementation of the federated data architecture, guiding data access, interoperability, and control. A second version of this document, planned for M44 of the EQUIFy project, will provide a detailed record of how individual datasets were configured within the architecture. This will include documentation that supports scientists in locating, accessing, and using the data most relevant to their research. The configuration records will enhance transparency and will serve as a practical guide for engaging with the federated system. Finally, a lessons learnt process will be initiated throughout the implementation of this data architecture. This process will capture observations from across the project, identifying both successful strategies and areas where improvements can be made. These insights will be analysed and used to inform future projects, contributing to the ongoing evolution and refinement of the federated data architecture approach. Together, these two document releases aim to provide a comprehensive view of the architecture and its application, while supporting continuous improvement and knowledge sharing across the scientific data community. 19 | P a g e 5 References [1] Plymouth Marine Laboratory, “PML Scientific Data Management Policy,” PML, Plymouth, UK, 2023 (Version 5). [2] M. Thorley and S. Callaghan, “NERC Data Policy - Guidance Notes,” UNRI Natural Environment Research Council, 2019. [3] National Ocanic and Atmospheric Administration (NOAA), “ERDDAP > Information,” NOAA, [Online]. Available: https://www.ncei.noaa.gov/erddap/information.html. [Accessed 22 September 2025]. [4] Mark D. Wilkinson et al., “The FAIR Guiding Principles for scientific data management and stewardship,” Scientific Data, vol. 3, no. 18, 2016. [5] UK Government, “Open Government Licence (V3),” The National Archives, [Online]. Available: https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/. [Accessed 22 September 2025]. [6] DAMA International, Data Management Body of Knowledge (DAMA BoK), Vancouver, USA.: DAMA International, 2017. [7] T. Mansfield; J. Wihsgott; J. Skakala; P. Menon; E. Gardner; S. Kay; and M. Palmer, , “Lessons Learned in Establishing an Environmental Digital Twin,” in IEEE OCEANS, pp. 1-10, doi: 10.1109/OCEANS58557.2025.11104760, Brest, France., 2025. [8] BODC, “NERC Vocabulary Server (NVS),” BODC, 2023. [Online]. Available: https://www.bodc.ac.uk/resources/products/web_services/vocab/. [Accessed June 2023]. [9] MEDIN Standards Working Group, “Guidance notes for the production of discovery metadata for the Marine Environmental Data and Information Network,” MEDIN Standards Working Group, Southampton, UK., 2022. [10] Plymouth Marine Laboratory, “PML Model data portal,” PML, [Online]. Available: https://portal.ecosystem-modelling.pml.ac.uk/. [Accessed September 2025]. [11] NEODAAS, “NEODAAS Data Portal,” NEODAAS is hosted at Plymouth Marine Laboratory, overseen by the National Centre for Earth Observation (NCEO) and funded by the Natural Environment Research Council (NERC), 2024. [Online]. Available: https://data.neodaas.ac.uk/visualisation/. [Accessed September 2025]. Annex A: EQUIFY data architecture; Opportunities for a multi-institute data staging area This annex contains a slide pack that has been presented at various EQUIFy and ECOFLOW program meetings to describe the opportunities and potential benefits of expanding the EQUIFy data architecture across the complete EQUIFY project portfolio.