Metadata and Data Services Activities-Functions MaDSAF Overview

Metadata and Data Services’ Activities and Functions Introduction 2 MaDSAF v02.00 6 Context 6 AF01 Identification & Contact 6 AF02 Mission & Scope 6 Digital Object Management 7 AF03 Conceive, Create, Collect 7 AF04 Deposit & Appraisal 7 AF05 Curation, Quality & Compliance 7 AF06 Discovery & Identification 7 AF07 Access 8 AF08 ReUse 8 AF09 Workflows 8 AF10 Preservation 8 AF11 Provenance & Authenticity 9 AF12 Support 9 Organisational Infrastructure 9 AF13 Governance 9 AF14 Policy & Standards 9 AF15 Rights Management 10 AF16 Resources 10 AF17 People & Expertise 10 AF18 Third Party Dependencies 10 AF19 Continuity of Service 10 AF20 External Engagement 10 AF21 Release & Publishing 11 AF22 Interoperability 11 AF23 Legal & Ethical 11 AF24 Criteria, Assessment, Improvement 11 AF25 Analysis & Impact 11 AF26 Training 11 AF27 Research & Development (R&D) 11 Technology 12 AF28 Storage & Integrity 12 AF29 Technical Infrastructure 12 Security 12 AF30 Security 12 Appendix A: Crosswalk References 13 Appendix B: MaDSAF Overview Diagram 14 1 Introduction This working paper1 defines a ‘super-set' of metadata and data services’ activities and functions (MaDSAF) derived from an analysis of the common features across current requirements, standards and criteria. There are four top-level headings: digital object management, organisational infrastructure, technology and security. Initial work on this topic was presented at the IASSIST 2022 Conference as Information Architecture for Secure Trustworthy Data Services2, which considered the suite of components necessary for an ecosystem of FAIR3 (Findable, Accessible, Interoperable, ReUsable) digital objects (data and metadata). A related document - FAIR Ecosystem Components4 - provides an infrastructure level perspective on a three-tier operational layer of digital objects, repositories and data services, and their dependency on shared registries of information. Standards, standards bodies and assessment processes/providers are the foundation for these ecosystem components. At the UK Data Service5 (UKDS), Trustworthy Digital Repository (TDR) functions are provided by the UK Data Archive6 (UKDA7) The UKDA is certified to the CoreTrustSeal8, a set of sixteen TDR Requirements9 adhered to by over 140 global repositories and a key influence on the TRUST Principles10 (Transparency, Responsibility, User focus, Sustainability and Technology). UK Data Service staff are active within the CoreTrustSeal and its Assembly of Reviewers. However, data services, including the UKDS, have responsibilities that may be wider, or requirements that are more rigorous, than those applied by TDR standards. The UKDS seeks to benchmark its activities against a wide range of relevant criteria. This comprehensive but high-level set of activities and functions relevant to metadata and data services is intended to support strategic planning and continuous improvement. More granular and specific 10 Lin, D., Crabtree, J., Dillo, I. et al. The TRUST Principles for digital repositories. Sci Data 7, 144 (2020). https://doi.org/10.1038/s41597-020-0486-7 9 https://doi.org/10.5281/zenodo.7051012 8 https://www.coretrustseal.org/why-certification/certified-repositories/ 7 Based at the University of Essex, the UKDA is the lead partner in the UK Data Service. 6 https://www.data-archive.ac.uk/ 5 https://ukdataservice.ac.uk/about/host-organisations-and-partners/ 4 FAIR Ecosystem Components https://doi.org/10.5281/zenodo.6726533 3 Wilkinson, M., Dumontier, M., Aalbersberg, I. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016). https://doi.org/10.1038/sdata.2016.18 2 Information Architecture For Secure Trustworthy Data Services https://doi.org/10.5281/zenodo.6822236 1 https://doi.org/10.5281/zenodo.17779868 [This document] 2 requirements such as FitSM11 (applied by the UKDS technical services team) and ISO27001 for information security management (to which the UK Data Archive is certified) can be aligned with these activities and functions. The development of MaDSAF was augmented with a review12 of the CoreTrustSeal Requirements to assess their general applicability to wider data services beyond TDRs. The outcome was that most of the CoreTrustSeal Requirements applied to all data services, with only Preservation Plan (Requirement 09) and ReUse (Requirement 13) applying to a more limited set of services. A review and crosswalk13 of characteristics, standards and requirements for repositories and data services from a variety of UK, European and International sources (listed in Appendix A: Crosswalk References) was used to generate a common ‘super-set’ of activities and functions14 Each activity or function can be augmented with more detailed subsets of locally applicable business processes and workflows. Each activity may be supported by business information artefacts (e.g. policies and procedures), and imply characteristics of the organisation or digital objects (expressed through metadata). If this information is made transparent to relevant parties it can act as evidence for assessment. Data and metadata services may be delivered through a wide variety of organisational models and partnerships. However, to apply standards and assess outcomes it must be possible to assign responsibilities for delivering activities and functions to a defined ‘organisational entity’. The overall model can also be used to specify and manage responsibilities and where there is a need for engagement (e.g. with data users). The table below provides a brief example of how activities and functions can be aligned with different local variables. MaDSAF Information Artefact Organisation Characteristic Digital Object Characteristic Transparency Method Engagement Method Responsibility Activity/ Function Collections Policy "Is Certified" "Is FAIR" Public Document User Survey Repository/ Third Party Supplier Table 1: MaDSAF Example Augmentations Since their initial release in 2023 the MaDSAF have been referenced by the European Open Science Cloud (EOSC) Association15Task Forces16, including those for Preservation17 and Retention18, and into the work of the sibling FIDELIS and EOSC EDEN project. The FIDELIS project has incorporated the MaDSAF into their Transparent Trustworthy Repositories Attributes Matrix (TTRAM)19, which aligns the activities and 19 FIDELIS MS12 First version of TTRAM. Zenodo. https://doi.org/10.5281/zenodo.17157958 18 https://eosc.eu/advisory-groups/long-term-data-retention-task-force/ 17 https://eosc.eu/advisory-groups/long-term-data-preservation/ 16 https://eosc.eu/eosc-association/eosc-task-forces/ 15 https://eosc.eu/ 14 Broadly, activities may be grouped into functions and some of these may be defined and delivered as internally or externally facing data and/or metadata services. This paper does not attempt to address the wide range of different interpretations of these three terms. 13 Metadata & Data Services Activities & Functions (MaDSAF): Crosswalk https://doi.org/10.5281/zenodo.17779955 12Review of CoreTrustSeal Applicability to non-Preservation (Trustworthy) Data Services https://doi.org/10.5281/zenodo.7646134 11 ‘Federated IT Service Management’ - a European standard for delivery of IT Services cf. https://www.fitsm.eu/ 3 functions with transparent repository information and the CoreTrustSeal Levels of Retention, Curation and Preservation (LoRCAP)20. In this version 2.0 of MaDSAF the identifiers, activity/function labels and descriptions have been updated to align with the work21 of the FIDELIS project and the related public consultation. The previous version of MaDSAF included examples of “Metadata Characteristics, Information Artefacts & Evidence” which have been incorporated into the FIDELIS examples of what repository information could or should be made available (“transparent”) for different ‘trust’ contexts. The body of this document provides an overview of the MaDSAF. Each brief description is not intended to conflict with or replace the more detailed descriptions provided in the source texts, standards and criteria from which they are derived. Key to the diagrams below: ● Data/metadata service provider (e.g. repository) boundary. Dotted line (red). ● External actor interactions. Ovals (green). ● Activities and Functions. Rounded boxes (white text on blue background). ● CoreTrustSeal and applicable to all services. Wider border (orange). ● Aligned with CoreTrustSeal and applicable to a subset of services. Wider dotted border (orange). All information present in the diagram below and the MaDSAF-only diagram in Appendix B is also represented in the body of the text. 21 FIDELIS TTRAMatrix v01.00 Guide (v01.00). Zenodo. https://doi.org/10.5281/zenodo.17144141 20 Curation & Preservation Levels: CoreTrustSeal Position Paper. Zenodo. https://doi.org/10.5281/zenodo.11476980 4 Diagram: MaDSAF Overview with CoreTrustSeal Alignment 5 MaDSAF v02.00 Headings that align with the CoreTrustSeal are presented in the text e.g. Deposit & Appraisal (R08). Headings that are not applicable across all repositories and data/metadata services are preceded by the text “Exception:” Each Activity or Function (A|F) is grouped under Context, Digital Object Management, Organisational Infrastructure, Technology and Security. Context Both AF01 and AF02 are delivered via ‘Organisational Infrastructure’ but are presented separately here as they provide essential context across all Activities and Functions. AF01 Identification & Contact Background Information & Context (R0) Names, abbreviations and contact details for the organisation to provide reference information and context for the rest of the activities and functions. AF02 Mission & Scope Mission & Scope (R01) Defining the purpose (or mission) of the organisation and the boundaries of activities for which the repository is responsible. Where activities involve partners or outsourced services, the organisation retains ultimate responsibility for the activities, functions and outcomes. Provides reference information and context for the rest of the activities and functions. 6 Digital Object Management Digital Object Management AF03 Conceive, Create, Collect The stages of the research data lifecycle, before data enters a repository. Working with data or metadata creators and owners can help improve quality and highlight the benefits of curation, preservation and reuse. AF04 Deposit & Appraisal Deposit & Appraisal (R08) Accepting custody of digital objects from depositors, transferring responsibility to the repository. It may also include appraising offered or requested deposits to ensure they meet established criteria for acceptance. AF05 Curation, Quality & Compliance Quality Assurance (R10) Ensuring digital objects reach a defined level of quality and standards compliance before they are made available for reuse. AF06 Discovery & Identification Discovery & Identification (R12) Applying persistent identifiers and descriptive metadata to digital objects to support resource discovery. Providing discovery systems and making metadata available for harvesting. 7 AF07 Access Determining the appropriate method of access—such as direct download or use within a secure remote environment—and facilitating user access accordingly. Access criteria are based on the characteristics of both the digital objects and the users, in alignment with rights management policies (see AF15 "Rights"). AF08 ReUse Reuse (R13) Making and keeping digital objects usable and understandable depends on deposit, curation, and preservation activities. Facilitating reuse depends on understanding the needs and expectations of users, including—but not limited to—identified ‘designated communities’, through ongoing external engagement (see AF20 "External Engagement"). Repositories may mediate reuse through secure remote access systems, safe rooms, or other tools that keep the digital objects fully or partially under the service provider's control. Exception: some repositories and data services, e.g. retention/storage-only, may not take account of the needs of those reusing digital objects’ data and or metadata. AF09 Workflows Workflows (R11) Processes and activities for managing, changing and maintaining digital objects and their associated data and metadata in line with legal, ethical, policy, and other standards and operating procedures (see AF23 "Legal & Ethical" and AF14 "Policy & Standards"). AF10 Preservation Preservation Plan (R09) Monitoring both the technology landscape (see AF29 "Technical Infrastructure") and the user community (see AF20 "External Engagement") for any changes that could impact the usability or understanding (see AF08 "Reuse") of digital objects. When necessary, preservation actions are taken—such as migrating data formats, updating metadata (e.g., revising ontologies), or emulating the environment in which the digital object is used. These actions help ensure the long-term viability and continued accessibility of the digital object. Exception: active preservation to ensure digital objects’ data and/or metadata remain understandable and reusable for the long term is not provided by all repositories or other data/metadata services. 8 AF11 Provenance & Authenticity Provenance & Authenticity (R07) Providing information about digital objects that details their history (provenance) and ensures their authenticity. Provenance and authenticity information can be captured at the point of deposit, generated during curation and preservation processes, and shared with data or metadata users. AF12 Support Providing guidance and responding to requests from depositors and users around deposit and appraisal, discovery, access and re-use of data and metadata. Organisational Infrastructure Organisational Infrastructure AF13 Governance Governance & Resources (R05) The organisational hierarchies and processes by which the entity's mission is managed and executed. AF14 Policy & Standards The adoption and development of policies, standards and other criteria to guide practice, aligned with governance structures (see AF13 "Governance") and operational workflows (see AF09 "Workflows"), and in compliance with the applicable legal and ethical framework (see AF23 "Legal and Ethical"). 9