scieee AI-readable full text Open interactive document viewer

de.KCD: German Competence Center Cloud‐Technologies for Data Management and Processing

Jünemann, Sebastian; Seidemann, Britta; Goesmann, Alexander; Eils, Roland; Bork, Peer; Kohlbacher, Oliver; Kummer, Ursula; Grüning, Björn; Buchhalter, Ivo; Sczyrba, Alexander

Publisher
Zenodo
DOI
10.5281/zenodo.17589764
Repository
zenodo.org
Language
en

Abstract

The German Competence Center Cloud Technologies for Data Management and Processing (de.KCD) is a cross-location and cross-domain contact point for teaching skills in data handling particularly utilizing cloud-based technologies, resources and methods for institutions, interconnected centers and researchers at all career levels. In the current phase of digital transformation, scientific and economic success as well as the connectivity of future-oriented projects depend on (i) the systematic and structured collection of relevant raw and metadata through expertise-based data management, (ii) the development and provision of innovative cloud offerings and automated workflows, (iii) the development and transfer of expertise in the field of cloud-based data processing, and (iv) the availability of a powerful and independent cloud infrastructure. The de.KCD specifically addresses these challenges by implementing suitable measures for cloud-based data management and standardized data analysis and providing and expanding the necessary hardware capacities and cloud services. It offers cloud infrastructure, storage and analysis options as well as generic training for knowledge transfer across different specialist areas, e.g. by implementing best practice examples that serve as guidelines and show how efficient and secure data management can be implemented in the cloud. The development of software stacks and workflows that are necessary for the scalable and flexible processing of distributed data sets will also be trained and actively supported. Hereby, the provision of virtual learning and working environments lower the barriers to access to cloud infrastructure. In addition, technologies are being developed to answer complex research questions and integrate data from different disciplines. Beyond these measures, de.KCD also aims to promote collaboration and knowledge exchange between research locations by creating a networked, collaborative data space for national and international research projects.

Full text

We are developing a structured training program and self‑learning units consisting of learning paths and modules, complemented by a scalable, cloud‑based training infrastructure with preconfigured learning environments: • Collection of FAIR training materials (GTN compatible formatting) • Sustainable life cycle of materials • Face‑to‑face, hybrid & online courses + summer schools & networking events • Best practices of cloud‑based RDM • Self‑learning offers & trainer lessons • Meet the Expert consulting hours • Continuous needs & self assessments • For researchers / developers / trainers at all career levels & across disciplines The de.KCD addresses the challenges of the current phase of digital transformation by bundling expertise in cloud computing and data mana gement, building upon the network, hardware capacities and services of the de.NBI Cloud. The project implements suitable measures for cloud‑based data mana gement, stan dardized data analysis and generic training content for know ledge transfer across different specialist areas. Beyond these, de.KCD also aims to promote colla boration and knowledge ex change between research loca tions by creating a net worked, colla ‑ borative data space for national and inter ‑ national research pro jects. Since the end of 2024, de.KCD has been an official partner project of de.NBI. Flexible virtualization and usage of specialized software for cloud‑based data analysis requires often dedicated solutions. To this end, scientists and data analysts from all specialist areas are actively supported in the development, establishment and usage of: • Scalable & automated processing via workflows (e.g. Nextflow & Galaxy) • Tailored software stacks (e.g. via Ansible, Docker & Conda) • Cloud‑based grid computing (e.g. BiBiGrid) • Virtual research & virtual learning environments (e.g. SimpleVM & Kallysto) • Sensitive processing in trusted environments de.KCD: German Competence Center Cloud‑Technologies for Data Management and Processing 3 The de.NBI Cloud Consortium: A. Goesmann, Justus‑Liebig‑University Giessen; R. Eils, BIH‑Zentrum Digitale Gesundheit, Charite ‑ Universitatsmedizin Berlin; P. Bork, European Molecular Biology Laboratory Heidelberg; O. Kohlbacher, Eberhard Karls University Tu bingen; U. Kumsmer, Heidelberg University; B. Gru ning, Albert‑Ludwigs University Freiburg; I. Buchhalter, Deutsches Krebsforschungszentrum Heidelberg; A. Sczyrba, Forschungszentrum Ju lich GmbH #de.KCD‑Public:uni‑bielefeld.de https://github.com/deKCD https://datenkompetenz.cloud de.kcd‑[email protected]‑juelich.de Sebastian Jünemann1 & Britta Seidemann2 on behalf of the de.KCD Consortium3 1 Institute of Bio‑ and Geosciences IBG‑5, Research Center Ju lich GmbH, c/o Centrum fu r Biotechnologie (CeBiTec), Bielefeld University, 33594 Bielefeld, Germany Training & Consulting Cloud Based RDM The Consortium 2 Center of Digital Health, Berlin Institute of Health (BIH) at Charite, 10117 Berlin, Germany For data science methods, we focus on the use of cloud‑based infrastructures for distributed and scalable data management, as well as the necessary skills for standardized and automated data processing. In accordance with the FAIR principles, this includes specific expertise for the reproducible handling of data and the use of appropriate software tools, for example, through the use of software container solutions (e.g. BioContainers) in conjunction with cloud‑based data management systems (e.g. FAIRDOM SEEK), object storage systems (e.g. Ceph RADOS) and workflow engines (e.g. nextflow). This will complement the efforts by resource providers and infrastructure initiatives within Europe (ELIXIR, EOSC) and Germany, specifically the different NFDI consortia. Services The Training Portal ‑ https://github.com/deKCD/Training‑Materials Registry of Data Management Platforms ‑ https://github.com/deKCD/FAIR‑DMP‑RegistryMaterials SimpleVM Portal for Workshops ‑ https://simplevm.denbi.de Needs Assessments ‑ https://akirbis.github.io/deKCD‑needs‑assessment Kallysto JHaaS System Overview ‑ https://jhaas.gi.denbi.de/ Ask the Expert Consultancy ‑ https://datenkompetenz.cloud/en/ask‑our‑experts/