Accelerating Scientific ML Workflows with Tapis - Tutorial

Accelerating Scientific ML Workflows with Tapis Joe Stubbs1, Gilbert Curbelo1, Steve Black1, Nathan Freeman1, Christian Garcia1, Texas Advanced Computing Center, Austin, TX, USA 1{jstubbs, gcurbelo, scblack, nfreeman, cgarcia}@tacc.utexas.edu Abstract: This tutorial aims to provide researchers with an introduction to the latest reproducible Machine Learning (ML) workflows and tools available through the NSF-funded Tapis v3 Application Programming Interface (API) and User Interface (UI). Through hands-on exercises, participants will gain experience in developing ML workflows and deploying them on Jetstream resources. We will emphasize the utilization of various Tapis core APIs, alongside specialized APIs such as Tapis Workflows and Tapis Pods, all seamlessly integrated within the user-friendly TapisUI. These production-grade services are designed to simplify the creation and facilitation of trustworthy, reproducible scientific workflows. By the end of this tutorial, researchers will be empowered to efficiently develop, deploy, and maintain their own ML workflows. Description and Format: To ensure broad accessibility, training accounts will be provided to all attendees, granting access to Tapis resources regardless of their current TACC account status. All course materials, including slides and hands-on exercises, will be published on Github pages and will remain available to the attendees during and after the tutorial. The tutorial will include theoretical concepts followed by practical, hands-on-exercises. Attendees will be able to run the exercises in TapisUI, a serverless interactive science gateway that runs entirely on GitHub pages to interact with different Tapis services. In the event of an internet outage, a pre-computed example Jupyter notebook will be available, detailing the process of building and running scientific workflows. The tutorial will have a balanced mix of presentation and hands-on exercises for the attendees to learn and implement their own scientific computing workflows. Dedicated proctors will be present throughout the session to provide assistance and support. The proposed tutorial schedule is as follows: Time Block Topics 30 min 1 Introduction to Tapis (Lecture) 30 min 1 Tapis Authentication, Tapis UI (Hands-on) 20 min 1 Intro to NLP (Transformers, Hugging Face)(Lecture) 10 min Break 30 min 1 Tapis Systems, Tapis Apps (Hands-on) 30 min 1 Running an ML Tapis application using Tapis jobs sentiment analysis (Hands-on) 30 min 1 Tapis Workflows (Lecture + Hands-on) Tutorial Duration: 3 hours (including a 10-minute break) Learning Outcomes: Upon completion of this workshop, attendees will be able to: ● Authenticate with Tapis to utilize both core and advanced Tapis APIs ● Use Tapis APIs for creation, execution, and deployment of scientific ML workflows ● Construct well-defined workflows for a real-world scientific use-case ● Have a basic understanding of utilizing HPC resources for research through Tapis Target Audience: This workshop is designed for three primary categories of participants: 1) Researchers who utilize national, campus and local cyberinfrastructure resources and seek to leverage them in a reproducible, scalable, and programmable manner. 2) Cyberinfrastructure Specialists, including research software engineers (RSE), gateway providers/developers, and infrastructure administrators, who can apply open-source technologies and state-of-the-art techniques to enable portable, reproducible computation. 3) Cyberinfrastructure Directors, Managers, and Facilitators who are seeking solutions to assist and educate their institutional researchers in optimizing the utilization of local and distributed computational and cyberinfrastructure resources. Content Level: ● Beginner: 50% ● Intermediate: 30% ● Advanced: 20% Audience Prerequisites: Attendees are required to bring their own laptops for the hands-on portions of the tutorial. Participants should either have existing TACC accounts or use the provided day-of training credentials. Acknowledgement: This workshop material is based upon work supported by the National Science Foundation Plant Cyberinfrastructure Program (DBI-0735191), the National Science Foundation Plant Genome Research Program (IOS-1237931 and IOS-1237931), the National Science Foundation Division of Biological Infrastructure (DBI-1262414), the National Science Foundation Division of Advanced CyberInfrastructure (1127210), (2112606), and (1931439 and 1931575). Presentation team: Dr. Joe Stubbs is a Research Associate and leads the Cloud and Interactive Computing (CIC) group at the Texas Advanced Computing Center at the University of Texas at Austin. Dr. Stubbs is currently the PI of two NSF-funded projects and has played a fundamental role in developing numerous national-scale cyberinfrastructure systems for various scientific and engineering communities used by thousands of researchers. Previous trainings: ● PEARC 24: Trustworthy, Reproducible Machine Learning with Tapis ● PEARC 23: Best Practices of CI/CD for High Performance Computation with Tapis Workflows API Nathan Freeman is an Engineering Scientist in the Cloud and Interactive Computing (CIC) group at the Texas Advanced Computing Center at the University of Texas at Austin. Nathan manages the development of the Tapis Workflows API and related services, libraries, and UI. Previous trainings: ● PEARC 24: Trustworthy, Reproducible Machine Learning with Tapis ● PEARC 23: Best Practices of CI/CD for High Performance Computation with Tapis Workflows API Christian Garcia is an Engineering Associate in the Cloud and Interactive Computing group (CIC) at the Texas Advanced Computing Center at the University of Texas at Austin. Christian manages the Tapis Pods Service and related services, libraries, and UI. Dr. Steve Black is an Engineering Scientist in the Cloud and Interactive Computing group at the Texas Advanced Computing Center at the University of Texas at Austin. Dr. Black has played a vital role in the development of Tapis APIs. Previous trainings: ● PEARC 25: (Accepted), scheduled for July 2025: Reproducible ML Workflows and Deployments with Tapis ● Gateways 2024: Reproducible AI/ML Workflows and Deployments with Tapis ● PEARC 24: Trustworthy, Reproducible Machine Learning with Tapis ● Gateways 23: ● PEARC 23: Best Practices of CI/CD for High Performance Computation with Tapis Workflows API ● Gateways 22: Building Portable, Scalable and Reproducible Scientific Workloads across Cloud and HPC for Gateways ● PEARC 22: Building Portable, Scalable and Reproducible Scientific Workloads across Cloud and HPC ● Gateways 21: Portable, Scalable, Reproducible Scientific Computing -- from Cloud to HPC ● PEARC 20: Portable, Reproducible High Performance Computing In the Cloud Gilbert Curbelo is an Engineering Scientist in the Cloud and Interactive Computing (CIC) group at the Texas Advanced Computing Center at the University of Texas at Austin. Gilbert contributes heavily to the Tapis Authentication service and Tapis usage tracking. Resources: 1) Tapis Project: https://tapis-project.org 2) Tapis v3 documentation: https://tapis.readthedocs.io/en/latest/