CDH Project Charter — Bringing HTR to the HPC 2024–2025
Abstract
Project charter for 2024 Research Partnership - Bringing HTR to the HPC. The charter is the foundational document that describes the research purpose, deliverables, overall timeline, terms and conditions, and outcomes of a CDH project. Charters are written by core members of a project team in a series of planning meetings before the work starts. Led by the CDH Technical Project Manager, the planning process is intensive, collaborative, and requires substantial input from everyone on a team. Charters serve as formalized agreements among team members on such crucial questions as scope, technical design, infrastructural needs, and success criteria. At the CDH, each project charter is peer-reviewed by the Assistant Director, Executive Director, and Faculty Director. The charter has a twofold importance: 1) it defines the contribution of the CDH Research Software Engineers within the faculty partner’s larger research agenda and 2) it embodies a commitment to a set of intellectual, ethical, and social values, providing space to negotiate topics like equity and fair labor, accountability, accessibility, sustainability, and community practices. Charters are the result of a social process, establishing communication norms and setting the tone for the collaborative work that follows. CDH charters and their planning documents exist in several forms as we have refined them over the years and tailored them to the several types of projects we have supported. For more about CDH project management, including the charter process, visit: https://cdh.princeton.edu/programs/project-design/.
Full text
Project Name: Bringing HTR to the HPC Updated As Of: 2024-12-02 Page: 1 of 7 Project Charter PROJECT OVERVIEW PROJECT NAME Bringing HTR to the HPC: A Pilot to Customize eScriptorium for Princeton part of the Princeton Open HTR Initiative: Creating Infrastructure for Modeling Historical Texts PRIMARY INVESTIGATOR Marina Rustow Helmut Reimitz Christine Roughan GRANT CDH Research Partnership EXPECTED START DATE 2024-07-17 EXPECTED END DATE 2025-05-30 DATE OF DOCUMENT 2024-07-12 PROJECT DETAILS PROJECT ABSTRACT The Princeton Open HTR Initiative is a research infrastructure project to test a Princeton-specific instance of the eScriptorium handwritten text recognition (HTR) software. While the digitization efforts of recent decades have revolutionized access to historical texts in libraries, archives and cultural heritage institutions, HTR is making these digitized materials machine readable, opening them up to text search and computational analysis at scale. An increasing number of humanities scholars – including at Princeton – are eager to integrate HTR into their research workflows. But barriers to entry can be significant, particularly in technical expertise and cost. In this research partnership with the CDH, we will set up and evaluate a test instance of eScriptorium, the current leader in open-source HTR software designed by and for scholars working with historical texts. Our aim is to assess, evaluate and document what it would take to set up and maintain an infrastructure for Princeton researchers, regardless of technical background, corpus size, language, or team size to use freely, efficiently, effectively and sustainably. This phase, exploring modifications to the eScriptorium software to function in a HPC environment like Princeton’s Research Computing, is a critical first step towards realizing that goal. PURPOSE/AUDIENCE 1. Princeton researchers who will be beta testers 2. Research Computing and PUL, who will consider implementing a
Project Name: Bringing HTR to the HPC Updated As Of: 2024-12-02 Page: 2 of 7 longer-term version of the instance 3. HTR community, who will benefit from the enhancements to the eScriptorium code and learn from the Princeton model of implementing an institutional instance DELIVERABLES ● Documentation and code for working with eScriptorium in Princeton HPC environments Code Must-Have ● Internal report assessing further code developments to eScriptorium, Kraken, or related libraries that improve HTR workflows (e.g., better permissions for team projects; better handling of duplicate model files; use of ATR engines besides Kraken) Writing (articl… Should-Have ● Internal report finalizing specs and requirements for long-term management of an HTR infrastructure at Princeton Writing (articl… Must-Have ● White paper published on Zenodo on findings and recommendations for other institutions/projects Writing (articl… Must-Have ● Internal presentation to Princeton stakeholders (RC, PUL, SC) Presentation (… Must-Have OUT OF SCOPE ● Continued support by the CDH for the test eScriptorium instance after the test phase or for any of the data or models used or generated during the test phase ● Production of HTR and segmentation models ● Production of transcriptions using the eScriptorium software ● Development of code for additional enhancements to eScriptorium beyond adaption for HPC environments SUCCESS CRITERIA ● Code contributions successfully allow deployment of eScriptorium in Princeton’s HPC environment SUSTAINABILITY PLAN ● The test instance will be shut down after the Research Partnership ends. ● The code will be available on GitHub so that Princeton or other institutions can fork it and use it.
Project Name: Bringing HTR to the HPC Updated As Of: 2024-12-02 Page: 3 of 7 MILESTONES Key Milestones Est. Start Date Est. End Date Pre-partnership meetings with collaborators (PUL, RC, UPenn, U Maryland, members of eScriptorium team) 2024-01-08 2024-05-31 Project kickoff 2024-07-17 2024-07-17 Assessments of eScriptorium deployment options (e.g. manual installation, docker, Ansible) and the “slurmer” library complete 2024-07-22 2024-07-26 Test instance of base eScriptorium using the chosen approach deployed 2024-08-12 2024-08-21 Code adaptations produced for integration with HPC 2024-08-19 2025-01-22 Meeting with PUL & RC to determine organizational constraints and requirements 2024-09-06 2024-09-06 Beta testers secured 2024-08-12 2025-01-31 eScriptorium test instance ready for beta testing on library VM (cloud or local) 2024-09-09 2025-01-22 Beta users training and kickoff 2025-01-31 2025-02-07 Mid-term feedback acquired from beta testers 2025-03-07 2025-03-21 End-term feedback acquired from beta testers 2025-04-22 2025-05-06 Internal report(s), white paper, and presentation written 2025-05-07 2025-05-30 Project closeout 2025-05-07 2025-05-30 DEPENDENCIES & RISKS DEPENDENCIES ● HPC access ● Potential TigerData integration ● Potential external software library (slurmer) ● PUL VMs
Project Name: Bringing HTR to the HPC Updated As Of: 2024-12-02 Page: 4 of 7 POTENTIAL RISKS ● The software might ultimately not work with existing Princeton infrastructure. ● The test instance might not be taken up for longer-term support. ● Beta users will need to be clear that this is a testing phase of the software; they will be advised to download any data they care about before the test instance is shut down. RIGHTS + PERMISSIONS ● eScriptorium code is open-source and licensed under The MIT License. PROJECT TEAM Name Role on Project Dept Role Description Marina Rustow Project Co-Lead NES ● Participate in planning discussions ● Test and review the Scriptorium instance Helmut Reimitz Project Co-Lead HIS ● Participate in planning discussions ● Test and review the Scriptorium instance Christine Roug… Project Co-Lead Project Manager CDH ● Coordinate planning meetings and community engagement ● Provide feedback on the technical development ● Co-author (with the RSE) the internal report(s) and white paper Rebecca Koeser RSE CDH ● Attend planning meetings as needed ● Review and test eScriptorium code and related tooling (potentially including, python API library for eScriptorium, Kraken OCR library, Slurmer) ● Coordinate installation of a test instance of eScriptorium ● Identify significant points where software engineering could improve an HTR pipeline built around eScriptorium and Kraken ● Identify the changes needed to deploy a long-term, supportable instance of eScriptorium at Princeton ● Co-author (with Christine) the internal report(s) and white paper Mary Naydan Technical Project Manager CDH ● Help define, queue, and prioritize work for RSE ● Advise Project Manager
Project Name: Bringing HTR to the HPC Updated As Of: 2024-12-02 Page: 5 of 7 Jeri Wieringa Project Advisor CDH ● Advise on the project BUDGET Item Funding Source Cost [budget available upon request] xxxxxxxxxxxxxxxxxxx xxxxxxxxxxx TERMS OF COLLABORATION DOCUMENTATION + COMMUNICATION This research partnership will store its meeting notes and documentation in a shared folder within the CDH Research Partnership shared Google Drive. Asana will be used for task assignment and a Slack channel will be created for technical communications between the project RSE and project manager. The project manager will circulate high level overviews (progress updates and meeting summaries) to all members of the team via regular email. A private Github repository under the Princeton-CDH organization will be used for managing technical issues, documentation, and code. There are a variety of meetings expected / planned over the course of this project: ● biweekly internal team meetings ● conversations with Princeton units that provide necessary or relevant infrastructure or services (Research Computing, PUL) ● meetings with peer institutions that have established local instances of eScriptorium ● discussions with the eScriptorium team CREDIT + ATTRIBUTION Rebecca will be credited for any code or documentation contributed to eScriptorium or other associated projects. All contributors will be credited as authors on the written outputs (preliminary/anticipated authorship: Christine, Rebecca as co-authors for internal report(s); Christine, Rebecca, Helmut and Marina as co-authors for the white paper). PUBLICITY The CDH Communications Manager may publicize the project and related information on the CDH website and newsletter, in annual reports, and on select social media (Bluesky, Instagram). The Communications Manager may also work with Princeton’s Office of Communications to publicize the project on Princeton’s main website or in its magazines.
Project Name: Bringing HTR to the HPC Updated As Of: 2024-12-02 Page: 6 of 7 EXTENSIONS The PI and RSE may request an extension beyond the allotted partnership timeframe by submitting a written rationale to the CDH PM and the CDH AD. Extensions may also be proposed and automatically granted by the CDH PM and CDH AD due to unforeseen delays in the CDH development schedule. AMENDMENTS Amendments to the charter should be requested in writing to the project team members, the CDH PM, and the CDH AD Changes will be recorded in the “Version History” section below. Please also update the date in the charter header. SIGNATORIES PREPARED BY Mary Naydan Christine Roughan 2024-05-24 Original draft here. REVIEWED BY Jeri Wieringa 2024-06-22 REVIEWED BY Rebecca Koeser 2024-07-03 APPROVED BY Jeri Wieringa Natalia Ermolaev Rebecca Koeser Christine Roughan Mary Naydan 2024-07-17 APPROVED BY PI Helmut Reimitz 2024-07-22 APPROVED BY CDH FACULTY DIRECTOR Meredith Martin 2024-07-24 VERSION HISTORY UPDATED BY Mary Naydan Christine Roughan 2024-07-12 Put original draft into new charter template & revised to incorporate JW & RSK suggestions. UPDATED BY Christine M. Roughan 2024-08-12 Adding line for RSE salary to budget, already accounted for by PLI seed grant.
Project Name: Bringing HTR to the HPC Updated As Of: 2024-12-02 Page: 7 of 7 UPDATED BY Christine Roughan Mary Naydan 2024-12-02 Adjusted RP end date and milestone dates to reflect extensions granted here: Re Extension for HTR2HP… Recommended Citation Naydan, Mary, Christine Roughan, Rebecca Sutton Koeser, Helmut Reimitz and Marina Rustow. CDH Project Charter – Bringing HTR to the HPC 2024. Center for Digital Humanities at Princeton. 2024. http://doi.org/10.5281/zenodo.17344496.