NDSA 2021 Fixity Survey

2021 Fixity Survey Report An NDSA Report Results of the 2021 Fixity Survey October 2021 Authored by the 2021 Fixity Survey Working Group Carol Kussmann (co-chair), University of Minnesota Libraries Sibyl Schaefer (co-chair), UC San Diego Robin Dean, Michigan State University Katherine Fisher, Ph.D., Emory University Martin Gengenbach, National Library of New Zealand Kimberly Gianfrancesco, Vassar College Nick Krabbenhoeft, New York Public Library Jenny Mitcham, Digital Preservation Coalition Patrice-André Prud’homme, Ph.D., Oklahoma State University 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 2 Table of Contents About the NDSA 3 Executive Summary 4 Introduction 5 Background 5 Recent Developments 6 2021 Fixity Survey 7 Methodology 8 Data files 100 Codebook 100 Findings 111 Section 1: The Basics 111 Section 2: Using Fixity Information 19 Section 3: Cloud Services 411 Section 4: Fixity Failures 48 Section 5: Demographic Information 555 Discussion and Analysis 58 Differences and similarities between 2021 and 2017 58 New 2021 questions 600 Manual processes 611 Cloud vendor usage 611 Failures 622 Case Studies 622 Large Academic Library with an Emerging Fixity Program 633 Small Nonprofit Archives with Significant Audiovisual Holdings and Emerging Fixity Program 645 Data Repository with Large Content Volume and Established Fixity Program 67 National Archives with Large Content Volume and Established Fixity Program 69 Conclusion 712 Recommendations for Future Surveys 723 Suggestions for survey questions 723 Suggestions for survey methodology 744 Appendix 1: Interpreting survey results for this report 745 Appendix 2: Survey questions 77 Appendix 3: Crosswalk between 2021 and 2017 survey questions 856 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 3 About the NDSA Founded in 2010, the NDSA is an international membership organization that supplies advocacy, expertise, and support for the preservation of digital heritage. The NDSA promotes a vision in which all digital material fundamentally important to our cultures receives appropriate, effective, and sustainable stewardship care from the international preservation community to protect and enhance its persistent value, availability, and (re)use. NDSA member institutions represent all sectors, and include universities, consortia, non-profits, professional associations, commercial enterprises, and government agencies at the federal, state, and local levels. More information about the NDSA is available at https://www.ndsa.org. Copyright © 2021 by NDSA. This work is licensed under a Creative Commons Attribution ShareAlike 4.0 International License. DOI: 10.17605/OSF.IO/2QKEA 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 4 Executive Summary The digital preservation community has long recognized the importance of fixity information in enabling and facilitating digital preservation activities. In particular, fixity information is used to review digital content to ensure that its bit-level representation remains unchanged over time, thus proving that the content (and indeed the digital preservation processes that manage and maintain it) can be trusted. To enable greater understanding about how fixity information was employed in practice, in 2017 the NDSA carried out a survey to gather this information from the community. The 2017 Fixity Survey Report summarized the results of the survey and provided a valuable snapshot of community fixity practices.1 Understanding that digital preservation is an emerging discipline and practices evolve over time, it was anticipated that the 2017 Fixity Survey would not be a one-time exercise and that future surveys would create a longitudinal dataset to increase our understanding of this evolving field. The NDSA Fixity Survey Working Group was re-established in 2021. Survey questions from 2017 were reviewed and new questions were added to cover additional areas of interest. To enable analysis of trends and evolving practices between the 2017 and 2021 surveys, a crosswalk was established. A total of 166 survey responses were recorded, of which 116 completed surveys were used for analysis. Several key points can be made from studying the survey results: ● The results demonstrate just how important fixity information is to the digital preservation community, with over 96% of survey respondents confirming that they utilize fixity information within their organization and over 98% of these using checksums (sometimes alongside other types of fixity information). The primary reason fixity information is used by the community is to determine whether data has been altered over time. ● Despite a clear consensus that the use of fixity information represents good practice, the results demonstrate huge variation in fixity practices across the community. There are a variety of practices reported across the survey questions, including at what point fixity information is verified, the frequency of checks, where fixity information is recorded, and the checksum algorithms in use. ● Variations in fixity practices within an organization are also common, with over 48% of respondents reporting that different fixity practices are employed for different content or media. ● The importance of recording and verifying fixity information is clear. Though nearly 27% of respondents never saw fixity checks fail, failures occasionally occurred for others and nearly 11% reported seeing fixity 1 NDSA Fixity Survey Working Group, “2017 Fixity Survey Report. An NDSA Report,” National Digital Stewardship Alliance, 2017, p. 4, https://osf.io/grfpa/. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 5 failures multiple times per year. Interrupted network transfers were reported as the most common reason for fixity failures. ● Receiving fixity information at the time of acquisition remains a challenge. ● Though fixity checking lends itself well to automation, for many it remains a fairly manual process, with a majority of respondents using manuallyrun software to carry out this activity. The Fixity Survey Working Group conducted follow-up interviews with some organizations to explore fixity practices in more detail. The resulting case studies, included within this report, provide a rich illustration of how fixity is used within specific organizations, and build on some of the findings of the survey itself. Introduction Background Fixity checking, also known as integrity checking, is a key element of digital preservation and is defined as the practice of reviewing digital content to ensure that it remains unchanged over time. By monitoring the fixity of digital content, organizations can provide assurance that they hold the authentic digital objects that they have been charged with preserving and that those objects have not been accidentally or deliberately altered or tampered with. Fixity checking is an essential element of bitstream preservation and can be used by an organization to demonstrate the trustworthiness of digital content and of the organization’s own professional practices. In 2017, the NDSA put out a survey to learn more about fixity practices across the digital preservation community. The 2017 survey and subsequent report provided an overview of key developments and publications relating to fixity checking.2 By reporting on patterns and trends from the 89 survey respondents, the report provided a window into fixity practices across the community. It was apparent from the 2017 survey that the vast majority of respondents were fixity checking their content (or planning to do so) and that this was recognized as good practice. However, the details of fixity checking practices in use (for example, software used or frequency of checks) varied widely. The 2017 survey results are referenced frequently in this report, particularly in respect to how practices have changed over time. Recent Developments Since the 2017 report was published, there have been a few relevant developments in this area. The 2017 report discusses the NDSA Levels of 2 NDSA Fixity Survey Working Group, “2017 Fixity Survey Report. An NDSA Report,” National Digital Stewardship Alliance, 2017, https://osf.io/grfpa/. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 6 Preservation and in particular the “File fixity and data integrity” functional area, which provides guidance on when to create and check fixity values. Since the publication of this report, the NDSA Levels of Preservation has been revised and several changes were made to this functional area (now known by the shorter heading of “Integrity”).3 FIGURE 1: Revised ‘Integrity’ row from Version 2 of the NDSA Levels of Preservation Matrix Though many of the key recommendations around fixity checking remain the same, the Levels of Preservation now includes recommendations to document fixity checking processes and their outcomes as well as a new requirement to back up and store fixity information separately from the content that it describes. In 2019, the Digital Preservation Coalition (DPC) released a new maturity model called the Rapid Assessment Model or DPC RAM.4 Like the NDSA Levels of Preservation, this model encapsulates digital preservation good practice in a framework for assessment and continuous improvement. The “Bitstream preservation” section of DPC RAM includes processes to monitor the integrity of digital content. Some examples from the model of what good practice around fixity activities might look like are as follows: Level Examples included within the DPC RAM model 2 - Basic ● Checksums are generated for all content. 3 - Managed ● Content is managed with a combination of integrity checking and content replication to one or more locations. ● Decisions on the frequency of integrity checking and the number of copies held take into consideration risks, value of the content and costs (both financial and environmental). ● Content failing integrity checks is repaired. 3 NDSA Levels of Preservation Working Group, “Levels of Digital Preservation,” National Digital Stewardship Alliance, 2019, https://ndsa.org/publications/levels-of-digital-preservation/. 4 Digital Preservation Coalition, “Digital Preservation Coalition - Rapid Assessment Model,” March 2021, https://www.dpconline.org/digipres/implement-digipres/dpc-ram. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 7 4 - Optimized ● Content integrity and processes to ascertain integrity are independently reviewed. TABLE 1: Relevant examples relating to fixity as recorded in the “Bitstream preservation” section of DPC’s Rapid Assessment Model The DPC RAM reflects the idea that fixity checking is closely linked with storage (thus the frequency of checks needs to be appropriate to the number of copies held and the frequency of content replication). It also recommends that decisions on frequency of checks and number of copies held should take into consideration a number of factors, including the risks, the value of the content, and costs (both financial and environmental). Further developments since the publication of the 2017 report include the publication of a new DPC Technology Watch Guidance Note in 2020 by Matthew Addis entitled “Which Checksum Algorithm Should I Use?”5 This short report was intended to answer one of the perennial questions in digital preservation and also provides helpful background information on checksums, why we should use them, and what tools can help us carry out fixity checking operations. The report also discusses where checksums should be kept, adding weight to the NDSA Levels of Preservation recommendation to store more than one copy and to keep them in different locations: “Put simply, just like your data, you should keep your checksums safe and secure.”6 2021 Fixity Survey The 2017 Fixity Survey was not intended to be a one-time exercise. It was recognized that there was value in surveying the community periodically to create a longitudinal dataset that could be used to monitor and report on changing practices. A proposal to start up the NDSA Fixity Survey Working group again to survey organizations stewarding digital content and use the resulting data to produce NDSA’s second fixity survey was approved by the NDSA Leadership team in March of 2021. The objectives of this survey reflected those of the 2017 survey and included the additional aim of comparing survey results with those gathered previously, thus providing some reporting and analysis on the changing digital preservation landscape. Survey objectives were as follows: ● Identify the fixity practices that institutions are employing ● Identify difficulties in employing fixity practices ● Continue to collect longitudinal data around fixity practices 5 Matthew Addis, “Which Checksum Algorithm Should I Use?” December 2020, http://doi.org/10.7207/twgn20-12. 6 Ibid. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 8 Methodology Survey questions were based on the question set used for the 2017 Fixity Survey. The new survey retained the basic structure of the 2017 survey but incorporated a new section to explore how fixity failures were handled. The Fixity Survey Working Group reviewed and discussed the 2017 questions and made further changes and additions where appropriate. Given the desire for longitudinal analysis of results across surveys, changes to existing questions were kept to a minimum whenever possible. The Working Group maintained a crosswalk7 to ensure that corresponding questions in 2017 and 2021 could be easily compared. In addition to gathering basic demographic information about respondents’ institutions, the survey contained four main sections: 1. What types of fixity information are used 2. How fixity information is used 3. How fixity practices are impacted by cloud storage 4. How fixity failures are handled In addition to questions covered by the 2017 survey, the following new topics were also introduced: ● What types of fixity information are utilized? ● Are different fixity practices employed for different types of content and/or storage media? ● How often do fixity checks fail, and why? ● Which actions are taken to address fixity failures? The survey included 40 questions in total, with a mixture of different question types, including several optional, open-ended questions that aimed to capture the nuance of local practice and the reasons for that practice. Survey logic was in place to ensure that relevant follow-up questions were displayed to respondents based on previous answers. The survey logic is documented in the codebook. Members of the Working Group sought participation from the global digital preservation community (including and reaching beyond NDSA member institutions). The survey announcement and reminders were sent to many professional listservs and groups to solicit participation as well as circulated through various channels on Twitter. A blog post announcing the release of the survey and encouraging participation was posted on the NDSA blog.8 A full list 7 A crosswalk of the 2017 and 2021 questions is provided as an appendix to this report. Some general changes made throughout include the following: the word “organization” was changed to “institution”; where appropriate, the phrase “collect fixity information” was changed to “receive fixity information”; where appropriate, the phrase “create fixity information” was changed to “capture fixity information’; and, where appropriate, the phrase “check fixity information” was changed to “verify fixity information.” 8 “It’s here, the 2021 NDSA Fixity Survey!” NDSA Fixity Survey Working Group, May 19, 2021, https://ndsa.org//2021/05/19/it-s-here-the-2021-ndsa-fixity-survey.html. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 9 of locations where the survey was announced is provided in the codebook. The survey was open to any institutions that steward digital collections and participation was voluntary. The survey was open and available for completion through Qualtrics from May 19, 2021, to June 20, 2021. The survey preamble gave participants a brief description of what fixity information is (for example, file manifests, file sizes, and cryptographic checksums). This holistic definition of fixity information was a new addition to this survey. An assumption of the 2017 survey was that fixity information relates only to checksums, but the 2021 survey acknowledged that other types of fixity information exist and may be used and actively monitored by digital preservation practitioners. Participants were informed that survey questions pertain primarily to preservation copies of files rather than access copies or other manifestations. It was also stressed that survey questions pertain to digital content that is managed for long-term preservation rather than working files or otherwise unmanaged content. After preliminary analysis of the survey responses, the Working Group conducted interviews with representatives of four organizations (three of which completed the survey) identified as representing a diverse range of fixity use cases. In addition to answering individualized questions prompted by their initial survey responses or comments during their interview, each case study participant responded to five general questions: ● Please provide a broad overview of how fixity is used in your organization. ● What about fixity practices do you find challenging in your organizational context? ● In an ideal world, how would your organization capture and manage fixity information? ● Has the NDSA fixity survey helped you to think about your organization’s fixity practices? If so, how? ● Do you consider any part of your organization’s fixity practices to be innovative or unique? Why, or why not? These conversations, summarized in the Case Studies section of the report, illustrate some of the nuances, challenges, and rationales behind real-world fixity practices. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 16 FIGURE 5: Responses to “Does your institution capture fixity information for digital content if it is not provided at the time of acquisition? Please indicate how often you capture fixity information.” Comparing the 2021 survey results to the 2017 survey results, 70 (63%) ‘Always’ capture fixity information in 2021 compared to 55 (74.3%) in 2017. In 2021 more respondents, 22 (19.8%), capture fixity information ‘At least half of the time’ as opposed to only 10 respondents (13.5%) in 2017. The number of respondents who ‘Never’ capture fixity information remained relatively consistent across the surveys, with five (6.8%) recording this response in 2017. Question 6: Please provide any relevant details about why you capture fixity information as frequently as you do. In this free-text question, 71 respondents provided insights for why they capture fixity as frequently as they do. In addition to commenting on the frequency of capturing fixity information, participants gave details about when fixity is used, how fixity is used, what fixity is used on, where fixity information is stored, what values are being captured, and the tools that are used to help with this work. Following standard procedures (23 or 32.4%) and ensuring integrity, authenticity, and trustworthiness (10 or 14%) were the top reasons given for capturing fixity information. One respondent commented, “All repository materials have fixity information to preserve chain of custody and 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 17 authenticity/integrity of the records over time,” and another stated that “Creating fixity information is part of our acquisitions process for all born-digital archives.” Other respondents expanded on this question, stating when, rather than why, they capture fixity. Respondents commented: “We capture fixity before and after content is transferred from one storage location to another.” ”We do not wait until preservation storage because processing might not be done for years after the content is acquired.” Many other comments indicated that respondents capture fixity at the point of ingest. Other reasons provided included that this information is captured as early as possible to establish provenance, prior to delivery, and before and after transfer. One respondent indicated that they plan to check fixity once a year, and another stated that they had recently checked fixity for the first time. ”We are beginning to capture this information for new collections. We are worried about the stability of our University provided storage and we use fixity information to monitor if there are any changes to our files.” Question 7: What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important):10 The 112 respondents to this question rated the eight provided reasons for using fixity information. Of these, ‘Determining if data has been corrupted or altered over time’ and ‘Determining if data has been corrupted during transmission’ were ranked as the two most critical reasons for using fixity information. Responses were more evenly spread for ‘Hardware monitoring,’ while ‘Permit[ting] an update to a portion of a content file while proving the other portions remain unchanged’ was selected as ‘Not important’ by most respondents. 10 In 2017, this question was worded differently. The question read: “What are the reasons your organization collects, checks, maintains, and verifies fixity information?” The scale for this question used five points rather than the four used in 2021. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 18 Reasons Extremely Important Moderately Important Somewhat Important Not Important Total Determine if the data has been corrupted or altered over time 90.2% (101) 7.1% (8) 1.8% (2) 0.9% (1) 112 Determine if the data has been corrupted or altered during transmission 76.6% (85) 16.2% (18) 4.5% (5) 2.7% (3) 111 To support the authenticity or trustworthiness of the digital objects 64% (71) 25.2% (28) 8.1% (9) 2.7% (3) 111 To monitor hardware degradation 20.2% (22) 29.4% (32) 29.4% (32) 21.1% (23) 109 For authenticity: To prove you are providing the digital object that has been requested 42.2% (46) 28.4% (31) 21.1% (23) 8.3% (9) 109 To permit an update to a portion of a content file while proving the other portions remain unchanged (ex: split video files) 8.3% (9) 6.4% (7) 21.1% (23) 64.2% (70) 109 Meet requirements or best practice guidelines 52.7% (59) 41.1% (46) 5.4% (6) 0.9% (1) 112 Help identify systemic or human error in the management of digital content 57.7% (64) 23.4% (26) 13.5% (15) 5.4% (6) 111 Other 50% (4) 12.5% (1) 12.5% (1) 25% (2) 8 Totals 461 197 116 118 892 TABLE 2: Reasons organizations use fixity information in response to “What are the reasons your institution uses fixity information? Please rate the importance of each of these items.” Within the ‘Other’ category, eight respondents provided additional insights about characteristics they considered to be ‘Extremely Important’ to ‘Somewhat Important.’ Some of these are listed below. “Maintain ISO 16363 certification.” (extremely important) “When transcoding from format to format to ensure the video and audio content is lossless (framemd5).” (extremely important) 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 19 “To facilitate archival appraisal, particularly the identification of duplicate files.” (moderately important) “The on-campus archives are part of a long term cold storage test with our on-campus IT partners.” (moderately important) It is difficult to make comparisons with the 2017 data, as the 2017 survey provided a five-point scale ranking in comparison to the 2021 survey’s four-point scale. The 2017 survey included a ‘Very important’ ranking between the ‘Extremely important’ and ‘Moderately important’ rankings. Section 2: Using Fixity Information Question 8: How much total content (preservation copies that are managed for long-term preservation only) are you running fixity on? Respondents were asked to select the range that includes the total amount of content they are running fixity on. These ranges were expanded from the 2017 survey based on the values provided in the ‘Over 500 TB’ category. The 2021 survey added four additional categories over 500 TB. With 111 respondents answering this question, there is no clear pattern in the results. Respondents are managing a large range of content from less than 100 GB to over 5 PB. Full results can be seen in Figure 6. From the perspective of survey analysis, these values were helpful for interpreting how fixity practices may differ depending on the total volume of digital content. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 20 FIGURE 6: Responses to “How much total content (preservation copies that are managed for long-term preservation only) are you running fixity on?” 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 21 Question 9: Do you employ different fixity practices for different types of content or storage media? Responses to this question were split almost evenly, with 57 (51.4%) of 111 reporting that they did not employ different fixity practices for different types of content or storage media and 54 (48.6%) responding that they did. This was a new question introduced for the 2021 survey. FIGURE 7: Responses to “Do you employ different fixity practices for different types of content or storage media?” Question 10: What factors influence your decision to use different fixity practices? (select all that apply) The 54 respondents who answered ‘Yes’ to Question 9, indicating that their fixity practices change based on different kinds of content or storage media, were provided with a follow-up question asking what factors influenced their decision. Respondents were able to select multiple choices. The majority (29 or 53.7%) of the 54 respondents indicated that ‘Storage media’ influences their decisions to use different fixity practices. Other factors that influence decisions include ‘Type of content’ (20 or 37%), ‘File format’ (16 or 29.6%), different practices for different ‘Collections’ (14 or 25.9%), and ‘Content value’ (9 or 16.7%). 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 22 FIGURE 8: Responses to “What factors influence your decision to use different fixity practices?” Twenty-two or 40.7% of respondents to this question indicated that ‘Other’ factors that were not listed influence their decisions to use different fixity practices. Some factors identified by respondents included: ● Storage location, particularly cost and feasibility considerations around cloud storage (This was viewed separately from the idea of “storage media” by the respondents.) “Restrictions around computing fixity in the cloud/AWS” “Nearline copies are only accounted for with minimal metadata checking, rather than retrieved for full fixity checking purposes due to high costs associated with Amazon S3 Glacier.” ● Software or system where the content is managed “It is based on what each architecture (within our preservation repository system) supports.” 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 23 ● Availability and capabilities of specific tools “We use a variety of tools when examining files… We are influenced by what fixity checks the tools provide and also by what tools are approved for which environment.” ● Collecting area, rather than specific collections “Our workflows are very format specific, so if the Library acquires or creates an object, the program responsible for that will perform the fixity workflows they have built out.” “The moving images and audio collections employ fixity extensively. The photo collection does not. Primarily due to a lack of technical expertise in that area.” Question 11: Do you verify fixity information after transferring data from one location to another? Seventy-four (66.7%) of the total 111 respondents who answered this question replied that ‘Yes,’ they verify fixity information after transferring data, and an additional 32 respondents (28.8%) said that they ‘Sometimes’ verify fixity after transferring data. Only five respondents (4.5%) replied that ‘No,’ they do not verify fixity information after transferring data from one location to another. These percentages are similar to those for the responses given in the 2017 fixity survey. In 2017, 51 respondents (68.9%) answered ‘Yes,’ they verified fixity information after transferring data; 18 respondents (24.3%) answered that they ‘Sometimes’ verified fixity information after transfer; and five respondents (6.8%) answered ‘No,’ they did not verify fixity information after transfer. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 24 FIGURE 9: Responses to “Do you verify fixity information after transferring data from one location to another?” Question 12: If ‘Yes’ or ‘Sometimes,’ when do you verify fixity information on any of the files you are preserving for long-term? The 106 respondents who answered ‘Yes’ or ‘Sometimes’ to Question 11 were asked to select how frequently they verify fixity when certain events happen. Not all respondents rated every event, so response totals differ for each event. The two most frequent events that ‘Always’ trigger fixity verification for respondents were ‘After placing files in preservation storage’ and ‘After moving files to new media.’ Seventy-nine out of 104 respondents (76.0%) replied that they ‘Always’ check fixity ‘After placing files in preservation storage.’ Fifty-eight out of 100 respondents (58.0%) responded that they ‘Always’ check fixity ‘After moving files to new media.’ Other provided fixity events had more evenly distributed responses. For example, out of the 88 respondents who ranked ‘After retrieving files for archival processing/description’ in response to this question, 16 (18.2%) chose ‘Never,’ 7 (8.0%) chose ‘Very rarely,’ 15 (17.0%) chose ‘Sometimes,’ 16 (18.2%) chose ‘Frequently,’ and 34 (38.6%) chose ‘Always.’ 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 25 When fixity is verified Always Frequently Sometimes Very rarely Never Total Upon receipt of materials 44.4% (44) 15.2% (15) 12.1% (12) 17.2% (17) 11.1% (11) 99 After moving files to new media 58.0% (58) 27.0% (27) 8.0% (8) 2.0% (2) 5.0% (5) 100 After placing files in preservation storage 76.0% (79) 12.5% (13) 6.7% (7) 2.9% (3) 1.9% (2) 104 After retrieving files for archival processing/ description 38.6% (34) 18.2% (16) 17.0% (15) 8.0% (7) 18.2% (16) 88 Other (please indicate) 70.6% (12) 5.9% (1) 0.0% (0) 5.9% (1) 17.6% (3) 17 Total 227 72 42 30 37 408 TABLE 3: Responses to “When do you verify fixity information on any of the files you are preserving for long-term?” by percentage and count (count in parentheses) Seventeen respondents selected ‘Other,’ and 12 of them provided text responses. Several of the respondents stated that they verify checksums before placing files in preservation storage rather than after, particularly if those files have been sitting on processing storage for a while or if the files are going to be uploaded to a cloud provider. “Rather than checking fixity every time data moves, we check it on receipt to create a baseline, and then again before deposit to preservation in AWS. Because we use [AWS Glacier] we cannot check fixity after deposit, but instead we pre-calculate the AWS etag and use that as confirmation of receipt.” Other respondents described additional triggers for fixity checking: ● When testing replacement or restoration of files from preservation systems 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 32 Question 18: Is your fixity verification done utilizing built-in hardware or is it software-based? The 107 responses to this question revealed that the majority of respondents (74 or 69.2%) are using ‘Software’ to carry out fixity checking, with a number of people using ‘Both software and hardware’ (32 or 29.9%). These results are very close to the 2017 survey, in which 49 out of 72 (68.1%) respondents recorded using only software for this purpose and 23 (31.9%) recorded using both hardware and software. Note that in 2021 only one respondent indicated that they were only using ‘Hardware,’ and in 2017 zero respondents selected hardware. FIGURE 13: Responses to “Is your fixity verification done utilizing built-in hardware or is it software-based?” Question 19: What software, tools, or services are you using to capture/verify fixity information? Select all that apply: This question was designed to gather more information about the types of software and services that respondents use to carry out fixity checking. More than one option could be selected for this question, so although 106 respondents answered this question, 196 answers were selected. The most selected response, at 63 (59.4%), was ‘Manually run software.’ High response rates were also noted for ‘Scripts / custom code’ (55 or 51.9%) and ‘Automated / scheduled software’ (51 or 48.1%). A smaller percentage of respondents selected ‘Third-party services’ (22 or 20.8%) and ‘Other’ options (5 or 4.7%). For these later two options, respondents were encouraged to provide further details, and a range of answers were received in response. Several third party service 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 33 providers were mentioned, including Preservica, AWS, Arkivum, Artefactual, and LIBNOVA. Some respondents spoke in more general terms, mentioning their digitization vendor, digital preservation system, or cloud service provider. One respondent mentioned that they do not know which third party service is being used. A range of software tools were mentioned (both as third-party services and as ‘Other’), including Fixity Pro, BagIt, md5deep, Gobi, BitCurator, Teracopy, hashdeep, rsync, and DROID. FIGURE 14: Responses to “What software, tools, or services are you using to capture/verify fixity information? Select all that apply.” These results are comparable to the previous survey, in which, out of 130 selections made by 73 respondents, the most selected response was ‘Automated or scheduled software’ (39 or 30%), with ‘Manually run software’ a joint second with ‘Scripts and custom code,’ with 35 respondents (26.9%) selecting each of these answers. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 34 Question 20: What type of fixity checking algorithm do you use? Select all that apply:14 The 104 responses to this question revealed that the most common fixitychecking algorithm in use is ‘MD5,’ with 81 respondents (77.9%) using it, followed by ‘SHA256,’ with 52 respondents (50%), and ‘SHA1,’ with 28 respondents (26.9%). The ‘SHA512’ and ‘CRC’ algorithms were less prevalent, with only 16 (15.4%) and nine (8.7%) respondents respectively. ‘Scripts / custom code’ was selected by 5 respondents (4.8%). These results are very similar to those shown in the 2017 report, though a slight decrease in the number of people using SHA1 was noted.15 Comparison of results for this question suggests there has been little change in practice in this area over the intervening years. FIGURE 15: Responses to “What type of fixity checking algorithm do you use? Select all that apply.” 14 A change was made to the wording of this question for clarity. The 2017 question was: “What type of fixity checking algorithm does your preservation software use?” 15 In the 2017 survey, 135 selections were made by 71 respondents. The largest number of respondents, 58 or 42.96%, selected the MD5 algorithm; 34 or 25.2% selected SHA256, followed by 28 or 20.7% who selected SHA1. CRC checksums were used the least, by ten respondents or 7.41%. Five respondents (3.7%) selected ‘Other.’ 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 35 Of those who selected ‘Other’ in answer to this question in the 2021 survey, responses included mention of double parity verification and AWS hashtree. In addition, a couple of respondents noted that they do not know which checksum algorithm is used. The following statement, also provided in the ‘Other’ response, details checksum practices that include using multiple algorithms: “Both Goobi and Archivematica generate SHA256 checksums, which are automatically verified upon ingest to our preservation repository. All files stored in our preservation repository must be accompanied by SHA256 checksums at ingest time. To date we’ve generated MD5 checksums with DROID, but we’ve now changed to SHA256 for consistency across all our systems. Goobi additionally creates SHA512 checksums, but these are not automatically verified.” This answer reflects the idea that many institutional practices employ a selection of algorithms for different purposes. Out of the 104 respondents to this question, 54 (51%) selected more than one checksum algorithm and 51 (49%) selected just one. Number of checksum algorithms selected Number of responses Percentage 1 51 49% 2 26 25% 3 16 15.4% 4 9 8.7% 5 2 1.9% TABLE 5: Breakdown on the Number of Algorithms Selected and the Number of Responses When taking into account the different combinations of answers selected in response to this question, the second most common response was both MD5 and SHA256 together, with 17 (16.3%) respondents selecting this pairing of algorithms. This was followed by SHA256 on its own, with 13 (12.5%) respondents selecting only this option. Other groupings included MD5, SHA1, and SHA256 (6 or 5.8%); MD5 and SHA1 (5 or 4.8%); MD5, SHA1, SHA256, and SHA512 (4 or 3.8%); and CRC, MD5, and SHA256 (3 or 2.9%). 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 36 Question 21: Who is responsible for verifying your content's fixity information (e.g., who runs manual scans, schedules automated scanning, analyzes reports or logs, etc.)? Select all that apply.16 There were 111 responses to this question, and answers revealed that the role responsible for verifying fixity information is most commonly an ‘Archivist, librarian, or curator,’ with 87 respondents (78.4%) selecting this response. ‘System administrator’ was the second most selected answer, with 37 respondents (33.3%) selecting this option. Note that further details provided by those selecting the ‘Other’ answer included mention of metadata specialists, conservators or conservation technicians, and digital workflow or digital curation specialists. One answer revealed that the role responsible for this process has not yet been defined. Note that 51% of those who answered this question selected more than one response, demonstrating that more than one role holder is responsible for this task. Some respondents selected as many as four or five options. 16 A change was made to the wording of this question for clarity. The 2017 question was “Who is responsible for fixity checking (e.g., running manual scans, scheduling automated scanning, etc.)?” 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 37 FIGURE 16: Responses to “Who is responsible for verifying your content’s fixity information (e.g., who runs manual scans, schedules automated scanning, analyzes reports or logs, etc.)? Select all that apply.” Direct comparison with the results from the 2017 survey is difficult as options have been simplified and rationalized into logical groupings in this latest iteration of the survey. ‘System administrator’ was the most popular answer in 2017, at 34 out of 160 selections (21.3%), but many variations of the ‘Archivist, librarian and curator’ role were listed as separate options within the 2017 survey and results were spread across these options.17 Question 22: Where do you store the preservation copies that are verified with fixity checking? Select all that apply:18 The 112 responses to this question revealed a range of storage types in use, with respondents in many cases selecting more than one answer. The most frequently selected option was ‘In-house online storage’ (80 or 71.4%), with ‘Offline storage (including cloud storage vendors)’ coming next (60 or 53.6%) and ‘In-house nearline storage’ less heavily utilized (26 or 23.3%). These results are similar to those recorded in the 2017 survey.19 Other options mentioned in free-text responses include in-house offline storage, in-house second storage, in-house networked storage and online cloud based storage, external hard drive, distributed digital preservation network, and M-DISC. Nuances in how fixity checking is managed across different storage locations are captured in one comment: “The in-house/online storage is the main copy used to verify content over time. However, the nearline and offsite (tape) copies are verified upon transfer to those locations.” 17 There were 74 responses to this question in the 2017 survey, and 160 options were selected in total. The results showed that system administrators were most often responsible for fixity practices (34 or 21.3%). Digital preservation managers and digital archivists followed, with 26 (16.3%) and 21 (13.1%) responses respectively. 18 A change was made to the wording of this question for clarity. The 2017 question was: “Where are the preservation copies stored, upon which the fixity checking occurs? Select all that apply.” 19 In the 2017 survey, 74 respondents made 117 selections in response to this question. The most common location selected was ‘In-house online storage’ at 67.6% (50 respondents), followed by ‘Offsite storage (including cloud vendors)’ at 46% (34 respondents) and ‘In-house nearline storage’ at 35% (26 respondents). 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 38 FIGURE 17: Responses to “Where do you store the preservation copies that are verified with fixity checking? Select all that apply.” Question 23: Where does your institution record fixity information? Select all that apply: There were 112 responses to this question and 220 selections made in total, illustrating that many of those surveyed record fixity information in more than one place. The answer most frequently selected was storage ‘In databases and logs’ (72 or 64.3%), with fixity information being stored ‘Alongside content’ also receiving a high number of responses (68 or 60.7%). Storage of fixity information ‘In object metadata records’ had 54 responses (48.2%) and a small number of respondents reported storing fixity information ‘In the files themselves’ (9 or 8%). There has been some change in the distribution of answers since the 2017 survey, where storing fixity as ‘Part of the metadata record’ came out as the second most popular response above storing the information ‘Alongside the content.’20 20 In the 2017 survey, 74 respondents answered this question. Recording fixity information in databases or logs was the most common response, with 54 respondents (73%) taking this action. Storing fixity as part of the metadata record was selected by 39 respondents (52.7%), and storing the information alongside the content was selected by 32 respondents (43.2%). Recording the fixity information within the file itself was only selected by nine respondents (12.1%). 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 39 FIGURE 18: Responses to “Where does your institution record fixity information? Select all that apply.” Those that selected ‘Other’ gave further information in the free-text field. Several responses mentioned storing the fixity information in text, CSV, or Excel files, and in some cases it was mentioned that these files were stored alongside the content, which was similar to one of the provided scenarios. A more detailed answer, describing a different fixity storage scenario, was given by one respondent: “Our AIP is a single file that contains the metadata and content (usually multiple metadata packages and content files). The fixity information (digital signature) is held as metadata within the AIP. So it is similar to a PREMIS XML file, but held in the file itself.” Another answer reported that fixity information is deleted once it has been verified. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 40 Question 24: What level of granularity do you utilize when running checksums? Select all that apply: There were 110 responses to this question. Results show that the most common level of granularity is ‘Per-file level checksums (one file per checksum)’ (99 or 90%), with several respondents also applying checksums ‘Per block, folder or bag’ (40 or 36.4%). A smaller number of responses were also received for the option of creating ‘Partial-file checksums (multiple checksums per file)’ (12 or 10.9%). Again, as with many of the preceding questions, more than one answer was often selected, demonstrating that for many practitioners, multiple options may be used in different circumstances. The results reported here reflect closely the findings of the 2017 survey. FIGURE 19: Responses to “What level of granularity do you utilize when running checksums? Select all that apply.” Question 25: If you run partial-file checksums (multiple checksums per file), what is your use case? This free-text question was answered by 11 of the 12 respondents who reported using partial-file checksums in the previous question (though one response was “n/a”). The majority of these answers (eight) specifically mention using this level of granularity for workflows involving audiovisual content, and several of those answers mention the use of FrameMD5 in this context. Details include “framelevel checksums in moving images” and “framemd5 are run for digitized AVobjects.” This focus on the audiovisual use case tallies with the findings of the 2017 report, which reported that use cases for the eight respondents who created multiple fixity values per file were all related to audio or video files. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 41 Section 3: Cloud Services This section asked questions about fixity services and information provided by cloud services vendors. These vendors include direct providers of cloud storage such as Amazon Web Services S3 and Glacier storage, or third-party software that runs on commercial cloud services such as Preservica or Arkivum. The 2017 survey also included this section with the same questions, but some question text was updated for clarity.21 Question 26: Are you using cloud service vendors that offer fixity services? Of the 112 survey respondents who answered this question, 60 respondents (53.6%) answered ‘No,’ they are not using cloud services vendors that offer fixity services, and 52 respondents (46.4%) answered ‘Yes,’ they are using cloud services vendors that offer fixity services. In the 2017 survey, 51 (68.9%) of respondents answered ‘No,’ and only 23 (31.1%) of the respondents answered ‘Yes.’ The proportion of respondents in 2021 who are using cloud service vendors that offer fixity services is 15.3% greater than the proportion of 2017 respondents who did so. FIGURE 20: Responses to “Are you using cloud service vendors that offer fixity services?” The table below shows how use of cloud-based fixity services is distributed based on the total amount of content an institution is running fixity on as reported in Question 8. 21 Use of the words “vendors” or “cloud services” in 2017 question text was updated to consistently say “cloud service vendors” to clarify that these questions have to do with third-party commercial cloud services. Emphasis was also added to the questions about receiving and using fixity information. For a complete list of changes, see Appendix 3. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 48 used a scale of ‘Never, Rarely, Infrequently, or Frequently’ to rate the relative frequency of each aspect of fixity failures. Question 32: How many times a year do you see fixity checks fail? Out of 112 respondents, 30 (26.8%) reported ‘Never’ seeing fixity failures. For the 82 respondents (73.2%) who experienced fixity failures, 39 (34.8%) see failures ‘Rarely,’ 31 (27.7%) see failures ‘Infrequently,’ and 12 (10.7%) see failures ‘Frequently’ during a year. FIGURE 26: Responses to “How many times a year do you see fixity checks fail?” Very large collection sizes, as reported in Question 8, do not necessarily seem to be a predictor of whether institutions ‘Never’ see fixity failures or ‘Frequently’ see them. Institutions storing more than 5 PB of content included three respondents who ‘Frequently’ see failures, two who ‘Infrequently’ see failures, and one who ‘Rarely’ sees failures. In the category of institutions with 1–5 PB of data, seven respondents reported that they ‘Never,’ ‘Rarely,’ or ‘Infrequently’ see failures, compared to four respondents who reported ‘Frequently’ seeing failures. Question 33: How often do you see fixity failures during the events listed below? This question asked for greater detail about the events in which fixity failure was found. The table below shows the responses for each scenario with the selected rankings. Please note that the options in the table below have been shortened. The full options listed in the survey were: 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 49 ● Never ● Rarely (less than once a year) ● Infrequently (a few times a year) ● Frequently (multiple times per year) ● I don’t know Event where fixity failure occurred Frequently Infrequently Rarely Never I don’t know Total Responses While verifying fixity for content at rest 6.7% (5) 16% (12) 32% (24) 32% (24) 13.3% (10) 75 While verifying fixity upon receipt 7.7% (6) 19.2% (15) 32.1% (25) 28.2% (22) 12.8% (10) 78 While verifying fixity after moving files to new media 5.2% (4) 23.4% (18) 29.9% (23) 28.6% (22) 13% (10) 77 While verifying fixity after storing files in repository 5.3% (4) 10.7% (8) 25.3% (19) 40% (30) 18.7% (14) 75 While verifying fixity after retrieving files for archival processing/description 1.3% (1) 12% (9) 28% (21) 41.3% (31) 17.3% (13) 75 Other (please indicate) 14.3% (1) 28.6% (2) 42.9% (3) 0% (0) 14.3% (1) 7 Total 21 64 115 129 58 387 TABLE 6: Responses to “How often do you see fixity failures during the events listed below?” by percentage and count (count in parentheses) When reviewing failures that occur ‘Frequently,’ the most common occurrence appears to be ‘While verifying fixity upon receipt,’ with six selections out of the 21 total for that option. Other options were not far behind, with ‘While verifying fixity for content at rest’ selected five times and both ‘While verifying fixity after moving files to new media’ and ‘While verifying fixity after storing files in a repository’ selected four times. On the other side of the frequency scale, 129 selections were made for events that ‘Never’ occur. In reviewing the data, it appears that failures are not often seen, especially ‘While verifying fixity after retrieving files for archival 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 50 processing/description’ (31, or 24%) and also ‘While verifying fixity after storing files in a repository,’ (30 or 23.2%) both of which were commonly reported as ‘Never’ occurring. Question 34: How often does fixity fail for the listed reasons? The table below shows a full summary of the results. Please note that the column titles in the table below have been shortened. The full options listed in the survey were: ● Never ● Rarely (less than once a year) ● Infrequently (a few times a year) ● Frequently (multiple times per year) ● I don’t know Fixity Failure Reason Frequently Infrequently Rarely Never I don’t know Total Responses Corrupted byte stream (e.g., flipped bits) 1.3% (1) 9.2% (7) 36.8% (28) 35.5% (27) 17.3% (13) 76 Interrupted network transfers (e.g., resulting in truncated files) 15.8% (12) 26.3% (20) 25.0% (19) 21.1% (16) 11.8% (9) 76 Missing files (e.g., files in a manifest but not available) 7.1% (5) 11.3% (8) 36.6% (26) 32.4% (23) 12.7% (9) 71 Extra files (e.g., files not in a manifest but in package) 8.1% (6) 14.9% (11) 25.7% (19) 39.2% (29) 12.2% (9) 74 File reference lost (e.g., bytestream fixity maintained by file name changed) 0.0% (0) 11.3% (8) 29.6% (21) 45.1% (32) 14.1% (10) 71 Other (please indicate): 22.2% (2) 11.1% (1) 33.3% (3) 11.1% (1) 22.2% (2) 9 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 51 Total 26 55 116 128 52 377 TABLE 7: Responses to “How often does fixity fail for the listed reasons?” by percentage and count (count in parentheses) Out of the 377 selections across this question, 26 indicated errors with a high frequency, or happening ‘multiple times a year.’ Of these, 12 were from ‘Interrupted network transfers,’ six from ‘Extra files,’ five from ‘Missing files,’ two for ‘Other’ reasons, and one for a ‘Corrupted byte stream.’ FIGURE 27: Number of Fixity failure events selected as occurring ‘Frequently’ On the other side of the scale, 118 selections were made for events that ‘Never’ occur, with the most common selection being ‘File reference lost’ with 32 selections. ‘Extra files’ and ‘Corrupted byte stream’ were selected 29 and 27 times respectively. ‘Missing files’ follows close behind, with 23 reporting that this never happens, while 16 indicated that they have never experienced fixity failures due to an ‘Interrupted network transfer.’ 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 52 FIGURE 28: Number of fixity failure events selected as occurring ‘Never’ In reviewing the data as a whole (Table 7), in nearly all cases, respondents reported a decreasing relative frequency of occurrence from ‘Never’ to ‘Frequently.’ The only outlier is ‘Interrupted network transfers,’ which has a much larger proportion of respondents having selected ‘Infrequently’ (20 or 26.3%) or ‘Frequently’ (12 or 15.8%). The eight free-text responses provided with the selection of ‘Other’ can mostly be categorized into either human errors or system issues. Human error may be caused by intentional but untracked changes in instances where a staff member updates, corrects, or edits a file, for example a metadata sidecar, without updating that file’s checksum. Four respondents report having seen fixity checks fail as a result of a process like this. One respondent provided the following: “Pretty much all our fixity failures are caused by human error – e.g. somebody downloading a known-good package, modifying one file, and uploading the new version without updating the fixity information.” System issues result in fixity failures not because the stability of the file is in doubt, but because the process performing the check does not work as expected. Causes range from storage being offline to software bugs. Causes like this were also reported by four respondents. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 53 Question 35: What actions have you taken to address fixity failures? Select all that apply. The total number of respondents to this question was 77, and respondents were able to select all that applied. The survey results suggest the primary strategy in addressing fixity failures is to replace the file, whether that is to ‘Replace the file with a known good copy from your storage,’ as selected by 64 respondents (83.1%); ‘Request a new copy of the file from creator or digitization source,’ selected by 40 (51.9%); or ‘Request a new copy from the third-party storage provider,’ selected by four respondents (5.2%). Other methods of addressing fixity failures provided as options in the survey included ‘Remov[ing] the extra files,’ ‘Accept[ing] the file as is and recording the fixity failure,’ and ‘Other.’ Twenty-three respondents (29.9%) indicated that they ‘Remove the extra files,’ and 17 (22.1%) ‘Accept the file as is and record the fixity failure.’ The 11 free-text responses in ‘Other’ offered additional processes being undertaken, including re-running the check because the process failed and not the fixity of the file (7), investigating the cause to improve processes (3), and recording the intentionality of the change (2). 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 54 FIGURE 29: Responses to “ What actions have you taken to address fixity failures?” Question 36: Are there any noteworthy fixity failure events and responses that you would feel comfortable sharing? If so, please describe them below. Twenty respondents provided greater details around fixity failure events. Many of these responses echoed technical issues such as errors during network transfers (5), when writing the files to a new storage system (5), or when the fixity checking process itself fails (5). As illustrated in the example below, these issues can require deep technical dives to both identify the cause and then put a solution in place: “We used to use storage that was case-insensitive. Switching to S3 protocols requires case-sensitivity so it is important that the paths to files are correctly recorded in your databases/xml files.” Some failures were only caught while investigating unexpected system behavior. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 55 “We had a storage vendor migrate our data from disk to tape and noticed performance issues. When we requested that the files be migrated back to disk, a hardware bug corrupted 10% of them.” Another source of frustration is unintentional extra files created by operating systems (3). “Mac OS temp files such as the ubiquitous .DS_Store files are a constant source of aggravation.” One final theme, apparent in three answers, was respondents having the tools to check for fixity and respond to failures but not to identify the causes of the failures. “...sometimes checksums change before and after transfer for no apparent reason, often solved by recopying.” Section 5: Demographic Information This section captured some basic demographic information about the survey respondents. While the survey was shared with an international audience through international listservs, it should be pointed out that the survey was written in and only available in English. Thus there is a bias favoring Englishspeaking countries or those more familiar/comfortable with the English language. Question 37: Which of the following most closely describes the type or function of your institution? Of the 116 respondents to this question, the majority, 61 (52.6%), are from ‘Academic libraries or archives.’ ‘Government entities’ are the second most represented group, with 18 respondents (15.5%). Other selections in the 2021 survey included ‘Non-profit institution (not one of the above types)’ (6.9% or 8 respondents); ‘National, federal or legal depository repository’ (5.2% or 6 respondents); ‘Museum (4.3% or 5 respondents); “Research data repository’ (4.3% or 5 respondents); ‘Independent Library or archives’ (2.6% or 3 respondents); ‘Historical society’ (0.9% or 1 respondent); and ‘For profit corporation’ (0.9% or 1 respondent). Eight respondents chose ‘other’ and indicated that they represent the following types of organizations: private archive, state archives, national archive for moving images, service provider, and consortium of university and academic libraries. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 56 Available options to select for this question were reduced from the 2017 survey, grouping some of the terms used previously. Initial comparison of results suggests that results were similar in the 2017 survey, with ‘Academia’ and ‘Government Entity’ being the top responses with 38 (47.7%) and 13 (20.5%) respectively. FIGURE 30: Responses to “Which of the following most closely describes the type or function of your organization?” Question 38: Where are you located? The 116 respondents represented a total of 12 countries, with results skewing toward English-speaking countries. Not surprisingly, 62.1% of respondents are from the United States (72) and another 19.8% are from the United Kingdom (23). Australia and Canada each represent another 5.2%, with 6 respondents each. Two respondents were from Austria (1.72%), with one respondent from Denmark, Finland, Germany, Ireland (Republic), Netherlands, New Zealand, and Singapore (0.9% each). 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 57 FIGURE 31: Graph showing countries with more than one respondent Question 39: Would you be willing to discuss your fixity practices with us? We would like to expand on the survey by providing selected use cases. These would be used in a final report about the survey and/or as individual blog posts. Just under half (48.5%) of the 99 respondents to this question were willing to discuss their practices further. Select respondents were chosen from this group of 48 based on a combination of factors such as organization type, collection size, and digital preservation program maturity. These discussions are included in the “Case Studies” section of this report. Each case study provides more details about an organization’s fixity practices and the rationale behind them. Question 40: Is there anything you would like to clarify about your survey responses or share with us about your fixity practices?22 This question provided respondents with the opportunity to clarify their answers and/or share anything additional about their fixity practices. Thirty-one respondents took the opportunity to clarify their results. Many offered general comments that provided clarification about rankings, described assumptions while taking the survey, noted that they expect failures with large amounts of data, or mentioned tools being used and issues with them. The most 22 A change was made to the wording of this question for clarity. The 2017 question was “Is there anything else you would like to tell us about your practices around fixity?” 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 64 The primary function of fixity information in the Archives and Special Collections is to ensure integrity in preservation storage, although future use cases might include deduping received files and ensuring the authenticity of acquired content prior to processing and ingest. Routine audits have turned up no fixity failures so far, which May sees as an indication that the process is working well and should continue as is: “I think of it as…so far, so good…. If it’s working, I’m not going to change it until there’s a good reason to.” Fixity in a new digital preservation program Over the last year, much of the foundational work for the new digital preservation program has centered on educating others in the organization, particularly those in positions to make programmatic and financial decisions, about what fixity is and why it matters. To get buy-in, the preservation librarian has worked to explain to colleagues the differences between backups and active preservation and the importance of fixity “to ensure that we are still seeing the same thing over and over and over again.” Even with greater support now in place, understanding of the potential future uses of fixity is limited because the library has not experienced any disasters or fixity failures that would test the effectiveness of the practices in place. The program’s draft digital preservation policy stresses the importance of fixity to ensure it will be a priority in tool selection and workflow development, although at this point nearly all fixity activity occurs in Preservica without library staff intervention. Archives and Special Collections began using the system in 2020 and has since relied primarily on Preservica’s built-in fixity monitoring, which is set to audit each object every 30 days. Using an automated, vendorprovided fixity solution allowed quick implementation of baseline practices during ongoing program development. Because the digital preservation program is so new, fixity practices will likely look very different in the future. Staff are still outlining next steps and additional use cases, but in the meantime, the focus remains on continuing advocacy and education, ensuring consistent capture of fixity information prior to or during Preservica ingest, and auditing checksums regularly. May emphasizes that the department’s approach to fixity aims to accomplish as much as possible with the staffing and resources available now and that the program’s goals will continue evolving to reflect new standards and practices. Case Study #2: Small Nonprofit Archives with Significant Audiovisual Holdings and Emerging Fixity Program Based on a conversation with Milo Thiesen, Media Asset Manager, Archives and Information Resources, Lincoln Center for the Performing Arts, Inc. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 65 Organizational context Lincoln Center for the Performing Arts, Inc. (LCPA) is the corporate body that provides shared services and facilities for its constituent organizations, which function as independent entities and, in some cases, run their own archives separate from those of LCPA. In addition to supporting its resident organizations, LCPA has its own educational mission, runs a program of art commissions, offers a variety of civic engagement initiatives, and presents artistic programming. Highlights of this programming, of which LCPA’s archives holds extensive records, include 2021’s Restart Stages, the Mostly Mozart Festival, Lincoln Center Out of Doors, and Live From Lincoln Center, a seventeen-time Emmy Award-winning television series that has broadcast world-class performances on PBS since 1976. LCPA’s holdings include organizational records, information about the physical campus and its history, and audiovisual recordings of LCPA programs. Other important holdings related to Lincoln Center’s founding from 1956 to 1959 are currently being processed in an effort to critically examine the institution’s complex history with the displacement of residents from San Juan Hill, the neighborhood that was razed to build the Lincoln Center campus. The archives unit has four staff positions, although some are currently vacant and only one person works primarily with digital material. The digital preservation program at LCPA is still emerging and led by a recently hired Media Asset Manager. Key priorities for the current stage of the program include assessing past practices, centralizing inventories and other documentation, implementing a new DAMS, and incorporating more whole-lifecycle workflows across all asset types. How the organization uses fixity In the past, LCPA’s electronic records and media were stored using a variety of archival databases, external drives, and other systems without consistently captured fixity information. Thiesen notes, “To my knowledge, there hasn't been any catastrophic data loss event”; however, there have been some scares. There is also the possibility that as-yet-unnoticed corruptions or losses were introduced during past data migrations that did not incorporate fixity checks, a concern Thiesen aims to protect against in the future. Now, during migrations and when working with high-value items and large file packages, tools such as hashdeep allow them to manually capture and log fixity information. A core strategy for integrating fixity practices more deeply into the organization’s asset creation and management processes has been attaching fixity to existing workflows in simple, automated ways that support integrity throughout assets’ full lifecycles. For example, the post-production tool ShotPut Pro can be used in existing video and audio workflows and also includes a built-in hashing algorithm that supports quick and easy fixity verification and audit logging during 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 66 production, transfer, and storage. Similarly, LCPA’s new DAMS, Cortext from Orange Logic, was chosen in part because it not only meets the functional needs of the organization by facilitating internal access and reuse but also calculates MD5 hashes on upload, whenever objects move within the system, and on regular intervals. Also, the archives coordinates closely with vendors to ensure they generate and verify fixity information throughout their processes, not just during the final transfer of assets. As LCPA’s preservation practices evolve, Thiesen hopes eventually to have a media archivist more deeply involved in video production, able to capture content and checksums closer to the point of creation and help content creators understand why fixity should be part of their daily workflows. Other plans include continued movement toward centralized integrity checks and audit trails with robust event notifications. Pragmatic fixity practices LCPA’s approach to fixity is necessarily adapted to local context and focused on incremental change. In an archive where the importance of content stems largely from its informational value and reuse potential, remastering and normalization are common. Fixity practices must therefore be flexible and allow for primary files to be replaced and checksums to change periodically while still ensuring bit-level integrity of primary files over time, and fixity information is thus part of a nuanced audit trail documenting when and how an object changed over time rather than a tool to ensure all content remains static. LCPA employs a tiered approach to fixity that directs more time and resources to high-value content. For example, preservation-quality DPX sequences of scanned 35mm historical films have more frequent and robust fixity checks than reference scans. Thiesen also emphasizes the importance of understanding the use cases for different hashing algorithms and why standards might vary among industries, noting that employing a less secure algorithm to expedite file transfers or make tasks easier to complete can be reasonable for certain scenarios where secure cryptographic hashes are not essential. Incremental progress depends not only on pragmatic technical choices but also on a realistic view of the non-technical aspects of fixity. LCPA’s practices are rooted in the idea that fixity is about people and knowledge management as well as tools and metadata. Lack of institutional memory, inadequate staffing, and incomplete documentation can become barriers to monitoring and maintaining the integrity of digital assets, whether because no one knows which version of a file should be fixed, discontinuity renders fixity logs less reliable, or the locations of fixity information are lost. While working toward larger technical and non-technical fixity goals, Thiesen explains, “I want to make sure that we’re doing what we can and that we have a 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 67 thoughtful strategy about pushing things forward.” That means focusing on better, if not best, fixity practices and taking small, consistent steps forward rather than becoming stuck in the planning phase trying to ensure perfection. Case Study #3: Data Repository with Large Content Volume and Established Fixity Program Based on a conversation with Sam Pepler, Curation Manager, Natural Environment Research Council Organizational context The UK-based Natural Environment Research Council (NERC) is an environmental science funding body that commissions the Center for Environmental Data Analysis (CEDA) as part of its Environmental Data Service. The core functions of the CEDA Archive are to supply useful data to researchers, including data sourced from space agencies and other research organizations, and to support transparency and traceability of research by acting as the long-term home for data produced through NERC-funded projects. Ensuring such data’s survival and integrity over time is critical to the mission of the parent organization and essential for longitudinal research—as Pepler notes, “If you make an environmental measurement, it’s not like a lab measurement; you can’t go back and measure the sea temperature in 1960 again.” NERC’s data curation activities primarily involve oceanographic, atmospheric, and other environmental datasets, which are heterogeneous in size, type, and source, ranging from complex climate models used for Intergovernmental Panel on Climate Change assessment reports to satellite images to historical temperature records. CEDA’s holdings comprise approximately 300 million unique digital objects totaling 18 petabytes. Depending on the data type and format and whether the CEDA copy is the version of record, NERC staff employ different storage locations, media types, layers of redundancy, and backup strategies. As well as running the archive for NERC, CEDA operates an inhouse supercomputer called JASMIN for data-intensive science. This compute infrastructure forms the backbone for the archives storage systems. Approximately twelve CEDA staff directly support archive functions, principally data scientists facilitating data ingestion and some developers creating and maintaining software systems. Six staff in CEDA support the JASMIN infrastructure, and a further twelve work on projects associated with the archive. How the organization uses fixity Affordable, efficient fixity monitoring is one reason NERC manages its own infrastructure rather than contracting with a cloud provider or other vendor. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 68 Checksumming massive quantities of data in a commercial storage solution would be cost prohibitive, and by handling fixity in house, NERC can take advantage of its supercomputer to speed up the process. Even so, checksumming the entire archive is time consuming. Regular audits at sixmonth intervals are made more manageable by organizing data into larger chunks and calculating checksums at an aggregate level. Audit logs, following the Checkm specification,23 record the number of new, corrupted, changed, and deleted files after every check, which allows for comparisons over time and deeper investigation when warranted. These recurring fixity checks, along with verification of checksums upon receipt (when values are provided by the producer) and generation of MD5 hashes upon ingest, become part of a long-term audit trail. As audit records grow in number and length, dealing with them has become a complex challenge in its own right. Pepler hopes to migrate text-based fixity logs into a robust database and devise a mechanism for assembling complete audit records and delivering to researchers full-lifecycle checksum manifests along with requested datasets. Such reports would include original checksums recorded during deposit, verification of non-MD5 checksums, and new fixity information captured when archive staff change or replace files. Although CEDA sees little demand from researchers at this point for checksums as externally visible metadata, and using MD5 as the primary checksum algorithm has worked well thus far, a more flexible, end-to-end fixity record would be good practice and support the security and reusability of data. Nevertheless, Pepler maintains that simplicity is key when it comes to using fixity in a large data repository; when tools and workflows become too complex, the barriers to ongoing maintenance and future advances inevitably increase. Dealing with fixity failures With large collections stewarded over decades through multiple migrations, NERC staff have seen many ways that storage media failures, software errors, and human mistakes can cause fixity problems. While many survey respondents capture fixity information and run regular checks simply as a precaution, NERC staff routinely encounter fixity failures and thus have established practices for responding when they do. To filter out false signals and ensure fixity failures reflect only unintentional changes, the CEDA Archive’s audit logs define “corruption” as an altered checksum without an altered modified date (as the latter would suggest intentional alteration or replacement of an object). Narrowly scoping the types of 23 Checkm is a text-based file manifest format designed to support fixity-verification tools. For more information, see the technical standard: John Kunze, “Checkm: A Checksum-based Manifest Format,” California Digital Library, 2009, https://ia801704.us.archive.org/29/items/ark_13030_c72z12p53/CheckmSpec.pdf. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 69 fixity failures that warrant investigation makes followup more practicable. Failures are monitored in certain automated workflows, and staff periodically review the reported list of corruptions. Whether failures result from random bit flips, firmware errors, system bugs, or prematurely checksumming files that are still active and changing, human intervention is needed to identify the cause and determine the best course of action. Despite the manual effort required to respond to failures, Pepler prefers this to the idea of a self-correcting system: “If you jump up and fix it right away, you lose the information about what went wrong” and “what the file looks like on the disk.” When systems automatically replace files without review, there is a risk of ending up with the wrong version or overlooking a bug in a storage device that could cause further problems. In the process of spotting and dealing with fixity failures, NERC’s best results come from a combination of tools and people: tools can alert staff to a problem and support investigation of the object in question, data producers can provide important input about how a file should look and what might have changed, and technologists and curation experts can track down the underlying problems and make informed decisions about how to replace and repair altered objects. Case Study #4: National Archives with Large Content Volume and Established Fixity Program Based on a conversation with Leslie Johnston, Director of Digital Preservation, and Elizabeth England, Digital Preservation Specialist, National Archives and Records Administration Organizational context The United States National Archives and Records Administration (NARA) has been collecting records in electronic formats since the 1970s, and current digital holdings are estimated at over 1 PB of data in 2.1 billion unique files. In 2008, NARA began development of its ERA (Electronic Records Archives) project to preserve records from federal agencies and has since developed ERA 2.0, a cloud-based environment for the processing and preservation of electronic records.24 Most of these records fall into one of three categories: permanent records created by federal agencies, presidential records, and legislative records consisting of materials produced by legislative offices, commissions, and committees. Each category of materials is governed by different policies, affecting the scope of materials directed to the archive and the point at which fixity is recorded. Heterogeneity is a key component of NARA’s electronic records program: while NARA can and does provide extensive guidance to 24 National Archives, “Electronic Records Archives,” National Archives, 2021. https://www.archives.gov/era/about. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 70 record creators and submitting agencies, there are few legal requirements to govern how records are submitted for preservation. In response, NARA must develop flexible fixity processes to manage the vast quantity of digital content that is in their custody. How the organization uses fixity The permanent records of more than 200 federal agencies are managed through record schedules, which organizations submit to NARA for approved record disposition. Less than 5% of records produced by federal agencies are identified as permanent and sent to NARA, where they are processed and then submitted to ERA 2.0 object storage. This submission to storage is also the point at which legal custody is transferred to NARA from the submitting agency and fixity information is generated for the records, if it has not been previously provided. Many federal agencies export records for NARA from other recordkeeping systems, where fixity information can be generated as part of the export package, so ERA 2.0 will validate fixity if present. However, fixity information and other metadata are not required for submission, so ERA 2.0 will generate fixity information if not provided. When generating fixity information, NARA uses SHA256 checksums to document object fixity. These checksums are maintained along with the submission package in object storage. Presidential records have a far broader scope than federal agency records: everything created within the executive office is considered a permanent record and must be transferred to NARA following the end of that administration. As such, there are a great variety of types and formats of digital information in presidential record submissions, and fixity information may or may not be created as a part of those submissions. Materials submitted to NARA undergo the same processes as agency records, with SHA256 checksums generated upon ingest. Legislative records are the third body of material collected by NARA. These materials are not the personal records of individual congresspeople, but the records created through defined congressional roles, such as the Office of the Speaker of the House, and also through congressional committees and commissions. Records may be deposited with little notice, and unlike other materials at NARA, legislative record creators retain legal custody of their records after submission. Records from legislative offices and committees are frequently subject to embargoes of up to 50 years and are not processed during these embargo periods. NARA maintains administrative control of legislative records as a “courtesy hold,” and NARA staff must be able to return materials within 24 hours if requested. Given these requirements, fixity information is generated at the moment of receipt, as this fixity information is key to 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 71 demonstrating the authenticity of materials returned to legislative records requestors. Managing fixity in large volumes at an established digital preservation program With an estimated collection size of over 2.1 billion files, the biggest fixity challenge described by Johnston and England is one of scale: there are simply too many files, and not enough active compute capability, to be able to conduct fixity auditing in a meaningful way. As Johnston puts it, if they were to undertake systematic auditing of NARA collections, “We would never not be auditing.” The ERA 2.0 system is built on the Amazon GovCloud service, and while the service provides fixity checking and fixity auditing, this activity is opaque and undocumented to the NARA team. In the future, existing fixity practices will require closer examination as NARA shifts to a tiered-storage model, potentially incorporating additional disk, tape, or cloud storage options (at time of interview this project is still under development). For digital content at each tier of storage, NARA will need to decide how to generate, audit, and update fixity information and how frequently to do it. They are currently investigating randomly, routinely sampling a subset of their digital collections for fixity auditing, but are also thinking about what to do with the fixity and auditing information that is generated and whether it could have any uses beyond that originally intended. As a core function of the digital repository, notes Johnston, “Fixity is not a place for innovation;” however, it is an area where the large quantities of information processed and audited could be useful for other analysis and decision-making. Conclusion The 2021 Fixity Survey and follow-up interviews highlight the importance of fixity checking as part of digital preservation practice. It is clear that recognized good practice (as defined by models and frameworks such as the NDSA Levels of Preservation25) is evolving, as are institutional practices as defined by survey respondents. The results of this survey do not point to a clear, one-size-fits-all approach to using fixity information, though they do clearly demonstrate its wide use at a variety of different stages in the digital preservation workflow. As this report summarizes a wide range of practices, prescriptive guidance around how fixity checking should be carried out, with which tools, and at what 25 NDSA Levels of Preservation Working Group, “Levels of Digital Preservation,” National Digital Stewardship Alliance, 2019, https://ndsa.org/publications/levels-of-digital-preservation/. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 72 frequency cannot easily be extracted. As with most areas of digital preservation practice, context is everything. Decisions made and techniques employed will depend on a number of factors, including technical infrastructure, organizational priorities, resources, and risk appetite. Although considerations around the environmental sustainability of digital preservation practices do not surface in the survey results, it is anticipated that this may emerge as an additional factor for practitioners to balance in the future. Use of fixity information should also be viewed with the bigger picture in mind. Clearly the workflows and methodologies reported here do not represent a neatly self-contained package of digital preservation practice; they typically exist as just one part of a broader digital preservation infrastructure. The influence of all of the factors that inform decisions on fixity checking was not captured within this survey but should be noted. The relationship between the number of copies held (or length of time backup copies are available) and the frequency of fixity checks, for example, is an important one. As noted in the 2017 Survey Report, what is considered best practice in use of fixity information is likely to evolve over time, and those working in digital preservation should consider their fixity practices within a wider framework of continuous improvement rather than as a finished piece. Benchmarking against practices recorded in this survey report may be a helpful place to start. It is noted again that a repeat of this survey in the future would be helpful in capturing further developments in the use of fixity information in digital preservation. Recommendations for Future Surveys To assist future iterations of the survey, this section provides information on issues the 2021 Fixity Survey Working Group discussed when analyzing the data and writing this report. The items listed below should be considered when preparing the next fixity survey. Suggestions for survey questions ● Question 6: Please provide any relevant details about why you capture fixity information as frequently as you do. ○ The wording of this question could be clearer. Some of the responses didn’t seem to answer the question that had been asked. However, it is a very general free-text question and providing respondents a space to add details could be helpful. ● Question 7: What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important) 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 73 ○ Working group members received feedback that some respondents (from an IT viewpoint) felt that the options ‘For authenticity’ and ‘Corrupted and altered files’ were similar. This could be interpreted this way, as the end result is the same; however, the reasons for doing the checks are different. ○ It was noted by the working group that the Likert scale may be difficult to use when trying to rank human error or monitoring hardware. ● Question 9: Do you employ different fixity practices for different types of content or storage media? ○ This question could be used to gather additional response granularity. Because almost half of the respondents (48.6%) employ different fixity practices for different content or storage media, it is difficult to determine which answers apply to which of their use cases. Future surveys may try to tease these relationships out a bit more by including survey logic to break out answers per each identified use case. ● Question 10: What factors influence your decision to use different fiixty practices? ○ ‘Other’ was chosen by 40% of respondents. This indicates that the reasons listed weren’t fully fleshed out. More work should be done to review the other responses to see if they could be added to the next iteration of the survey. ○ While ‘Storage media’ was a provided option, many listed cloud storage, which the group would consider to be ‘Storage media.’ A clarification or additional option could be added to the next survey. ● Question 34: How often does fixity fail for the listed reasons? ○ The next group may want to consider if they should add ‘Human error’ to the list of options provided. Another option to add may be ‘Hardware failure,’ addressing issues where the hardware went bad and caused the files to become corrupted. The result would be ‘Corrupted byte stream’ which is a current option; however, the reason for the corrupted byte stream is specific. ● Suggested new questions or topic areas ○ The 2021 survey didn’t assume that fixity checking involved using checksums. However, it may be interesting to know what tools they are using for creating/verifying fixity. Some of these tools were mentioned in the free-text fields, but adding a question specifically asking for this information would allow for analysis of the tools being used. ○ Additional questions that could be asked include: 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 80 [Instructional Text: Answer the following questions based on any type of fixity practices you are doing on either any portion or all of the content you are preserving for the long-term.] Question 11: Do you verify fixity information after transferring data from one location to another? ● Yes ● No ● Sometimes Question 12 [Matrix]: If Yes or Sometimes, when do you verify fixity information on any of the files you are preserving for long-term? [Scale: Never, Very rarely (<25% of the time), Sometimes (25-50% of the time), Frequently (>50% of the time), Always] ● Upon receipt of materials ● After moving files to new media ● After placing files in preservation storage ● After retrieving files for archival processing/description ● Other (please indicate) Question 13: For data at rest (i.e. in storage) do you check fixity information at regular time-based intervals? If so, please specify the intervals that your institution uses. Time intervals listed are for the interval on which the fixity verification is started, not necessarily completed. Select all that apply or the closest to the time interval that you employ: ● Hourly ● Daily ● Weekly or Biweekly ● Monthly ● Quarterly ● Every six months ● Yearly ● Every two years ● Continuously (automatically on a rolling basis) ● Do not check at regular intervals ● Other (please indicate) Question 14: Please provide any other relevant details about how often you verify fixity or rationale around differing fixity checking frequencies (e.g. based on storage location/collection/file format). Question 15: Do you check fixity information at regular intervals of all your preserved digital content? ● Yes ● No 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 81 Question 16: Do you check fixity information at regular intervals of a sampling of your digital content? ● Yes ● No Question 17 [Matrix]: What factors does your institution consider when determining fixity check frequency? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): ● Concern of media failure due to increased use of storage media (e.g. tape) ● Storage media reaching end of expected lifespan ● Throughput limitations (e.g., network bandwidth) ● Number and size of files or objects that require fixity checks ● The number of copies of the digital content that are held (i.e., the difference between if you preserve 2 copies versus if you preserve 7 copies) ● Reliance on checksums generated by storage providers (i.e. cloud providers and others) ● Terms of access by service providers (e.g., access to servers, cost of downloading data) ● Regular checks done at the block level via a system ● Environmental cost of computing checksums ● Available staff Question 18: Is your fixity verification done utilizing built-in hardware or is it software-based? ● Hardware ● Software ● Both hardware and software Question 19 [displayed if Software or Both hardware and software was selected in question 18]: What software, tools, or services are you using to capture/verify fixity information? Select all that apply: ● Scripts/custom code ● Automated/scheduled software ● Manually run software ● Third-party services (if yes, please provide details) ● Other (please indicate) Question 20 [displayed if Software or Both hardware and software was selected in question 18]: What type of fixity checking algorithm do you use? Select all that apply: ● Scripts/custom code 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 82 ● CRC ● MD5 ● SHA1 ● SHA256 ● SHA512 ● Other (please indicate) Question 21: Who is responsible for verifying your content's fixity information (e.g., who runs manual scans, schedules automated scanning, analyzes reports or logs, etc.)? Select all that apply. ● Administrator or manager ● Archivist, librarian, or curator ● System administrator ● Software developer/programmer ● Other IT staff ● Third-party service provider ● Other (please indicate) Question 22: Where do you store the preservation copies that are verified with fixity checking? Select all that apply: ● In-house online storage ● In-house nearline storage ● Offsite storage (including cloud service vendors) ● Other (please indicate) Question 23: Where does your institution record fixity information? Select all that apply: ● In object metadata records (e.g. a PREMIS XML file) ● In databases and logs ● Alongside content (e.g. an MD5 sidecar or bag manifest) ● In the files themselves (e.g., stored in the file header of a A/V file) ● Other (please indicate) Question 24: What level of granularity do you utilize when running checksums? Select all that apply: ● Per-block/folder/bag/etc. checksums (multiple files per checksum) ● Per-file checksums (one file per checksum) ● Partial-file checksums (multiple checksums per file) Question 25: If you run partial-file checksums (multiple checksums per file), what is your use case? [Section 3: Cloud Services; This section addresses fixity issues specific to using cloud storage services. For the purposes of this survey, cloud storage services are any remote third-party service used to store digital collections. Examples 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 83 include, but are not limited to: Amazon Web Services, Microsoft Azure, DuraCloud, APTrust, Chronopolis, and MetaArchive.] Question 26 [required]: Are you using cloud service vendors that offer fixity services? ● Yes ● No [if selected, skip to section 4] Question 27: Do you have the ability to run your own fixity services on the cloud vendor services? ● Yes ● No Question 28: Do you RECEIVE fixity information from the cloud service vendors that you may use as you see fit? ● Yes ● No Question 29 [displayed if response to question 28 was yes]: Do you USE the fixity information the cloud service vendors are providing? ● Yes ● No - if not, why not? Questio 30: Do you record fixity information provided by the cloud service vendors? ● Yes, in a software-based management system (such as a collections management or digital asset management system) ● Yes, outside of a formal management system ● No Question 31: Please provide any other information or requirements around using fixity information in conjunction with cloud vendor services, in as much detail as possible. [Section 4: Fixity Failures; This section asks about when the fixity verification process results in a failure and the actions you have taken to address those reports.] Question 32 [required]: How many times a year do you see fixity checks fail? ● Never [if selected, skip to section 5] ● Rarely (less than once a year) ● Infrequently (a few times a year) ● Frequently (multiple times per year) 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 84 Question 33 [Matrix]: How often do you see fixity failures during the events listed below? [Scale: Never, Rarely (less than once a year), Infrequently (a few times a year), Frequently (multiple times a year), I don’t know] ● While verifying fixity for content at rest ● While verifying fixity upon receipt ● While verifying fixity after moving files to new media ● While verifying fixity after storing files in repository ● While verifying fixity after retrieving files for archival processing/description ● Other (please indicate) Question 34 [Matrix]: How often does fixity fail for the listed reasons? [Scale: Never, Rarely (less than once a year), Infrequently (a few times a year), Frequently (multiple times a year), I don’t know] ● Corrupted byte stream (e.g. flipped bits) ● Interrupted network transfers (e.g. resulting in truncated files) ● Missing files (e.g. files in a manifest but not available) ● Extra files (e.g. files not in a manifest but in package) ● File reference lost (e.g. bytestream fixity maintained by file name changed) ● Other (please indicate): Question 35: What actions have you taken to address fixity failures? Select all that apply: ● Replace the file with a known good copy from your storage ● Request a new copy of the file from creator or digitization source ● Request a new copy from the third-party storage provider ● Remove the extra files ● Accept the file as-is and record fixity failure ● Other (please indicate) Question 36: Are there any noteworthy fixity failure events and responses that you would feel comfortable sharing? If so, please describe them below. [Section 5: About your Institution; This section provides us with basic demographic information about your institution.] Question 37 [required]: Which of the following most closely describes the type or function of your institution? ● Academic library or archives ● Academic institution department (not a library or archives) ● For-profit corporation ● Government entity ● Historical society ● Independent library or archives 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 85 ● Kindergarten - 12th grade ● Museum ● National, federal or legal deposit library ● Non-profit institution (not one of the above types) ● Public library ● Research data repository ● Research group ● Other (please indicate) Question 38: Where are you located? [drop down list] [Drop down country list was imported into the survey and is available on GitHub27 under kalinchernev/countries] Question 39: Would you be willing to discuss your fixity practices with us? We would like to expand on the survey by providing selected use cases. These would be used in a final report about the survey and/or as individual blog posts. Question 40: Is there anything you would like to clarify about your survey responses or share with us about your fixity practices? Appendix 3: Crosswalk between 2021 and 2017 survey questions To assist with comparing data between the 2021 and 2017 survey, the following table compares the questions from 2021 with the questions from 2017. This table does not include any information about survey logic, which is shown in the respective codebooks. 2021 Survey Questions 2017 Survey Questions Section 1: The Basics; This section addresses the basics of if and why your institution uses fixity information. For the purposes of this survey, fixity information is any information that can be used to monitor the stability of an object. Examples include but are not limited to: File names, File counts, File sizes, Checksums/Hash values Section 1: The Basics; this section addresses the basics of if and why your institution uses fixity information. 27 Kalinchernev / countries, GitHub, accessed September 8, 2021,https://gist.github.com/kalinchernev/486393efcca01623b18d. 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 86 Q1 Do your institutional practices include utilizing fixity information at any point in time? Q1 Do your organizational practices include utilizing fixity information at any point in time? Q2 What types of fixity information do you employ on files you are managing for the long-term? (Check all that apply) - Selected Choice NA Q2_5_TEXT What types of fixity information do you employ on files you are managing for the long-term? (Check all that apply) - Other (please enter) - Text NA Q3 Does your institution receive fixity information (created by another institution or a separate entity within your institution) along with digital content at the time of acquisition if it is available? Q2 Does your organization collect fixity information (created by another institution or separate entity within your organization) along with digital content at the time of acquisition if it is available? Q4 Please provide any relevant details about why you receive fixity information as frequently as you do. Q3 Please provide any relevant details about why you collect fixity information as frequently as you do. Q5 Does your institution capture fixity information for digital content if it is not provided at the time of acquisition? Please indicate how often you capture fixity information (never, very rarely, sometimes, frequently, always): Q4 Does your organization create fixity checks for digital content if they are not provided at the time of acquisition? Please indicate how often you collect fixity information: Q6 Please provide any relevant details about why you capture fixity information as frequently as you do. Q5 Please provide any relevant details about why you create fixity information as frequently as you do. Q7_1 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - Determine if the data has been corrupted or altered over time Q6_1 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - Determine if the data has been corrupted or altered over time Q7_2 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - Determine if the data has been corrupted or altered during transmission Q6_2 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 87 extremely important): - Determine if the data has been corrupted or altered during transmission Q7_3 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - To support the authenticity or trustworthiness of the digital objects Q6_3 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - To support the authenticity or trustworthiness of the digital objects Q7_4 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - To monitor hardware degradation Q6_4 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - To monitor hardware degradation Q7_5 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - For authenticity: To prove you are providing the digital object that has been requested Q6_5 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - For authenticity: To prove you are providing the digital object that has been requested Q7_6 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - To permit an update to a portion of a content file while proving the other portions remain unchanged (ex: split video files) Q6_6 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - To permit an update to a portion of a content file while proving the other portions remain unchanged (ex: split video files) 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 88 Q7_7 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - Meet requirements or best practice guidelines Q6_7 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - Meet requirements or best practice guidelines Q7_8 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - Help identify systemic or human error in the management of digital content Q6_8 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - Help identify systemic or human error in the management of digital content Q7_9 What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - Other (please indicate and rank as appropriate) Q6_9 What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - Other Q7_9_TEXT What are the reasons your institution uses fixity information? Please rate the importance of each of these items (not important, somewhat important, moderately important, extremely important): - Other (please indicate and rank as appropriate) - Text Q6_9_TEXT What are the reasons your organization collects, checks, maintains, and verifies fixity information? Please rate the importance of each of these items (not at all important, slightly important, moderately important, very important, extremely important): - Other - Text Section 2: Using Fixty Information; This section helps to communicate when, where and how fixity is being used in your institution for materials that are managed for long-term preservation purposes. Section 2: Where, When, and How; this section helps to communicate when, where and how fixity is being used in your institution. Q8 How much total content (preservation copies that are managed for long-term preservation only) are you running fixity on? - Selected Choice Q10 How much total content (preservation copies that are managed for long-term preservation only) are you running fixity on? - Selected Choice Q8_12_TEXT How much total content (preservation copies that are managed for long-term preservation only) are you running fixity on? - More than 5 PB: [Enter amount] - Text Q10_TEXT How much total content (preservation copies that are managed for long-term preservation only) are you running fixity on? - More than 500 TB. Please provide your answer in total number 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 89 of TB: Q9 Do you employ different fixity practices for different types of content or storage media? NA Q10 What factors influence your decision to use different fixity practices? (select all that apply): - Selected Choice NA Q10_7_TEXT What factors influence your decision to use different fixity practices? (select all that apply): - Other (please indicate) - Text NA Q11 Do you verify fixity information after transferring data from one location to another? Q7 Do you check fixity information after transferring data? Q12_1 If Yes or Sometimes, when do you verify fixity information on any of the files you are preserving for longterm? - Upon receipt of materials NA Q12_2 If Yes or Sometimes, when do you verify fixity information on any of the files you are preserving for longterm? - After moving files to new media NA Q12_3 If Yes or Sometimes, when do you verify fixity information on any of the files you are preserving for longterm? - After placing files in preservation storage NA Q12_4 If Yes or Sometimes, when do you verify fixity information on any of the files you are preserving for longterm? - After retrieving files for archival processing/description NA Q12_5 If Yes or Sometimes, when do you verify fixity information on any of the files you are preserving for longterm? - Other (please indicate) NA Q12_5_TEXT If Yes or Sometimes, when do you verify fixity information on any of the files you are preserving for longterm? - Other (please indicate) - Text NA Q13 For data at rest (i.e. in storage) do you check fixity information at regular time-based intervals? If so, please specify the intervals that your institution uses. Time intervals listed are for the interval on which the fixity verification is started, not necessarily completed. Select all that apply or the closest to the time interval that you employ: - Selected Choice Q8 Do you check fixity at regular intervals - please specify the intervals that your organization uses. - Selected Choice 2021 Fixity Survey Report; Results of the 2021 Fixity Survey 96 Q40 Is there anything you would like to clarify about your survey responses or share with us about your fixity practices? Q31 Is there anything else you would like to tell us about your practices around fixity?