scieee AI-readable full text Open interactive document viewer

When AI Met IRs: Bot Activity and the Institutional Repository

Makula, Amanda

Abstract

High download counts are a good thing -- right? But what happens when those numbers become unusually -- even alarmingly -- high? Are there truly human users at the other end, or is this an indication of AI bot activity? When the Digital Initiatives team at the University of San Diego noticed a sudden and substantial uptick in download numbers, we began asking questions: What's really going on? Is this cause for concern? How do we address this? Will our users be impacted? To explore and answer them, we consulted with the library’s Scholarly Communications Committee and corresponded with our Digital Commons representative. We enrolled in DC’s pilot program for anti-bot blocking, designed to reduce the number of suspicious downloads from targeted regions. Although these efforts helped us learn more about the situation, they also generated even more questions, such as long-term planning by commercial IR platforms to effectively address these issues, and what it means for open content more broadly, especially as it impacts human authors, creators, and users. Is this a "dark side" to open access? As AI bots become more and more sophisticated, how will IRs rise to the challenge? This presentation will encourage robust discussion among attendees about their own experiences and reflections surrounding AI bots’ mining of open access institutional repositories. Let’s get this important conversation started!

Full text

When AI Met IRs: Bot Activity and the Institutional Repository Amanda Makula Associate Professor & Digital Initiatives Librarian Institutional Context ●Private Catholic institution ●<10,000 students ●Undergraduate and graduate programs; R2 Carnegie classification ●Digital Commons repository platform, launched in 2014: “Digital USD” In early 2025, we began to notice . . . ● In download numbers; robust conversation on DC Google Group ●Lots of commercial entities heavily represented on our Dashboard, such as “BytePlus Pte Ltd”, a Singapore-based technology company, “Octopus Web Solutions” (Hong Kong), and “Menlo Security” (Silicon Valley) ●April 2025: A survey from a librarian at Appalachian State University circulated, asking IR managers about their experiences re: AI use of IR content So we began to ask . . . ●Are AI bots crawling our repository? (yes) ●Is this cause for concern? (depends) ●What should we do? Accurate download metrics Feeding AI high-quality scholarship Slowing down service Distinguishing “good” and “bad” actors Difficult/costly to stop Bots as intermediary to bring content to human users What should we do? Enroll in DC’s pilot program for anti-bot blocking Consult with our Digital Commons representative Bring this issue to the Scholarly Communications Committee Digital Commons Anti-Bot Pilot Program ●●Tests incoming hits from Brazil, Singapore, and Finland – countries with “high bot activity” according to DC ●Allows incoming hits from known search engine user agents ●Invisible to human users but block “a lot” of bots Results ●Reduced downloads from Brazil to roughly ⅕ – 1/10 of previous downloads ●Still seeing “suspicious” commercial entities in top downloading institutions on the Dashboard ●Bots are sophisticated and designed to mimic human use; do not always respect the rules that DC tried to implement ●DC rolls out “enhanced” program for ALL countries; 6 participating institutions (one reported a 90 % drop in downloads) Questions remain . . . Will “best practices” be developed, and who will lead them? Is this just the new reality for open content? Where does the responsibility lie? (Vendors? Managers? Authors?) “Aggressive AI Harvesting of Digital Resources” by Lyrasis “AI summaries insert themselves between a user seeking information and sources of the information, resulting in lower visits to the source websites where the information is created and maintained. Although this is primarily a concern to sites that rely on traffic for revenue, it also makes the impact of an institution's online presence more difficult, if not impossible, to measure.”