Building User-Centered Science Gateways Through User Imagination: Lessons from KnowCOVID-19

Building User-Centered Science Gateways Through User Imagination: Lessons from KnowCOVID-19 MD Ashraful Goni Media and Communication Texas Tech University Lubbock, Texas, USA [email protected] Roland Oruche Computer Science University of Missouri-Columbia Columbia, Missouri, USA [email protected] Prasad Calyam Computer Science University of Missouri-Columbia Columbia, Missouri, USA [email protected] Kerk F. Kee Media and Communication Texas Tech University Lubbock, Texas, USA kerk.k[email protected] Abstract—Science gateways are digital platforms that offer streamlined access to computational tools and data through intuitive user interfaces. While many have demonstrated technical success, achieving broad user adoption remains a challenge. Recent research highlights the growing importance of usercentered design in improving the usability and impact of these gateways. One critical yet often overlooked element of this approach is user imagination—the ability of users to envision features and functions that address their unique needs. This paper explores how user imagination influenced the development of the KnowCOVID-19 Science Gateway, a platform created to support medical professionals in navigating the vast body of COVID-19 research. Through interviews with ten medical professionals, the study illustrates how user insights shaped key design features, such as evidence-based filtering and chatbot support. The findings offer practical guidance for developers seeking to incorporate user imagination into the design process, emphasizing its essential role in building user-focused and widely adopted science gateways. Index Terms—Science Gateways, User-Centered Design, KnowCOVID-19 Gateway, User Imagination, Usability I. INTRODUCTION Science gateways are digital platforms designed to provide seamless access to computational and data resources through user-friendly interfaces, enabling research and education across various disciplines [1]. With the expansion of big data and increasingly complex research workflows, the design of science gateways must evolve to meet the diverse needs of researchers and professional communities. Traditional development approaches often prioritize technical feasibility over human-centered perspectives, potentially limiting user engagement and long-term adoption. Recent scholarship and practice, however, have begun to emphasize the importance of user-centered design in science gateways, calling for greater inclusion of end-users throughout the development process [2], [3]. This project was funded by the US National Science Foundation under grant numbers NSF-2006816 and NSF-2007100. We thank Eric Milman and Chaitra Kulkarni for their assistance in the early usability study. The KnowCOVID-19 science gateway was developed to address the urgent need among researchers and clinicians for fast, reliable, and organized access to COVID-19 literature during the early stages of the pandemic [2], [4]. During this period, the rapid increase in publications led to an overwhelming volume of information, making it challenging to quickly identify high-quality and relevant content. This problem is also known as ‘data-deluge.’ Science gateways provide an effective solution by streamlining access to trusted research. In response, our team created KnowCOVID-19 science gateway that features an artificial intelligence powered chatbot named Vidura. Vidura is designed to help medical professionals navigate the gateway more efficiently and locate the information they need. The gateway enables users to search and filter academic publications based on different levels of evidence, allowing them to focus on the most reliable and clinically useful sources for patient care [2], [4]. Despite numerous technical achievements, science gateways still face challenges in gaining widespread user adoption. Technical quality alone is not sufficient to ensure that these platforms are embraced by diverse research communities. In [5], seven external communication practices were highlighted that, when implemented, can support broader adoption of science gateways across scientific fields. Since gateways are fundamentally designed to serve users, it is essential to understand how users imagine, accept, and integrate these platforms into their workflows to ensure their long-term success [1]. A particularly underexplored yet critical aspect of usercentered design is user imagination. This refers to the ability of users to envision ideal features, workflows, and systems that do not currently exist but have the potential to enhance their research processes. Although imagination is not typically emphasized in usability studies, it plays an important role in how users express their needs, anticipate the usefulness of tools, and support innovative design ideas. According to diffusion of Innovations theory, the perceived attributes of an innovationrelative advantage, compatibility, complexity, trialability, and observability—influence the likelihood of its adoption [6]. User imagination plays a key role in shaping these perceptions early in the innovation process, especially in situations where systems are being created from the ground up or modified to serve new purposes. The use of imagination-based inquiry in the KnowCOVID19 project provides a valuable model for enhancing user involvement in the development of science gateways. In this project, users were not limited to reacting to existing prototypes; instead, they actively participated as co-designers by sharing their visions and ideas for what the gateway could become. This paper explores how user imagination shaped the design of the KnowCOVID-19 science gateway and supported the creation of a user-centered platform. By positioning user imagination as a key element in the design process, we offer practical insights for improving the development of science gateways. The central question guiding this study is: How can user imagination be effectively utilized during the design and development phase of a science gateway? To achieve our goal, this paper is organized as follows. First, we offer a brief review of key concepts, including usability, user experience, and user-centered design. Next, we describe the methods used for data collection. We then present our findings through a thematic analysis and share practical insights based on the results. Finally, we conclude the paper with a summary of key takeaways and implications for science gateway development. II. LITERATURE REVIEW User-Centered Design (UCD), Usability, and User Experience (UX) are foundational concepts in the design and evaluation of digital systems, including science gateways. While these terms are closely related, each represents a distinct but complementary perspective on how systems should be designed with users in mind. UCD is an iterative design process that places the needs, preferences, and limitations of end-users at the core of every stage of development [7]. Usability, a central goal of UCD, refers to the extent to which a product can be used by specific users to achieve specific goals with effectiveness, efficiency, and satisfaction in a particular context [7]. UX, while related to usability, encompasses a broader scope that includes emotional responses, engagement, and overall perception of the system [8]. A recent study [3] on Gateways developers highlights the importance of integrating UX principles into the development of science gateways. It shows that early and intentional attention to UX can result in platforms that are more accessible, effective, and centered around user needs. Drawing on semi-structured interviews with 18 individuals from various science gateways who had previously participated in usability consulting through the Science Gateways Community Institute (SGCI), the study examined how UX is understood, implemented, and valued within different teams. The findings suggest that embracing a proactive and design-focused approach, rather than treating UX as a secondary concern, can significantly improve the usability and long-term adoption of science gateways [3]. For instance, the Event Horizon Science Gateway applied UX strategies to reduce barriers to adoption and enhance user-friendliness [9]. In user-centered fields such as science gateways, involving end users such as scientists, educators, and researchers in the design process can uncover important needs and ideas that might otherwise be missed. Research in co-design and participatory innovation shows that when users are actively engaged in imagining future technologies, the outcomes are often more relevant and more likely to be adopted. For example, one study [10] describes a community-based speculative design project in which local participants worked together to envision future urban technologies. Through a series of design fiction workshops, community members imagined new tools such as augmented reality applications and smart wearable devices that could support the common good. The technology concepts that emerged from these workshops were closely aligned with the community’s values and needs, illustrating how user foresight and creative thinking can play a direct role in shaping the direction of innovation. Science gateway communities can adopt similar approaches to develop user-centered platforms tailored to the specific needs and values of their communities. III. METHOD 1) Interview Protocol: This study employed a qualitative interview approach to explore the imagination of medical professionals on designing a science gateway. The interview protocol consisted of 15 open-ended questions, with the first six focused on understanding participants’ typical routines and experiences when searching for research publications related to their medical or professional fields. The interviews were conducted using a semi-structured format, allowing for flexibility in both the order and phrasing of questions. This structure enabled interviewers to adjust their questions based on participants’ responses and to follow relevant lines of inquiry as they emerged. Before beginning the interview, participants were asked to watch a short video introducing the KnowCOVID-19 science gateway and explaining how the level of evidence pyramid can support medical professionals in refining their literature searches. If a participant had not watched the video beforehand, they were given the opportunity to do so after the sixth question. Following the video, the remaining interview questions focused on gathering participants’ perspectives on how the KnowCOVID-19 gateway, with its evidence-based filtering features, could support more efficient and targeted literature searches during the pandemic. 2) Analysis: Our analysis centers on the themes that emerged from interviews with medical professionals, focusing specifically on their imagination of how a science gateway could address the challenges they faced in searching for COVID-19 literature. To facilitate this analysis, we used Otter.ai, an artificial intelligence-based transcription service, to produce verbatim transcripts of each interview. We carefully reviewed the audio recordings alongside the transcripts to ensure accuracy and to identify recurring themes. We began our thematic analysis with open coding, a qualitative method used to break down and categorize data into distinct concepts [11]. This allowed us to compare and refine emerging ideas. We then applied axial coding to examine how these themes relate to one another, organizing them into coherent categories [11]. This process helped us uncover key barriers and design considerations relevant to science gateway development. To protect the confidentiality of our participants, we assigned each one a unique code ranging from P01 to P10, which we use throughout the paper in place of their names. IV. FINDINGS To address three major challenges faced by medical professionals when searching for COVID-19 literature – slow search speed, diverse user needs, and the absence of a quality-based ranking system—the KnowCOVID-19 science gateway incorporated a level of evidence pyramid [4]. This feature allows users to quickly access high-quality publications, improving both the speed and reliability of their searches. In addition, the gateway introduced Vidura, an artificial intelligence powered chatbot designed to support users as they navigate the platform and filter search results. Although the development team had an initial plan to integrate the evidence pyramid and the Vidura chatbot, the imagination and input of participants helped refine these features. Their perspectives were valuable in shaping a more usercentered gateway by placing users and their experiences at the heart of the design process. The following section presents our findings under two main themes: User imagination of the level of evidence pyramid and user imagination of the Vidura chatbot. These themes reflect how participant input influenced and enhanced the development of the KnowCOVID-19 science gateway. 1) User Imagination of the Level of Evidence Pyramid: Participants shared their imagination on how the level of evidence pyramid could support their search processes and enhance the usefulness of the KnowCOVID-19 science gateway. One participant emphasized the importance of focusing on highquality sources, stating, “What we always strive for, is to find articles with the highest level of evidence because you can always find any article out there” (P02). This reflects a strong desire for filtering mechanisms that prioritize credibility over quantity. Another participant appreciated the feature’s ability to organize information meaningfully: “I like the level of evidence in the fact that we can search by level of evidence. I think I think that’s a very, very cool feature” (P03). This participant also shared a preferred method for reviewing content, noting, “I would like to review probably the highest level of evidence first, for example, randomized control trials first, and systematic reviews” (P03). The pyramid also addresses gaps in access to overlooked sources. As one participant noted, “But it [level of evidence pyramid] definitely will help at least the journals that that I don’t subscribe their emails and all they probably miss and it can help in that” (P04). Another participant pointed to its practical grounding in medical standards, saying, “We can actually bang it off the level of evidence that really exists” (P05). Specific preferences also emerged, such as, “I want to concentrate on systematic reviews only” (P09), and others highlighted the potential for contextual integration: “If you can link in the context of COVID-19” (P01). Another participant pointed to the importance of trustworthy filtering, stating, “Giving you confidence in the results that you have with the understanding that to get to the highest level of evidence, you have the smallest number of articles” (P06). The overwhelming volume of research during the pandemic was also a factor: “I think it [level of evidence pyramid] would be helpful, just because there has been such an explosion” (P07). Additional insights suggested tailoring the gateway experience based on different types of users. As one participant explained, “The thing that you have to realize is clinicians are a different group of people than researchers” (P08). This distinction highlights how clinicians may prioritize speed and summaries, while researchers may focus on in-depth analysis. Another participant provided a helpful ranking structure, stating, “So I would say first relevancy. And then publication type, then research design, maybe” (P10). These imaginations offer valuable guidance for refining the level of evidence feature to accommodate a range of professional needs in the KnowCOVID-19 science gateway. 2) User Imagination of the Vidura Chatbot : Participants offered a range of perspectives based on their imagination on the Vidura chatbot, reflecting both their expectations and concerns regarding its role in enhancing user support within the KnowCOVID-19 science gateway. One participant, reflecting on past experiences with other digital platforms, emphasized the value of guidance during use: “It would be helpful to have some guidance rather than floundering around and then by yourself” (P01). Another participant expressed a clear willingness to engage with the chatbot, stating, “Yeah, I think that that’s a feature that I will use. Because I may have questions during the search, and perhaps the chatbot can guide me in the right direction” (P03). These comments underscore the perceived usefulness of real-time, interactive support during complex information-seeking tasks. The combination of automation and evidence-based ranking was seen as a unique strength of Vidura by another participant: “To me it [Vidura] combines keywords with a objective ranking of the quality of the data. I think those two things alone are a huge boon” (P05). Still, some participants raised critical questions that helped the development team refine the chatbot. One asked, “Then why do we need chatbots? When we have Google” (P04), while another expressed uncertainty, stating, “I have some doubts as to whether it would be able to filter that information to give me the quality metrics that I want” (P06). The need for simplicity and clarity in chatbot interactions was also raised. One participant noted frustration when chatbots fail to interpret user input accurately and emphasized the need for systems that understand user queries well. He mentioned “For a guy like me, who [he] needs user friendly [Chatbot]” (P09). In contrast, another participant highlighted a powerful benefit of chatbot technology: “I really love about a chatbot is if that kid then can connect someone, they can actually ask a question in real time” (P10). Participants also offered forward-looking suggestions. One envisioned broader application: “But if you could ultimately build something like this that would do this for any particular set of keywords I was interested in searching for. That would be tremendous” (P01). Not all participants were comfortable with chatbot use. As one stated, “I’m also old school and if I really had an issue, I’m going to talk to someone” (P02), and another admitted, “I’m just not a big chatbot system kind of person” (P07). These perspectives made it clear that some users may prefer traditional forms of support, reinforcing the need for an intuitive user interface that minimizes reliance on chatbot assistance. Finally, comparisons with human help highlighted further expectations. A participant spoke fondly of a librarian who “Helps you with everything that you need” (P06), suggesting that if Vidura could imitate such personalized support, it would be transformative. Similarly, another participant reflected on chatbot limitations, stating, “What comes to mind when you think about a chatbot that gave you that, that impression that it may not be like operating as effectively as a human being like a as a librarian” (P10). These insights call for designing chatbot features that prioritize empathy, clarity, and relevance and are capable enough to help KnowCOVID-19 science gateway users. V. CONCLUSION In this paper, we presented an example of how user imagination can be effectively utilized during the design and development phase of a science gateway, using KnowCOVID19 as a case study. Insights from users about the level of evidence pyramid and the Vidura chatbot played a critical role in shaping the design of the gateway to better align with their needs. We received positive feedback in later usability studies, which we attribute to incorporating user imagination early in the development process [2]. However, not all user suggestions could be fully implemented. For example, some participants imagined the chatbot functioning as effectively as a human librarian. In practice, our usability testing revealed that certain users were dissatisfied with the chatbot’s performance, reflecting a gap between expectations and current capabilities. This highlights the ongoing and iterative nature of user-centered design. We continue to improve the Vidura chatbot to better meet user needs, but our experience demonstrates that involving users early in the design process leads to more effective and responsive science gateways. A key insight from this study is that user imagination plays a crucial role in creating user-centered science gateways and in driving the successful adoption of innovation. Science gateways are most effective when they are not only technically robust but also thoughtfully imagined and strategically developed. Two communication strategies discussed in prior research [5]: networking with the community and building relationships based on trust—can be applied during the early stages of science gateway development through approaches such as market interviews, participatory workshops, or focus group discussions. By incorporating user imagination into the design process, developers can create gateways that align more closely with user needs and demonstrate clear advantages that users can easily recognize. In essence, a science gateway shaped by users’ own ideas is more likely to address the problems that matter to them and integrate smoothly into their existing workflows. This approach not only fosters greater user satisfaction but also encourages long-term engagement and trust in the platform. As the development of science gateways continues to evolve, placing user imagination at the center of design will be essential for building tools that are both innovative and impactful. REFERENCES [1] S. Gesing, S. Brandt, S. Bradley, M. Potkewitz, K. Kee, N. Whysel, M. Perri, S. Cleveland, A. Rugg, and J. Smith, “A vision for science gateways: Bridging the gap and broadening the outreach,” in Practice and Experience in Advanced Research Computing 2021: Evolution Across All Dimensions, ser. PEARC ’21. New York, NY: Association for Computing Machinery, Jul. 2021, pp. 1–8. [Online]. Available: https://dl.acm.org/doi/10.1145/3437359.3465579 [2] M. A. Goni, R. Oruche, M. Uddin, O. Lawal, P. Calyam, and K. Kee, “User competence metrics for science gateways: The case of the KnowCOVID-19 gateway,” in Gateways 2024, Bozeman, MT, Oct. 2024. [Online]. Available: https://zenodo.org/records/13868494 [3] P. C. Parsons, S. Krishnaraj, M. Berent-Spillson, and J. Gerber, “What is user experience to science gateways?” in Gateways 2024, Bozeman, MT, Oct. 2024. [Online]. Available: https://zenodo.org/records/13864426 [4] B. W. Shibo, R. Oruche, M. A. Goni, X. Cheng, E. Milman, P. Calyam, and K. Kee, “Challenges faced by medical professionals as gateway users: The case of Knowcovid-19,” in Science Gateways 2023 Annual Conference, Pittsburgh, PA, Oct. 2023. [Online]. Available: https://zenodo.org/records/10028564 [5] K. F. Kee, B. Le, and K. Jitkajornwanich, “If you build it, promote it, and they trust you, then they will come: Diffusion strategies for science gateways and cyberinfrastructure adoption to harness big data in the science, technology, engineering, and mathematics (stem) community,” Concurrency and Computation: Practice and Experience, vol. 33, no. 19, p. e6192, 2021. [6] E. M. Rogers, Diffusion of Innovations, 5th Edition, 5th ed. New York London Toronto Sydney: Free Press, Aug. 2003. [7] T. Jokela, N. Iivari, J. Matero, and M. Karukka, “The standard of user-centered design and the standard definition of usability: analyzing iso 13407 against iso 9241-11,” in Proceedings of the Latin American conference on Human-computer interaction, 2003, pp. 53–60. [8] G. Walton, “What user experience (ux) means for academic libraries,” New Review of Academic Librarianship, vol. 21, no. 1, pp. 1–3, 2015. [9] R. Quick, E. G. Shelton, J. Wang, and A. West, “The event horizon science gateway,” in Proceedings of the Practice and Experience in Advanced Research Computing (PEARC ’24). New York, NY: Association for Computing Machinery, 2024. [Online]. Available: https://doi.org/10.1145/3626203.3670585 [10] K. Baumann, B. Stokes, F. Bar, and B. Caldwell, “Infrastructures of the imagination: Community design for speculative urban technologies,” in Proceedings of the 8th International Conference on Communities and Technologies, 2017, pp. 266–269. [11] J. A. Holton, “The coding process and its challenges,” The Sage handbook of grounded theory, vol. 3, pp. 265–289, 2007.