Diversification of Research Methods in Information Science

Diversification of Research Methods in Information Science Brady D. Lund University of North Texas, Department of Information Science Abstract This chapter provides a historical overview of the evolving landscape of research methods in information science, examining articles from seven well-established information science journals for the decades of 1960 to 2020. The chapter begins by presenting an overview of the early studies that categorized research methods in information science, emphasizing the groundbreaking work of researchers like Jarvelin and Vakkari in the 1990s, among others. It then details the emergence and trajectory of non-traditional methods within information science over recent decades. An original analysis of research methods in seven top journals illustrates the emergence of qualitative approaches and greater methodological diversity within information science, while providing a compass for exploring the discipline’s future trajectory. Keywords: information science, research methods, overview, qualitative methods, methodological diversity, content analysis The study of information science is a dynamic field that is constantly evolving. One key aspect of this evolution is the growth of methodological diversity within the discipline. This chapter explores the various research methods that have been used within the field of information science over time, and how these methods have evolved and changed in response to the needs and goals of the research being conducted. The chapter begins by reviewing past studies that document the history of research methods within information science, and then presents an original analysis of research methods used in articles published in seven long-published information science journals. The chapter concludes by discussing some of the more compelling and emerging research methods within information science, and how these methods are helping to shape the direction of the field. Evolution of the Study of Research Methods in Information Science: A Historical Overview Trends that occur in the usage of research methods tell researchers much about the evolving nature of our disciplines. Disciplines that lack methodological diversity and rely almost exclusively on one category of data (quantitative or qualitative) are less resilient when faced with the natural shifts in the popularity and relevance of research topics and timely issues, whereas those disciplines that are methodologically diverse are able to adapt to solve new issues and questions that arise. An understanding of popular research methods and their change over time is valuable for directing new researchers in a field of study. Though researchers should not be afraid to adopt novel methods in their work, academic journals, editors, and peer reviewers tend to look for the popular methods when making a determination of a study’s reliability (Roberts & Priest, 2006). Across the past half-century, interest in the nature of information science research methods has grown tremendously. Schick’s (1962) study, “library science research,” was among the first to explore this topic. This study categorized research studies published between 1956 and 1961 based on whether they used statistical (156) or non-statistical (398) means of analysis, with surveys being common among those statistics-based studies. Over the next couple of decades, several other articles followed that showed that a majority of articles in information science research seemed to use non-empirical methods or focus on single cases (i.e., case studies), though some areas of the emerging discipline (such as information behavior and user studies) were more likely to use diverse methods. These studies of methodological innovation in early information science included Atherton (1975), Bates (1971), and Peritz (1980). Feehan et al. (1987), in looking at information science research articles published in 1984, found that this research employed relatively simplistic research methods, but there were some indications of growing sophistication. A period of maturation had begun to occur that was mirrored in broader developments within information science, such as the shift from a systems to a users focus (Dervin & Nilan, 1986). It is Jarvelin and Vakkari’s (1990; 1993) research during the early 1990s that really revolutionized the study of research methods. Using a comprehensive content analysis approach, these researchers documented trends in research methods from 1965 to 1985 in top information science research journals. The authors found that information science research during this period was largely dominated by conceptual, literature-based, and questionnaire/survey approaches. Qualitative methods were particularly limited, with less than 1% of studies using this type of approach in 1965 and 1975, and only above 1% in 1985. Jarvelin and Vakkari’s work has received considerable attention and acclaim, receiving hundreds of citations over the past three decades. Julien (1996) adopted an approach inspired by Jarvelin and Vakkari’s work to examine research methods used in the specific subject area of information behavior. In this study, Julien found that information behavior research overwhelmingly employs survey-based approaches, with the written questionnaire representing 44% of methods used and the interview representing 11%. Julien excluded articles that lacked an empirical basis (i.e., conceptual papers/commentaries). Julien et al. (2011) followed up the 1996 study by examining trends in information behavior research from 1999 to 2008. In the follow-up study, only modest change was found in methods used, with questionnaires representing 34.7% of studies and interviews representing 10%. Information behavior has always been a bit more progressive methodologically than information science research as a whole, so trends in this research area are not necessarily indicative of all of information science research. However, it is noteworthy that the past studies of Feehan et al. (1987) and Jarvelin and Vakkari (1990; 1993) showed information science studies as largely descriptive and likely to undergo rapid change but Julien’s studies show much more variety and stability, suggesting that a shift may have indeed occurred during this period of the early 1990s and has since stabilized. Rochester (1995) used Jarvelin and Vakkari’s approach to study Australian library journals, where they found that Australian researchers particularly favored the survey/questionnaire method (44% of studies used this approach from 1985 to 1994). Yontar and Yalvac (2000) adapted Jarvelin and Vakkari’s approach to specifically study information science research in Turkey. Similar to many past studies, the researchers found that most of the information science research in Turkey used a conceptual or literature-based approach, though methodological diversity had improved somewhat in the most recent years. Hider and Pymm (2008) also directly adopted Jarvelin and Vakkari’s methods but followed Feehan et al. in only examining a single year (in this case, 2005), which they then compared back to Jarvelin and Vakkari’s (1993) past findings. Interestingly, Tuomaala, Jarvelin, and Vakkari’s (2014) article also looked at trends in 2005 compared to Jarvelin and Vakkari’s past studies. The percentages found for each study were not identical, but were relatively similar. For instance, Hider and Pymm (2008) had the survey/questionnaire method dropping in popularity from 41.0% in 1985 to 30.5% in 2005, whereas Tuomaala et al. (2014) had it dropping to 34.4% in 2005. Both studies also noted growth in qualitative methods from 1985 to 2005, with Hider and Pymm (2008) having it jumping from 2.9% to 5.5% of studies and Tuomaala et al. (2014) having it going from 2.9% to 7.5%. One of the issues with comparison of research methods over time and different researcher groups is the inclusion criteria for journals/literature included in the analysis. Tuomaala et al. (2014) used a different set of “key” journals for each year studied (1965, 1985, 2005). Subsequent studies, such as Hider and Pymm (2008) and Chu (2015) also used different sets of journals. This is important because the methods used in information science research can very greatly among journals. Aytac and Slutsky (2014) noted that most literature in librarianship-oriented journals remains largely descriptive and quantitative in nature. Fidel (2008) notes substantial differences in the rates of mixed-methods research published among different information science journals. Zhang et al. (2016) note major differences in the use of statistical methods across information science journals. Jarvelin and Vakkari’s presence in the study of library and information science research and research methods remains ubiquitous into recent years. Articles that either directly reference Jarvelin and Vakkari’s articles as inspiration for their approach or show clear influence through the classification taxonomies or terminology used include Van Scoy and Fontana’s (2016) study of reference services studies (more directly influenced by Julien’s work, however Julien was influenced by Jarvelin and Vakkari’s studies), Ullah and Ameen’s (2018) study of library and information science research trends, and Ma and Lund’s (2021) study of recent evolution of information science research subtopics. Jarvelin and Vakkari themselves returned for a 2021 study that summarizes 50 years of research trends in information science (1965 to 2015), as well as a 2023 paper that examines the relationship of researchers’ disciplinary backgrounds on their contributions to library and information science (Vakkari et al., 2023). Recently, several studies have looked at one particular research method and how it has evolved in information science research publications. Khoo et al. (2012) looked at ethnographic approaches, finding a considerable growth in these approaches in the 2000s. Armann-Keown and Patterson (2020) found a giant increase (833%) in the use of content analysis as a research method in select information science publications. Lund (2020) similarly notes considerable growth in the use of the Delphi method in information science research. While the studies discussed in this section generally indicate that library and information science research has traditionally been dominated by the positivist-structuralist paradigm, with quantitative methods favored over qualitative, and objective over subjective/interpretive, the recent studies by Tuomaala et al. (2014), Ma and Lund (2021), and Jarvelin and Vakkari (2021) indicate a sea change that has resulted in greater adoption of the qualitative approach, while researchers like Julien (1996) suggest that certain research areas within information science have also been more likely to embrace methodological diversity. By documenting these changes, it becomes obvious to readers that acceptance of non-traditional methods has grown and that they themselves may feel encouraged to adopt such methods in their own studies. A Look at Commonly-Used Research Methods Before delving further into the evolving trends in research within the field of Library and Information Science (LIS), it is crucial to establish a foundation by familiarizing ourselves with the most commonly-used research methods. This section is designed to provide a comprehensive orientation to these methods, ensuring that readers have a clear understanding of their fundamental principles and applications. By gaining proficiency in these research methods, readers will be better equipped to appreciate the major shifts and developments that have shaped LIS research over time. Conceptual/Essay The conceptual/essay approach has historically been widely used in research studies, but its popularity has diminished in recent decades. This method involves introducing and discussing new concepts, theories, or ideas through persuasive arguments, sometimes with references to existing literature. Unlike a literature review, a conceptual study does not follow a structured approach to analyzing literature, but rather selectively incorporates relevant literature to support the argument. It is important to note that this approach does not involve empirical analysis of data. Case Study Case studies have been a prevalent research method in the field of information science, especially in library or practitioner-focused journals, over the past few decades (Lund & Wang, 2021). However, similar to other disciplines like education and business research, case studies have experienced a decline in recent years, possibly due to concerns about their perceived lack of rigor. Nonetheless, well-designed and theoretically informed case studies can still provide valuable insights for both practitioners and researchers. The close connection between information science and service-based professions further contributes to the prevalence of case studies in this field. Survey The survey method has gained popularity in recent years, likely because it is perceived as more rigorous and allows for statistical analysis through diverse sampling. While some information science journals have shifted towards embracing more diverse methods, particularly qualitative approaches, surveys have increasingly been adopted by library-focused journals. The convenience of online questionnaire administration, particularly for busy practitionerresearchers, may also be a contributing factor to its popularity. Interview Qualitative methods, particularly interviews, have witnessed a surge in popularity within the field of information science. Although the use of this method was relatively limited in the 1960s and 1970s, it has seen significant growth since the 1980s, with recent studies indicating that approximately one in ten articles published in the past decade have employed this method (Lewis-Beck et al., 2003). Interviews involve posing questions to participants and collecting their responses, with the structure and nature of the questions ranging from rigid lists to more conversational approaches. Regardless of the specific approach, the fundamental method of data elicitation remains the same. Interviews are generally regarded as rigorous yet straightforward research tools. Experiment Experiments are a key part of scientific discovery, particularly in the natural sciences and psychology, where they are used to uncover the causes and correlations of natural phenomena. The experimental method has also been used in some instructional studies within the field of information science, particularly for evaluating the effectiveness of different pedagogical approaches. However, this method has not been as prominent in library-centric journals compared to other areas of the discipline. A Closer Look at Trends in Research Methods From 1960 to 2020 To present an original study that illustrates the trend in information science research methods, an analysis was performed by the author that examines all research, or data collection, methods used in articles published from 1960 to 2020 in seven, long-running and renowned journals: • Journal of the Medical Library Association (JMLA), which was formerly called the Bulletin of the Medical Library Association, and began its publication in 1911; • Library Quarterly (LQ), which has been published under the same name since its inception, and which began publication in 1931; • College and Research Libraries (C&RL), which has been published under that name since its inception, and began its publication in 1939; • Journal of Documentation (JDOC), which has been published under the same name since its inception, and which began publication in 1945; • Aslib Journal of Information Management (ASLIB), which was formerly known as the Aslib Proceedings, and which was first published in 1949; • Journal of the Association for Information Science and Technology (JASIS&T), which was formerly known by the names The Journal of the American Society for Information Science and Technology, Journal of the American Society for Information Science, and American Documentation, and began publication in 1950; • Library Trends (LT), which has been published under that name since its inception, and began its publication run in 1952. The newest of these journals began publication in 1952, while the oldest is over 110 years old, and the average age is approximately 83 years (publication start year of 1940). All of these journals are highly-regarded in the discipline of information science, both in terms of quantitative and qualitative measures. All seven journals rank among the 18 journals that Nixon (2014) indicates are “tier one” journals in library and information science, and all rank highly based on Manzari’s (2013) findings of LIS journal prestige, including three of the top four journals. Each of these journals is also included in the Social Science Citation Index (SSCI), with an average journal impact factor (JIF) of 1.87, which places them among the very top journals of information science. A total of 16933 peer-reviewed articles were published by these seven journals over the sixty-year period. 29.1% of these articles were published in JASIST, 14.5% in JMLA, 14.2% in LT, 13.4% in both ASLIB and C&RL, 9.2% in JDOC, and 6.2% in LQ. The rate of publication among the journals increased by decade: 2307 articles were published in the 1960s, 2309 in the 1970s, 2210 in the 1980s, 2712 in the 1990s, 3481 in the 2000s, and 3914 in the 2010s. Articles were sorted into one or more of 13 methods categories based on the method(s) used in the study. These methods are case study (including “how we did it” articles), conceptual/essay (proposing theories or new concepts for library and information science), Delphi method, ethnography, experiment, focus group, interview, literature review, observation, scientometric methods, and survey/questionnaire. Many articles were sorted in an automated fashion, using keyword matching (e.g., if the word “experiment” appeared within the article content downloaded from Web of Science, but no other keywords like “interview,” “questionnaire,” or “survey,” then it could be coded as “experiment”). Articles that could not be matched using keywords were then reviewed manually be the author. The percentage of papers to utilize each method was then calculated. The results are shown in Table 1. Table 1. Trend in Usage of Methods in Information Science Research 1960s 1970s 1980s 1990s 2000s 2010s Average Case Study 30.1% 24.7% 20.9% 16.6% 13.1% 10.3% 19.3% Conceptual 39.0% 37.7% 37.6% 23.0% 15.8% 13.4% 27.7% Content Analysis 0.1% 0.4% 0.7% 2.3% 3.4% 4.4% 1.9% Data Analytics 6.7% 8.5% 9.5% 12.3% 16.5% 16.2% 11.6% Delphi 0.0% 0.0% 0.0% 0.4% 1.0% 0.3% 0.3% Ethnography 0.0% 0.0% 0.1% 0.6% 0.6% 1.0% 0.4% Experiment 5.3% 6.8% 8.3% 14.3% 16.6% 14.8% 11.0% Focus Group 0.0% 0.0% 0.0% 0.7% 1.2% 1.9% 0.6% Interview 0.2% 0.2% 1.3% 3.5% 6.5% 10.3% 3.7% Literature Review 1.3% 2.2% 2.1% 4.3% 3.5% 3.1% 2.8% Observation 0.3% 0.6% 0.7% 2.3% 3.4% 3.6% 1.8% Scientometric 1.6% 2.2% 3.4% 1.8% 1.0% 1.1% 1.9% Survey 15.4% 16.7% 15.3% 17.8% 17.3% 19.3% 17.0% Several notable shifts occurred in research methods over the past six decades. In the 1960s, 69.1% of studies used a case study or conceptual approach – in essence, a method that does not require direct empirical investigation and data collection. In the 2010s, only 23.7% of studies used one of these approaches. That is a decline of nearly two-thirds (65.7%) over that time period – a difference in frequency of these studies that is demonstrated as statistically significant based on a chi-square test, X2 = 1238.7, p < .001. In fact, the drop in usage of these two methods is almost identical, with a 65.6% drop for conceptual studies and a 65.8% drop for case studies. Meanwhile, there were several methods that were virtually non-existent in LIS research in the 1960s that are common today, such as content analysis (4300% growth), interview (5050% growth), and observation (1100% growth). Data analytics and experimental methods have also experienced considerable growth in usage. All methods except case study, conceptual, and scientometrics have experienced some growth in recent decades. Methodological diversity has become increasingly common. As of the 2010s, 17% of studies used some type of qualitative method (ethnography, focus group, interview, observation), compared to only 2% of studies in the 1980s and only 0.5% in the 1960s, which again represents a statistically significant difference based on a chi-square test, X2 = 406.0, p < .001. Since the 1980s, growth in the percentage of qualitative studies has been consistent at about 5% of all studies each decade: 2% of studies were qualitative in the 1980s, 7.1% in the 1990s, 11.6% in the 2000s, and 16.9% in the 2020s. However, these findings indicate that the research published in these information science journals continues to be driven by the quantitative research paradigm. For many of the methods studied, there is no significant difference in usage between “library” journals (journals like JMLA, C&RL, LQ, and LT, which have the word “library” in their titles) and other information science journals (JASIS&T, JDOC, and ASLIB), however there are a few exceptions. Library journals have consistently had higher proportions of case studies. In the 1960s an 1970s, this distinction was substantial. Library journals averaged 44.9% of articles using a case study/”how we did it” approach, whereas only 13.4% of articles in the other information science journals used this approach. In the most recent two decades, 16.2% of articles in library journals used a case study, while only 8.5% of articles in other information science journals did the same. There is also a difference in the use of experimental methods. Only 6.6% of library journal articles used this approach (8.2% in the most recent decade) compared to 13.7% of articles in the other information science journals (18.6% in the most recent decade). Though the survey method appears to have maintained an overall fairly steady usage rate over the past six decades, when library journals are compared to other information science journals two very distinct trajectories are revealed. Among library journals, the rate of survey method usage remained low for the first three decades examined (1961-1990) at only 4.2%, as case studies were extremely common. However, as case studies suddenly became far less common in the 1990s, a shift occurred and the use of the survey method among library journals jumped to 21.6% from 1991-2020. The exact opposite trend occurred among the other information science journals. 25.8% of articles published in these journals between 1961 and 1990 used survey methods; however, as methodological diversity increased in information science research, the use of the survey approach dropped to only 15.3% of studies between 1991 and 2020. Qualitative methods became more common during the last three decades, especially among information science journals, where 12.0% of articles used a qualitative approach, compared to only 7.0% of the library journals. Emergence of Unique Methods Several methods that are more unique or emerging within information science were found in this examination, including ethnography, Delphi method, and data analytics. Here, a closer look at the nature of these methods within the literature is provided. Ethnographic Methods Ethnography is a broad approach to research that often incorporates several other forms of data collection, such as observation and interviews, to understand a specific culture or group of people, as depicted in Figure 1. Jamali (2018), in his study of all library and information science publications from 2003 to 2013, notes that the use of ethnographic methods grew 600% from 2003 to 2013, with much of the growth occurring in the top journals, including JASIS&T and JDOC. In this study, 71 ethnographic papers were identified among the seven journals studied, with 19 published in “library” journals and 52 in the other information science journals. The earliest of these articles was published in the mid-1980s, but these articles emerged with greater frequency in the 1990s. Early ethnographic studies utilized the approach primarily to examine information behaviors in context, especially in schools and workplaces. The approach was Forster, M. (2019). Ethnographic thematic phenomenology: A methodological adaptation for the study of information literacy in an ontologically complex workplace. Journal of Documentation, 75(2), 349-365. Gerring, J. (2017). Case study research: Principles and practices. New York, NY: Cambridge University Press. Hider, P., & Pymm, B. (2008). Empirical research methods reported in high-profile LIS literature. Library and Information Science Research, 30(2), 108-114. Jamali, H. R. (2018). Does research using qualitative methods (grounded theory, ethnography, and phenomenology) have more impact? Library and Information Science Research, 40(3/4), 201-207. Jarvelin, K., & Vakkari, P. (1990). Content analysis of research articles in library and information science. Library and Information Science Research, 12(4), 395-421. Jarvelin, K., & Vakkari, P. (1993). The evolution of library and information science, 1965-1985. Information Processing and Management, 29(1), 129-144. Jarvelin. K., & Vakkari, P. (2021). LIS research across 50 years: Content analysis of journal articles. Journal of Documentation, 78(7), 65-88. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. Julien, H. (1996). A content analysis of the recent information needs and uses literature. Library and Information Science Research, 18, 53-65. Julien H., Pecoskie, J. L., & Reed, K. (2011). Trends in information behavior research, 19992008: A content analysis. Library and Information Science Research, 33(1), 19-24. Khoo, M., Rozaklis, L., & Hall, C. (2012). A survey of the use of ethnographic methods in the study of libraries and library users. Library and Information Science Research, 34(2), 8291. Lewis-Beck, M., Bryman, A. E., & Liao, T. F. (2003). The Sage Encyclopedia of Social Science Research Methods. Sage Publications. Linstone, H. A., & Turoff, M. (1975). The Delphi method: Techniques and applications. Reading, MA: Addison-Wesley. Logue, M. D., & Effken, J. A. (2013). Validating the personal health records adoption model using a modified e-Delphi. Journal of Advanced Nursing, 69(3), 685-696. Lund, B. D. (2020). Review of the Delphi method in library and information science research. Journal of Documentation, 76(4), 929-960. Lund, B. D., & Ma, J. (2021). A review of cluster analysis techniques and their uses in library and information science research: k-means and k-medoids clustering. Performance Measurement and Metrics, 22(3), 161-173. Lund, B. D., & Wang, T. (2021). An analysis of research methods utilized in five top, practitioner-oriented LIS journals from 1980 to 2019. Journal of Documentation, 77(5), 1196-1208. Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74(5), 570-581. Ma, J., & Lund, B. D. (2021). The evolution and shift of research topics and methods in library and information science. Journal of the Association for Information Science and Technology, 72(8), 1059-1074. Ma, J., Wang, T., & Lund, B. D. (2023). Analyzing public libraries as civic agents in advocating for COVID-19 vaccine uptake. Public Library Quarterly. https://doi.org/10.1080/01616846.2023.2197842 Manzari, L. (2013). Library and information science journal prestige as assessed by library and information science faculty. The Library Quarterly, 83(1), 42-60. Nixon, J. M. (2014). Core journals in library and information science: Developing a methodology for ranking LIS journals. College & Research Libraries, 75(1), 66-90. Popoviciu, L., Haywood, C., & Mac an Ghaill, M. (2006). The promise of post-structuralist methodology: Ethnographic representation of education and masculinity. Ethnography and Education, 1(3), 393-412. Roberts, P. & Priest, H. (2006). Reliability and validity in research. Nursing Standard, 20(44), 41-46. Rochester, M. K. (1995). Library and information science research in Australia, 1985-1994: A content analysis of research articles. Australian Academic and Research Libraries, 26(3), 163-170. Schick, F. L. (1962). Library science research. Journal of Education for Librarianship, 3(2), 8392. Tuomaala, O., Jarvelin, K., & Vakkari, P. (2014). Evolution of library and information science, 1965-2005. Journal of the Association for Information Science and Technology, 65(7), 1446-1462. Ullah, A., & Ameen, K. (2018). Account of methodologies and methods applied in LIS research: A systematic review. Library and Information Science Research, 40(1), 53-60. Vakkari, P., Jarvelin, K., & Chang, Y. (2023). The association of disciplinary background with the evolution of topics and methods in library and information science research, 19952015. Journal of the Association for Information Science and Technology, 74(7), 811827. Van Scoy, A., & Fontana, C. (2016). How reference and information service is studied: Research approaches and methods. Library and Information Science Research, 38(2), 94-100. Wright, C. (2018). Photo-ethnography. In the International Encyclopedia of Anthropology. John Wiley & Sons. https://doi.org/10.1002/9781118924396.wbiea2017 Yontar, A., & Yalvac, M. (2000). Problems of library and information science research in Turkey. IFLA Journal, 26(1), 39-46. Zhang, J., Zhao, Y., & Wang, Y. (2016). A study on statistical methods used in six journal of library and information science. Online Information Review, 40(3), 416-434.