Interdisciplinary Value

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY 4.0 https://creativecommons.org/licenses/by/4.0/ Interdisciplinary Value © 2024 Veli-Matti Karhulahti Published version Karhulahti, Veli-Matti Karhulahti, V.-M. (2024). Interdisciplinary Value. Meta-Psychology, 8, Article 3679. https://doi.org/10.15626/MP.2023.3679 2024 Meta-Psychology, 2024, vol 8, MP.2023.3679 https://doi.org/10.15626/MP.2023.3679 Article type: Commentary Published under the CC-BY4.0 license Open data: Not Applicable Open materials: Not Applicable Open and reproducible analysis: Not Applicable Open reviews and editorial process: Yes Preregistration: No Edited by: Daniel Lakens Reviewed by: Sven Ulpts, Alejandro Sandoval-Lentisco Analysis reproduced by: Not Applicable All supplementary files can be accessed at OSF: https://doi.org/10.17605/OSF.IO/27NF6 Interdisciplinary Value Veli-Matti Karhulahti1 1University of Jyväskylä This is a commentary on interdisciplinary value in the special issue "Responsible Research Assessment: Implementing DORA for hiring and promotion in psychology." Keywords: Assessment, Axiology, Meta-science I write this commentary with a conflict of interest and confirmation bias. As an interdisciplinary researcher, it is my personal interest to highlight the value of such research, which could also be debated. From this position, I argue that the four largely welcome research assessment principles for psychology, proposed by Schönbrodt et al. (2022) and Gärtner et al. (2022), should include a fifth one: interdisciplinary value. Not everyone needs to be interdisciplinary, but those who are should be evaluated as such. For the working purposes in the present commentary, we may think of interdisciplinarity in research as the merging of elements from different disciplines. The term “elements” is important, as what is being merged may differ from constructs and methods to practices and theories. Without entering deeper terminological debates, it is important to separate interdisciplinary efforts from multidisciplinary ones, the latter of which operate on multiple disciplines not in interconnected ways but respectively (e.g., Nicolescu, 2014). In one sense, psychology itself could be considered an interdisciplinary science due to its historical blending across methodological, pragmatic, and other domains. One cannot have missed the recurring authoritative calls for better and more interdisciplinarity across sciences. For instance, a policy brief from the European Commission explicitly asserts that “to foster, harness, and leverage collaborative interdisciplinarity should become a key priority for EU research and innovation policy” (Allmendinger, 2015, p. 4). Critically related to the target articles, the policy brief also notes how interdisciplinary research: challenges common quality assurance and evaluation devices which zoom in research excellence as the main, or sometimes only, criteria for measuring performance and impact. This is why interdisciplinary work also carries much higher risks for academic careers than research that is firmly based in traditional disciplines which still serve as gatekeepers (p. 5) Interdisciplinary research is difficult to carry out and those who pursue it seriously often struggle for many related reasons. Some years ago, I was involved in a workshop series where we discussed and sought solutions for the practical challenges of interdisciplinarity (NOSHS Collaboration, 2020). One of the collectively identified key issues concern the poor recognition of heavy interdisciplinary efforts, especially for early career researchers whose work is typically assessed by (and in) monodisciplinary units (see also Goring et al., 2014). If the evaluators are not sufficiently familiar with the constructs, methods, practices, and theories applied by the assessed researcher—or interdisciplinary complexity in general—it is but natural for them to not perceive these elements of high value. The research outputs and evaluation schemes proposed by Schönbrodt et al. (2022) and Gärtner et al. (2022), while carrying clear potential for quantitative psychology, also exemplify the challenges facing many interdisciplinary psychologists. Because psychology, in its current form, has developed a rigid identity as a field driven by statistical methods, hypothesis testing, and publication of various effects (and “the effects”), researchers who do things differently are easily interpreted as lacking in rigor or not meeting the best criteria or practices. The present space allows for a few examples, with focus on the section publications. When presenting publications as the first output type (of three), Schönbrodt et al. (2022) start by explicitly suggesting “journal articles” as the defining category. From an interdisciplinary perspective, the journal article is but one of many relevant publications. For instance, in computer science and related fields, due to their distinct lineages, the foremost publication category is the proceedings article (for an historically contextualized disciplinary diagnosis, see Fortnow, 2009). Meanwhile, in fields like anthropology, literature, and sociology, which often engage in theoretical dialogue via in-depth cases, the monograph has been a key publication format and, not uncommonly, also an explicit 2 tenure requirement (e.g., Levine 2007). By the principles of Schönbrodt et al. (2022), interdisciplinary psychologists who publish their core research also outside the journal article category would be overlooked in (or completely left out of) evaluations already by the nature of their different publication practices. On the other hand, when researchers publish journal articles in psychology journals, the evaluation schemes proposed by Gärtner et al. (2022) are so specifically crafted for statistical designs that even conventionally trained psychologists who deviate from the norm—for instance, by doing qualitative research (interdisciplinary or not)—would not be properly assessed. Because the quality concepts have always been different in qualitative research (e.g., Stenbacka, 2001), they remain difficult to score in the presented format; meanwhile, items like “verified computational results” would hardly be applicable to any qualitative analyses at all. These issues would leave qualitatively specialized applicants in a weaker position by default. Similar notes can be made of the two other publication types, data and software, which remain unconventional in the qualitative domain due to various methods-specific reasons, such as the excessive resource demands of safely publishing qualitative datasets and the lesser relevance of software in their analyses (see Karhulahti, 2022). To be clear, I completely agree with Schönbrodt et al. (2022) and Gärtner et al. (2022) when it comes to discarding journal or publisher metrics, and I am not against valuing journal articles, datasets, and software. My point is that assessment should not focus—or worse, be limited to—such categories, as this would only contribute to further narrowing of what good (psychological and other) research is or should be. Where I am currently based, the Finnish National Board on Research Integrity (https://tenk.fi/en) has developed multidisciplinary assessment guidelines that take into consideration a wider spectrum of achievements, rewards, and skills. With reference to these guidelines, the Research Council of Finland already highlights that the “use of journal-based metrics in the assessment is prohibited, nor may applicants add metrics to their applications”; moreover, researchers are explicitly asked to list publications in nine categories ranging from “peer-reviewed scientific articles” to “scientific monographs” and “audiovisual material & software” (Research Council of Finland, 2022b). Other contributions, such as “datasets”, “guides”, “infrastructures”, “methods”, and “promoting open science” are reported and evaluated respectively (Research Council of Finland, 2022a). This dynamic model welcomes diversely specialized scholars, including interdisciplinary ones, with less predefined bias. To accommodate interdisciplinarity even further as part of research assessment, the below what,how, and why could be considered by any motivated committee. What? Some three decades ago, Nissani (1995) suggested that interdisciplinary “richness” could be weighted by four variables: number of disciplines, distance between them, novelty, and integration. In other words, when evaluating the degree to which an applicant or study thrives in interdisciplinarity, one should pay attention to a) how many disciplines are successfully combined, b) to what intensity, c) whether such links are common or pave the way for new bridges, and d) to what degree are these bridges integrated meaningfully. In theory, these variables could also be turned into a metric, but a good start would be to keep them in mind when qualitatively assessing the merits of interdisciplinary applicants and their publications. How? To carry out high-level assessment of applicants with diverse interdisciplinary (including methodological) experience and expertise, evaluators, too, should represent experience and expertise in various ways. Because a pre-set committee would unlikely have the knowledge of future applicants (Kekecs et al., 2023), it is often practical to recruit external experts based on the applicants’ backgrounds. In addition to this alreadycommon procedure, it is essential that at least some evaluators also have personal experience of interdisciplinary research (not merely representing multiple disciplines respectively). Successfully carrying out interdisciplinary projects typically involves challenges and creative solutions that may be visible only to those who engage in interdisciplinary work themselves. Why? Some pieces of empirical evidence and common sense imply that interdisciplinary research operates by delayed gratification. For instance, being trained in multiple disciplines (their histories, methods, and theories) is time-consuming and usually results in lower quantities of initial productivity, which nonetheless can enable higher productivity and quality later (see Goring et al., 2014). Likewise, whereas interdisciplinary studies tend to gain less attention and citations in the first years after publication—perhaps due to their unorthodox research designs—it seems that 13 years after publication the same studies already exceed the citations of monodisciplinary work (Noorden, 2015). If the value of interdisciplinary research accumulates exceptionally over time, any psychology committee with long-term interests can harvest benefits from identifying it early on. 3 In closing, I commend Schönbrodt et al. (2022) for their important suggestion to provide applicants “the opportunity to explain in a few sentences if and why they think that something important is being overlooked when using these indicators” (p. 5). Although interdisciplinary applicants could use these few sentences to explain the efforts, outcomes, and limitations involved in interdisciplinary research, this would unlikely have any meaningful effect on a committee that is otherwise unprepared for such assessment. Interdisciplinary value should be one of the research assessment principles in psychology. Author Contact [email protected] https://orcid.org/0000-0003-3709-5341 Conflict of Interest and Funding No meaningful conflicts of interest beyond the ones mentioned at the start. Funded by the European Union (ERC, ORE, 101042052). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. Author Contributions This is a single-author paper. Open Science Practices This article is a commentary and is not eligible for any Open Sciemce badges. The entire editorial process, including the open reviews, is published in the online supplement. References Allmendinger, J. (2015). Quests for interdisciplinarity: A challenge for the era and horizon 2020. Policy Brief by the Research, Innovation, and Science Policy Experts (RISE). Brussels: European Commission. https://data.europa.eu/doi/10.2777/ 499518 Fortnow, L. (2009). Viewpoint: Time for computer science to grow up. Communications of the ACM, 52(8), 33–35. https : / / doi . org / 10 . 1145 / 1536616.1536631 Gärtner, A., Leising, D., & Schönbrodt, F. (2022). Responsible research assessment ii: A specific proposal for hiring and promotion in psychology. https://psyarxiv.com/5yexm/ Goring, S. J., Weathers, K. C., Dodds, W. K., Soranno, P. A., Sweet, L. C., Cheruvelil, K. S., et al. (2014). Improving the culture of interdisciplinary collaboration in ecology by expanding measures of success. Frontiers in Ecology and the Environment,12(1), 39–47. https://doi.org/ 10.1890/120370 Karhulahti, V.-M. (2022). Reasons for qualitative psychologists to share human data. British Journal of Social Psychology.https://doi.org/10.1111/ bjso.12573 Kekecs, Z., Palfi, B., Szaszi, B., Szecsi, P., Zrubka, M., Kovacs, M., et al. (2023). Raising the value of research studies in psychological science by increasing the credibility of research reports: The transparent psi project. Royal Society Open Science,10(2), 191375. https://doi.org/10.1098/ rsos.191375 Nicolescu, B. (2014). Multidisciplinarity, interdisciplinarity, indisciplinarity, and transdisciplinarity: Similarities and differences. RCC perspectives, (2), 19–26. Nissani, M. (1995). Fruits, salads, and smoothies: A working definition of interdisciplinarity. The Journal of Educational Thought (JET)/Revue de la Pensée Éducative, 121–128. https : / / www. jstor.org/stable/23767672 Noorden, R., van. (2015). Interdisciplinary research by the numbers. Nature,525(7569), 306–307. https://doi.org/10.1038/525306a NOS-HS Collaboration. (2020). Research policy brief on the practical challenges of interdisciplinarity. Policy Brief by the Place of the Cognitive in Literary Studies 2018–2019. Oslo: University of Oslo. https://research.utu.fi/converis/portal/detail/ Publication/46300826 Research Council of Finland. (2022a). Cv guidelines. https : / / www. aka . fi / en / research - funding / apply-for-funding/how-to-apply-for-funding/ az - index - of - application - guidelines2 / cv - guidelines/ Research Council of Finland. (2022b). List of publications.https : / / www. aka . fi / en / research - funding/apply-for-funding/how-to-apply-forfunding/az-index-of-application-guidelines2/ list-of-publications/ Schönbrodt, F., Gärtner, A., Frank, M., Gollwitzer, M., Ihle, M., Mischkowski, D., et al. (2022). Responsible research assessment i: Implementing dora for hiring and promotion in psychology.https: //psyarxiv.com/rgh5b Stenbacka, C. (2001). Qualitative research requires quality concepts of its own. Management De- 4 cision,39(7), 551–556. https: / / doi. org / 10 . 1108/EUM0000000005801