The CoARA process at LUT University

LUT-yliopisto LUT University lut.fi Yliopistonkatu 34 FI-53850 Lappeenranta Mukkulankatu 19 FI-15210 Lahti tel +358 294 462 111 Y-tunnus 0245904-2 ALV/VAT FI 02459042 12.12.2025 The CoARA process at LUT University 1 Introduction This document presents how LUT University starts the process of reviewing and developing the criteria, tools, and processes for research and researcher assessment in line with the international Agreement on Reforming Research Assessment (ARRA). As a signatory of ARRA, LUT agrees on the need to reform research assessment, and is committed to implement tangible changes within the agreed time frame and monitor progress transparently. LUT belongs to the Coalition for Advancing Research Assessment (CoARA), which provides a shared framework, guidance, and tools for implementing research assessment reform. This action plan is part of LUT’s CoARA process. 2 Links to other related reforms currently underway at LUT LUT is committed to several national and international policies that aim at good scientific practice, equality, fairness, and recognition of diversity. Our current guidelines for assessing research and researchers are based on, among other things, the following (the links point to the implementation of these guidelines at LUT University, if internal guidelines have been published): • DORA – San Francisco Declaration on Research Assessment (2012) • The Finnish code of conduct for research integrity and procedures for handling alleged violations of research integrity in Finland (2023) • Guidelines for ethical review in human sciences (2019) • Declaration for open science and research (2025) • National recommendation on the responsible use of publication metrics (2020) • Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland (2020) • HR Strategy for Researchers HRS4R The principles of ARRA are largely based on the same foundation as the above guidelines. ARRA principles for overarching conditions require us to comply with ethics and integrity rules and practices; safeguard freedom of scientific research; and ensure independence and transparency of the data, infrastructure and criteria for research assessment and for determining research impacts. ARRA principles for assessment criteria and processes, in turn, require us to focus on quality and impact; recognize the diversity of research activities, practices, and roles; and ensure equality and inclusiveness. What distinguishes ARRA from the other guidelines, however, is its specific focus on reforming research assessment, and the commitment of organizations to decide on the concrete steps they will 2 take in their reform journey. While reforming research assessment has been underway at LUT for some time, the CoARA process adds an important dimension to this ongoing development. 3 Research assessment contexts at LUT The Agreement focuses on reforming the assessment of research performing organizations and research units, research projects, and individual researchers and research teams. LUT is a research performing organization that conducts all these types of research assessments. Of the purposes of research assessment mentioned in the Agreement, we recognize the following as applicable to us: i) University and its units • allocating funding (e.g. internal allocation of state funding received by the university) • informing decisions on research priorities (e.g. selection of strategic focus areas) • improving the definition and implementation of research strategies (e.g. strategic development of research) ii) Research projects etc. • allocating funding (e.g. doctoral school funding, other internal funding for projects) • informing project management and future research funding decisions (e.g. research platform funding) iii) Individual researchers and research teams • allocating funding (e.g. salary based on research performance) • recruitment and promotion (e.g. tenure-track and non-tenure-track positions) • professional development review • prize and award decisions (e.g. publication bonuses) However, research assessment in these contexts is not always a primary or even an explicit activity, but it may occur as a minor part of the decision-making process. One aspect of LUT’s CoARA process is to raise awareness of how research is valued and assessed in these situations. 4 Key challenges at LUT in relation to the core commitments According to ARRA, the assessment of research, researchers and research organizations should recognize the diverse outputs, practices and activities that maximize the quality and impact of research. This includes the consideration of research contributions to scientific, technological, economic, cultural and societal impact. It requires basing assessment primarily on qualitative judgement, for which peer review is central, supported by responsible use of quantitative indicators. The core commitments of ARRA include two commitments to enable better recognition of the diverse practices and activities that maximise the quality of research, as well as two commitments to enable a move away from inappropriate uses of metrics. These requirements are not new, and LUT has systematically developed its research assessment to this direction. In the table below, we highlight some of the key questions or concerns at LUT in relation to the core commitments. These could serve as a starting point for our reform journey. 3 Core commitments Key questions or concerns at LUT 1. Recognize the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research Do we sufficiently recognize the following in assessing academic staff and research units? • Academic service tasks like leadership, mentoring, peer review • Diverse outputs beyond journal publications • Teamwork, early sharing of data and results, open collaboration • Roles like data steward, technical roles, public outreach, science diplomacy, science advice • Research contributions for the benefit of society 2. Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators Researcher promotions and performance assessments are largely based on quantitative indicators. While this provides a standardized and transparent approach, does it leave room for contextdependent considerations? The achievement of the university’s strategic goals for research is assessed based on quantitative indicators. Does this guide the research community to also pursue valuable goals that cannot be measured? 3. Abandon inappropriate uses in research assessment of journaland publicationbased metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and hindex Journal-based metrics (JUFO-classification) are used in researcher promotion criteria, work performance assessments, and personal bonuses. This does not necessarily treat everyone fairly or encourage publishing in the most appropriate channels. Author-based metrics (counting papers, citations, grants, etc.) dominate the calculation of research contributions in work performance assessments. This does not necessarily encourage teamwork or open collaboration. 4. Avoid the use of rankings of research organizations in research assessment University rankings are one way to show off, monitor, and benchmark our research performance. Rankings are also used as one source of information in partner scouting and networking. What is the role of rankings in relation to our research community’s own goals and values? To what extent do they help us understand, develop, or manage our research activities? Do we want to track our progress with data, criteria, and analyses controlled by a few companies? 5 Who and what does the reform concern at LUT In addition to the four core commitments, the Agreement includes six supporting commitments. The latter include three commitments that enable the transition to new research assessment criteria, tools, and processes, as well as three commitments that facilitate mutual learning, communicate progress, and ensure that new approaches are evidence-based. In the table below, we specify who and what these commitments should concern at LUT, as this "anchors" the reform in our organization. 4 Supporting commitments Who or what does this concern at LUT? 5. Commit resources to reforming research assessment as is needed to achieve the organisational changes committed to Whose time / resources would be needed? • Research management (provost, deans, department and unit heads). Attention and joint forum to consider the research assessment reform and make decisions about it, including decisions on resourcing. • Management services. Resources to coordinate the reform at LUT and develop and implement new practices in university-level assessments. • HR services. Time to review and develop practices in researcher assessment. Connect the work to HRS4R process. • Supervisors of academic staff. Time to adopt and implement new practices in researcher assessment. Connect the work to HRS4R process and its current action plan. • Library. Resources to develop the current research information system to better document and analyze researchers’ diverse research contributions. Resources to provide support for responsible publishing, data management, and targeted training on responsible metrics. Connect the work to open science action plan. 6. Review and develop research assessment criteria, tools and processes What criteria, tools, and processes does this entail? ▪ For units and institutions • Strategy indicators for research – balancing quantitative and qualitative assessment and recognizing the diversity of contributions • RIA assessment framework – tools and processes for recognizing diverse contributions and ensuring the responsible use quantitative indicators • Current research information system (LUT Research Portal) – supporting the documentation of diverse research outputs and providing sufficient metadata for their analysis and responsible use • External data sources – progressing towards open-access and/or community-owned research information infrastructures (such as OpenAlex bibliometric database or Lens patent search service) • LUT impact monitoring – critically reviewing the use of university rankings in communication, monitoring, and benchmarking of LUT’s research performance ▪ For projects • Internal research funding instruments – reviewing criteria, tools, and processes to ensure transparency, fairness, and equity 5 ▪ For researchers • Career path models – considering the necessary diversity of research career paths, and ensuring transparency and consistency of goals and criteria • Recruitments and promotions (tenure-track, non-tenure-track) – ensuring balanced use of quantitative and qualitative evidence, and abandoning inappropriate or uncritical uses of journal-based metrics • Salary system – recognizing diversity of contributions, ensuring balance between quantitative and qualitative assessment, and eliminating inappropriate uses of journal-based metrics • Bonus system – recognizing diverse contributions, and abandoning inappropriate uses of journal-based or other narrow metrics 7. Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use Whose awareness is needed, and how it is raised? • Research management. Key actors in starting the reform. Ensure regular discussions in the strategic management team for research. • Supervisors of academic staff. Key actors in implementing new practices in researcher assessment. Include a CoARA module in the supervisors’ handbook or training package. • Tenure-track committee. A key actor in considering reforms to researcher career paths. Provide targeted training on assessment reform. • Selection and promotion committees + external reviewers. Provide instructions on assessment criteria, unconscious bias, etc. Use training materials produced by the CoARA network. • Contributors to RIA. Prepare research units for the next RIA round. Pay attention to skills to identify and communicate how their research can benefit various stakeholders and knowledge users beyond their specific scientific, technical, or business domains. • External reviewers in RIA. Provide instructions on assessment criteria, unconscious bias, etc. Use training materials produced by the CoARA network. • All academic staff. Develop skills to better identify and communicate various tasks, roles, skills, and contributions, which should be considered merits in the academic world. Provide easyto-find guidance and training on responsible use of quantitative indicators in research assessment, leveraging synergies with LUT’s open science actions. • HR services. Provide targeted training on assessment reform, especially on tools and practices. Leverage synergies with HRS4R actions. • Other university services. Provide information on the assessment reform and opportunities to discuss and develop processes together. 6 8. Exchange practices and experiences to enable mutual learning within and beyond the Coalition With whom are practices and experiences exchanged? • Participate in CoARA national chapter network for tips, feedback and mutual learning. • Benchmark national or international best practices in technical and business universities most likely to adopt a performance measurement culture such as LUT. • Exchange practices and experiences with LAB University of Applied Sciences, and search for synergies within LUT Universities in implementing the CoARA action plans. 9. Communicate progress made on adherence to the Principles and implementation of the Commitments To whom and when is progress communicated? • Publish this action plan on the CoARA and LUT webpages. • Communicate progress annually to the university community, for example via an internal blog. Also communicate progress to the CoARA national chapter when the opportunity arises. 10. Evaluate practices, criteria and tools based on solid evidence and the state-of-the-art in research on research, and make data openly available for evidence gathering and research What evidence and research are used to evaluate our practices? • Feedback surveys for our research staff • Lessons learned from HRS4R actions • Evaluate LUT research assessment beyond business logic; critically examine criteria and practices against the state-of-the-art research assessment. Discuss with other professionals. 6 How do we proceed Below is LUT University’s planned process for reforming research assessment during 2025–2028, presented as chronological steps. • University management initiates the process by agreeing on this action plan. Based on the plan, they communicate to the university community what the research assessment reform means for LUT, whom it concerns, and how it will be implemented. (See table above, Commitment 9) – Winter 2025–2026 • University management services, together with other service units, conduct a gap analysis to evaluate how well LUT’s current assessment criteria, tools, and processes align with the core commitments of research assessment reform. The analysis identifies specific areas where LUT practices need to be developed or adapted. The work is organized into three task forces according to assessment context: (i) university and its units, (ii) research projects, and (iii) individual researchers and research teams. Relevant tools and practices developed by other CoARA signatories are reviewed as potential models. 7 (See table above, Commitment 6) – Spring 2026 • Observations from the gap analysis inform discussions and decisions on what to reform and where to begin. Schools, relevant service units, and university management take part in these discussions. Required resources for implementing the reforms must also be considered. Decisions are made by the university management. (See table above, Commitments 6, 5) – Spring–Autumn 2026 • The LUT community begins developing and testing new assessment criteria, tools, and practices in the selected areas. If large-scale impacts are expected, such as those from reforming the salary system, piloting may first take place within one or a few units before extending the reform to the entire university. For instance, new tools such as the FIN-CAM career assessment matrix could be piloted with an appropriate group, such as the department of social sciences. Conversely, if a reform would place a significant burden on specific individuals or teams, a different type of pilot may be needed. For example, shifting to open-access databases and tools in research monitoring and assessment may require parallel use of a new database alongside the old one for a transition period. During this developing and testing period, LUT will exchange experiences with other CoARA signatories. (See table above, Commitments 6, 8, 9) – Autumn 2026 • Following piloting and testing, LUT implements the developed and revised criteria, practices and tools. Transparent communication, guidance, and training are provided for all involved. (See table above, Commitment 7) – 2027 • Feedback is collected, and outcomes are evaluated after 6–12 months. Lessons learned are shared within the CoARA network to support mutual learning. (See table above, Commitments 9, 10) – 2028 Contact: Katri Huutoniemi Senior Expert, Science Policy and Research Management Support LUT University Yliopistonkatu 34, 53850 Lappeenranta, Finland Mukkulankatu 19, 15210 Lahti, Finland [email protected]