Joint Statement on Informatics Research Evaluation

Joint Statement on Informatics Research Evaluation Informatics Europe and the undersigned National Informatics Associations, considering A. that it is of major importance to strengthen the quality and impact of informatics research over quantity [1, 2, 5], B. that publication counts (or derivative bibliometrics) and securing research grants per se do not primarily indicate research quality but rather demonstrate productivity [1, 2, 3, 7, 8], C. that research evaluation forms the basis for decision making (hiring, tenure, promotion, resource allocation, etc.) on the one hand, and the basis for feedback (with a view to improving quality and making strategic decisions) on the other hand, D. that, besides journals, conferences and books play a crucial role within the publication culture of Informatics [2], E. that publication quality is only one aspect of research quality and that the impact of research is not limited to publications but also includes different types of artefacts (software, etc.) [2, 3, 8], F. that some databases used for evaluation (such as the Web of Science) do not represent Informatics research in a proper way [2, 3], G. that publication patterns vary across disciplines and within Informatics, and therefore a comparative evaluation primarily based on quantitative metrics might produce biased results [2], H. that research assessment should foster research integrity and the ethical awareness of researchers [3, 4], 1. agree on the following basic principles of research assessment in Informatics with regard to the evaluation of departments / research units as well as to the evaluation of individuals: (a) Qualitative evaluations based on key data ("informed peer-review process") are the preferred means of research assessment. (b) Quantitative metrics are never to be used as the only or predominant evaluation instrument. Their role is to underpin and supplement the findings of a qualitative assessment where appropriate. This is especially true of comparisons across disciplines or across subfields of Informatics. (c) Publication counts (or derivative bibliometrics) cannot be allowed to replace the reading and assessment of the most relevant publications of an individual scientist or of a research unit. The basis of assessment within the framework of an evaluation has to be the scientific content and not any kind of publication index [5]. (d) Prior to an evaluation, clear goals for the assessment have to be formulated. The evaluation of research in departments/research units must take into consideration the respective strategies and goals of the department/research unit [6]. (e) Scientific conferences (and books, if appropriate) have to be regarded as being as important as journals. (f) Research evaluation must refer to all kinds of research output (research publications, artefacts such as software, etc.) and take into consideration their value as well as their impact on science and society. (g) Research assessment is to take into account Open Science principles and value interdisciplinary collaboration in research. 2. encourage policymakers, institutions, funding agencies and researchers to implement and apply all afore-mentioned basic principles of research assessment. CODDII - Conferencia de Directores y Decanos de Ingeniería Informática CPHC - Council of Professors and Heads of Computing FTI - Fakultätentag Informatik GI - Gesellschaft für Informatik GII - Gruppo Ingegneria Informatica GRIN - GRuppo di INformatica Informatik Austria IPN - ICT-research Platform Netherlands SCIE - Spanish Scientific Society of Informatics SIF - Société Informatique de France SIRA - Swiss Informatics Research Association UKCRC - UK Computing Research Committee ------------------------------------- 1) Batya Friedman and Fred B. Schneider : Incentivizing Quality and Impact: Evaluating Scholarship in Hiring, Tenure, and Promotion. Computing Research Association Best Practices Memo. 2015 (https://cra.org/wp-content/uploads/2016/02/BP_Memo.pdf) 2) Bertrand Meyer, Christine Choppy, Jørgen Staunstrup, Jan van Leeuwen: Research Evaluation For Computer Science. Communications of the ACM, April 2009, Vol. 52 No. 4, Pages 31-34 3) Floriana Esposito, Carlo Ghezzi, Manuel Hermenegildo, Helene Kirchner, Luke Ong: Informatics Research Evaluation. An Informatics Europe Report. Informatics Europe, 2018 (www.informaticseurope.org) 4) Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M. H., Barbour, V., … Dirnagl, U.. The Hong Kong Principles for Assessing Researchers: Fostering Research Integrity. 2019 (https://doi.org/10.31219/osf.io/m9abx) 5) San Francisco Declaration on Research Assessment. 2013 (https://sfdora.org) 6) Dutch Organisation for Scientific Research: Researcher and department/institute evaluation. 2019 (https://www.informatics-europe.org/images/nationalassociations/Research_Evaluation_in_the_Netherlands.pdf) 7) David L. Parnas: Stop the Numbers Game. Counting papers slows the rate of scientific progress. Communications of the ACM, November 2007, Vol. 50, No. 11, Pages 19-21 8) Institut de France – Académie des Sciences: On the Proper Use of Bibliometrics to Evaluate Individual Researchers. Report presented on 17 January 2011 to the Minister of Higher Education and Research. 2011