Report on Exploration Workshops (D4.2)

Ref. Ares(2023)8069052 - 27/11/2023 Report on Exploration Workshops Deliverable 4.2 Authors’ name: Sara Gentilini, Giulia Pietrollini, Alessia Careccia, Gjis Kleter, Jeanne Marie Membré, Marilena Dimitrakopoulou, Wenjuan Mu, Thomas Zunder, Maria Scherbov Date 27/11/2023 Ref. Ares(2024)6839728 - 27/09/2024 Deliverable 4.2 2 Report on Exploration Workshops D4.2 30/11/2023 Public 4 0.9 Document ID Due date Submission date Dissemination Level Work Package Document Version Author(s) 27/11/2024 24/09/2024 (updated version) Sara Gentilini, Giulia Pietrollini, Alessia Careccia, Gjis Kleter, Jeanne Marie Membré, Marilena Dimitrakopoulou, Wenjuan Mu, Thomas Zunder, Maria Scherbov 101059813 October 2022 Grant Agreement Start Date September 2026 48 Months Duration End Date D4.2 Deliverable 4.2 3 Contributors Name Organisation Sara Gentilini APRE Giulia Pietrollini APRE Gijs Kleter WUR Jeanne-Marie Membré INRAE Marilena Dimitrakopoulou AGROKNOW Mu Wenjuan WUR Thomas Zunder UNEW Maria Scherbov EUFIC Alessia Careccia APRE Revision history Version Date Reviewer Modifications 0.1 31/08/2023 Ine van der FelsKlerx First set up of the report 0.2 14/10/2023 Jeanne-Marie Membré Adding information 0.3 01/11/2023 Sara Gentilini Adding Report data 0.4 03/11/2023 Giulia Pietrollini Adding Report data 0.5 06/11/2023 Thomas Zunder Adding LL2 data 0.6 22/11/2023 Maria Scherbov Added information justifying chosen format for LLs 1 & 3 for stakeholder engagement 0.7 24/11/2023 Ingeborg van Leeuwen-Bol Editing Deliverable 4.2 4 0.8 25/11/2023 Ine van der FelsKlerx Final report review, rewriting and edits 0.9 04/07/2024 Giulia Pietrollini, Alessia Careccia, Thomas Zunder Updated version including information concerning the Delphi study conducted by LL2. Disclaimer: The information and views set out in this report are those of the author(s) and do not necessarily reflect the official opinion of the European Union. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained herein. Deliverable 4.2 5 Index of Contents 1 Introduction ........................................................................................................................... 8 2 Methodology ........................................................................................................................ 11 3 Results .................................................................................................................................. 23 4 Discussion and Conclusions ............................................................................................... 29 5 Set up and planning next steps Living Labs ...................................................................... 30 6 Annex 1. Announcement of LL1 and LL3 workshop ........................................................ 31 7 Annex 2. Preview of LL2 Delphi Round 1 Survey .............................................................. 32 Deliverable 4.2 6 Index of tables Table 1: Topics and responsibilities of the 3 LLs .................................................................. 11 Table 2: Stakeholders list for LL2 .......................................................................................... 15 Table 3: List of external experts from AGROKNOW............................................................. 18 Index of figures Figure 1: Organisation type and Country of work for panel ............................................... 16 Figure 2: Years of experience and gender identity of the panel......................................... 17 Figure 3 Poll outcome on participant's origin ...................................................................... 17 Figure 4 Poll outcome on field work origin of participants ................................................. 18 Figure 5 Project booth staffed by HoliFood partners during the Synergy Days ................ 20 Figure 6: Welcome and introduction to the workshop by W. Mu ....................................... 21 Figure 7: Moderation of the workshop by T. H. Zunder ...................................................... 21 Figure 8: Poll outcome: capabilities of emerging risk prediction tools .............................. 23 Figure 9: Poll outcome on priority hazards to be screened ................................................ 24 Figure 10: Poll outcome on requirements for a successful tool ......................................... 24 Figure 11: Poll outcome on source data as input for the predictive tool ........................... 24 Figure 12: : Wooclap activity’s result for the question “What opportunities do you think AI tools present to your organization for food safety?” ....................................................... 26 Figure 13: Wooclap activity's result for the question “ What are the specific technical requirements your organization needs to implement AI?” ................................................. 26 Figure 14: Wooclap activity's result for the question “What challenges are you currently facing in implementing AI in your organization?” ................................................................. 27 Deliverable 4.2 7 Executive summary As part of WP4 of the HOLiFOOD project, three Living Labs (LL) have been established, linked to WP1, WP3 and WP6, respectively. Each LL goes through the three different phases of Exploration, Experimentation and Evaluation during the four years of the project. This Deliverable report presents the set up and results of the first phase of the Living Labs, i.e., the Exploration phase. The LL development and composition mostly followed the Guidelines which had been set up in Task 4.1, and were described in Deliverable 4.1. For the Exploration phase, LL1 and LL3 organized a workshop, and LL2 has set up a Delphi Survey, each aimed to retrieve the needs and experiences from stakeholders, and to co-create a tailor-made action plan, that addresses the identified needs. The workshop of LL1 was based on the identification of needs of stakeholders as related to emerging risk identification (ERI) models and tools (to be developed in WP1). The workshop not only aimed to list stakeholder needs, but also to find ways to best address them, and to recognize possible problems that could be encountered. The Delphi survey of LL2 focused on the identification of stakeholder's expectations in term of holistic risk assessment dimensions, preferential methods of risk aggregation, and formats of holistic assessment outputs. The workshop of LL3 was focused on the identification of technical requirements of stakeholders related to the ERI infrastructures, in terms of data, models and computation. This Deliverable report shows how Co-Creation acts as a “key-component” when including the analysis of the common problems and needs, involving key players in the ideation and co-design through brainstorming sessions and focus groups; while the quantitative data collection (interviews and questionnaires) is also crucial for the baseline developed of the research processes. Deliverable 4.2 8 1 Introduction 1.1 HOLiFOOD WP4 The overall objective of WP4 entitled “Stakeholder engagement and codesign in living labs” is to establish three Living Labs (LLs) following the innovation development phase approach which consists of three phases: exploration, experimentation and evaluation. This co-creation activity is particularly necessary for a wide variety of food system actors to work together to facilitate the early identification, assessment and mitigation of risks related to existing and emerging food safety hazards. The three LLs are planned to be conducted in both virtual and face-to-face modes and are aimed at addressing three different priorities: 1) Identification and monitoring of emerging food safety risks (LL1), 2) Holistic risk assessment and acceptance (LL2), 3) Codesign of the platform for emerging risk identification (LL3). The respective contents of each specific LL are addressed to the following HOLiFOOD WPs: LL1: WP1 “Big Data technologies and Artificial Intelligence (AI) for food safety detection and prevention”, LL2: WP2 “Holistic risk assessment for regulation”, LL3: WP6 “Integrated Decision Making & Mitigation”. The overall objective the HOLiFOOD project can be achieved through these three LLs by bridging the gap between research and practice by facilitating discussions between stakeholders, hereby systematically integrating the multi-actor approach (MAA). 1.2 Living Labs Introduction In the course of HOLiFOOD project, the use of Living Labs (LL) as co-creation approach is implemented throughout the entire project. HOLiFOOD uses the following definition of LL - based on the European Network of Living labs (https://enoll.org/): “User-centered, open innovation ecosystems based on systematic user co-creation approach, integrating research and innovation processes in arenas where both open innovation and user innovation processes can be studied”. LLs operate as intermediaries among the so called quadruple Helix (Civil society, Public Administration, Business and Research & Education) for joint value co-creation or validation to scale up innovation and businesses. They act as ‘brokers’ between citizens and organizations (academia, local government, private companies etc.), ensuring that each participant is able to contribute with its knowledge and experience. HOLiFOOD implements a multi-actor codesign approach, using the LL methodology, to ensure stakeholders needs and requirements are considered throughout the project: stakeholders are indeed involved in setting the research objectives, research activities, innovation roll-out and translational activities. Deliverable 4.2 9 Co-creation in LLs is a method of tackling real-life problems, as this methodology allows complex issues to be unbundled by submitting them to multi-actor groups that can grasp all the nuances. A structured co-design process is followed throughout the project, defined as a "transparent process of value creation in ongoing, productive collaboration with, and supported by all relevant parties, with end-users playing a central role and covering all stages of a development process”. Especially if we talk about food, new methods are needed that take into account the increasingly connected and convergent arena of the food system for adequately addressing new challenges, and to develop new solutions with a holistic and transsectoral approach. Resorting to collaboration thus allows the views of experts from different ecosystems to be interwoven, finding common ground in the HOLiFOOD project goal. Another interesting issue is that co-creation activities highlight that actors do not have all the answers, so it is beneficial for all to listen to different points of view and work together to find solutions. In this context, the use of co-creation tools in the Living Labs is crucial to foster creativity, especially in the exploration phase, to find innovative solutions and to improve the engagement of actors. 1.3 What Exploration Living Lab means in HOLiFOOD The exploratory approach of the LLs was translated within the project activity into an inductive content analysis which implied the collection and analysis of data without specific categories of experts or preconceived theories on the topic. This flexibility went very well with the co-creation approach and created a well-identified space within the specific research activities. Co-creation therefore made it possible to guide the researcher's analysis to identify emerging patterns, themes and concepts that can be experimented in the next phase. Moreover, this approach helped identify patterns, categories, or themes that may not have been considered at the early stage of the WP’s activities and ensured that researchers explored multiple perspectives and viewpoints. On the other side of the coin, the inductive approach declined as co-creation activity can be timeconsuming compared with a traditional analysis. Another disadvantage, of course, can be represented by the influence of researcher's personal biases and preconceptions, but these are constantly overcome by the analysis and wrap-up sessions done after the labs. Deliverable 4.2 16 WU Academics M WU Academics F WU Academics F UNEW Academics M UNEW Academics M UNEW Academics F Dialogik Not-for-profit research institute M BfR Bund Regulatory scientific organization F Nationally the panel was split evenly between Denmark, France, Germany, Hungary, the Netherlands and the United Kingdom. In terms of experience the panel had a good range: 42% of respondents had 1 to 10 years of experience, about 42% had 11-20 years of experience and around 16% had 21 years or more. It is important to note that due to the anonymity of the Delphi approach, it is not possible to confirm the makeup of the panel in the same way of LL1 and LL3; nevertheless, this anonymity is viewed as having yielded more objective insights: respondents were in fact unaffected by peer pressure and charisma/age/gender characteristics of others, as it happens in open workshops and forums. Figure 1 and 2 present the demographic of the survey participants. They include the type of organization, the country in which they work and the gender of participants. Figure 1: Organisation type and Country of work for panel Deliverable 4.2 17 Figure 2: Years of experience and gender identity of the panel After the Delphi Survey, an online workshop to discuss results was organized on 28 February 2024. Participants to this workshop included 16 external people and 6 internal members of the consortium. 2.3.3 Stakeholder list exploration phase LL3 Through the Wooclap smartphone app, the 33 workshop participants of the LL3 workshop entered answers to poll questions, including questions about their country of origin (Figure 3) and field of work (Figure 4). No attribution was made in the answers in the Wooclap to the individual participants, consistent with the Chatham House Rules under which the workshop was operated. Figure 3: Poll outcome on participant's origin Various countries were represented (numbers of participants in brackets): • Greece (6) • Serbia (1) • Poland (3) Deliverable 4.2 18 • Mexico (1) • Netherlands (3) • Spain (1) • United Kingdom (1) • France (1) • South Africa (1) • Germany (1) • Romania (1) • Ireland (1) Several sectors were represented in the LL3 stakeholder group, amongst others 8 stakeholders from the Food safety, two from Agrifood innovation sector, two from the Data analysis and data science sectors; and two from the Dissemination sector. Figure 4 Poll outcome on field work origin of participants Several external experts have been involved in the LL3 workshop directly by AGROKNOW, supported by APRE, as shown in Table 3. Table 3: List of external experts from AGROKNOW Organization name Category Social category Men/ Women Country Technical University of Cartagena/Spain Academics Professor M SP Technical University of Cartagena/Spain Academics Researcher M SP Mega Yeeros Poultry Industry Research & Development Specialist F GR Deliverable 4.2 19 Mega Yeeros Poultry Industry Quality Assurance Director F GR INRAE Research Researcher F FR Agritrack Tech company/research AI engineer M GR Lamia Stevia Industry and Farmers Agronomist M GR Intrasoft Tech company AI engineer M GR Effost Food safety agency Project & Communications Manager F NL Effost Food safety agency Managing director M NL Biocos/ Watson Research Researcher F GR Wikifarmer Industry and Farmers Agronomist, Plant Science F GR Wikifarmer Industry and Farmers Agronomist, Plant Science F GR Agricultural University of Athens Research Agronomist F GR The consistent number of participants coming from Greece is justified by the fact that the Synergy days were held in Thessaloniki (Greece), hence it was easier for Greek stakeholders to participate in presence to the event. This, however, did not influence the Living Labs in any way for the heterogeneity of the expertise that the stakeholders brought to the workshop tables. 2.4 Process of the LLs The following sections present the process of the Exploration activity of each LL, with one section per LL, including the workshop held at the Synergy days of LL1 and LL3, and the Delphi survey held in the course of LL2. Annex 1 presents the announcement of the two workshops at the Synergy days. At the Synergy days, an information booth on the HOLiFOOD project was organized as well, as shown in Figure 5 below. Deliverable 4.2 20 Figure 5: Project booth staffed by HOLiFOOD partners during the Synergy Days 2.5 Process LL1 Welcome and introduction As shown in Figure 6 below, WP1 leader Wenjuan Mu (WFSR) provided the background of the HOLiFOOD project and introduced the concept of a holistic approach for building a food safety emerging risk identification system, with an emphasis on the importance of combining knowledge from different fields in food and agriculture and data science knowledge when constructing such a system. 00:15 Ice breaker questions in Wooclap Workshop participants were asked to access the Wooclap website on their smartphones to answer poll questions. The Wooclap poll was operated by Naomi Dam and colleagues from WUR. The first questions (#1-#3) were of a general nature, asking about participants’ provenance and field of work. Questions and results were also displayed on the main screen, with moderation by Tom H. Zunder from UNEW (Figure 7). 00:20 Questions about emerging risk identification systems The following questions (#4-#17) concerned the use of AI for the development and usage of predictive tools for emerging food safety risks, such as the use of AI, the kinds of hazards to be scrutinized, data sources, etc. Results were shown as bar diagrams or word clouds, depending on the type of question. 00:50 Summary, conclusion, and discussion The workshop moderator wrapped up the polls, summarized the main observations and conclusions, allowing participants to raise comments and questions to animate the discussion. Deliverable 4.2 21 Figure 6: Welcome and introduction to the workshop by W. Mu Figure 7: Moderation of the workshop by T. H. Zunder 2.6 Process LL2 Within LL2, input was collected from expert by using a traditional Delphi consisting of the following steps: 1. Scoping: this step implies asking experts and stakeholders to scope baselines, metrics, factors and other elements that they view as important in the estimation of food risk in general, and more specifically in the HOLiFOOD supply chains of Legumes, Maize and Chicken. This step was carried out through anonymous online surveys. Deliverable 4.2 22 2. Exploring Consensus. In this step, the information collected in the first step through the Delphi method was elaborated in order to understand the views of experts in terms of consensus and dissent about food risks, regulations and policies. 3. Workshop. A workshop was run to validate, expand, nuance and critique the outputs from anonymous survey work. The virtual workshop was organized on February 28, 2024 to receive feedback on the structure of the survey and on the relevance of the interim results collected through the survey. 2.7 Process LL3 Welcome and introduction Presentation by Maria Scherbov (EUFIC), providing info for the purpose of the workshop. Ice breaker questions in Wooclap Workshop participants were asked to access the Wooclap website on their smartphones for answering poll questions. The poll was operated by Naomi Dam and her colleagues from WUR. The first questions were of a general nature, asking about participants’ provenance and field of work. Questions and results were also displayed on the main screen, with moderation by Maria Scherbov (EUFFIC). Questions about novel digital infrastructure for food safety in Wooclap A series of follow-up questions were posed. Summary, conclusion, and discussion The moderator wrapped up the poll results and summarized the main observations and conclusions, allowing participants also to raise any comments are questions they'd like to share with other participants and the hosts. Deliverable 4.2 23 3 Results 3.1 Results Living Lab 1 The LL1 stakeholder group of the workshop was composed as: approximately a ~50/50 division of users and developers of tools amongst the audience (+- 20 people). The poll and discussion outcomes were the result of participants views, their experiences, and recommendations regarding the technical features of emerging risk prediction tools, providing project partners with a view on users’ and developers’ needs and preferences. This LL allowed to establish connections with experts and representatives of parallel projects. The visibility of the project was enhanced through the distribution of promotional materials (brochures, merchandise) at the project booth and several relevant contacts were collected. The experts' views/recommendations on the following aspects were retrieved. Results are summarized in the Figures: • Capabilities of emerging risk prediction tools (Figure 8) • Priority hazards and factors to be screened for (Figure 9) • Requirements for a successful prediction model (Figure 10) • Source data to be used as inputs for a prediction tool (Figure 11) Figure 8: Poll outcome: capabilities of emerging risk prediction tools Deliverable 4.2 24 Figure 9: Poll outcome on priority hazards to be screened Figure 10: Poll outcome on requirements for a successful tool Figure 11: Poll outcome on source data as input for the predictive tool Deliverable 4.2 25 The answers given by participants in the poll and the active discussion that took place afterwards, which extended over the expected time, confirmed the stakeholders’ interest for the topic and the high level of engagement created. A positive surprise was the participants’ interest in the field of AI for food safety. Workshop results will serve as a basis for further development of AI tools within HOLiFOOD WP1. 3.2 Results Living Lab 2 The first phase of the Delphi survey can be described as a productive session, in which at times the collective anonymous opinions of the peers were unexpected. The participants observed that the results prompted them to reassess the priorities and decisions they had planned for their work. It was acknowledged that this added value, but also that researchers were experts and whilst this Delphi process may inform their research planning, in the end they should adopt approaches that are defensible, practical and in line with contractual obligations. The workshop conducted on 28 February 2024 was necessary to receive feedback on the structure of the survey and on the relevance of the interim results collected. In the survey’s questions, a high degree of consensus with regard to emerging risks, their identification and the metrics used to monitor them was seen; For all questions with statements that did not reach consensus, they should be repeated as before as is the case with classic Delphi methodology. The survey also showed evidence of the need to move towards sustainable, healthy and inclusive food systems. By doing this, co-benefits can be delivered also for climate mitigation and adaptation, environmental sustainability, and circularity, sustainable healthy nutrition, safe food consumption, food poverty reduction, the inclusion of marginalised people, the empowerment of communities, and flourishing businesses. Specific results and insights of the survey results are detailed fully in D3.1. 3.3 Results Living Lab 3 Three main questions were asked to stakeholders to get feedback. Answers to these questions are summarized in the figures below: a. Which opportunities may arise in the organization represented regarding possible applications of AI technologies (Figure 12) b. What are the technical requirements for implementing AI in the organizations (Figure 13) c. What are the challenges you are facing within the implementation of AI (Figure 14) 7 Annex 2. Preview of LL2 Delphi Round 1 Survey