Ethics of AI and Data in Education The Situated ETH-TECH Perspective: a Framework for Practice_Project Deliverable –Work Package II «ETH-TECH Framework»

Ethics of AI and Data in Education The Situated ETH-TECH Perspective: a Framework for Practice Project Deliverable –Work Package II «ETH-TECH Framework» The Project & Partnership Anchoring Ethical Technology (AI and Data) Usage in the Educational practice (ETH-TECH) The ETH-TECH project aims to promote an ethical approach to the use of technologies, especially data and artificial intelligence (AI), in higher education (HE) teaching and learning. It is grounded on European Union initiatives, such as the ethical guidelines on the use of AI and data in education published in 2022, which encouraged educators to reflect on the consequences of the digital transformation. ETH-TECH channels the complex professional expertise of university educators and researchers from 4 EU countries, to provide a culturally sensitive and contextualized framework for the ethical use of AI in HE. This project deliverable is the result of joint collaboration between all partners. Final document authored by BBU and UNIPD teams (Work Package Leaders). 2 Let’s make a start! ETH-TECH objectives To raise awareness among future educators about ETH-TECH perspectives in HE courses. To develop a set of practical tools for selfassessment of the effective integration of an ETHTECH approach in HE, based on EU guidelines. To support the development of an ETH-TECH approach through a set of OER. To foster creative engagement of key stakeholders towards the ETH-TECH. ETH-TECH activities Awareness-raising sessions, after the participatory analysis of syllabi and teaching practices on AI ethics and data use in education. Creation of a self-assessment framework and tools to promote dialogue on current gaps and future challenges of the ETH-TECH approach. Development of Open Educational Resources and a 25-hour course (1 ECTS) to facilitate the ETH-TECH approach in institutions. Organization of Educathon events to validate the ETH-TECH approach and resources with NGOs, SMEs, and civil society. Understanding the ETH-TECH objectives and activities will help us understand the value of the present framework 3 The aim of this framework is to provide support to future educators (and teachers and educators’ trainers) to reflect about the ethics of AI and data in education*. ETH-TECH critically integrates the 7 principles on the use of AI and data in education proposed by the European Commission in 2022. Moreover, we consider the recent developments related to the AI Act (European Commission, 2024) for Higher Education. Principle Short explanation and key guiding question Human Agency and Oversight AI can help teaching and learning goals and collaborative work with colleagues. It is key that educators have control and oversight of the AI -supported products being developed. Who controls and monitors the AI you use for education? Transparency AI systems need to clearly explain how they function and what data they process. Do you know how the AI you use for education works? Diversity, non - Discrimination and Fairness AI systems should be designed to accommodate the diversity and needs of all teachers and students, who should be able to use these systems equally. Could the AI system lead to discrimination or unfairness towards some users? Societal and Environmental Wellbeing The increasing use of AI tools for educational activities should not have a negative impact on broader ethical concerns, such as toxic dependence, lack of academic integrity or environmental impact. How does the AI you use for education make you feel? Privacy and Data Governance In the “ datafied” society user interactions with AI systems are recorded and potentially monetized. It is important to understand how personal data is collected, stored, and used by AI systems. How is your data collected when you use AI for education and how is it used? Who has access to this data? Technical Robustness and Safety Many AI systems operate as “black boxes,” which means that their algorithms work in unclear ways, or the interfaces do not explain how data is handled. You depend on systems you don’t fully understand, which can undermine trust and pedagogical alignment. Is your interaction with AI for education secure and you can trace each step of the interaction? Accountability Teachers and universities need to understand and monitor how Ed -Tech enhanced by AI works, to be in contact with their developers for troubleshooting, and to tackle potential problems. Who can you contact when something goes wrong with your using AI for education? Table 1. Synthetic presentation of the 7 principles on the use of AI and data in education (European Commission, 2022) Take your time... And explore the several principles. Use the questions in the table to think about the AIpowered systems in use in the institution where you take part as a teacher or a student. During an initial analysis carried out over more than 300 syllabi across four member states (Germany, Italy, Romania, Spain) we found a troubled picture of how the Ethics of AI and data is taught to future educators. Read more HERE. 4 Certainly, these principles are very abstract and noncontextual. The ETH-TECH framework recommends to critically integrate EU regulations on ethical use of AI. Here, you will see several things to reflect upon before considering the ethical principles. 5 Analyse the role of culture Each local context has a dominant cultural orientation (for example, collectivism versus individualism), which is reflected on how legislators, institutions, and educators approach AI for education. These cultural orientations place more or less responsibility on the individual (administrator, teacher, student) or the legislators (national or local administrative institutions) to define, enforce, and support ethical AI use in education. The trust that people have in the system is the first line of analysis that needs to be considered. Guiding reflection questions Who regulates the ethical use of AI in education in your country? How much flexibility does your university have in enforcing ethical use of AI in education? Do you feel protected yet responsible when you experience a breach in ethical use of AI in education in your practice? 6 Consider the national and local socioeconomic dynamic Local (embedded in national) contexts may have more or less resources in educating HE institutions about the ethical use of AI in education. This creates inequalities at national and regional level and requires a tailored approach of what is possible to be implemented in terms of procedural regulations and training of teachers. Guiding reflection questions Who provides resources (information, training, assistance for critical problems) for ethical use of AI in your regions? Is the ethical use of AI for education legally regulated and are there resources to ensure the implementation of these regulations? Do teachers and students have the socioeconomic resources (time, education about how AI works, tools and skills for critical reflection) to ensure ethical use of AI in their educational practice? 7 Understand the structure of the specific educational system Each educational system works differently, starting from how HE is structured to who can access HE. Moreover, the autonomy HE institutions have differs among EU countries. These structural realities greatly shape WHAT and WHEN can be done to implement ethical use of AI in education. Guiding reflection questions Does your university or your national Ministry of Education decide on the ethical guidelines for AI in education? Do teachers and students in your university benefit from training on the ethical use of AI in education? Can you autonomously integrate ethical guidelines of AI use in education in your course syllabi? Who can you contact when there is an ethical breach of AI use in education? 8 The ETH-TECH team conducted Awareness Raising Sessions with students and teachers in the 4 project countries*. These sessions uncovered the multilayered dynamic of ethical use of AI in education. Participants in the Awareness Raising sessions conceptualized three hierarchical levels of ethics in practice: 1. Normative/Technological; 2. Institutional; 3. Personal (teacher, student, classroom as regulated interaction of individuals). This dynamic is synthetized in Figure 1 and detailed below. *The country-level reports detailing the sessions and national conclusions are available online: Germany (HSU team) - Italy (UNIPD team) - Rumania (BBU team) - Spain (UB team) Normative/ Technological Level Institutional Level Personal Level •International regulations. •National and institutional information available. •Institutional information available. •Institutional tech tools. •Institutional regulations and codes of practice. •Teaching practice and professionalism. •Academic integrity. •Personal positionality. Levels to understand the ethics of AI and data in the educational practice 9 Ed-Tech enhanced by AI should help teachers and students reach their teaching and learning goals and work with colleagues to create better academic and educational work. At each point, it is key that teachers have control and oversight of the AI-supported products so they can intervene in cases of errors, misinformation, discrimination and student overreliance on the systems. Human Agency and Oversight Friendly Definition In a software certification course, there is a very active group of participants who promote informal support for the study. In this regard, they have opened a WhatsApp channel to support each other in their learning efforts. Within this group, it emerges that the use of AI tools such as Claude or Copilot is perfect for writing a programming assignment required in one of the teachings. The teacher is not aware of the tool and does not have access to efficient tools for detecting AI-generated content, as they have not been developed yet. Therefore, many students create their entire assignment with AI. Despite some surprise at the unusually high work quality of this generation of students, the teacher does not worry much: the more participants are certified, the higher the success rate of the course, the better the remuneration. The use of AI in students' work is not discussed during the course, and students begin to use AI as a shortcut to complete their assignments rather than a tool that can assist them in their learning. Case Study Guiding Questions Can you relate to this situation? In your local educational context, do you think students can use AI as a tool that impairs their learning process, despite seemingly leading to good results? 16 Human Agency and Oversight Action Points: Institution Problem: Teachers ask the university leadership to clarify the role of genAI in EdTech products that are formally used by the university to facilitate teaching and student learning. They stress that more genAI has been introduced in the last versions of these products and fear that student decisions are now controlled by AI. Action point: Require human oversight of automated decisions. AI-guided decisions for different features need to be transparent to the teachers who can then facilitate or restrict student access to these features. Especially when AI features reflect content creation or student grading, teachers need to be able to make decisions regarding feature activation or customization. Problem: Teachers and students complain that they don’t fully understand the limitations of genAI use for education. The former ask for institutional regulations and the latter claim that they can decide themselves how to use AI for education, as there are no clear rules. Action point: Considering national and university regulations regarding the use of AI for education, create an ethical code and implementation rules (in terms of actions for specific issues) that all teachers must adhere to and integrate into their syllabi and all students must formally acknowledge at the beginning of the academic year. Problem: The ethics committee of your university faces new and more complex ethical complaints brought by teachers and students regarding the fraudulent use of genAI and cannot keep up with updating the ethical code of the university of these issues. Action point: Make a database of types of problems and complaints on the unethical use of AI and genAI in education, to gradually create categories of problems. By making legal advisors, tech specialists, teachers, and students aware of types of problems, a community focusing on transparent problem-solving can be developed. This can be a procedural manner to ensure transparent and participatory decision-making for AI-related issues in education. 17 Human Agency and Oversight Action Points: Teacher Problem: The learning community of a course has been greatly affected by the ambivalence of AI use for education, which some students perceive as cheating and others see as a natural enhancement to their academic learning process. Action point: Ensure meaningful consent for AI use in education and offer alternatives without penalty, by co-creating "didactic contracts" at the course outset. Problem: The assignments students turn in have many fake references (including of your own publications) and use a complex and somehow unnatural type of phrasing. You suspect that they have been generated with the use of genAI. Action point: Shift towards process-oriented student assessment, that relies on here-and-now actions and is not influenced by gen AI. Oral evaluations and project work that occurs during seminars as “inbasket” or situational tests can better capture students’ competences. Problem: Students don’t understand how the excessive use of genAI for education can negatively impact their critical thinking. Action point: Teach students to critically reflect on contents generated by genAI for questions you use in your own course. You can introduce the development of critical reflection on genAI use for education as a transversal skill that your course develops. 18 Human Agency and Oversight Action Points: Student Problem: Teachers do not clearly state in the course syllabi if and which genAI you can use for academic learning. Action point: Openly discuss with the professoriate as the beginning of the academic year which is the official position of the university and each teacher’s specific course regarding the use of AI for academic learning. Problem: All your colleagues have been using genAI for most academic work and you feel you also must use it to remain competitive. Action point: Critically analyze what “competitive” means for a specific course. Using genAI for generating academic work does not enhance your knowledge and competences and often creates just an illusion of competency. Problem: The answers that the genAI conversational agent has been giving you for a question that your teacher approached in a course are very different from what the teacher presents during the course. Action point: Critically analyze each answer and discuss with the teacher the differences you identified. Fact checking and careful reading of references used by the AI and the teacher are very important in understanding why the answers are different. 19 AI systems need to clearly explain how they function, what data they collect and for what purposes. Students, teachers and universities should be informed about these aspects so they can give their informed consent when using AI systems. Friendly Definition A university introduces an AI-powered software that assists students in their learning. It works as a virtual assistant which gives students detailed instructions and feedback on their tasks but also includes emotional support to help students manage their mental health during times of academic stress. Both students and teachers are happy to use this free system: students appreciate the immediate and personalized assistance, while teachers appreciate the reduced workload. However, some students notice that they began receiving ads for paid study materials, online courses and tutoring services. Some of them also received ads for mental health services, and apps targeted on issues like those discussed with the software’s chatbot. Over time, students and teachers become sure that the data is shared with third parties and used for commercial profiling. Case Study Guiding Questions Do you know what data is collected during your interactions with AI systems you use in your educational context and how it is later used? Would knowing your data is shared for commercial profiling influence if and how you (as student, teacher or institutional educational staff) use AI systems? Transparency 20 Action Points: Institution Transparency Problem: Teachers and students point out that the AI systems officially used by the university are linked to commercial platforms that offer personalized solutions based on class work. Action point: Promote transparency about how data and algorithms work. Request the AI system provider to provide and explain how data collected from university users are used and shared. Problem: Administrators and teachers point that existing university-level are not available or outdated to deal with genAI ethical issues in HE. Action point: Make shared decisions about tool usage and curriculum-integrated ethics as approaches aligned with participatory pedagogies and democratic education. Problem: Though the university provides an ethical code for AI use or references the EU-level regulations, teachers and students do not understand these regulations and cannot link them to their teaching and learning practice. Action point: Provide training for both teachers and students on ethical AI use, using co-design scenarios that deal with learners and community problems. 21 Action Points: Teacher Transparency Problem: Integration of AI-generated materials in the course can be useful as practical examples for many subjects. Nevertheless, when the teacher generates material using complex prompts for a genAI agent, it is not clear who owns the product and how it will be used by the AI developer company in the future. Action point: When given the possibility, always opt for an open-source solution to generate new course material or a genAI solution that is transparent about data storage and sharing. Problem: As teachers use multiple AI and genAI tools in their teaching practice, it may be difficult to keep track which tools are transparent on their data sharing practices. Action point: Take time at the beginning of each year to choose the AI tools you will use for education. Then gather information on how the information you input into AI systems for education is being stored, shared, and who owns the products you develop with the help of AI. Problem: It is not clear how GenAI powered learning tools evaluate student knowledge and competence acquisition, which makes it hard for the teacher to accurately assess how much students have learnt. Action point: Become informed how the AI systems you use for student learning gather information to provide assessments of student learning progress. 22 Action Points: Student Transparency Problem: genAI is easy to use and offers quick answers starting from very simple questions. But students become increasingly aware that their choices when using AI for education are transferred to other platforms that may not be linked to education. Action point: Understand what the “digital footprint” means. If private companies are behind AI tools, data will be collected for commercial purposes. There are multiple negative ethical implications of educational contents and educational decisions that are monetized without the awareness and active consent of the user. Problem: It is difficult to understand how AI and genAI work and where to search for easy-tounderstand information on how these systems function, who is responsible for their mechanics, and how data is being used. Action point: Read and watch presentations developed by legislators that try to regulate AI for education. Here are presentations of the AI Act developed by the European Commission (2024): https://artificialintelligenceact.eu/ * Take into account that the complexity of the technological systems behind genAI cannot be fully grasped from a short presentation.. Website maintained by the Future of Life Institute (FLI). FLI is an independent non-profit working to reduce large-scale, extreme risks from transformative technologies. https://artificialintelligenceact.eu/about/ 23 All teachers and students should be able to access the AI (or Ed-Tech enhanced by AI) in the same manner and the AI system should be designed to accommodate for the diversity of all students, including those with special needs. AI systems should not facilitate discrimination or other inequitable practices. Friendly Definition A professor at a multicultural university created a presentation of the university for prospective students. Dall-E (an image generation system) and Canva (freemium versions) are used to generate the presentation. In creating some of the images, the professor realizes that all the images of scientists generated by AI include middle-aged men, usually Caucasian and shown in a central position. When the prompts are changed to ask for female and disabled scientists, they are usually presented in a supporting role. Case Study Guiding Questions Do you think AI can reinforce pre-existing stereotypes and biases in your context? Do you think that in your university/educational institution AI is equally accessible for all students, regardless of background and possible special needs? Diversity, nonDiscrimination and Fairness 24 Problem: AI and genAI educational products implicitly promote stereotypical presentation of learners and expected learning outcomes. Gender stereotypes, mental health stereotypes, performance stereotypes, educational achievement expectations are embedded in images, types of tasks, and evaluation options. Action point: Choose an AI tools for education that is customizable to the characteristics of the student learners in your university and that offer teachers the possibility of customization for a specific course. Problem: Many AI solutions for education do not accommodate the learning needs and learning pace of students with physical and/or cognitive disabilities. They promote the idea that everyone can become competent and can reach the same high levels of performance. Action point: Provide teachers technical support to tailor AI tools that can be adopted by diverse students. Technical staff needs to have operational competences in choosing the most appropriate AI tools for education for specific types of student disabilities. Refuse AI tools that does not align with diversityProblem: AI instruments for education do not acknowledge the diversity of learner needs and do not promote the role of acceptance of diversity from teachers and students. Action point: Focus on offering teachers training programs that help them recognize and work with student diversity by fostering inclusion and acceptance. Diversity, nonDiscrimination and Fairness Action Points: Institution 25