Evaluating The Outcomes of Electrical Engineering Workshop Classes

Practice Paper Recommended citation: Buskes, G., Xavier, M. D. S., Beuchat, P., & Balendhran, S. (2025). Evaluating The Outcomes of Electrical Engineering Workshop Classes. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631569. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. EVALUATING THE OUTCOMES OF ELECTRICAL ENGINEERING WORKSHOP CLASSES G. Buskes a, 1 , M. S. Xavier b, S. Balendhran c, P. N. Beuchat d a The University of Melbourne, Melbourne, Australia, 0000-0002-7920-8052 b The University of Melbourne, Melbourne, Australia, 0000-0002-6581-705X c The University of Melbourne, Melbourne, Australia, 0000-0002-4390-3491 d The University of Melbourne, Melbourne, Australia, 0000-0002-1044-3994 Conference Key Areas: Engineering skills, professional skills, and transversal skills; Curriculum development and emerging curriculum models in engineering Keywords: Engineering laboratories, evaluation, learning outcomes, workshops ABSTRACT Student experience surveys in engineering education typically assess the overall experience of a subject, covering aspects such as lectures, tutorials, laboratories and assessments. While they provide valuable insights into broad satisfaction, they often lack a focused evaluation of how students engage in specific environments such as laboratory or workshop classes, which are pivotal in providing students with hands-on experience, developing critical skills, and simulating real-world engineering environments. This study aims to evaluate student learning experiences across three learning domains - cognitive, psychomotor and affective - in a laboratory-based environment through a tailored survey instrument, adapted from existing frameworks. This survey was administered to students across thirteen subjects, spanning an electrical engineering degree programme, to evaluate the perceptions of their learning outcomes in their workshop classes. The results highlight strengths in instrumentation, guided experimentation and teamwork, while revealing several areas for improvement, including sensory awareness and safety, although there were some significant variations across some subjects. Subjects with more intensive teamwork and open-ended design projects tended to improve students’ perceived learning in the cognitive and affective domains. The study has provided insights into how the workshop classes can be optimised to better align with professional engineering practices and accreditation requirements. 1 Corresponding Author: G. Buskes, [email protected] 1 INTRODUCTION Engineering laboratories are essential components of the university learning experience, playing a pivotal role in engineering education by providing students with invaluable hands-on experience that connects theoretical knowledge with practical application (Feisel & Rosa, 2005). Beyond this practical exposure, laboratories promote the development of critical skills necessary for future engineers through inquiry-based learning and problem-solving. Students learn to identify and define problems, formulate and test hypotheses, conduct experiments, collect and interpret data, and draw conclusions. These processes significantly enhance their critical thinking and independent reasoning abilities (Johnstone & Al-Shuaili, 2001). Additionally, through authentic, open-ended design work grounded in rigorous analytical techniques, laboratories stimulate student motivation and creativity while encouraging higher-order cognitive skills (Prince & Felder, 2006). Laboratory work often involves collaboration, which helps students develop interpersonal skills, teamwork, peer teaching, negotiation, and effective communication (Trevelyan, 2007; Winberg & Winberg, 2021). Adhering to safety procedures and cultivating a sense of responsibility are also crucial aspects that students learn in these settings, hence, in this way, laboratories simulate authentic engineering workplace environments and scenarios (Gustavsson et al., 2009). Moreover, engineering laboratories facilitate the development of discipline-specific skills and capabilities necessary for the workforce, preparing students for their future careers while also allowing them to learn from failure - an important aspect of engineering practice. These laboratories can be recognised as addressing multiple learning domains: cognitive skills are enhanced through improved conceptual understanding and problem-solving; psychomotor skills are developed through practical and technical training; and affective skills are nurtured by fostering teamwork, communication, and professional attitudes (Feisel & Rosa, 2005). The Faculty of Engineering at the University of Melbourne employs a ‘workshop’ model as part of the timetabled teaching and learning activities for most of its subjects. The workshop class model flexibly combines traditional, more proceduredriven ‘laboratory’ class activities that utilise devices and equipment to gather data and understand specific phenomena, with active learning activities that provide increased opportunities for problem solving, discussion, and collaboration. This model allows the demonstrators that staff these classes to transition seamlessly between facilitator and teacher roles, creating a relatively informal and collegial learning environment that encourages all students to participate and enhances their learning experience. Furthermore, this approach enables teaching staff to monitor students in person and provide immediate feedback. However, the effectiveness of these workshop classes in developing skills across the three learning domains, both within individual subjects and more broadly across a degree program, has not been captured by traditional ‘whole of subject’ surveys. To this end, this paper presents the adaption and development of a laboratory survey instrument to provide empirical evidence into the effectiveness of the workshop classes as part of an electrical engineering degree program in developing students’ knowledge and skills across the three learning domains: cognitive, psychomotor and affective. Results from the survey administered in subjects across the degree program are presented and used to evaluate students’ perceived learning experiences across these three domains. Furthermore, the results across multiple subjects are used to identify patterns, gaps, and opportunities for curriculum and workshop improvement across the program. A discussion of the benefits and limitations of the study is presented and directions for future work are identified. 2 METHODOLODY Feisel and Rosa (2005) outlined thirteen learning objectives for engineering laboratories, spanning the three domains of cognitive, psychomotor and affective: instrumentation, models, experiment, data analysis, design, learn from failure, creativity, psychomotor, safety, communication, teamwork, ethics in the laboratory, and sensory awareness. The Measuring the Learning Outcomes of Laboratory Work (MeLOLW) survey instrument (Salim et al., 2013) developed a collection of statements to assess students’ perception of their experience of each of these learning objectives and explicitly mapped them to the cognitive, psychomotor and affective domains. This instrument has since been adapted and used as a selfassessment tool for engineering laboratories, for example, by students at the start and end of two electrical engineering laboratory subjects (Nikolic et al., 2015) and to understand the relationships between student evaluation scores and perceived learning across domains between two universities (Nikolic et al., 2020). The survey instrument used in this work follows the three domains outlined in Salim et al. (2013) but expands the thirteen objectives proposed by Feisel and Rosa (2005) to cover experiment, design and teamwork in more detail. The updated subobjectives are motivated by a desire to understand the differences between subjects with respect to how often these subjects go beyond guided procedures and limited design aspects into open-ended design practices which better emulate professional engineering practice. The updated list also allows us to better capture the development of professional skills, particularly around complex engineering problem solving, systematic engineering design processes, systematic management of engineering projects, and effective team membership and leadership, as suggested by accreditation requirements in Australia (Engineers Australia, 2019). Overall, as shown in Table 1, the survey comprised twenty statements across the three domains and an additional item to assess students’ perception of their overall workshop learning experience. The survey was deployed on paper to students in workshop classes in the final two weeks of semester, depending on the subject’s schedule. The final two weeks were chosen to ensure that the students have substantial subject matter expertise and can comment on their workshop experience over the entire semester. Thirteen subjects offered by the Department of Electrical and Electronic Engineering, spanning multiple year levels, were surveyed. A member of the evaluation team who was not associated with the teaching or assessment of the subject, visited each workshop class to administer the survey. The purpose of the study was explained to the participants and a written version of this description was also offered to any participants. Subsequently, a total of 768 students completed their surveys in the workshop classroom, taking approximately 5 to 10 minutes to complete at the start of the class. Students responded to each statement on a 5-point Likert scale ranging from “1” representing “Strongly Disagree” to “5” representing “Strongly Agree”. The surveys were anonymous to encourage honest responses and to protect participant identity. The research was conducted with approval from the university’s Human Research Ethics Committee application HREC 30767. 3 RESULTS AND INSIGHTS 3.1 Statement responses Table 1 shows a summary of the mean scores and standard deviations for the twenty statements on the workshop outcomes survey across all subjects. Surveys with identical responses for all statements (e.g., all responses as ‘strongly agree’) were removed from the analysis, leaving a total of six-hundred and ninety-one surveys. Some surveys had no response entered for some statements, and these were treated as missing values in the statistical analysis that follows. Table 1 also includes the domain each statement maps to and its specific measurable objective. Table 1. Question means and standard deviations for all subjects (N=691) Statement Description Domain Objective Mean SD 1 Apply appropriate sensors, instrumentation, and/or software tools to make measurements of physical quantities Cognitive Instrumentation 4.36 0.74 2 Identify the strengths and limitations of theoretical models as predictors of real-world behaviours Cognitive Models 4.15 0.83 3 Read and comprehend datasheets, circuit diagrams, user-manuals or previous literature Cognitive Experiment (comprehend) 3.90 1.00 4 Follow a guided/given experimental or simulation approach, and utilise appropriate tools (software, equipment and procedures) Cognitive Experiment (guided) 4.35 0.78 5 Devise and implement an experimental or simulation approach to characterise an engineering material, component, or system, and specify appropriate tools required (software, equipment and procedures) Cognitive Experiment (open-ended)) 4.27 0.75 6 Demonstrate the ability to collect, analyse, and interpret data, and to support conclusions Cognitive Data Analysis 4.23 0.76 7 Design and build (or simulate) a product or system following a guided/given procedure Cognitive Design (guided) 4.28 0.80 8 Design and build (or simulate) a product or system within an open-ended project which involves developing system specification from requirements and/or meeting client requirements Cognitive Design (open-ended) 4.01 1.00 9 Test and debug a prototype using appropriate tools to satisfy requirements Cognitive Design (prototype) 4.00 0.95 10 Demonstrate competence in selection, modification, and operation of appropriate engineering tools and resources Psychomotor Operation of Equipment 4.02 0.85 11 Use human senses to gather information and to make sound engineering judgments in formulating conclusions about real-world problems Psychomotor Sensory Awareness 3.84 0.95 12 Identify unsuccessful outcomes due to faulty equipment, parts, code, construction, process, or design, and then re-engineer effective solutions Affective Learn from failure 3.97 0.95 13 Demonstrate appropriate levels of independent thought, creativity, and capability in real-world problem solving Affective Creativity 3.98 0.87 14 Identify health, safety, and environmental issues related to technological processes and activities, and deal with them responsibly Affective Safety 3.56 1.19 15 Communicate effectively about laboratory work with a specific audience, both orally and in writing, at levels ranging from executive summaries to comprehensive technical reports Affective Communication 3.93 1.02 16 Work effectively in teams during workshop activities conducted in class Affective Teamwork (in-class) 4.31 0.87 17 Continue to work effectively in teams outside workshops for assessment tasks related to workshops Affective Teamwork (out-of-class) 4.12 0.95 Statement Description Domain Objective Mean SD 18 Work through a collaboration framework which included structuring individual and joint accountability; assigning roles, responsibilities, and tasks; monitoring progress; meeting deadlines; and integrating individual contributions into a final group report Affective Teamwork (structured) 3.99 1.01 19 Behave with highest ethical standards, including reporting information objectively and interacting with integrity Affective Ethics 4.14 0.90 20 Overall, I had a very good learning experience in the workshops Overall 4.19 0.86 The statements with the highest means were S1, S4, and S16 covering Instrumentation, Experiment (guided), and Teamwork (in-class), respectively. This does not come as a surprise, given the collaborative nature of the workshop classes and the fact that they typically use equipment to validate theoretical concepts through experiment. However, it can be seen that these three statement means vary considerably across subjects, as Table 2 shows. Note that for ease of reading, each subject code in subsequent tables has been re-coded in terms of its sequence in the electrical engineering degree programme: an initial ‘2’ representing undergraduate foundational material, ‘3’ representing undergraduate core fundamentals and ‘9’ representing higher-level and advanced masters-level material. Table 2. Subject descriptions and means for S1, S4 and S16 for all subjects Subject Subject description N S1 S4 S16 2A Circuits I 164 4.62 4.61 4.26 3A Circuits II 24 4.42 4.50 4.21 3B Engineering electromagnetics 44 4.52 4.39 4.41 3C Signals and systems 14 4.36 4.36 4.00 3D Electronic systems design 83 4.40 4.23 4.08 9A Analog electronics I 9 4.33 4.33 4.56 9B Control systems I 136 4.14 3.99 4.21 9C Signal processing 59 4.15 4.36 4.41 9D Communication networks 7 4.14 4.71 4.86 9E Control systems II 60 4.42 4.50 4.58 9F Embedded systems 63 4.36 4.32 4.48 9G AI for robotics 21 3.81 4.24 4.52 9H Hardware accelerated computing 5 4.20 4.60 4.60 Subject 2A is the first subject in the degree sequence, where students learn in a guided fashion how to use a portable electronics device (Analog Discovery 2) to build and test circuits, which likely explains the highest mean for S1 and second highest mean for S4. On the other hand, 3B introduces students to standard electrical benchtop equipment such as signal generators, power supplies and oscilloscopes, which likely reflects that it had the second highest mean for S1. 9G, an advanced elective, has the lowest mean for S1, likely due to the students being removed from the actual sensors and instrumentation so that they may focus on implementing their own code for decision making algorithms for a robot. Subjects scoring high on S16 include 9A, 9D, 9E, 9F, 9G and 9H, which all involve intensive group work in order to complete the tasks during workshop classes. The lowest mean for S16 was 3C, which could be attributed to a primarily guided workshop structure and challenges with team collaboration and workload distribution within the student cohort. Interestingly, 3D, which is team-based and involves a open-ended design project in the latter part of the semester, also figures low on the mean score for S16. This low mean may be associated with workshop classes finishing early or poor teamwork dynamics and project management practices. The statements with the lowest means were S11 (3.84) and S14 (3.56), covering Sensory Awareness and Safety respectively. However, the standard deviation of S14 on Safety was relatively high, indicating that students’ experiences of safety in the workshop class likely varied significantly according to subject. A one-way ANOVA revealed that there was only one statement, S5, with no statistically significant difference in means between any subjects (F(12, 674) = [1.122], p = 0.339). The compound phrasing of the S5 statement, ‘Devise and implement an experimental or simulation approach…’ might be interpreted by students as a disjunctive (OR) condition between ‘devise’ and ‘implement’, rather than a conjunctive (AND) condition. As all subjects’ workshops implement some form of experimental or simulation approach, the responses for individual subjects for this statement have likely become indistinguishable. 3.2 Learning Domains Recalling that each statement corresponds to one of the three learning domains, Figure 1 gives the means of each domain across all subjects. The overall domain means across all subjects is 4.17 (Cognitive), 3.93 (Psychomotor), and 4.00 (Affective). Interestingly, level ‘2’ and level ‘3’ subjects on average outperformed the advanced, level ‘9’ subjects across the three domains. Subject 3A has the highest mean for the Cognitive domain, which is likely associated to the combination of a large amount of technical content in the lectures, guided workshops for fundamental concepts and open-ended workshops for a design project. This is a combination that many students find unusual, particularly as project-based learning tends to be concentrated in later, more advanced subjects, rather than core or foundational (content-heavy) subjects like 3A. Fig. 1. Domain means across all subjects (N=691) Subject 9A has the highest means for Psychomotor and Affective. We recall this subject is project-based and involves printed circuit boards, which reasonably explains the Psychomotor mean. In addition, students may keep their circuit designs developed throughout the semester, which could justify the large Affective mean. Note that while being in the mid-range for means for the Cognitive and Psychomotor domains, 3C had the equal lowest mean for the Affective domain. This may be due to not only the teamwork challenges highlighted before, but also the mathematically intensive content and a lack of tangible connection to real-world relevance making it 3.6 3.7 3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5 2A 3A 3B 3C 3D 9A 9B 9C 9D 9E 9F 9G 9H cognitive psychomotor affective difficult for the students to emotionally engage with the content. To improve affective engagement in this subject, real-world examples should be integrated into the workshops with improved opportunities for collaboration and hands-on activities. 9E incorporates a large simulation-based project, which is completed individually and separately from workshop classes. Hence, a lower affective score could be expected, particularly since this setup negatively impacts S14, S17 and S18. 3.3 Comparing guided versus open-ended design workshops Subjects 2A and 3A provide an overview of fundamental tools for the analysis of electric circuits and form the foundation of many subsequent subjects in the electrical engineering degree. While 2A follows a conventional structure with guided workshops where students work through a list of tasks with limited aspects of innovation, 3A includes an open-ended design project where students are provided with a complex engineering problem and asked to collaboratively “think as engineers” (Xavier & Wong, 2024). Due to their close relationship in terms of technical content, but different forms of workshop delivery and assessment, these subjects provide a useful point of comparison. An independent samples t-test was performed across all statements, resulting in five statements with means that were statistically significant, as shown in Table 3, along with their effect sizes (Cohen’s d). It can be seen that the introduction of project-based learning in 3A has resulted in statistically significant improvements in S8 (due to the open-ended nature of the project), S9 (requirement to build a physical prototype of the team’s proposed solution), S11 (project aims to emulate a real-world problem), S13 (project requires an innovative solution) and S15 (students are required to present results both orally and in writing throughout the semester). However, we note that the overall workshop learning experience for both subjects (S20) did not show any statistically significant difference. The same observation applies to aspects of team collaboration (S17) and project management (S18), suggesting that students have perceived minimum improvements in these important professional skills. Table 3. Statements with statistically significant (p < 0.05) differences in means for subjects 2A and 3A. Subject 2A (N=164) Subject 3B (N=24) Statement Cohen’s d Mean SD Mean SD S8 Design 1.05 3.82 1.09 4.57 0.59 S9 Design 0.94 3.85 0.96 4.38 0.77 S11 Sensory Awareness 0.93 3.82 0.94 4.17 0.97 S13 Creativity 0.91 3.84 0.95 4.33 0.56 S15 Communication 1.06 3.77 1.10 4.29 0.69 4 DISCUSSION AND FUTURE WORK 4.1 The value of perception This study focused on students’ perception of their learning and workshop experience via the twenty statements, hence the value and limitations of the insights must be interpreted in that context. Some of the statements are conceptually easier to perceive, for example, a student presentation delivered in workshop classes is clearly mapped to the S15 statement about communication, hence leading to higher average for this statement. However, distinguishing between whether an experimental approach (S4 and S5) or design procedure (S7 and S8) was guided or open-ended can have more variability in perception since students may struggle to accurately differentiate between simpler experimental/design procedures and more authentic, open-ended learning experiences. Additionally, in a future semester, perceptions of a past subject can change as their engineering knowledge and skills develop and scaffold over time. This ongoing learning process may influence how they retrospectively evaluate their earlier experiences. 4.2 Deployment strategy One key consideration is that numerous subjects conduct assessment during workshop classes, thus, some students may view this (relatively short) survey as taking away time from their assessment preparation. The demonstrators generally announced the survey to the class and highlighted that its importance. However, students would sometimes immediately put the survey paper aside, continue with their work, and then complete it in a shorter time than others. Additionally, depending on enrolment, an individual student completed the survey instrument several times (for different subjects), meaning that familiarity and fluency with the questions could change the interpretation. With the longer-term view of embedding this survey instrument in the feedback and continuous improvement cycles of teaching and learning, it is valuable for future work to investigate whether and how this workshop evaluation process becomes engrained into the culture of our program. 4.3 Educating a holistic engineer Accreditation standards are an important guide for the knowledge and skills we aim to impart through our teaching. As such, our expansion of this survey instrument to twenty statements, from the thirteen in Salim et al. (2013), means it can more precisely map onto individual elements of an accreditation standard. Moreover, it is not expected that every subject scores high on every statement. Thus, a robust and reliable measurement of student perception of workshops contrasted alongside teaching materials and pedagogies is a valuable contribution to program-wide evaluation at various levels of the university. Future research will also aim to investigate synergies between the learning domains where the perception of certain statements tend to vary together, e.g., the suggested synergy between open-ended design (S8) and sensory awareness (S11) requires further investigation. Moreover, investigating other ‘domain’ groupings may provide new insights for program-wide evaluation, for example, authentic perceptions of ethics (S19), safety (S14), and reading manuals, literature (S3) can be elusive despite an educator’s best efforts. 5 CONCLUSION This paper presented the development of a survey instrument to assess student learning in workshop classes across the three domains of Cognitive, Psychomotor, and Affective in an electrical engineering degree programme. Results indicated that there were significant differences between subjects, linked to variations in delivery mode, teaching content, and learning activities and can inform the refinement of workshop content and delivery strategies to better align with the intended learning outcomes across the three domains. Notably, the results highlight opportunities to improve teamwork experiences in subjects where groupwork does not transition into truly collaborative teamwork experiences. Another key takeaway is the importance of embedding safety education and practices consistently throughout the programme. Overall, this survey framework can serve as a valuable tool for engineering educators to assess the student learning experiences in laboratories or workshops either in particular subjects or throughout a whole programme.