Assessing Course Learning Outcomes, Engineers Australia Competencies (EA PE) and Sustainable Development Goals (SDGS) Attainment in Engineering Education: Pilot Study

Research Paper Recommended citation: Nair, D. J., Rashidi, T. H., Mario, J. V., Chen, R., Zhang, C., & Barati, K. (2025). Assessing Course Learning Outcomes, Engineers Australia Competencies (EA PE) and Sustainable Development Goals (SDGS) Attainment in Engineering Education: Pilot Study. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631503. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. ASSESSING COURSE LEARNING OUTCOMES, ENGINEERS AUSTRALIA COMPETENCIES AND SUSTAINABLE DEVELOPMENT GOALS ATTAINMENT IN ENGINEERING EDUCATION: A PILOT STUDY D. J. Nair a,1, T. H. Rashidi a, J. Videla Mario a, R. Chen a, C. Zhang a, K. Barati a a School of Civil and Environmental Engineering, UNSW Sydney, Sydney, Australia Conference Key Areas: Engineering skills, professional skills, and SDGs Keywords: Professional competencies, SDGs, engineering curriculum ABSTRACT Engineering education is undergoing significant transformations to address global technological advancements, industry demands, and sustainable use of resources. While traditional curricula emphasize technical knowledge, modern engineers require interdisciplinary collaboration, innovation, and sustainability awareness. However, existing educational frameworks lack a structured integration of Course Learning Outcomes (CLOs), Engineers Australia Professional Engineer (EA PE) competencies, and Sustainable Development Goals (SDGs), making it difficult to assess students’ competencies effectively. To address this gap, this study proposes a new assessment model integrating CLOs, EA PE competencies, and SDGs. The model employs statistical modeling techniques, specifically Structural Equation Modeling (SEM) to evaluate students' learning progress, competency development, and sustainability awareness. By applying quantitative and qualitative analysis, this framework provides a systematic approach to improving engineering curricula. The findings will contribute to better assessment methods and curriculum development strategies, ensuring that future engineers are well-equipped to meet evolving industry and sustainability challenges. 1 Corresponding Author D. J. Nair [email protected] 1 INTRODUCTION Engineering education is rapidly evolving to meet global technological advancements, industry demands, and Sustainable Development Goals (SDGs). While traditional curricula emphasize technical knowledge, modern engineers increasingly require competencies in interdisciplinary collaboration, innovation, teamwork, ethical responsibility, and sustainability awareness. This shift necessitates competency-based education (CBE) frameworks capable of systematically evaluating students' diverse skill sets and development trajectories. Course Learning Outcomes (CLOs) form a critical component of CBE, ensuring students acquire technical and professional skills, such as mathematics, engineering design, computational tools, communication, teamwork, and ethical judgment (Mukhopadhyay & Smith, 2010). The adoption of Outcome-Based Education (OBE), employing rubrics, self-assessments, and performance analytics, has improved students’ employability and alignment with industry needs (Male et al., 2011; Swamy et al., 2024). Nevertheless, challenges persist, including inconsistent definitions and implementation of CLOs across institutions, disconnect between theoretical curricula and practical industry applications, and limited sustainability integration (Hadgraft & Kolmos, 2020; Swamy et al., 2024; Beagon et al., 2023). In the Australian context, The Engineers Australia Professional Engineer (EA PE) competencies (Engineers Australia, 2019) provide benchmarks for Australian engineering graduates, categorized into three domains: foundational engineering knowledge, engineering application abilities (e.g. problem-solving and system design), and professional attributes including communication, leadership, teamwork, and lifelong learning. Despite initiatives to align CLOs with EA PE Competencies, research indicates persistent integration. Traditional assessments often prioritize theoretical knowledge, overlooking competencies such as problem-solving, ethical reasoning, sustainability awareness, leadership, and interdisciplinary collaboration (Chen et al., 2022; Durrans et al., 2020). To address these gaps, scholars advocate approaches like project-based learning, industry placements, and case-based assessments, promoting holistic evaluation and skill development (Chen et al., 2022; Durrans et al., 2020). Integration of SDGs into engineering curricula has become increasingly vital due to global demands for sustainable infrastructure, energy-efficient technologies, and environmentally responsible design practices (Leal Filho et al., 2019). However, inconsistencies persist in embedding SDGs within curricula due to the lack of integration frameworks, limitations of traditional assessment approaches, and insufficient interdisciplinary exposure (Gutierrez-Bucheli et al., 2022; Beagon et al., 2023; Hadgraft & Kolmos, 2020; Chen et al., 2022). Recent research has proposed effective strategies to address these challenges, including the use of Project-Based Learning (PBL), competency-based assessments such as portfolios, case studies, peer evaluations, and enhanced industry-academic collaborations to offer students practical exposure to sustainability and real-world applications (Serafini et al., 2022; Swamy et al., 2024; Beagon et al., 2023). To bridge the curriculum gap between CLOs, EA PE competencies, and SDG integration, there is a need for a structured, integrated, data-driven competency assessment model. Statistical modeling techniques, particularly Structural Equation Modeling (SEM), have proven effective in evaluating causal relationships among different observed variables, such as student learning outcomes, competency development, and sustainability awareness (Yin & Huang, 2021). By systematically analyzing these elements, SEM provides valuable quantitative insights for enhancing curricula and ensuring alignment with professional and sustainability standards. This research study proposes a comprehensive competency assessment model integrating CLOs, EA PE competencies, and SDGs, employing SEM to systematically evaluate students' learning trajectories, competency development, and sustainability awareness. The outcome of this research aims to deliver actionable insights and practical recommendations for curriculum improvements, ensuring engineering graduates are thoroughly prepared to address contemporary industry demands and global sustainability challenges. 2 METHODOLOGY This study employs a mixed-methods approach to develop and validate a competency-based assessment framework. It applies Structural Equation Modeling (SEM) to evaluate how engineering curricula align with industry standards and sustainability objectives. 2.1 Research Design The research is structured into three phases: 1. Data Collection: Student performance records, competency self-assessments, and SDG awareness survey data. 2. Model Development: Construction of SEM to analyze relationships between CLOs, competencies, and SDG knowledge. 3. Model Validation & Curriculum Recommendations: Using statistical validation methods to assess model accuracy and proposing curriculum adjustments. 2.2 Data Collection Primary data are collected through an online survey . Survey design considerations and a link to the survey are provided in Appendix B. The survey gathers: • Student self-assessment on competencies in technical skills, teamwork, communication, and sustainability awareness. • Demographics: gender, academic background, and prior work experience to support subgroup analyses. • Qualitative feedback through open-ended survey questions capture students’ perceptions of their learning experiences and competency development. Secondary data consist of information generated through course delivery and assessments. This includes: • Course assessment data: Student scores and grades from quizzes, midterms, assignments, project reports, reflective journals, and final exams. • Assessment artifacts: Assignment submissions, project reports, and reflective journals are reviewed to analyze how students demonstrate technical and professional skills in their coursework. • Course outlines, assessment structures, and mapping documents that link assessments to CLOs, EA PE competencies, and SDGs. The study adheres to institutional ethical guidelines (iRECS7821) by obtaining informed consent from participants, ensuring data anonymity to protect student privacy, and utilizing aggregated performance data without identifying individual students. 2.3 SDG and EA Competencies Mapping A critical piece of this model is the link between Learning outcomes, EA professional competencies and SDGs. The SDG mapping employed in this research project is initially based on the mapping technique proposed by Adams et al. (Adams, Jameel, & Goggins, 2023) and further revised by course coordinator. Engineers Australia provides a detailed descriptions of their professionals’ competencies (Engineers Australia, 2019). The mapping of CLOs to EA professional competencies is performed using their guidelines. 2.4 Data Analysis Techniques The primary analytical technique employed in this study is SEM. SEM is particularly suitable for evaluating complex relationships among students' achievement of CLOs, their development of professional EA PE competencies, and their awareness of SDGs. The SEM framework incorporates both primary and secondary data sources. Primary data, collected through student self-assessment surveys, capture student perceptions of their competency development, teamwork skills, communication abilities, and sustainability awareness. Demographic information supports subgroup analyses, further informing the model. Secondary data, including course outlines, assessment structures, and artifacts, facilitate mapping individual assessments to CLOs, EA PE competencies, and SDGs. This mapping clarifies observed variables and their theoretical relationships within the SEM, providing objective measures of student performance and detailed context regarding intended learning outcomes and competencies. SEM Model Specification and Structural Paths: Demographics → CLO Achievement (CLO_Achieve); Demographics → EA Competency (EA_Comp); Demographics → SDG Awareness (SDG_Aware); CLO Achievement (CLO_Achieve) → Overall Performance (OP); EA Competency (EA_Comp) → Overall Performance (OP); SDG Awareness (SDG_Aware) → Overall Performance (OP). These defined structural paths evaluate how demographic factors influence learning and competency outcomes, and subsequently, how each area contributes to holistic student preparedness. The SEM model framework (Figure A1), the latent variables and indicators are provided in Appendix A. 3 RESULTS The research aims to collect comprehensive data from over 20 engineering courses, encompassing introductory, core, elective, and capstone courses. Initially, a pilot study has been conducted on a civil engineering elective 4th year course focused on transportation. This course has five CLOs and four evaluations. Primary data show the results of the students’ survey, marks in their assessments and their alignment to EA competencies and SDGs, while secondary data provides the SDG and EA professional competencies mapping. 3.1 Primary Data After data cleaning, 23 valid samples were collected. Given the limited size of this initial dataset, it is not sufficient to construct and validate the complete SEM model. Therefore, the SEM model has been subdivided into manageable sub-models for preliminary analysis. Once enough data is gathered from additional courses, the complete SEM model will be constructed and validated to provide a comprehensive assessment of student competencies and learning outcomes. The following section presents an overview of the survey data, including participant demographics, assessment performance, course experiences, professional competencies, and engagement with the UN SDGs. Table 4 in Appendix A summarizes the key demographic characteristics. Most respondents were international postgraduate students enrolled full-time, aged between 20 and 25 years, and predominantly male. Students were evaluated through four assessment components: an online quiz (A1), a mid-term exam (A2), an assignment (A3), and a final exam (A4). Table 1 presents the distribution of grades across various percentage ranges and mean scores for each assessment. Table 1: Assessment Grade Distribution Assessment 85–100% 75–84% 60–74% 50–59% <50% Mean % A1 Quiz 60.87% 17.39% 13.04% 0% 0% 79.20 A2 Mid-Term 47.83% 30.43% 13.04% 0% 0% 77.50 A3 Assignment 73.91% 13.04% 4.35% 0% 0% 81.76 A4 Final 4.35% 39.13 17.39% 0% 17.39% 51.57 Students provided feedback on course experience, alignment of course content with CLOs, professional competencies, and engagement with the UN SDGs. Table 2 presents grouped results for clearer interpretation. Students reported positive experiences with the course, with over 80% expressing satisfaction and alignment with CLOs. The highest proficiency was observed in knowledge-based competencies (PE1), while engineering application (PE2) emerged as the most challenging area for more than half the respondents. Regarding sustainability education, familiarity and perceived relevance of the SDGs were moderate overall, though a significant proportion (47.8%) found the course impactful in improving their SDG understanding. Students most commonly felt connected to SDG11 (Sustainable Cities) and SDG9 (Industry and Infrastructure), and prioritized infrastructure and environmental themes in their professional field. Table 2: Summary of Course Experience, CLO Alignment, Professional Competencies, and SDG Engagement Dimension Category Percent. Dimension Category Percent. Course Experience High (4–5) 82.61% SDG Familiarity High (4-5) 26.09% Moderate (3) 17.39% Moderate (3) 34.78% Low (1–2) 0% Low (1–2) 30.43% CLO Alignment High (4–5) 82.61% SDG Impact on Understanding High (4–5) 47.83% Moderate (3) 13.04% Moderate (3) 34.78% Low (1–2) 4.35% Low (1–2) 8.70% PE1 – Knowledge Base High (4–5) 69.57% SDG Relevance to Transport Engineering High (4–5) 60.87% Moderate (3) 26.09% Moderate (3) 21.74% Low (1–2) 4.35% Low (1–2) 8.70% PE2 – Engineering Application High (4–5) 52.17% Connectednes s to SDGs SDG11 65.22% Moderate (3) 30.43% SDG9 52.17% Low (1–2) 17.39% SDG3, SDG4, SDG8 ~34.78% PE3 – Professional Accountabilit y High (4–5) 56.53% SDG Themes Prioritized Infrastructure 34.78% Moderate (3) 34.78% Environment 17.39% Low (1–2) 8.70% Others (Tech, Policy, Equity) <10% each Most Challenging Competency PE1 17.39% PE2 56.52% PE3 21.74% 3.2 Secondary Data Secondary data analysis is based on existing assessment records (quizzes, midterms, assignments, and final exams) mapped to CLOs, EA PE competencies, and SDGs. Assessment-CLO, PE-CLO, and SDG-CLO mappings are presented in Figure A2 (a), (b), and (c). Average assessment scores and their distribution are presented in Figure A3 (a) and (b), while the average CLO achievement and distribution are illustrated in Figure A3 (c) and (d). Both Figure A2 and Figure A3 are presented in Appendix A. Student performance analysis shows stronger achievement in CLO1, CLO2, and CLO3, associated with foundational knowledge and technical understanding. In contrast, student achievement in CLO4 and CLO5, focusing on applied, integrative, and reflective skills, was lower. This disparity suggests that the current assessment structure emphasizes knowledge-based learning outcomes. To address this imbalance, curriculum designers could adjust assessment weighting and incorporate opportunities that specifically target the development of higher-order thinking, problem-solving, and application skills aligned with CLO4 and CLO5. Against the EA PE competencies, student performance shows a strong alignment with the Knowledge and Skill Base category through CLO1 to CLO3. However, the Engineering Application Ability and Professional and Personal Attributes categories, involving practical application, ethics, communication, and teamwork, show weaker representation in assessment and performance. The alignment of CLOs with the SDGs, particularly SDG 9 (Industry, Innovation and Infrastructure) and SDG 11 (Sustainable Cities and Communities), provides insight into how student performance connects with global sustainability competencies. Data suggests stronger student performance in CLOs mapped to SDG 9, reflecting a solid understanding of infrastructure and innovation-related topics. However, outcomes linked to SDG 11 exhibit relatively weaker performance, potentially reflecting a need for more focused instruction on sustainability in urban contexts. To strengthen this, sustainability principles should be explicitly embedded within assessments and learning activities, through interdisciplinary projects, urban case studies, or partnerships with local communities. While not all courses need to cover the full range of EA competencies or SDGs, especially foundational courses, this higherlevel elective course is particularly well positioned to engage students in more advanced learning. It provides an opportunity to reinforce competencies in engineering application and professional practice by embedding authentic tasks, interdisciplinary perspectives, and reflective components within the curriculum. 3.3 SEM Results The SEM model conceptualizes Overall Performance (OP) as a second‐order latent variable that is influenced by CLO Achievement (CLO_Achieve), EA Competency (EA_Comp), and SDG Awareness (SDG_Aware). The path from OP to CLO_Achieve was fixed at 1, while paths from OP to EA_Comp (Estimate = 0.02, p < 0.001, ***) and to SDG_Aware (Estimate = 0.061, p < 0.001, ***) were statistically significant, indicating that higher overall performance is associated with improved professional competency and greater sustainability awareness. The measurement model further supports the construct validity of CLO_Achieve, as indicated by robust and significant loadings for CLO indicators (e.g., CLO2: 0.865, CLO3: 1.000, CLO4: 0.624, CLO5: 0.491, all with p < 0.001) and the alignment measure (CLO_Con: 0.019, p < 0.001). Similarly, within the EA competency, the significant paths for PE2 (Estimate = 1.364, p < 0.001, ***) and PE3 (Estimate = 0.956, p < 0.001, ***) indicate that specific professional competencies are well captured by the model. For the SDG, the significant relationships between SDG_Aware and both SDG_Imp (Estimate = 1.010, p < 0.001, ***) and SDG_Rel (Estimate = 1.075, p < 0.001, ***) further validate the model, suggesting that increased SDG awareness translates into greater perceived importance and relevance of sustainable development. Results provide preliminary evidence that the model’s structure is sound, with meaningful and significant paths linking educational outcomes, professional competencies, and sustainability awareness, even though further data collection is needed to fully develop and validate the complete SEM model. This is presented in Table 3. Table 3: SEM - Significance of each variable Path Estimate Significance Path Estimate Significance CLO_Achieve ~ OP 1 CLO_Achieve ~~ CLO_Achieve 114.129 * EA_Comp ~ OP 0.02 *** EA_Comp ~~ EA_Comp 0.308 *** SDG_Aware ~ OP 0.061 *** OP ~~ OP 329.214 *** CLO1 ~ CLO_Achieve 1 SDG_Aware ~~ SDG_Aware 0.009 CLO2 ~ CLO_Achieve 0.865 *** CLO1 ~~ CLO1 4.5 ** CLO3 ~ CLO_Achieve 1 *** CLO2 ~~ CLO2 13.125 *** CLO4 ~ CLO_Achieve 0.624 *** CLO3 ~~ CLO3 4.499 ** CLO5 ~ CLO_Achieve 0.491 *** CLO4 ~~ CLO4 22.419 *** CLO_Con ~ CLO_Achieve 0.019 *** CLO5 ~~ CLO5 7.192 *** PE1_Comp ~ EA_Comp 1 CLO_Con ~~ CLO_Con 0.455 *** PE2_Comp ~ EA_Comp 1.364 *** PE1_Comp ~~ PE1_Comp 0.122 * PE3_Comp ~ EA_Comp 0.956 *** PE2_Comp ~~ PE2_Comp 0.448 *** SDG_Fam ~ SDG_Aware 1 PE3_Comp ~~ PE3_Comp 0.357 *** SDG_Imp ~ SDG_Aware 1.01 *** SDG_Fam ~~ SDG_Fam 0.676 *** SDG_Rel ~ SDG_Aware 1.075 *** SDG_Imp ~~ SDG_Imp 0.119 * SDG_Rel ~~ SDG_Rel 0.211 *** The SEM model shows moderate overall fit, with acceptable values for CFI (0.927) and TLI (0.905) indicating a reasonably well-fitting structure. While the chi-square statistic (χ² = 82.68, p = 0.0033) is statistically significant, this remains suboptimal in small-samples where chi-square is known to be highly sensitive. The elevated RMSEA (0.168) and slightly suboptimal GFI (0.834), AGFI (0.785), and NFI (0.834) suggest potential model refinement. These fit challenges are likely influenced by the limited sample size in the pilot study, which can destabilize parameter estimates and inflate certain indices like RMSEA. As this preliminary analysis was based on a decomposed version of the full SEM model, designed to accommodate the current data constraints, these results serve as an exploratory step. With more comprehensive data from future course iterations, the full SEM model can be validated and optimized, likely resolving its current fit limitations and enhancing its overall explanatory power. 4 DISCUSSION AND CONCLUSIONS The study developed and validated a preliminary SEM to assess engineering students’ competencies by integrating CLOs, Engineers Australia EA PE competencies, and SDGs. Results from the pilot data indicate strong, statistically significant paths between OP and both EA competency and SDG Awareness, confirming the model’s conceptual soundness. The model showed internal consistency, with key CLOs (especially CLO2 and CLO3), professional competencies (PE2 and PE3), and SDG dimensions (Importance and Relevance) all significantly contributing to their respective latent variables. Assessment-competency alignment shows that student learning, professional readiness, and sustainability engagement can be assessed together. These findings show that the integrated SEM model provides a valuable framework to holistically evaluate student preparedness, bridging the academic-industry competency gap. Future work will expand the model using a larger and more diverse dataset, encompassing a wider range of engineering courses across different academic levels and specializations. The full model incorporates demographic influences and includes CLO Achievement, EA competency, and SDG Awareness as interrelated latent constructs feeding into OP. Once validated with a larger sample, this model can serve as a strategic tool for curriculum evaluation and enhancement. Its integrated structure offers insights into how assessment practices align with intended learning outcomes and broader professional expectations. This enables curriculum developers, educators, and accreditation bodies to better identify strengths, address competency gaps, and ensure alignment with national and global standards in engineering education. Ultimately, this approach supports a more holistic and evidence-based pathway for preparing future engineers to meet complex and evolving societal needs. 5 ACKNOWLEDGEMENTS This research project has been funded by the EFFECT (Education Focused Career Support) grant entitled “Development of a Comprehensive Framework to Align Students’ Performance with PE Competencies and SDGs in Engineering Education”. REFERENCES Adams, T., Jameel, S. M., & Goggins, J. (2023). Education for sustainable development: mapping the SDGs to university curricula. Sustainability, 15(10), 8340. doi:https://doi.org/10.3390/su15108340