Understanding the Use of Generative AI in the Engineering Design Process: A TAM-Based Comparative Analysis Across Academic Levels

Research Paper Recommended citation: Deng, Y., Delaney, E., & Liu, W. (2025). Understanding the Use of Generative AI in the Engineering Design Process: A TAM-Based Comparative Analysis Across Academic Levels. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17632064. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. UNDERSTANDING THE USE OF GENERATIVE AI IN THE ENGINEERING DESIGN PROCESS: A TAM-BASED COMPARATIVE ANALYSIS ACROSS ACADEMIC LEVELS Y. Deng a, 1 , E. Delaney a, W. Liu a, a King's College London, United Kingdom Conference Key Areas: Digital tools and AI in engineering education, Improving higher engineering education through researching engineering education Keywords: Engineering education, GenAI, Design Thinking, Technology Acceptance Model ABSTRACT The emergence of generative artificial intelligence (GenAI) tools has introduced new opportunities and challenges for engineering design education. These technologies offer significant potential to enhance creativity, accelerate iteration, and expand the exploration of design solutions. However, understanding how engineering students at different academic levels perceive and utilize GenAI tools during structured conceptual design processes requires further investigation. A mixed-methods approach was adopted, combining survey data (n = 30) with a structured design challenge involving 16 students across undergraduate (UG), postgraduate taught (PGT), and doctoral programmes. The Technology Acceptance Model (TAM) provides the theoretical framework, focusing on three core dimensions: perceived usefulness (PU), perceived ease of use (PEOU), and application intention or attitude (ATT). These dimensions were adapted to the context of engineering design, reflecting the cognitive and technical demands of ideation, prototyping, and evaluation. Thematic analysis of interview data revealed both common and divergent patterns in students’ experiences. Across all levels, students valued GenAI tools for early-stage ideation and creative support. However, key differences emerged by academic level. UG and PGT students perceived GenAI as highly accessible and uniformly useful across the design process yet often demonstrated surface-level engagement with limited output verification. In contrast, doctoral students reported lower ease of use, citing the cognitive demands of effective prompting, output 1 Corresponding Author Y. Deng [email protected] adaptation, and verification. They also distinguished more clearly between GenAI’s high utility in divergent phases and its limited relevance in convergent stages requiring technical precision. 1 INTRODUCTION The rapid development of generative artificial intelligence (GenAI) tools such as ChatGPT, DeepSeek, and Midjourney presents new opportunities and challenges for design practice and education. While these tools offer unprecedented capabilities for ideation, visualisation, and problem-solving, they also require users to develop new competencies and adaptation strategies. Understanding how engineering students perceive and use these technologies is essential for developing effective educational approaches. A preliminary survey revealed a significant gap: while engineering students frequently use GenAI to assist with homework and general knowledge queries, they rarely apply these tools to design tasks, despite GenAI's growing capabilities and applications in design contexts. Daalhuizen and Schoormans (2018) found that engineering students frequently struggle with the ambiguity and openness of design processes, preferring structured problem-solving approaches. This situation raises critical questions about how students with different academic backgrounds understand and implement GenAI tools in design tasks. This situation raises a central question: how do students with different academic backgrounds perceive and implement GenAI tools in conceptual design tasks? Specifically, this study is guided by the following research question: How do engineering students at different academic levels (undergraduate, postgraduate taught, and doctoral) perceive and utilize GenAI tools during conceptual design processes? Why do their perceptions of the ease of use, usefulness, and intention to apply GenAI tools differ during design tasks, as analysed through the Technology Acceptance Model (TAM)? To address these questions, this study adopts the Technology Acceptance Model (TAM) as a theoretical framework and investigates students’ perceptions of GenAI’s ease of use, usefulness, and their attitudes toward its integration in design processes. 2 CONTEXT AND PRACTICAL WORK 2.1 GenAI in Engineering Design Education For the purposes of this study, GenAI is defined as computational systems that can create novel content (text, images, code, audio, etc.) by learning patterns from large datasets and then producing new outputs that reflect these patterns while introducing novel variations. These GenAI systems employ complex neural network architectures, such as transformers and diffusion models, enabling them to generate content that often closely resembles human-created work. Studies suggest that AI tools can enhance creativity(Hwang, 2022), reduce cognitive load during ideation tasks(Gandhi et al., 2023), and enable students to explore solution spaces more effectively(Chen et al., 2023). However, concerns have been raised about overreliance on AI-generated solutions(Zhai et al., 2024) and potential skill atrophy among students who heavily depend on these tools (Morandini et al., 2023). While engineering students frequently use GenAI to assist with homework and general knowledge queries, they rarely apply these tools to design tasks, despite the growing capabilities and applications of AI in design environments. Additionally, the survey highlights that engineering students often demonstrate deficiencies in design skills compared to other disciplines, a finding supported by previous research (Morandini et al., 2023). This suggests a missed opportunity to leverage AI to enhance design learning experiences in engineering education. It is important to note that in UK engineering education, there are significant structural differences between BSc, MSc, and PhD programs, which may influence how students at different academic levels interact with GenAI. Despite these differences in overall program structure, AI education is predominantly delivered through general elective courses that are common across academic levels. These courses typically focus on technical proficiency and understanding of GenAI technologies, rather than offering specialized applications for different degree levels. This approach creates an interesting context for examining how students with varying levels of disciplinary expertise engage with similar AI training but demonstrate different implementation strategies in design tasks. 2.2 Technology Acceptance Model (TAM) in Educational Contexts Lorem The Technology Acceptance Model, originally developed by Davis (1989), has been widely used to understand how users adopt new technologies. Perceived usefulness is defined as "the degree to which a person believes that using a particular system would enhance his or her job performance," while perceived ease of use refers to "the degree to which a person believes that using a particular system would be free of effort" (Davis, 1989). Subsequent extensions of TAM have incorporated additional factors relevant to educational contexts, including emotional responses, social influences, and self-efficacy (Scherer et al., 2020). Specifically, in engineering education, research has shown that perceived relevance to future career goals significantly influences technology adoption(Willis et al., 2013). Studies applying TAM to AI adoption in educational settings have found that perceived usefulness typically outweighs ease of use in determining adoption intentions (Or, 2025). However, these studies have primarily focused on general educational applications rather than design-specific contexts, leaving a gap in understanding how TAM dimensions manifest in engineering design education. 3 METHODOLOGY 3.1 Research Design This study employed a mixed-methods approach combining quantitative and qualitative data collection techniques to understand engineering students' perceptions and use of GenAI in conceptual design processes. The research was conducted in two phases: a preliminary survey (n=30) to understand current AI usage patterns and design challenges among engineering students, and a design challenge study (n=16) with equal representation across academic levels (8 UG or PGT students, 8 doctoral students). This multi-case study approach allowed for exploration of perspectives within each group and enabled comparative analysis of attitudes and behaviours across different groups (Gustafsson, 2017). The design challenge required participants to design an assistive tool for elderly individuals with hand arthritis, completing the entire Double Diamond process from problem discovery to prototype delivery. All participants had access to a suite of GenAI tools and were free to use these tools throughout the design process, along with traditional free-form search methods. To control for the rapid evolution of GenAI technologies, all experiments were conducted within a condensed timeframe in January 2025, divided into three sessions. This approach ensured that all participants had access to the same generation of GenAI tools with identical capabilities, preventing technological advancements from becoming a confounding variable. The challenge lasted 3 hours, designed to ensure participants engaged with all phases of the design process, the study flowchart is shown in Fig.1. Fig. 1. Experiment Flowchart 3.2 Data Collection and Analysis Data were collected through mixed methods including a pre-survey gathering demographic information, design experience, prior AI usage patterns, and design thinking abilities; a custom post-design challenge questionnaire measuring TAM dimensions, AI interaction patterns, and verification strategies; semi-structured interviews based on TAM constructs; and focus group discussions organized separately for each academic level. The semi-structured interviews included questions such as "Please describe your experience using GenAI for engineering design," "In what ways do you think GenAI helps your design work?", and "What strategies did you adopt to improve efficiency in using AI?". Qualitative data analysis followed a deductive approach, with three preset perception categories aligned with our theoretical framework: Perceived Usefulness (PU), Perceived Ease of Use (PEOU), and Application Intention/Attitude (ATT). Using NVivo 14, two researchers independently performed initial coding to identify preliminary themes and patterns, followed by thematic coding based on the three theoretical dimensions while remaining open to emergent themes. Two coders separately coded 20% of the data, calculating Cohen's Kappa coefficient (K=0.87) to ensure coding reliability and consistency. We compared themes across cases, first analysing data for each academic level group separately to determine within-group patterns, then conducting cross-group comparisons to identify similarities and differences across academic levels. 4 RESULTS AND INSIGHTS 4.1 GenAI Tasks and Applications Data collected from 30 engineering students revealed varied engagement with GenAI across both general capability areas and academic applications. The most frequently reported use was tutoring, which included concept explanations and question answering. This was followed by content generation, reported by 36.7% of students, and research assistance, cited by 23.3%. Creative tasks such as design ideation were reported by 20.0%, while assessment-related applications, including practice tests and feedback, were less common at 16.7%. Additionally, 30.0% of students reported other uses, indicating a growing range of informal or personalised interactions with GenAI beyond traditional academic contexts. In terms of academic-specific applications, GenAI was most used for generating content for assignments and projects, reported by 50.0% of respondents. This was followed by tutoring to support understanding of complex topics (46.7%) and obtaining personalised learning support (43.3%). Research assistance, including literature review and data processing, was reported by 33.3%, while design simulations were mentioned by only 13.3%. None of the students selected "other" for academic use, which suggests a more clearly defined role of GenAI in conventional educational tasks. These findings suggest that engineering students primarily use GenAI as a learning support tool rather than as a means of simulation or decision-making. The emphasis on tutoring and content generation is consistent with subsequent findings related to perceived ease of use and perceived usefulness, particularly in early-phase ideation and knowledge acquisition. At the same time, the relatively limited engagement with assessment and design simulation indicates a need for educational interventions to help students develop awareness and capability in these areas. 4.2 Perceptions of GenAI in Engineering Design Tasks The results of thematic analysis reveal both similarities and differences in how taught students and doctoral students perceive and interact with GenAI tools in engineering design contexts. These findings are organized according to the three key dimensions identified in the coding framework. Due to space limitations, detailed analysis and representative quotes for each code are not presented in this paper. The qualitative data analysis followed a systematic coding approach to identify patterns in students' perceptions across academic levels. Tables 1-3 present the coding frameworks developed for each dimension of analysis. Perceived Ease of Use (PEOU) Table 1. Coding Framework and Categories for Perceived Ease of Use (EU) Category Code Label Description Summary Interface Interaction EU1.1 Basic Accessibility Perceived ease of accessing and navigating GenAI tools Initial ease of access and responsiveness of GenAI tools EU1.2 Response Time Satisfaction Perceived responsiveness and speed of GenAI system Prompt Engineering EU2.1 Prompt Formulation Cognitive effort required to craft effective AI prompts Skill and effort required to craft and refine prompts EU2.2 Prompt Iteration Effort involved in refining prompts to improve AI outputs Output Management EU3.1 Output Evaluation Effort Perceived workload in assessing and verifying AI-generated content Effort needed to assess and adapt GenAI-generated outputs EU3.2 Output Adaptation Complexity Perceived difficulty in modifying and applying AI outputs to design needs Workflow Burden EU4.1 Cognitive load Cognitive load imposed by switching between multiple tools Impact of using multiple tools and switching between them EU4.2 Workflow Fragmentation Perception of AI disrupting the continuity and flow of the design process Table 2. Coding Framework and Categories for Perceived Usefulness (PU) Category Code Label Description Summary Design Thinking Expansion PU1.1 Creative Quantity Increase in number of design concepts generated with AI Extending idea generation and expanding solution space PU1.2 Creative Novelty Degree to which AI supports novel or unconventional idea generation PU1.3 Design Diversity Perceived expansion of the design solution space through AI Efficiency Enhancement PU2.1 Time Savings Reduction in time required to complete ideation or development tasks Time-saving and faster design iterations PU2.2 Iteration Acceleration Speed of revising and testing design alternatives with AI assistance Quality Impact Assessment PU3 Output Quality Perception Perceived improvement in the technical or aesthetic quality of final designs Beliefs about GenAI's influence on final design quality Technical Limitations PU4.1 Accuracy Limitations Concerns about factual correctness or validity of AI-generated content Concerns about precision and domain knowledge gaps PU4.2 Domain Expertise Gaps Perceived lack of engineering-specific or contextual knowledge in AI StageSpecific Utility PU5 Phase Relevance Understanding of AI’s differing value across design stages Understanding the varying usefulness across design stages Table 3: Coding Framework and Categories for Application Intention/Attitude (IA) Category Code Label Description Summary Skill Development IA1.1 Output Inconsistency Concerns regarding variability in AI output quality and relevance Impact of AI use on long-term design ability IA1.2 Core Skill Erosion Perceived risk of diminished foundational design skills due to AI reliance Ethical Considerations IA2.1 Privacy Concerns Apprehensions about data safety and transparency in AI systems Concerns over design authorship, transparency, and data use IA2.2 Design Authenticity Concern over loss of originality and authorship in AI-supported outputs Professional Impact IA3.1 Career Readiness Perception Beliefs about GenAI’s relevance to future professional roles in engineering Views on how GenAI supports future employability IA3.2 Anticipated Industry Adoption Expectations of GenAI integration in engineering and design industries 4.3 Key Differences Between Academic Levels Thematic analysis revealed several differences in how students at varying academic levels perceived and engaged with generative AI (GenAI) tools across the three dimensions of the Technology Acceptance Model (TAM). Perceived Ease of Use (PEOU) Compared to doctoral students, UG and PGT students consistently reported higher perceived ease of use. While this inverse relationship might seem counterintuitive, it reflects deeper engagement by more experienced students. Specifically, doctoral research participants recognized the complexities involved in leveraging GenAI for sophisticated design tasks such as technical specification, constraint assessment, and solution refinement. Their lower ease of use ratings stemmed not from interface difficulties but from awareness of the cognitive and procedural demands involved in critically integrating GenAI outputs. In contrast, UG and PGT students exhibited surface-level interactions. They tended to copy-paste design queries, rarely engaged in iterative prompting, and frequently accepted GenAI responses without rigorous verification. This overestimation of GenAI technical accuracy and underestimation of its limitations suggests inflated perceptions of ease of use, driven by interface fluency rather than deep understanding of the tool. These patterns were particularly evident in EU3.2 (Output Adaptation Complexity) and EU4.1 (Cognitive load), where doctoral students reported significantly more cognitive burden and verification workload than their less experienced peers. Perceived Usefulness (PU) While participants across all groups acknowledged the value of GenAI in various design tasks such as ideation and visual exploration, perceptions diverged for more complex stages of the design process. UG and PGT students generally held optimistic and unconditional views of GenAI's contribution to final design quality. Many directly attributed tangible improvements in their outputs to GenAI assistance, aligning with PU3 (Output Quality Perception) and PU2.1 (Time Savings). First, regarding PU3 (Output Quality Perception), UG and PGT students typically believed that AI directly improved the overall quality of their design outcomes. Their reflections indicated strong confidence in AI's value not just for ideation but for enhancing and elevating results. In contrast, doctoral students expressed a more conditional perspective, emphasizing that any quality improvements depended on the degree and appropriateness of AI integration. As one doctoral student explained, "AI can suggest interesting ideas, but I must use engineering knowledge to filter and reshape them". Doctoral students also showed differences in their interpretation of AI's role across double diamond design phases (PU5). Undergraduate participants generally considered AI equally useful across all tasks, with limited differentiation between divergent and convergent phases. This suggested nascent understanding of design process structure and stage-specific tool fit. In contrast, doctoral students demonstrated stronger phase distinction, recognizing AI's high utility in divergent phases (e.g., brainstorming, visual generation) but explicitly noting its limitations in convergent phases. One doctoral student emphasized: "I stop using AI at the refinement stage, by then, engineering judgment and feasibility dominate". Application Intention and Attitude (ATT) While intentions to use GenAI in future work were similar overall, differences emerged in the depth and framing of these intentions. UG and PGT students exhibited broad enthusiasm and willingness to adopt GenAI across a wide range of design tasks. Their reflections emphasized its potential to enhance productivity, creativity, and career readiness. Although ethical considerations and design authenticity concerns (IA2.2) were occasionally mentioned, they were largely secondary to perceived functional advantages. In contrast, doctoral students expressed a more selective and strategic approach. They emphasized responsible integration, preservation of authorship, and concerns about skill atrophy (IA1.2). Many viewed GenAI as a valuable but fallible collaborator requiring rigorous verification and clear application boundaries. Concerns about inconsistency (IA1.1), over-reliance, and erosion of core design capabilities were particularly prominent. One doctoral student shared: "I worry that dependence on AI might weaken our design reasoning abilities over time". Privacy and institutional transparency (IA2.1) were considered relevant factors across all levels but not primary determinants of adoption decisions. Most students desired clear responsible use policy guidance from educational institutions but were unlikely to self-regulate in the absence of institutional frameworks. As one PGT student explained, "We definitely need clear policies dictating AI application in education, but I wouldn't stop using it solely due to privacy concerns". This indicates that students expect institutional guidance but are unlikely to self-limit usage without it. 5 CONCLUSIONS AND IMPLICATIONS This study employed the TAM as an analytical framework to explore how engineering students at different academic levels utilize GenAI tools in conceptual design tasks. By integrating survey data with qualitative insights from structured design challenges, the research reveals both commonalities and significant differences in perceptions and usage patterns among undergraduate, taught postgraduate, and research postgraduate students. The research establishes a specialized TAM framework for understanding AI adoption in engineering design education. By examining how students across academic levels interact with GenAI tools during structured design challenges. This framework helps bridge the gap between general technology acceptance theories and the specific context of designfocused engineering education. Instructional design should articulate GenAI's variable utility throughout the design process. Educators should clearly distinguish between AI use in divergent tasks and its limitations in convergent tasks. Embedding this understanding within design curricula can help students strategically integrate GenAI tools without compromising necessary engineering judgment. Educational strategies should focus on students' academic maturity and disciplinary expertise. For undergraduates and taught postgraduates, emphasis should be placed on developing critical evaluation skills, fostering awareness of GenAI limitations, and guiding prompt iteration strategies. For doctoral students, curricula should support ethical reflection, authorship protection, and integration of AI tool methodologies into research-driven design practices. Ease of use in design environments should not be limited to interface navigation but must also consider the