Reimagining The MSc Dissertation as an Agile-Like Group Project: A Mixed-Methods Evaluation and Recommendations for Best Practice

Research Paper Recommended citation: Johnson, H., Fraser, H., Craddock, I., Brigden, A., Mccallum, C. H., Dawe, K., & Isotalus, H. K. (2025). Reimagining The MSc Dissertation as an Agile-Like Group Project: A Mixed-Methods Evaluation and Recommendations for Best Practice. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17632072. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. (4 spaces) REIMAGINING THE MSC DISSERTATION AS AN AGILE-LIKE GROUP PROJECT: A MIXED-METHODS EVALUATION AND RECOMMENDATIONS FOR BEST PRACTICE H Johnson a, 1 , H Fraser b, I Craddock c, A Brigden d, CH McCallum e, K Dawe f, HK Isotalus g (1 space) a Digital Health, School of Computer Science, Electrical and Electronic Engineering, University of Bristol, Bristol, United Kingdom, 0000-0002-2001-4544 b Digital Health, School of Computer Science, Electrical and Electronic Engineering, University of Bristol, Bristol, United Kingdom, 0000-0001-9752-0042 c Digital Health, School of Computer Science, Electrical and Electronic Engineering, University of Bristol, Bristol, United Kingdom, 0000-0001-6552-8541 d Digital Health, School of Computer Science, Electrical and Electronic Engineering, University of Bristol, Bristol, United Kingdom, 0000-0002-7958-7881 e Digital Health, School of Computer Science, Electrical and Electronic Engineering, University of Bristol, Bristol, United Kingdom f Digital Health, School of Computer Science, Electrical and Electronic Engineering, University of Bristol, Bristol, United Kingdom, 0000-0003-2173-9851 g Digital Health, School of Computer Science, Electrical and Electronic Engineering, University of Bristol, Bristol, United Kingdom, 0000-0002-3393-9263 (2 spaces) Conference Key Areas: Curriculum development and emerging curriculum models in engineering; Dialogue between engineering and society – effects on education Keywords: group-based dissertation, employability, teamwork, digital health, mixed methods ABSTRACT Traditional MSc dissertations, while academically rigorous, often fail to develop the collaborative skills increasingly demanded by employers. This paper presents a novel alternative: a group-based agile-like project model implemented within a Digital Health MSc programme at a university. Through a mixed-methods evaluation involving quantitative surveys (n=6) and qualitative focus groups with students (n=6), academics (n=3), and industry partners (n=3), we assessed the effectiveness of this innovative approach. Our findings demonstrate consistent improvements in students' self-reported knowledge and confidence across six project phases, with quantitative data showing +0.9 point Likert improvement for knowledge and qualitative data 1 Corresponding Author H Johnson [email protected] revealing four key themes: managing uncertainty in innovation processes, balancing multidisciplinary learning with specialisation, navigating team dynamics whilst maintaining individual accountability, and optimising workload distribution. Based on these findings, we propose evidence-based recommendations for implementing similar models in engineering education, addressing the inherent tensions between authentic workplace preparation and academic assessment requirements. This approach offers a promising pathway to enhance graduate employability while simultaneously addressing faculty workload challenges in an increasingly resourceconstrained higher education environment. 1 INTRODUCTION Employers recognise the benefits of students developing specialist knowledge. However, they also desire graduates with practical experience and skills (Bhagra & Sharma, 2018). The traditional dissertation project, supervised by an academic, typically focuses on a narrow research question, promoting independent study and a focus on personal achievement (Mamas, 2018). This does not require the multidisciplinary and team-based approach to problem-solving, completing tasks, and decision-making that is necessary within Digital Health workplaces. To foster these skills and improve employability, we piloted an innovative student project. The structure of the project was designed by faculty staff in consultation with stakeholders from the UK National Health Service (NHS) and industry and modelled on “Agile methodology”. Agile is a real-world project management process used within the technology industry (Salza et al., 2019). Universities are highly motivated to meet the employability needs of students (Maxwell et al., 2010). There is currently a significant gap between educational practice and workplace requirements. Employers should therefore be recognised as stakeholders in course design and be involved in designing and delivering course content (Dickinson et al., 2015; Maxwell et al., 2010). Further, academics face increasing workload pressures (Pace et al., 2021). The traditional MSc project model generates a substantial workload, with academics acting as designated supervisors for a number of students, providing support throughout the dissertation period, then marking each. Although the relationship between workload and performance among academic staff is not clear (Yousefi & Abdullah, 2019), it is conceivable that such factors may contribute to overall programme delivery, while negatively impacting staff retention and career development (Godfrey et al., 2010). It should be noted that many MSc programmes already develop teamwork and employability skills through alternative formats, such as industry‐sponsored projects, multi-disciplinary capstone courses, and long-term internships that similarly mirror workplace practice (Nilsson, 2010; O'Leary, 2015). In close collaboration with an industry advisory board, the authors have developed an Agile-like group-based project (the “Agile MSc Project”) to replace the traditional individual dissertation project for this degree programme. Agile methodology involves breaking projects down into phases and emphasises continuous collaboration and improvement. It originated in software development; however it is increasingly being used for project management in general, particularly in areas of innovation and uncertainty. Agile techniques have been used in the development of non-software products such as computers, medical devices, food, clothing, and music (Rico & Sayani, 2009). The sprint model contrasts with traditional dissertation approaches that might involve months of continuous work toward a single deliverable. Instead, it breaks the project into manageable chunks with regular feedback points, allowing teams to adapt their approach based on what they learn in each phase, much like professionals would in industry settings (Srivastava et al., 2017).Collaborative skills also play a key role in academic careers. Academics are increasingly expected to collaborate widely; across their department, with other Universities, and with external partners. If found to be effective, we believe this innovative model could be adopted by other Engineering MScs beyond Digital Health. To this end, to complete the project, students worked in groups on a real-life health and innovation project, receiving both academic support and consultation sessions from industry stakeholders. In this paper, we report the evaluation of this new model. We explored stakeholders' views (students, academics and industry partners), to understand their experiences and perceptions of the project. 2 METHODOLOGY 2.1 Group Project Model In the Agile MSc Project students work through a mock product development cycle (over 4 months) in groups of 5-6. All project teams from the 2020–21 cohort (each n=5–6) were invited to complete our surveys; the quantitative arm drew responses from 6 individuals. The project forms one third of the 1-year postgraduate degree (60 UK credit points). The project is broken down into six phases: • Phase 1: Market research • Phase 2: Patient and public involvement • Phase 3: Quantitative analysis • Phase 4: Clinical evaluation design • Phase 5: Regulatory submission • Phase 6: Post-market surveillance Groups have 2-4 weeks to complete each phase. This mimics industry practices by giving student teams a concentrated period (2-4 weeks) to complete each phase. This structure helps students develop time management skills, learn to prioritize tasks, and experience the iterative nature of real-world project development. Students attend workshops and are given contact hours with consultants who are academics and industry professionals with expertise related to that particular phase. These meetings typically last 30-60 minutes per group and require little preparation from the consultants, reflecting real-world consulting practices while reducing faculty workload. During the project, groups have weekly meetings with teaching assistants (TAs) who assess group engagement and monitor progression. Student groups submit written reports at the end of each phase. The group's project dissertation mark combines these reports, with individual marks weighted according to engagement, contributions, and final oral presentation. 2.2 Evaluation Study Design We conducted a mixed-methods involving: (1) quantitative surveys administered to students, and (2) qualitative focus groups with students, academics and industry partners. We recognise the modest sample size (n=6 for surveys) and frame our findings as exploratory, generating hypotheses for larger-scale follow-up studies. We recruited three stakeholder groups: • Students enrolled on the Digital Health MSc (2020-2021) • Academics who delivered teaching or acted in a consultancy capacity for the MSc project • Industry partners from the Digital Health Industry Advisory Group Students completed surveys at the start and end of each phase (12 surveys total), rating their self-reported knowledge and confidence on a 5-point Likert scale, plus an end-of-project survey evaluating their overall experience. For the qualitative arm, we conducted focus groups with students, academics, and industry partners using semistructured topic guides. Focus group facilitators were not involved in project assessment to encourage candid responses. Focus groups were recorded, transcribed, pseudonymised, and analysed thematically, adhering to Braun and Clarke’s thematic analyse principles (Clarke & Braun, 2017). All participants provided informed consent with ethical approval from the Faculty of Engineering Research Ethics Committee (Ref = 8363). 3 RESULTS 3.1 Quantitative Findings Responses from students (N=6) showed a general pattern of improvement in knowledge and confidence across phases (Table 1, Fig. 1). Most participants reported increased or maintained knowledge and confidence, with some exceptions in Phases 1 and 3 where a few students reported slight decreases in knowledge. Table 1: Change in pre-to-post phase self-report on knowledge and confidence. All means reported in Figure 1 are unweighted averages of 1–5 Likert responses. ‘Prephase’ and ‘Post-phase’ averages were computed per student for each phase, then averaged across respondents (excluding missing data) Phase Change in "Knowledge" Change in "Confidence" Higher (n) No change (n) Lower (n) Higher (n) No change (n) Lower (n) Phase 1: Market Research 5 0 1 5 1 0 Phase 2: Patient and Public Involvement 4 2 0 4 2 0 Phase 3: Quantitative analysis 1 3 2 3 3 0 Phase 4: Clinical Evaluation Design 2 4 0 5 1 0 Phase 5*: Regulatory Submission 5 0 0 3 2 0 Phase 6**: Post-market surveillance 3 1 0 4 0 0 *One missing data point; **Two missing data points Fig. 1: Unweighted mean Likert scores (1–5) for student self-rated knowledge and confidence before (‘Pre-phase’) and after (‘Post-phase’) each phase. Means were calculated per student then averaged across all survey respondents (n=6); missing data points omitted End-of-project survey results (n=5) revealed high satisfaction with the project. All students agreed or strongly agreed they were satisfied with project quality, enjoyed the experience, found it engaging, understood expectations, and believed it enhanced their employability. Four students agreed the project was useful for their training as Digital Health professionals and encouraged efficiency with industry partners. Employability: Each student either agreed or strongly agreed that (i) the project was useful for their professional training, that (ii) the project equipped them with skills / knowledge, and that (iii) having done the project will improve their employability. Regarding group dynamics, all students felt their skillset was valued by team members, but perceptions of work quality varied considerably. Three students disagreed that work quality was equal across members, and opinions were divided on whether groups shared a common understanding of quality standards. Four students preferred the group project over an individual dissertation and would recommend it to others, while two disagreed they could have completed the project independently. Comparison of dissertation types: Four students reported that (i) compared to working with an individual project with a supervisor, they preferred the group project, with one student disagreeing. Similarly, while four students agreed or strongly agreed that (ii) they would recommend an MSc with a group-project model to their friends, one student disagreed with the statement. The peer rating data revealed distinct team development patterns, with one group achieving consistent ratings early while the other experienced more variable perceptions until later phases, suggesting teamwork skills develop at different rates across groups. 3.2 Qualitative Findings Our thematic analysis consisted of two focus groups with students (n=3 per focus group, total n=6), one academic focus group (n=3), and one industry focus group (n=3). We identified four major themes with associated subthemes (Table 2). The findings indicate that the Agile MSc Project paradigm has both benefits and drawbacks from the perspectives of all stakeholder groups. Table 2: Thematic analysis of qualitative data (N=6) Theme Subtheme Description Example quote Uncertainty and fuzziness in innovation Desire for structure Students sought clarity on how phases interconnected and how decisions in one phase affected subsequent work "Each phase was like a standalone thing, so you sort of drop your idea and start again." (Student) Authentic workplace ambiguity Industry partners emphasised that uncertainty is inherent in real-world product development "The reality of how these projects go in real life is that they're often very fuzzy." (Industry) Guidance for consultants Academics sought better guidance on their consulting role "I wasn't exactly sure what I was going to be asked... a suggestion might be to write guidance for consultants." (Academic) Multidisciplinary working vs specialisation Jack of all trades concerns Industry expressed concern about graduates becoming generalists without sufficient depth "The danger could be... is this person like a jack of all trades." (Industry) Value of diverse expertise Students and industry recognised benefits of diverse backgrounds in team composition "It really pulled on each other's expertise and background, so that was really valuable." (Student) Complementary skillsets Industry values specialists who complement existing team skills "Companies have been trying to bring in somebody with psychology background... to complement the existing team." (Industry) Teamwork dynamics Individual contribution and ownership Students desired individual components to demonstrate personal accountability "I would have liked some more individual aspects as well as a combination." (Student) Measuring contribution Challenges in assessing individual contributions within group work "It was a little bit tricky when someone contributed less to the group work, or the quality of their work wouldn't match others." (Student) Peer assessment challenges Difficulty providing honest feedback in group settings "If we're all on a group call talking about it in front of each other, people aren't going to be honest." (Student) Theme Subtheme Description Example quote Evolution of teamwork skills Improvement in group working efficiency over time "By the end we had 10-minute meetings, sorting it out really quickly, and were really organised." (Student) Workload management Staff workload reduction Group projects reduce supervision bottlenecks for academics "That's why we're doing this approach, because the bottleneck with increasing MSc numbers is always supervisors." (Academic) Phase workload variability Students experienced unequal workloads across different phases "The regulations phase, the workload was massive... but for the previous one, I didn't have to do it consistently." (Student) New content challenges Phases introducing unfamiliar content created additional workload "We had hardly any time on that phase, and it was completely new content." (Student) 4 DISCUSSION The Agile MSc Project involved students working in multi-disciplinary teams through an accelerated process of product design, evaluation, and implementation, closely mimicking real-world scenarios within an academic timeframe. The findings indicate that the programme model was beneficial to both student and academic stakeholders (Neumann & Baumann, 2021). Reported benefits for students included enhanced collaborative learning, evolution of teamworking skills over time, and enjoyment of team working. Survey results also indicated perceived employability enhancement. However, drawbacks included challenges with ensuring equal work distribution, differences in contribution quality, and decreased sense of individual ownership (Marra et al., 2016). From the academic perspective, benefits included reduced supervisory bottlenecks and more efficient communication with student cohorts, improving workload management for project directors. A key tension emerged between students' desire for clarity and structure versus the inherent uncertainty of real-world innovation processes. While students sought clear connections between phases, industry partners emphasised that ambiguity is characteristic of actual workplace projects. This suggests that managing uncertainty is itself a valuable skill for graduates, though it requires careful scaffolding within the educational context. Each project phase is underpinned by earlier taught content, spanning research methods, quantitative analysis, clinical evaluation, and regulatory frameworks, so that students draw directly on and reinforce their existing knowledge as they progress through each stage. Another tension concerned individual accountability versus collaborative achievement. Students valued collaborative learning but expressed concerns about individual ownership and fair recognition of effort. This reflects higher education's traditional focus on individual achievement contrasted with industry's emphasis on teamwork (Mamas, 2018). 5 RECOMMENDATIONS FOR GOOD PRACTICE Drawing on our findings, we propose five recommendations for implementing Agilelike dissertation projects: 5.1 Balance Structure with Authentic Uncertainty Students should have opportunities to ask logistical questions while understanding that some uncertainty is intentional and reflects the realities of innovation. Q+A sessions at the beginning of each phase can address procedural concerns while maintaining the authentic "fuzziness" that develops problem-solving capabilities. Frame uncertainty explicitly as a learning opportunity that develops valuable professional skills. 5.2 Incorporate Individual Assessment Components Include individually authored sections within group reports, such as discussion sections or abstracts to provide students with intellectual ownership while maintaining collaborative benefits. This addresses students' desire for personal accountability while developing both independent and team-based skills. Consider confidential individual contribution assessments (Cheng & Warren, 2000) rather than group-based evaluations to encourage honest feedback. 5.3 Collaborate with Industry Partners Involve industry stakeholders in identifying valued specialist skills and co-creating project content. This ensures relevance to employer needs while giving students direct exposure to industry perspectives. Field-specific industry involvement is particularly important for multidisciplinary programs like Digital Health (El-Maaddawy et al., 2018; Teixeira et al., 2020). 5.4 Support Effective Teamwork Provide preparatory training on communication, conflict resolution, and team working practices (Ercan & Khan, 2017). Encourage teams to establish clear boundaries such as not arranging meetings on weekends and communication protocols. Recognise that teamwork skills develop over time, as evidenced by our peer rating consistency data (available on request), and provide appropriate scaffolding. 5.5 Communicate Expectations Clearly Ensure program marketing and documentation clearly describe the group-based dissertation approach so applicants understand what will be expected. While employers value teamwork skills, some students may prefer traditional dissertations, so transparency is essential. Create comprehensive project guides detailing phase connections and progression. 6 CONCLUSION Our evaluation demonstrates that reimagining the MSc dissertation as a groupbased agile-like project offers significant benefits while presenting distinct challenges that require careful consideration. The approach successfully develops teamwork, communication, and adaptability skills valued by employers, while also addressing faculty workload challenges. The key themes identified in our qualitative analysis, uncertainty management, multidisciplinary versus specialised learning, team dynamics, and workload considerations, highlight the tensions inherent in bridging academic assessment with authentic workplace focused preparation. By implementing our recommended best practices, multidisciplinary engineering