Bridging the Gap: Implementing Doctoral Study Process in a Postgraduate University Course

Practice Paper Recommended citation: Li, C., & Eskelinen, H. (2025). Bridging the Gap: Implementing Doctoral Study Process in a Postgraduate University Course. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631758. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. Bridging the Gap: Implementing Doctoral Study Process in a Postgraduate University Course C Li a, 1 , H Eskelinen b a Mechanical Engineering, School of Energy Systems, LUT University, Lappeenranta, Finland, 0000-0001-9400-3998 b Mechanical Engineering, School of Energy Systems, LUT University, Lappeenranta, Finland Conference Key Areas: Engineering skills, professional skills, and transversal skills; Improving higher engineering education through researching engineering education Keywords: Postgraduate study, Doctoral study, Peer-review assessment ABSTRACT Doctoral researchers play an important role in both educational institutions and industry. As a doctoral researcher, it is expected that the original research addresses existing knowledge gaps and introduces novel ideas. In an academic context, doctoral researchers can also work as educators, teaching students. In an industrial context, doctoral researchers can transfer knowledge into commercial output. However, many doctoral candidates spend a considerable amount of time attempting to understand their role at the beginning. This is because, during their master’s studies, they may not have had the opportunity to understand the specific responsibilities and requirements of a doctoral candidate. Consequently, they may lose interest or become scared of pursuing a doctoral degree, particularly if they are facing financial challenges. Therefore, in this paper, a way of implementing the doctoral study process into a postgraduate university course is introduced. This approach incorporates peer-review publication, final dissertation writing, and the final doctoral examination into the course, which enables students to gain a deeper understanding of the life of a doctoral candidate. 1 Corresponding Author C Li [email protected] 1 INTRODUCTION In Europe, full-time doctoral programmes typically last between three and four years (Bitusikova, 2009). However, in most countries, the time to degree (TTD) tends to be longer. Furthermore, only approximately half of doctoral candidates complete their education (Taylor et al., 2017). The survey reported by the European University Association (EUA) provides an overview of the current landscape of doctoral education in Europe, involving more than 250 universities from 36 countries (Hasgall et al., 2019), it tells the current chanllenges in doctoral education. Doctoral programs respond to external forces such as national governments, and this has direct implications for globalization, government innovation policies, and postgraduate education. There are numerous advantages to the recruitment of doctoral graduates in various areas. Bryan’s and Kay’s (2018) work discussed the value of the doctoral, with the results indicating the value to doctoral candidates and the job market. It is therefore important for researchers to be aware of the potential career paths (Mangematin, 2000; Vekkaila et al., 2019). Adequate funding will ensure that the majority of doctoral candidates have sufficient time and energy to complete their doctoral studies. However, they may face various challenges, particularly during the period of being a doctoral student. Wollast et al.’s study (2018) demonstrated that the marital status and funding of doctoral candidates affect their likelihood of dropping out. Furthermore, the supervisor plays a significant role in the success of PhD candidates (Ferguson & Sharples, 2014). Based on data from 10 years, the results indicate that on-campus employment increases the probability of completing a doctoral programme, while off-campus employment is negatively associated with the completion of a doctoral programme (Bekova, 2021). Therefore, doctoral education is highly valued in both academia and industry, and more attention should be paid especially on how to attract doctoral candidate. Master’s studies are usually short in duration, to motivate students to continue doctoral study is even more challenges especially in high workload during their master’s study. Therefore, how to integrate doctoral education into master’s studies using different pedagogical tools can be one of the approaches to motivate students. In various educational institutions, the choice of pedagogical tools play an important role in improving teaching quality (Silva-Maceda et al., 2016). Depending on the number of students, appropriate and efficient tool should also be selected (Batra & Andreas, 2021). During the research, it is also recognized that suitable tools must be selected for students with disabilities carefully (Seale & Martyn, 2010). In crosscultural collaboration, the learning environment of one university is typically selected. However, teachers from another university may be unfamiliar with the educational tools, such as Slack and Zoom. However, after getting familiar with the tools, it is recognized that all the tools are beneficial to the teaching and education collaboration (Deng & Jackie, 2021). Furthermore, there are differences between different study programmes. For instance, the pedagogy tool used in business differs from that used in engineering. In addition, the literature contains information about teachers’ perspectives on the pedagogical tools for their teaching practice (Diwakar, 2012). So it would be beneficial to students if suitable pedagogical tool is found and used in the course to simulate the paper peer-review process. However, in the past few years, AI has rapidly increased its ability in many aspects. In higher education, teachers, students, and staff have started using AI for different purposes, such as question generation, feedback provision, and essay grading. However, practicality and ethicality are the main concerns (Yan et al., 2024). It is recognized that students are using it for different purposes. Therefore, there is an urgent need to develop a new evaluation method to evaluate students’ performance in the course and improve the quality of the course. In addition, the relation between higher education and lifelong learning is discussed in (Bearman et al., 2024). The challenge is that students may not give their original contribution, which results in loss of motivation to produce original work, since teachers cannot always identify which work is made with the support of AI. Therefore, a more inclusive approach is required to address this problem (Luo, 2024). This paper presents a case study of one postgraduate course that implements a doctoral study process in its exercises. The objective is to assist students in comprehending the process of becoming a doctoral candidate and ultimately a doctor. In some exercises, a peer-review system is employed to simulate the peerreview process observed in journal paper publication. Subsequently, the quantitative and qualitative feedback from students is gathered and analyzed. 2 DESCRIPTION OF THE METHOD 2.1 Course setting The course’s name is reliability-based machine element design, which is a first-year master’s course mainly for mechanical engineering students. The course has been running for a long time. However, considering the validity, the information about the course in the past three academic years was collected in this study. Over the course of the three-year period, approximately 100 to 150 students have participated in the course each year. The course is attended by three groups of students.: (1) Group 1: international students from all over the world, full-time students. (2) Group 2: local students, full-time students. (3) Group 3: local students, working while studying. Since students in group 3 are working while studying, the lectures are recorded and made available for later viewing. The course runs from the beginning of September to the middle of December each year, with a total duration of 13 weeks. The course is worth 5 European Credit Transfer and Accumulation System (ECTS) credits, with each ECTS equating to 27 hours of workload. The language of instruction is English. Since this course is one of the classic courses, the learning outcomes and the content do not change much over the years. The learning outcome and contents are not described here as it does not have direct connection to this study. In the academic year 2024-2025, the course comprises four lectures. Lectures are delivered every two weeks, allowing students sufficient time to complete the exercises. Lectures are delivered online and recorded. All the course materials are updated using the same template and structure to enhance readability, based on feedback from students from the previous year. Regarding the exercises, there are four in total. The number of exercises was decided because it is more suitable in terms of schedule and content. The fourth exercise in the academic year 2024-2025 is equivalent to the fourth and fifth exercises in the previous academic year. Each exercise is relevant to the lecture given in that week. Students are given approximately two weeks to complete each exercise. The practical arrangements for the group exercise, mid-term presentation and final report are the same as those of the previous year. However, regarding the final presentation, each group is required to give a presentation and listen to other two presentations, rather than remaining in the classroom throughout the duration of the whole day event. This arrangement has been implemented to accommodate students who have presentations or examinations scheduled for the same day as the presentation, as it is not practical for them to be in the classroom for the entirety of the event. Nevertheless, by listening to the other two other presentations, students should still be able to gain valuable insights and identify areas for improvement. The full grading system of the course is presented in Table 1. Table 1. The grading scheme of the course. Assessment Description Share of the final grade Group exercise Four group exercise (10%) 40% Final presentation Group presentation at the end of the course 35% Final report Group report submitted at the end of the course 25% 2.2 Development of the course It is recognized that some students will pursue a doctoral degree after they have finished their master’s studies, and then continue their career. Alternatively, some students do not know the process from PhD candidate to doctor at university. Therefore, in this course, we design the course in a way that reflects the progress from a PhD candidate to a doctor. In general, a student needs to publish several peer-reviewed journals papers, then they will summarize their publications into their dissertation and send it for review, and they need to defend their dissertation in a public examination, if they defend it successfully, they will receive a doctoral degree. This course aims to provide a representation of the doctoral education process. The illustration of how the course compares with doctoral education in a European university is shown in Fig.1. The 4 peer-reviewed publications are similar as the exercise 1-4 in the course, the dissertation is similar as the final report, and the presentation/defense is similar as the final presentation. Fig. 1. Flow chart of the course compared with doctoral education in a European university Over the last few years, with an increasing number of students and an increasing number of online courses, educators have experienced challenges with students’ homework evaluation processes. For example, students know little about this peerreview system, they do not see this as a learning tool when they receive and produce feedback reviews from and for peers (Cho & MacArthur, 2011). However, there are many advantages to using the peer-review system. It is found that in undergraduate courses (Van et al., 2006), postgraduate courses (Dickson, 2019) and doctoral programs (Ciampa & Zora, 2023), the oral peer review exercise improves understanding, confidence and satisfaction. When feedback is given by multiple peers, the feedback is more reliable covers different aspects, sometimes, the feedback is written in more accessible language. In massive open online courses (MOOC), peer review is also a solution to motivate students to learn (Meek et al., 2017). In fundamental courses such as mathematics, students can benefit from engaging in the peer review and evaluation process (Thomas et al., 2011). The peer-review system is also referred to as the "Workshop" activity in Moodle. Students submit their work, which is then distributed amongst their peers for assessment based on the evaluation matrix specified by the teachers. The submission system will be opened for 1.5 weeks, during which students can send their work at any time and make modifications if necessary. Once the submission system is closed, no further submissions can be made. Subsequently, the phase transitions from submission to peer-review. Each student will receive three randomly selected reports for evaluation based on the provided evaluation matrix. Once the peer-review stage is complete, the grades will be calculated. The assessment criteria is a rubric matrix, considering five aspects. (1) How well is the report documented? (2) How does the report reflect the discussion item 1? (3) How does the report reflect the discussion item 2? (4) How does the report reflect the discussion item 3? (5) How does the report reflect the discussion item 4? And under each aspect, different items should be checked, for example, in aspect 1, the following items should be considered when giving the grades: (1) The texts should be readable and easy to understand. (2) The figures and tables should be described in the text. (3) The figures and tables should be clear to readers, such as ensuring the fonts in the figures must be readable. (4) In the text, the references should be in the proper position so that they refer to the right content. (5) The report template is used properly (it is not strict to use the template in the exercise stage, for example, abstract or abbreviation are not needed but suggested to be included). Finally, examples of giving different points are listed: (1) Point 1: the report is difficult to understand, and there are several mistakes which contradict the report. (2) Point 3: the report is acceptable and easy to understand, even though some modifications are required in the future. (3) Point 5: the report has all the elements, references are well cited in the reference section, not much is needed in the future. As an example, student A will submit the exercise to Moodle before the submission deadline. Then right after the deadline, student A’s submission will be randomly distributed to student B, C, and D. Next, student B, C, and D will evaluate student A’s submission based on the given evaluation matrix with short explaination, which is the similar as this paper’s peer-review process. Once the deadline of evaluation is finished, the average point of student B, C, and D will be calculated and used as the final point of this exercise for student A. In the end, if students have questions about this exercise, they can contact the responsible teacher and ask for re-evaluation. Overall, the whole process is automated. 3 RESULTS 3.1 Student learning As for quantitative analysis, the grades of each task and the final grade are exported and then analysed. Firstly, the scale of the grades of each exercise is from 0 to 5, and these grades are divided into five intervals: 1) 0<grade≤1; 2) 1<grade≤2; 3) 2<grade≤3; 4) 3<grade≤4; 5) 4<grade≤5. Different colours/markers represent different tasks in different years. Fig. 2 illustrates line charts showing the grade distribution of each exercise, the final report, the final presentation, and the final grades over three academic years. Fig.3 and Fig. 4 illustrate students’ point statistics in different tasks in different academic year. Table 2 summarizes the average points and standard deviation of each task in different academic years. Students feedback of the peer review system is analysed by qualitative analysis of their feedback left in the end of the course. Overall, students made roughly equal number of positive and negative comments, some comments provide constructional suggestions as well. In their negative comments, the students highlighted several areas for improvement. They suggested that the instructions should be more clearly defined, the time allocated for exercise submission and peer review processes should be adjusted, and that sometimes, the reviewers do not provide any comments or feedback. However, they did not express opposition to this peer review tool, but rather, suggested ways to enhance it. On the positive side, The students highlighted that the continuous engagement of the course motivates them to study, although they occasionally found the deadlines challenging. The final report is a combination of the earlier reports, which they found a relief. The report-based exercise was seen as an effective learning tool and a valuable approach. Some students also observed that the presentation was beneficial for them, as it required them to step outside their comfort zone and improve their skills. Furthermore, in the context of peer review, many students recognize the value of learning from the experiences of others and applying this knowledge to subsequent reports, informed by the insights gained from feedback. Fig. 2. Grades distribution of each exercise, final report, final presentation, and final grades Fig. 3. Students’ points in average in each task in different academic years. Fig. 4. Students’ points in standard deviation in each task in different academic years Table 2. Tasks’ points in average value x , and standard deviation σ 2022 2023 2024 x σ x σ x σ E1 8.12 1.65 8.19 1.03 7.99 2.81 E2 8.86 1.27 8.67 0.84 7.78 3.08 E3 8.08 1.46 8.18 1.14 7.88 2.76 E4 8.61 1.14 9.50 1.27 8.14 2.73 Final report 15.45 3.14 17.31 2.39 18.46 6.51 Final presentation 21.55 4.12 25.34 4.52 28.57 4.26 Final grade 3.82 0.95 4.27 0.86 3.84 1.40 3.2 Teacher and student experiences As mentioned previously, the goal of introducing the peer-review system is to imitate the process from a PhD candidate to a doctoral graduate in a university. Compared with previous years, teachers only monitor the evaluation process of E1-E4 instead of evaluating them one by one. If students are not satisfied with the results, they can contact teachers during the weekly office hour. In this way, it saves time and encourages communication between teachers and students. Both teachers in the same course feel the workload is decreasing. In addition, a similar approach can be applied to other MOOC courses or postgraduate course. From the students’ perspective, the peer-review system allows students to check how other groups write their reports. They can also learn from these reports and recognize the errors in their own submissions. Since it is already common that students compare their grades with others outside the class. This peer-review system makes it transparent. This is different from the traditional method, where students cannot see others’ work, and do not know how well they have done. In addition, students can recognize those who work hard, and have a goal of achieving higher grade in the course, they may choose each other as potential group members in future courses. 4 SUMMARY The statistical results show that peer-review assessment supports this course, there was concern that students would give random points to others. However, this phenomenon was not recognized. With the development of exercise instructions and the enrichment of course materials, the performance of students has improved, especially in final report writing and final presentations. Some random exercises samples were checked during the course, and teachers found the evaluation results to be aligned with the evaluation matrix. From the feedback on this course, some students expressed interest in continuing their journey in academia, and they have realized the requirements of acquiring a doctoral degree in this field. Overall, the teachers are satisfied with this approach and the results. It is suggested that a similar approach can be adopted also in master’s courses in other universities. Minor adjustments can be made to better fit the local education system. REFERENCES