Development of an Application for the Automated Evaluation of 3D Modelling in Industrial Parts

Depósito de Investigación de la Universidad de Sevilla https://idus.us.es/ This version of the conference paper has been accepted for publication, after peer review and is subject to Springer AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-3-031- 51623-8_88 Development of an application for the automated evaluation of 3D modelling in industrial parts Laura GARCÍA-RUESGAS1*, Jaime AREÑAS-MORALES1, Francisco SALMERÓN-MEDINA1-2, Cristina TORRECILLAS1 1 Dpto. Ingeniería Gráfica, Escuela Técnica Superior de Ingeniería, Universidad de Sevilla. Av. De los descubrimientos s/n, Seville, 41092, España 2 Dpto. Ingeniería, Universidad Loyola Andalucía. Av. de las Universidades, s/n, 41704 Dos Hermanas, Seville, España * Corresponding author. E-mail address: [email protected] Abstract In the teaching field, the existence of automated procedures that allow self-assessment or self-correction of exercises or exams is a great help. Most web learning applications generate tests with single, multiple, or text answers, or the use of closed-solution problems where only the student's final answer is validated. The complexity increases when we try to score a text, but the most difficult evaluation is in the field of Graphic Engineering, inside CAD/CAM environments. The student assessment usually uses the revision of a digital file with a solution. While it is true that some software may offer tools to compare files, these are focused on the search for differences in their own versioning. When the 3D industrial model is created by students, the tools, planes or geometric sketches are very varied, and an automated assessment is not a simple task. This paper attempts to offer a possible way for the automated correction of the 3D modelling in industrial parts based on some parameters proposed. A programme, called CPIT (Corrector of Three-Dimensional Industrial Parts), has been designed to evaluate the similarity between parts within the CATIA® V5R19 environment using Visual Basic for Applications (VBA). This first version has been tested successfully with several exercises and it is necessary to improve some aspects of it, such as its performance in complex pieces or Boolean operation. Keywords: Self-assessment; Industrial Design; 3D Parameters; Dimensions; Gravity Centers. 2 1 Introduction Within the framework of teaching industrial design subjects, teachers meet students who do not have extensive knowledge of the subject. This is due to pre-university training (and first university courses in some cases) extremely focused on more abstract aspects and supported by hand drawing. Thus, teachers face different challenges for the subject development that can be split into four categories: “1) Handling the software, 2) Using ready-made models, 3) Manufacturing and creating printed models, and 4) Designing" [1]. This paper focuses on the first category, which is essential to be able to advance through the later ones. Besides, in a digital environment where CAD/CAM subjects are carried out, teachers have to develop the competences needed to be fulfilled, on one hand, the five general digital competencies [2] are combined, since it is developed in an entirely digital environment and more specifically focus on the third competence “Digital content creation” and the fifth one “Problem solving”; and, on the other hand, the specific competences of the subject in question [3]. Furthermore, in the field of Graphic Engineering, inside CAD/CAM environments, the assessment is complex [4–9]. The students´ drawing exercises are solved on paper or in digital files. Their resolution usually resorts to the use of templates (paper), or the revision of each file (digital). While it is true that some software may offer tools to compare files, these are focused on the search for differences in their own versioning. In the case of a versioned file of a 3D modelling of an industrial part, this file has similar features and the same coordinate origins as the previous file. When the 3D model is created by students, the tools, planes, or geometric sketches are very varied, and an automated assessment is not a simple task. Thus, in this special context, a respectable number of tests have been carried out in the field of CAD subjects, trying to emulate many tests and studies of automatic correction of exercises in different areas within STEM subjects, with corroborated success in many of them. The use of automatic corrections is spreading in multiple fields, as it has benefits such as the time teachers save in this process, giving them the time to prepare more exercises, the impartiality and accuracy of the correction, and the particularization of the teaching to the specific students’ needs [10]. This is in addition to the fact that after the COVID pandemic, the use of online assessment has been reinforced and increased. Most of these automatic assessment processes focus on the correction of numerical concepts such as control systems and programming [11] and others work with tests with single, multiple, or text answers, or the use of closed-solution problems where only the student's last answer is confirmed. The complexity increases when we try to score a text. In these cases, it could be possible to assign a quantitative evaluation to the answer given, for instance, by searching for some keywords. These automated assessment applications, understood as those carried out by software, offer a closed and quantified value once the test is over [12]. There are researches focus on finding the automatic generation of statements and their correction [13, 14] using as software 3 support (SolidWorks® in this case) in which the analysis of physical variables for correction is proposed, in some way the modelling problem is being taken to a numerical issue; other studies such as [15] proposes a software called TrainCAD in which a self-evaluation of the exercises performed in AutoCAD is proposed -as well as [16]- and that can be executed by the students themselves to verify their errors or [17] support on iGeom to compare geometry objects. Other studies, such as [18], analyse two tools for the correction of different aspects, the first of which compares by STEP file the surfaces of the student's exercise with that of the target result and, on the other hand, using Creo Parametric® by PTC as a support, comparing the magnitudes of volume and/or surface, and looking for certain operations within the student's exercise. In other cases, studies have been developed along the lines of measuring the execution times and the number of features used, supported by Pro- Engineer® and later Wildfire® software, of some ad-hoc defined exercises to design empirical learning curves to analyse how students develop the necessary skills in CAD subjects [19]. This paper attempts to offer a feasible way for the automated correction of the 3D modelling in industrial parts based on some parameters proposed in [20]. The idea is to generate a program that automatically compares 3D files with a double objective: on one hand, to make possible errors or decisions in their modelling visible to the students who are able to correct them by means of the information provided by the program (self-learning and encouraging interaction with the user as pedagogical contents [21]); and, on the other hand, to offer a possible mark to help in their assessment. 2 Levels and parameters to assess A rubric plays a crucial role in an evaluation. In a 3D industrial model design, this could be related to the initial positioning, order of operations, a correct use of them, and well-defined sketches. Visual Basic for Application of CATIA V5R19 is the programming environment. Ordered by importance, three detail levels have been classified and defined with a weight depending on some parameters inside them (see Fig.1): • Level 1. General parameters: Analyse the general parameters of the parts. The initial position of the model for a correct approach of general planes in, e.g., symmetries or patterns, is the first step. Linked to this idea, ten parameters that appear in CATIA V5R19's Measure Inertia tool are evaluated, which can be accessed from Toolbars – Measure, see Table 1. • Level 2. Modelling tree and order: Analyses the modelling tree and the number of part operations. • Level 3. 3D and 2D analysis of dimensions: Analyses those 3D and 2D dimensions that are part of the same operations in both files (student and reference). 4 The diagram in Fig.1 shows the weight of each level in the final evaluation. These values are the results of years of experience evaluating these types of exercises. The analysis has been organised in this way to encourage user learning. The idea is that as the programme runs, it reports the results on the status of the analysis and the corrections to be made through a window. This methodology makes it easier for the user to know where errors have been made and how to solve them. Furthermore, the fact of separating the programme into different levels makes it possible to distribute the evaluation of the part among a multitude of aspects. Consequently, the final score is as complete and precise as possible. On the other hand, it should be noted that the levels are structured in increasing order of detail, from the general parameters of the part (Level 1) to the three and two-dimensional analysis of the operations that make up it (Level 3). Since this programme is designed to evaluate parts containing operations and sketches that are common in industry, it is possible that when evaluating, for instance, special parts that have very complex sketches and few operations, more singularities than expected may appear. The design of the reference part file is important and, in our subject, our goal is the manufacturing, so this design and tree model sequence are according to this objective. The parts to be assessed should have a clear use of industrial operations and order. First of all, these part files should have a main body with some transform operations (pattern or symmetry). Secondly, the dress-up features which involve minor changes (holes, edge fillets or chamfer) might be added. Finally, in this way, the result will be suitable with the design level of an industrial modelling course for beginners. Fig. 1. Self-assessment flow and weight levels. 5 Table 1. General elements to be assessed following [20]. Elements to be assessed Description Weight Level 1 BBLx, BBLy, BBLz (mm) Volume (m3) COGx, COGy, COGz (m) Same Bounding Box X, Y, Z maximum Same Volume Same X, Y, Z Gravity centre 1,8 1 0,6 M1, M2, M3 (kg*m2) Same Inertia Moment 1, 2, 3 0,6 Level 2 Tree Use of suitable operations 2 Order Use of an appropriate level of design detail 1 Level 3 3D analysis Use of right 3D dimensions 1.5 2D analysis Use of right 2D dimensions 1.5 From our experience, we have been able to verify how student learning increases when the available initial information is a template instead of a modelled part. In this case, the programme serves as a guide, informing the user about the number of operations to carry out or the kind of operation to use in order to improve the result. By not showing the final solution, the user is obliged to think. 3 Programme design A software called CPIT (Corrector of Three-Dimensional Industrial Parts) has been designed to evaluate the similarity between parts within the CATIA® V5R19 environment. For this purpose, Visual Basic for Applications (VBA) [4] has been used. Fig. 2 shows the CPIT graphical environment. In the upper text box of the menu, the directory of the part to be analysed is shown, which must be open in CATIA before executing the programme macro. In this way, the user is assured that CPIT will analyse the desired part. The main menu offers the user the possibility to view the results obtained. Since CPIT informs through the results window (C in Fig. 2) about the analysis and potential corrections to be made, it is not necessary to show the obtained solutions. If the top menu box (R in Fig. 2) is ticked, the solution Excel file will be displayed once the analysis process is finished. There are 2 options (A or B) to evaluate our part, which are associated with the use of a part or an Excel template as a reference, respectively. The latter option, which is derived from the correct model, is considered to facilitate better learning for the student, as the final solution is not initially visualised. Once one of the options is activated, the other is disabled. In addition, when any option is checked, the tool is applied to the part modelled by the user. The evaluation presented in the status box is based on three detail levels (“Level 1”, “Level 2” or “Level 3”). These levels are geometric comparisons which are established on: extracting general parameters of the part (volume or the centre of 6 gravity), the operations that make it up, or the 2D and 3D used dimensions. At the end of the analysis process, some tips are displayed in the status box related to wrong dimensions or what to do to improve the model. This information, shown in figure 2, is also included in the Excel® template (in the same directory as the part file) as shown in figure 3. The search button opens a window that allows the user to select the reference file. Once opened, the reference file directory is copied into the adjacent text box. In the case of option A, the reference part is opened in CATIA, and the Measure Inertia tool is applied to it as well as to the user model. Clicking on the run button, the software is run. Finally, following the discrepancies mentioned above, considering the different levels, the score is estimated and shown in the CPIT window (D). Fig. 2. CPIT environment: R (to obtain comments in an Excel file); A (to evaluate with a CATIA file); B (to evaluate with an Excel file); D (assessment); and C (comments). CPIT is capable of analysing sixteen operations of features from CATIA's Part Design module distributed between the Sketch-Based Features, Transformation Features and Dress-Up Features windows. In addition, the possibility of analysing models using Geometrical Set and parameterised models with the "Formula" tool has been included in CPIT. It is important to mention that in some cases the time performance of the programme is not optimal (some minutes), especially in parts with a high number of operations and dimensions to assess. Additionally, there are operations that are not correctly analysed in some cases, such as Multi-sections Solid, generally because the sketches do not appear associated with the operation. A multitude of tests have been carried out on individual part files using a wide catalogue, offering the users a wide range of freedom when modelling their parts. The result is an Excel file with three sheets, one for each level. 7 4 Testing and results Some tests have been performed to evaluate CPIT software. In detail, three exercises were analysed: • A pulley: a simple part with 3 different tools and 5 operations (see Fig.3) • A complex part: a part with 8 different tools and 10 operations (see Fig.4) • A simple part: a sketch with only 2 pads (Fig. 5). Fig. 3: Level 1 result on Excel® template after analysing Pulley file from two different students. In Fig. 3, the assessment mark, the tree model, and the evaluation of level 1 for two different parts are shown. The part on the left obtained a 9.5 as result of a bad initial positioning of the initial Shaft sketch. The part is correct, but in Mechanical Design, the approach of the symmetrical shapes and positional centering is important. The pulley part on the right is correctly positioned, but two radio measures are wrong. This affects to the volume and inertial moments. Fig. 4 shows the results of another part more complex than the first one based on two different student files. In this case with the 'Circle pattern' and 'Mirror' tools. On the left, the evaluation mark is 9.4 even though the positioning and volume part is correct. Table 2 shows an extract of the Level 2 sheet of the Excel file. In this table, the number of operation type is not the same. The student used the “Circle pattern” twice instead of only once and “Mirror” is missing, probably the student includes the operation inside the sketch. Obviously, the part is correct, but it could 8 be better without the use of a double circle pattern and running the mirror in a 3D environment, so 0.6 points were subtracted from the final mark. The exercise with 5 points (on the right in Fig 4) is the result of some wrong dimensions and incorrect operations (using the “Pocket” instead “Hole”, 2D fillets instead of 3D fillets or additional “Pad”s). Fig. 4: Up, the reference part. Down, Level 1 result on Excel® template after analysing complex part file from two different students. Table 2. Extract of the Level 2 Excel® sheet of complex part with a 9.4 mark of Fig. 4. Operation Type Number Operation type Reference Number Pad 3 Pad 3 Draft 1 Draft 1 Pocket 1 Pocket 1 Stiffener 1 Stiffener 1 CircPattern 2 CircPattern 1 Hole 1 Hole 1 ConstRadEdgeFillet 1 ConstRadEdgeFillet 1 Mirror 1 Finally, Fig. 5 shows a simple part with only 2 Pads but a complex base sketch. The objective of this example is to demonstrate the influence on the final mark of the