Usage of real time machine vision in rolling mill

sustainability Article Usage of Real Time Machine Vision in Rolling Mill JiˇríDavid *, Pavel Švec, Vít Pasker and Romana Garzinová   Citation: David, J.; Švec, P.; Pasker, V.; Garzinová, R. Usage of Real Time Machine Vision in Rolling Mill. Sustainability 2021,13, 3851. https:// doi.org/10.3390/su13073851 Academic Editors: Bernardi Pranggono, Alex Shenfield and Augustine Ikpehai Received: 1 February 2021 Accepted: 26 March 2021 Published: 31 March 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). Department of Automation and Computing in Industry, VŠB-Technical University Ostrava, 17. Listopadu 2173/15, 70800 Ostrava, Czech Republic; [email protected] (P.Š.); [email protected] (V.P.); [email protected] (R.G.) *Correspondence: [email protected]; Tel.: +420-723-858-551 Abstract: This article deals with the issue of computer vision on a rolling mill. The main goal of this article is to describe the designed and implemented algorithm for the automatic identification of the character string of billets on the rolling mill. The algorithm allows the conversion of image information from the front of the billet, which enters the rolling process, into a string of characters, which is further used to control the technological process. The purpose of this identification is to prevent the input pieces from being confused because different parameters of the rolling process are set for different pieces. In solving this task, it was necessary to design the optimal technical equipment for image capture, choose the appropriate lighting, search for text and recognize individual symbols, and insert them into the control system. The research methodology is based on the empiricalquantitative principle, the basis of which is the analysis of experimentally obtained data (photographs of billet faces) in real operating conditions leading to their interpretation (transformation into the shape of a digital chain). The first part of the article briefly describes the billet identification system from the point of view of technology and hardware resources. The next parts are devoted to the main parts of the algorithm of automatic identification—optical recognition of strings and recognition of individual characters of the chain using artificial intelligence. The method of optical character recognition using artificial neural networks is the basic algorithm of the system of automatic identification of billets and eliminates ambiguities during their further processing. Successful implementation of the automatic inspection system will increase the share of operation automation and lead to ensuring automatic inspection of steel billets according to the production plan. This issue is related to the trend of digitization of individual technological processes in metallurgy and also to the social sustainability of processes, which means the elimination of human errors in the management of the billet rolling process. Keywords: metallurgy; machine vision; neural networks; control; applications; steel industry; optical devices; artificial intelligence; machine learning; intelligent systems; artificial neural networks; image processing; image classification; real-time systems 1. Introduction In today’s industrial companies and manufacturing companies, the emphasis is on product quality and in this respect on the number of control points at each stage of production. Checks can be performed manually, automatically, or in combination. With the growing modernization of business and the development of automation technologies, many operations are moving to automated management. Accordingly, various quality control methods are used at control stations, such as machine vision, surface scanning, precision shaping, color and size measurement, or inspection of semi-finished products and product identifiers during production and storage. Many of these control methods use an industrial camera as an essential element to provide the required information in image form. Sustainability 2021,13, 3851. https://doi.org/10.3390/su13073851 https://www.mdpi.com/journal/sustainability Sustainability 2021,13, 3851 2 of 18 The application of artificial intelligence methods in industrial applications is today a trend both in the field of process digitization and in the field of sustainability, where it is also necessary to apply new methods and approaches. There has been a lot of talk lately about the Industry 4.0 initiative or the 4th Industrial Revolution. New trends in automation, digitization, or data exchange in industrial processes and technologies can be heard from all sides. On the other hand, the issue can also be seen in the context of sustainable development. The above-described solution of using machine vision in a rolling mill is an example of a partial solution contributing to the digitization of the process in order to create a so-called “Smart Factory”, which combines the physical and cyber world of production, but also as a solution that is in line with sustainable development of society in all factors—increasing the quality of production, reducing production costs, and from a social and environmental point of view. This connection significantly takes the digitization of industry to a new level. But what does the combination of the physical and cyber world really mean for manufacturing companies? What opportunities does the digitalization of industry bring? The answer can be contained in one term—intelligent data. Intelligent data is closely linked to digitization and data analytics. We can define them as data that are unambiguous and cleaned up and that are linked directly to a specific action. Intelligent data is the result of digitization and data analytics. In order to work out the information and value that data represents within the company, it is first necessary to collect it (integrate different data sources), process it (e.g., transform it into the same data type or clean it from data impurities) and analyze it (using different types of data analysis, such as predictive analysis). Today, however, the field of intelligent data faces a large number of challenges. These include the speed of new data, security, and the lack of experts in the market (with knowledge of the company’s production processes and also knowledge of data analytics) or the integration of several different data sources. This raises the question of whether intelligent data is really worth the effort and what digitization and data analytics actually bring to manufacturing companies. Simply put, it helps companies understand what and how it is done, and then helps identify opportunities for improvement. In the field of digitization, we are not only talking about the transfer of the physical world to the cyber world, the processes of data collection, or the elimination of communication in paper form (the concept of the “paperless company”). Digitization in the context of production also includes concepts such as the digital twin, advanced data mining of production and non-production processes, or the entire production and product life cycle. Digitization is thus a cornerstone of Industry 4.0. We can say that it is the technical advances in digitization that enable the complete interconnection of the entire production process and the entire customer-consumer chain. Subsequently, it is possible to apply advanced data processing methods, apply statistical and prediction models, monitor the behavior of systems and machines in real-time, or prevent outages in production processes. Today, technological advances allow us to collect data directly from production lines (Internet of Things: IoT), pre-process it at the local level and visualize it in clear reports or work surfaces (edge computing), send it to the cloud for further processing (e.g., machine learning), or enrich it with non-production data and visualize on mobile devices. Individual technologies thus really become the standard and in today’s highly competitive environment are certainly a necessity to maintain their own competitiveness. Industry 4.0, the digitization of production processes, and intelligent data are not a goal, but a tool to achieve set goals, identify new business models, or optimize production processes. These tools use data as a basic input and prerequisite for better and more accurate decision-making. Digitization and data analytics then represent something as a cornerstone of the whole Industry 4.0 initiative, where the physical and cybernetic world is interconnected in a smart factory into a system that monitors processes, creates Sustainability 2021,13, 3851 3 of 18 a digital twin (workpiece or process), and is able to make its own decisions (artificial intelligence elements). Applications aimed at creating smart factories change the functioning not only of industry and production but also of society as a whole. Productivity is a sore spot in the Czech labor market. A 3% growth in labor productivity growth in 2017 was above average after the economic crisis, but still below pre-crisis levels. The level of labor productivity remains relatively low, especially compared to developed economies, due to significant skill mismatches, the low impact of R&D on innovation, and the size of low-skilled industries. To achieve economic value, the Czech economy must move from low-skilled to high-skilled, thereby increasing value-added and improving the position of companies. The process of automation and digitization is absolutely crucial for the Czech Republic. Both processes lead to a reduction in production costs, shortening the production cycle, increasing production efficiency, increasing the useful properties of products and services, humanizing work, improving the working environment, and increasing leisure time. According to various studies and ongoing digitization and automation projects, the greatest impact is expected on employment in the middle class, where many professions are routine and therefore most prone to computer replacement. The automation and digitization of industrial industries alone may not result in a reduction in the number of employees; on the contrary, it may be a way to compensate for the lack of labor due to unfavorable demographic developments in developed countries. Advanced automation thus jeopardizes more work tasks than entire jobs. In line with existing findings, it is expected that new technologies will shift the ratio of man-made to machine-driven tasks, with the nature of the changes varying both in scope and speed of application. Interoperability, decentralization, intelligence, and reconfigurability are features that currently bring other benefits to companies, for example in the areas of safety and environmental studies. The Industrial Internet of Things (IIoT) and the vast amount of data that this network generates create greater clarity in business processes, and thus reduce the risk of human or technological error. Industrial Internet of Things (IIoT) technologies and Smart Industry systems have a direct impact on improving the safety, reliability, and sustainability of the production environment, and ultimately on preventing negative impacts on product quality and delivery time. Greater transparency also helps to reduce waste, leakage, and waste and identify their origin, thus minimizing the environmental footprint of the manufacturing plant. This paper deals with the issue of optical character recognition using artificial neural networks. The main part of the paper is a description of the designed and implemented algorithm for optical identification of a character string from the front of a steel billet using artificial intelligence. The paper has five sections. The introductory chapter outlines the issue of process digitization and the development of sustainability in the current conditions of process digitization. Section 2of the article briefly describes the billet identification system in terms of technological and hardware resources. Section 3describes the issue of optical chain recognition on steel billets. Section 4describes the use of neural networks for the recognition of individual elements of the chain and the achieved results. The conclusion of the article then presents the achieved results in terms of the proposed algorithm and in terms of operational use. 2. System Design for Identification of Steel Billets The basic semi-finished products for the production of rolled wire are steel billets, which are stored in the area of the rolling mill and further distributed to the production line according to the requirements of the production plan. Before entering the heating furnace, the billets are inspected only during the process in the charging network without inspection from the service station. In some cases, inaccurate checks and errors in billet changes occur here. As the material properties of individual billets vary greatly depending Sustainability 2021,13, 3851 4 of 18 on their use for specific products, their changes result in fatal manufacturing defects and large financial losses [1]. In order to modernize the operation and limit the exchange of billets, a billet identification system based on machine learning was designed and subsequently implemented. The design of the billet identification system was realized by four basic working procedures divided according to thematic tasks: • The first basic procedure was the selection of individual components, in the selection of which it was necessary to observe the requirements and conditions of operation. • The second procedure was to set up and install a network camera, including mounting the camera and accessories, including wiring. • The third procedure was the creation of an algorithm to adjust the billet conveyor control, including the addition of new commands in the programmable logic controller (PLC) control program and camera control settings. • The fourth procedure was the design of an algorithm for the identification of billets using machine learning and storing the information obtained in the Billet Stamping program. Furthermore, the article is focused on the description of the proposed algorithm for billet identification using machine learning [2–4]. Input information and operating requirements 1. Steel billets have, for identification purposes, a serial number, which is created by a marking automatic machine immediately after the billets exit from the continuous casting machine. 2. The numeric character contains a combination of 5 to 9 digits, possibly accompanied by two letters. 3. The steel billets are arranged on the charging grate and individually conveyed to the heating furnace. 4. For the purpose of billet identification, the possibility of digitizing numerical characters on the computer monitor of the operator’s station and the possibility of archiving the obtained information is required. The selection of technical means for scanning and subsequent identification was made with regard to the properties of the scanned objects, which were steel billets. The dimensions of the billet are 8 m in length, 150 mm in width, and 150 mm in height. The weight of one billet is approximately 2 tons. The billets are arranged on a loading grid and individually transported to a heating furnace, from which they continue to the rolling line after reaching the required temperature. For identification purposes, they are marked with a serial number string, which is created by an automatic stamping machine on the front of the billet. The numeric character contains a combination of 5 to 9 digits, possibly supplemented by two letters. The basic components were an industrial camera, a camera lens, a camera cover, a camera cover holder, a power supply, a light reflector, and related cabling. When choosing an industrial camera and accessories, it was necessary to take into account, in particular, the method of use and the properties of the environment. Components must be designed for continuous operation in an industrial environment. Also, the emphasis was on reliable operation, software, and reasonable price. Based on these conditions, the camera IP 1343 was selected by Axis (see Table 1for parameters) and lens Computar TG10Z0513FCS (see Table 2for parameters). Sustainability 2021,13, 3851 5 of 18 Table 1. Axis P1343 camera parameters. Camera Resolution 800 ×600 Image sensor type CMOS 1/4” progressive scan Minimum lighting value 0.05 lux Image compression MPEG 4, MJPEG, H.264 Number of frames per second 30 Operating temperature −30 ◦C to 50 ◦C Power voltage 8–20 V DC Consumption 6.4 W Table 2. Computar TG10Z0513FCS lens parameters. Focal Distance 5–50 mm Aperture 1.3–360 Aperture control Automatic DC Format 1/3” Type of attachment CS Due to the production environment having both high temperatures and considerable dustiness, it was necessary to place the camera in a camera cover. This cover protects the camera from dust, moisture, and mechanical damage. The type used is TP607H protection class IP66, it is designed for a temperature range from − 30 ◦ C to +50 ◦ C. The cover was mounted on a bracket equipped with a swivel joint that allows the camera cover to be swiveled and tilted to the desired position. Camera power was provided by a linear 12 V/800 mA stabilized power supply. The power supply was placed in the camera housing using a cover kit. A bracket was made to mount the camera housing, welded from L-type steel profiles 40 mm × 40 mm with dimensions of 300 mm × 1000 mm, and treated with gray paint, see Figure 1. Sustainability 2021, 13, x FOR PEER REVIEW 5 of 17 T ABLE 2. Computar TG10Z0513FCS lens parameters. Focal Distance 5–50 mm Aperture 1.3–360 Aperture control Automatic DC Format 1/3“ Type of attachment CS Due to the production environment having both high temperatures and considerable dustiness, it was necessary to place the camera in a camera cover. This cover protects the camera from dust, moisture, and mechanical damage. The type used is TP607H protection class IP66, it is designed for a temperature range from −30 °C to +50 °C. The cover was mounted on a bracket equipped with a swivel joint that allows the camera cover to be swiveled and tilted to the desired position. Camera power was provided by a linear 12 V/800 mA stabilized power supply. The power supply was placed in the camera housing using a cover kit. A bracket was made to mount the camera housing, welded from L-type steel profiles 40 mm × 40 mm with dimensions of 300 mm × 1000 mm, and treated with gray paint, see Figure 1. A halogen reflector with a power of 100 W was used to illuminate the billets. Interconnection of network camera and data network was solved by FTP (Foiled Twisted Pair) (cable category 5e, cable CYKY-J 3 × 2.5 was used for power supply of camera and lighting. Existing cable routes were used to store the cabling. Figure 1. Camera mount including an incandescent light source. Pre-installation settings and installations of network cameras, network camera settings, lens testing, and network communication, as well as power supply functions, were performed. The camera was connected to a power source and a network cable to connect it to the computer. The camera was configured using a web browser. The network address, netmask, and gateway address were set on the camera. Then the administrator password was set. After these steps, it was necessary to restart the camera to save the newly set values. Camera configuration continued by setting the time zone, current time, and time synchronization server time. The next step was to adjust the image quality and resolution. The image quality and resolution were set to maximum, MJPEG was selected for the video format. The camera was equipped with a lens and several focusing were performed, including the extreme positions of the lens. This was followed by testing the auto iris function by aiming the camera at dark and light places alternately. In this way, the camera, lens, and power supply, i.e., the components that have the greatest impact on the final image quality, were easily tested. The camera, including the lens and power supply, was placed in the camera housing to which the hinge bracket is attached. The location of the camera installation was chosen Figure 1. Camera mount including an incandescent light source. A halogen reflector with a power of 100 W was used to illuminate the billets. Interconnection of network camera and data network was solved by FTP (Foiled Twisted Pair) (cable category 5e, cable CYKY-J 3 × 2.5 was used for power supply of camera and lighting. Existing cable routes were used to store the cabling. Pre-installation settings and installations of network cameras, network camera settings, lens testing, and network communication, as well as power supply functions, were performed. The camera was connected to a power source and a network cable to connect it to the computer. The camera was configured using a web browser. The network address, netmask, and gateway address were set on the camera. Then the administrator password was set. After these steps, it was necessary to restart the camera to save the newly set values. Camera configuration continued by setting the time zone, current time, and time Sustainability 2021,13, 3851 6 of 18 synchronization server time. The next step was to adjust the image quality and resolution. The image quality and resolution were set to maximum, MJPEG was selected for the video format. The camera was equipped with a lens and several focusing were performed, including the extreme positions of the lens. This was followed by testing the auto iris function by aiming the camera at dark and light places alternately. In this way, the camera, lens, and power supply, i.e., the components that have the greatest impact on the final image quality, were easily tested. The camera, including the lens and power supply, was placed in the camera housing to which the hinge bracket is attached. The location of the camera installation was chosen with regard to the production technology, optical possibilities of the camera, cable routing, and camera testing. A bracket for the camera cover was welded at the selected location, and a light reflector with a power supply terminated by a double metal socket with IP 55 protection was placed above the console. The camera was placed in the center of the conveyor approximately 30 cm above the passing billet, as shown in Figure 2. The scanning distance with respect to the required details and lens capabilities was determined by a camera test approximately one meter from the camera lens. Sustainability 2021, 13, x FOR PEER REVIEW 6 of 17 with regard to the production technology, optical possibilities of the camera, cable routing, and camera testing. A bracket for the camera cover was welded at the selected location, and a light reflector with a power supply terminated by a double metal socket with IP 55 protection was placed above the console. The camera was placed in the center of the conveyor approximately 30 cm above the passing billet, as shown in Figure 2. The scanning distance with respect to the required details and lens capabilities was determined by a camera test approximately one meter from the camera lens. Figure 2. Modification of billet conveyor control. During camera tests, it was found that it is not appropriate to scan billets in motion (Figure 3), especially because the effect of transmitted vibrations, which cannot be completely eliminated, significantly impairs the image quality. Much better results are achieved by scanning billets at rest. Based on these findings, it was necessary to stop the passing billet at a given distance from the camera before saving the image. Figure 3. A picture of a passing billet and a billet at rest from the Billet Stamping program. The rolling of the billet is controlled by a programmable logic controller (PLC) Simatic S7 300. This PLC controls the movement of semi-finished products on the charge grate and conveyors on which the semi-finished products are transported to the heating furnace. The PLC program for conveyor control had to be modified so that the billet stopped at the specified location, at this point the required time remained and continued to move to the heating furnace. The start command for the PLC is the “prepare billet” command from the main control system. The movement of the billet from the filling grate to the heating furnace takes place in the following steps, which are also shown in Figure 4. • Transfer of billet from charging grid to conveyor No. 1. • Start conveyor travel 1 and 2. • Stop the conveyor No. 1 and 2 for 5 s after signaling the sensor on conveyor No. 2. • After stopping, send the “stop camera” command to the master control system. Figure 2. Modification of billet conveyor control. During camera tests, it was found that it is not appropriate to scan billets in motion (Figure 3), especially because the effect of transmitted vibrations, which cannot be completely eliminated, significantly impairs the image quality. Much better results are achieved by scanning billets at rest. Based on these findings, it was necessary to stop the passing billet at a given distance from the camera before saving the image. Sustainability 2021, 13, x FOR PEER REVIEW 6 of 17 with regard to the production technology, optical possibilities of the camera, cable routing, and camera testing. A bracket for the camera cover was welded at the selected location, and a light reflector with a power supply terminated by a double metal socket with IP 55 protection was placed above the console. The camera was placed in the center of the conveyor approximately 30 cm above the passing billet, as shown in Figure 2. The scanning distance with respect to the required details and lens capabilities was determined by a camera test approximately one meter from the camera lens. Figure 2. Modification of billet conveyor control. During camera tests, it was found that it is not appropriate to scan billets in motion (Figure 3), especially because the effect of transmitted vibrations, which cannot be completely eliminated, significantly impairs the image quality. Much better results are achieved by scanning billets at rest. Based on these findings, it was necessary to stop the passing billet at a given distance from the camera before saving the image. Figure 3. A picture of a passing billet and a billet at rest from the Billet Stamping program. The rolling of the billet is controlled by a programmable logic controller (PLC) Simatic S7 300. This PLC controls the movement of semi-finished products on the charge grate and conveyors on which the semi-finished products are transported to the heating furnace. The PLC program for conveyor control had to be modified so that the billet stopped at the specified location, at this point the required time remained and continued to move to the heating furnace. The start command for the PLC is the “prepare billet” command from the main control system. The movement of the billet from the filling grate to the heating furnace takes place in the following steps, which are also shown in Figure 4. • Transfer of billet from charging grid to conveyor No. 1. • Start conveyor travel 1 and 2. • Stop the conveyor No. 1 and 2 for 5 s after signaling the sensor on conveyor No. 2. • After stopping, send the “stop camera” command to the master control system. Figure 3. A picture of a passing billet and a billet at rest from the Billet Stamping program. The rolling of the billet is controlled by a programmable logic controller (PLC) Simatic S7 300. This PLC controls the movement of semi-finished products on the charge grate and conveyors on which the semi-finished products are transported to the heating furnace. The PLC program for conveyor control had to be modified so that the billet stopped at the Sustainability 2021,13, 3851 7 of 18 specified location, at this point the required time remained and continued to move to the heating furnace. The start command for the PLC is the “prepare billet” command from the main control system. The movement of the billet from the filling grate to the heating furnace takes place in the following steps, which are also shown in Figure 4. •Transfer of billet from charging grid to conveyor No. 1. •Start conveyor travel 1 and 2. •Stop the conveyor No. 1 and 2 for 5 s after signaling the sensor on conveyor No. 2. •After stopping, send the “stop camera” command to the master control system. •Start the conveyor travel 1 and 2 after 5 s. Sustainability 2021, 13, x FOR PEER REVIEW 7 of 17 • Start the conveyor travel 1 and 2 after 5 s. The following new commands have been added to the sequence of commands in the PLC control program: • Travel stop after sensor signaling from conveyor No. 2. • After stopping, send the “stop camera” command. • After 5 s, start conveyor travel 1 and 2. All these changes were made in STEP 7 version 5.5. The billet stop sensor was used by the LT 100L and LR 100L in a light barrier with a PA 11B evaluation unit from Telco. This solution excels above all in its high vibration and dust resistance. Figure 4. System function diagram. Camera control and image storage were performed using the Axis Media Control SDK development software (version 6.32), which is available on the Axis camera manufacturer’s website. With the help of this software, a program was created to control the functions of IP cameras. The program was run on a camera server, which was equipped with enough memory to store images. The driver was created in Visual Studio 2010 express and C #. This programming language created a program with a simple graphical interface called Punching Billets, see Figure 5. The graphical interface contains several function keys that can be used to call up the respective camera and program functions. After starting the Billet Punching program and displaying the graphical interface, it was necessary to set the data for communication with the IP camera. These include the IP address of the camera, the login name, and password of the camera administrator, the type of video playback (with options between MJPEG, MPEG 4, and H.264), and the recording path. After pressing the Play Live button, the program establishes communication with the IP camera, verifies the login name and password, and displays the current image from the IP camera in the preview window [5,6]. Creating a snapshot was done by calling the Save current image function in Billet Stamping. The function creates an image in the camera and saves it according to the set path. This function is also available directly in the graphical interface on the SnapShot button. Several functions have been programmed using the graphical interface buttons, such as the already mentioned Play Live function, the Play File function which plays the video stored on the server, the Recording function which starts recording the image from the camera and saves the recording according to the set path, and the Stop button which activates the function to stop the current operation, i.e., play the current image or play the saved video. The possibilities of the program are very wide and allow you to control a large number of functions and control the connected camera. The described billet identification system mainly uses the image saving function. Other functions are mainly used to test the system. The Billet Punching program can also be controlled using commands and this option is used in the described image storage system. The camera server receives Figure 4. System function diagram. The following new commands have been added to the sequence of commands in the PLC control program: •Travel stop after sensor signaling from conveyor No. 2. •After stopping, send the “stop camera” command. •After 5 s, start conveyor travel 1 and 2. All these changes were made in STEP 7 version 5.5. The billet stop sensor was used by the LT 100L and LR 100L in a light barrier with a PA 11B evaluation unit from Telco. This solution excels above all in its high vibration and dust resistance. Camera control and image storage were performed using the Axis Media Control SDK development software (version 6.32), which is available on the Axis camera manufacturer’s website. With the help of this software, a program was created to control the functions of IP cameras. The program was run on a camera server, which was equipped with enough memory to store images. The driver was created in Visual Studio 2010 express and C #. This programming language created a program with a simple graphical interface called Punching Billets, see Figure 5. The graphical interface contains several function keys that can be used to call up the respective camera and program functions. After starting the Billet Punching program and displaying the graphical interface, it was necessary to set the data for communication with the IP camera. These include the IP address of the camera, the login name, and password of the camera administrator, the type of video playback (with options between MJPEG, MPEG 4, and H.264), and the recording path. After pressing the Play Live button, the program establishes communication with the IP camera, verifies the login name and password, and displays the current image from the IP camera in the preview window [5,6]. Sustainability 2021,13, 3851 8 of 18 Sustainability 2021, 13, x FOR PEER REVIEW 8 of 17 a command to save the image from the parent controller and invokes the image storage function in Billet Punching to create the image and save it according to the set path. The component of the Save Image command is the billet ID that is used for the image name. Saved images are accessible through the production information system to users with the appropriate permissions. Image archiving was set for at least two years. Figure 5. Graphical interface of Billet Stamping program. 3. Testing the Optical Recognition of Characteristics A partial goal of the solution was to test the optical character recognition. The commercial programs Abbyy Fine Reader version 12 and Free image OCR (Optical Character Recognition) were used for this purpose. These programs allow you to convert text from a photo to plain text format. Twenty images from the billet identification system were selected for testing. Selected images were tested in the mentioned programs. All tested images were in JPEG format and the programs were set to save in txt format [7]. In the first phase, the images were tested in their original colors (Figure 6), followed by black and white images (Figure 6), and in the final phase, the images were turned into negative colors (Figure 6). Each image was tested separately. The test results in these programs were insufficient. Changing the color spectrum also did not produce better results. The programs could not recognize the billet numbers from the tested images and also did not export satisfactorily to the txt format. The basic problem was the high error rate caused by the automatic vectorization of the OCR evaluation software. With automatic vectorization, the software must be able to determine the basic entities that make up an image based on the color information of each pixel. When applied to embossed characters on billets that contained many different OCR deformations, the software did not perform well. The introduction of pre-processing tools would improve this situation. Figure 6. Modification of images of billets from the Billet Stamping program. 3.1. Machine Vision Computer vision is one of the most advanced areas of computer technology. Used to recognize the required information from the image captured by the industrial camera, computer vision is widely used, especially in the industrial sector, where it is referred to as machine vision [8]. Figure 5. Graphical interface of Billet Stamping program. Creating a snapshot was done by calling the Save current image function in Billet Stamping. The function creates an image in the camera and saves it according to the set path. This function is also available directly in the graphical interface on the SnapShot button. Several functions have been programmed using the graphical interface buttons, such as the already mentioned Play Live function, the Play File function which plays the video stored on the server, the Recording function which starts recording the image from the camera and saves the recording according to the set path, and the Stop button which activates the function to stop the current operation, i.e., play the current image or play the saved video. The possibilities of the program are very wide and allow you to control a large number of functions and control the connected camera. The described billet identification system mainly uses the image saving function. Other functions are mainly used to test the system. The Billet Punching program can also be controlled using commands and this option is used in the described image storage system. The camera server receives a command to save the image from the parent controller and invokes the image storage function in Billet Punching to create the image and save it according to the set path. The component of the Save Image command is the billet ID that is used for the image name. Saved images are accessible through the production information system to users with the appropriate permissions. Image archiving was set for at least two years. 3. Testing the Optical Recognition of Characteristics A partial goal of the solution was to test the optical character recognition. The commercial programs Abbyy Fine Reader version 12 and Free image OCR (Optical Character Recognition) were used for this purpose. These programs allow you to convert text from a photo to plain text format. Twenty images from the billet identification system were selected for testing. Selected images were tested in the mentioned programs. All tested images were in JPEG format and the programs were set to save in txt format [7]. In the first phase, the images were tested in their original colors (Figure 6), followed by black and white images (Figure 6), and in the final phase, the images were turned into negative colors (Figure 6). Each image was tested separately. The test results in these programs were insufficient. Changing the color spectrum also did not produce better results. The programs could not recognize the billet numbers from the tested images and also did not export satisfactorily to the txt format. The basic problem was the high error rate caused by the automatic vectorization of the OCR evaluation software. With automatic vectorization, the software must be able to determine the basic entities that make up an image based on the color information of each pixel. When applied to embossed characters on billets that contained many different OCR deformations, the software did not perform well. The introduction of pre-processing tools would improve this situation. Sustainability 2021,13, 3851 9 of 18 Sustainability 2021, 13, x FOR PEER REVIEW 8 of 17 a command to save the image from the parent controller and invokes the image storage function in Billet Punching to create the image and save it according to the set path. The component of the Save Image command is the billet ID that is used for the image name. Saved images are accessible through the production information system to users with the appropriate permissions. Image archiving was set for at least two years. Figure 5. Graphical interface of Billet Stamping program. 3. Testing the Optical Recognition of Characteristics A partial goal of the solution was to test the optical character recognition. The commercial programs Abbyy Fine Reader version 12 and Free image OCR (Optical Character Recognition) were used for this purpose. These programs allow you to convert text from a photo to plain text format. Twenty images from the billet identification system were selected for testing. Selected images were tested in the mentioned programs. All tested images were in JPEG format and the programs were set to save in txt format [7]. In the first phase, the images were tested in their original colors (Figure 6), followed by black and white images (Figure 6), and in the final phase, the images were turned into negative colors (Figure 6). Each image was tested separately. The test results in these programs were insufficient. Changing the color spectrum also did not produce better results. The programs could not recognize the billet numbers from the tested images and also did not export satisfactorily to the txt format. The basic problem was the high error rate caused by the automatic vectorization of the OCR evaluation software. With automatic vectorization, the software must be able to determine the basic entities that make up an image based on the color information of each pixel. When applied to embossed characters on billets that contained many different OCR deformations, the software did not perform well. The introduction of pre-processing tools would improve this situation. Figure 6. Modification of images of billets from the Billet Stamping program. 3.1. Machine Vision Computer vision is one of the most advanced areas of computer technology. Used to recognize the required information from the image captured by the industrial camera, computer vision is widely used, especially in the industrial sector, where it is referred to as machine vision [8]. Figure 6. Modification of images of billets from the Billet Stamping program. 3.1. Machine Vision Computer vision is one of the most advanced areas of computer technology. Used to recognize the required information from the image captured by the industrial camera, computer vision is widely used, especially in the industrial sector, where it is referred to as machine vision [8]. Computer vision basically replaces human abilities in terms of understanding and interpretation of the image by technical means. Typical tasks in machine vision may be the recognition and counting of products using industrial cameras, positioning, dimensioning, or optical quality control. In conjunction with other product handling systems, some manufacturing operations can be fully automated using machine vision systems. The basis of a reliable machine vision system is suitable illumination of the scanned scene in which the scanned objects are located [ 9 ]. The image data obtained by a scanning device, such as a digital industrial camera, is then processed by the system to recognize the desired information. The system is connected to other control parts of the production line, as required, with which it communicates and can autonomously intervene in the production process or provide signaling to the operator. From automated interventions, we can consider stopping the machine, discarding defective and poor quality products, rotating parts in inappropriate positions, etc. Access to the system can also be enabled remotely via an application or web interface. Machine learning methods have been used to improve computer vision to identify embossed characters. There are many such methods, for example [ 10 – 15 ], which have certain limitations. In [ 16 ] off-line character recognition is presented, which is applied to the billet identification system in the steel industry. Identification characters are so damaged in this application that we need noise-resistant methods and robust character segmentation and extraction algorithms. We propose new local methods of adaptive thresholding and character extraction. We use a subspace classifier using KLT (Karhunen– Loève Transform—principal component analysis—robust pattern recognition method) to classify characters. The methods have been tested on real industrial applications and have proven to be successful and allow implementation in technological processes. Unlike the above publications, the proposed machine learning solution for the identification of embossed characters is unique to the metallurgical industry and works online in real-time. The basic difference is in the method of finding characters and thus in the identification of the whole string at the head of the billet and in the method of learning the character, where only selected pixels are used for learning and therefore not the whole pattern. This approach allows the chosen principle to be used to support real-time play and for control models that must be simple, fast, but also accurate. Another difference is that even though no filter is applied in the method when digitizing the billet face, the method is completely accurate. 3.2. Techniques to Reduce Character Reading Error An image may contain (and usually does) contain certain errors, whether it is the input device (camera), the surface of the material, or the imperfections of the characters themselves. Some of the main errors and ways to eliminate them are [17]: • Removal of stains—This is a basic process for reducing error rates. Spots are usually caused by the dirty surface of the sensor (camera). The most effective method to stop this error is prevention, by regularly cleaning the camera lens and general hardware scanning. Once this error is embedded in the resulting image from which the char- Sustainability 2021,13, 3851 16 of 18 Although the system enables auto-associative processing of embossed numbers, situations of misidentification occur (of the order of 0.01%), which are, however, caused by the non-embossing of the symbol in the identification chain and thus by the error of the previous technological process. 5. Conclusions This article presents a partial part of the machine vision system for automatic identification of the character string of billets on the rolling mill line. The system enables the conversion of image information from the front side of the billet into a character string, which is further used to control the technological process of rolling and to archive the process parameters. The identified character string here serves as a unique identifier of the billet. Currently, all parts of this control system are in full operating mode. The creation of this system was started on the basis of operational requirements, due to higher automation of operation and ensuring the control of steel billets according to the production plan. The machine vision system uses a camera to scan the area in front of the furnace and allows automatic inspection. The output of the described part of the system is digital information of numerical characters on the computer monitor of the operator’s station and enabling archiving of the obtained information. This reduces the operator’s workload and reduces manufacturing errors. The identified character string is the result of a system of an algorithm for automatically identifying the character string and individual characters and an algorithm for automatically digitizing individual characters. Heuristic algorithms for finding the identification chain in combination with multilayer artificial neural networks of the perceptron type with topology 69-38-42-10 were used for automatic identification of embossed numbers. A training set of photographs of the fronts of billets with identification strings was created for the learning process. The identification strings were divided into individual characters, thus creating a training set for the identification of individual characters. The training set for this testing contained 45 patterns in the original, black and white, and negative colors with numerical characters on the billets. The application of artificial neural networks in the field of image processing is not new, but the uniqueness of the proposed solution lies in the modification of this processing, which results in a significant simplification. The designed and implemented algorithm for automatic identification of embossed numbers does not suppress background noise, so it does not perform preprocessing and segmentation, which greatly simplifies the solution and speeds up the final processing while maintaining 100% classification of embossed numbers. These side effects are eliminated by the proposed neural network. The proposed system of automatic identification does not reduce the production cycle of technological processes, as the process is not continuous and provides enough time (approximately 2 min) for the entire processing. However, the proposed algorithm provides digital information in a matter of seconds and is therefore usable in continuous processes. Although the system enables auto-associative processing of embossed numbers, situations of misidentification occur (of the order of 0.01%), which are, however, caused by the non-embossing of the symbol in the identification chain and thus by the error of the previous technological process. The whole solution was done on real pictures. This solution is an example of the digitization of technological processes, which is reflected in sustainability in the social and economic field. This example of a solution also shows the possibilities of artificial neural networks and other methods of artificial intelligence, in general, to solve complex problems in industrial and non-industrial environments. The proposed solution is unique for the following reasons: - it is applied in a metallurgical environment, which shows large differences in digital image processing (impurities, lighting, heterogeneity of shapes and surfaces, specific temperature conditions, etc.); - the method can find the character string on the face of the billet; - the method can identify a character without the use of a filter in digital image processing; Sustainability 2021,13, 3851 17 of 18 - the method identifies the character, while only selected pixels of the character are used for learning, which enables real-time work; - the method is accurate and fast, which allows it to be applied to decision support, process management, or quality control systems; - the method works using standard hardware and software. This solution is an example of an innovative approach to the issue of digitization of technological processes, which is reflected in sustainability in the social and economic field. Author Contributions: Conceptualization, J.D., P.Š., V.P. and R.G.; methodology, J.D., P.Š., V.P. and R.G.; software, V.P.; validation, J.D., P.Š. and V.P.; formal analysis, R.G.; investigation, J.D. and P.Š.; resources, V.P. and R.G.; data curation, J.D. and V.P.; writing—original draft preparation, J.D., P.Š., V.P. and R.G.; writing—review and editing, J.D. and P.Š.; visualization, J.D., P.Š. and V.P.; supervision, J.D., P.Š., V.P. and R.G.; project administration, J.D. and P.Š.; funding acquisition, J.D. All authors have read and agreed to the published version of the manuscript. Funding: The work was supported by the specific university research of Ministry of Education, Youth and Sports of the Czech Republic No. SP2020/18, SP2020/61 and SP2021/23. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: Not applicable. Conflicts of Interest: The authors declare no conflict of interest. References 1. David, J.; Švec, P.; Frischer, R. Support for maintenance and technology control on slab device of continuous casting. In Proceedings of the Metal 2013: 22nd International Conference on Metallurgy and Materials, Brno, Czech Republic, 15–17 May 2013; pp. 1650–1655. 2. Šonka, M.; Hlaváˇc, V.; Boyle, R. Image processing, Analysis, and Machine Vision, 3rd ed.; Thomson: Toronto, ON, USA, 2008; p. 829. 3. Horák, K.; Kalová, I.; Petyovský, P.; Richter, M. PoˇcítaˇcovéVidˇení, 1st ed.; VysokéUˇceníTechnickév Brnˇe: Brno, Czech Republic, 2008; p. 132. 4. Bovik, A. Handbook of Image and Video Processing, 2nd ed.; Academic Press: London, UK, 2000; p. 891. 5. Szeliski, R. Computer Vision: Algorithms and Applications, 1st ed.; Springer Publishing: New York, NY, USA, 2011; p. 812. 6. Forsyth, D.A.; Ponce, J. Computer Vision: A Modern Approach, 2nd ed.; Pearson: London, UK, 2011; p. 800. 7. Seidl, D.; Koštial, P.; Janˇcíková, Z.; Ružiak, I.; Rusnáková, S.; Farkašová, M. Modal analysis—Measurements versus FEM and artificial neural networks simulation. Commun. Comput. Inf. Sci. 2011,188, 170–175. 8. Pujari, P.; Karim, M.R.; Sewak, M. Practical Convolutional Neural Networks, 2nd ed.; Packt Publishing—ebooks Account: Birmingham, UK, 2018; p. 218. 9. Janˇcíková, Z.; Koštial, P.; Bakošová, D.; Seidl, D.; David, J.; Valíˇcek, J.; Harniˇcárová, M. Neural Model for Prediction of Tires Eigenfrequencies. Adv. Struct. Mater. 2013,41, 1–11. 10. Rui, W.; Dong, A. The Method of Pattern Recognition based on Weighted Intuitionistic Fuzzy Relative Entropy. In Proceedings of the 2013 Fourth International Conference on Digital Manufacturing and Automation (ICDMA), Qingdao, China, 29–30 June 2013; pp. 1487–1489. 11. Yang, P.; Xing, K.; Huang, J. A Novel Feature Reduction Method for Real-Time EMG Pattern Recognition System. In Proceedings of the 2013 25th Chinese Control and Decision Conference (CCDC) 2013, Guiyang, China, 25–27 May 2013; pp. 1500–1505. 12. Zhang, J.; Liang, L. New Pattern Recognition Method based on Wavelet De-Noising and Kernel Principal Component Analysis. Appl. Mech. Mater. 2012,235, 74–82. [CrossRef] 13. Yuan, X.; Wu, X.; Wei, X. A Pattern Recognition Method Using Parallel Pyramid Neural Networks. In Proceedings of the 2011 International Conference on Computers, Communications, Control and Automation (CCCA 2011), Hokkaido, Japan, 1–2 February 2011; Volume I, pp. 284–287. 14. Valdez, F.; Melin, P.; Castillo, O. Fuzzy Logic Adaptation of a Hybrid Evolutionary Method for Pattern Recognition. In Proceedings of the Joint 2009 International Fuzzy Systems Association World Congress and 2009 european society of Fuzzy Logic and Technology Conference 2009, Lisbon, Portugal, 20–24 July 2009; pp. 815–820. 15. Yu, J.; Zheng, X.; Wang, S. A deep autoencoder feature learning method for process pattern recognition. J. Process Control 2019 ,79, 1–15. [CrossRef] 16. Kandel, E.R. Principles of Neural Science, 5th ed.; McGraw-Hill: New York, NY, USA, 2012; p. 1504. 17. Theodoridis, S.; Koutroumbas, K. Pattern Recognition, 4th ed.; Academic Press: Cambridge, MA, USA, 2008. Sustainability 2021,13, 3851 18 of 18 18. Rice, S.V.; Nagy, G.; Nartker, T.A. Optical Character Recognition: An Illustrated Guide to the Frontier; Springer: New York, NY, USA, 1999. 19. David, J.; Frischer, R.; Stranavova, M. Usage of RFID wireless identification technology to support decision making in steel works. In Proceedings of the Metal 2012: 21st International Conference on Metallurgy and Materials, Brno, Czech Republic, 23–25 May 2012; pp. 1734–1738. 20. Frischer, R.; David, J.; Svec, P.; Krejcar, O. Usage of analytical diagnostics when evaluating functional surface material defects. Metalurgija 2015,54, 667–670.