Custom automotive grade Linux image for production diagnostics

Universidade do Minho Escola de Engenharia Eduardo João Fernandes de Sousa Custom Automotive Grade Linux image for Production Diagnostics julho de 2022 UMinho | 2022 Eduardo Sousa Custom Automotive Grade Linux image for Production Diagnostics Eduardo João Fernandes de Sousa Custom Automotive Grade Linux image for Production Diagnostics Dissertação de Mestrado Engenharia Eletrónica Industrial e Computadores Sistemas Embebidos Trabalho efetuado sob a orientação do Professor Doutor Paulo Cardoso Universidade do Minho Escola de Engenharia julho de 2022 COPYRIGHT AND TERMS OF USE OF THIS WORK BY A THIRD PARTY This is academic work that can be used by third parties as long as internationally accepted rules and good practices regarding copyright and related rights are respected. Accordingly, this work may be used under the license provided below. If the user needs permission to make use of the work under conditions not provided for in the indicated licensing, they should contact the author through the RepositoriUM of Universidade do Minho. License granted to the users of this work Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International CC BY-NC-SA 4.0 https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en iv Acknowledgements I would like to thank my advisor, Professor Paulo Cardoso, for helping me all the way through the writing process this dissertation. I would also like to express my gratitude towards Bosch Car Multimedia for taking me in as an intern and for letting me use their offices and technologies. I would like to give special thanks to my colleagues of Bosch Car Multimedia for teaching me, for their extensive help through the development and spending their time with me when needed. A special thanks to Miguel Azevedo, my project superviser at Bosch Car Multimedia, for passing his knowledge onto me. A huge thanks to my family for supporting me in every decision along the way, for giving me the tools that made me what I am and for always picking me up when I went down. A very big thank you to my girlfriend for helping me through the tough times this dissertation presented and for always being there for me. To all sincerely, thank you! v STATEMENT OF INTEGRITY I hereby declare having conducted this academic work with integrity. I confirm that I have not used plagiarism or any form of undue use of information or falsification of results along the process leading to its elaboration. I further declare that I have fully acknowledged the Code of Ethical Conduct of the Universidade do Minho. vi Resumo Os sistemas presentes nos automóveis de hoje em dia, têm vindo a tornar-se cada vez mais complexos conforme as tecnologias e as preferências dos consumidores têm evoluído. Os sistemas, tais como consolas centrais ou paineis de instrumentos, têm o papel não só de informar os passageiros como também de entertenimento e ajuda na navegação. Estas e outras capacidades traduziram-se numa maior complexidade destes sistemas, tanto a nível do software como do hardware . A maior parte dos fornecedores de peças e equipamentos eletrónicos para a industria automóvel têm a necessidade de acelerar o processo de testes dos seus produtos, para poderem acompanhar a procura mantendo a qualidade. Para este fim, software de testes é usado nos sistemas desenvolvidos com o objetivo de testar o hardware e software desenvolvido para o cliente. No software de testes incluí-se o Production Diagnostic Software (PDS) . A solução de PDS atualmente utilizada pela Bosch é composta por uma pilha de software, QNX a correr em paralelo com AUTOSAR da Vector. O AUTOSAR é um Real-Time Operative System (RTOS) usado como ponto de entrada dos comandos de diagnostico e oferece acesso ao hardware e à interface de comunicação. O QNX é um sistema UNIX de alto nível que lida com dispositivos externos, como ecrãs e audio. O Automotive Grade Linux (AGL), em comparação com outros sistemas operativos direcionados à indústria automóvel, é mais recente e apresenta software totalmente open-source . Devido a isto, apresenta custos reduzidos quando comparado com soluções closed-source e um maior grau de personalização em termos de código. Este projeto de dissertação implementou uma prova de conceito de um PDS usando o AGL como o unico sistema operativo, com o objetivo de competir e substituir, totalmente ou parcialmente, a pilha de software existente na solução atual da Bosch. A conclusão deste projeto de dissertação é que o AGL não consegue substituir a pilha de software devido à falta de qualidades real-time por parte do AGL. Mas, por outro lado, é um possível candidato a substituir o QNX na pilha de software, pois o AGL apresenta as mesmas capacidades que o QNX a menor custo e com maior personalização. Palavras-chave: Automotive, PDS, AGL, Diagnostico vii Abstract The systems in today’s cars have become increasingly complex as technology and consumer preferences have evolved. Systems, such as center consoles or instrument clusters, have the role of not only inform passengers but also to entertain and aid navigation. These and other capabilities increased the complexity of these systems, in terms of software and hardware. Most suppliers of electronic parts and equipment to the automotive industry have the need to speed up the process of testing their products so that they can meet demand while maintaining quality. For this purpose, test software is used with the objective of testing the hardware and software developed for the client. Test software includes Production Diagnostic Software (PDS). The PDS solution currently used by Bosch is composed of a software stack, QNX running alongside Vectors AUTOSAR. AUTOSAR is a Real-Time Operative System (RTOS) used as the entry point for diagnostic commands and provides access to the hardware and communication interface. QNX is a high-level UNIX system that handles external devices, such as displays and audio. The Automotive Grade Linux (AGL), compared to other Operative Systems targeted at the automotive industry, is newer and features a open-source approach. Because of this, it offers lower costs when compared to other closed-source solutions and a higher level of customization of code. This dissertation project implemented a proof-of-concept of a PDS using AGL as the only operating system, with the aim to compete and replace, totally or partially, the existing software stack in the current Bosch solution. The conclusion drawn from this dissertation project is that AGL cannot replace the software stack, due to it’s lack of real-time capabilities. On the other hand, it is a possible candidate to replace QNX in the software stack, for the reason that AGL has the same capabilities at a lower cost and higher customization level. Keywords: Automotive, AGL, PDS, Diagnostic viii Contents List of Figures xi List of Tables xii Acronyms xiii 1 Introduction 1 1.1 Context...................................... 1 1.2 Motivation .................................... 2 1.3 Objectives .................................... 2 1.4 DissertationStructure............................... 2 2 Background 4 2.1 Production Diagnostics Software . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1.1 Unified Diagnostics Service . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Support Software for PDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.1 Current Bosch solution . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.2 Automotive Grade Linux . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 PDS application specification 11 3.1 Problemanalysis ................................. 11 3.1.1 Diagnostics Behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.1.2 BlockDiagram.............................. 14 3.2 ApplicationDesign ................................ 15 3.2.1 Entities Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.2 Diagnostics ............................... 21 4 Implementation and Tests 25 ix CHAPTER 1. INTRODUCTION diagnostics. The two OSs are running in different processors in the SoC and the communication between them is made through a custom inter-processor communication protocol, this is what synchronizes the whole system and makes it function as a single system. The inter-processor communication is used to send the messages received via CAN on AUTOSAR to the application running in QNX, and also to send responses from the diagnostics to AUTOSAR. This solution presents some problems, due to its architecture and implementation. One of the problems is the usage of the same diagnostics engine for the production tests and the end user. A diagnostics engine is what interprets the UDS messages and executes the diagnostics, and the current PDS uses the same engine for both end user application and production diagnostics application. Since the engine is shared between use cases, it is very complicated and the majority of its functionalities go unused in the PDS application. Other problems are the non removal of the PDS and the utilization of two OS. The utilization of two OS increases the complexity of the system and makes developing for the system harder. 1.2 Motivation A current trend in the automotive industry is the use of open-source software, this led to the development of the Automotive Grade Linux (AGL) in recent years to standardize the software present in the devices and accelerate the development of new applications. The main motivation in this dissertation is the exploration of AGL as a way to produce a custom environment for PDS. 1.3 Objectives This dissertation, besides exploring a different OS, creates a PDS application that tackles some of the problems with the current solution. The main objective of this dissertation is the exploration of AGL as a potential platform for PDS applications. For this end, it is developed a proof-of-concept system of a PDS application running in AGL, with the aim of testing and exploring this new OS. 1.4 Dissertation Structure In this dissertation, the reader is presented with the Background chapter, which explains all the topics involved in the implementation of the project, allowing to understand the needed technologies and the subjects to be used or that help development. The following chapter, PDS application specification, gazes upon the analysis of the system to be implemented, it’s requirements and how this dissertation tries to tackle the problems presented. But also, the design, choices made to make the project happen and the decisions taken to make the project function on paper. 2 1.4. DISSERTATION STRUCTURE The Implementation and Tests chapter comes next and uncovers the implementation of the project. The chapter closes with tests and tools used to test the system to then evaluate the success of the developed project. Finally, this dissertation ends with the Conclusion, where the final solution is warped up and summarized, terminating with the necessary work to perfect it and publish. 3 2 Background In this chapter, the concepts used by this dissertation are explained, starting on more abstract concepts and narrowing it down to the core of the problem presented in this dissertation. Starting with a definition of Production Diagnostic Software (PDS) and Unified Diagnostic Services (UDS), and ending with the current solution used by Bosch and the new emerging Automotive Grade Linux (AGL) OS. 2.1 Production Diagnostics Software Production Diagnostic Software is software designed to automate the tests necessary to validate the hardware before it is integrated in the final device [1, 2]. Test automation is a way to simplify and accelerate the diagnostics of faulty hardware and it is essential to realize time-sensitive action sequences [3]. With the increase in market demand, the PDS and its automation assume a greater relevance given the need to increase the testing cadence while maintaining the level of robustness of the tests [2]. Diagnostic software is software created and used to retrieve fault information stored in an ECU, this information will then be used to diagnose failures in the hardware [1]. This software follows the Unified Diagnostic Services (UDS) standard to support the communication part of the test system and the definition of diagnostic. Next, can be visualized the Figure 1 with a example EOL test system. Figure 1: Typical test setup for EOL tests 4 2.1. PRODUCTION DIAGNOSTICS SOFTWARE 2.1.1 Unified Diagnostics Service Unified Diagnostic Services (UDS) is a communication protocol developed for the automotive industry, specified by the ISO 14229-1 [4], to define common diagnostics practices independently of company or car. For that purpose the UDS standard uses as a base the Open Systems Interconnection (OSI) basic reference module, which divides the communication systems into seven layers. At the top layer, or application layer, is where UDS resides. The standard also defines all the values for the information bytes, values for errors, error states and program sequences to be implemented. The UDS standard specifies that all the diagnostics in a car are comprised of diagnostic services, that are predefined by the standard and have a Service ID (SID), and their respective parameters and Routine ID (RID), if available [4]. A diagnostic service, as per the specification, is an exchange of information started by a client in order to retrieve relevant information from a server. Here, a client is the tester or the equipment that sends the diagnostic messages and the server is the hardware being tested. The UDS specifies a multitude of diagnostic services. Some of the services specified are SessionControl, ECUReset, DiagnosticSessionControl, RoutineControl. The UDS standard, not only defines the available services but also the message structure and a plethora of codes used in the response messages. The request messages are defined as starting with the SID, next the Service Sub function (if applicable), then the Diagnostic ID, and ending with the Diagnostic parameters, as Figure 2 shows. The response is similar to the request message, it is specified that if the execution of the diagnostic is correct the response shall start with the SID value plus 40 in hexadecimal (i.e. for the SID 0x31, the positive response will be 0x31+0x40 = 0x71), this is followed by the Diagnostic ID and by the data, as Figure 2 demonstrates. For the negative responses, the specification created a special SID called Negative Response Code (NRC) ID used for the purpose of sending negative responses to the tester. This NRC ID as the value of 0x7F, and in the negative response format, is followed by the SID that was called and ends with the negative response code, as Figure 2 shows. Figure 2: UDS messages format 5 CHAPTER 2. BACKGROUND For this dissertation, the most important service specified by the standard for diagnostics purposes is the RoutineControl service. The RoutineControl service is used by the tester to execute a sequence of defined steps and obtain results. It is a service of high flexibility, since the implementation of the steps is defined by the project requirements and it is used for complex control sequences. The Routine Control is defined as having an ID of 0x31h and having one parameter to control if the routine is start, stop or request result, thus giving control to the tester if he wants to start, stop or receive a result from a given routine service. Table 1 helps visualize the request message for a RoutineControl service. The message contains the SID for RoutineControl, and then, the RoutineControlType variable that controls the type of execution, either start (0x1h), stop (0x2h) or request result (0x3h). After that comes the RID, that defines the sub-function to execute in the diagnostics program and has a two byte length ID, the next part of the request message is optional depending on the need for execution parameters or not and it is the parameters section, every parameter is one byte. Data byte Parameter Name Value #1 Service ID 0x31h #2 RoutineControlType 0x1h, 0x2h, 0x3h #3 #4 Rotuine ID [] = [MSB, LSB] 0x0h - 0xFFFFh #5 - #n Optional Parameters[] = [param1,..., paramN] 0x0h - 0xFFh ... 0x0h - 0xFFh Table 1: Routine Control Request message format [4] 2.2 Support Software for PDS The PDS, as with all applications, runs on a OS and has software that supports its implementation and functionality. For this end, it is usually developed a software stack composed of two OSs. Currently, at Bosch, the software stack is composed of QNX on top of AUTOSAR. Next subsection gives a brief explanation and insight of this solution. This subsection also shows some technical concepts and state of the art about AGL, as a newly emerging automotive focused OS. 6 2.2. SUPPORT SOFTWARE FOR PDS 2.2.1 Current Bosch solution As explained in the contextualization, the current solution, as seen in Figure 3, uses two OS running in different processors on the SoC and are connected via a custom inter-node communication, or INC, used for communication between processors. The two OS are Vector AUTOSAR and QNX, and each have their own tasks. The AUTOSAR deals with hardware directly, offers implementation of device drivers, communication interfaces, offers abstraction from hardware, implements the majority of the communication part and it’s responsible for the main state machine that controls the flow of execution of the diagnostics. AUTOSAR also has a diagnostic engine that it is native to the OS. QNX offers access to a file system, control of displays and audio and a custom diagnostic engine to execute UDS diagnostics. Figure 3: Block Diagram of Bosch Solution The current booting process is composed of two bootchains A and B, each containing two instances of AUTOSAR and QNX. The bootchain B is used for production line tests and the other for the end user software. The control of what bootchain is booted is done by the bootloader, through a custom boot flag. By default, the boot flag enables the bootchain B. The problem with this solution is that, when the tests are done, the tests bootchain is not deleted but is, however, deactivated through the change and subsequent locking of the boot flag to boot the end user bootchain. The current solution uses the Vector AUTOSAR for the communication and diagnostics part of the UDS specification, and QNX only for specific diagnostics. On the QNX part of the system, the diagnostics 7 CHAPTER 2. BACKGROUND engine used for the production tests is the same as the one that it is used for the end user software. This diagnostics engine, called MicroCore, has more functionalities implemented than what is needed for production tests, so, it presents the developer with high complexity. Also, the MicroCore does not have any communication capabilities apart from the inter-node communication, or INC, used for communication between processors, and no simple way of adding more communication interfaces. The flow of execution of the current solution is as follows. Upon receiving a UDS message in the AUTOSAR communication interface, the AUTOSAR application parses the message and checks if the diagnostic exists and if the diagnostic is supposed to run on AUTOSAR or QNX. If the diagnostic is supposed to run on QNX, the UDS message is sent, through the inter-node communication interface, to QNX’s MicroCore application. Afterwards, MicroCore upon receiving the message proceeds to parse it and execute the diagnostic. Upon finishing the diagnostic execution, MicroCore sends the response, via the inter-node communication to AUTOSAR, that then sends the response to the CAN communication bus. As with all known systems, the system used by Bosch currently contains problems. The largest problem currently is the high complexity of the architecture. The architecture as it stands is the same as the end-user architecture, which for production diagnostics is highly complicated and goes mostly unused. The high complexity also means more development time for new features and a less flexible system. Other problem with the current system is the usage of closed-source software, which limits the flexibility of the system, brings with it high licensing costs and the access to the code is limited. 2.2.2 Automotive Grade Linux Automotive Grade Linux (AGL) is an open-source operating-system targeted to the automotive industry based on the Linux kernel. The main focus of AGL is to create OSs for infotainment systems and a framework to build automotive software applications [5]. Automotive software dates back to the 1970s where systems where programmed in assembly and only in the 1990s, the C language became a industry standard for systems programming, but with the introduction of new technologies and increased complexity, software became complex too and the creation of AUTOSAR and recently AGL came to help with the problem [6, 7]. Since the complexity of cars tends to increase, the need for a more complex and robust OS and hardware to keep up with the increased demand for safer, faster and reliable automotive systems. Recent production vehicles can have upwards of 100 million lines of code across all systems and take 2 to 4 years to develop, compared to smartphones with only 12 to 15 million lines of code [6]. Nowadays automakers are using Windows Embedded Automotive, QNX or Ubuntu for the OS of the systems, but, each car on the lineup can use a different customization of an OS. AGL comes to fix this lack of standardization [8]. AGL became available in 2012 [8], but some articles from 2005 already refer to some automotivegrade Linux, not AGL but a concept or idea of a Linux distribution for automotive [9]. The main concept of the AGL is that the OS acts as a framework that can be used by OEMs and suppliers to make their own versions by integrating their own modules. The main target applications for 8 2.2. SUPPORT SOFTWARE FOR PDS AGL are telemetry, HMI and instrument clusters. The base building blocks can also be reused from other Linux embedded distributions due to the similar kernel [8]. The AGL uses Unified Code Base (UCB) and the ”code first”approach to reduce time-to-market for the product and does this by implementing 70-80% of the necessary software bases for the supported boards, thus reducing the workload to the developers to only 20-30%, which consists of specific application software and customization [10]. AGL in the real world AGL has already been used in the automotive world, with some new cars infotainment being totally developed with this OS, as it is been showed, the Toyota Camry was the first Toyota car in the United States market with an infotainment system totally developed in AGL [5]. Other proof-of-concepts have been made with this OS, one of them presented in [7], which goes into the concept of connected vehicles by binding adaptive AUTOSAR with AGL. The author goes into exploring the usage of well documented and well implemented AUTOSAR to develop the low level of the project, but developing the whole Graphical User Interface (GUI) with AGL. The benefit is that the AUTOSAR software can focus activities into secure software and time sensitive networks, while AGL focuses on GUI. More uses of AGL in the automotive world were explored by papers such as [11]. In this paper the authors develop AGL applications for Advanced Driver Assistance Systems (ADAS) using Linux as a basis, but refer AGL as a candidate to develop ADAS with the use of a GPU for acceleration. The authors also mention the lack of reports on the real-time performance of such systems developed with AGL. AGL software architecture AGL presents the user with 4 layers in its software architecture [8]. The top layer, and most abstract, is the application/HMI layer. The layer is composed of the things the driver and passengers can interact with, mainly the infotainment apps, radio, GPS, etc. The next layer in the architecture is the application framework, this layer provides all the necessary Application Programming Interface (API) to develop applications. The layer bellow is the service layer which contains all the user-space resources and automotive services. At the bottom of the stack, the OS layer includes the Linux kernel and all the device drivers. 9 CHAPTER 2. BACKGROUND Figure 4: AGL software architecture [8] 10 3 PDS application specification 3.1 Problem analysis To validate AGL as a possible platform for PDS applications, it is intended to use a custom developed PDS application. This PDS application must contain different diagnostics that will be used to test the viability of AGL, as well as, simulate a real PDS application in the real world. As requirements for this application the diagnostics to be conducted need to mimic the Bosch solution including using the UDS specification. The communication must use CAN protocol. The set of diagnostics that are going to be implemented, as a proof-of-concept, seen in Table 2, are a subset of the tests performed by the current Bosch solution. Diagnostics Read MAC Address Read embedded MultiMediaCard (eMMC) Card Identification (CID) Play/Stop Audio Read/Write General Purpose Input Output (GPIO) pin Read/Write Memory Display Test Picture Display Generated Image Table 2: Diagnostics to be implemented The diagnostics to be implemented are presented in the next paragraphs. Read MACAddress This diagnostic reads the MACAddress of the main Ethernet interface and should return error either if the Ethernet interface is not configured or reading its MACAddress fails. 11 CHAPTER 3. PDS APPLICATION SPECIFICATION For the storage of diagnostics it is used the C++ class map. A map is a specific table that can store and retrieve content using a ID or key. The content and key can be whatever data structure the developer wants (i.e. strings, classes, integers, etc.), this enables high flexibility and the ability to use complex data structures, either for keys or content. For this module the map is used because of its properties, such as, unique keys (no two elements can have the same key), self dynamic memory allocation and associative nature. Also, it enables the storage of diagnostics by using a custom data structure for the map key. This custom data structure is called UDS ID and it is composed by the SID and DID. This was done to make searching for diagnostics easy, by just searching its SID and DID. For the insertion of elements in the Diagnostics map a COTS algorithm and design, previously developed by Bosch, is used. This works by compiling the diagnostics as a shared object library and storing them in a directory inside the target hardware filesystem. The libraries are then fetched in run-time by a algorithm that creates a pointer to the class in the map and an association is then made between the Diagnostic ID and the Diagnostic by using configuration text files, that associate the ID with the Diagnostic Name. The flowchart for the storage of diagnostics can be seen in the following Figure 10. Figure 10: Diagnostic Lookup Store diagnostic flowchart For searching and retrieving the diagnostic, it is used the UDS ID and the map function find, this 18 3.2. APPLICATION DESIGN function takes in the map key as a parameter (in this case the UDS ID) and returns an iterator to the specific map position where the key was found. This iterator contains the key and the value it stores, in this case the value will be the diagnostic. The map will be encapsulated in a class, that contains a find and a store function, and the UDS ID structure and the map as class members. The store function will be used on at the start of the application to store all the available diagnostics. The find function will be the one used by the scheduler to retrieve the diagnostic. The searching and get algorithm can be visualized further with the flowchart presented next in the Figure 11. Figure 11: Diagnostic Lookup Get diagnostic flowchart Diagnostics Processor This entity is responsible for the retrieval and execution of diagnostics. This entity is divided into two parts, a synchronously called part for retrieving the diagnostics and a asynchronous part for executing diagnostics. The asynchronous part is where the diagnostics are called to execute and where the response 19 CHAPTER 3. PDS APPLICATION SPECIFICATION is sent after their execution is terminated and it is totally independent of the rest of the application. The synchronous part retrieves the diagnostic from the table and places it in a queue to later be executed. The design of this module uses custom algorithms for both the synchronous and asynchronous parts. For the synchronous part, the algorithm receives the diagnostics IDs and parameters from the Message Processor and receives the corresponding diagnostic from the Diagnostics Lookup table entity. The algorithm for the asynchronous part uses a thread and a queue with the First In First Out (FIFO) method of organization maintaining the events in the same order of reception. When this part enters execution it retrieves the diagnostic from the queue and executes it. As the Figure 13 shows, the thread starts by checking if the queue of events is empty, if it is not, it pops the top of the queue and proceeds to call the diagnostic execution synchronously. After the diagnostic executes, its response will be sent to the communication part to later be sent via the communication bus. After all this, the thread executes again from the top of the algorithm. Figure 12: Diagnostics Processor synchronous part flowchart Figure 13: Diagnostics Processor thread flowchart 20 3.2. APPLICATION DESIGN 3.2.2 Diagnostics This entity is based on a COTS design that is already used by Bosch and it is based on UDS protocol specifications for diagnostics and services. This design consists of a hierarchy of classes, presented on Figure 14. It is based on a fully abstract class that represents the concept of Service in the UDS protocol specification. This class is inherited by different classes that represent the different Services available and each Service can, in turn, be inherited by several classes that implement different diagnostics. Figure 14: Diagnostics Class diagram Diagnostics Specification Each diagnostic class, referred above and in Figure 14, will equate to the diagnostics to be executed by the system, referred on table 2. Read MACAddress This diagnostic reads the MACAddress of the Ethernet device present on the board and retrieving that information to the tester. This diagnostic will take leverage of the Linux method of presenting devices as files. For this diagnostic it will be used COTS functions that are available in the programming language, to read the file that contains the MACAddress of the main Ethernet device of the board. Read eMMC CID For this diagnostic it was designed a custom algorithm that takes advantage of the Linux way of showing devices as files in the filesystem. The algorithm for this diagnostic works by searching a specific file on the sys folder of Linux, this file is called ”cid”and it contains the CID value for the specific eMMC card. This diagnostic on call will proceed to open the cid file and read its contents. Later it parses the contents to the response buffer to be sent to the tester. The response of this diagnostic will contain a 16 byte value of the register. The CID is a special register on every eMMC device that contains relevant manufacturer data. 21 CHAPTER 3. PDS APPLICATION SPECIFICATION Play/Stop Audio This diagnostic uses a COTS library, namely the ALSA library, for controlling the audio device. The ALSA library, or Advanced Linux Sound Architecture library, offers APIs to control the sound devices of a Linux system, it can be used to control volume or send specific values to the DAC. For this diagnostic it will be used the PCM interface from ALSA library. This diagnostic is composed of two actions, a start play audio and a stop audio. For this the parameter of the Routine Control Service will be used. For the start play audio, the diagnostic works by opening and configuring the target device first. Next is setting the volume given by the parameters. Afterwards it is calculated a sine wave with the frequency parameter and the values are saved to a buffer. After all the setup is complete the diagnostic proceeds to create a thread to send the values of the sine wave to the device by using the ALSA API function snd_pcm_writei. A thread is used to enable the output of sound without blocking the system execution and to output sound until a stop command arrives. For the stop audio, the diagnostic works by signaling the thread to stop, using an atomic variable, this makes the thread finish sending the pending values and stop its execution. It then ends by cleaning up the configurations made to the device and closing all the open devices. Read/Write GPIO For the design of this diagnostic it will be used a COTS API for Linux called libgpiod. This library offers an abstraction layer from the ioctl functions to control the GPIO pins of a board. For this diagnostic it will only be used the start function of the Routine Control Service. The parameters for the read, as explained in the previous chapter, are the GPIO channel number and the pin number. The parameters for the write are the same as the read with one extra, the value to set the pin to, which is either 0 or 1 which translates to either high or low. The way the reading diagnostic works is by first opening the gpio chip whose number is received in the parameters by using the libgpiod APIs, afterwards, it is opened the pin by using the number received in the parameters. If all the processing is done correctly and does not raise error, the diagnostic shall return the value that the specified pin in the specified gpio chip is set at. For the writing the diagnostic sends the channel, pin and value to the GPIO driver. The response should contain the value the tester sent. Read/Write Memory This diagnostic is mostly used to read/write values from certain addresses on the RAM. For this the read receives the size of the values (8, 16 or 32 bit) to be read and the memory address. The write receives the size, the memory address and the value to write. 22 3.2. APPLICATION DESIGN The design of the read memory diagnostic is simple, basically it starts by opening the system RAM file descriptor in read mode and mapping its address to a pointer. Afterwards it just basically reads the value of the pointed address and passes it to a byte array. For the write memory diagnostic, it also opens the system RAM file descriptor, but, in write mode. It maps the address to a pointer and then simply proceeds to copy the received value from the parameters to the values of the pointed address. All of the operations for opening, mapping, reading and writing will be encapsulated in a custom class to help the portability of the code and to not repeat code, since this is used in two diagnostics. Display Test Picture This diagnostic is responsible for reading the pictures inside a specified folder in the filesystem, index them, process them and show them accordingly in the display connected to the development board. The chosen file type for the pictures is Portable Network Graphics (PNG), since there are COTS libraries available to process this file type in Linux, and for this diagnostic the libpng and png++ wrapper are used. The libpng is the reference library for PNG manipulation and supports almost all PNG features. The png++ is a C++ wrapper interface for the libpng that offers better C++ compatiblity and reduces complexity of data manipulation and handles initialization/deinitialization of structures. This COTS modules are used due to the immediate availability and ease of integration they presented. For storing the pictures in the filesystem it is used a specific folder to only contain PNG pictures. The diagnostic will read the pictures in the directory and index them in a array. To make it possible to show the same picture every time the same index is received, the pictures are stored in a orderly fashion by using their file name to store the index number (i.e. ”1-TestPicture.png”). When a diagnostic start is called the diagnostic will search for the specified picture index, retrieve it and process it to convert from PNG to raw binary data buffer. The conversion to raw is done through the png++ wrapper functions. After the image is transformed from PNG to raw binary, the raw binary buffer is sent to the display driver to be displayed in the screen. Display Generated Image This diagnostic is responsible for taking the position of two vertices of a quadrilateral and display it on the screen, with a background color and a color for the quadrilateral. The diagnostic will receive a 3 bytes for the RGB values of the quadrilateral, followed by 3 bytes for the RGB values of the background. The vertices are given as pairs of x and y values of pixels, each x and y is given by 2 bytes each, forming a value like 0xXXXX 0xYYYY for one vertex. The following Figure 15 helps visualize this idea. For this diagnostic, the design uses a COTS code developed by Bosch to convert the vertices position to a final image. Since the vertices are received as the absolute position on the screen, the calculations involved are not complicated. The algorithm works by representing the screen as a matrix with the size of the screen in pixels, each coordinate in the matrix containing the RGB value of the pixel. 23 CHAPTER 3. PDS APPLICATION SPECIFICATION Figure 15: Display Generated Image diagram On execution, the algorithm, first, fills the matrix with the background color values, afterwards, it goes through the whole matrix and checks if the coordinates are between the x values and the y values. If the previous condition is true, the algorithm fills that pixel with the RGB values for the quadrilateral. Upon finishing filling the matrix the algorithm then converts the matrix to a single dimension array to be dispatched to display driver to display the final image on the screen. 24 4 Implementation and Tests This chapter discusses the aspects of the implementation, as well as, the validation tests of the developed application. First it is shown the implementation of the application and its entities. Next, the configuration of the AGL as a platform for the application. This chapter ends with the tests made on this dissertation project. The testing section shows the testing environment used and what tests were conducted on the project. This enables one to gauge the feasibility of the project when deployed on real hardware, and draw conclusions. 4.1 PDS Application The setup used to develop and implement this system was a computer running Ubuntu Focal Fossa, since it’s lightweight and offers a lot of flexibility and documentation. For the programming language, it was used C++ version 14 since it is supported by AGL, it is very powerful and flexible and one of the most used programming languages for embedded systems. To implement the application, in accordance with the requirements, the solution reached was a core shared library and executable, with different modules implemented through shared libraries and fetching of those plugin libraries in runtime. This solution enables modularity since only the shared library for the plugin needs to be compiled to be added to the program. This modularity works by compiling the core modules into the executable and its library, and compiling the diagnostics and communication plugins as separate libraries. The core application and library is composed of the following entities, the Diagnostics Processor, the Diagnostics Lookup, Message Processor, and the Communication Interface. The plugins for the app are the individual diagnostics and the specific communication protocol plugin (for example serial communication plugin). The basic structure of the program is a core DiagEngine class that contains and initializes all other classes. The main classes of the program called by the DiagEngine are, the UDSConfig class, that reads the ”.conf”configuration files used to configure plugins, and configures the program accordingly, the UDSBinding class, this class fetches and initializes the shared libraries in runtime, and, the UDSComLibrary class, this class configures the communication layer of the program and has a pointer to the abstract 25 CHAPTER 4. IMPLEMENTATION AND TESTS communication class. The startup of the final iteration of the PDS application, as seen in Figure 16, is conducted as follows. When started, the application searches the predefined path for the configuration files, that contain the diagnostics names and DIDs, saves these values to a map to later be synced with the diagnostic libraries. Subsequently, the UDSBinding class searches a predefined path for the diagnostics libraries (in this case .so or shared object files), and proceeds to bind the map created by the configuration files with pointers to the diagnostics classes. Afterwards, the UDSCommLibrary starts the communication protocol, by searching a predefined path for the library of the communication interface wanted (for example, CAN or serial), and runs the communication thread. When all the previous steps are done, the application waits for UDS messages to arrive at the communication interface to execute the diagnostics. All this can be further visualized in the flowchart presented in the Figure 16. Figure 16: Application startup diagram 26 4.1. PDS APPLICATION 4.1.1 DiagEngine class This class, is the main class of the application, used to initialize all the other modules of the application and to contain all the other classes. For the initialization of the application, it starts with the configuration of the system by reading the diagnosis.default.conf file that includes other .conf files for the diagnostics, this ”.conf”files for the diagnostics contain the diagnostic name and SID. First it is configured the supported diagnostics and their respective names and SIDs via the diagnosis.default.conf file. Afterwards, the search for the shared libraries of all the diagnostics and the binding of them to the lookup class used to store and fetch the pointers of the diagnostics. Lastly the configuration, initialization and start of the communication protocol, which can be either using CAN or serial console. All the configurations use user defined macros that contain the absolute path of the configuration folders, communication and diagnostics library paths. 4.1.2 Communication Interface The implementation of the Communication Interface, taking the requirements into account, it is done with the usage of three classes and inheritance. The three classes are a base abstract class, and both the CAN communication class and the serial communication class. The specific communication classes, CAN and serial, inherit from the base class. For the base abstract class the idea is to create an interface with the DiagEngine class and abstract it from the communication protocol, for this, the base class has functions to retrieve messages and to send messages, as well as an abstract initialization function. This way any class that inherits the base class has the control of what is done for the initialization, reception and relay of the UDS messages. For the implementation of the CAN communication class, it is first done a configuration on the class constructor to open a socket and to configure the baudrate and datarate of the CAN communication, this ends by starting the execution thread. This thread calls the recv function from the Linux CAN libraries that waits until a message is received. After the message is received, it is copied to a byte vector that it is sent to the Scheduler using the Scheduler’s setRequest function. For the serial communication class, the process is similar, only that the serial communication does not need configuring and the thread uses different methods to retrieve information from the serial console. The contents from the serial console are retrieved using the C++ libraries function fgets and checking if the string is a digit or not and passing the ASCII digits to their value in bytes. This information is passed to a buffer that then is copied to a byte vector and sent to the Scheduler, using the setRequest function. 4.1.3 Message Processor The implementation of the message processor, contrary to the other modules, is not a class but a set of simple lines of code to retrieve the relevant information from the UDS message. These lines of code use standard C++ functions to, first, remove the first byte which is the SID. Next, it checks if the SID extracted 27 CHAPTER 4. IMPLEMENTATION AND TESTS 4.2 AGL Image The AGL image built is based on the AGL jellyfish version 10.0.3 for the Renesas RCar H3 Salvator-XS board. The Renesas RCar H3 Salvator-XS board is a development board based on the RCar H3 SoC, this processor is a nine core processor with four cortex A-57, four cortex A-53 and a cortex R7 core, this board was used because of it’s availability and it’s similarity with a custom board developed by Bosch that it’s based on this development board. To explore more the AGL build environment and capabilities a yocto layer was created. The layer is based on the already available layer for the Renesas RCar platform provided by Renesas to AGL, but, with less clutter, less configurations and only supporting the development board used for this dissertation. In the end the layer provided the capability of building a image, based on a pre-configured default image, for the development board with the CAN drivers and interface activated and the libraries necessary for compiling and use the PDS application. The base image choice that the created layer provides is based on the ”agl-cluster-demo”image. This image is a very good starting point with most of the functionalities needed for the project, but it needed some fine tweaks. It was added the option of having the GPIO pins present on the filesystem to enable the configuration of the Control Area Network (CAN) hardware module through scripts provided by Renesas. It was also added to the image the libgpiod library, this enables the implementation of simpler GPIO controls through a abstraction layer provided by that library. After the GPIO modifications, it needs to have enabled the support for Control Area Network (CAN) communication and to have enabled the specific System-onChip (SoC) CAN device drivers, which is done in the kernel configuration menus. 4.3 Tests This is dedicated to the tests and the results of the tests conducted on the system. First there’s a brief exposition of the test environment used, what tools and software was used to conduct said tests. After the tests done are shown and explained, it’s exposed the results of the tests, to later be discussed in the Conclusion chapter. It’s important to keep in mind that this dissertation project is a proof-of-concept, so its tests are mostly functional with little to no real world test conducted, since it’s difficult to do something in such a embryonic solution. 4.3.1 Test environment The test environment used was the CANoe application by Vector running on a Windows machine. This application is used through out Bosch to simulate CAN devices and busses and the interactions between the hardware and a simulated car. The application offers a UDS diagnostics test framework to introduce 34 4.3. TESTS the diagnostics the tester wants to run on the target and their respective responses. This tool was used since it has a simple configuration interface and it supports the CAN interface hardware available. The usage of a CAN simulation application is not enough, it is also needed a interface to physically connect to the hardware and to transform the messages into CAN messages. For this, it is used the Vector VN1630A CAN interface. This interface supports the transmission and reception of CAN and CAN Flexible Data-Rate (CAN FD). To configure the test environment, first it was configured the diagnostics messages to be sent and their positive responses. Then it was created a BasicDiagnosticECU, which is a virtualization of the ECU under test and it’s used to send the diagnostics to the board and to receive the responses. Afterwards it was configured the CAN timings, baudrate and datarates of the CAN bus. The chosen baudrate and datarate are 500000 and 2000000 bits per second. After, it was needed to enable CAN FD and choose the CAN IDs. The test environment connections is as follows, the VN1630A is connected to the computer via Universal Serial Bus (USB) and to the board via a 9 pin DSUB connector. The next figure 18 is a diagram of the test connections. Figure 18: Test environment connection diagram. 4.3.2 Tests and Results Since the project is a proof-of-concept the tests conducted on the board are mainly functional tests, to check if the implementation does what is supposed to and functions in accordance with the requirements and design. First it will be explained the tests conducted to the communication of the system with the test environment. Afterwards it will be shown the tests made for each of the system’s modules, like the Scheduler. The tests end with the tests conducted to the individual diagnostics. Communication tests For the communication tests it was used a set of applications for Linux CAN subsystem, the CAN-utils, as well as the CANoe application, for testing the sending of CAN messages from the board to the tester computer. CAN-utils offers many utilities, the ones used for the test where cangen, candump and cansend. The cangen and cansend are used to send CAN messages to the bus, cangen sends random messages and cansend sends a message specified in it’s parameters. Candump is used to dump received messages 35 CHAPTER 4. IMPLEMENTATION AND TESTS to the terminal. These utilities where used to test the correct configuration and connection of the CAN interface. The results of this test were good and as expected. The CANoe application on the tester computer received all of the messages sent from the board and the board also received and displayed the messages received from the tester computer correctly on the command line, with the use of the candump application. The last tests where also on the full system, but this time, testing the diagnostics itself. The diagnostics where tested to know if the execution is compliant with their requirements and tested against errors. Some diagnostics tests where short and others more intensive. To show better the tests conducted on each of the diagnostics, the next part goes into each of the diagnostics and shows what was done. System Tests This tests simulate the reception of diagnostic messages and the initialization of the system. The system was tested to know if it interpreted the message correctly, what would happen if it received a non compliant message or unknown SID or DID. The initialization tested the system against lack of libraries for each of the components, lack of configuration files and what would happen if it tried to fetch a diagnostic without it’s library present in the filesystem. The results of the tests mediocre, the system, when poorly configured or lacking it’s base libraries, doesn’t initialize, as expected. On the other hand, if the system initializes it’s base libraries but lacks the component libraries it enter a error state and doesn’t recover, which is non ideal. For the reception of non compliant messages or unknown SID or DID the system performed well, it returned a error message to the tester showing that either the message is incorrect or the IDs are not supported by the system. In the Figure 19 it can be seen a snipped of the output of the initialization of the application. Figure 19: Terminal output of the initialization of the application 36 4.3. TESTS Diagnostic Tests This test is characterized by checking the individual diagnostics supported by the system against bad received information or some midway problem with the information that it’s passed between entities. This also tests all the interactions in the system, starting at receiving a message and ending in calling a diagnostic. This is done to check the interaction between the entities, how the modules perform and what happens if a error occurs in the middle of execution. Read MACAddress For this diagnostic the tests where simple, just test what happened if the diagnostic receives parameters or the ethernet interface doesn’t exist. The results are good and as expected, the diagnostic returns the UDS protocol error code for incorrect message length when the number of parameters is higher than the needed number. When the ethernet interface or any processing between the acquisition of the MACAddress and the return of it fails the system correctly returned the UDS protocol error code for ”conditions not correct”. Read eMMC CID This diagnostic is like the Read MACAddress one, just check what happens if the diagnostic receives parameters when it should receive none and one extra test check if the response message contains the exact same values as the CID register. The results are good, the diagnostic sends the correct negative responses to the incorrect number of parameters and if the CID is not available. Read/Write GPIO The tests for this diagnostic check the parameters and test the design and implementation against errors of reading or writing pins that are locked by other programs or processes or do not exist. The results are as expected, like read MACAddress the diagnostic responds correctly to a incorrect number of parameters and responds correctly to non existent GPIO pins, thanks to the libgpiod API error handling. When a pin is locked by other processes of the system the diagnostic also responded correctly with the ”conditions not correct”negative code, due to the libgpiod API excellent error handling. Read/Write Memory The Read/Write Memory tests are checking what happens when restricted memory is read or written to and check the outcome of write by reading the value after the write operation. The results are unexpected and bad, the system could not recover and reset when a restricted memory address was accessed in read or write, this can be a problem with the algorithm used as well as the functions used to retrieve the contents of the memory. On the other hand, when something is written into a address the reading of that same address returned the contents written to it. Play/Stop Audio 37 CHAPTER 4. IMPLEMENTATION AND TESTS In this diagnostic, the test where about what would happen if the interface is not correctly configured, what happens if the thread doesn’t start and what would happen if the audio could not be stopped. The results of this tests are good and bad. If the diagnostic fails at configuring the interface the system responded with the correct negative code and the system didn’t crash, as well as, when the thread could not be started the response was correct. On the other hand, if the audio thread somehow failed, the system would play the audio indefinitely and could only stop when a reset is triggered. Display Test Picture This diagnostic tests are straight forward, connect the display and see what’s outputted, but also, check the outcome of searching a non existent image index and what happens if the image is too big for the screen. The results are as expected, the diagnostic when a out of bounds index is sent returns the UDS error code ”conditions not correct”. When a selected image is bigger than the display, which a occurrence that should not happen since the developer controls the pictures present in the system, the image is displayed, although, it gets cut since it has more information than the display can output. The Figure 20 shows the display output of a given image and the Figure 21 shows some debug output of the diagnostic being called. Figure 20: Display output to a give image Figure 21: Terminal output of some debug lines showing the available images Display Generated Image The Display Generated Image tests include check the displayed image if it’s correct with the parameters, check the output of parameters out of range and check if what happens if the Display Control Wrapper is not working properly. The results of this diagnostic tests are great, the diagnostic deals with the corner 38 4.3. TESTS cases as expected. If the Display Control Wrapper is not working the diagnostic since it could not retrieve a non null pointer throws a error and a negative UDS response, as expected. 39 5 Conclusions This is the closing chapter of this dissertation. Here, the project will be wrapped up and the results to the tests will be discussed to find an answer to the dissertations questions. Currently in the automotive world of Bosch the usage of a open source Linux based OS like AGL can be of high interest. The company can reduce the costs of using closed source options and have a more flexible solution due to open source nature. The solution here presented, a PDS application running on AGL, has proved to be very promising in this field of production diagnostics. 5.1 Conclusions The purpose of this dissertation project was to create a proof-of-concept Production Diagnostics System that uses AGL as a OS, in order to be a competing platform to the solution currently used by Bosch. Nowadays, the existing PDS applications at Bosch are supported by a QNX on AUTOSAR stack. This dissertation aims to verify if it is possible to remove this software stack and replace it with AGL and also, it aims of exploring the automotive features of AGL, for example the implementation of a diagnostics engine compliant with UDS. AGL, as a Linux project, offers high levels of customization, as well as a repository of open-source software packets and COTS modules to support development of new applications. On the other hand, AGL being a recent technology does not follow some automotive standards and lacks the code base and maturity for certain functionalities when comparing with other operating systems like QNX. A proof-of-concept was made, in this dissertation, of the use of AGL to support the implementation of a UDS compliant application interacting with AGL. The proof-of-concept developed integrated a PDS application with AGL. This system is capable of receiving UDS requests, execute Production Diagnostics and send back UDS responses to the tester. The finding of this dissertation is that the proof-of-concept developed cannot replace the current solution used by Bosch. This is because AGL doesn’t offer the same features, have the same behaviour and be compliant with the same automotive standards that Vector AUTOSAR offers, this can be attributed to 40 5.2. FUTURE WORK the fact that AUTOSAR is a RTOS with low level hardware interfaces and a ecosystem made for automotive development. On the other hand, AGL can be seen as a competitor to QNX in this software stack. AGL being a opensource Linux project, aimed for the automotive industry, offers a cost reduction and higher customization when compared to QNX. The challenge, however, is to expand its code base and reach a higher level of maturity to be viable option to future solutions and replace QNX. 5.2 Future Work As mentioned above, AGL is a viable solution for incorporating the PDS support software stack. Although the PDS application developed in this dissertation is functionally correct, it needs to implement a good error handling framework and mechanisms to prevent system lockups in certain corner cases. The application also needs testing against all the automotive ISO specifications, specially ASIL. Other good features to implement in late iterations are, data logging to help tackle errors and understand what the system is doing, more services implemented to increase the number of diagnostics and a fully functional UDS specification framework. AGL OS it is a promising alternative to QNX on the software stack. For the future it will be very important to explore the integration of AGL into the software stack, taking the place of QNX. 41 Bibliography [1] R. Ramya, S. Sripada, and K. B. Binu. “Breakthrough approach to automotive diagnostic verification”. In: SAE Technical Papers (2011). issn: 26883627. doi: 10.4271/2011-28-0030. [2] Y. Li, D. Zhan, and H. Li. “Eol testing method based on ecu diagnostic function”. In: FISITA 2016 World Automotive Congress - Proceedings . 2016. [3] E. Bringmann and A. Krämer. “Model-based testing of automotive systems”. In: Proceedings of the 1st International Conference on Software Testing, Verification and Validation, ICST 2008 . 2008, pp. 485–493. isbn: 076953127X. doi: 10.1109/ICST.2008.45. [4] I. Standard. “INTERNATIONAL STANDARD Road vehicles — Unified diagnostic”. In: 2006 (2008). [5] J. Knox. Automotive grade linux platform debuts on the 2018 toyota camry . 2017. url: https: / / www . automotivelinux . org / announcements / automotive - grade - linux - platform-debuts-on-the-2018-toyota-camry/. [6] J. Mössinger. “Software in automotive systems”. In: IEEE Software 27.2 (Mar. 2010), pp. 92–94. issn: 07407459. doi: 10.1109/MS.2010.55. [7] S. Aust. “Paving the way for connected cars with adaptive AUTOSAR and AGL”. In: Proceedings of the 43rd Annual IEEE Conference on Local Computer Networks, LCN Workshops 2018 (2019), pp. 53–58. doi: 10.1109/LCNW.2018.8628558. [8] P. Sivakumar et al. “Automotive Grade Linux Software Architecture for Automotive Infotainment System”. In: Proceedings of the 5th International Conference on Inventive Computation Technologies, ICICT 2020 . 2020, pp. 391–395. isbn: 9781728146850. doi: 10.1109/ICICT48043.2020.9 112556. [9] M. O’Donnell. Automotive telematics: Open-source automotive-grade linux is the future . July 2005. [10] AGL. Automotive Grade Linux . 2014. url: https://www.automotivelinux.org/. 42 BIBLIOGRAPHY [11] S. Mizuno et al. “Real-time Performance of Embedded Platform for Autonomous Driving Using Linux and a GPU”. In: International Journal of Informatics Society 10.1 (2018), pp. 31–40. url: http: //www.infsoc.org/journal/vol10/IJIS%7B%5C_%7D10%7B%5C_%7D1%7B%5C_%7 D031-040.pdf. Thisdocumentwascreatedusingthe(pdf/Xe/Lua)L A T EXprocessor,basedontheNOVAthesistemplate,developedattheDep.InformáticaofFCT-NOVAbyJoãoM.Lourenço.[1] [1] J.M.Lourenço.TheNOVAthesisL A T EXTemplateUser’sManual.NOVAUniversityLisbon.2021.URL:https://github.com/joaomlourenco/novathesis/raw/master/template.pdf. 43