sensors
Article
Configurable Sensor Model Architecture for the Development
of Automated Driving Systems
Simon Schmidt 1,* , Birgit Schlager 2,3 , Stefan Muckenhuber 2,4 and Rainer Stark 5
Citation: Schmidt, S.; Schlager, B.;
Muckenhuber, S.; Stark, R.
Configurable Sensor Model
Architecture for the Development of
Automated Driving Systems. Sensors
2021,21, 4687. https://doi.org/
10.3390/s21144687
Academic Editor: Iván García Daza
and Javier Alonso Ruiz
Received: 9 June 2021
Accepted: 5 July 2021
Published: 8 July 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1Volkswagen AG, 38436 Wolfsburg, Germany
2Virtual Vehicle Research GmbH, 8010 Graz, Austria; bir[email protected] (B.S.);
[email protected] (S.M.)
3Institute of Automation and Control, Graz University of Technology, 8010 Graz, Austria
4Department of Geography and Regional Science, University of Graz, 8010 Graz, Austria
5Institute of Machine-Tools and Factory Management, Department of Industrial Information Technology,
*Correspondence: [email protected]
Abstract:
Sensor models provide the required environmental perception information for the devel-
opment and testing of automated driving systems in virtual vehicle environments. In this article, a
configurable sensor model architecture is introduced. Based on methods of model-based systems
engineering (MBSE) and functional decomposition, this approach supports a flexible and continuous
way to use sensor models in automotive development. Modeled sensor effects, representing single-
sensor properties, are combined to an overall sensor behavior. This improves reusability and enables
adaptation to specific requirements of the development. Finally, a first practical application of the
configurable sensor model architecture is demonstrated, using two exemplary sensor effects: the
geometric field of view (FoV) and the object-dependent FoV.
Keywords:
sensor model architecture; configurable sensor model; sensor effects; automated driving;
virtual testing; functional decomposition; model reusability
1. Introduction
Automated and autonomous vehicles are an important future topic in automotive
industry and politics. The rising degree in automation will lead to major changes in
transportation as well as public and personal mobility [
1
,
2
]. Advantages are promised by
increasing automotive safety, energy efficiency in transportation, and increasing comfort
for passengers [3].
Automated driving functions are realized as complex and usually architectural dis-
tributed systems [
4
]. They are highly integrated with other vehicle systems and process
information from the vehicle itself as well as from the vehicle environment. Automated
driving systems are safety-relevant systems since they affect the driving behavior by influ-
encing the longitudinal and/or lateral control of the vehicle [
5
]. This results in special safety
requirements that must be taken into account during the development and for release [
4
].
The requirements have to be validated with an increasing demand in testing [
6
,
7
]. To meet
this demand, development and test activities can be transferred to a virtual vehicle envi-
ronment using simulative methods. Therefore, the perception of the vehicle environment
is an integral part of the development and is needed to operate automated driving systems.
Sensor models provide the perception information from the virtual vehicle environment.
The systems development process (SDP) in the automotive industry is changing from
a mechanical- and component-oriented development to a function-oriented development
due to increasing interconnectivity and a growing share of software. A method to practice
function-oriented development is systems engineering (SE), where large, complex, and
interdisciplinary systems are systematically broken down into subsystems. In the case of
Sensors 2021,21, 4687. https://doi.org/10.3390/s21144687 https://www.mdpi.com/journal/sensors
Sensors 2021,21, 4687 2 of 14
mechatronic systems, such as automated driving systems, models can be used to represent
subsystems or parts of them. This approach, which uses a rich mix of digital models
for the system description and development and emphasizes the deployment of links
in between the individual model parameters and characteristics, is called model-based
systems engineering (MBSE). Providing perception information through sensor models
supports this idea.
There are two kinds of approaches for creating sensor models, which can be distin-
guished. The first one is explicit modeling. Sensor characteristics must be known in order
to model a specific behavior directly in source code. Schlager et al. present an overview
about the current state-of-the-art of explicit modeling approaches of perception sensors
in [
8
]. The second kind of creating sensor models is using data-driven procedures, where a
specific behavior is extracted from recorded measurement data and thus modeled indirectly.
This can be achieved, for example, by means of a neural network. Data-driven modeling
offers the advantage that complex and even unknown sensor behavior can be represented.
However, the disadvantages are that only recorded behavior can be represented and the
large amount of data required to generate sensor models. Furthermore, these models can
only be used as black-boxes, without having access to their internal structure [
9
]. Therefore,
with regard to the conceptual design and synthesis of sensor models, the first approach of
direct modeling is followed in the present work. This is associated with greater effort, but
supports the deeper understanding from the internal structure to the overall behavior of
sensor models.
Since automated driving systems go through different phases in the course of SDP and
are constantly changing, the provided perception information must also be continuously
adapted. A single-sensor model specified at the beginning of the SDP is unlikely to meet
the requirements of later development phases [10].
In Figure 1, selected exemplary requirements are shown that can arise in the course
of the development of an automated driving system. The SDP is presented in the form of
the V-model [
11
]. The descending branch of the V-Model describes the specification phase.
At the beginning, sensor models are required to specify the perception concept of the
automated driving system. For example, it must be determined which areas of the vehicle
environment are to be perceived and at what distance objects must be detected. Later in
the development, sensor models can support the selection of the sensor technology to be
used for the automated driving system (radar, lidar, camera, etc.). For this purpose, typical
sensor characteristics must be modeled. After the specification phase, the implementation
begins. Now, for the execution of prototypes, sensor models are required that provide
a sensor behavior that is as physically correct as possible. The ascending branch of the
V-Model describes the integration phase. Interfaces must be tested before the integration
of the automated driving system into the overall vehicle begins. The interfaces with
regard to perception information or sensor data processing can be checked with the help of
sensor models, which represent the interface behavior. Sensor models provide the required
perceptual information in order to assess the system behavior of the automated driving
system in the overall vehicle, which behaves similarly to that in subsequent real-world
operation. These must, for example, be consistent with additional information from other
vehicle systems. With regard to the functional safety, faulty system states are also checked
later on in a targeted manner in order to exclude unintentional system reactions. Here,
sensor models that represent special fault conditions in addition to the desired standard
behavior can be used.
Sensors 2021,21, 4687 3 of 14
Specification Phase
Integration Phase
Implementation
Time
Level of Detail
Perception
Concept
Perception
Technology
Physical Perception
Behavior Interface
Testing
System
Behavior
Functional
Safety
Figure 1.
Exemplary sensor model requirements resulting from the SDP. The automotive development
of an automated driving system is depicted by means of the V-model.
In the literature, lots of very specific sensor models are described [
8
], but less publica-
tions deal with structuring sensor models in general. A flexible sensor model architecture
is needed to enable continuous adaptation of sensor models to requirements from the
SDP. The basic idea of a configurable sensor model architecture is presented first in [
12
]
from Hanke et al. They introduce a generic modular approach, where an environment
simulation provides ground truth perception information, which is then modified by a
number of sensor modules in sequence. In [
13
], Linnhoff et al. introduce a parameterizable
architecture for sensor models based on the functional decomposition of real world sensors.
The approach of Hanke et al. describes a sensor-type independent procedure, whereas
Linnhoff et al. refer to radar, lidar, and camera sensors. Both recommend further devel-
opment and implementation of a fundamental architecture but have not demonstrated a
practical application on an automated driving system so far.
This article presents an approach that uses a configurable sensor model architecture to
provide perception information. Perceptual information can be provided in a configurable
and requirement-adaptable way by combining different exchangeable modules containing
modeled sensor effects. The configurable sensor model architecture presented in the present
work is designed to support sensor-type dependent as well as sensor-type independent
models. This is necessary because the perception technology in the SDP is not always
fixed from the beginning. The individual modules have to be exchanged flexibly, so the
standardization of data types and interfaces is important. For this purpose, the upcoming
standard open simulation interface (OSI) [
14
] is used here, as mentioned in [
13
] before.
Hanke et al. and Linnhoff et al. describe their architectures purely from a functional per-
spective. In comparison, the logical and technical perspectives are additionally considered
here. These are intended to improve comprehensibility and support usability. Furthermore,
the architecture introduced in this article works with individual and separately modeled
sensor effects. The focus is more on sensor effect level than in the previous approaches.
This enables flexibility and allows us to decide about every single-sensor property. In
this way, both basic generic properties can be combined to form sensor-type independent
models as well as sensor-type specific properties can be combined to form type-dependent
models. The focus of this article is on the application of the configurable sensor model
architecture. Therefore, the architecture is presented in an application-oriented form of the
configurable sensor model template.
Sensors 2021,21, 4687 4 of 14
In Section 2, a simulation setup for automated driving systems is illustrated to clarify
the application context. An essential part of this setup is the configurable sensor model
template. After the integration into the simulation setup has been explained, Section 3
presents the configurable sensor model template in detail. Subsequently, a case study of
the configurable sensor model architecture is demonstrated, using the configurable sensor
model template, by means of an adaptive cruise control (ACC) system and two selected
sensor effects in Section 4.
2. Simulation Setup
The general functionality of automated driving systems can be described by the sense–
plan–act principle, which originated in robotics [
15
]. First, information is perceived (sense).
Based on this information, a behavior is planned (plan) and afterwards transformed into
an action (act). The sense part is fulfilled by the sensor model. It processes perception
information from the virtual vehicle environment and makes it available to the automated
driving system, as shown in Figure 2.
Virtual
Vehicle
Environment Sensor Model Automated
Driving
System
SENSE ACTPLAN
Figure 2. Perception process for simulation of automated driving systems.
In order to be able to operate the automated driving system the same way as it would
function in a real vehicle, it is integrated into the simulation setup, shown in Figure 3. The
setup provides all necessary interfaces to other vehicle systems, as well as interfaces to
sensor technology for the perception of the vehicle environment and, if required, interfaces
to the human driver. Closed-loop operation is established via a feedback-loop using vehicle
dynamic models. Although the automated driving system is the device under test, this
article will concentrate on the sensor models used.
The simulation setup consists of components that can be updated and exchanged
during the SDP. This is supported by a co-simulation platform. Co-simulation is used
to meet the challenges in development of complex and interdisciplinary systems where
distributed modeling is used [
16
]. Gomes et al. explain in [
17
]“It consists of the theory and
techniques to enable global simulation of a coupled system via the composition of simulators" and
provide an overview of the current state of technology and application. The components
shown in Figure 3are in interaction with the automated driving system and are therefore
required for operation, comparable to the use in a real vehicle later. The virtual vehicle
environment is generated by an environment simulation and includes the necessary per-
ception information for environmental sensors. The sensor model processes the perceptual
information and provides it in the required form to the automated driving system. In
addition to perceptual information, the automated driving system also receives information
about the current status of the vehicle from interacting vehicle systems. Afterwards, it
plans a behavior based on this information and calculates the necessary control values for
longitudinal and/or lateral movement in order to execute this behavior. The control values
are converted into a vehicle movement by the vehicle dynamics. Optionally, chassis and
tire models can be integrated. A driver model is required for driving systems that are not
autonomous (lower than level 5, according to SAE J3016 [
18
]). The driver model operates
the interfaces of the human driver, such as steering wheel, pedals, and direction indicators.
Sensors 2021,21, 4687 5 of 14
Co-Simulation Platform
Environmental Perception
Runtime Environment
Sensor Model
Virtual Vehicle Environment
(Environment Simulation)
Automated
Driving System
Vehicle Dynamics
(Actuator Models, Body
Model, Chassis Model,
Tire Models)
Interacting
Vehicle
Systems
Human Driver
(Driver Model)
Direction of Information Transfer System Boundaries
Vehicle
Figure 3.
Simulation setup for development of automated driving systems. A sensor model provides
the environmental perception.
After this overview about the simulation setup, Section 3describes in detail how to
use sensor models in a continuous and flexible way. For this purpose, the configurable
sensor model template is introduced.
3. Configurable Sensor Model Template
A continuous, flexible, and requirement-oriented use of sensor models must be en-
abled in the SDP of automated driving systems to extract perception information from
the virtual vehicle environment. The configurable sensor model template is introduced in
order to demonstrate and apply the configurable sensor model architecture.
With a meta-perspective towards the simulation setup (shown in Figure 2), a sensor
model is the linking element between the environment simulation and the automated
driving system. Using functional decomposition, an overall behavior of the sensor can be
separated into individual elements, called sensor effects. A sensor effect covers either a
general sensor-independent characteristic e.g., the field of view (FoV) or a characteristic
of a sensor technology e.g., an occlusion modeling, which is physically different for radar,
lidar, and camera. Further, a property of a specific real sensor e.g., hardware dependent
noise can be covered. The configurable sensor model template uses the combination of
individual sensor effects to provide either sensor or sensor-type specific behavior. The
overall behavior generated in this way allows the template to be adapted to requirements
of the different stages and milestones in the SDP of automated driving systems. The
consistency and reusability achieved using the configurable sensor model template follows
the idea of MBSE, which is a formalized application of modeling to support development
according to INCOSE [
19
]. Thus, a high availability is achieved through modularity, which
allows a fast and direct feedback regarding viability and robustness during the execution
of automated driving systems at all stages.
In this article, the idea of modularity in sensor simulation is developed further. With
the goal of a systematic integration into the SDP, different perspectives of the configurable
sensor model template architecture are presented. The focus is on functional aspects in
terms of a function-oriented development. The approach is illustrated by means of the
functional decomposition of sensors. For this purpose, a prototype according to require-
ments of the SDP of automated driving systems is presented. In model-based approaches,
connecting different models is an important issue in order to meet the requirements of
consistency and flexibility. For this reason, the prototype uses the upcoming standard OSI
for interfaces and communication.
Loading more pages...