Closing the Gap Between Automated Mobility in Smart Cities: Smart Cities: Automated Vehicle and Shuttle Transportation in the Test Area Autonomous Driving Baden-Württemberg

- 1 - Closing the Gap Between Automated Mobility in Smart Cities: Automated Vehicle and Shuttle Transportation in the Test Area Autonomous Driving Baden-Württemberg Nico Lambing, Sven Ochs, Stefan Orf, Tobias Fleck, Christian Hubschneider, Marc René Zofka, Alexander Viehl, J. Marius Zöllner 1FZI Research Center for Information Technology, Division for Intelligent Systems and Production Engineering (ISPE), Haid-und-Neu-Str. 10-14, 76131 Karlsruhe, Germany E-mail: (lambing, ochs, orf, tfleck, hubschneider, zofka, viehl, zoellner)@fzi.de Abstract Based on the rapid development of a multitude of autonomous vehicles (AVs), their applications have risen and are currently studied globally to shape future mobility. Accordingly, real-life demonstrations at different scales are conducted or are in the making worldwide. Within the context of the Horizon2020 project SHOW (SHared automation Operating models for Worldwide adoption (SHOW, 2021)), the test site Karlsruhe within the Test Area Autonomous Driving Baden-Württemberg (TAF-BW) is part of the German Mega Site and is leveraged to research and demonstrate AV transportation services with a focus on shared mobility for different types of AVs. The services under consideration are strongly connected to road users’ daily travel needs, such as demand responsive transport on the last mile in peri-urban quarters. This work aims to provide a deeper insight into the conducted use cases in the SHOW project at the test site Karlsruhe. The respective system concept and framework expected results and possible issues will be explained and discussed consequently. Keywords: real-life demonstration, shared AV mobility, demand responsive transport, autonomous driving, intelligent road infrastructure Περίληψη Με βάση την ταχεία ανάπτυξη πλήθους αυτόνομων οχημάτων (ΑΟ), οι εφαρμογές τους έχουν αυξηθεί και μελετώνται σήμερα σε παγκόσμιο επίπεδο για τη διαμόρφωση της μελλοντικής κινητικότητας. Κατά συνέπεια, πραγματοποιούνται ή βρίσκονται σε εξέλιξη επιδείξεις σε διάφορες κλίμακες παγκοσμίως. Στο πλαίσιο του έργου SHOW (SHared automation Operating models for Worldwide adoption (SHOW, 2021)) του προγράμματος Horizon2020, ο χώρος δοκιμών της Καρλσρούης στο πλαίσιο του Test Area Autonomous Driving Baden-Württemberg (TAF-BW) αποτελεί μέρος του γερμανικού Mega Site και αξιοποιείται για την έρευνα και την επίδειξη υπηρεσιών μεταφοράς AV με έμφαση στην κοινή κινητικότητα για διαφορετικούς τύπους AV. Οι εξεταζόμενες υπηρεσίες συνδέονται στενά με τις καθημερινές ανάγκες μετακίνησης των χρηστών του οδικού δικτύου, όπως οι μεταφορές που ανταποκρίνονται στη ζήτηση στο τελευταίο μίλι σε περιαστικές συνοικίες. Η παρούσα εργασία αποσκοπεί στην παροχή μιας βαθύτερης κατανόησης των περιπτώσεων χρήσης που πραγματοποιήθηκαν στο έργο SHOW στο χώρο δοκιμών της Καρλσρούης. Θα εξηγηθούν και θα συζητηθούν, κατά συνέπεια, τα αναμενόμενα αποτελέσματα και τα πιθανά ζητήματα της αντίστοιχης έννοιας και του πλαισίου του συστήματος. Λέξεις-κλειδιά: επίδειξη σε πραγματικές συνθήκες, κοινή κινητικότητα AV, μεταφορές που ανταποκρίνονται στη ζήτηση, αυτόνομη οδήγηση, ευφυείς οδικές υποδομές - 2 - 1. Introduction Due to the rapid innovation and research conducted in automated driving technology, we firmly belief that autonomous vehicles (AVs) are going to be an important part of our future daily life. AVs' capabilities for higher levels of automation are continuously refined, tested and validated (Yurtsever, 2020). At the same time, various AV application strategies and services have been initiated and studied with the goal of shaping future mobility regarding sustainability, safety, efficiency and comfort. Accordingly, many real-life demonstrations, conducted at different scales, are currently being planned or performed worldwide. (Worschech, 2020) In Germany, like in many other countries, various projects have been initiated with the goal to research and demonstrate Highly Automated Driving (HAD). Such demonstrations are not only intended for functional verification in real, complex traffic environments. Moreover, they aim to examine proposed strategies and to explore the potential for improvement and further innovation in AV technology and deployment strategies. In addition to research results, corresponding infrastructure facilities, test fields and labs, covering private and public areas, have been established and steadily enhanced in different regions (Riener, 2020) (Narayanan, 2020). Within the context of the European research project SHOW (SHared automation Operating models for Worldwide adoption), the test site Karlsruhe is part of the Mega Site Germany (SHOW, 2021). The main focus of this is the examination and demonstration of AV transportation services with a focus on shared mobility and its interaction with intelligent infrastructure. In addition to a technological evaluation and optimization of the HAD functions of different types of AVs, such as shuttles and individual passenger vehicles, the test site Karlsruhe focuses on the question how public automated transportation of passengers and cargo can benefit from smart and connected infrastructure. From our point of view, intelligent infrastructure (IIF) provides three major benefits: • IIF can provide environment information, which otherwise has to be provided by an AV’s perception system. A simple example for this case is the current status of traffic lights. • IIF is able to provide additional information, which by traditional means would not be available or cannot be perceived. Using the traffic light example, this could be the remaining time for the current signal phase. Another example for an extension of the AV`s perception is the possibility to provide additional information, e.g. in intersections that are impossible to be fully perceived from a single point of view on street level. Such intersections could be equipped with additional sensors to communicate a model of the whole environment to the AV. • IIF can serve as a connection node to allow communication with other vehicles or internet services. The latter is of particular interest in case mobile communication systems such as LTE are not available, e.g. in tunnels, or to provide highly localized information, e.g. construction site warnings via ITS G5. - 3 - The test site Karlsruhe tries to exploit all three presented major benefits of IIF in order to demonstrate how HAD can leverage additional information. Furthermore, an investigation how fleets consisting of different types of AVs can be supervised or controlled by a remote operator is of interest. The demonstration of these concepts will be performed with multiple autonomous shuttles and passenger cars. The remainder of this paper is structured as follows: Sec. 2 focuses on the mobility services addressed and the overall technology concept of the test site Karlsruhe. In Sec. III we conclude with an outlook of the addressed challenges and expected results. 2. Enhancing shared mobility services In order to study and elaborate new deployment strategies for shared autonomous mobility, it is necessary to evaluate HAD functions in specific scenarios under realistic conditions. Providing a realistic setting in a controlled environment is the goal of the Test Area Autonomous Driving Baden-Württemberg (“Testfeld Autonomes Fahren BadenWürttemberg”, abbr.: TAF-BW) (TAF, 2021). The TAF-BW consists of selected areas within the cities Karlsruhe, Heilbronn and Bruchsal in the south-western part of Germany. Various kinds of online and offline data to support the development and testing of autonomous driving functions are provided, as well as high-definition road maps (HD maps), currently consisting of 223 km with 691 traffic signs and 223 signal lights. To research how HAD can leverage additional information from IIF, intersections along a 2.5 km and 7 km long route are equipped with roadside units (RSUs) and additional environmental sensors such as cameras, radars and weather sensors. In order to focus on shared mobility and the integration of AVs into smart cities, special areas within the TAF-BW were selected, which are suitable to study typical use cases that arise from individual transportation in peri-urban quarters. The deployed system architecture consists of several technological components, which integrate seamlessly into a comprehensive mobility system. This concept is visualized in Figure 1. The different AVs and the IIF are both connected to the operation center, which aggregates information from the diagnostics system of individual AVs, as well as additional information from the IIF, enabling a potential supervisor to manage several different AVs being operated in distinct operation modes in a peri-urban quarter. Furthermore, the IIF and AVs are connected bidirectionally, enabling the AVs to utilize additional information provided by the infrastructure. The individual parts and components are explained in the following sections. - 4 - Figure 1: The system architecture within the test site Karlsruhe that enables aggregation of information from AVs and additional IIF data in an operation center. Incoming data is visualized in a dashboard enabling the operator to monitor the current diagnostic data of the AVs in the field and the incoming environment data from the intelligent infrastructure. 2.1 TAF-BW - a field laboratory for smart city mobility concepts To pave the way for autonomous vehicles in different fields of use, such as individual or public transport and logistics, public test sites are crucial for reference data and testing process. Although simulation-based testing is used excessively, there still remains a considerable gap to real environments. This includes sensor errors and artifacts due to adverse weather conditions and environmental aspects that may be absent in simulation. Furthermore, complex multi-agent traffic participant behavior must be considered, which is hard to model and test in state-of-theart simulation environments. Hence, real world test sites are inevitable in the development process and in order to close the gap between proving grounds and the final rollout, while preserving the operational design domain (ODD). This is achieved by using smart and connected infrastructure to obtain additional support for the HAD functions as well as gaining insight into the ground truth of the present situation. Within the test area of TAF-BW, selected streets of different types and complexity are extended with intelligent roadside infrastructure. Two sections of the test area are shown in Figure 2. The infrastructure is used to observe, process and communicate with connected vehicles in the present traffic situation. - 5 - With these applications in mind, low cost sensors and local road-side units, which are connected to a central back-end for cost-efficient intelligent infrastructure coverage, are utilized. Since the start of operation of TAF-BW in 2018, the catalog of available data and test services is expanding. This includes high-definition mapping in different formats, including Lanelet2 (Poggenhans, 2018) and OpenDrive, restricted test area access for testing and supplying online and offline environment data (TAF, 2021). The test area is continuously extended by new sensors, services and testing capabilities in order to adapt to new requirements and technology. Figure 2: Exemplary routes in the the TAF-BW in the cities of Karlsruhe (left) and Heilbronn (right.). The blue and yellow dots mark the intersections that are equipped with sensors.(TAF, 2021) Fleck et. al (Fleck, 2018) introduced the concept of local traffic road-side infrastructure in the test area, laying the groundwork for vehicle evaluation, road side perception and communication as a testbed for connected autonomous driving applications. Local traffic roadside infrastructure consists of perception sensors, e.g. fixed mounted cameras, local V2X communication devices (currently ITS-G5) and a wide area network that connects the sensors and devices to a central back-end server architecture. For real-time traffic detection and classification, the concept of stationary perception sensors has been extended to real-time applications (Fleck, 2020). A camera-based, real-time capable, multi-object tracking system has been developed, evaluated and deployed, that is capable of tracking, classifying and locating objects in global coordinates (WGS-84). This way, all information from the test side is condensed and agglomerated in a centralized, accessible way for the support of HAD functions on the one hand, as well as for the evaluation of such functions on the other hand. Furthermore, an open data set has been published and made accessible, which provides insights into the condensed and post-processed information collected at the different intersections. This data consists of HD maps, globally referenced road user trajectories as well as traffic light signal phase diagrams recorded in the test area (Zipfl, 2020). The opportunities of such data have been shown by Zipfl. et al. by demonstrating a semantic scene model that is used to assess the criticality of a traffic scene based on constellations of traffic participants and their relations to each other and the underlying map. Future work might extend this methodology to an online use case where the criticality of a situation may be communicated to participating test vehicles via V2X communication. - 6 - In summary, TAF-BW provides a laboratory under real world conditions to test future mobility concepts. Through the available test sites and infrastructure it enables the development of future-oriented solutions for individual transport and local public transport. 2.2 Considered use cases The research presented in this work focuses on the deployment of innovative shared autonomous mobility concepts, assisted by smart and connected infrastructure. Since the potential applications in this rather broad environment are numerous, exemplary use cases have been defined to demonstrate the general improvements and opportunities that arise from the use of intelligent infrastructures. In addition to mixed-use of vehicles for autonomous passenger and cargo transport under normal and complex environmental conditions, we focus on the aspect of managing the fleet via a connection to an operation center, including remote supervision as well as potential teleoperation or tele-guidance. City areas of operation are chosen based on their potential for varying operation modes, traffic complexity and regulatory restrictions. This use-cases were chosen because they align with our vision of the future of transportation services and the potential provided by a fleet of AV, serving different purposes and being managed by a central operation center, which has access to a vast amount of real-time data, in turn provided by the AVs and the intelligent infrastructure. Furthermore, the operation center should be able to interact with the vehicles in order to resolve possible issues. We belief that this kind of manageable fleet will be necessary in the future, when regulatory requirements permit the operation without safety drivers and fleet sizes will be much bigger. Furthermore, such an operation center is necessary to distribute driving tasks optimally within the managed fleet. 2.3 The test site Karlsruhe As part of the German mega site, the Karlsruhe test site, which in turn is part of the Test Area Autonomous Driving Baden-Württemberg, was designed to represent the use cases mentioned above. In this chapter we present the test site Karlsruhe, the technical infrastructure and the utilized AVs. The test site consists of two sub-sites as visualized in figure 3. - 7 - Figure 3: Test test site Karlsruhe. The left image shows the sub-site "Campus-Ost" of the Karlsruhe Insitut of Technology (KIT). Since the sub-site Campus Ost is a restricted area, experimental functions can be tested before they are tested on public roads. Public road tests are performed in the second subsite (right), a quarter in Weiherfeld-Dammestock, a suburb of Karlsruhe. All sites are part of the TAF-BW. (Graphhopper, 2021) The first test sub-site is comprised of an area of the Karlsruhe Institute of Technology (KIT), called "Campus-Ost" (eastern campus). The KIT Campus-Ost is a restricted area with an open space of about 5.000 m² and 2 km of drivable street around it. Within this area, autonomous vehicles can be tested safely before they are used on public roads. Since the access to the site is limited, low traffic volume and a controllable environment can be taken advantage of to test and evaluate experimental HAD functions like the operation center based tele-operation. The second sub-site provides a more complex environment and is located in WeiherfeldDammerstock, a suburb in the south of Karlsruhe. In this area, our autonomous shuttles have the legal permit to drive autonomously with a maximal speed of 20 km/h on public roads and in real traffic. In the sub-site, a general speed limit of 30 km/h is given, but autonomous driving still poses a number of challenges like narrow streets with parked vehicles alongside the road, a high number of cyclists and pedestrians, bad road conditions and missing road markings. This combination makes the environment challenging and interesting for perception and trajectory planning algorithms. Also navigating narrow street corridors presents challenges on the localization component of the automated vehicle, since GPS-Signals in those corridors can diverge and drift due to occlusions and reflection. 2.3.1 Intelligent Infrastructure We equipped two intersections in the sub-site Weiherfeld-Dammerstock with intelligent roadside infrastructure consisting of ITS-G5 (IEEE WLAN 802.11p) based roadside units (RSUs) that are connected to the traffic light controller of a pedestrian crossing. Figure 4 (left) shows the intersection. - 8 - Figure 4: One of the RSU equipped intersections in the test site in Weiherfeld-Dammerstock (left). The RSSI of 802.11p based communication has been measured with a commercially available V2X-device in an autonomous shuttle to gain insight on the maximum range of the ITS-G5 based communication (right). Low signal strength is encoded in yellow, high signal strength is encoded in red. (OpenStreetMap contributors, 2017) (leaflet, 2021) The RSU communicates the traffic light states making it possible to deploy, test and evaluate intelligent trajectory and maneuver planning incorporating such information. For the use cases to be realized in the project, we consider two types of communication as relevant: First, the bidirectional communication between roadside infrastructure and the AV in order to support the HAD functions by serving as an additional sensor. Therefore, signal phases (SPaT) or road geometry and topological quantities (MAP) are exchanged. Also, information of second order, such as received signal strength indications (RSSI) enable the derivation of additional states about the environment. Second, our infrastructure allows to incorporate global information by connecting all roadside devices in a wide area network, making all information available in a backend service infrastructure. In order to enable human operators located in the operation center to gain insight into the interoceptive vehicle states especially Cooperative Awareness Messages (CAM) and Collective Perception Messages (CPM), reflecting a vehicle’s perception and environmental understanding, are of interest. Local, temporary events, such as accidents or construction sites, are communicated between backend, local roadside units, automated vehicles and shuttles via DENM. This allows to live monitor, record, replay and analyse test drives and demonstrations regarding the aspects global information like booking-services, V2X communication, environment perception of the infrastructure and the vehicle itself and the behaviour of the HAD-function. Through the globally agglomerated data we are able to reconstruct challenging situations, for example weather conditions or unforeseen behaviour of other traffic participants. Based on the insights into these challenging events the overall performance of the HAD function can be improved. - 9 - 2.3.2 Autonomous vehicles at the test site Karlsruhe Figure 4: The EVA-Shuttle in the subsite Weiherfeld-Dammerstock, Karlsruhe (left). Our shuttles are equipped with additional sensors at the corners and on top of the vehicle. The automated driving platform Cocar (right) of FZI Research Center for Information Technology. CoCar is based on a Audi Q5 which was equipped with LiDAR and dGPS sensors. In SHOW we demonstrate multi-modal transportation approaches for passengers and cargo using different types of automated vehicles. Our fleet of AVs consists of two modified EasyMile EZ10 Gen2 shuttles and CoCar the automated driving platform of FZI, which is based on an Audi Q5, both shown in Figure 4. Since the shuttles are designed for slow speeds and routes with a high frequency of stops, we utilize them to serve the first and last mile scenario in the second subsite, a suburban area of Karlsruhe. The requisite work was done in the project EVA Shuttle (EVA, 2021) including hardware and software setup. Currently, a multi-sensor setup with 5 Velodyne VLP 16 and one VLP 32 is used, which are placed at the corners and top of the vehicle. The sensor-setup is extended through four radar sensors, which have the ability to detect obstacles at further distances than the LiDAR-Sensors. The SAE Level 4 permit requires the addition of a safety driver/operator to ensure that the vehicles can be brought from automated driving mode to a standstill at any time. In addition, manual steering of the shuttle is available if a traffic situation occurs which the HAD function is not yet able to master fully. In this case the driver closes the last possible safety gap. This safety assurance enables the vehicles to plan trajectories without the usage of virtual rails. In use cases that involve individual transport with higher speeds, we are using our AV CoCar (Cognitive Car). Extensive modifications were made in order to enable safe autonomous driving, advanced perception and control. Additional sensors and hardware components were installed, fitting well into the car design and appearance, while securing the flexibility of use for various use cases at the same time. A set of five LiDAR sensors (Ibeo Lux) is integrated in the frontand rear valance and on the sides of the vehicle. The position of the vehicle is estimated by an inertial measurement unit, combined with D-GPS thus providing centimeteraccuracy. The safety concept for CoCar consists of a safety driver which intervenes in critical situations. In autonomous mode, the vehicle is completely software controlled. However, as soon as the driver touches the brake pedal or steering wheel, the vehicle is instantaneously switched to the normal mode and the driver gets full control of the vehicle. The second driver, who has the supervision of the software, has additional brake pedals, which also deactivates the autonomous driving mode. We utilize the same planning algorithm on CoCar and the shuttles, except of vehicle specific parameters.