Characterizing zebra crossing zones using LiDAR data

DOI: 10.1111/mice.12968 INDUSTRIAL APPLICATION Characterizing zebra crossing zones using LiDAR data Alberto M. Esmorís1David L. Vilariño2David F. Arango3 Francisco-Alberto Varela-García3José C. Cabaleiro1,2Francisco F. Rivera1,2 1Centro Singular de Investigación en Tecnoloxías Intelixentes, Universidade de Santiago de Compostela, Santiago de Compostela, Spain 2Departamento de Electrónica e Computación, Universidade de Santiago de Compostela, Santiago de Compostela, Spain 3cartoLAB, Grupo de Visualización Avanzada e Cartografía, Departamento de Ingeniería Civil, E.T.S. Ingeniería de Caminos, Canales y Puertos, Universidade da Coruña, A Coruña, Spain Correspondence Francisco F. Rivera, Centro Singular de Investigación en Tecnoloxías Intelixentes and Departamento de Electrónica e Computación, Universidade de Santiago de Compostela, 15782 Santiago de Compostela, Spain. Email: [email protected] Funding information Consellería de Cultura, Educación e Ordenación Universitaria, Grant/Award Numbers: accreditation 2019-2022 ED431G-2019/04, 2022-2024, ED431C2022/16, ED481A-2020/231; European Regional Development Fund (ERDF); CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System; Ministry of Economy and Competitiveness, Government of Spain, Grant/Award Number: PID2019-104834GB-I00; National Department of Traffic (DGT) through the project Analysis of Indicators Big-Geodata on Urban Roads for the Dynamic Design of Safe School Roads, Grant/Award Number: SPIP2017-02340 Abstract Light detection and ranging (LiDAR) scanning in urban environments leads to accurate and dense three-dimensional point clouds where the different elements in the scene can be precisely characterized. In this paper, two LiDAR-based algorithms that complement each other are proposed. The first one is a novel profiling method robust to noise and obstacles. It accurately characterizes the curvature, the slope, the height of the sidewalks, obstacles, and defects such as potholes. It was effective for 48 of 49 detected zebra crossings, even in the presence of pedestrians or vehicles in the crossing zone. The second one is a detailed quantitative summary of the state of the zebra crossing. It contains information about the location, the geometry, and the road marking. Coarse grain statistics are more prone to obstacle-related errors and are only fully reliable for 18 zebra crossings free from significant obstacles. However, all the anomalous statistics can be analyzed by looking at the associated profiles. The results can help in the maintenance of urban roads. More specifically, they can be used to improve the quality and safety of pedestrian routes. 1 INTRODUCTION Walking is one of the most common ways of transport. Hence, advances in the development of methods to analyze the walkability of pedestrian routes can help to improve This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. © 2023 The Authors. Computer-Aided Civil and Infrastructure Engineering published by Wiley Periodicals LLC on behalf of Editor. the quality of life in urban environments (Olszewski, 2007). These advances are even more essential to improve the quality of life for people with reduced mobility (Lima & Machado, 2019). They are also crucial in improving the safety of pedestrians. More concretely, discovering Comput Aided Civ Inf. 2023;1–22. wileyonlinelibrary.com/journal/mice 1 2ESMORÍS et al. and repairing crosswalks’ damage and aging situations is essential to keep crosswalk pedestrians safe (Chen et al., 2021). This fact holds especially for children who need to walk from house to school multiple times during the week. Therefore, it is helpful to have an accurate and detailed characterization of some features of the zebra crossing, including its geometry, road surface condition, painting, and height of the sidewalks at the extremes. While this information can be approximated from traditional mapping techniques typically based on representing the earth’s surface projected into a plane (Kumar, 2018), we propose to use light detection and ranging (LiDAR) technology. It is based on an active remote sensing sensor that emits pulses of polarized light to measure distances to the emitting point (McManamon, 2019).Themainadvantages of LiDAR over conventional imaging cameras are its great accuracy and the fact that it provides more data to characterize the studied surfaces in detail. Its primary disadvantage is that it is usually more expensive. For the case study in this paper, the data were obtained using mobile LiDAR scanning (MLS). The scanner was placed in a vehicle that acquires the data as it moves (Guan et al., 2016). When using MLS, it is possible to capture data from close distances, leading to highly accurate threedimensional (3D) point clouds. Moreover, it is possible to georeference the acquired data using a navigation system composed of a Global Navigation Satellite System and an inertial measurement unit (IMU). Therefore, MLS is an interesting option to accurately characterize transport infrastructures such as streets, roads, highways, and railroads (Haala et al., 2008). All the data used in this work were acquired using an Optech Lynx Mobile Mapper. It comprises two LiDAR sensors, four cameras, and an IMU. The information from each sensor is linked by a GPS time stamp, obtaining a point cloud with geometric and radiometric information. All the equipment specifications can be consulted at Optech-Incorporated (2021). Note that the Lynx Mobile Mapper includes the Lynx Survey software to handle the inertial/positional processing and the LiDAR postprocessing. The methods proposed in this work are meant to be applied to the generated output point cloud instead of the raw data. The work in this paper comes from BIG-GEOMOVE, a project funded by the Spanish General Traffic Authority, which provided the case study to develop and validate our approach. In the framework of this project, the aim was to characterize the zebra crossings on school routes using MLS data. These data are publicly available as ground truth for researchers in the field (Cartolab, 2021). Zebra crossings are critical in pedestrian routes because they are used by motor vehicles and pedestrians. Therefore, they represent crucial points in road safety. The misuse of zebra crossings causes 15% of driver offenses on Spanish streets. These figures increase up to 40% of the road crashes on urban roads (Servicio de Estadística. Observatorio Nacional de Seguridad Vial, 2018). Having a method to identify and accurately characterize the crosswalks is essential to keep pedestrian routes in good and safe conditions. That is the aim of the algorithm introduced in this paper. Our proposal can be divided into two different parts. The first one deals with the identification of zebra crossings. The second part focuses on the profiling and characterization of the zebra crossings. There are many works about segmenting and classifying objects from LiDAR point clouds. However, there are few about what to do with the classified objects. This second part represents the main contribution of this paper. The profiling of the zebra crossing surface is useful for anomaly detection purposes and to accurately analyze slopes, curvature, height, and size. The characterization summarizes the state of the zebra crossing as a set of quantitative parameters. Finally, the structure of the paper is briefly summarized. First, different related works are discussed in Section 2. Second, a description of the method to identify and characterize zebra crossings is detailed in Section 3. Afterward, the case study results are discussed in Section 4. Finally, the future work is discussed in Section 5, and a summary of the main conclusions is presented in Section 6. 2 RELATED WORK There is rich literature regarding both segmentation and characterization of objects of interest from LiDAR data. For instance, Jung et al. (2019) proposed a RANSAC-based algorithm to segment road surfaces using a double polynomial fitting. Then, they rasterize to avoid the prohibitive computational burden and use an algorithm based on morphological filters to extract lane markings. Alternatively, Yan et al. (2016) presented a method to extract road markings working with the scan lines. They outperformed the previous work from Yu et al. (2015) based on hierarchical features . Other works, such as Yang et al. (2020), focused on road marking segmentation on noisy point clouds coming from low-cost MLS . The method is divided into three steps. First, the road surface is segmented; then, road marks are extracted, combining median filtering of intensity values with edge detection, and, finally, a refinement stage is applied. There are other general-purpose methods, like the one proposed by Wang et al. (2019), in which, using a density-based spatial clustering of applications with noise-based approach (Ester et al., 1996), they managed to segment vegetation and different types of buildings from a scene of Baltimore, USA, sensed using an aerial LiDAR scanner. Some works use terrestrial laser scanning to scan a steel structure (Park et al., 2007). The acquired 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ESMORÍS et al. 3 point cloud is then processed by an algorithm that estimates the local coordinate system of the structure and computes the displacements in the estimated coordinate system to reach a final estimation of the deformation with millimetric precision. Similar approaches were proposed to deal with the problem of health monitoring of structures (Park et al., 2015) and walls (Riveiro et al., 2016) to distinguish masonry from joints in a similar way we identify crossing marks. Also, more particular segmentation proposals exist, such as the one of Smith and Sarlo (2021), where a method for extracting structural layout is proposed. It is based on aligning point cloud axes with primary structure axes and then transforming the point cloud to a stack of images, so convolutions and morphological operations have a lower computational cost. Then, centroids from images are brought back to 3D space to build a sparse point cloud to which a linear region growing algorithm is applied to extract centroidal axes defining the structure layout. Also, in Pauly et al. (2002), an analysis and comparison of several surface simplification methods for point-sampled geometry are presented. These methods are incremental and hierarchical clustering, iterative simplification, and particle simulation algorithms to approximate point-based models. However, these methods and those coming from nonmobile LiDAR scanners are beyond the scope of this work. On the other hand, there are works on characterizing roads from mobile LiDAR data. For instance, Holgado- Barco et al. (2014) proposed a method to characterize roads based on longitudinal profiles. They use cross sections by fitting a plane using principal component analysis to compute the slope and superelevation from the eigenvalues and eigenvectors contained in the best fitting plane. A methodology for the automated extraction of the topographical parameters of road axes using datasets obtained by LiDAR sensor is also proposed by Holgado-Barco et al. (2015). In particular, the road centerline is extracted and modeled based on scan angle and intensity segmentations. The horizontal alignment of the road axis is parameterized based on azimuth and curvature parameters. Furthermore,Díaz-Vilariño et al. (2016) introduced a method based on computing roughness and its absolute average, mean squared root, skewness, and kurtosis for a road section. Then, they apply a k-means clustering strategy to characterize roads concerning their material, either paving or asphalt. Some works integrate point clouds with building information modeling and geographic information systems (GIS), where LiDAR is preferred to optical methods for road characterization because it can penetrate canopies. It is the case for the work of Barazzetti et al. (2020), which integrated LiDAR data to segment and characterize roads, trees, and buildings, considering characterization parameters for a road in different situations such as a tunnel, a bridge, or a standard road. Other proposals take advantage of the detailed point clouds obtained through MLS to analyze surfaces in detail, as shown in the work of Famili, where pavement rutting is detected by fitting a surface from which features such as aspect, slope, and plan and profile curvatures can be computed (Famili, 2020). Cai and Rasdorf (2008) concluded that LIDAR point clouds could be directly used in modeling linear objects in a 3D space instead of being interpolated into traditional elevation data. Their method was used to predict 3D distances for road centerlines. The use of artificial intelligence methods to characterize 3D point clouds is possible using tools such as PointNet, which can extract features from the points efficiently and robustly (Qi et al., 2017). This fact is shown by Ren et al. (2015),who presented a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position, which is part of the problem we deal with in our paper. Also, in Munoz et al. (2009), a functional gradient approach is adapted for learning high-dimensional parameters of random fields to perform multilabel classification of point clouds. Focusing on road markings detection, Wen et al. proposed a deep learningbased approach to detect, classify, and even complete marks feeding a convolutional neural network (Le Cun et al., 1989). They use mobile LiDAR data projected onto a horizontal plane under the assumption that road markings are a two-dimensional (2D) structure (Wen et al., 2019). Moreover, Cheng et al. (2017) presented an algorithm for the segmentation and classification of road markings based on transforming the 3D MLS point cloud to the 2D space of imagery. Riveiro et al. (2015) developed a method for detecting zebra crossings from mobile LiDAR data tested to be applied for road management purposes, using rasterization techniques and image-processing algorithms . In L. Li et al. (2016), a stepwise procedure for recognizing and reconstructing zebra crossings using mobile laser scanning data was presented. Zebra stripes are recognized according to the rectangular feature and fixed size. While most works on marks are mainly oriented to segmentation and classification tasks, our proposal focuses on a deep 3D characterization of zebra crossings as a primary and novel contribution. Thus, road markings are not understood as fully reducible to a 2D structure, which differs from state-of-the-art approaches that focus more on segmentation and classification than on characterization. Moreover, our work is also related to the topic of autonomous driving. While our proposal focuses on road safety and ease of use from the pedestrian perspective, 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 4ESMORÍS et al. FIGURE 1 The entire workflow of the algorithm the primary concern on safety comes from the potential interactions between vehicles and pedestrians. Thus, works like Gouda et al. (2021) proposed a simulation-based method to assess how adequate a road is for autonomous driving using octrees for volumetric analysis over point clouds can be complemented with our methods to analyze crossing zones and include pedestrian-safety criteria in their assessment. Other works, such as Q. Li et al. (2022), also use a simulation-based approach to analyze the behavior of automated vehicles in contexts involving lane-changing. The method is focused on the autonomous vehicle perspective. Hence, it could benefit from including an analysis similar to ours to assess pedestrian safety in urban scenarios. One more autonomous driving work is the one of Verstraete and Tampère (2022) that extends the route choice component of a dynamic traffic assignment model to analyze traffic propagation . These models often focus on traffic analysis from the vehicle perspective, for instance, considering the behavior of vehicle queues to predict traffic behavior. Crossing zone analysis could be used to extend these models to work better in urban environments where pedestrian behavior impacts traffic. 3ZEBRA CROSSING IDENTIFICATION AND CHARACTERIZATION The entire process of zebra crossing identification and characterization is illustrated in Figure 1. First, the zebra crossings are identified from the MLS point cloud. Then, each of these pedestrian crossings is divided into strips to which a profiling stage is applied. The profiling results will detect anomalies that might condition the selection of the zebra crossing for the subsequent quantification, as this is only performed on well-captured zebra crossings. The whole process generates qualitative and quantitative information describing the state of the zebra crossings and their surroundings. As the director vectors and the vertices of the zebra crossing are available, it is possible to load the georeferenced output from the quantitative characterization into GIS software. This type of software helps study pedestrians’ mobility in urban areas. For instance, some methods propose loading data from a Topographic Database into a GIS system to assist decision-making tasks related to accessibility and transport in urban contexts (Rossetti et al., 2020). Similar works propose using GIS software to analyze the walkability in urban environments from the pedestrian perspective (Caselli et al., 2021). The importance of these initiatives is justified by the necessity of improving walkability and transport in modern cities. That is what models such as the 15-min city model aim to do by reducing the amount of lost time because of traffic conditions (Moreno et al., 2021). There are also algorithmic proposals that use GIS software to estimate the travel time of pedestrians through a load of different layers of information characterizing the different paths (Rossetti et al., 2015). Finally, the whole workflow of the algorithm is depicted in Figure 1. 3.1 Zebra crossing identification The first stage to identify zebra crossings is a presegmentation of road markings from the LiDAR point cloud. 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ESMORÍS et al. 5 This initial segmentation is performed under the premise that roads are large planar areas, and road markings feature higher intensity than the surrounding background. Therefore, the position (𝑥,𝑦,𝑧)and the intensity (𝐼) must be considered for each point. The first step is to perform a surface variation filtering to discard all the points that do not belong to a planar area. The surface variation metric is a well-known tool to measure the deviation of the points with respect to the tangent plane (Pauly et al., 2002). It can be calculated using Equation (1). In this equation, 𝜆1≥𝜆2≥𝜆3≥0are the ordered eigenvalues of the 3×3covariance matrix of the neighborhood. These neighborhoods are computed using an octree, a well-known data structure, to speed up spatial queries (Meagher, 1980). 𝑃𝜆=𝜆3 𝜆1+𝜆 2+𝜆 3 .(1) Afterward, a plane-based clustering is carried out in such a way that points associated with planes with similar orientations are labeled into the same cluster. To this end, the normal vector of the best fitting plane associated with the neighborhood of each point is considered. That corresponds with the smallest eigenvalue of the covariance matrix mentioned above (𝜆3). There is a high variability of the normal vectors coming from the presence of multiple elements and structures in urban environments, for example, trees, sign poles, and curbs, among others. However, most road points are included in the same cluster, usually the largest one. The road cluster can also be identified by considering the scanning angle. Since the LiDAR sensor is deployed on a vehicle driving on the road, most of the LiDAR points in the vehicle trajectory should be included in the road cluster. The most significant feature of the road marking points is the intensity, usually higher than the surrounding background. Therefore, an adaptive intensity filtering is helpful to identify road marking points. Filtering based on a global threshold is not accurate enough due to the high variability of road intensity in different areas. Hence, a local adaptive threshold that analyzes the surrounding area of each point is proposed. This threshold is calculated from the average and maximum intensities of road points around the point under consideration, as shown in Equation (2). In this equation, 𝐼𝐿𝑚 and 𝐼𝐿𝑚𝑎𝑥 are the average and the maximum intensities in the surrounding area, respectively. 𝑇𝐼=𝐼𝐿𝑚 +𝐼 𝐿𝑚𝑎𝑥 2.(2) The initial processing states described until now are summarized in the pseudocode of Algorithm 1. Inthis pseudocode, 𝜆𝑖isthe set ofeigenvaluesestimatedfor the 𝑖th point, and 𝜆𝑖1,𝜆 𝑖2,𝜆 𝑖3 stand for the first, second, and third eigenvalues sorted as previously described. Furthermore, 𝑣𝑖are the eigenvectors associated with the 𝑖th point. Thus, 𝑣𝑖3 is the eigenvector that estimates the orthonormal vector of the best fitting plane in the neighborhood of the 𝑖th point with radius 𝑟>0. The adaptive intensity threshold function details how to compute the local adaptive threshold for the intensities of the points in a given neighborhood. The eigen_analysis function computes the eigenvalues and eigenvectors of the structure’s space covariance matrix of the points inside the given neighborhood. The 𝜖threshold is a decimal error tolerance value that governs the minimum required likelihood to be a plane. In the final step, the algorithm returns the 3 point cloud that is the result of applying all the previously explained filters. After the previous step, all the identified road marking points are included in the same group. With the aim of identifying individual road markings, a connected component labeling is carried out (He et al., 2017). As a result, new groups containing only connected road marking points are created. Some objects, like the upper part of cars, can pass previous filters and produce clusters with planar and highintensity points. However, they are significantly above the road surface, so a simple local average of the 𝑧coordinate is enough to filter them out. Next, the identification and reconstruction of zebra crossings from the previous road marking segmentation are carried out. The identification is approached by multiple operations labeled as point density and geometric 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 6ESMORÍS et al. filteringin Figure 1. The first step discards groups with only a few points because they cannot be reliably categorized as zebra crossing bars at this stage. Then, its minimum area bounding box is calculated for any cluster of road marking points. The computational burden of the problem is low due to the significant reduction of data volume in the previous road marking segmentation. Thus, a naive approach based on comparing the areas of the bounding boxes in a linear search process over the rotation angle is suitable. However, more sophisticated algorithms could also be used, such as the minimum area encasing rectangle based on the minimal perimeter convex polygon (Freeman & Shapira, 1975). Once the minimum area bounding boxes are known, those with a long edge significantly greater than the short edge are discarded. The main goal here is to discard solid lane and edge markings. These road markings are easily identified because their length is multiple times greater than their width. Assuming that zebra crossing bars fit a rectangular geometry well, most of the points inside a minimum area bounding box containing them are expected to be segmented as road marking points. Therefore, all those clusters with a significant amount of nonroad marking points are discarded. Finally, note that consecutive zebra crossing bars are expected to have a similar orientation. Hence, the short edge angle of each bounding box is considered to perform orientation comparisons. This angle can be efficiently computed using the line equation to find the slope of the short edge. Those bars with similar slopes and short edges fitting the same line are likely to be part of a zebra crossing. There are multiple solutions to identify zebra crossings. However, a common issue for any method that uses restrictive filtering criteria to prevent false positives is that it will discard some zebra crossing bars in poor conditions or partially occluded. Also, they might have failed to be properly segmented either due to limitations of the algorithm or to measurement errors in the input data. Therefore, a reconstruction of missing zebra crossing parts is usually required. The so-called iterative rebuilding process aims to identify missing bars by using the likelihood of being a zebra crossing bar for each cluster. In this step, a search process for each reliable candidate to be a zebra crossing bar is carried out as illustrated in Figure 2. This search process is performed in both backward and forward directions. It expands the bounding box of the known zebra crossing bar considering an expansion factor parameter (𝑒𝑝). Then, the expanded bounding box is shifted along the zebra crossing direction, considering an expansion step parameter (𝑒𝑠). Once the search space has been determined, the rebuilding process consisting of three consecutive steps can be applied. For the first step, it is necessary to calculate an adaptive intensity threshold defined by a quantile factor (𝑐𝑓). Note that 𝑐𝑓 = 0.5 implies considering the median. However, a slightly higher value of 𝑐𝑓 = 0.67, which implies considering the second tertile instead, has been found to work better. The quantile factor is used to multiply the number of points inside the bounding box (𝑛), leading to the quantile index (𝑐𝑖). Assuming an index space where points are sorted from minimum to maximum intensity, it is possible to define the low median point index (𝑚𝑙𝑖) as the index of the median point for points between 0 and 𝑐𝑖. Analogously, the high median point index (𝑚ℎ𝑖) is defined as the index of the median point for points between 𝑐𝑖 and 𝑛(Figure 3). The second step considers a minimum contrast difference parameter (𝑚𝑐𝑑) that determines when the difference between high- and low-intensity points is relevant. First, the contrast difference (𝑐𝑑) is calculated as the difference between the intensity for the point at 𝑚𝑙𝑖 and the point at 𝑚ℎ𝑖 (𝑐𝑑 = 𝐼𝑚ℎ𝑖 −𝐼 𝑚𝑙𝑖). If 𝑐𝑑 is less than 𝑚𝑐𝑑, then the rebuild process is aborted. Otherwise, all the points are segmented again by comparing their intensity with the point at the 𝑐𝑖 index. More formally, any 𝑖th point will be labeled as a road marking point if 𝐼𝑖>𝐼 𝑐𝑖. Finally, the third step is based on an adaptive contrast function to relax 𝑚𝑐𝑑 in subsequent reconstruction, assuming that the first rebuild must be more exigent to avoid false positives. On the other hand, the farther from the LiDAR sensor, the lower the intensities and, in consequence, their differences. A simple yet valid adaptive contrast function is suggested in Equation (3). This function is applied once for each reliable zebra crossing bar. Our experiments show that 𝑙𝑐𝑑 values slightly greater than 1 work properly to gradually relax the rebuild process criteria at each step. 𝑚𝑐𝑑𝑡+1 =𝑚𝑐𝑑𝑡 𝑙𝑑𝑐 .(3) At this stage, any reliable zebra crossing bar that has not been assigned to any zebra crossing is iteratively projected. For this purpose, its bounding box is expandedin the direction of its short edge. The expansion at any iteration (𝑖𝑡𝑒𝑟) is defined as 𝑝𝑓 ⋅ 𝑖𝑡𝑒𝑟 ⋅ 𝑆𝐸length, where 𝑝𝑓 is the projectionfactor parameter.Ateachexpansion,all nonpreviously considered bounding boxes with a non-null intersection with respect to the expanded one are analyzed. More concretely, a line equation fit is used to find the orientation and the intercept for each edge of the bounding box. Afterward, the line intersection between the edges of both bounding boxes is checked as shown in Figure 4. Once all zebra crossing bars are known, it is possible to proceed with the merging of split zebra crossings. For this purpose, zebra crossings that are close enough 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ESMORÍS et al. 7 FIGURE 2 This figure illustrates the search space of the zebra crossing rebuilding stage. The known trust bar represents a bar that fits an ideal zebra crossing bar well enough. From this bar, we start a search process based on the geometric constraints of an ideal zebra crossing. The search is carried out in both ways along the short edge of the known well-fitting bar until a region not likely to be a zebra crossing bar is found FIGURE 3 Finding 𝑚𝑙𝑖 and 𝑚ℎ𝑖 from a given 𝑐𝑓 FIGURE 4 This figure illustrates the zebra crossing projection stage for a given factor. The rectangle labeled 𝑆represents the starting bar of the projection. 𝐿𝑖and 𝑅𝑖represent the 𝑖th iteration of the leftward and the rightward projections along the short edge of the starting bar. The magnitude of the projections is given by the magnitude of the short edge length 𝑆𝐸length scaled 𝑖th times the projection factor 𝑝𝑓. The process is repeated until a nonconvergent iteration is reached are merged into a single zebra crossing. A maximum distance threshold of 1.5 m was found to bring satisfactory results. It is based on the idea of roughly approximating the ideal short edge as a 0.5-m segment; thus, the second bar is expected to start at 1.0 m and end at 1.5 m. Consequently, it is expected to be big enough to merge bars even when they are significantly separated. Nevertheless, at the same time, it is expected to be small enough to avoid merging different zebra crossings. Notwithstanding, the max distance criterium is easily adaptable to the local legislation on zebra crossings. Finally, the minimum area bounding box is calculated for all merged zebra crossings. Once the zebra crossings are correctly identified, they can be characterized. To this end, a profiling of the zebra crossing (Section 3.2) is performed, and, depending on the profiling results, a subsequent detailed quantitative characterization is computed (Section 3.3). 3.2 Profiling Three user parameters define the profiling. The first one is the desired number of strips 𝑛, so one profiling per strip will be performed. The second one is the margin between strips 𝑚. It forces consecutive strips to be separated in 𝑚 meters. The third is the expansion units 𝑘, such that each strip will be expanded outside the zebra crossing bounds in, at most, 𝑘meters. For this paper, the profiling was carried out with 𝑛=4, 𝑚=0.2,and𝑘=1,asshowninFigure5. In this figure, the points are a subsample of the zebra crossing. The fat dashed rectangle is the bounding box containing the zebra crossing. The upper left point is the starting vertex of the zebra crossing. The black edges are the projections of the director vectors of the zebra crossing, and the inner rectangles are strips whose profile must be calculated. 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 8ESMORÍS et al. FIGURE 5 Profiling with 𝑛=4,𝑚=0.2and 𝑘=1 FIGURE 6 Profiling inside a strip For the profiling, LiDAR points are mapped to its projection in the central path (the line in the middle of the strip), as shown in Figure 6. Consequently, the (𝑥, 𝑦) coordinates of the zebra crossing points are transformed into a one-dimensional domain leading to the profiles of the 𝑧 coordinate with respect to the 𝑋𝑌 plane. Four different profiles are generated: ∙Minimum profile. The minimum value in the neighborhood of a given point. ∙Maximum profile. The maximum value in the neighborhood of a given point. ∙Mean profile. Mean of the neighborhood. ∙Median profile. Median of the neighborhood. When analyzing the profile for each strip of the same zebra crossing, the presence of obstacles becomes evident. Whether a person or a car passes through the zebra crossing, some of its strips will present a significant deviation between their minimum, maximum, and mean profiles. If the zebra crossing is free from obstacles, then the minimum, maximum, and mean profiles will be close to each other. Moreover, the profiling method is useful for an exhaustive road surface analysis that considers all available points inside the same strip. Thus, sudden slope changes can be detected, whether they appear because of potholes or a too-elevated sidewalk. For potholes, the length of the 𝑋𝑌 axis between the start and endpoints of a sudden road descent gives their approximated diameter. For sidewalks, the height with respect to the road can be estimated with the difference in the 𝑧coordinate between the last smooth road point and the points after a sudden rise at the end of the road. Furthermore, some zebra crossings are lower in the middle but higher at the extremes. That can be accurately measured with the profiling method simply by considering the height difference between the strips at the extremes and those in the middle. Finally, the profiling method can be used to characterize those zebra crossings with partial occlusions and those for which there are obstacles on their surface. That is possible when at least one of the following assumptions holds. The first one is that most obstacles and partial occlusions appear in some strips but not all. For this case, it is possible to consider the characterization from the clean strips. The second one is that all the anomalies in the strip are placed either above or below the zebra crossing. On the one hand, there might be obstacles above the zebra crossing, yet the minimum profile is smooth and continuous. But then, the minimum profile can be considered the best representation of the road profile along the zebra crossing. On the other hand, there might be anomalies below the zebra crossing, yet the maximum profile is smooth and continuous. But then, the maximum profile can be considered the best representation of the road profile along the zebra crossing. It is still possible to analyze the road surface by considering the proper profile in both cases. Further details can be found in Section 4.2, where the profiling results are discussed. 3.3 Quantification Based on the profiling results, the quantitative characterization will be performed only on those zebra crossings free of relevant obstacles. This characterization is mainly subject to two user parameters. The first one is the distance 𝑑 between grid nodes. The smaller it is, the greater the quantification accuracy and the greater the computational cost. In this work, the distance between nodes was configured to 𝑑=0.3m. The second parameter is the radius 𝑟, defining the spherical neighborhood for each node. If 𝑟=𝑑∕2, then the neighborhoods will have the maximum possible size under the condition that the intersection between any pair of neighborhoods must be the empty set. If 𝑟<𝑑∕2, then the neighborhoods will have a nonmaximum size under the intersection condition mentioned above. If 𝑟>𝑑∕2, then for some pairs of neighborhoods, their 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ESMORÍS et al. 9 TABLE 1 The parameters that quantitatively characterize each zebra crossing Parameter Description Transversal vector The vector defining the transversal edge Longitudinal vector The vector defining the longitudinal edge Width Total width of the identified zebra crossing Length Total length of the identified crosswalk Height 1 Height with respect to the sidewalk Height 2 Height with respect to the sidewalk on the opposite side Transversal slope Mean and std deviation for height differences in the transverse direction Longitudinal slope Mean and std deviation for height differences in the longitudinal direction Plane slope Mean and std deviation for slope measured as angle between planes Roughness Mean and std deviation of roughness for each point Global paint quality Average quality of each crosswalk band based on points intensity Number of bars Number of strips identified on the zebra crossing Bar width Width of each identified strip at each zebra crossing Bar length Length of each identified strip at each zebra crossing Paint quality Paint quality for each identified strip at each zebra crossing Paint intensity Mean and std deviation of the intensity for each crosswalk band Paint LiDAR angle Mean and std deviation of the angle of incidence for each crosswalk band FIGURE 7 Different mesh configurations for the quantitative characterization intersection set will have a non-null cardinality. In other words, some points belong to more than one neighborhood. The proposed meshing method uses the transversal and longitudinal vectors as the basis of the plane where the mesh is placed. A visual representation of the meshing process is shown in Figure 7. For this paper, the radius defining the neighborhood of the nodes was configured to 𝑟=0.3m. The features of a zebra crossing that have been considered are summarized in Table 1. Once the mesh is built, all the edges parallel to the transversal director vector are studied to calculate statistics for the transversal slope. Analogously, all the edges parallel to the longitudinal director vector are studied to calculate statistics for the longitudinal slope. The height of the sidewalk with respect to the road is computed from the vertices at the extremes of the mesh. The roughness statistics are obtained from the roughness estimation ℜat each node’s neighborhood. This estimation is calculated as the distance between the closest point to the node and the best fitting plane with respect to its neighborhood, as shown by Equation (4). In this equation, the point 𝑝=(𝑝 𝑥,𝑝 𝑦,𝑝 𝑧)represents the closest point to the node, and the vector  𝑢=(𝑢 𝑥,𝑢 𝑦,𝑢 𝑧)is the orthonormal vector of the best fitting plane. It is worth mentioning that some geometric quantifications, such as the width and the length of the zebra crossing, are directly taken from its minimum area bounding box with no need for mesh analysis. ℜ=⟨ 𝑢,𝑝⟩=𝑢 𝑥𝑝𝑥+𝑢 𝑦𝑝𝑦+𝑢 𝑧𝑝𝑧.(4) Moreover, as the bounding box for each bar of the zebra crossing is known, it is possible to characterize the bars quantitatively. In this case, the quantitative characterization includes the number of bars, statistics describing the separation between bars, and the geometry of each bar in terms of width and length. The intensity distribution for each bar is also statistically described. A paint 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 16 ESMORÍS et al. Crosswalks marked with paint are the safer places to cross a street because they are visible to drivers. When the marking patterns are zebra crossings, the degree of identification by drivers increases compared to other marking patterns such as bar pairs or transverse lines (Fitzpatrick et al., 2010). The regulation related to crosswalks changes from one countryto another. Even in Spain,wherethezebracrossing is the most common crosswalk, there are many different regulations for its dimensions and performance characteristics. Therefore, measuring parameters that directly affect the main elements of crosswalks is an additional complementto the otherparameters to characterizethe crosswalk. The number of bars that compose the crosswalk, the length and the width of each bar, their paint quality, and their intensity distribution are parameters that we have measured directly from the LiDAR point cloud to know and assess the quality of each crosswalk. Despite all but one of the zebra crossings being identified, not all are adequate for detailed quantitative characterization. This situation occurs for several reasons, such as the degradation of the intensities for points too far from the LiDAR sensor, degenerated shapes, and occlusions. On the one hand, intensity-related problems affect only a small subset of the quantification attributes. Thus, intensityrelated issues do not prevent absolute quantification but only affect those parameters that describe the paint of the zebra crossing. On the other hand, space-related problems, such as big occlusions or significant obstacles, may degenerate the data, so the quantitative characterization might not provide valid results. For these cases, profiling is the only reliable characterization method. When strange results are found in the quantitative summary of a zebra crossing, it is not necessary to do a field check. When necessary, a visual inspection of the profiles should explain what is going on without requiring in-place checks. First, we found a correlation between roughness and zebra crossings. Looking at Figure 12, it can be seen that the greater roughness values are often located on the zebra crossing bars. We think that this is explained because deteriorated zebra crossings imply the presence of different paint fragments. In consequence, there are frequent height differences in the painted area. We conclude that the calculus of roughness over LiDAR data is accurate enough to detect these differences. Next, some different cases are depicted to highlight why, under certain conditions, it is expected that some of the results of the quantitative characterization are not representative of the zebra crossing state. Figure 8a shows one zebra crossing for which quantitative characterization is totally admissible as well as Figure 8e. However, Figure 8c contains both an inappropriate case and a missing zebra crossing. The missing zebra crossing is not characterized FIGURE 12 The visualization of the roughness of two zebra crossings. The intensity view of the point cloud is on the left side. On the right side, the point-wise roughness overlaid on the zebra crossing. Lighter colors represent the lesser roughness, while darker colors represent the greater roughness because it is not identified. Concerning the inappropriate case, it cannot be fully quantified because, from those three bars, it is impossible to obtain the length, the height of the sidewalk, the paint quality, and the number of bars. Nonetheless, it is possible to know its width and a close approximation of its roughness, transversal slope, longitudinal slope, and location. Next, an inappropriate zebra crossing due to the occlusion generated by a vehicle is shown in Figure 8b. In this case, it is impossible to obtain the number of bars, paint quality metrics, length, and transversal slope because of the pedestrian in the middle of the zebra crossing. The sidewalk height at the extreme where the vehicle is placed cannot be appropriately summarized either. Nevertheless, obtaining the width, the longitudinal slope, the height of one of the sidewalks, the location, and an approximate roughness estimation is possible. Another inappropriate zebra crossing due to the occlusion generated bythe movement of a vehicle is shown in Figure 8d. In this case, the problem causes a lack of data at one of the extremes. Thus, it is impossible to find the sidewalk’s height at that extreme. It is neither possible to know the exact number of bars nor the length of the zebra crossing. Nevertheless, all the other parameters can be correctly calculated. There are more scenarios where the zebra crossings cannot be properly segmented. Some of these cases may come from the unexpected presence of vehicles above the zebra crossing, as illustrated in Figure 8f. In this case, the characterization of the longitudinal and transverse vectors, the width, the height, the slope, the roughness, and an approximated analysis of its bars can be adequately quantified. Nonetheless, one of its heights has a value of 0.84 m. That is an anomaly caused by the presence of a car 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ESMORÍS et al. 17 TABLE 2 Summary of the 18 well-adjusted zebra crossings. For each repeated zebra crossing in the MLS point clouds, only the best identified case is considered Feature AVG. Min. Max. Range Std. Dev. Calculated Area (m2) 32.19 14.92 53.21 38.29 8.65 Length (m) 7.88 4.18 13.15 8.97 1.91 Width (m) 4.07 3.56 4.54 0.98 0.20 Length/width (m) 1.93 1.17 3.25 2.08 0.44 Rougness (mm) 2.32 1.43 6.32 4.89 1.27 Num. bars 8.20 413 91.90 Painted Area (m2) 18.76 8.90 31.98 23.08 5.12 Length/Num. bars 0.97 0.90 1.08 0.17 0.05 Height 1 (m) 0.14 0.04 1.00 0.96 0.33 Height 2 (m) 0.17 0.02 1.00 0.98 0.39 Global paint quality 0.91 0.81 0.96 0.16 0.04 Global paint intens mean 2141.1 1309.4 2932.6 1623.2 358.5 Global paint intens std dev 589.2 423.0 1103.7 680.6 155.9 at the corresponding extreme of the zebra crossing. Nevertheless, a field check is not necessary to find the problem. It is enough to check the corresponding profile. Doing this will show that there is a car-like anomaly in the profile. A similar case is discussed in Section 4.2. In this case, there is a car at one of the extremes of the zebra crossing. Nonetheless, looking at the profiling represented in Figure 9b,it is easy to see that the presence of an obstacle above the zebra crossing is causing inaccurate quantification. Another problem is the one illustrated in Figure 8g.In this case, the zebra crossing quality is not optimal, and there is also a pedestrian walking over it. Hence, while the zebra crossing is identified, there are problems with its quantitative characterization. On the one hand, the location of the zebra crossing, its width, and an approximation of its roughness, transversal slope, and longitudinal slope are feasible. On the other hand, the total number of bars, the length of the zebra crossing, and the paint quality metric cannot be appropriately calculated or even approximated. One of the most uncommon yet possible problems is the one illustrated in Figure 8h, which belongs to a zebra crossing that the algorithm cannot safely unify. The problem here comes from the ghostly projection of a moving car which splits the zebra crossing and distorts the LiDAR data. Despite this might be fixed by adapting the projection and merging stages of the algorithm, it is important to note that the problem is not the presence of an obstacle but distorted scanning. Therefore the data are not reliable. The results of the quantitative characterization are summarized in two different tables. Table 2contains the mean TABLE 3 Summary of the bars from the 18 well-adjusted zebra crossings. For each repeated zebra crossing in the MLS point clouds, only the best identified case is considered Feature AVG. Min. Max. Range Std. Dev. Bar max length (m) 4.05 3.95 4.51 0.56 0.13 Bar max width (m) 0.65 0.53 0.80 0.27 0.11 Bar min length (m) 3.22 0.68 3.94 3.26 0.73 Bar min width (m) 0.48 0.34 0.53 0.19 0.05 Bar mean length (m) 3.82 3.20 4.40 1.20 0.24 Bar mean width (m) 0.54 0.49 0.64 0.14 0.04 Bar mean quality 0.91 0.79 0.96 0.18 0.05 Bar mean intens 1674.7 580.7 2694.2 2113.4 465.3 Bar max intens 3446.0 2820.7 3837.2 1016.5 285.1 Bar min intens 400.8 49.6 878.9 829.3 223.6 Bar mean dev intens 491.6 169.9 1082.0 912.1 192.9 values obtained for some of the calculated parameters. The quantification shows that the zebra crossings have an average width of 4.07 m and an average length of 7.88 m. It also shows that there are very different cases. On the one hand, the minimum length case has 4.18 m. On the other hand, the maximum length case has 13.15 m. The difference in height between the roadway and the sidewalk shows minimum values of 2 and 4 cm. This difference indicates the existence of crosswalks with ramps to facilitate the movement of people with reduced mobility, using wheelchairs, or carrying a stroller. However, we also observed excessive height measurements, characterized by maximum values of 1 m, indicating the existence of some obstacles in the crosswalk. If we do not consider these extreme values, the average values of these heights at the ends of the zebra crossing are 14 and 17 cm. These values show that many pedestrian crossings were not adapted for reduced mobility. The average number of bars for characterized zebra crossings is 8, which gives an average width of 97 cm for each pair composed of a bar and its associated nonpainted space. Table 3contains the values of different parameters measured for all the bars of the characterized zebra crossings. As the average width of each bar is 54 cm, the gaps between bars have a size of 43 cm for the average case. However, there are bars with widths ranging from 48 to 65 cm. The measured width is very close to the expected ideal length of 50 cm given by the ONCE, an official Spanish organization that aims to improve the quality of life for disabled people (Pereda & Móuriz, 2000). We can calculate the painted area of each zebra crossing by having the length and width measurements of all the bars that compose it. Table 3gives us the mean value of the painted area of the characterized crosswalks, which is 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 18 ESMORÍS et al. 18.76 m2. This mean value approximately represents 60% of the entire zebra crossing surface. We detected cases where the painted area exceeds 66% of the crosswalk surface. The paint quality on the characterized zebra crossings is very high, with a mean value of 0.91 out of 1, a small range, and a reduced standard deviation. We argue that LiDAR intensity is a useful feature to characterize the paint of road markings based on the work of Burghardt et al. (2021) on machine vision and LiDAR data concerning the visual recognition of road markings. In this work, different road markings made of different materials with different colors and retroreflectivities were studied under different lighting, rain, and fog conditions. The experiments were performed in a 100 m long climatic wind tunnel operated by Rail Tec Arsenal Fahrzeugversuchsanlage GmbH (Vienna, Austria). They found it is possible to achieve a correlation between the measured contrast ratio and the LiDAR intensity above the 80% when using the proper LiDAR scanner. They also reported highquality recognition from typical cameras and LiDAR under dry conditions. Moreover, in the case of adverse conditions, their experiments with one LiDAR scanner with a wavelength of 1550 nm and another with 905 nm showed that it is possible to have good measurements by selecting the proper scanner. With the purpose of validating the results from the quantitative characterization of the zebra crossings, the output generated by the processing of LiDAR data was compared with the manually digitized geometry of the zebra crossings. The polygon enclosing each crosswalk was manually digitized using the intensity values to represent the LiDAR points. That polygon is compared with the one automatically generated by the crosswalk identification algorithm. The results give average variations of 8.6% andinnocase reach 20%. The cases with the most remarkable difference are due to some circumstance that makes it difficult to automatically identify the complete crosswalk, as mentioned above. Figure 13 shows two examples corresponding to the crosswalks identified in Figures 8h and 10a.The manually digitized polygon is colored in green, while the polygon generated from the identification algorithm is colored in orange. A field check performed by civil engineers was used to verify the results’ accuracy. The validation was applied to the 37 different zebra crossings in the surroundings of a school in the city of A Coruña, as illustrated in Figure 14. The verification showed that 18 of the 37 cases were fully and correctly characterized, both in the number of bars and geometric parameters. The differences are mainly due to the nonidentification of bars of much shorter length located at the ends of zebra crossings placed at curved intersections. According to the different reasons described above, the other 19 zebra crossings could not be adequately quantified for at least one of the parameters. It is interesting to note that most of the problems are due to deficiencies in the paint quality of the zebra crossing bars. This fact reinforces the importance of the method to identify and characterize the zebra crossings and detect those in poor condition. 5FUTURE WORK Crosswalks are a fundamental element in the safety of walking routes as they are spaces where pedestrians and cars coincide. Therefore, their knowledge and analysis should be the object of special attention for those responsible for urban road infrastructures. Our work demonstrates that it is possible to identify and characterize zebra crossings from LiDAR data. Thus, the full integration of our method as a plugin for GIS software would significantly improve our proposal, at least for application purposes. From the data acquisition perspective, it should be feasible to use an algorithm to extract best conditioned data for quantitative statistical characterization. The most straightforward approach would be selecting the best conditioned data from each scanning for each zebra crossing. However, in many cases, moving obstacles on the zebra crossing surface suggest that a second pass might be clean or have a moving obstacle in a different position. Our profiling method can determine the regions occupied by the obstacles. Thus, merging the point clouds and selecting points from clean regions to fill problematic regions should be studied. Furthermore, our profiling method has the potential to automatize the validation of the quantitative statistical analysis. Even for the most problematic cases, we often have at least one profile that approximates well the road surface and others that do not. Those not fitting well on the road surface must have a significantly greater deviation with respect to the best fitting line that can be easily obtained by linear regression. Once the problematic cases are detected, if they have slope changes greater than a few centimeters, it is clear that curbs cannot cause them. Thus, problems must be expected regarding some of the values in the statistical summary. Weshouldalsoadvanceinobtainingparametersthat characterize other features of the crosswalks, such as the field of visibility, a more thorough characterization of sidewalk pavements concerning the easiness for blind people, vertical signs, lighting elements, and many more. In any case, we believe that our methodology can help the development of crosswalk inventories that might be incorporated into databases for analysis in GIS or in applications that deal with pedestrian routes for people with any mobility condition. 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ESMORÍS et al. 19 FIGURE 13 The location of the 37 zebra crossings analyzed in the surroundings of the Fogar de Santa Margarida school, in the city of A Coruña (Spain). Note that some zebra crossings were analyzed twice. This explains why there are 49 zebra crossings in the MLS dataset FIGURE 14 Manually digitized zebra crossings. The green color represents the manually digitized georeferenced layer. The orange color represents the zebra crossing from the LiDAR point cloud There are two straightforward extensions of our work. The first one is the use of our profiling method to characterize different road and sidewalk regions. These can be, for instance, different types of crosswalks, road intersections, and elevation changes. We expect that the profiling works well for any other case as long as it is a ground region whose boundaries can be represented by a rectangle in the plane. The second extension is the generation of different quantification summaries for other crosswalks different from zebra crossings. Moreover, from the perspective of crossing zones, it should be feasible to apply both the profiling and the statistical summary methods to cases distinct from the ones in our study case. For instance, the profiling method can also be applied to trapezoidal or curved crossing zones. It should also be feasible to compute the profiles of speed 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 20 ESMORÍS et al. tables. Since they have similar rectangular geometry to our study case, computing the profiles over speed tables should be straightforward. Unfortunately, there are no speed tables in the regions contained in the project; thus, we could not explore this in our work. Finally, another research issue to be addressed is validating LiDAR optical features, such as intensity, concerning the analysis of road marking paint. While there are some works in this direction, they were conducted under controlled conditions. However, real applications often present challenges absent in laboratory-like contexts or controlled testing environments. Further research oriented to real applications characterizing road markings is necessary to understand the true potential of LiDAR regarding the maintenance of urban roads. 6CONCLUSIONS LiDAR point clouds are an interesting alternative to imagery for the detailed characterization of surfaces. There are multiple methods to identify road markings in point clouds. It is even possible to project the LiDAR data to an image to perform the comparison with state-of-the-art methods, such as convolutional neural networks. However, a detailed analysis of the segmented surfaces is not feasible from images. For these purposes, the LiDAR point clouds are much more adequate. There are two main reasons for that. The first one is the isometric nature of the scanning, which preserves the distances between points; unlike images, geometrical analysis of the point cloud results in spatial magnitudes corresponding to the real world. The second one is the increased amount of information. These facts make the LiDAR data more robust against scanning problems. Therefore, it is possible to handle both scanning anomalies and real obstacles. Our method to identify zebra crossings can detect 98% of all the zebra crossings in our dataset. Nonetheless, this is not a challenge nowadays. Our major contributions are a method to extract accurate profiles and a method to generate a quantitative summary of the condition of the zebra crossings. The proposed profiling method is robust to outliers, capable of detecting anomalies, and well-suited for a detailed characterization of curvature, slope, height differences, and size. It can accurately describe all the identified zebra crossings, even in the presence of cars, pedestrians, and potholes. Furthermore, it can also be used to characterize the potholes and the obstacles themselves. For those cases with a clean enough profile, our characterization method correctly summarizes the state of the zebra crossing in a set of quantitative parameters. For instance, it is possible to know the paint area and assess the need to repaint some crosswalks. However, the coarse-grain nature of the statistical summary makes this method sensible to obstacles. Thus, only half of the zebra crossings have a perfect statistical summary free from anomalous measurements. Nonetheless, the profiling and the quantitative summary complement each other. If problematic values are found in the statistical summary, doing a field check or repeating the data acquisition is unnecessary. Looking at the associated profile is enough to understand the actual condition of the zebra crossing. Consequently, anomalous values in the statistical summary can be directly understood by looking at the output of the profiling method. Moreover, we found that the profiling method works well for all detected zebra crossings and that there is always at least one profiling that accurately represents the surface of the zebra crossing. In summary, LiDAR data’s capabilities go far beyond identifying objects. For this purpose, image processing is usually cheaper and provides excellent results. Nevertheless, LiDAR data outperforms images for the accurate spatial characterization of identified objects. Consequently, we consider LiDAR a promising data source for analyzing and maintaining urban infrastructures. ACKNOWLEDGMENTS This work has received financial support from the Consellería de Cultura, Educación e Ordenación Universitaria (accreditation 2019-2022 ED431G-2019/04, reference competitive group 2019-2021, ED431C 2018/19 and the predoctoral grant of Alberto Esmorís Ref. ED481A-2020/231) and the European Regional Development Fund (ERDF), which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System. This work was also supported by the Ministry of Economy and Competitiveness, Government of Spain (Grant Number PID2019-104834GB-I00) and the National Department of Traffic (DGT) through the project Analysis of Indicators Big-Geodata on Urban Roads for the Dynamic Design of Safe School Roads (Grant Number SPIP2017-02340). CONFLICT OF INTEREST The authors declare no potential conflict of interest. REFERENCES Barazzetti, L., Previtali, M., & Scaioni, M. (2020). Roads detection and parametrization in integrated BIM-GIS using LiDAR. Infrastructures,5(7), 55. Burghardt, T. E., Popp, R., Helmreich, B., Reiter, T., Böhm, G., Pitterle, G., & Artmann, M. (2021). Visibility of various road markings for machine vision. Case Studies in Construction Materials,15, e00579. 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ESMORÍS et al. 21 Cai, H., & Rasdorf, W. (2008). Modeling road centerlines and predicting lengths in 3-D using LiDAR point cloud and planimetric road centerline data. Computer-Aided Civil and Infrastructure Engineering,23,157–173. Cartolab (2021). Mobile mapping Geomove.https://cartolab.udc.es/ geomove/datos/ Caselli, B., Carra, M., Rossetti, S., & Zazzi, M. (2021). From urban planning techniques to 15-minute neighbourhoods. A theoretical framework and GIS-based analysis of pedestrian accessibility to public services. European Transport/Trasporti Europei,85, (10). Chen, Z., Luo, R., Li, J., Du, J., & Wang, C. (2021). U-Net based road area guidance for crosswalks detection from remote sensing images. Canadian Journal of Remote Sensing,47(1), 83–99. Cheng, M., Zhang, H., Wang, C., & Li, J. (2017). Extraction and classification of road markings using mobile laser scanning point clouds. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,10(3), 1182–1196. Díaz-Vilariño, L., González-Jorge, H., Bueno, M., Arias, P., & Puente, I. (2016). Automatic classification of urban pavements using mobile Lidar data and roughness descriptors. Construction and Building Materials,102,208–215. Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD’96: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (pp. 226–231). KDD. Famili, A. (2020). Pavement surface evaluation using mobile terrestrial LiDAR Scanner (unpublished Ph.D. thesis). Graduate School of Clemson University, Clemson SC. Fitzpatrick, K., Chrysler, S. T., Iragavarapu, V., & Park, E. S. (2010). Crosswalk marking field visibility study (Technical report). United States Federal Highway Administration. Office of Safety Research and Development, Washington, DC. Freeman, H., & Shapira, R. (1975). Determining the minimum-area encasing rectangle for an arbitrary closed curve. Communications of the ACM,18, 409–413. Girardeau-Montaut, D. (2020). CloudCompare version 2.11, GPL software. http://www.cloudcompare.org/ Gouda, M., Mirza, J., Weiß, J., Ribeiro Castro, A., & El-Basyouny, K. (2021). Octree-based point cloud simulation to assess the readiness of highway infrastructure for autonomous vehicles. Computer- Aided Civil and Infrastructure Engineering,36(7), 922–940. Guan, H., Li, J., Cao, S., & Yu, Y. (2016). Use of mobile LiDAR in road information inventory: A review. International Journal of Image and Data Fusion,7,219–242. Haala, N., Peter, M., Kremer, J., & Hunter, G. (2008). Mobile LiDAR mapping for 3D point cloud collection in urban areas - A performance test. Applied Geomatics,37. He, L., Ren, X., Gao, Q., Zhao, X., Yao, B., & Chao, Y. (2017). The connected-component labeling problem: A review of state-of-the- art algorithms. Pattern Recognition,70,25–43. Holgado-Barco, A., Gonzalez-Aguilera, D., Arias-Sanchez, P., & Martinez-Sanchez, J. (2014). An automated approach to vertical road characterisation using mobile lidar systems: Longitudinal profiles and cross-sections. ISPRS Journal of Photogrammetry and Remote Sensing,96,28–37. Holgado-Barco, A., González-Aguilera, D., Arias-Sanchez, P., & Martinez-Sanchez, J. (2015). Semiautomatic extraction of road horizontal alignment from a mobile lidar system. Computer-Aided Civil and Infrastructure Engineering,30(3), 217–228. Jung, J., Che, E., Olsen, M. J., & Parrish, C. (2019). Efficient and robust lane marking extraction from mobile lidar point clouds. ISPRS Journal of Photogrammetry and Remote Sensing,147, 1–18. Kumar, S. (2018). Causes of replacing geographical traditional mapping into new world digital mapping through geospatial techniques. International Journal of Engineering Research in Computer Science and Engineering,5(1), 26–28. Le Cun, Y., Jackel, L., Boser, B., Denker, J., Graf, H., Guyon, I., Henderson, D., Howard, R., & Hubbard, W. (1989). Handwritten digit recognition: Applications of neural network chips and automatic learning. IEEE Communications Magazine,27(11), 41–46. Li, L., Zhang, D., Ying, S., & Li, Y. (2016). Recognition and reconstruction of zebra crossings on roads from mobile laser scanning data. ISPRS International Journal of Geo-Information,5,125. Li, Q., Li, X., Huang, Z., Halkias, J., McHale, G., & James, R. (2022). Simulation of mixed traffic with cooperative lane changes. Computer-Aided Civil and Infrastructure Engineering,37(15), 1978– 1996. Lima, J. P., & Machado, M. H. (2019). Walking accessibility for individuals with reduced mobility: A Brazilian case study. Case Studies on Transport Policy,7(2), 269–279. López Pazos, G., Balado Frías, J., Díaz Vilariño, L., Arias Sánchez, P., & Scaioni, M. (2017). Pedestrian pathfinding in urban environments: Preliminary results. In ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Geospace 2017, Kyiv, Ukraine, December 4–6. 2017 (Vol. IV-5/W1). Mascetti, S., Civitarese, G., El Malak, O., & Bettini, C. (2020). Smartwheels: Detecting urban features for wheelchair users’ navigation. Pervasive and Mobile Computing,62, 101115. McManamon, P. F. (2019). LiDAR technologies and systems (1 ed.). SPIE. Meagher, D. (1980). Octree encoding: A new technique for the representation, manipulation and display of arbitrary 3-D objects by computer (Technical report). Image Processing Laboratory, Rensselaer Polytechnic Institute, Troy, NY. Ministerio de Fomento & Dirección General de Arquitectura (2010). Vivienda y Suelo, Gobierno de España. Documento Básico SUA. Ministerio de Fomento & Dirección General de Carreteras (2016). Gobierno de España. Norma 3.1-IC Trazado, de la instrucción de Carreteras. Moreno, C., Allam, Z., Chabaud, D., Gall, C., & Pratlong, F. (2021). Introducing the “15-minute city”: Sustainability, resilience and place identity in future post-pandemic cities. Smart Cities,4(1), 93–111. Munoz, D., Bagnell, J. A., Vandapel, N., & Hebert, M. (2009). Contextual classification with functional max-margin Markov networks. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (pp. 975–982). IEEE. Olszewski, P. (2007). Walking as a mode of transport—a planning and policy perspective. Oficyna Wydawnicza Politechniki Warszawskiej, Warsaw Poland. Optech-Incorporated (2021). Lynx mobile mapper web.https:// pdf.directindustry.com/pdf/optech/lynx-mobile-mapper/25132- 387481.html 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 22 ESMORÍS et al. Park, H. S., Lee, H. M., Adeli, H., & Lee, I. (2007). A new approach for health monitoring of structures: Terrestrial laser scanning. Computer-Aided Civil and Infrastructure Engineering,22(1), 19–30. Park, S., Park, H., Adeli, H., & Kim, J. (2015). 3D displacement measurement model for health monitoring of structures using a motion capture system. Industrial Metrology,59, 352–362. Pauly, M., Gross, M., & Kobbelt, L. (2002). Efficient simplification of point-sampled surfaces. In IEEE Visualization, 2002 (VIS 2002). (pp. 163–170). IEEE. Pereda, P. L., & Móuriz, E. N. (2000). Manual de vatos y pasos peatonales (1 ed.). Escuela Libre Editorial. Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. CoRR, https://arxiv.org/abs/1706.02413 Ren, S., He, K., Girshick, R. B., & Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. CoRR, https://arxiv.org/abs/1506.01497 Riveiro, B., González-Jorge, H., Martínez-Sánchez, J., Díaz-Vilariño, L., & Arias, P. (2015). Automatic detection of zebra crossings from mobile LiDAR data. Optics and Laser Technology,70,63–70. Riveiro, B., Lourenço, P. B., Oliveira, D. V., González-Jorge, H., & Arias, P. (2016). Automatic morphologic analysis of quasiperiodic masonry walls from LiDAR. Computer-Aided Civil and Infrastructure Engineering,31. Rossetti, S., Tiboni, M., Vetturi, D., & Calderòn, E. J. (2015). Pedestrian mobility and accessibility planning: some remarks towards the implementation of travel time maps. CSE Journal,1(1), 67–78. Rossetti, S., Tiboni, M., Vetturi, D., Zazzi, M., & Caselli, B. (2020). Measuring pedestrian accessibility to public transport in urban areas: A GIS-based discretisation approach. European Transport/Trasporti Europei,76,2. Servicio de Estadística. Observatorio Nacional de Seguridad Vial. (2018). Anuario estadístico de accidentes 2018 (Technical report). Dirección General de Tráfico. Smith, A., & Sarlo, R. (2021). Automated extraction of structural beam lines and connections from point clouds of steel buildings. Computer-Aided Civil and Infrastructure Engineering,37,110–125. Verstraete, J., & Tampère, C. M. J. (2022). Consistent iterative algorithm for stochastic dynamic traffic assignment with a stable route set. Computer-Aided Civil and Infrastructure Engineering,37(10), 1223–1242. Wang,C.,Ji,M.,Wang,J.,Wen,W.,Li,T.,&Sun,Y.(2019).An improved DBSCAN method for LiDAR data segmentation with automatic eps estimation. Sensors,19(1), 172. Wen, C., Sun, X., Li, J., Wang, C., Guo, Y., & Habib, A. (2019). A deep learning framework for road marking extraction, classification and completion from mobile laser scanning point clouds. ISPRS Journal of Photogrammetry and Remote Sensing,147,178–192. Yan, L., Liu, H., Tan, J., Li, Z., Xie, H., & Chen, C. (2016). Scan line based road marking extraction from mobile LiDAR point clouds. Sensors,16(6), 903. Yang,R.,Li,Q.,Tan,J.,Li,S.,&Chen,X.(2020).Accurateroadmarking detection from noisy point clouds acquired by low-cost mobile lidar systems. ISPRS International Journal of Geo-Information,9, 10. Yu, Y., Li, J., Guan, H., Jia, F., & Wang, C. (2015). Learning hierarchical features for automated extraction of road markings from 3-D mobile LiDAR point clouds. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,8(2), 709–726. How to cite this article: Esmorís, A. M., Vilariño, D. L., Arango, D. F., Varela-García, F.-A., Cabaleiro, José~C., & Rivera, F. F. (2023). Characterizing zebra crossing zones using LiDAR data. Computer-Aided Civil and Infrastructure Engineering,1–22. https://doi.org/10.1111/mice.12968 14678667, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/mice.12968 by Universidade de Santiago de Compostela, Wiley Online Library on [20/06/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License