A methodology for semi-automatic documentation of archaeological elements using RPAS imagery

2019 IMEKO TC-4 International Conference on Metrology for Archaeology and Cultural Heritage Florence, Italy, December 4-6, 2019 A methodology for semi-automatic documentation of archaeological elements using RPAS imagery E. Angelats1, M. E. Parés1, C. Mas-Florit2, M.A Cau-Ontiveros2,3 1 Geomatics division, Centre Tecnològic de Telecomunicacions de Catalunya (CTTC/CERCA), Av. Carl Friedrich Gauss, 7. Building B4, 08860, Castelldefels, Spain, eduar[email protected], eulalia.par[email protected] 2 ERAAUB, Departament d’Història i Arqueologia, Universitat de Barcelona, c/ Montalegre 6-8, 08001 Barcelona, Spain, [email protected], [email protected] 3ICREA, Pg. Lluís Companys 23, 08010 Barcelona, Spain Abstract – One of the main tasks of archaeologists is to document (map) their sites at a level where stones may be clearly distinguished and to provide centimetrelevel reliable measurements of man-built structures. One of the goals of the project ARCHREMOTELANDS is to develop a methodology to generate those maps semi-automatically combining RPAS imagery, photogrammetry and machine learning techniques. In this context, this paper presents a conceptual framework to determine the minimum number of pixels, and so the required Ground Sampling distance and altitude needed to fly a RPAS allowing very detailed documentation and digitalization of archaeological sites by processing the acquired images. These images are used firstly to generate a high resolution orthophoto. Then, archaeological elements of interest are documented almost automatically using 2D unsupervised clustering techniques. The approach has been experimentally tested and validated in the early Christian complex of Son Peretó (Manacor, Mallorca, Spain). I. INTRODUCTION Documentation is an important task in archaeology and cultural heritage domains. It allows to reflect what happens in an archaeological site both from temporal, spatial and historical perspectives. In this context, geomatics techniques and technologies such as GNSS, total stations, photogrammetry, and laser scanning have become very useful tools for obtaining precise georeferenced data but also for generating 2D, 3D and 4D documentation and the digitalization of sites. These products have contributed significantly to the management of sites providing multiple levels of detail and resolutions. In addition, these may complement and help the analysis, connection and interpretation of materials collected during archaeological campaigns [1],[2]). Moreover, 3D and 4D data may help to valorise and spread the information to an extended audience[3]. Latest advances in Remotely Piloted Aircraft System (RPAS) technology, Commercial-Of-The-Self (COTS) miniaturized cameras, and photogrammetric software have opened the access to this technology beyond research and experimental projects to the archaeological community [4,5,6]. Recent works have shown the potential and usefulness of the outputs of these technologies (orthophotos, point clouds and Digital Surface Models (DSM)) for the management of archaeological sites, both for documenting and 3D modelling perspectives [4,7,8]. Thanks to the capability of flying at low altitudes, dense point clouds and orthophotos with a ground sampling distance (GSD) of a few centimetres can be generated, increasing the quality provided by satellite (meter level) and aerial imagery (decimetre level) [9]. This increased resolution may allow to digitalize an archaeological site to a stone level [4,9] or to provide high detail of the structures. These outputs become a suitable alternative to total stations by relaxing requirements in terms of metric accuracy. Moreover, these technologies may also help to detect potential archaeological buried remains when multispectral orthophotos are generated. The advantage in this case is the capability to acquire data in the optimal time window for detecting soil or cropmarks [10]. Techniques to allow the automatic processing of point clouds and 2D images have made significant progress in recent years. These techniques can be used for segmenting point clouds/images in a set of objects, to 7 classify elements of interests or both of them. In the context of cultural heritage, some works can be highlighted. Riveiro et al. [11] developed an algorithm to automatically segment masonry blocks from 3D point clouds based on a marked-controlled watershed using 2.5D derived images. Grilli et al. [3] proposed a methodology to classify a 3D dataset of several heritage scenarios based on a 2D classification on a first step and back-projecting in a second step. Poux et al. [12] proposed a methodology to segment and classify directly from a point cloud, each single quasi-planar object such as the ones that are normally mapped in an archaeological plan. The previous works on the generation of orthophotos and 3D models derived from RPAS together with the advances on automation reveal the potential to document every single stone or archaeological element of interest using these techniques. This would give the archaeological community the capability to generate almost automatically near real time documentation of archaeological sites as well as monitoring with high level of detail their temporal evolution. In this context, the following question arise: which is the required ground resolution that allows to visually identify every stone or archaeological element from RPAS imagery but also to segment them almost automatically? To answer this question, this study builds a conceptual framework to determine the minimum number of pixels, and so the required GSD and altitude needed to fly a RPAS allowing very detailed documentation and digitalization of archaeological sites using the images acquired by the aforementioned RPAS. These images will be used to generate a high resolution orthophoto permitting to document semi-automatically every archaeological element using 2D unsupervised clustering techniques from this orthophoto. Preliminary results based on the proposed approach, have been obtained from the early Christian complex of Son Peretó (Manacor, Mallorca, Spain). II. PROPOSED APPROACH The semi-automatically detection and classification of individual stones, as archaeological elements of interest, in RPAS collected images will be based on a set of machine learning and image processing techniques. The research presented on this paper relies on the use of unsupervised clustering techniques (k-means). A review of these techniques, unsupervised and supervised, can be found in Grilli et al. [3] but a brief definition of cluster is introduced hereafter. A cluster is a set of points sharing a common feature. In this case, the aim is that the algorithm groups all the pixels representing the same stone as part of the same cluster. Thus, we will focus on grouping pixels that share very similar RGB values. The algorithm relies on the assumption that within every stone the RGB values are more or less the same while areas between stones (mortar area, sand etc..) have different RGB values than stones. An example is shown in Figure 1 where pixels representing the same stone share similar RGB values. Fig. 1. Image of a stone wall. In order to be able to group pixels belonging to the same stone, there should be a minimum number of them sharing the same feature. In the frame of this research, we have considered than 5 is the minimum number of pixels needed for the algorithm to classify elements. This value was selected heuristically after a preliminary testing using several images with different GSD, and so covering stones with different number of pixels, and varying the number of clusters and the minimum number of pixels per clusters. Taking this into account, the workflow proposed has been developed as follows. Firstly, the user has to decide which elements of an archaeological site must be digitalized. To do so, an estimation of the mean area and perimeter of the stones to be digitalized have to be computed. Note that not all the stones need to be measured but only a significant subset of them. Then, the estimation of the required GSD has to be done. This estimation should be based on the fact that the mean area of a stone should be covered by at least 5 pixels. Taken the GSD and the RPAS camera characteristics (focal length and sensor size), estimate the required flying height. Then, prepare the flight plan, execute the flight and acquire the images. Next step is to generate the orthophoto following the standard structure from motion approach involving the use of GCP previously measured; the use of imagery acquired with a RPAS with on-board GNSS RTK receiver or the use of alternative approaches as the one suggested by [13]. After that, an unsupervised classification k-means algorithm using a predefined number k of clusters should be applied. Finally, a manual selection and merging 8 clusters belonging to archaeological elements should be done. The last step is to extract the desired values and convert them from raster to vector layer. III. CASE STUDY: THE EARLY CHRISTIAN COMPLEX OF SON PERETÓ. The basilica of Son Peretó (Manacor) is located in the eastern part of Mallorca. The site was discovered at the beginning of the 20th century. The excavations uncovered a church with three naves, a baptistery with two baptismal fonts as well as a large cemetery and a series of rooms attached to the baptistery [14] [15]. Recent geophysical surveys, undertaken in the accessible area surrounding the remains of the excavated basilica have demonstrated the presence of a different group of buildings and walls connected to the archaeological remains, as well as numerous pits and ditches and the remains of a large villae. Continuous excavations are still being carried out and are expected to be developed in the forthcoming years, making Son Peretó an ideal case study for the current research. Fig. 2. Location and view of early Christian complex of Son Peretó (Source [15]) A. Tools The proposed procedure was validated using the following tools: xa DJI Phantom-3 RPAS with built-in camera. The camera has a focal length of 3.61 mm and a pixel size of 1.19 μm. Although it is a low cost RPAS, GSD lower than 3cm can be obtained with flying heights below 50 m. x12 GCP were used for the orthophoto generation. 6 checkpoints (CHP) were used to assess the quality of the orthophoto generation step. The GCP and CHP coordinates were estimated using triangulation techniques and raw aerial imagery, available from Spanish Plan Nacional de Ortofotografía Aérea (PNOA), following the approach presented in [13]. The raw imagery includes aerial images as well as exterior orientation parameters (position and attitude) for each image and they have a GSD of 21 cm. According to specifications [16], the planimetric accuracy of the PNOA orthophoto should be better than 0.5 m for the planimetric components and 1 m for the height component. Therefore, similar values can be expected for the GCP and CHP coordinates since these coordinates are estimated from the same raw imagery used to generate PNOA orthophoto. xAgisoft Metashape [17] was used as photogrammetric software for the orthophoto generation step. The k-means implementation of Orfeo toolbox library [18] running on QGIS v3.6.1 [19] was used for the clustering step together with some standard GIS tools. B. Data processing Planimetric maps and existing documentation for the archaeological site [14] [15] were used as input for identifying different types and sizes of the stones which are part of this archaeological site. Although we proposed to use the mean area of all stones that should be mapped, for this research, we used as target value, the size of the smallest stone that needs to be mapped. In our case, the smallest area was 5.5 x 5.5cm that translates to a GSD of 1.1 cm. Taking 1.1 cm as targeted GSD and DJI Phantom-3 characteristics, the flying height was set to 25 m. The mission plan consisted of parallel strips with an overlap of 60%. 53 images were acquired. The images were processed with Agisoft Metashape following the methodology proposed by [13] to generate a high resolution orthophoto (Fig. 3). The methodology relies on the use of GCP obtained from PNOA imagery to assure a high resolution map, but also a proper coregistration (Fig. 3) with historical aerial orthography, allowing also a temporal documentation. Nevertheless, it should be pointed out that an 9 alternative approach is possible. The desired results may be also achieved even with low-accuracy GCPs if their use is consistent through the whole sequence of RPAS flights in a “4D” survey. This alternative approach delivers quick, adequate and coherent results between RPAS datasets (coming from different flights) without the need for a more accurate, but time consuming, GNSS ground survey for GCPs. However, this approach may not achieve a proper spatial co-registration with historical datasets in a first step, requiring an additional coregistration step. Fig. 3. Orthophoto from Son Peretó In order to validate the semi-automatic documentation, we focused on a previously documented wall (Fig.1). Firstly, a manual segmentation was done for validation purposes (Fig.4). After that, the k-means clustering algorithm implemented in Orfeo toolbox library running on QGIS v3.4 was used (Fig.5). k-means was configured as follows: number of cluster k = 5; number of iterations = 10. Fig. 4. Ground truth of validation area Fig. 5. Result from the semi-automatic workflow. A final step was performed to manually correct the output of the clustering. This might need to be done because sometimes two stones are too closed to be distinguished and appear as the same one. After a visual inspection, final clusters were decided. C. Results evaluation The analysis of the CHP residuals was the tool to evaluate the relative precision quality of the orthophoto generation step. The residuals of these points show a planimetric error around 1 (aerial) GSD for the three components (0.3, 0.23 and 0.27 m) ensuring that aerial PNOA and RPAS orthophotos are co-registered. The validation of the semi-automatic detection of stones was carried out for a selected area (Fig. 4). All the target stones were detected (Fig. 5). Some of them, mainly small ones, have been clustered together and lately separated. In order to quantify the precision, for each of the stones, their areas have been computed and compared with their real areas (Table 1). Table 1. Results from the semi-automatic workflow. Total number of stones 50 Undetected stones 0 Reference stones that are split in two clusters 5 Area computation precision 71.42% IV. CONCLUSIONS A methodology to segment almost automatically archaeological elements from RPAS high resolution orthophoto has been presented. The approach has been tested and validated with RPAS and aerial imagery of the archaeological site of Son Peretó. The aim of this work was not to assess the potential of RPAS imagery, for monitoring and documenting the site, already known by 10 archaeological excavations or remains, but to assess the feasibility of automation of the documentation process. Further research will include a more complete validation including the entire archaeological site instead of a small part. Next steps might include also the use of supervised classification techniques [20] in a near term and the use of deep learning algorithms [20] in the longterm if larger datasets are available. The supervised classification algorithms such as Support Vector Machines or Random Forest, might use directly the RGB band or a combination of RGB bands and additional band obtained after applying enhanced edge detection techniques such as the ones proposed in [11]. V. ACKNOWLEDGMENTS This contribution is part of the results of the project Archaeology, Remote Sensing, and Archaeometry: A multidisciplinary approach to landscape and ceramics from the Roman to the Medieval period in Mallorca (Balearic Islands) (ARCHREMOTELANDS) (HAR2017-83335-P), PI: Miguel Ángel Cau Ontiveros, funded by the Ministerio de Economía, Industria y Competitividad, with contribution from the European Regional Development Fund from the European Commission. This is also part of the activities of the Equip de Recerca Arqueològica i Arqueomètrica de la Universitat de Barcelona (ERAAUB), Consolidated Group (2017 SGR 1043), thanks to the support of the Secretaria d’Universitats i Recerca del DEC de la Generalitat de Catalunya. The work of Catalina has been possible thanks to a Juan de la Cierva research fellowship from the Spanish Ministerio de Economia, Industria y Competitividad (MINECO), reference IJCI-2017-32853. REFERENCES [1] G. Bitelli., V.A. Girelli, G. Sammarini “4- dimensional recording and visualization of urban archeological excavations”, Appl Geomatics, 2018, 10, pp. 415–26. [2] E.S. Malinverni, C.C. Barbaro, R. Pierdicca, C.A. Bozzi, A.N. Tassetti, “UAV Surveying for a complete mapping and documentation of archaeological findings. The early neolithic site of Portonovo”, ISPRS Archives, 2016-January, pp. 1149-1155 [3] E. Grilli,F. Remondino, “Classification of 3D Digital Heritage”, Remote Sens., 2019, pp. 11, 847. [4] S. Campana, “Drones in Archaeology. State-of-the- art and Future Perspectives”, Archaeol. Prospect., 2017, 24, pp. 275–296. [5] F.Nex, , F. Remondino, “UAV for 3D Mapping Applications: A Review”, Applied Geomatics, 2013, 6, pp. 1-15. [6] R. Opitz, J. Herrmann, “Recent trends and longstanding problems in archaeological remote sensing”, Journal of Computer Applications in Archaeology, 2018, 1(1), pp. 19-41. [7] D.C. Cowley,C. Moriarty, G. Geddes, G.L. Brown, T. Wade, C.J. Nichol, “UAVs in Context: Archaeological Airborne Recording in a National Body of Survey and Record”, Drones, 2018, 2 (2), p. 2. [8] M. Lo Brutto, A. Garraffa, P. Meli, “UAV platforms for cultural heritage survey: first results”, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., 2014, II-5, pp. 227-234. doi: 10.5194/isprsannals-II- 5-227-2014. [9] K.G. Nikolakopoulos, K. Soura, I.K. Koukouvelas, N.G., Argyropoulos, “UAV vs classical aerial photogrammetry for archaeological studies”, Journal of Archaeological Science, 2017, vol. 14, pp. 758- 773. [10] N. Masini, C. Marzo, P. Manzari, A. Belmonte, C. Sabia, R. Lasaponara, “On the characterization of temporal and spatial patterns of archaeological cropmarks”. Journal of Cultural Heritage, 2018, vol. 32, pp. 124-132. [11] B. Riveiro, P.B. Lourenço, D.V. Oliveira, H. González-Jorge, P. Arias, “Automatic Morphologic Analysis of Quasi-Periodic MasonryWalls from LiDAR”, Comput.-Aided Civ. Infrastruct. Eng, 2016, vol. 31, pp. 305–319. [12] F. Poux,R. Neuville, L. Van Wersch, G.A. Nys, R. Billen, “3D Point Clouds in Archaeology: Advances in Acquisition, Processing and Knowledge Integration Applied to Quasi-Planar Objects”, Geosciences, 2017, 7, 96. [13] E. Angelats, M.A. Cau-Ontiveros, C. Mas-Florit, “High resolution orthophotos and a digital surface model of the Roman city of Pollentia (Mallorca, Spain) using RPAS imagery, aerial images, and open data archives”, inProceedings of the 2018 IEEE International Conference on Metrology for Archaeology and Cultural Heritage, 22–24 October 2018, Cassino (Italy) [14] P. de Palol, “La arqueología cristiana en la Hispania romana y visigoda. Descubrimientos recientes y nuevos puntos de vista”, in Actas del XIème Congrés International d´Archéologie Chrétienne (Lyon-Vienne-Grenoble-Aosta, 1986), (Roma, 1989), pp. 1975- 2022. [15] M.A. Cau, M. Riera Rullan,M. Salas, “The early Christian complex of Son Peretó (Mallorca, Balearic Islands): Excavations in the ‘west sector’(2005- 2008)”, Archeologia Medievale, 2012, XXXIX, pp. 231-243. [16] pnoa.ign.es. ‘PNOA imagen, especificaciones técnicas’Available online: 11 http://pnoa.ign.es/caracteristicas-tecnicas. [Accessed: 10- November- 2019]. [17] Agisoft Metashape (Version 1.5.5). (2019). [18] M. Grizonnet, J. Michel, V. Poughon, J. Inglada, M. Savinaud, R. Cresson, “Orfeo ToolBox: Open Source Processing of Remote Sensing Images.” Open Geospatial Data, Software and Standards, 2017 2 (1), p. 15. https://doi.org/10.1186/s40965- 017-0031-6. [19] QGIS SW (Version 3.6.1). (2019) [20] S.E. Jozdani, B.A. Johnson, D. Chen, “Comparing Deep Neural Networks, Ensemble Classifiers, and Support Vector Machine Algorithms for Object- Based Urban Land Use/Land Cover Classification”, Remote Sens., 2019, 11, 1713. 12