Investigating Fractional Vegetation Cover (fcover) from Sentinel-2 as Input for Bare Surface Composites
Karlshoefer, Paul; Kühl, Kevin; Schwind, Peter; Heiden, Uta
- Publisher
- Zenodo
- Language
- en
Abstract
Keywords: Sentinel-2, EnMAP, Fractional Vegetation Cover, Temporal Composites, Bare surface The creation of reliable bare surface composites from Sentinel-2 (S2) data presents a significant challenge due to the variability of surface conditions and vegetation dynamics. This study aims to develop a novel approach for generating such composites by leveraging fractional vegetation cover (fCover) maps derived from a neural network trained on paired S2 and EnMAP observations. This approach is compared to traditional methods that rely on spectral indices alone. Such composites hold significant potential for various downstream applications, including the development of digital soil property models. Bare surface composites from temporally stacked data can be generated through various methodologies. A previous approach (Heiden et al., 2022) relied on rule-based classification using spectral indices to separate multispectral pixels into different surface classes. In contrast, this study proposes a machine learning-driven method leveraging paired Sentinel-2 (S2) and EnMAP observations acquired within close temporal proximity. Fractional vegetation cover (fCover) maps are first derived from EnMAP scenes using the EnMAP fCover processor (developed by Martin Bachmann, David Marshall and Kevin Kühl, DLR). These fCover maps serve as labelled training data for a neural network, which learns to predict fCover directly from Sentinel-2 spectra. The trained network is then applied to the full temporal stack of Sentinel-2 data, providing pixel-wise probabilities of bare surface conditions. Finally, these probabilities are aggregated to create a bare surface composite. The neural network demonstrated convergence, with training and validation losses stabilizing at 0.015 and 0.021 mean squared error (MSE), respectively, after 5 epochs and 24 hours of training. The resulting composites, derived from predicted vegetation fractions, closely resemble those produced using traditional spectral indices. Further analysis will include explaining the network's decision-making process using SHAP and a detailed comparison of the composites generated by both methods. To enhance interpretability, auxiliary data such as the acquisition date of each contributing pixel is tracked. Additionally, uncertainty estimates for each band and pixel in the composites will be provided, offering insight into the reliability of the results. Future work will integrate the generated bare surface composites with additional datasets, such as the soil map of Bavaria provided by the Landesamt für Umwelt (LfU). By combining these resources, we aim to establish correlations between the composite spectra and specific soil properties, like soil colour. This approach has the potential to enhance our understanding of surface-soil relationships and broaden the applicability of the composites for soil property modelling and environmental studies.
Full text
Training and prediction of Fcover Abundances for Sentinel 2 imagery Paul Karlshöfer, Kevin Kühl, Peter Schwind, Uta Heiden DLR Oberpfaffenhofen (DLR), Earth Observation Center (EOC), Germany Non-photosynthetic active vegetation A: Xylan & Cellulose B: Lignin & Cellulose C: Cellulose Bare soil D: Clay E: Carbonates Sentinel-2 (multispectral) EnMAP (hyperspectral) S2 bands modified HybridSNmodel EO-based bare soil reflectance composites (SRC) are important: •for large scale soil parameter analysis •to evaluate the impact of soil erosion processes. Pros and cons of hyperspectral image data •Enables pixel level separation of bare soil (BS) and dry vegetation (NPV) using spectroscopy •Limited coverage and infrequent observations reduce usability for largescale, time-sensitive applications Downstream Task: Soil Reflectance Composite SCMaPtrained SRC PV+IR2 PV+IR2index-basedSRC EnMAP trained SRC s2f 60 Based on min 60% BS abundance / pixel 2,5 km Bare soil reflectance Conclusion •The model makes good prediction, but training bias is problematic, especially in unclear situations •Usage of spectral information meaningful •Spatial features appear very important, even if spectral information is constant •SRC qualitatively convincing and comparable to index-based SRC (mean RSME 0.013 on Tile in Belgium ) •Focus on central Europe for training From 656 S2EnMAP matchups, the most promising 115 are selected (based on footprint overlap, cloudcover, etc) (103 training, 12 validation) •41374077 trainingpatcheswith size 25x25x10; randomly flipping training patches in X/Y to reduce risk of overfitting •Training on 4 A100-SXM4-80GB GPUs and 1TB RAM •Tensorflow2.6, Cuda11.0, numpy1.21 •Training for 4 epochs (8h per epoch) with min validation loss 0.011 Provided by and modified from M. Bachmann ³Schwind, P. et al. (2024). Using Deep Learning To Generate Fractional Vegetation Cover From Multispectral Data. 13th EARSeL Workshop on Imaging Spectroscopy, 2024-04-16 -2024-04-19, Valencia, Spain. Concept: Leverage the growing hyperspectral archive of EnMAPto predict fractional vegetation cover in Sentinel-2 temporal stacks³ Training of a modified HybridSNmodel (see below) with S2-EnMAP temporal pairs (flowchart left) that predicts fcover(percentage of PV/NPV/BS per pixel. Footprints (left) (red) S2 scenes involved in training process (black) EnMAPlabels Distribution of labels (right) Per class (PV, NPV, BS), frequency (histogram) of various abundance levels (10% size of bins) -> training data is not uniformly distributed frequency Binned Fcover per class Distribution of Input Labels per fcover class Result (Validation –not seen during Training) Investigation of model’s decision making Influence of spectral features (via Occlusion) Green Vegetation (PV) Red-edge (Chlorophyl-a) S2-Bands (5 and 6) most important, followed by B11 –however all bands significant Dry Vegetation (NPV) OnlyRed-edge S2-Bands (5 and 6) and Band 11 is important Bare surface / soil (BS) Importance is well distributed across all features Shape of spectra is most important Influence of spatial features (via Patch-Alteration) Examples of Patch Alteration to test model response →done for 1000s of patches Original 25x25x10 patch Baseline / labeled data Streaking Spetraintact, but shape worse Shuffle set of pixels this case, 3 deep from border Lin. Comb. With cloud spectra 20% Method mean( pred –baseline_pred) Comment Flip entire Patch [ 0.002 0.003 -0.005] No impact, since part of training routine Streaking Patch b) [ 0.040 -0.010 -0.029] Very resilient Shuffle Periphery (1) [ 0.003 -0.004 0.000] Very resilient to 1px deep changes at border Shuffle Periphery (3) c) [ 0.665 -0.495 -0.118 ] Very sensitive to 3px deep changes, huge drift to PV Rand. Center (3x3) [ 0.379 -0.314 -0.061 ] Sensitive, but more resilient than c) (fewer pixels affected) Artificial Cloud (d) [ 0.386 -0.276 -0.106 ] Sensitive, lin. combination of (.8 * spectra, .2 * cloud) ba cd Model enforces that sum of fcoveradds up to 1, With increasing uncertainty, model tends towards PV (bias of training data) Height of green bar indicates feature importance NPV BS PV PV PV Investigating Fractional Vegetation Cover (fcover) from Sentinel-2 as Input for Bare Surface Composites Existing product: SoilSuite5y •Mosaic and temporal composite of all “bare surface pixels” found in S2 data •Classification is binary (soil, not soil), -- →fractional cover is more detailed Heiden et al, SoilSuite–Sentinel-2 –Europe, 5 year composite (2018-2022), DOI: 10.15489/qkud8cudg596 PV (green) NPV (blue) and BS (red). Clouds and urban masked. Left: Labels (fcoverderived from EnMAP), right: Prediction (fcoverpredtictedfrom Sentinel-2). NW of Basel Airport True color example of SRC in 31UFS (Belgium) Mean RSME between both SRCs is 133 for reflectance data scaled to [0,10000]
