Quantification of AWS data quality and timeliness

Quantification of AWS data quality and timeliness Deliverable 9, European Space Agency Project -Performance Evaluation of Arctic Weather Satellite Data (No. 4000136511/21/NL/IA) Per Dahlgren1, Peter McEvoy2, Patrick Eriksson2, Adam Dybbroe3, Stephanie Guedj1, Trygve Aspenes1, Panu Lahtinen4 1 MET Norway 2Chalmers University 3 SMHI 4 FMI Introduction The Arctic Weather Satellite was successfully launched 16 August 2024 and a longer LEOP (Launch and Early Operation Phase) phase started with initial testing and adjustment of orbit, during which the project team had no access to data. Direct Broadcast was not turned on until spring 2025. However, in December 2024 a data stream was set up in which global AWS acquired at Svalbard and processed at Tromsø was transferred to Eumetsat and made available to early evaluators. We consequently set up a retrieval and processing system in which data from the Eumetsat data store was regularly retrieved in L1B format and processed locally to obtain L1C netCDF files using the ESA level-1c processor [1]. In this report we will present data from near real time runs with the HARMONIE AROME Data Assimilation (H-A DA) system. Additionally, to complement NWP-related analyses, we perform a statistical comparison between early AWS observations and radiative transfer simulations to assess channel behaviour across atmospheric conditions, including the novel 325 GHz band channels. Direct Data Broadcast reception and processing in the Nordic AWS Ground Segment The NWP data evaluation presented herein is done entirely with the global data acquired from EUMETSAT. However, in preparation for the operational use of AWS data in the Nordic regional NWP models and as part of this project a Nordic ground reception and processing segment has been set up (see [2]). For regional short range forecasting and nowcasting timeliness is crucial. Therefore requirements for the Nordic AWS ground segment have a goal timeliness of better than 6 minutes and threshold requirement of 15 minutes. In Figure 1 the end-to-end timeliness of the Greenland data is displayed over July and first part of August 2025. We see that the timeliness is stable over time and in general lower than 2 minutes calculated from the observation time of the last scanline observed. The reception in Kangerlussuaq is configured for a 3 minute segmentation, so this means that on average no data are older than 5 minutes. Thus the goal timeliness is reached for the Greenland data. The two other stations are configured to have no pass segmentation, so processing starts only after the loss of signal (LOS) when the satellite descends under the antenna horizon. But the processing is equally quick there as at SMHI, thus the timeliness for Oslo and Sodankylä meet the threshold requirement, as seen from Figure XXX and YYY in annex 1. Figure 1: Data granule lengths and level-1c data latency for the data received at Kangerlussuaq, Greenland, relayed to DMI-Copenhagen, and further to SMHI/Norrköping and processed at SMHI. Here for the period of July and first part of August 2025. The latency is calculated as the time difference between the observation time and the time the level-1c netCDF file is available on disk. The NWP system We use a branch of the H-A NWP system based on version cy43. In this configuration, AWS data is used in L1C NetCDF format. The L1C files used were produced from L1B using the ESA processor [1] in which footprints from feedhorns 1,2 and 4 were mapped to the grid of feedhorn 3. I.e., data is mapped to the grid of the 183GHz channels. The first thing that happens in H-A DA is that the NetCDF file is read by a tool called BATOR which reads the input file and writes the contents into Observation Data Base (ODB) format. ODB is the format which the DA system uses. Then the Screening is done which is where model equivalents are computed and quality control of observations is performed. In terms of AWS, model equivalents means calculating the AWS brightness temperatures from NWP model profiles via RTTOV. In these experiments, RTTOV version 10 is used and the assimilation is done in clear sky mode. After Screening comes Minim in which the actual analysis is calculated by minimizing a cost function. Experiment configuration and domains used We use a 3 dimensional variational data assimilation (3D-Var) technique to perform the analysis and an analysis is done every third hour starting from 00UTC. The 3 hour forecast from each analysis is used as first guess for the next DA cycle. The horizontal grid is at 2.5 km resolution and there are 65 levels in the vertical and the model top is at 10hPa. We run over two separate domains; Arome Arctic (AA) which is operational at MET Norway, and METCOOP which is operational at both MET Norway and SMHI, see Figure 2. It should be clarified that we do not run the operational NWP model(s) configuration, we are using the H-A configuration developed in the ESA-AWS project. ECMWF operational forecasts are used as lateral boundary conditions in the same way as our operational NWP production. The experiments are set up so that they can warm start using data from the routine operational NWP production. Warm start means that the first guess, in our case a 3 hour forecast, is taken from the operational runs. It also means that we take the variational bias correction coefficients from the operational suite(s). The benefit of warm starting from operational data is that it gives flexibility. At whatever date AWS is available, we can start assimilation experiments on that date directly, assimilating not only AWS but other heritage sensors used in operations as well e.g. AMSU-A, MHS and ATMS. Figure 2. The two domains used in these NWP experiments. Black is the Arome Arctic domain and red is the METCOOP domain. AWS Assimilation - quality control In these experiments, AWS data is assimilated in passive mode. This means that AWS will undergo quality control (Screening) and enter the minimization (Minim) but the data will be prevented from influencing the analysis. Data is set to passive in the blacklisting, which is part of the quality control procedure. Screening also does a first guess check in which the observation minus the model equivalent, (O-B), is checked to see if it exceeds a preset threshold. If the deviation is too large, the observation is rejected. Radiances also undergo what is called cloud clearing. For microwave radiances this means checking if the observation has been influenced by scattering from rain or very dense clouds. When setting up the H-A DA system to assimilate AWS radiances we used the same type of cloud clearing as MHS, AMSU-A and ATMS, which looks at (O-B) for window channel(s). For AWS channels 2 (AWS-12 at 52.8GHz) and 10 (AWS-31 at 165.5 GHz) are used. If (O-B) exceeds 5K for any of these two channels; we reject all channels from that FOV. This is a first crude setup that can be improved and made more sophisticated. For an early evaluation of the quality of AWS radiances we think this works well as we will get rid of most cloud affected radiances. Further refinement of cloud clearing will make more sense when the actual impact of active assimilation of AWS is being studied. The technical implementation of passive mode assimilation poses some problems for the first guess check. When an observation is flagged as passive, the observation error becomes automatically inflated. This, in turn, has the effect that the preset threshold in the first guess check becomes larger which means that it will be difficult to exceed the threshold. In these tests presented here it does not pose a major problem. The way cloud clearing is done will anyway remove most of the larger departures. When we go on to active assimilation of AWS, this inflation of the observation error will not occur. AWS NWP based quality assessment with a stringent data selection over the METCOOP domain Here we will present long time-series of O-B statistics where the data selected has passed the quality control explained above. Also, we will only select data that is near nadir, namely +/- 10 degrees scan angle. Since comparison to other heritage sensors (ATMS, AMSU-A, MHS) will also be done, we also want to select data over a sub domain to ensure all sensors have been seeing the same air mass with the same surface characteristics. To facilitate this, statistics of near nadir satellite overpasses were computed for AWS, NOAA-20 and METOP-C. The statistics have been calculated by defining a grid-mesh over the NWP domain and then count how many data points there are in each grid cell and the results presented as color coded observations density maps. The period is from the start of near real time data, i.e. 2024-12-22 to 2025-02-20 and the results can be seen in Figure 3. It can be seen that AWS (Figure 3a) has a lot of overpasses in the north part of the METCCOP domain while NOAA-20 ATMS (Figure 3b) and METOP-C MHS (Figure 3c) have a lot of overpasses in the north west part of the domain. By selecting the sub-area marked in sub-figures (a) and (b) we ensure a large enough sample roughly over the same geographical domain. In order to obtain more stringency when comparing different sensors, the sub area should ideally be smaller and there should also be a time tolerance threshold set to ensure the different sensors have measured almost at the same time. Such a data selection would make it difficult for us to get a large enough statistical sample within our NWP domain and the subarea we have used can be seen as a compromise. Furthermore, different sensors have different size footprints so the comparison between sensors will not be completely fair from that perspective either. With that being said, from an NWP user perspective it is still a useful exercise to compare O-B statistics from a new sensor with heritage sensors. Figure 3. Near nadir observation density maps for the period 2024-12-22 to 2025-02-20. (a) is AWS, (b) is NOAA-20 ATMS and ( c) is METOP-C MHS. Note that the legends do not have the same range. In (a) and (b), the sub area selected for statistics has been marked with a black line in the north west corner of the maps. In Figure 4, time-series of STDEV(O-B) are shown for AWS channels 4-8 (AWS-14 to AWS-18) and 11-15 (AWS-32 to AWS-36). Channels 4-8 are temperature sounding channels in the 50GHz oxygen absorption band and 11-15 are humidity sounding channels in the 183GHz water vapor absorption band. AWS is shown in blue and the red line is the corresponding NOAA-20 ATMS channel that has a similar weighting function. What is shown are daily averages, i.e. we compute the STDEV of all O-B departures from all data assimilation cycles in each day. In our case that means we accumulate data from the 00, 03, 06, 09, 12, 15, 18 and 21 DA cycles. With regards to the discussion above about stringency, that means our time tolerance is 24 hours. So when comparing AWS to ATMS, each data point in Figure 4 is data inside the sub-area shown in Figure 3a and b during a 24 hour period. The first thing to point out is that the AWS channels performance is stable in time. Data gaps in the blue line (AWS) are data outages because ESA was still working with the satellite. AWS channel 8 (AWS-18) is out of specification which is a known problem even before launch and a solution exists which will be implemented for EPS-Sterna. AWS channels 4-7 have a higher noise level (STDEV) than the corresponding ATMS channels. This is expected as the integration time is shorter for AWS compared to ATMS. This gives a higher level of noise in the 50 GHz AWS channels compared to heritage sensors. However, the AWS temperature channels exhibit a considerable amount of oversampling (large overlap between consecutive pixels in a scan and between scan lines) similar to ATMS that should be utilised to lower the noise. So just as what is normally done with ATMS data (3x3 averaging is applied)averaging, or super-obbing, is also the solution for reducing STDEV for AWS which will be shown later. AWS channels 11-15 have comparable STDEV to ATMS. Figure 4. Time series of STDEV(O-B) for AWS (blue) and NOAA-20 ATMS (red) over the METCOOP domain. Data samples come from the sub area marked in Figure 3a and b. Now the same AWS statistics as in Figure 4 will be shown but in comparison to AMSU-A and MHS on METOP-C, Figure 5. The observation density for METOP-C AMSU-A/MHS show that it has a high density of observations in marked sub-region used to collect data for these plots, Figure 3c. WIth regards to Figure 5, METOP-C AMSU-A ch 8 has a problem with increased noise and the noise level of AWS ch 8 (AWS-18) is out of specification. It is therefore only meaningful to use the top three plots in Figure 5 for comparison between AMSU-A and AWS. For AWS ch 4 (AWS-14), the STDEV seems comparable to AMSU-A ch 5, while AWS ch 5 and 6 have higher STDEV than the corresponding AMSU-A channels. For the 183 GHz humidity sounding channels AWS has comparable STDEV to METOP-C MHS. Figure 5. Time series of STDEV(O-B) for AWS (blue) and METOP-C AMSU-A and MHS (red) over the METCOOP domain. Data samples come from the sub area marked in Figure 3a and b. METOP-C AMSU-A ch 8 has a problem with increased noise and is blacklisted and AWS ch 8 (AWS-18) noise level is out of specification. AWS NWP based quality assessment over the Arome Arctic domain Here we will present time-series of STDEV of O-B departures for the same period as in Figures 4 and 5, i.e. 2025-01-01 to 2025-04-10 but with data from the Arome Arctic domain. The statistics will be based on data over open sea that has passed quality control and also from the full swath, i.e. all scan angles. We can again see that the AWS performance is stable over time. On 2025-03-20 a 3x3 averaging was introduced for AWS ch 1-8 (AWS-11 to AWS-18). This can be seen in the blue lines in Figure 6 as a sharp drop in STDEV. For AWS ch 4-6 the STDEV of AWS decreases to be just slightly larger than those of AMSU-A. The STDEV of the 183 GHz AWS sounding channels (ch 11-15, AWS-32 to AWS-36) are comparable to those of MHS. Figure 5 shows data from the METCOOP domain with a stringent data selection described earlier and Figure 6 shows data from the Arome Arctic domain and different, less stringent, data selection. Still, the results are overall very similar. Figure 6. Time series of STDEV(O-B) for AWS (blue) and METOP-C AMSU-A and MHS (red) over the Arome Arctic domain. On 2025-03-20 a 3x3 averaging was introduced for AWS 50 GHz sounding channels, i.e. ch 1-8 (AWS-11 to AWS-18). METOP-C AMSU-A ch 8 has a problem with increased noise and is blacklisted and AWS ch 8 (AWS-18) noise level is out of specification. Quality assessment with radiative transfer simulations In this section, actual AWS observations are compared with radiative transfer simulations for channels AWS-21 to AWS-44, providing a statistical view of the instrument's radiance measurements and how they compare with theoretical expectations, based on simulations. The simulated observations are produced in a simulation framework based on the one described in May et al. (2024) [3]. In this framework, the AWS instrument is modelled with channels described by measured Spectral Response Functions, that describe each channel's sensitivity across the frequency range. Scenes used for simulations are populated with surface and atmospheric data from ERA5 and other sources, and clouds in the scenes are populated using cloud radar data from the now-decommissioned CloudSat satellite. The high-quality simulations used for this comparison span the year 2015 and represent a substantial computational effort to generate sufficient statistical data for meaningful instrument assessment. Ideally, simulations concurrent with 2025 AWS observations would be used for this comparison. However, this is not feasible because, even though the CloudSat’s successor EarthCARE was recently launched, it is not yet operational. Despite this temporal gap, the statistical comparison remains valid, since the fundamental radiative transfer physics and instrument response characteristics are unchanged. Methodology for Comparison The observed AWS radiances in this comparison have bias corrections applied. For each channel, a constant offset is added according to O-B analyses performed by ECMWF. As mentioned in previous sections, the AWS-3X channels show a linear and constant slope dependent on the mirror angle through the scan; a compensating bias correction for this scan dependence is applied. Finally, ECMWF's O-B analyses indicate variation in bias dependent on latitude for channel AWS-35. Therefore, a compensating cosine bias is also applied. When comparing with simulations, we differentiate between brightness temperatures (TB), pure pencil beam radiances for a single frequency, and antenna temperatures (TA), which take the instrument channel’s broader spectral and spatial sensitivity into account. In order to focus on atmospheric performance and avoid introducing surface modelling effects, only oceanic scenes are considered in these comparisons. It should be noted that while the simulations are produced with a method that has been proven on previous microwave sounders, the simulations should not be seen as a ground truth. Especially when considering that AWS is the first operational weather satellite with 325 GHz channels. Nevertheless, comparisons with radiative transfer simulations provide a useful quantitative comparison with what we can expect from the instrument based on current theory. Distribution of channel antenna temperatures In Figure 14 the distribution of antenna temperature values (TA) are shown per channel for actual and simulated AWS observations. The distributions for all channels show excellent agreement in shape and position, meaning that the instrument channels are observing the expected range and occurrence frequency of radiance values. The immediate takeaway is that the instrument is behaving well in orbit and delivers on what can be expected based on instrument specifications, even with complications such as the scan dependent changes mentioned in previous sections. Looking more closely at AWS-21 (a window channel, sensitive to the surface), there are observations down to 150 K and observations above 300 K, while there are no such values for simulations. These are very low and high values, respectively, to see in oceanic scenes. Thus, this difference is attributed to some snow/ice or land cases slipping through when filtering for oceanic scenes. Figure 14. Distribution of Antenna Temperature values (TA) from instrument observations from 2025 and simulated instrument observations based on data from 2015. Simulations and observations are filtered to only include oceanic scenes. The panels are paired and ordered according to the altitude of peak humidity sensitivity, going from lowest peaking on the left to the highest on the right. Note also that, except for AWS-21, the distributions agree down to 100 K, showing that the instrument’s good performance holds even for cases with these lower values. This complements the O-B analyses above, which focus on warmer clear-sky cases. Channel-pair distributions of antenna temperatures A more detailed view of the instrument's behaviour for different atmospheric scenarios is obtained by plotting the distribution of paired channel values from the observations and simulations. Figure 15 shows 2D histograms for actual and simulated observations for AWS-33 and AWS-44. Since this pair matches in altitude for peak sensitivity of humidity, the joint distribution shows interesting structures. Specifically, the majority of the observations have the same TA values for AWS-33 and AWS44 and lie in the 300 to 250 K range. These correspond to clear-sky cases, with varying amounts of humidity, where both channels are expected to measure the same TA. In contrast, cases with clouds show up as a curved arm, where the 325 GHz channel values are lower than the 183 GHz channel values, a consequence of 325 GHz channels being more sensitive to the effects of hydrometeors than the 183 GHz channels. Comparing the observations with simulations, clear agreement in the overall shape and range of values can be seen. This indicates that the instrument gives expected values, even for this joint view of the channels. Looking closer, however, we see differences. First, the arm of values along the 1:1 line in observations extends down to 200K, while the arm in simulations only goes down to 225K. Second, there are missing values just below the 1:1 line in the simulations. The former is likely due to cold surface (such as sea-ice) cases incorrectly being classified as pure oceanic cases, thus slipping though the filtering. The latter is likely due to the simulations not having sufficient particle models and related assumptions that would produce the 325 GHz radiances that are slightly below those at 183 GHz. This joint view with 183 GHz and the novel 325 GHz observations shows expected performance across the full range of atmospheric conditions. However, the differences highlight a gap in current hydrometeor models, representing opportunities to improve atmospheric particle scattering models. While acknowledging these modelling gaps, the comparison with simulations provides an encouraging assessment of the instrument's capabilities for sensing hydrometeors and related meteorological applications. Figure 15. 2D histogram of joint AWS-33 and AWS-44 channel values from actual observations and simulations. The two channels have a matching altitude for peak sensitivity to humidity. Annex 1 - Timeliness statistics for the Sodankylä and Oslo antennas Both the Sodankylä and Oslo antenna stations are configured to acquire AWS passes in full local passes, without any segmentation as is the case for the Kangerlussuaq station. This means L0 and L1b/L1c processing is triggered only after AWS LOS, and processing is done on one full pass. From figure A1 we see that L1b data for full passes from Sodankylä are ready on disk mostly within 3 minutes after LOS. The L1c processing is ready within seconds, so the statistics shown for L1b are applicable also for L1c. At Oslo the processing is a bit faster and the local AWS passes are ready in L1c format usually within one minute after LOS, see figure A2. With a typical local AWS pass length around 12-14 minutes the average observation timeliness for both stations are thus well below the threshold requirement of 15 minutes, and for Oslo no AWS data are on average older than 15 minutes. Figure A1: Timeliness of the AWS reception and L1b processing at Sodankylä, Finland. Statistics over three months from June 1st till September 1st, 2025. The latency or timeliness is calculated as the time difference between the observation time of the last scanline received and the time when the L1b file is available on disk. Figure A2: Timeliness of the AWS reception, acquisition and L1c processing at Oslo, Norway. Statistics over three months from June 1st till September 1st, 2025. The latency is derived by comparing the time of the last scan line observed and the time when the L1c file is written to disk. From figure A1 and A2 we also see that Oslo is currently receiving only very few AWS passes per week, whereas AWS is received much more extensively at Sodankylä. This is because FMI are currently able to schedule AWS on two L-band capable antennas in Sodankylä, and at Oslo AWS is until further put lowest on the reception priorities. The optimised hierarchical reception scheduling, as described in [2], is not currently implemented in the Nordic Ground Segment. The reception schedules from Kangerlussuaq are made available daily to the two other antennas, and Sodankylä is also sharing their reception schedule plans for the Oslo station. But it has been decided to wait with the schedule optimisation till the time when the Arome Arctic and MetCoOp NWP regional models are ready to ingest local AWS data operationally. References [1] Rydberg, B. (2025). Remapping of AWS data. Zenodo. https://doi.org/10.5281/zenodo.15049254 [2] Dybbroe, A. (2024). Performance evaluation of Arctic Weather Satellite data - Ground Segment Development Plan. Zenodo. https://doi.org/10.5281/zenodo.10782614 [3] May, E., Rydberg, B., Kaur, I., Mattioli, V., Hallborn, H., and Eriksson, P. (2024). The Ice Cloud Imager: retrieval of frozen water column properties, Atmos. Meas. Tech., 17, 5957–5987, https://doi.org/10.5194/amt-17-5957-2024