Impact assessment of AWS in regional NWP and NWC

Impact assessment of AWS in regional NWP and NWC Deliverable 10, European Space Agency Project -Performance Evaluation of Arctic Weather Satellite Data (No. 4000136511/21/NL/IA) Per Dahlgren1, Stephanie Guedj1, Eleanor May2 1 MET Norway 2 Chalmers technical university Introduction The Arctic Weather Satellite was successfully launched 16 August 2024 and a longer LEOP (Launch and Early Operation Phase) phase started with initial testing and adjustment of orbit, during which the project team had no access to data. Direct Broadcast was not turned on until spring 2025. However, in December 2024 a data stream was set up in which global AWS acquired at Svalbard and processed at Tromsø was transferred to Eumetsat and made available to early evaluators. We consequently set up a retrieval and processing system in which data from the Eumetsat data store was regularly retrieved in L1B format and processed locally to obtain L1C netCDF files using the ESA level-1c processor [1]. Then followed near real time runs with the HARMONIE AROME Data Assimilation (H-A DA) system with AWS data assimilated in passive mode. A big effort was put into this phase of the project and included meetings with ESA, EUMETSAT and instrument providers to provide feedback. The main results of the passive mode, near real time, runs are presented in deliverable 9 [2]. In the near real time passive mode runs, a test was performed to reduce noise in the temperature sensing 50 GHz channels. This was achieved by doing a 3x3 averaging of these channels which did indeed reduce the noise to a level comparable to AMSU-A. The averaging was only introduced in one of the domains used: Arome Arctic. When moving on to active assimilation of AWS we therefore only test on that domain. Since such a big effort was put into the passive mode runs for early evaluation and the project was coming to an end, the impact experiments shown here are only a preliminary first test and is work in progress. More comprehensive studies will follow after the end of this project. For example MET Norway has just started a project called Arctic Weather Satellite All-sky Radiance Data Assimilation Implementation (AWARI) that is funded by the Norwegian Space Agency and it focuses on assimilation of AWS data. The NWP system We use a branch of the H-A NWP system based on version cy43. In this configuration, AWS data is used in L1C NetCDF format. The L1C files used were produced from L1B using the ESA processor [1] in which footprints from feedhorns 1,2 and 4 were mapped to the grid of feedhorn 3. I.e., data is mapped to the grid of the 183 GHz channels. The first thing that happens in H-A DA is that the NetCDF file is read by a tool called BATOR which reads the input file and writes the contents into Observation Data Base (ODB) format. In the 1 experiments presented here, the 3x3 averaging of 50GHz channels is also done in the BATOR step. Then the Screening is done which is where model equivalents are computed and quality control of observations is performed. In terms of AWS, model equivalents means calculating the AWS brightness temperatures from NWP model profiles via RTTOV. In these experiments, RTTOV version 10 is used and the assimilation is done in clear sky mode. After Screening comes Minim in which the actual analysis is calculated by minimizing a cost function. Experiment configuration We use a 3 dimensional variational data assimilation (3D-Var) technique to perform the analysis and an analysis is done every third hour starting from 00UTC. The 3 hour forecast from each analysis is used as first guess for the next DA cycle. The horizontal grid is at 2.5 km resolution and there are 65 levels in the vertical and the model top is at 10hPa. We run here one domain; Arome Arctic (AA) which is operational at MET Norway, see Figure 1. It should be clarified that we do not run the operational NWP model configuration, we are using the H-A configuration developed in this project. ECMWF operational forecasts are used as lateral boundary conditions in the same way as in our operational NWP production. The passive mode runs for early evaluation carried on until the end of April 2025. Therefore, to save time, we ran active assimilation of AWS for that month as the reference experiment (no active AWS assimilation) was already in place. Experiment REF means no active AWS is used, and experiment EXP is the same as REF except AWS is actively assimilated. Both experiments use conventional observations (SYNOP, SHIP, DRIBU, TEMP, AIRCRAFT), MW observations from the sensors listed in Table1, IASI data from METOP-B and METOP-C, ASCAT winds from METOP-B and METOP-C. Instrument Satellite Channels used AMSU-A METOP-B 7,8,9 AMSU-A METOP-C 6,7,8,9 AMSU-A NOAA-18 6,7,8 AMSU-A NOAA-19 6,9 MHS METOP-B 3,4,5 MHS METOP-C 3,4,5 MHS NOAA-19 4,5 ATMS S-NPP 6,7,8,9,18,19,20,21,22 ATMS NOAA-20 6,7,8,9,18,19,20,21,22 2 MWHS2 FY-3D 4,5,6,11,12,13,14,15 MWHS2 FY-3E 4,5,6,11,12,13,14,15 AWS (only in EXP) 4,5,6,7,11,12,13,14,15 Table 1. Microwave sensors used in the experiments In this first impact assessment, only 3h forecasts are done and we evaluate the impact of AWS assimilation on 3h forecasts. This will be further explained later. Figure 1. The two domains used in the near real time runs for early evaluation experiments. Red is the Arome Arctic domain and blue is the METCOOP domain. The impact experiments are only done with the Arome Arctic domain. AWS Assimilation - quality control AWS undergoes quality control (Screening) before entering the minimization (Minim). Screening does a first guess check in which the observation minus the model equivalent, (O-B), is checked to see if it exceeds a preset threshold. If the deviation is too large, the observation is rejected. Radiances also undergo what is called cloud clearing. For microwave radiances this means checking if the observation has been influenced by scattering from rain or very dense clouds. When setting up the H-A DA system to assimilate AWS radiances we used the same type of 3 cloud clearing as MHS, AMSU-A and ATMS, which looks at (O-B) for window channel(s). For AWS, channels 2 (AWS-12 at 52.8 GHz) and 10 (AWS-31 at 165.5 GHz) are used. If (O-B) exceeds 0.7K for channel 2 then AWS channels 4,5,6,7,8 are rejected. If (O-B) for channel 10 exceeds 5K then channels 11,12,13,14,15 are rejected. Variational bias correction A regression based scheme is used to correct biases in radiance assimilation. The bias, b, is estimated by multiplying a set of predefined predictors, p, to coefficients, beta. Originally, these coefficients were determined offline by regression between (O-B) statistics and the predictors. In the VARiational Bias Correction (VARBC) framework, the same thing is done inside the minimization which allows for the coefficients to automatically adjust to changes in the biases. The bias, b, is applied by subtracting it from the (O-B). We use for AWS the same predictors as for other heritage MW sensors: p0 constant offset (=1) p1 1000-300hPa thickness p8 nadir viewing angle p9 nadir viewing angle ** 2 p10 nadir viewing angle ** 3 Table 2. List of predictors used for bias-corrections of AWS Since AWS is a new sensor, the runs in passive mode will start with the coefficients set to zero and they will then gradually spin up before stabilising. Once VARBC has been spun up, active assimilation can take place. Predictors p8, p9 and p10 are there to adjust for scan dependent biases and if we take a look at the data availability, per scan position, in each assimilation cycle, Figure 2, we see that some DA cycles only a small part of the scan. E.g. 00 and 06 UTC. If VARBC gets a varied sample of different scan positions it is more likely to be able to adjust for the scan dependent part of the bias. Therefore we will only actively assimilate AWS at 09, 12,15, 18 and 21 UTC. For now we keep 15UTC even if it only sees one part of the scan. 4 Figure 2. Number of observations available in each data assimilation cycle as a function of scan position. The sample period is 2025-02-21 to 2025-04-05. The 03 UTC cycle has no AWS data in the AA domain and is not plotted here During the early evaluation the AWS data was in passive mode and the VARBC coefficients were therefore adjusting. It should be noted that the data presented in the early evaluation part of the project had the effect of VARBC removed, i.e. raw data. For various reasons, the VARBC coefficients for AWS were reset several times in the period 2024-12-22 to 2025-04-01 and the last time that was done was 2025-02-21. Therefore, we present time-series for the spin-up and adjustment of VARBC from that date. Figure 3 shows the time evolution for channel 6 (AWS-16) of each coefficient separately. If we focus on the cycles where AWS is actively used, explained above, we see that the coefficients adjust in about one week and then they readjust again when sidelobe correction is introduced 2025-03-13. After that the coefficients are stable. The same behaviour is true for AWS channel 15 (AWS-36), Figure 4. 5 Figure 3. Time evolution of each VARBC coefficient separately for AWS channel 6 (AWS-16), for each data assimilation cycle. The plot shows the beta coefficients associated with predictors p0, p1, p8, p9, p10 6 Figure 4. Time evolution of each VARBC coefficient separately for AWS channel 15 (AWS-36), for each data assimilation cycle. Impact assessment We here refer to experiment EXP as the one where AWS is actively assimilated and REF as the one where it is not. When assessing if forecasts improve due to a given modification to the H-A NWP system we usually compare the forecasts to a predefined set of SYNOP (for surface) and TEMP (for upper-air) observations. Given the uneven distribution of such data, and the limited number of TEMP stations in the Arome-Arctic domain we instead use ATMS from NOAA-20 as a reference and AMSU-A/MHS from METOP-C. By comparing the (O-B) departures for NOAA-20 ATMS in EXP (AWS assimilated) and REF (no AWS) we are verifying if the 3h forecast has changed by assimilating AWS since B in (O-B) is a 3h forecast. If assimilation of AWS improves 3h forecasts, the (O-B) from NOAA-20 ATMS should be decreased in EXP. If 7 then we assume AWS has improved 3h forecasts. The impact assessment started with a sensitivity test which was 14 days long. Such a short period, with only 3h forecasts, will complete in only 2 days which allows for many tests to be done. This is mainly to see what type of change, e.g. changing the AWS observation errors, that change the impact according to the formula above. During the sensitivity test, the impact signal was more or less the same; positive impact on temperature sensing 50 GHz channels (ATMS 6-10) and negative on humidity sensing 183 GHz channels (ATMS 18-22). The following things were done in these tests: ● Changing AWS observations errors ● Changing thresholds for cloud screening ● Apply 3x3 averaging for the 183 GHz AWS channels ● Assimilating only humidity sensing channels. This led to negative impact on humidity and neutral to slightly negative for temperature The results presented here are for the full month of April 2025, 3x3 averaging is applied to 50GHz channels only and the following observation errors (Table 3): AWS channel observation error 4 (AWS-14) 0.27 5 (AWS-15) 0.20 6 (AWS-16) 0.23 7 (AWS-17) 0.24 11 (AWS-32) 0.9 12 (AWS-33) 0.9 13 (AWS-34) 0.9 14 (AWS-35) 0.9 15 (AWS-36) 0.9 Table 3. Observation errors used in the experiment presented in this document The baseline setup for the AWS observation errors were the same as for AMSU-A and MHS. During the sensitivity test, Desroziers diagnostics [6] were applied to check the consistency between the applied observation errors and the ones suggested by the diagnostics. The Desroziers diagnostics indicated that the obs errors for the 50 GHz channels should be slightly increased and the obs errors decreased for the 183 GHz channels. Eventually we ended up with the settings in Table 3. In Figure 5, the left plot, we see that the impact is positive for most temperature sensing ATMS channels, 6-10, except the lowest peaking (ch 6) and negative for the humidity sensing channels, 18-22. The signal is weaker in METOP-C, right plot in Figure 5, and 8 neutral for temperature sensing channels and slightly negative for humidity sensing channels. Figure 6 shows how the assimilation system has handled the AWS observations by comparing first guess departures STDEV(O-B) to analysis departures STDEV(O-A). If the red line, STDEV(O-A), is below the blue line then the minimization has drawn the analysis closer to the assimilated observation. From Figure 6 it is clear that the analysis has been drawn to the AWS observations. It seems like the humidity sensing channels, 11-15, are drawn a bit too aggressively meaning that the observation errors should probably be inflated. It should however be mentioned that in the sensitivity test mentioned above, inflation of the observation errors for the humidity sensing channels did not change the impact signal from negative to positive. One reason for the stronger impact signal on ATMS compared to AMSU-A/MHS lies in which assimilation cycle the data is available. In Figure 7 the number of observations available from AWS, METOP-C and NOAA-20 in each assimilation cycle is shown. We can see that AWS and METOP-C are often available in the same assimilation cycle while the availability of NOAA-20 is shifted about 6 hours. It might be so that the effect of assimilating AWS is projected through the system via cycling and is then seen in the first guess by ATMS in DA cycles where less other microwave sensors are available. In deliverable 6 [3] the assimilation of AWS radiances was also assessed for the same time-period, but over the Metcoop domain and with 4D-Var instead of 3D-Var. The experiment setup was also different in the sense that AWS was added on top of a baseline experiment where METOP-B was not included. In that experiment the impact on ATMS by AWS assimilation was neutral on temperature sensing channels and positive on humidity sensing channels. The main difference between those experiments and the ones presented in this report is the domain used. This would suggest that one month long impact experiments are very sensitive to the weather conditions, i.e. the Metcoop domain sampled different conditions than the Arome-Arctic domain. To further emphasize this, experiments with AWS are being carried out with the pre-operational Arome-Arctic system for the period 1-20 July 2025. In this case the NWP system is a different H-A cycle (cy46), but still with 3D-Var. Results from these tests show that the impact on ATMS by AWS assimilation was neutral on temperature sensing channels and positive on humidity sensing channels, similarly to the 4D-Var experiment over the Metcoop domain reported in [3] and mentioned above, see Figure 8. 9 Figure 10. STDEV of first guess departures (O-B) for AWS channels 11-15 in the period 2025-04-20 to 2025-04-30. Black: STDEV(O-B) when original cloud filtering is used. Red: STDEV(O-B) when Chalmers cloud filtering is used. 16 References [1] Rydberg, B. (2025). Remapping of AWS data. Zenodo. https://doi.org/10.5281/zenodo.15049254 [2] Dahlgren, P., McEvoy, P., Eriksson, P., Dybbroe, A., Guedj, S., Aspenes, T., Lahtinen, P., (2025). Quantification of AWS data quality and timeliness. Deliverable D9. Performance Evaluation of Arctic Weather Satellite Data. ESA Contract 4000136311 [3] Reima, E. Dahlgren, P. Schoenach, D. Dybbroe, A. (2025). Impact assessment of AWS constellation using real AWS data. Deliverable D6. Performance Evaluation of Arctic Weather Satellite Data. ESA Contract 4000136311 [4] May, E., Rydberg, B., Kaur, I., Mattioli, V., Hallborn, H., and Eriksson, P. (2024). The Ice Cloud Imager: retrieval of frozen water column properties, Atmos. Meas. Tech., 17, 5957–5987, https://doi.org/10.5194/amt-17-5957-2024 [5] Kaur, I., Eriksson, P., Pfreundschuh, S., Duncan, I. (2021). Can machine learning correct microwave humidity radiances for the influence of clouds?, Atmos. Meas. Tech., 14, 2957–2979, https://doi.org/10.5194/amt-14-2957-2021 [6] Desroziers G, Berre L, Chapnik B, Poli P. Diagnosis of observation, background and analysis-error statistics in observation space. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography 2005;131(613):3385–3396. 17