Validation of the STORM model used in IRI with ionosonde data

Validation of the STORM model used in IRI with ionosonde data G. Miro ´ Amarante a,* , M. Cueto Santamarı ´a b , K. Alazo c , S.M. Radicella a a Aeromony and Radiopropagation Laboratory, Abdus Salam ICTP, Strada Costiera 11, 34014 Trieste, Italy b Geophysics and Meteorology Department, Faculty of Physics, Complutense University, Avda. Complutense, S/N 28040, Madrid, Spain c Instituto de Geofı´sica y Astronomı´a, Calle 212, N. 2906, C. Habana 11600, Cuba Abstract The empirical ionospheric storm correction model STORM included in the new version of the IRI model is driven by the previous time-history of the ap index and is designed to be dependent on latitude and season. The behavior of this correction model has been analyzed comparing foF2 values obtained by the IRI model with or without storm correction and those measured by ionosondes distributed around the world. A statistical analysis was done using two geomagnetic storms during 2003 (medium solar activity). Keywords: Ionospheric storm model; IRI; foF2 values; Ionosonde data; Validation 1. Introduction The International Reference Ionospheric model (IRI) (Bilitza, 1990, 2001) is the result of an international project sponsored by the Committee on Space Research (COSPAR) and the International Union of Radio Science (URSI), which aims at producing a reference model of the ionosphere based on available experimental data sources. The IRI model can be run on-line at: http://nssdc.gsfc.nasa.gov/space/model/ models/iri.html and the source code is available at: ftp:// nssdcftp.gsfc.nasa.gov/models/. One of the major limitations of IRI was that it provided only monthly averages for magnetically quiet conditions. The storm correction model STORM (Araujo-Pradere et al., 2002) was included in the new version of IRI and was designed to be dependent on the intensity of the storm (ap index over the 33 previous hours), latitude, and season. The model considers a threshold of about 9 in the ap index before correcting the foF2 value for quiet conditions. The real time version of the model is available at: http://sec. noaa.gov/storm/. The purpose of this paper is to evaluate this storm corrections by comparing the foF2 values obtained by the IRI model, with and without storm correction, with those measured by ionosondes. Data from about 15 observing sites have been analyzed for two geomagnetic storms in 2003. Similar validation studies have recently been done by Araujo-Pradere and Fuller-Rowell (2002), Araujo-Pradere et al. (2003, 2004), and Miro ´Amarante et al. (2004). The results obtained by Araujo-Pradere et al. (2004) show that the storm model captures very well the direction and magnitude of the changes in the summer stations where there is a clear tendency for a negative phase, this was not the case, however, for winter stations likely because of the increased variability in foF2. 2. Data Table 1 shows the two geomagnetic storms (ap > 150) analyzed in this study: October 2003 (smoothed sunspot number: 58.2) and November 2003 (smoothed sunspot number: 56.7). Ionosonde data were obtained directly from several ionospheric groups (National Observatory of * Corresponding author. E-mail addresses: [email protected] (G. Miro ´Amarante), mcs@fis.ucm.es (M. Cueto Santamarı ´a), [email protected] (K. Alazo). Athens, Greece; Communications Research Laboratory, Tokyo, Japan; Istituto Nazionale di Geofisica e Vulcanologia, Rome, Italy; Instituto de Geofı ´sica y Astronomı ´a, La Habana, Cuba, CASLEO, San Juan, Argentina, and IPS Radio and Space Services, Australia), or downloaded from the University of Massachusetts Lowell DIDBase (http:// ulcar.uml.edu/DIDB/DIDBHome.html). In order to avoid possible errors in the automatic scaling those data were manually scaled using the Sao-Explorer software (http:// ulcar.uml.edu/SAO-X/SAO-X.html). Ionograms from 30 ionosonde locations were scaled but only those with a continuous 5-day period of foF2 values were included in the evaluation. Geographic coordinates of these observing sites, as well as their geomagnetic latitude and modified dip latitude, are shown in Table 2. Unfortunately, latitudinal dependence cannot be discussed because of the scarcity of high latitude data. 3. Methodology For each storm hourly measured foF2 values have been compared with model predictions using the IRI with STORM and without STORM. The statistical analysis presented here is based on the method developed by Araujo-Pradere et al. (2004). That paper evaluates the empirical storm-time ionospheric correction model Table 2 Geographic coordinates, geomagnetic latitudes, and modified dip latitudes of the observing sites used in the storm IRI model evaluation Station Lat. Long. Lat. geomag. Modip October 2003 Midlatitude (40/60) Fairford (FAI) 51.70 358.50 54.55 56.06 Chilton (CHI) 51.50 359.40 54.18 55.92 Rome (ROM) 41.90 12.50 42.40 49.17 Dyess Afb (DYE) 32.50 260.30 42.16 49.61 Midlatitude (20/40) Athens (ATH) 38.00 23.50 36.48 45.96 Wakkanai (WAK) 45.40 141.70 35.42 51.15 Kokubunji (KOK) 35.70 139.60 25.60 43.44 Low latitude (0/20) Jicamarca (JIC) 12.00 283.20 0.68 0.65 Ascension Is (ASC) 7.95 345.60 1.40 23.24 Midlatitude (20/40) Townsville (TOW) 19.63 146.85 28.60 41.26 Norfolk Is. (NOR) 29.03 167.97 34.60 46.45 Brisbane (BRI) 27.53 152.92 35.58 46.87 Midlatitude (40/60) Mundaring (MUN) 31.98 116.22 43.31 51.55 November 2003 Midlatitude (40/60) Chilton (CHI) 51.50 359.40 54.18 55.92 Millstone H (MIL) 42.60 288.50 53.99 55.90 Rome (ROM) 41.90 12.50 42.40 49.17 Dyess AFB (DYE) 32.50 260.30 42.16 49.61 Eglin AFB (EGL) 30.40 273.20 41.23 49.50 San Vito (SAV) 40.60 17.80 40.11 48.11 Midlatitude (20/40) Athens (ATH) 38.00 23.50 36.48 45.96 Wakkanai (WAK) 45.40 141.70 35.42 51.15 Ramey (RAM) 18.50 292.90 29.89 42.48 Kokubunji (KOK) 35.70 139.60 25.60 43.44 Low latitude (0/20) Ascension Is (ASC) 7.95 345.60 1.40 23.24 Midlatitude (20/40) Concepcion (CON) 36.80 286.90 25.42 35.59 Townsville (TOW) 19.63 146.85 28.60 41.26 Learmonth (LEA) 21.80 114.00 33.17 45.14 Grahamstown (GRA) 33.30 26.50 33.80 50.97 Norfolk Is. (NOR) 29.03 167.97 34.60 46.45 Brisbane (BRI) 27.53 152.92 35.58 46.87 Midlatitude (40/60) Mundaring (MUN) 31.98 116.22 43.31 51.55 The ionosonde stations are sorted into three geomagnetic groups (0–20, 20–40, 40–60). Table 1 Geomagnetic storm periods selected in the storm model evaluation Day Month Day year Year Ap index Day Month Day year Year Ap index 27 10 300 2003 11 19 11 323 2003 12 28 10 301 2003 25 20 11 324 2003 150 29 10 302 2003 204 21 11 325 2003 42 30 10 303 2003 191 22 11 326 2003 30 31 10 304 2003 116 23 11 327 2003 22 Fig. 1. Hourly variation of foF2 during the October 2003 storm at Mundaring, Rome, Athens, Brisbane, and Ascension Island. Circles, model predictions of foF2 with Storm model; triangles, foF2 predictions without Storm model; squares, foF2 ionosonde values (measurements). Fig. 2. Hourly variation of foF2 during the November 2003 storm at Learmonth, Kokubunji, Athens, Millstone Hill and Ascension Island. Circles, foF2 values with Storm model; triangles, foF2 predictions without Storm model; squares, foF2 ionosonde values (measurements). STORM during all the significant geomagnetic storms in 2000 and 2001 (high solar activity). The results obtained in the current study apply to 2003, a middle-low solar activity period, allowing us to establish a possible solar activity dependence of the storm model. We calculated the daily root mean square error (RMSE) for the entire day (RMSE 24 hours), for the period from 08 to 16 LT (RMSE daytime), and for the period from 20 to 04 LT (RMSE nighttime) RMSE ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P N1 i¼0 foF2modifoF2expi ½ 2 N v u u u tð1Þ were foF2mod is the model prediction and foF2exp the measured value. We calculated RMSE using IRI without STORM and with STORM. Nis the number of hours for each case. The percentage improvement (AraujoPradere et al., 2004) is then given as improvementð%Þ¼RMSEwithout RMSEwith RMSEwithout 100: ð2Þ Negative values of ‘‘improvement’’ indicate that the STORM model is worse than the quiet day model. 4. Results Fig. 1 compares the measured foF2 values with the model predictions using the Storm and no Storm versions of the IRI model. The plots show the results for some of the stations for a 5-day period (120 h, DOY 300–304) in October 2003. It can be seen that storm effects appear for DOY 302, 303, and 304. The STORM model shows mixed success. For negative ionospheric responses on days 303 and 304 it captures the direction of the changes at Mundaring, Brisbane, Townsville, and Norfolk Is. (summer), as well as at Fairford and Chilton (winter). However, STORM does not capture the direction of the changes at Rome and Athens (winter). For positive ionospheric response on day 302, STORM captures the direction of the changes at Athens, Kokubunji, and Rome (winter), but does not capture the direction of the changes at Brisbane, Mundaring, and Townsville (summer). The STORM model prediction for days 303 and 304 at Ascension Island and Jicamarca (summer) and Dyess and Wakkanai (winter) are not improving the quiet-day IRI predictions. A similar analysis was made for the November 2003 storm. Fig. 2 shows some examples of the comparison between measured and predicted foF2 values for DOY 323–327 (November 19–23). For the negative ionospheric response on day 325, STORM captures the direction of the changes at Learmonth, Brisbane, Grahamstown, Mundaring, Townsville, Norfolk Is., and Concepcion (summer), but does not capture the direction of the changes at Kokubunji (winter). For the positive ionospheric response, STORM captures the direction of the changes at Athens, Ramey, San Vito and Rome (winter), but does not capture the direction of the changes at Millstone Hill and Chilton Table 3 Percentage improvement (% Imp) during the 5-day period of the October 2003 (left panel) and November 2003 (right panel) storms at each ionosonde station October 2003 November 2003 Name Lat. geomg 300 % Imp 301 % Imp 302 % Imp 303 % Imp 304 % Imp Name Lat. geomg 323 % Imp 324 % Imp 325 % Imp 326 % Imp 327 % Imp FAI 54.55 3.8 11.7 27.1 50.0 48.9 CHI 54.18 1.3 2.1 100.0 4.3 7.3 CHI 54.18 10.9 26.7 45.6 40.7 MIL 53.99 11.5 9.5 52.9 10.1 4.5 ROM 42.40 6.8 9.0 17.3 41.2 37.1 ROM 42.40 5.2 8.0 9.6 6.5 24.2 DYE 42.16 3.6 10.7 2.4 1.4 2.7 DYE 42.16 0.0 4.8 0.3 15.2 15.3 ATH 36.48 3.4 8.9 22.6 108.2 70.8 EGL 41.23 5.6 4.8 2.6 12.0 15.0 WAK 35.42 4.3 9.7 4.0 1.8 0.0 SAV 40.11 2.5 8.9 5.1 3.4 13.0 KOK 25.60 2.2 4.3 15.5 29.4 27.5 ATH 36.48 2.0 15.0 13.4 2.2 19.5 JIC 0.68 2.6 5.1 2.8 0.5 10.0 WAK 35.42 11.7 10.3 7.0 34.2 0.0 ASC 1.40 1.9 3.0 1.0 3.3 2.5 RAM 29.89 10.0 17.9 13.2 16.0 20.1 TOW 28.60 0.0 1.0 23.5 50.7 16.1 KOK 25.60 15.9 27.6 66.7 7.5 5.1 NOR 34.60 0.0 4.4 8.1 61.5 48.0 ASC 1.40 0.0 2.6 1.8 11.6 6.3 BRI 35.58 1.6 1.2 6.4 38.3 63.4 CON 25.42 0.5 28.7 28.8 4.9 2.6 MUN 43.31 0.0 2.5 15.6 39.4 59.9 TOW 28.60 0.7 11.7 58.6 13.8 16.5 LEA 33.17 1.0 1.2 51.9 17.5 8.8 GRA 33.80 2.2 10.9 58.8 9.0 24.3 NOR 34.60 1.6 36.6 54.4 38.0 7.0 BRI 35.58 0.9 28.2 46.0 17.8 12.0 MUN 43.31 0.0 15.2 73.9 6.3 11.1 (winter). STORM does not improve the predictions on day 325 at Ascension Island (summer), Wakkanai, Dyess, and Eglin (winter). Clearly, STORM does not improve the predictions the two low latitude stations analyzed, Ascension Island and Jicamarca, during the storm periods. This is to be expected because of the scarcity of data that were available for the development of STORM and the still poor understanding of the different low latitude physical processes involved (Araujo-Pradere et al., 2002). Table 3 shows the percentage improvements, as defined above for the 5-day storm periods in October (left panel) and November (right panel) 2003. The RMSE was calculated considering the previous 24 h. Table 3 is sorted by geomagnetic latitude since the STORM model was designed with a latitudinal dependence. Negative values Table 4 RMSE for Northern and Southern Hemispheres for each of the 5 days in October 2003 storm Day Ap RMSE 24 hours RMSE daytime RMSE nighttime Count Storm No Storm Count Storm No Storm Count Storm No storm North hemisphere (Oct 2003) 300 11 142 1.34 1.39 52 1.81 1.89 54 0.62 0.64 301 25 165 1.36 1.50 61 2.06 2.28 62 0.51 0.56 302 204 139 1.70 2.02 56 2.26 2.71 48 1.12 1.30 303 191 123 2.90 2.91 55 2.18 2.20 34 4.18 4.15 304 116 154 2.02 1.94 62 2.45 2.34 52 1.86 1.82 Average storm period 1.86 1.95 2.15 2.28 1.66 1.69 Average storm days 2.21 2.29 2.30 2.42 2.39 2.42 % improvement 3.5% 4.9 % 1.2% South hemisphere (Oct 2003) 300 11 135 1.52 1.55 51 1.37 1.42 52 1.75 1.76 301 25 131 1.96 2.01 48 1.74 1.82 47 2.18 2.21 302 204 108 2.38 2.25 41 2.20 2.05 42 2.69 2.62 303 191 97 2.09 2.58 19 2.31 2.77 45 2.02 2.26 304 116 105 2.00 2.46 21 2.39 2.99 52 1.79 2.04 Average storm period 1.99 2.17 2.00 2.21 2.09 2.18 Average storm days 2.16 2.43 2.30 2.60 2.17 2.31 % improvement 11.1% 11.5% 6.1% Averages for the entire storm period and for the specific storm days (in bold) are also shown. The percentage of improvement is calculated for the storm days. Results are also shown for daytime (08–16 LT) and nighttime (20–04 LT). Table 5 RMSE for North and South Hemispheres for each of the 5 days in November 2003 Day Ap RMSE 24 hours RMSE daytime RMSE nighttime Count Storm No storm Count Storm No storm Count Storm No storm North hemisphere (Nov 2003) 323 12 237 0.86 0.85 89 0.86 0.94 88 0.82 0.75 324 150 203 1.97 2.13 82 2.37 2.63 66 0.95 0.88 325 42 213 2.43 2.38 81 2.35 2.02 80 3.06 3.16 326 30 202 1.17 1.26 73 1.51 1.73 76 0.75 0.66 327 22 214 1.23 1.40 90 1.42 1.74 67 1.01 0.98 Average storm period 1.53 1.60 1.70 1.81 1.32 1.29 Average storm days 2.20 2.26 2.36 2.32 2.00 2.02 % improvement 2.6% 1.7% 1.0% South hemisphere (Nov 2003) 323 12 179 1.31 1.30 64 1.27 1.26 71 1.37 1.36 324 150 161 1.54 1.89 63 1.52 1.93 59 1.40 1.61 325 42 107 1.79 2.80 16 2.93 4.02 57 1.56 2.52 326 30 146 1.36 1.28 55 1.45 1.34 52 1.36 1.31 327 22 161 1.28 1.28 61 1.53 1.46 61 1.14 1.22 Average storm period 1.46 1.71 1.74 2.00 1.37 1.60 Average storm days 1.66 2.34 2.22 2.97 1.48 2.06 % improvement 29.0% 25.2% 28.1% Averages for the entire storm period and for the specific storm days (in bold) are also shown. The percentage improvement is calculated for the storm days. Results are also shown for daytime (08–16 LT) and nighttime (20–04 LT). mean that STORM worsens the predictions (shadowed cells). The predicted values for the negative effects observed in the Australian region and at two European stations (Chilton and Fairford) during October 2003 (days 303 and 304) are improved by up to 63%. In the case of November 2003 (day 325), the improvement reaches values of 74% in the Australian region. It is important to point out that Chilton and the Australian stations have been included in the 75 ionosonde stations used in the development of the STORM model (Araujo-Pradere et al., 2002). The model improvement is moderate for positive effects predictions, up to 23% in October 2003 and 13% in November 2003, both of them in Athens station. Table 4 shows the RMSE for the Northern and Southern Hemispheres for each of the 5 days in October 2003. Storm days are marker in bold. The average for the entire storm period and for the specific storm days (bold) are also shown. The percentage of improvement is calculated for the storm days. As it has been noticed before, the best results are found for the Southern Hemisphere (summer stations) where the percentage improvement is more than 10%. In order to find a possible hourly dependence, RMSE and the percentage improvement has been separately calculated for dayand nighttime periods. The results obtained clearly show that the model works better between 08:00 and 16:00 LT. The analysis results for the November 2003 storm are shown in Table 5. In this case, the STORM model improves the predictions up to 29% for summer stations. For this particular storm, results do not show differences between daytime and nighttime behavior. The percentages of improvement found by Araujo-Pradere et al. (2004) for October and November 2001 storms are calculated with the previous 24 h and in general, they are larger than the corresponding values obtained in this paper. These differences could be related to the higher solar activity in 2001, the sunspot number was around 115, double that in 2003. 5. Conclusions Two geomagnetic storms (ap > 150) have been analyzed: October 2003 (13 ionosonde stations; smoothed sunspot number: 58.2), and November 2003 (18 ionosonde stations; smoothed sunspot number: 56.7). Measured foF2 data were compared with IRI model predictions using the STORM or no-STORM versions. STORM captures quite well the direction of the changes for negative effects during summer conditions. The improvement reaches values up to 63% during October 2003 and 74% for November 2003. For positive effects during winter conditions STORM, in general, improves the predictions in Europe and Japan with improvements reaching 23% for October 2003 and 13% for November 2003. For low latitudes (Ascension Island and Jicamarca) no clear improvement is achieved. In the case of the October 2003 storm the model works better during daytime with improvement of 11% at southern locations. Further studies including high latitude data are required to establish a possible latitudinal dependence of the accuracy in foF2 predictions. Acknowledgements The authors thank the University of Massachusetts Lowell for providing access to the digital ionogram data base (DIDBase), and the ionospheric groups (National Observatory of Athens (Greece), Communications Research Laboratory (Tokyo, Japan), Istituto Nazionale di Geofisica e Vulcanologia (Rome, Italy), the AF DISS network (USA), Instituto de Geofı ´sica y Astronomı ´a (La Habana, Cuba), CASLEO (San Juan, Argentina), and the IPS Radio and Space Services (Australia)), whose data were used in this study. References Araujo-Pradere, E.A., Fuller-Rowell, T.J., Codrescu, M.V. STORM: an empirical storm-time ionospheric correction model, 1, Model description. Radio Sci. 37 (5), 1070, doi:10.1029/2001RS00246, 2002. Araujo-Pradere, E.A., Fuller-Rowell, T.J. STORM: an empirical stormtime ionospheric correction model, 2, Validation. Radio Sci. 37 (5), 1071, doi:10.1029/2002RS00262, 2002. Araujo-Pradere, E.A., Fuller-Rowell, T.J., Bilitza, D. Validation of the STORM response in IRI2000. J. Geophys. Res. 108 (A3), 1120, doi:10.1029/2002JA00972, 2003. Araujo-Pradere, E.A., Fuller-Rowell, T.J., Codrescu, M.V., Anghel, A. Evaluation and prospects for storm-time corrections in the International Reference Ionosphere. Adv. Space Res. 33, 902–909, 2004. Bilitza, D. International Reference Ionosphere 1990. NSSDC 90-22, Greenbelt, Maryland, 1990. Bilitza, D. International reference ionosphere 2000. Radio Science 36 (2), 261–275, 2001. Miro ´Amarante, G., Alazo, K., Cueto, M., Radicella. S.M. Comparison between experimental and IRI foF2 values under quiet and storm conditions. Proceedings IRI TFA 2004. in press.