Validation of a new SAFRAN-based gridded precipitation product for Spain and comparisons to Spain02 and ERA-Interim

Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 www.hydrol-earth-syst-sci.net/21/2187/2017/ doi:10.5194/hess-21-2187-2017 © Author(s) 2017. CC Attribution 3.0 License. Validation of a new SAFRAN-based gridded precipitation product for Spain and comparisons to Spain02 and ERA-Interim Pere Quintana-Seguí1, Marco Turco2, Sixto Herrera3, and Gonzalo Miguez-Macho4 1Observatori de l’Ebre (OE), Universitat Ramon Llull – CSIC, Horta Alta 38, 43520 Roquetes, Spain 2Department of Applied Physics, University of Barcelona (UB), Av. Diagonal 647, 08028 Barcelona, Spain 3Meteorology Group, Department of Applied Mathematics and Computer Science, Universidad de Cantabria, Avenida de los Castros s/n, 39005 Santander, Spain 4Grupo de Física No Lineal, Universidade de Santiago Compostela, Calle Xosé María Suárez Núñez s/n., 15782 Santiago de Compostela, Spain Correspondence to: Pere Quintana-Seguí ([email protected]) Received: 13 July 2016 – Discussion started: 15 September 2016 Revised: 5 March 2017 – Accepted: 24 March 2017 – Published: 21 April 2017 Abstract. Offline land surface model (LSM) simulations are useful for studying the continental hydrological cycle. Because of the nonlinearities in the models, the results are very sensitive to the quality of the meteorological forcing; thus, high-quality gridded datasets of screen-level meteorological variables are needed. Precipitation datasets are particularly difficult to produce due to the inherent spatial and temporal heterogeneity of that variable. They do, however, have a large impact on the simulations, and it is thus necessary to carefully evaluate their quality in great detail. This paper reports the quality of two high-resolution precipitation datasets for Spain at the daily time scale: the new SAFRAN-based dataset and Spain02. SAFRAN is a meteorological analysis system that was designed to force LSMs and has recently been extended to the entirety of Spain for a long period of time (1979/1980–2013/2014). Spain02 is a daily precipitation dataset for Spain and was created mainly to validate regional climate models. In addition, ERA-Interim is included in the comparison to show the differences between local high-resolution and global lowresolution products. The study compares the different precipitation analyses with rain gauge data and assesses their temporal and spatial similarities to the observations. The validation of SAFRAN with independent data shows that this is a robust product. SAFRAN and Spain02 have very similar scores, although the latter slightly surpasses the former. The scores are robust with altitude and throughout the year, save perhaps in summer when a diminished skill is observed. As expected, SAFRAN and Spain02 perform better than ERA-Interim, which has difficulty capturing the effects of the relief on precipitation due to its low resolution. However, ERA-Interim reproduces spells remarkably well in contrast to the low skill shown by the high-resolution products. The high-resolution gridded products overestimate the number of precipitation days, which is a problem that affects SAFRAN more than Spain02 and is likely caused by the interpolation method. Both SAFRAN and Spain02 underestimate high precipitation events, but SAFRAN does so more than Spain02. The overestimation of low precipitation events and the underestimation of intense episodes will probably have hydrological consequences once the data are used to force a land surface or hydrological model. 1 Introduction Good knowledge of the continental water cycle and its quantification is critical for society, mainly because it represents both a resource and a hazard. As a consequence, a better understanding of the continental water cycle is a key issue for climate change impact and adaptation studies, and it has become a strategic topic in national and international climate programs, such as the Global Energy and Water Cycle Exchanges Project (GEWEX; http://www.gewex.org/) at the global scale and the Hydrological cycle in the Mediterranean EXperiment (HyMeX; http://www.hymex.org/) at the Published by Copernicus Publications on behalf of the European Geosciences Union. 2188 P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim Mediterranean scale. Simulating the continental water cycle is difficult. For example, precipitation is a challenging variable due to its spatial heterogeneity, spatial variability and time variability. Evapotranspiration is a very complex process that is difficult to measure and depends on the atmosphere, soil and vegetation in a complex manner (e.g., Peters- Lidard et al., 2011; Nasonova et al., 2011; Long et al., 2014). There are different approaches to simulating the continental cycle. These range from easy-to-calibrate simple water balance models (e.g., Orth and Seneviratne, 2015) to the more complex land surface models (LSM). LSMs simulate the physical processes at the interface among the soil, vegetation and the atmosphere. These models may be run offline and forced by a gridded dataset of screen-level meteorological variables or online and coupled to an atmospheric model. The main advantage of using offline LSMs is that they avoid atmospheric model biases because they are forced by gridded observational datasets, but they miss many feedback processes. Such systems have been applied at the global, continental, national and basin scales (Rodell et al., 2004; Decharme et al., 2012; Balsamo et al., 2012; Cosgrove, 2003; Mitchell, 2004; Chen et al., 2007; Habets et al., 2008; Artinyan et al., 2008; Szczypta et al., 2012; Barbu et al., 2014; Martínez de la Torre, 2014) in a wide range of work that includes the study of water resources, the initialization of meteorological models and the study of the continental water cycle. The performance of offline LSM simulations depends strongly on the quality of the meteorological datasets used to force the models. In addition, the nonlinear nature of the LSMs, which may amplify the errors, makes it essential to ensure the good quality of the meteorological forcing datasets for land surface and hydrological modeling. Unfortunately, it is not an easy task to build such gridded datasets that include all of the meteorological parameters and reach the spatial (at least 5 km) and temporal (intra-daily) resolutions needed to force an LSM. Meteorological analysis systems such as CANARI (Taillefer, 2002), SPAN (Rodríguez et al., 2003; Navascués et al., 2003; Cansado et al., 2003), MESAN (Häggmark et al., 2000) MESCAN (Soci et al., 2016) and SAFRAN (Durand et al., 1993, 1999; Quintana- Seguí et al., 2008; Vidal et al., 2010; Quintana-Seguí et al., 2016) are well suited for this task. Concerning precipitation, which is the variable examined in this paper, there are other interesting reference datasets that might not be able to force an LSM because they may not have sufficient temporal or spatial resolution (they are often daily products with resolutions that range from 50 to 12 km); however, they are well known and widely used for other applications, such as the validation (e.g., Gómez-Navarro et al., 2012; Turco et al., 2013a) and post-processing of regional climate models (Turco et al., 2011; Casanueva et al., 2015) or the statistical downscaling of global climate models (Casanueva et al., 2016). It is interesting to compare the analysis systems mentioned in the previous paragraph with these daily reference datasets. At the European scale, E-OBS (Haylock et al., 2008) is probably the best known of such products. However, in some countries, the station density considered when building E- OBS was very low, thus limiting the capability of that dataset to properly reproduce the climatic variability in those regions. As a result, some products have been developed for specific regions or countries, such as the gridded dataset EURO4M-APGD (Isotta et al., 2014) for the greater Alpine region and Spain02 (Herrera et al., 2012, 2016) in Spain, which consider a denser station network. A careful validation of these products should be performed to ensure that they correctly reproduce the important characteristics of the climate in the region under study. The validation should test both the temporal and spatial accuracies of the products, not only for the mean values but also for the extremes. Some examples of such studies are listed in the following. Ensor and Robeson (2008) analyzed the statistics of precipitation in gridded products in the USA and found that the interpolation process significantly increased the number of low precipitation events while greatly reducing the intensity and frequency of heavy precipitation episodes. Prein and Gobiet (2016) evaluated many regional datasets in Europe and demonstrated that the differences among them have the same magnitude as the precipitation errors found in regional climate models. Flaounas et al. (2012) validated the E-OBS dataset in the Mediterranean region and found that those data must be used with care in complex environments such as coastal areas and mountainous regions. Turco et al. (2013b) compared three different gridded datasets in the greater Alpine region (E-OBS and two local grids) and found that specific regional products are often more trustworthy than E-OBS. Herrera et al. (2012) compared Spain02 and E-OBS in Spain and found that the station density severely affects the results. Turco and Llasat (2011) confirmed the high quality of the Spain02 dataset in northeastern Spain. Belo-Pereira et al. (2011) compared IB02, an Iberian daily precipitation dataset built by joining two methodologically equivalent gridded products for Portugal (PT02) and Spain (Spain02 v2), to global datasets and found that the global products produce better results in western Iberia than on the Mediterranean side. Finally, Katsanos et al. (2016) validated the CHIRPS global remote-sensing product (Funk et al., 2015) in Cyprus and compared it to E-OBS and local rain gauge data and found that at the monthly scale, the results showed a good correlation between the CHIRPS values and the recorded precipitation. The objective of this paper is 2-fold. Firstly, this paper validates SAFRAN’s precipitation as reproduced in the most recent Spanish SAFRAN dataset (Quintana-Seguí, 2015), which is a temporal and spatial extension of the dataset presented in Quintana-Seguí et al. (2016). The new dataset spans a period of 35 years (1979/1980–2013/2014) and geographical covers mainland Spain and the Balearic Islands. Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 www.hydrol-earth-syst-sci.net/21/2187/2017/ P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim 2189 Figure 1. Maps of the study area. The left panel shows the relief (using the same 5 km resolution grid that SAFRAN uses) and the main river basins in Spain (note that for the sake of simplification, some of the smaller river basins have been grouped). The right panel shows SAFRAN’s climatically homogeneous zones, the rain gauges used for the study and their mean yearly precipitation. The dependent and independent stations are shown as dots and triangles, respectively. Secondly, this paper compares the SAFRAN precipitation dataset with Spain02, which is a well-known, high-quality daily precipitation dataset for Spain, and ERA-Interim, the ECMWF’s reanalysis dataset. ERA-Interim is not the same kind of dataset as SAFRAN or Spain02, in the sense that it is a reanalysis product into which gauge observations are not assimilated. Precipitation in ERA-Interim is the direct result of a model forecast, even though the dynamics conducive to it are indeed constrained by other assimilated observational data in the reanalysis procedure. Nevertheless, due to its high quality, ERA-Interim is of special interest in hydrology because it is the basis for other derived products that are tailored for hydrological applications such as WFDEI (Weedon et al., 2014). Moreover, ERA-Interim is a dataset suitable to force LSMs, so it is introduced in the comparison to show how a low-resolution global product compares to local higher resolution ones, which is one of the objectives of the eartH2Observe project and thus of this paper. E-OBS has not been included for several reasons. On the one hand, E-OBS can be considered a different kind of product in comparison to SAFRAN and Spain02 due to its much lower density of stations and the lower resolution of the final product. On the other hand, the E-OBS station density is especially low in Spain; as a consequence, this dataset cannot correctly reproduce the Iberian precipitation regimes, which primarily affects the variability and the extremes of the precipitation events; these are wellknown problems reflected in previous studies (e.g., Herrera et al., 2012). Finally, in contrast to ERA-Interim, it is not suitable to force LSMs, which is the main objective of SAFRAN. 2 Study area The geographical domain of this study is peninsular Spain and the Balearic Islands, located in southwestern Europe. Two main factors modulate the climate in this area: the influence of both the Atlantic Ocean and the Mediterranean Sea, and the complex orography (Fig. 1a). On the one hand, frontal systems from the Atlantic Ocean swipe the region regularly bringing wet and cold air, whilst wet and warm air arrives from the Mediterranean Sea. On the other hand, the complex orography and the configuration of the mountain ranges drive the wet air masses distributing the precipitation across the region. The combination of both factors leads to a marked NW–SE precipitation gradient ranging from large amounts of precipitation (from 900 to 2500 mm yr−1) without a dry season on the North Atlantic coast to semiarid and desert regions in the southeast with less than 100 mm yr−1 concentrated in some severe events; there are variants of Atlantic and Mediterranean climates in between (AEMET, 2011). The impact of a surrounding water mass and a complex orography is particularly relevant in the Ebro valley in the northeast of Spain. With both Atlantic and Mediterranean influences, the region has a low mean precipitation that is caused by the shadowing effect of the surrounding orography. This is an example of the important role of Spain’s marked relief on the distribution of precipitation, enhancing it in some areas and decreasing it in others, which in turn has an important effect on runoff generation. This is an added difficulty for gridded precipitation products. Figure 1a shows the study area, its relief and the main river basins. 3 Datasets and methods 3.1 SAFRAN The SAFRAN (Système d’Analyse Fournissant des Renseignements Atmosphériques à la Neige) meteorological analysis system (Durand et al., 1993, 1999) was initially created with the objective of forcing the CROCUS snow model (Brun et al., 1989) in the French Alps to improve avalanche www.hydrol-earth-syst-sci.net/21/2187/2017/ Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 2190 P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim prediction. Currently, it is extensively used in a wide range of applications. SAFRAN uses an optimal interpolation algorithm (OI) (Gandin, 1966) that combines observations and a first guess (e.g., the outputs of a meteorological model) to produce a gridded dataset of precipitation, temperature, wind speed, relative humidity and cloudiness. SAFRAN is also able to calculate downward visible and infrared radiation (Ritter and Geleyn, 1992). SAFRAN is currently used for operational and research purposes in France (Quintana-Seguí et al., 2008; Vidal et al., 2010; Habets et al., 2008). It is also used for research purposes in Spain (Quintana-Seguí et al., 2016) and Morocco (Szczypta et al., 2015). In Spain, SAFRAN was first implemented and tested in a pilot study for the northeastern Iberian Peninsula using only 1 year of data (Quintana-Seguí et al., 2016). This study is based on the recent extension of SAFRAN to the whole of Spain over a period of 35 years (1979/1980–2013/2014) (Quintana-Seguí, 2015). One of the main characteristics of SAFRAN is the use of climatically homogeneous zones to divide the space into the analysis areas shown in Fig. 1b. These zones have irregular shapes and cover an area that is generally smaller than 1000 km2. Ideally, they should have no horizontal gradient because, within the zone, SAFRAN cannot reproduce it. As it is impossible to design zones without horizontal gradients, the design of the zones should minimize them as much as possible, especially for precipitation. The zones have several vertical levels spaced at 300 m. The lowest level is on the ground; the other levels are located at 300 m, 600 m etc. (provided that they are above the ground). The highest level of each zone is the first level above the highest point of the relief. SAFRAN estimates the value of the analyzed variable on each level. These values are subsequently horizontally interpolated to the regular grid, which is how the data are presented to the users. In our implementation, the grid has a resolution of 5 km (Fig. 1a shows the relief of Spain using the exact same grid as SAFRAN). Each grid point has a location and an altitude according to the relief and belongs to a given zone (each zone has, on average, 40 grid points). To interpolate the variable to the altitude of the grid point, the values of the two adjacent vertical levels of the zone are used. A consequence is that, even though the zones are initially homogenous, the grid points within a zone have different values if they are at different altitudes. By means of the vertical levels, SAFRAN accounts for variations in precipitation with altitude using the real gradients at the time of analysis. A downside of the climatically homogeneous zones is that they create artificial discontinuities at the borders of the zones. For precipitation, SAFRAN analyzes daily observations, and all other variables are analyzed every 6 h. The data are then time interpolated to the hourly scale using different methods for each variable. For precipitation, relative humidity is currently used to hourly distribute the daily precipitation. For each analysis, SAFRAN uses as much data as possible after performing a quality control based on an iterative procedure that compares the observed and analyzed quantities at the observation location. The number of stations used in the analysis thus changes with time; this optimizes the quality of the daily precipitation but makes the dataset untrustworthy for trend analysis. We note that in this paper we do not study the hourly distribution of the precipitation, but only the quality of the daily data. For more details on SAFRAN, please see Quintana-Seguí et al. (2008, 2016). For this study, SAFRAN has been extended to mainland Spain and the Balearic Islands; for this purpose, a new set of climatological zones has been designed. The new zones, which are shown in Fig. 1b, also cover Portugal for a possible future extension. Quintana-Seguí et al. (2016) tested two methods to define the zones, one based on the contours of the river basins and the other based on the meteorological alert zones used by the Spanish Meteorological State Agency (Agencia Estatal de Meteorología; AEMET). The results showed that using the meteorological alert zones worked slightly better, but the differences were small. In other words, SAFRAN is quite robust regardless of the zone map used, provided it has physical sense and the zones have the right sizes. In order to expand the zone map to the whole of mainland Spain and the Balearic Islands, we first tried to use the same meteorological alert zones, but found that, in general, the zones were too big. As a consequence, we decided to create new zones following a subjective method based on the meteorological alert zones, the basin limits and our expert knowledge. In flat areas, the divisions created were more subjective than on the relief because there is no clear objective climatic delimitation of the terrain. However, it is precisely in these flat areas where SAFRAN is less sensitive to the shape of the zones, as the horizontal gradients are very weak. In addition, this newer version of SAFRAN uses ERA- Interim as a first guess for most variables, excluding precipitation (the variable analyzed in this paper), for which the first guess is deduced from the observations. The ERA- Interim dataset is available from 1979, which determines that the start date of SAFRAN be the same. The meteorological station data come from the AEMET network (see Sect. 3.5). Our dataset finishes in August 2014, which is the date that the data were requested from AEMET for this study. 3.2 Spain02 Spain02 (Herrera et al., 2012, 2016) is a series of high-resolution daily precipitation and temperature gridded datasets developed for peninsular Spain and the Balearic Islands. Version 4 of the product has been produced using the standard EURO-CORDEX (European Coordinated Regional Climate Downscaling Experiment) grids at resolutions of 0.44, 0.22 and 0.11◦and different interpolation approaches (Herrera et al., 2016). To compare Spain02 and SAFRAN, we chose the AA-3D version for this work, which considers the orography to be a covariable (3D) in the in- Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 www.hydrol-earth-syst-sci.net/21/2187/2017/ P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim 2191 terpolation process. It is areally representative (AA); more specifically, the AA-3D version is constructed by calculating the areal average of the interpolation, which is performed by applying thin plate splines (TPS) with the orography at the monthly scale and ordinary kriging (OK) on the daily anomalies. In fact, this is the same method employed by the widely used European-scale product E-OBS (Haylock et al., 2008). As a result, both SAFRAN and the version considered for Spain02 are representative of the areas and account for the relief in the interpolation process. In addition, Spain02 presents the same problems for reproducing trends as described for SAFRAN due to the station network considered (2756 rain gauges), which prioritizes spatial density over temporal consistency. We will use Spain02, or SP02, to refer to the AA-3D version of Spain02, which has a resolution of 0.11◦. 3.3 ERA-Interim ERA-Interim (Dee et al., 2011) is the current global atmospheric reanalysis produced by the European Centre for Medium-Range Weather Forecasts (ECMWF). It covers the period from 1979 onward. ERA-Interim has a spatial resolution of ∼79 km (T255). 3.4 Time periods Figure 2 shows the time periods covered by the SAFRAN and the Spain02 datasets. The latter covers the period from 1971 to 2010, whereas the current 35-year SAFRAN dataset has been produced within two projects and can be divided into three subperiods, as shown in Fig. 2. The central subperiod (labeled SAFRAN1 in the figure) covers the hydrological years (a hydrological year starts on 1 September and finishes on 31 August of the next calendar year) from 1995/1996 to 2006/2007. The other subperiod (SAFRAN2) was developed later in another research project and includes the remaining years in the series between 1979/1980 and 2013/2014. For each of the SAFRAN subperiods, a set of randomly selected stations was used a set of randomly selected stations was excluded from the analysis in order to obtain an independent validation dataset. The excluded stations were selected randomly, but making sure that the spatial separation between the selected stations was larger than a minimum distance in order to guarantee an even spatial distribution of the validation stations. Unfortunately for the SAFRAN1 and SAFRAN2 subperiods, the same validation datasets were not considered, and a common set of independent stations for the whole 35-year period is thus not available for SAFRAN. Furthermore, Spain02 also has its own set of independent stations. The common set of stations that are independent for both SAFRAN and Spain02 is not useful for the validation because there are too few stations that are poorly spatially distributed. Nevertheless, Spain02 has already been validated Figure 2. The data availability for SAFRAN and Spain02. with independent data by Herrera et al. (2012, 2016); thus, in this study, the comparisons between SAFRAN and Spain02 are performed using dependent data. As a consequence, in this study 1. SAFRAN is validated with independent data for the 1995/1996–2006/2007 period (SAFRAN1), 2. and the other comparisons, which also included Spain02 and ERA-Interim, are performed over a common period (1980–2010; natural years) using dependent stations. 3.5 Station data Figure 1b shows the locations of the daily precipitation stations used for this study, which were provided by the Spanish Meteorological State Agency (Agencia Estatal de Meteorología, AEMET). The figure only depicts the 1237 stations for which the time series have at least 90 % of the data, which are the stations that were used for our analysis. The triangles correspond to the independent dataset used to validate SAFRAN for the 1995/1996–2006/2007 time period (249 stations). The smaller dots correspond to the remaining stations, which are used for all other validations (988 stations). 3.6 Comparison measures The methodology used in this paper is based on the approach used by Turco et al. (2013b) with some modifications to account for the hydrological context in which the SAFRAN dataset has been developed. All of the metrics used in this study compare the products with rain gauge data, considering for each station the nearest grid points in terms of the Euclidean distance. The dual nature of precipitation in terms of its occurrence and amount makes it necessary to explicitly consider both components in the analysis and to include appropriated validation measures (e.g., the ROC skill area) and indicators (e.g., the wet-day frequency). First, the temporal agreement between the time series and the binary (the occurrence or non-occurrence of precipitation) sequences were studied using the relative mean absolute error (MAEr) and the Pearson’s correlation (CORR), and the relative operating characteristic (ROC) skill area (RSA) and its components (HIR: hit rate; FAR: false alarm rate; and CAR: correct alarm rate), respectively. In a second stage, the spatial agreement is studied www.hydrol-earth-syst-sci.net/21/2187/2017/ Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 2192 P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim Table 1. The climatic mean and extreme indices for the precipitation used in this work based on ETCCDI (Expert Team on Climate Change Detection and Indices; http://cccma.seos.uvic.ca/ETCCDI). Label Description Units PRCPTOT Total precipitation mm R1 Number of days with precipitation over 1 mm day−1(i.e., wet days) day SDII Mean precipitation on a wet day mm R10 Number of days with precipitation over 10 mm day−1day R20 Number of days with precipitation over 20 mm day−1day RX1D Maximum precipitation in 1 day mm RX5D Maximum precipitation in 5 days mm CWD Consecutive wet days (>1 mm) day CDD Consecutive dry days (<1 mm) day Table 2. The annual and seasonal correlation (CORR), relative mean absolute error (MAEr) and ROC skill area (RSA) of SAFRAN compared to the independent (ind.) and dependent (dep.) observations for the period 1995/96–2005/06. CORR MAEr RSA Mean Q25 Q75 Mean Q25 Q75 Mean Q25 Q75 Annual Ind. obs. 0.82 0.75 0.86 0.62 0.54 0.75 0.77 0.73 0.81 Dep. obs. 0.82 0.77 0.87 0.62 0.53 0.72 0.79 0.75 0.83 Winter Ind. obs. 0.84 0.77 0.90 0.57 0.48 0.71 0.78 0.72 0.82 Dep. obs. 0.85 0.79 0.89 0.58 0.48 0.69 0.79 0.74 0.83 Spring Ind. obs. 0.82 0.76 0.86 0.62 0.54 0.74 0.77 0.73 0.80 Dep. obs. 0.83 0.78 0.87 0.61 0.53 0.71 0.78 0.74 0.82 Summer Ind. obs. 0.71 0.62 0.79 0.88 0.74 1.05 0.70 0.64 0.77 Dep. obs. 0.74 0.66 0.80 0.82 0.71 0.94 0.75 0.70 0.80 Autumn Ind. obs. 0.83 0.76 0.87 0.60 0.51 0.71 0.78 0.73 0.81 Dep. obs. 0.83 0.77 0.88 0.60 0.51 0.71 0.79 0.75 0.83 using the standard precipitation indicators shown in Table 1. All indices are calculated on an annual scale in the common period (see Sect. 3.4), although some indices have also been calculated for each season (PRCPTOT, R1 and RX1D) or at a monthly scale (PRCPTOT and RX1D) for each river basin (see the river basin map in Fig. 1). Finally, the spatial pattern obtained for each index and gridded dataset is compared to the observations using the correlation (CORR), the variability (relative standard deviation, or STDEVr), the relative centered root mean square error (CRMSEr) and the relative bias (BIASr). To make the different indices comparable, the “relative” statistics (STDEVr and CRMSEr) have been normalized by dividing both the CRMSEr and STDEVr by the standard deviation of the observations (the reference dataset). Therefore, an STDEVr of 1 means that the standard deviation of the product is the same as that of the observations. The BI- ASr is relative with respect to the mean of the observations. The Taylor diagram (Taylor, 2001) summarizes these measures on a single plot, which has been produced to show the seasonal statistics. Note that there are differences between the spatial representativity of the considered datasets, ranging from local (stations) to low-resolution (∼79 km2) areal averages (ERA- Interim). These differences have been taken into account by performing, on the one hand, a comparison among the validation scores of the different gridded datasets with respect to rain gauges and, on the other, by defining the global scores based on the spatial averages over large enough areas Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 www.hydrol-earth-syst-sci.net/21/2187/2017/ P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim 2193 (b) CORR:0.82 IQR:0.77/0.87 (c) CORR:0.82 IQR:0.75/0.87(a) CORR:0.26 IQR:0.14/0.43 ERA-Interim SAFRAN Spain02 0 1 0.8 0.6 0.4 0.2 (e) MAEr:0.62 IQR:0.53/0.72 (f) MAEr:0.61 IQR:0.49/0.74(d) MAEr:1.30 IQR:1.14/1.44 (h) RSA:0.79 IQR:0.75/0.83 (i) RSA:0.82 IQR:0.77/0.85(g) RSA:0.36 IQR:0.26/0.52 0 2 1.6 1.2 0.8 0.4 0 1 0.8 0.6 0.4 0.2 Interim SAFRAN Spain02 Interim SAFRAN Spain02 Interim SAFRAN Spain02 Interim SAFRAN Spain02 Interim SAFRAN Spain02 Interim SAFRAN Spain02 CORR MAEr RSA HIR (j) (l) FAR CAR 2 0 0.2 0.4 0.6 0.8 1.2 1.4 1.6 1.8 1 1 0 0.2 0.4 0.6 0.8 Figure 3. The correlation (a–c), relative mean absolute error (d–f) and ROC skill area (g–i) of ERA-Interim, SAFRAN and Spain02 with regard to the observations. The lower row shows the box plots of the (j) CORR, MAEr, and RSA as well as the (l) HIR, FAR, and CAR for the three datasets, reflecting the spatial variability of the scores. (e.g., river basins). Finally, the aforementioned problem has been considered in the following sections in the interpretation and discussion of the results. 4 Results 4.1 Validation of SAFRAN using independent data The first step in our analysis is to validate the new SAFRAN dataset for Spain using independent stations for the period 1995/1996–2006/2007 (SAFRAN1). Table 2 summarizes the results at the annual and the seasonal scale, showing that the CORR, MAEr and RSA are slightly degraded when independent stations are employed (not used to perform the analysis) but have nevertheless a rather similar mean and interquartile range, which is an indicator of the robustness of SAFRAN. SAFRAN is quite robust across seasons with summer as the season with the worst performance. 4.2 Evaluation of the time similarity The next step in the analysis is to compare the three studied products (SAFRAN, Spain02 and ERA-Interim) to a common set of observations considering the same period (1980 to 2010). Figure 3 shows the correlation (CORR), relative mean absolute error (MAEr) and ROC skill area (RSA) between the three products and the observations. The SAFRAN and Spain02 correlations are very similar, and the main difference is located on the southeastern coast where SAFRAN has larger correlations. This similarity is also reflected in the statistics in Table 3, with both products having the same mean annual correlation (0.82) and an almost identical interquartile range (approximately 0.1). As expected, the ERA-Interim correlation is lower. In this case, there is a strong east–west gradient with the correlations higher in the Mediterranean due to the binary component (rain or no rain) of the precipitation, which is moderately well represented in www.hydrol-earth-syst-sci.net/21/2187/2017/ Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 2194 P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim 0 2 4 6 8 10 12 0 5 10 15 20 25 30 35 Month mm d-1 0 2 4 6 8 10 12 8 10 12 14 16 18 20 22 24 Month mm d-1 0 2 4 6 8 10 12 0 5 10 15 20 25 30 Month mm d-1 0 2 4 6 8 10 12 10 15 20 25 Month mm d-1 0 2 4 6 8 10 12 0 5 10 15 20 25 30 Month mm d-1 0 2 4 6 8 10 12 0 5 10 15 20 25 Month mm d-1 0 2 4 6 8 10 12 5 10 15 20 25 30 35 40 Month mm d-1 0 2 4 6 8 10 12 10 15 20 25 30 35 40 Month mm d-1 0 2 4 6 8 10 12 0 5 10 15 20 25 Month mm d-1 0 2 4 6 8 10 12 0 5 10 15 20 25 30 35 Month mm d-1 0 2 4 6 8 10 12 0 5 10 15 20 25 30 Month mm d-1 (a) North (b) Duero (c) Tajo (d) Guadiana (e) Guadalquivir (f) Southeast (g) Segura (h) East (i) Ebro (j) Northeast (k) Balearic Islands STATION SAFRAN SPAIN20 ERA−INTERIM Figure 4. The annual cycle of maximum precipitation in 1 day (RX1D) for the Spanish river basins. The yaxis range is different for each figure to highlight the differences between the datasets. the reanalysis. Precipitation is a variable with many occurrences of zero, which may suggest the use of the Spearman’s rank correlation coefficient instead of Pearson’s. In order to verify that both measures give the same qualitative information, we calculated the Spearman’s rank correlation coefficient and found that it is lower for both SAFRAN (0.73) and Spain02 (0.75). The corresponding interquantile ranges are 0.68 to 0.78 for SAFRAN and 0.69 to 0.80 for Spain02. Therefore, all results are shifted when the Spearman’s coefficient is used instead of Pearson’s. Notwithstanding, the width of the interquantile range remains almost the same for SAFRAN, and it is narrower by only 0.01 for Spain02; thus, we conclude that both Spearman’s and Pearson’s provide the same outlook when used to assess the relative performance of either product with respect to the observations. The results obtained with the correlation are, in general, in agreement with those obtained for the RSA. However, for the latter score, Spain02 shows a more homogeneous spatial pattern with similar values on the southeastern coast and in the rest of the area. For ERA-Interim, the spatial pattern shown in the correlation is extended for the RSA, with an improvement in the Ebro valley and on the southeastern coast. Concerning the MAEr, again SAFRAN and Spain02 show very similar results, as Table 3 corroborates. Spain02’s larger number of errors are mainly concentrated in the southeast, as seen in the correlation, whereas SAFRAN shows a more random spatial pattern. In the case of Spain02, the affected region (close to Murcia) has the highest station density for the entire peninsula, so the interpolation process obtains an areally averaged precipitation value that differs from the local observations of each particular station. This is more relevant when intense local precipitation events occur, which are very frequent in the Mediterranean region, leading to an overestimation of the interpolated values in the surrounding Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017 www.hydrol-earth-syst-sci.net/21/2187/2017/ P. Quintana-Seguí et al.: Validation of SAFRAN precipitation and comparisons to Spain02 and ERA-Interim 2195 ERA-Interim Stations SAFRAN Spain02 M = −0.16 S = 0.81 R = 0.80 C = 0.63 PRCPTOT (mm) RX1D (mm) CWD (days) M = 0.01 S = 1.00 R = 0.39 C = 0.92 M = 0.04 S = 0.97 R = 0.34 C = 0.94 M = 0.26 S = 1.08 R = 0.89 C = 0.64 M = 0.44 S = 1.47 R = 0.75 C = 0.88 M = 0.27 S = 1.27 R = 0.55 C = 0.91 M =−0.29 S = 0.26 R = 1.04 C = −0.02 M = −0.18 S = 0.96 R = 0.59 C = 0.82 M = −0.15 S = 0.78 R = 0.46 C = 0.90 500 1000 1500 2000 5 10 15 20 50 100 150 (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) Figure 5. The PRCPTOT (a, d, g, j), CWD (b, e, h, k) and RX1D (c, f, i, l) for the stations (a–c), ERA-Interim (d–f), SAFRAN (g–i) and Spain02 (j–l). area. The more spatially distributed nature of SAFRAN’s errors points to a close relation with the position of the zone borders, which produces a homogeneous spatial distribution over the study area. ERA-Interim’s errors are more numerous and quite spatially homogeneous, with the exception of the Mediterranean coast and an area located in the Basque Country (on the eastern side of the Bay of Biscay). The box plots shown in Fig. 3j and l corroborate the similarity between SAFRAN and Spain02, not only in terms of the mean but also in the spatial variability of the scores. They also corroborate the worse results and the higher spatial variability in terms of the box altitude for ERA-Interim. Note that there is remarkable agreement between SAFRAN and Spain02 on all scores related to the binary sequence. In addition to the correlation, this reflects the very similar temporal structure of both datasets. Table 3 also shows the results by season, clearly indicating that the seasonal behavior of SAFRAN and Spain02 is the same. They are both very robust across seasons with the exception of summer when the scores are degraded, especially for CORR and MAEr. Both SAFRAN and Spain02 (the AA-3D version used in this paper) consider the relief in their algorithms, and the same statistics were thus calculated for a subset of highaltitude stations located at 1000 m or higher. The results, which are shown in Table 3, do not reflect significant differences in terms of the correlation and MAEr between high- and low-altitude stations. Both SAFRAN and Spain02 perform equally well with only some slight differences shown for the RSA, which are more relevant for ERA-Interim. The trend in the errors with altitude for all stations was calculated, and it was not significant in any case, even for ERA-Interim. However, this result does not account for the fact that highaltitude areas are not well represented by the observations because there is a lack of stations at the highest elevations. Evaluation of the spatial similarity by means of precipitation indices In this section, we evaluate how the different analyzed products are able to reproduce a standard set of precipitation indices. First, the annual cycles of the monthly mean and the daily maximum precipitation are studied for the 11 river basins shown in Fig. 1. The global statistics of each index are then summarized and analyzed. Figure 4 shows the annual cycle of the maximum daily precipitation (RX1D) for each of the main Spanish river basins. A first look at the figure shows that, as expected, all www.hydrol-earth-syst-sci.net/21/2187/2017/ Hydrol. Earth Syst. Sci., 21, 2187–2201, 2017