Driving licenses based on points systems: Efficient road safety strategy or latest fashion in global transport policy? A worldwide meta-analysis

Driving licenses based on points systems: Efficient road safety strategy or latest fashion in global transport policy? A worldwide meta-analysis Jose ´I. Castillo-Manzano n , Mercedes Castro-Nun ˜o Facultad de Ciencias Econo ´micas y Empresariales, University of Seville (Spain), Avda. Ramo ´n y Cajal, 1, 41018 Seville, Spain article info Keywords: Road safety Driving license Point-record systems Meta-analysis approach abstract One of the most popular coactive measures developed to prevent road traffic accidents in recent decades is the implementation of driving licenses based on points systems (PS) which penalize repeat offenders with suspension or withdrawal of their licenses. This paper analyzes their rapid spread worldwide through an in-depth review of the existing literature. A comprehensive meta-analysis of the effects of PS on road traffic accidents and the duration of these effects has been conducted. The findings show that the strong initial positive impact (15 to 20% reductions in accidents, fatalities and injuries) seems to wear off in under eighteen months. This limited effectiveness is related to the absence of complementary enforcement to back up these measures. Without them, points systems could turn into a boomerang road safety policy, and even be abandoned at a later date. The implications of the conclusions for legislation and future research are considered. &2012 Elsevier Ltd. All rights reserved. 1. Introduction The extraordinary economic growth experienced throughout the 20th century has raised rates of motor vehicle ownership all around the world while, at the same time, road traffic accidents (RTA) have become one of the lethal epidemics affecting humankind, with 1.2 million people killed and up to 50 million more injured every year (WHO, 2009). This is, however, a silent killer which for years has been the object of much less media attention than other, less everyday but more eye-catching tragedies, such as air accidents or natural disasters. According to the same source, unless a preventative strategy is adopted on a global scale, RTA will become the fifth leading cause of death worldwide by 2030. In many countries, the implementation of increasingly severe road safety policies, together with the fact that the current economic crisis has reduced traffic volumes (at least in the short term), have produced a large fall in the main traffic accident indicators during the first decade of the 21st century (see OECD, 2010a,2010b). However, this improvement has not been uniformly distributed in geographical terms, with huge differences between high-income countries and middleincome/low-income countries (IRF, 2009). As they have achieved a more-or-less tested minimum degree of effectiveness (WHO, 2009), the majority of the interventions in the developed countries have been progressively imitated in developing countries. We found basically two groups of actions: firstly, strategies that focus on elements that are external to the road user (such as infrastructure improvements—analyzed by A ´lvarez et al., 2007 or safer vehicle design—studied by Christensen and Elvik, 2007); and secondly, strategies that focus on preventing or correcting unsafe road-user behaviors, based on the evidence that the human element is the factor that explains the majority of accidents (Stanton and Salmon, 2009). This last block of actions, which has been greatly developed in the academic literature, can in turn be classified as: preventive measures that seek to persuade or dissuade through education and information (such as communication campaigns and road safety advertisements, considered by Castillo-Manzano et al., 2012 and Lewis et al., 2008); or corrective or sanctioning measures (monetary and non-monetary) which try to internalize the social costs of RTA through coactive dissuasion (Castillo-Manzano et al., 2011). Apart from the application of severe criminal penalties by Law (judicial sentences and imprisonment) to punish the severest infringements, other softer regulations for minor infringements based on economic sanctions (by fines and insurance payments) and license deprivation (by suspension or withdrawal) also stand out (Bourgeon and Picard, 2007). In road traffic legislation in many countries, license deprivations have evolved into a type of driving license based on a points systems (henceforth PS) which, despite differences between countries, respond to a common philosophy that can be summarized following Nole ´nandO ¨stlin, (2008): several specific offenses committed by drivers cause the addition/loss of a certain number of points and when the number of points reaches the maximum allowed or all points are lost, the driver’s license is suspended. Contents lists available at SciVerse ScienceDirect journal homepage: www.elsevier.com/locate/tranpol Transport Policy 0967-070X/$ - see front matter &2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.tranpol.2012.02.003 n Corresponding author. Tel.: þ34 954 556727; fax: þ34 954 557629. E-mail addresses: [email protected] (J.I. Castillo-Manzano), [email protected] (M. Castro-Nun ˜o). Transport Policy 21 (2012) 191–201 Despite their widespread use in practice, scientific studies on the effectiveness of PS are not developed methodologically in the academic literature to the same degree, and there is a certain amount of controversy regarding the duration of their effects (Castillo-Manzano et al., 2010). There are some studies that are more optimistic about the impact and sustainability of the measure’s positive effects over time, including contributions to the success of other road safety measures (such as Zambon, et al. 2008 which attribute to it a major increase in the wearing of seatbelts); while other authors, such as Montag (2010), claim that its impact is only short-term and that, paradoxically, there might even be some perverse effects, either because traffic law enforcement produces more risk-taking behaviors, or because there is a decrease in the intensity of police activities due to trust in the deterrent effect of the system on its own. 1 In this line, PS can encourage undesirable behaviors, such as hit-and-run crashes (SWOV, 2010) and/or driving without a valid license (Knox, et al., 2003). In fact, following Engstr¨ om et al., (2003), very little is known about the effects of the PS on road safety, because as SWOV (2010) states, it is very difficult to isolate the impact that these systems have from the effects of other complementary types of enforcement being applied at the same time (i.e., an increase in police or law enforcement, public campaigns). The aim of this paper is to discuss some features of the pointsrecord mechanism, while simultaneously developing an empirical framework to investigate in more detail if this really is an efficient global strategy for improving road safety or simply a fashion policy which is imitated from one country to another. We use a meta-analysis approach to investigate the effectiveness and short- and long-term effects of this system, developed in the framework of a fixed-effects model and a random-effects model. Rather than just a simple state-of-the-art literature review, we select academic research from a wide search of studies developed around the world considered as relevant on the basis of the methodologies used and empirical findings, and we summarize a result. This understanding is considered to be especially useful for future expansion of the policy to developing or undeveloped countries as it provides the keys required for its optimum design. The paper is organized as follows: following this introduction, Section 2 examines driving licenses based on PS as an international strategy for reducing traffic accidents. In Section 3,weexplainthe design of the meta-analysis model applied, and discuss the findings. In Section 4 the main conclusions are drawn and the implications for road safety that derive from this research are set out. 2. Points systems to improve road safety Driving licenses based on a points-record have their predecessors in the license endorsement systems that were introduced in the nineteen-thirties (Milulik, 2007). They have currently been extrapolated to other areas of public politics not only related to the transportation sector (for infractions in fisheries activity in the European Union; for illegal immigration in countries such as Canada, Australia and New Zealand; for strict codes of discipline at high schools in Hong Kong and India). In the present day, this road safety strategy has been particularly recommended by international institutions because of its potential, at least in the initial period after its introduction, (see Global Road Safety Partnership, 2008;SWOV, 2010). It has progressively gained popularity and acceptance in public opinion worldwide (Nole ´n and O ¨stlin, 2008) because it is considered to be more just when dealing more strictly with repeat offenders and fairer than a monetary fine, (the real effect of which depends on the offender’s purchasing power, ETSC, 2008). According to Elvik and Vaa (2004), this measure forms part of a wider road safety strategy that includes two other enforcement measures: warning letters 2 and driving license revocations. It is also an umbrella strategy, as it covers all types of traffic violations compared to other measures included in the general legislative system which are exclusively related to one specific offense area (speeding, alcohol). A review of the PS in different countries reveals that, although their structures are all similar, there are differences in their designs and application (Nole ´n and O ¨stlin, 2008). From their analysis it can be concluded that they vary considerably in the selection of offenses and their scoring, the ways of counting points, the thresholds for disqualification or duration of disqualification and administration procedures. Even in the same country differences can be found between states as, in certain countries, these are systems that are transferred to the authorities of the various jurisdictions. This is the case of the United States, Australia and Canada. Furthermore, the punishment applied by PS to a traffic violation may not be standardized, as in countries such as Ireland and the United Kingdom the number of points can rise for the same violation depending on whether the driver is found guilty in court. The penalization is therefore adjusted for each violation depending on its seriousness or whether it is a repetition of the same violation (ETSC, 2008). 3 There are two equivalent variations with different mechanisms used to compute the points that penalize offenses (see, Nole ´n and O ¨stlin, 2008): (1) Penalty PS (PPS) or point deduction: all drivers who possess a valid license are allocated an initial number of points from which points are deducted depending on the offense committed. Once this initial credit runs out the license is withdrawn. ‘‘The point is to gain no point’’ (ETSC, 2008); (2) Demerit PS (DPS) or point accumulation: all drivers who possess a valid license start from scratch, i.e., no points, and accumulate points, depending on the offense committed. The license is withdrawn after a certain number of points are accumulated. In both cases, the deduction or apportioning of points is of a personal nature, that is, it is directly linked to the driving license owned by the offender so that when she/he is caught by the police committing a traffic violation the points cannot be used against the vehicle owner via the vehicle registration number (SWOV, 2010). Similarly, in both of the versions, all drivers start in the same situation (maximum points/ zero points), although in some countries like Latvia or the United Kingdom, differential systems are in place for groups of drivers with special circumstances, such as novice drivers (with a stricter system) 4 ; or professional drivers (with a more permissive system). According to Bourgeon and Picard (2007), in some cases such as France or Spain, ‘‘redemptive systems’’ are applied that are not 1 According to Dionne et al., (2011),‘‘ythe police are the most important enforcers of the different incentive schemes’’. 2 Generally-speaking, ‘‘warning letters’’ are an incentive system used to modify negligent driving behavior in road safety (Wilde and Murdoch, 1982) when a driver commits a certain traffic offense (e.g., exceeding the speed limit, or in photo radar enforcement). However, in some countries or US states with points systems, like France, Michigan (U.S.), Ontario (Canada) and Spain, warning letters are also used as ‘‘advisory written warnings’’ which are sent to drivers who are beginning to approach the limit for license withdrawal (see, for example Castillo- Manzano et al., 2010 and Haque, 1990). 3 This modality, which consists of graduating the penalization applied to each violation depending on its seriousness, is not entirely sanctioned by experts such as Elvik and Vaa (2004), who believe it is not very reliable due to its subjectivity. 4 PS can be specifically targeted at novice drivers (Netherlands and Finland) or under Gradual Licenses (GDL) and probationary systems, which consist of traffic violations during an intermediate stage of licensing possibly delaying the novice driver attaining a full license (e.g., a new Graduated Demerit Point System for novice drivers in Australia, since December, 2010). J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201192 only of a punitive nature but also have a rehabilitative aspect to them. In these systems the driver who has been penalized can have the points she/he has accumulated and removed, or recover those that have been lost, once a certain period of time has passed with no further offenses being committed, after attending a reeducation and road-safety awareness course self-financed by the offender, and even after retaking the driving test to recover the license. On occasion, extra points are added or deducted from the licenses of good drivers who do not commit any offense during a certain period of time (Nole ´nandO ¨stlin, 2008); this is the case in Malaysia, where the so-called KEJARA System is applied, and Spain. In jurisdictions such as the state of New South Wales (Australia), the ‘‘double penalty points’’ system is used to increase the level of penalization for certain offenses (such as speeding) during holiday periods (higher points during high-risk periods). In some states in the US and European countries like Germany and Ireland, PS can also produce other external effects: insurance companies require drivers’ points-records to be reported to them to determine whether or not to renew an insurance policy (ETSC, 2008 is analyzing the topic). This is an attempt to create a stronger incentive for reckless drivers who do not respond to that of insurance pricing (Dionne et al., 2011). Irrespective of the particular features of the PS variant used, Basili and Nicita (2005) highlight the fact that the measure also aspires to trigger a number of favorable effects in driver behavior: 1. Deterrence or the ability to make offenders associate repeat traffic violations with more severe sanctions to the point that they lose their licenses (Zambon et al., 2008). It could be said that the repeated violation of traffic regulations is producing a growing marginal cost for the offender; 2. Selection or removal of the mass of drivers who repeatedly violate traffic regulations with the objective of reducing the likelihood of traffic accidents (Diamantopoulou et al., 1997); 3. Correction or incentive for drivers to rectify their inappropriate behavior at the wheel (Poli de Figueiredo et al., 2001); 4. Education or Prevention, as drivers are shown which are the most serious punishable violations and most dangerous behaviors whilst (in some countries) it is at the same time established that courses on road safety have to be attended (Roca and Tortosa, 2008). In Table 1, we use the aggregation of countries in regions proposed by WHO (2009) as an initial study criterion to distinguish between countries with PS (44) whilst also indicating which variant has been implemented, PPS or DPS. The third column of this Table also provides a sample of countries from each WHO region for which some kind of study has been found on the effects of the PS, irrespective of the methodology used for its measurement, and together with its implementation date. Detailed analysis of Table 1 has allowed a series of findings to be determined:  The areas where the development of the PS has been most widespread can be seen to be those which include more highincome countries (HIC), especially in the European Region. This suggests that countries need to have the appropriate technical means to implement the system, as well as a certain degree of road safety culture (as described by Dodge and Kitchin, 2007). Furthermore, the system that is most adopted in the majority of the countries considered is the DPS method, i.e., the original model.  Twenty-four countries have opted for the system in the European Region. This measure has become a generalized standard for road safety in western countries, whereas its application in Eastern European countries has been much more limited and mostly confined to countries that are in the EU, such as the Czech Republic or Latvia, amongst others, and also Turkey, which has applied to join. It seems clear that belonging to the EU and its area of influence has been the thread that has led countries to imitate one another. Indeed, it could be said that after the European Commission set a target of a 50% reduction in traffic fatalities between 2000 and 2010 (European Commission, 2001), it was not so much an imitation effect that could be spoken of among the group of measures that European countries adopted, but rather a contagion effect. 5 There are also some EU countries where driving licenses are not in fact based on the PS, although they do bear many similarities. One example is Portugal, where a driving license can be withdrawn if a court decides the offender has been guilty of very serious offenses three times, or of serious offenses five times, in three years, and, furthermore, drivers must retake their driving tests to get their licenses back.  Chronologically, after its pioneering introduction in the US over fifty years ago, the system spread to Asia and Australia and thereafter to a European country (Germany). It has only really expanded and been perfected in recent times as it was in the last decade that the greatest number of changes and converts were seen, especially in Europe. The latest regions to introduce PS have been Africa and the Eastern Mediterranean.  Finally, we find cultural similarities in the PS variant adopted. After first being introduced in the US, the DP system has been replicated in other Anglo–Saxon countries irrespective of the WHO region to which they belong (Australia, New Zealand, South Africa, Canada, UK, and Ireland or their former colonies, such as Hong Kong). In more recent times, it has been the PPS that has been implemented in Latin European countries (France, Italy and Spain), and of late in these countries’ former American colonies, such as Ecuador and Argentina, although not in colonies that come under the cultural influence of the US, like Mexico and Panama, which use the DPS. Researchers such as Priya and Uteng (2009) find correlations between cultural elements and accidents on the basis of lifestyle and socioeconomic aspects. These studies would uphold the idea that the international spread of this road safety strategy is not only explained by the imitation effect that arises between countries because of their geographical proximity, but also because of cultural issues, as it would appear that the variant that is applied is closely linked to the peculiarities of each country. 3. Meta-analysis of effectiveness of point systems mechanisms One of the objectives of this study is to synthesize a series of results on the size and duration of PS effects worldwide from previous studies that have used various methodologies. The metaanalysis approach has advantages over a literature review for this because of its statistical nature and its inference process (Glass et al., 1981). This approach has been broadly developed over the last 20 years, basically in experimental sciences and environmental economics, although a variety of precedents can also be found of its being applied to a range of road safety strategies in recent years (see for example Erke, 2009, for red-light cameras; Høye, 2010, for front airbags; or Phillips, et al., 2011, for advertising campaigns). As far as PS are concerned, there is a precedent in Elvik et al. (2009),who synthesized a number of specific studies related to DPS-linked elements and license suspensions (warning letters, special driving tests and improvement courses). 5 As has been seen from the measures that have been progressively applied by the various countries, according to OECD, (IRTAD) (2010). J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201 193 3.1. Study design: data coding and variables An initial search was conducted in the academic literature to identify the most recent research on the topic of the effects of PS around the world, with the constraint of the publication form (articles in peer reviewed scientific journals or institutional reports). 6 Using general search terms in order that no study should be excluded apriori, these studies have been found in the following sources: the reputed Transportation Research Information Services (TRIS) database (U.S. Department of Transportation), PubMed (online database of the U.S. National Library of Medicine), the ISI Web of Knowledge, Scopus and Science direct (Elseviers online databases), reports from research institutes such as SWOV (The Netherlands) and TRL (UK) and Internet searches by Google Scholar. In this way almost fifty studies with these characteristics were located although they are all greatly heterogeneous, especially with regard to their methodologies. Some are eminently theoretical (such as Basili and Nicita, 2005 and Bourgeon and Picard, 2007), whilst others are based on a comparison of before-and-after statistics (Poli de Figuereido et al., 2001). Only a sample of suitable studies was selected, using typical inclusion criteria in meta-analysis (see Elvik, 1999), i.e., those which: 1. Include an estimation of PS effectiveness and/or duration using statistical or econometric methodology that seeks to separate out the effects of PS from other variables that might have had an effect during the period analyzed (both meta-analyses and studies that only evaluate elements such as warning letters and reeducation courses were rejected since, as was indicated in Section 2, these are not applied equally in all countries); 2. Report the standard deviation, confidence interval, sample size or similar statistical information about the accuracy and robustness of the results. The initial data sample was refined by the application of these quality indications and a total of 26 studies from around the world were finally included in the meta-analysis, many of them with multiple relevant results which could be used to expand the final sample. This suitable method of data collection was characterized by a coding procedure, as seen in Tables 2–5. Following the methodology described in Section 3.2, below, two scenarios were envisaged for the meta-analysis: SCENARIO (I)Estimation of summary effects of PS: To minimize the likely heterogeneity among the 26 suitable studies, these were classified into 3 subgroups depending on the type of variables that were considered for evaluating the effects of the PS. As there are studies with multiple results, the sample size will be the total number of results computed 7 :  I VTRA: Variables related directly to RTA, in terms of reductions in the number of accidents, casualties, fatalities or injured. These variables were found in 13 studies with 24 results (m¼24). Table 1 Countries with points systems in the who regions. Source: Authors. based on WHO (2009), ETSC (2008) and road traffic legislation in each country. WHO region No. of countries/ income level Countries with a points system driving license (DPS or PPS) such asySelected countries/year points system implemented 141 (0 HIC, 11 MIC, 30 LIC) South Africa (DPS) South Africa (in Pretoria, 2006) 232 (6 HIC, 26 MIC, 0 LIC) Argentina (PPS), Bermuda (DPS), Brazil (DPS), Canada (DPS), Ecuador (PPS), Jamaica (DPS), Mexico (DPS), Panama (DPS), Peru (DPS), United States of America (DPS) – USA (state by state, i.e. Connecticut, 1957) – Canada (state by State, i.e. Quebec, 1978) – Brazil (1998) 310 (0 HIC, 6 MIC, 4 LIC) – – 420 (5 HIC, 12 MIC, 3 LIC) Qatar (DPS), The United Arab Emirates (DPS) The United Arab Emirates (2008) 549 (25 HIC, 21 MIC, 3 LIC) Austria (DPS), Bulgaria (DPS), Cyprus (DPS), The Czech Republic (DPS), Denmark (DPS), France (PPS), Germany (DPS), Greece (PPS), Hungary (DPS), Iceland (DPS), Ireland (DPS), Israel (DPS), Italy (PPS), Latvia (DPS), Lithuania (DPS), Luxembourg (DPS), Malta (DPS), Norway (DPS), Poland (DPS), Romania (DPS), Slovenia (DPS), Spain (PPS), Turkey (DPS), United Kingdom (DPS) – Germany (1974) – United Kingdom (1988) – France (1992) – Finland (1996) a – Ireland (2002) – The Netherlands (2002) a – Italy (2003) – Austria (2005) – Spain (2006) – The Czech Republic (2006) 626 (6 HIC, 15 MIC, 5 LIC) Australia (DPS), China (DPS), Japan (DPS), Malaysia (DPS), New Zealand (DPS), Republic of Korea (DPS), Singapore (DPS), – New Zealand (1967) – Japan (1968) – Australia (state by state, i.e. Victoria, 1970) – China (jurisdiction by jurisdiction, i.e. Hong Kong, 1984) Notes: DPS: Demerit point system, PPS: Penalty point system. HIC: High-income countries; MIC: Middle-income countries; LIC: Low-income countries. 1. African Region; 2. Region of the Americas; 3. South-East Asia Region; 4. Eastern-Mediterranean Region; 5. European Region; 6. Western Pacific Region. a Only Points Systems for novice drivers on probation. 6 In order to reduce any possible bias, sources and bibliography were chosen blindly by the authors of the study. 7 Following Elvik and Vaa, (2004) we assume the independence hypothesis between multiple results. J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201194  I VDB: Variables related to driver behavior, in terms of reductions in risky or dangerous conduct, driving less unsafely (a smaller percentage of drivers not wearing seatbelts, for example) or a reduction in law violations (speed limits, drinkdriving). These variables were found in 8 studies with 18 results (m¼18).  I VHCD: Variables related to health care data, in terms of reductions in RTA related A&E admissions, surgery or hospitalizations. These variables were found in 5 studies with 8 results (m¼8). SCENARIO (II)Estimation of the duration of the PS effect: The variable considered in this case (LENGTH OF EFFECTS) is the duration of PS effects, in number of months, estimated for each of the studies. As can be seen in Tables 2–5, not all the suitable studies analyze this issue. To be precise, we shall work with the sample of 20 studies that do (m¼20). 3.2. Methodological aspects. The estimation method used was the ‘‘log-odds model’’ found in Elvik and Vaa (2004). In this method, the estimations of the suitable studies are combined to obtain a summary-measure using mean weighting according to the statistical weight (the inverse of the precision of the study). According to the following formula, the mean weighted effect for ‘‘m’’ estimations ðYÞwill be: Y¼exp P m i¼1 w i y i P m i¼1 w i  ð1Þ When: y i ¼log (i-th estimation obtained by each study). w i ¼the statistical weight of the i-th estimation obtained as w i ¼ð1=v i Þ,when v i is the variance of each i-th estimation, whereby P m i¼1 w i ¼1: The possible heterogeneity of the studies can be taken into account using the random-effects model (REM), or not included if the fixed-effects model (FEM) is used. The FEM assumes that there is one single effect on the population and does not take into account the variability of the results among the different studies. The size of the study and its own variance (intra-study variability) are the only determinants of its weight in the meta-analysis. The Dersimonian and Laird Q statistic (1986) is calculated to verify the validity of this hypothesis, distributed in accordance with a Chi- Square with m-1 degrees of freedom (m¼no. of combined estimations): Q¼X m i¼1 w i y 2 i ðP m i¼1 w i y i Þ 2 P m i¼1 w i ð2Þ If the null hypothesis is rejected (H 0 : non heterogeneity) it is recommended that REM be used as this does take into account Table 2 Selected studies in the Americas region: coded data. Source: Authors. Country/ income level Researcher/place Proxy of effects Methodology Results TIME series studied/–time– span of effects Brazil (MIC) Liberatti et al. (2001)/ (city of Londrina) VTRA Before/ After analysis using Chi-squared and Fisher’s tests. A reduction of 20% of car and 9.1% of motorcycle accident victims. An increase of 39% in use of safety belts and of 112% in use of helmet. (1997–1998)/6 monthsVBD Maffei de Andrade et al. (2008)/(city of Londrina) VTRA Linear regression. A 28.4% greater reduction in mortality levels than under other previous measures. (2000–2005)/12 months Canada (HIC) Chen et al. (1995)/ (British Columbia) VTRA Logistic regression. The results showed a consistent increase in post period accidents per driver with increasing pre period number of crashes. (1985–1990)/24 months Dionne et al. (2011)/ (Quebec) VBD Proportional Hazards Model. The frequency of traffic violations was reduced by 15% and the risk of a RTA after a traffic offense was 20% lower. (1985–1996)/12 months Redelmeier et al. (2003)/(Ontario) VBD Case-crossover study The risk of a fatal crash in the month after a conviction was about 35% lower. (1988–1999)/2 months United States of America (HIC) Gebers and Peck (2003)/(California) VBD Both multiple linear regression analysis and canonical correlation. An improvement of 14.9% in classification for predicting and identifying accident-involved drivers. (1992)/No quantified period of effects HIC: High-income country; MIC: Middle-income country; LIC: Low-income country. VTRA: Variables related directly to RTA, in terms of reductions in the number of accidents, casualties, fatalities or injured; VBD: Variables related to driver behavior, in terms of reductions in risky or dangerous conduct, driving less unsafely or a reduction in law violations; VHCD: Variables related to health care data, in terms of reductions in RTA-related A&E admissions, surgery or hospitalizations. Table 3 Selected studies in the Eastern-Mediterranean region: coded data. Source: Authors. Country/ income level Researcher/ place Proxy of effects Methodology Results Time series studied/time-span of effects The United Arab Emirates (HIC) Mehmood (2010)/(Al Ain) VBD Independent sample t-test using SPSS. The DPS has statistically no significant impact on the speeding behavior of drivers because of the lack of effective traffic monitoring system. (3 months before/3 months after introduction)/No quantified period of effects HIC: High-income countries; MIC: Middle-income countries; LIC: Low-income countries. VTRA: Variables related directly to RTA, in terms of reductions in the number of accidents, casualties, fatalities or injured; VBD: Variables related to driver behavior, in terms of reductions in risky or dangerous conduct, driving less unsafely or a reduction in law violations; VHCD: Variables related to health care data, in terms of reductions in RTA-related A&E admissions, surgery or hospitalizations. J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201 195 possible heterogeneity by considering that the effects of exposure/intervention on the population are varied and that the effects analyzed by the included studies are only a random sample of all possible effects. Weighting studies with REM considers not only their own variance (intra-study variability), but also the variance that might exist between studies (interstudy variability): s y¼½Q2ðm1Þ cð3Þ where: Q¼Dersimonian and Laird statistic. m¼no. of estimations considered. c¼estimator calculated as c¼P m i¼1 w i ½P m i¼1 w 2 i =P m i¼1 w i whereby the variance and the statistical weight of each i-th estimation would be, respectively v n i ¼ s yþv i and w n i ¼1=v n i In both cases (FEM and REM), the upper/lower limits of the confidence intervals (CI) (in our case, at 95%), are calculated by the formula (4): CI ¼exp Y71:96 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pstatistical weights p ! ð4Þ Any possible publication biases can be identified analytically using the Begg (Begg and Mazumbar, 1994) and Egger (Egger et al., 1997) tests, which test the null hypothesis for the lack of said bias. Table 4 Selected studies in the European region: coded data. Source: Authors. Country/income level Researcher/ place Proxy of effects Methodology Results Time series studied/– time–span of effects Czech Republic, The (HIC) Montag (2010)/ VTRA Standard differences-in- differences methodology. Fatalities were about 30% lower during first three months after the law was introduced. (2004–2008)/3 months Ireland (HIC) Butler et al. (2006) VHCD Statistical analysis using a two-factor ANOVA and Chi-squared test. In the first 6 months, there was a significant reduction (48.4%) in RTA-related spinal admissions. In the first year, there was a 25% reduction in deaths. (1998–2004)/6 months Healy et al. (2004) VHCD Statistical analysis with a Chi-squared test. The number of RTA- related admissions fell to 17 compared to an average of 33 in the preceding 4 years. (1998–2003)/6 months Hussain et al. (2006) VHCD Retrospective cohort study. Sixty-one percent reduction in operations on injuries caused by collisions. (2001–2003)/12 months Saeed et al. (2010) VHCD Statistical analysis was performed using the Chisquared test. After the offense of not wearing a seatbelt was added to the DPS in 2003, there was a 60% decline in the proportion of admissions for RTA-related eye-injuries. (2001–2007)/No quantified period of effects Italy (HIC) Benedettini and Nicita (2009) VTRA 3SLS regression and Poisson Regressions. A strong ‘announcement effect’ two years before the implementation of the PPS. A decrease of 72.87% in speeding tickets, of 17.80% in total RTA and 26.42% in fatal RTA. (2001–2008)/24 months before introductionVBD De Paola et al. (2010) VTRA Regression discontinuity design. A reduction of about 10% in RTA and of about 25% and 15% in traffic fatalities and injuries, respectively. (2001–2005)/No quantified period of effects Farchi et al. (2007)/(The Lazio Region) VHCD Poisson models. Twelve percent fewer RTA A&E visits; hospitalizations decreased by 16% and RTA-related deaths decreased by 4%. (2001–2004)/6 months Zambon et al. (2007)/ (Veneto Region) VBD Time-series analysis with ARIMA model. An increase in seatbelt use of 51.8% among drivers, of 42.3% among front passengers and of 120.7% among rear passengers. The number of RTA fatalities and injuries declined, with a reduction of 18% and 19% respectively. (1999–2004)/18 months VTRA Zambon et al. (2008)/ (Veneto Region) VBD Poisson models. An increase in seatbelt use of 83% for drivers and 76% for passengers. (2003–2005)/15 months Spain (HIC) Aparicio- Izquierdo et al. (2011) VTRA Time-series analysis with ARIMA models. A mean reduction in fatalities of between 11.27% and 13.9%. (1995–2009)/Effects maintained throughout the study period Castillo- Manzano et al. (2010) VTRA Time-series analysis with unobserved components model. An average reduction of 12.6% in the number of deaths in RTA. (1980–2007)/24 months (for deaths) and 12 (for injured) Novoa et al. (2010) VTRA Quasi–Poisson regression models. The overall number of people injured showed a reduction of 5% and 4% among men and women, respectively, and an 11% and 12% reduction for seriously injured people. (2000–2007)/18 months Pulido et al. (2010) VTRA Time-series analysis through ARIMA models. A reduction of 14.5% in overall fatalities. (2000–2007)/18 months United Kingdom (HIC) Simpson et al. (2002) VBD Questionnaire survey and empirical analysis using Chi-squared test. The results of a new PPS for novice drivers led to a decrease in the percentage of drivers (5.8 to 4.5) offending in their 2nd year of post-test driving. (1992-94; 1997–1998)/ 24 months HIC: High-income countries; MIC: Middle-income countries; LIC: Low-income countries. VTRA: Variables related directly to RTA, in terms of reductions in the number of accidents, casualties, fatalities or injured; VBD: Variables related to driver behavior, in terms of reductions in risky or dangerous conduct, driving less unsafely or a reduction in law violations; VHCD: Variables related to health care data, in terms of reductions in RTA-related A&E admissions, surgery or hospitalizations. J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201196 This analysis must be complemented with an interpretation of socalled funnel plots diagrams to avoid any false positives or false negatives due to the loss of power of detection that these tests can be subject to in small samples (Sterne et al., 2000). A funnel plot is a tool used to compare the estimation in a graph using some precision measure, such as a standard error function, for example. If the graph is symmetrical, in an inverted ‘‘V’’ shape, it is interpreted as proof of there probably being no publication bias. If the graph is asymmetric, it is interpreted as publication bias probably existing. 3.3. Results Table 6 shows (in columns) the main results obtained for the 4 meta-analyses formulated in all the scenarios (in rows), using a total sample of m¼70 individual estimations for 26 suitable studies. In order to analyze the heterogeneity of the sample the values are taken for the Dersimonian and Laird Q (1986) statistics and the Takkouche et al. (1999) R I coefficient (¼proportion of the total variance due to between-study variance). The pooled odds ratios obtained (OR) for each scenario with FEM and REM, respectively (with the confidence interval limits) are set alongside the Begg and Egger test p-values enabling the presence of publication biases that alter the final result to be identified. All the Q statistics are significant. This would, in principle, allow the homogeneity null hypothesis to be rejected at 99% in all cases. Following Takkouche et al. (1999), given that in the three analyses done for scenario I, R I is clearly greater than 0.75, we understand that the Dersimonian and Laird Q statistic is sufficiently powerful to detect heterogeneity, overcoming the limitations that this statistic presents for small samples (see Fleiss, 1993). Therefore, for scenario I, the OR estimated by REM is more reliable (which is why columns 7and8ofTable 6 with the REM results have been highlighted). This presence of random elements means it can be deduced that the results of the intervention analyzed are in reality varied, and that the estimations offered by the various studies included in our metaanalysis are only a random sample of possible effects. The differences from one country to another would be based on the countries’ own peculiar features, such as the advertising campaigns or the other more forceful enforcement measures (fixed speed cameras, lower blood alcohol limits, fines and heavier legal sanctions, policing) that accompany the application of PS. In any case, in quantitative terms no significant differences can be appreciated between the results obtained with FEM and REM, which enables us to conclude that our estimations are robust, as is Table 5 Selected studies in the Western-Pacific region: coded data. Source: Authors. Country/ income level Researcher/place Proxies of effects methodology Results Time series studied/–time– span of effects China (MIC) Wong et al. (2008)/(Hong Kong) VBD Multinomial logit model. An increase in penalty points produced a positive deterrence effect in combating red light violations (coefficient¼0.0959). (2005–2006)/12 months Sze et al. (2011)/ (Hong Kong) VTRA Binomial regression model. The number of red light offense RTA and casualties dropped by 23% and 29%, respectively. (2005–2007)/12 months Australia (HIC) Diamantopoulou et al. (1997)/ (Victoria) VTRA Logistic regression model. Adding a driver’s prior offenses (whether as demerit point levels or categories of offense) into this model produced the best predictive ability in identifying drivers with future RTA. (1991–1994)/No quantified period of effects VBD Haque (1990)/ (Victoria) VBD Poisson model using maximum likelihood estimation methods. The estimated mean inter-offense time interval between 2 nd and 3 rd offenses for drivers was significantly longer than the interval between 1 st and 2 nd offenses. (1982–1985)/No quantified period of effects HIC: High-income countries; MIC: Middle-income countries; LIC: Low-income countries. VTRA: Variables related directly to RTA, in terms of reductions in the number of accidents, casualties, fatalities or injured; VBD: Variables related to driver behavior, in terms of reductions in risky or dangerous conduct, driving less unsafely or a reduction in law violations; VHCD: Variables related to health care data, in terms of reductions in RTA-related A&E admissions, surgery or hospitalizations. Table 6 Meta-analysis outcomes. Source: Authors. Scenario Sample size Q statistic (p-value) R Ia OR b (FEM)/ CI limits 95% Result d (FEM)/ CI limits 95% OR c (REM)/ CI limits 95% Result d (REM)/ CI limits 95% Begg’s test (p-value) Egger’s test (p-value) I (VTRA) m¼24 0.0000 nnn 0.9944 2.9588 (2.9444; 2.9734) 19.2748 (18.993; 19.5583) 2.7570 (2.5516; 2.9789) 15.7525 (12.8276; 19.6662) 0.1574 0.3945 I (VBD) m¼18 0.0000 nnn 0.9998 3.4592 (3.4536; 3.4647) 31.7915 (31.6140; 31.9669) 3.3119 (2.9009; 3.7811) 27.4372 (18.1905; 43.8643) 0.2889 0.8765 I (VHCD) m¼8 0.0000 nnn 0.9992 4.0938 (4.0909; 4.0967) 59.9673 (44.1282; 61.6825) 3.9510 (3.7871; 4.1220) 51.9873 (59.7937; 60.1415) 0.7105 0.1318 II (LENGTH OF EFFECTS) m¼20 0.0002 nnn 0.6912 2.8088 (2.7305; 2.8893) 16.5900 (15.3406; 17.9807) 2.7663 (2.5602; 2.9840) 15.8997 (12.9384; 19.7667) 0.1119 0.2286 Notes: Definition of variables for Scenarios: VTRA: Variables related directly to RTA, in terms of reductions in the number of accidents, casualties, fatalities or injured; VBD: Variables related to driver behavior, in terms of reductions in risky or dangerous conduct, driving less unsafely or a reduction in law violations; VHCD: Variables related to health care data, in terms of reductions in RTA-related A&E admissions, surgery or hospitalizations; Length of effects: number of months.Significance at nnn 1%, nn 5%, n 10%, respectively. a R I indicates the proportion of the total variance due to between-study variance. b Odd Ratios for fixed-effects model and random-effects model expressed in logarithm. c Odd ratios for fixed-effects model and random-effects model expressed in logarithm. d Results (exponential OR) expressed in % for all variables of Scenarios I and number of months for Scenario II. J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201 197 also shown by the sensitivity analyses done. 8 In both models, the variables that experience a more positive effect with the coming into effect of the PS are the VHCD (with reductions of over 50% both in the number of hospitalizations and in A&E admissions), followed by the VBD (with a reduction in the number of reckless behaviors of around 30% on average). Meanwhile, the weakest effect seems to be on the VTRA (with reductions in accidents, fatalities and injuries of between 15 and 20%), i.e., the variables most directly related to road traffic accidents. Considering scenario II, a number of doubts could be raised about which of the two methods for estimating the Length of Effects is more appropriate as, despite the Q statistic rejecting the homogeneity null hypothesis at 99%, the R I coefficient is slightly under 0.75, indicating that its detection is less powerful (Takkouche et al., 1999). Notwithstanding, this problem is irrelevant given how similar the estimations by both FEM and REM are, with a difference of only 21 day between them (16.6 months compared to 15.9 months effects duration). In any event, the duration of the effects of PS is on average less than one and a half years. When this result is tested against Tables 2–5, it can be seen that of the 26 assessments analyzed in 11 different countries, 20 determine the length of the effects of a PS, and of these 9 give durations of over 12 months. Even in these cases no effect lasted a period of over 2 years, and there was simply an interim effect compared to other road safety measures with more structural effects (see e.g., Aparicio Izquierdo et al., 2011 for policing and Castillo-Manzano et al., 2010 for mandatory seatbelt use in Spain). Another interesting aspect is that 7 of these 9 evaluations with effects of over 12 months are for the same two European countries, Italy and Spain, which started out with road accident rates that were higher than the mean for the Eurozone (WHO, 2009). The system’s implementation in these countries was also accompanied by widespread dissemination of advertising campaigns in the institutional media and in all private news media, which shaped a favorable perception as far as public opinion is concerned (Novoa et al., 2010 for Spain; Benedettini and Nicita, 2009 for Italy). Specifically in the case of Italy, Benedettini and Nicita (2009) found there was an effect even before the PPS law came into force, with the media interest bringing about a positive ‘‘announcement effect’’, i.e., the effects of PPS introduction were anticipated even before it had been effectively adopted as law. The remaining 11 studies coincide in 12 months being a quite likely duration for the effects of the measure and that the effects would not remain stable throughout this period of a year, but would gradually decrease. Specifically, after a major initial shock during the months that immediately followed PS coming into force, the effects quickly wane with time. There is clear evidence that this major shock was noted even in countries where the effects immediately fizzled out. In Canada, for example (see Redelmeier et al., 2003), although it was estimated that the effects only lasted 2 months, there was a 35% fall in the risk of a fatal crash and in the Czech Republic (see Montag, 2010), despite the effects being limited to only 3 months, fatalities were about 30% lower. The p-values obtained in the Egger and Begg Tests in Table 6 show that, in principle, our estimations could be stable from the point-of-view of publication biases, as the lack of bias null hypothesis cannot be rejected in all the scenarios. However, as these detection methods are known to not always be very powerful (e.g., Sterne et al., 2000), it is recommended that the tests are complemented with an interpretation of the funnel plots diagrams (Palma-Pe ´rez and Delgado-Rodrı ´guez, 2006) which are included for all the analysis scenarios in Figs. 1–4. The parameter standard error (SE) is represented on the x-axis, the parameter for each study (odds ratio,OR) on the y-axis and each estimation by a point. The result is a funnel which narrows to the left as higher precision studies (smaller SE, equivalent to a larger sample size) are on the left-hand side of the figure and there is less variability between them. A symmetrical figure rising from a horizontal axis that passes through the parameter’s weighted value would indicate the absence of this error. As can be seen in Figs. 1–4, more or less asymmetrical figures are obtained for all scenarios which contradict the tests done in Table 6 and indicate the existence of publication biases. However, caution should be used when interpreting this. In fact, according to Thornton and Lee (2000), symmetry can be subjectively defined by the researcher, especially when there are small numbers of points. Heterogeneity between the estimations can also interfere when assessing of publication bias. Notwithstanding, following Høye and Elvik (2010) publication biases are very frequently found in meta-analysis to evaluate the effects of a road safety measure. These authors state that there are certain prejudices about omitting the publication of statistically nonsignificant studies or those with adverse results, as it is not reasonable for public opinion to expect a road safety measure that has no effect. Another possible source of these publication biases could be the strict filter applied to select the study sample for the Standard Error (SE) Effect Estimation Fig. 1. VTRA scenario. Note: Effect estimation in log scale. Source: Authors. Standard Error (SE) Effect Estimation Fig. 2. VBD scenario. Note: Effect estimation in log scale. Source: Authors. 8 The results of the sensitivity analysis are available from the authors upon request. J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201198 meta-analysis (see Section 3.1). Certain studies from several countries have been deliberately omitted either because the methodologies that they applied were not very rigorous (before and after statistical comparisons), or because they were ‘suspicious’ for having been carried out by governmental organizations and not being widely disseminated academically. 4. Conclusions and implications for road safety Since their pioneering implementation in the US in the mid 20th century, driving licenses based on points systems have rapidly expanded to countries with high and medium income levels on all continents, especially in the European region. As Nole ´n and O ¨stlin, (2008) and ETSC (2008) state, their popularity and social acceptance as one of the fairest corrective and punishment measures for internalizing the social costs of road traffic accidents seem to be the key to their wide-ranging development over the last decade. Despite recommendations from prestigious international organizations, very little is known in the academic literature about the effects of points systems on road safety, because it is very difficult to isolate their impact from the effects of other concurrent complementary types of enforcement applied (SWOV, 2010) (policing, laws or advertising campaigns). In this regard, studies such as Castillo-Manzano et al. (2010) point to the most controversial aspect of points systems linked to their overall effects in the short- and long-term. Our study analyzes whether points systems really are effective for bringing down the traffic accident rate or whether this is just a fashionable policy that countries in close geographical proximity or with similar cultural patterns imitate. The effects of points systems and the duration of the effects have been estimated by applying a meta-analysis approach to a sample of 70 estimations taken from 26 scientific assessments from 11 countries. Although it might result in publication bias, a strict sample selection criterion was opted for and a number of suspicious and possibly less scientifically rigorous studies were omitted. The results of the sensitivity analyses enable it to be robustly stated that all the variables benefit from the implementation of a points system. To be precise, reductions of over 50% in numbers of road traffic accident-related emergency admissions, surgery and hospitalizations are recorded. Reductions of around 30% in the number of law violations and risky and dangerous driver behaviors are also estimated. However, the most significant effect, (given that the outcomes are directly linked to road traffic accidents), is the reduction between 15 and 20% seen in the number of accidents, fatalities and injuries. By contrast, the high expectations that points systems generate among public opinion and the road safety authorities with regard to duration in time have not been subsequently supported by long-term results as, in general terms, the effects wear off in less than 18 months. Only 7 of these studies state that the effects have durations of over 12 months, but also indicate that they last under 2 years. Most of these seven studies, incidentally, refer to Spain and Italy, countries, precisely, where the system has been accompanied by major advertising campaigns, as shown by Novoa et al., (2010) and Benedettini and Nicita, (2009). It seems that this measure has only a major initial shock on a wide range of road safety variables and indicators, although the effect quickly declines over time when there are no other complementary enforcement measures. In short, as Twisk and Stacey (2007) state, many of the countermeasures that affect a licensing process are not effective without enforcement that increases the feeling that you could be caught, and complemented with media coverage that reduces the incentive for offenders to persist in their unsafe behaviors. Given that the benefits seem to be running out in terms of road safety, it is necessary to compute the estimated cost of implementation. To date, little has been provided regarding quantification other than, according to Elvik and Vaa, 2004, that these costs, a priori, logically seem to be theoretically lower than for other road traffic accident prevention methods, such as road improvements. For Elvik. et al. (2009), the lack of a full estimation of these costs would explain why to date a more-than-required Cost- Benefit Analysis has not been conducted for this measure. For all these reasons, a future line of research that begs to be suggested is a realistic estimation of the costs of this measure. This estimation should take into account both the design of the information campaigns that support them and changes in the country’s policing, legal and administrative systems that result from their application. With respect to the information campaigns, these have to be extended to cover the whole of the population and also need to be intensive in order to deal with all aspects of the system, from the various violations that are envisaged, to the license withdrawal process and its later recovery. With regard to the policing and legal-administrative systems being adapted, to prevent injustices there should be full cooperation between all the authorities responsible for road safety with the aim of ensuring that the monitoring, detection and recording of all sanctions is standardized wherever they take place. If the measure is implemented in a way that is lacking in this respect it could lead either to the measure being futile, as would seem to have been the case in The Arab Emirates (see Mehmood, 2010), or that systematic injustices are committed among Standard Error (SE) Effect Estimation Fig. 4. VHCD scenario. Note: Effect estimation in log scale. Source: Authors. Standard Error (SE) Effect Estimation Fig. 3. VHCD scenario. Note: Effect estimation in log scale. Source: Authors. J.I. Castillo-Manzano, M. Castro-Nun ˜o / Transport Policy 21 (2012) 191–201 199