Effectiveness of public training programs reducing the time needed to find a job

____________ Artículo recibido en diciembre de 2009 y aceptado en enero de 2011 Artículo disponible en versión electrónica en la página www.revista-eea.net, ref. ə-29103 ISSN 1697-5731 (online) – ISSN 1133-3197 (print) ESTUDIOS DE ECONOMÍA APLICADA VOL. 29 - 1 2011 PÁGS. 1 - 26 Effectiveness of Public Training Programs Reducing the Time Needed to Find a Job JOSÉ MANUEL CANSINO MUÑOZ-REPISO Departamento de Análisis Económico y Economía Política. UNIVERSIDAD DE SEVILLA. SPAIN. E-mail: [email protected] ANTONIO SÁNCHEZ BRAZA Departamento de Análisis Económico y Economía Política. UNIVERSIDAD DE SEVILLA. SPAIN. E-mail: as[email protected] ABSTRACT This paper estimes the average effect of a binary treatment on the time needed to find a job. Such a treatment is the training public schools program implemented in Seville. The research compares one training group and a control group. Two methods are developed with an interesting database. The first uses an estimator which weights observations by the inverse of the propensity score. This estimator let us conclude that, for participants, the time needed to find a job is reduced in 471 days. The second one is the differences estimator, it let us conclude that the time needed to find a job is reduced in 448 days. The evaluated program works as an active labour market policy with favourable effects on unemployed young people. Compared with other research in Spain, the obtained results show evidence in the same way as most of evaluation but stronger. A similar conclusion is derived from a comparison with international evidence disposable. Keywords: Training programs evaluation, public policies, propensity score, differences estimator Eficacia de los programas públicos de formación en la reducción del tiempo necesario para encontrar un empleo RESUMEN El artículo estima el efecto promedio de un tratamiento binario sobre el tiempo de búsqueda de empleo. Dicho tratamiento consiste en el Programa de Escuelas taller y Casas de Oficio desarrollado en la Provincia de Sevilla. Los autores comparan un grupo de tratamiento y otro de control. Para la estimación se utilizan dos métodos con los datos ofrecidos por una importante base de datos. El primero utiliza un estimador que pondera las observaciones por la inversa de la probabilidad de participar en el programa de formación (“propensity score”). Este estimador permite concluir que, para los individuos participantes en el programa, el tiempo de búsqueda de empleo se reduce en 471 días. El segundo método de estimación consiste en el estimador de diferencias. Con este segundo método el tiempo de búsqueda de empleo se reduciría en 448 días. El programa evaluado funciona como una política activa en el mercado de trabajo con efectos positivos sobre los jóvenes desempleados. En comparación con investigaciones similares realizadas en España, los resultados obtenidos arrojan evidencia en el mismo sentido aunque muestran un efecto mayor. Una conclusión similar se obtiene de comparaciones con la evidencia internacional disponible. Palabras clave: Evaluación de programas de formación, políticas públicas, probabilidad de participar, estimador de diferencias Clasificación JEL: H30, C10 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 2 1. INTRODUCCIÓN 1 From Heckman, Clements and Smith (1997), it is generally accepted that social programs impact differently on individuals when they differ in characteristics like the previous educational level, pre-training labour market status, sex, age, earnings, family background, etc. The individual’s addecuate assignement to the set of programs disposable begins a crucial issue in political decisions. After controlling by covariates and by knowing the average effect of a program on an appropiatte outcome, the public decisor can decide which program would provoke the best impact on individuals by taking into account the program’s average effect on subpopulations previously estimated. Social welfare can improve if public decisors follow an assignment rule which let them determine which individuals must receive which treatment. Manski (2001) theorized this assuming the case of a finite set of rival treatments. Cansino and Román (2007) explored this for the Spanish Accounting Court. The aim of this paper is to estime the average effect of a binary treatment on an appropiate outcome. Such a treatment is the Training Schools Program (TSP) implemented in Seville between 1997 and 1999. Speciffically, the paper estimes the average effect of this training program on the time needed to find a job. To choice this outcome we followed Zweimüller and Winter-Ebmer (1996) who consider that similar evaluations in USA usually focused on impact on earnings as outcome while in Europe focused on outcomes like the one used in this paper2. We select the province of Seville as this is the zone in Spain with the most widely developed number of training policies until now. The evaluation is carried out by estimating the program’s average effect over the individuals' ability of the sample to find a job, individuals being unemployed between 16 and 25 years old. Following Hirano et al. (2003), the paper estimes the average effect by using an estimator which weights observations by the inverse of the estimation of the propensity score. The differences estimator is also implemented to compare results. Evaluations of training are very scarce in Spain but largely developed in others countries like France, Germany, United Kingdom and USA. This may be due to the difficulty in collecting quality microdata. This paper contributes to 1 Authors acknowledge comments and suggestions made by two anonymous reviewers although take full responsibility for the content. Authors are also grateful for the financial support received from the project SEJ132. 2 A recent exemption is debt to Caparrós et al. (2010). Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 3 the literature in the sense that no evaluation based of non experimental methods has been applied to this training program in Spain before, apart from Cansino and Sánchez (2008 a y b), Mato and Cueto (2008) and Cueto and Mato (2009). Paper structures as follow. The model and program characteristics are described in section 2. Section 3 is destined to explain the database used. In section 4, we define the estimator which weights observations by the inverse of the propensity -Hirano et al. (2003)- and results are obtained. The differences estimator is defined in section 5 and results are also obtained. Section 6 concludes. 2. CHARACTERISTICS OF THE PROGRAM EVALUATED The TSP is one of the active labour market policies used in Spain. The program under evaluation is the TSP implemented in Seville between 1997 and 1999. The TSP was designed as a nationwide experimental one implemented by the Spanish Department of Labor (more specifically, by the National Institute of Employment). Considering the first results of the program, the Spanish Department of Labor decided to convert it into a permanent program regulated by the Department Labor’s order of march 29st - 1988. Finally, the Department Labor’s order of august 3st - 1994 added this program into the set of the national policies of employment until now. TSP is a vocational training program which is managed in a descentralized way. It is offered free of charge and it involves voluntary access to training. The objective of the TSP is to act as an iniciative of young unemployed minors (less than 25 years old) in finding a job. The TSP organizes its activities into two steps; the first one gives unemployed a theoretical education and the second one offers a professional stage. In order to judge the interest of this public training program, three parameters have been considered. The first is the number of participants. After the experimental period, the average number of participants had a range of between 45000 and 50000 young unemployed for every year. Compared to the whole of Spain, Seville is the zone with the largest number of implemented projects since 1985. This justifies the geographical focus of the paper and that implies a minor size of the sample. Secondly, the size of the public funds absorbed as Table 1 shows. Thirdly, the EU authorities supported this training program allowing the use of the European Social Fund to finance it. Table 1 shows the total budget for joint employment actions between the Spanish Department of Labor and the European Social Fund (ESF) for the period 2000-2006. Figures in Table 1 show the importance that the ESF has as a co-financer of the training programs. Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 4 Table 1 Joint budget actions between the Spanish Department of Labour (SDL) and the European Social Fund (ESF) (2000-2006) TOTAL 9.540.537.406 € SDL 3.772.328.490 € 39’54 % ESF 5.768.208.916 € 60’46 % Source: INEM (National Institute of Employment) 3. THE BASEVAFOR DATABASE The BASEVAFOR database has been constructed from individuals who have participated in the TSP carried out in Seville. Individual data came from the oficial employment agency. We have selected those individuals who have finished the TSP in 1999, the last year in which information was available when we started the evaluation3. Depending on the length of the program (1 or 2 years), programs finishing in 1999 started in either 1997 or 1998. Only individuals who finished the TSP have been considered, rejecting those who left the program before the end. The total of individuals was 1528 and from that figure we have selected a sample of 150 (around 10%)4. The selected individuals make up the group of participants in our investigation. To complete the database, the oficial employment agency provided us a control group5. To this end, 75 individuals6 with similar characteristics to the participants have been selected. 3 The requirements of the individuals were to be between the ages of 16 and 25 years, to remain unemployed and have signed up to the unemployment office. 4 The dimension of the selected sample is comparable to similar evaluations included in Dehejia and Wahba (1999). We assume some data are missing at random (Rubin, 1976; Little and Rubin, 1987). 5 For the purpose of this paper, the control group is an external one (Friedlander et al., 1997). This type of group represents a sample of individuals who, during the period of time considered, show the same characteristics of participants and have the requirements to participate in the training program, but they have not participate in it. Ashenfelter (1978), Heckman et al. (1994) and Dehejia and Wahba (1999) justified the use of external control groups. Friedlander et al. (1997) criticize the use of internal groups in evaluations because it was quickly recognized that nonparticipants are likely to be quite different from participants by virtue of the fact that they have been excluded by program staff. In a similiar way Bell et al., (1995), have pointed out that Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 5 The database includes two types of data related not only to participants but also with the controls. Firstly, this database gives us information about the periods of employment and unemployment of individuals, including data related to the number of times the individual has applied for a job. We will use this information to construct Y variable defined in next section. Secondly, this database contains information related to the covariates considered: sex, age and residence zone. 4. MODEL CHARACTERISTICS 4.1. The potencial outcome model The development of public policies evaluation has benefited from the use of causal inference7. One of the results is the Potencial Outcome Model –POM-, which allows us to compare participants and non participants in public programs8. A prolific development of the POM with regard to training programs evaluation comes thanks to Roy (1951) and Rubin9 (1974, 1978). This paper support the Roy-Rubin Causal Model (RRM). In the implementation of POM and RRM, the individual values of the main variables can be extracted from randomized experiments or from observational data. Both types of data will notably determine the evaluation and will promote different methodological developments. In social sciences, randomized experiments face important problems related to cost, moral limitations, attrition and problems derived from the Hawthorne effect -Burtless (1995) and Cameron and Trivedi (2005)-. This can be solved by using observational data. In these cases, Rosenbaum (1999) says that the researcher should design a treatment group and a control group from the ‘secreed out’ applicants by definition differ from participants only on factor (both objetive and subjetive) observable to staff. 6 We choice the ratio 2/1 which was used in the evaluation of the National Job Training Partnership Act (JTPA) study. The JTPA Study was commissioned by the U.S. Department of Labour in 1986 to measure the benefits and cost of selected employment and training programs for economically disadvantaged adults and out-of-school youths. See Heckman, Ichimura and Todd (1997, 1998) and Heckman, Ichimura, Smith and Todd (1998). 7 Refering to the theoretical approach of casuality and its use in randomized experiments, see Cox (1992). Other authors such as Dawid (1979, 2000), Holland (1986), Heckman (1990) and Pearl (2000) also discuss the meaning of casuality in such an enviroment. Finally, in the specific case of training programs, we have refered to the seminal papers of Rubin (1974) and Heckman and Hotz (1989). 8 Cameron and Trivedi (2005) expose the POM advantages compared to alternative models. 9 The first references that Rubin considered were Neyman (1923, 1935) and Fisher (1928, 1935). Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 6 individuals who have or have not been treated. The objective is to reproduce a scenario which is as similar as possible to a randomized experiment10. However, the comparison of the two outcomes for the same unit is not possible because they are conterfactual events (like an individual participating and not participating at the same time in a training program).We can at most observe one of these outcomes. Then, models which include counterfactual events are ineffective in individual causal effects estimation. Holland (1986) refers to this as “the fundamental problem of causal identification”. The construction of the counterfactual is one of the major problems in evaluation studies (Imbens and Wooldridge, 2009). 4.2. Definition of D and Y variables. We define D as the binary variable which indicates the participation of the individuals in the program. So, Di=1 will indicate that individual i has participated in the program and Di=0 will indicate that individual i has not participated. Otherwise, we consider Y as the response variable from which the program’s average effects will be evaluated. We define Y as the ability of the individual i to find a job, and shows how much time he has to spend searching for a job11. nobservatioofdurationtotal jobafindiindividualtheuntildayscutiveconseofnumber Yi "" 1 (1) The choice of the response variable is justified because individuals of the sample, both participants and non participants (control group), are initially unemployed and included in the oficial census of people who are searching for jobs. Most of them have not been working before or have a short labor experience because of their age and lack of experience. As was mentioned above, the objective of the TSP is to act as an initiative focus on young unemployed minors (less than 25 years old). For this reason, it is relevant for the program evaluation to consider a response variable which allows us to measure the abilities of these people to find jobs. 10 The seminal papers in this topic were implemented in Medicine. The papers of Cameron and Pauling (1976), Billewicz (1965) and Cochran (1968), must be highlighted. An interesting comment about these seminal papers is contained in Rosenbaum (1995, 1996). Some wellknown papers in observational methods are Kiefer (1979), Bassi (1984) and Moertel et al. (1985) 11 For a further investigation, we could define two outcomes in an alternative way. The first one let us know the treatment average effect on the individual probability to find a job. The second would be the time needed to find a job conditioned to the unemployments subset (treated and controls) who have found a job. Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 7 The period of observation we have considered consists of three years12 (1095 days). We started to measure this time from the moment participants finished the training program (generally at the end of 1999) and january 1st 2000 for individuals of the control group13. The value of Y varies between 0 and 1. If Y is equal to 0, it means that individual i has not found a job during the period considered. This is the worse scenario for the program’s effectiveness. If Y is close to 1, the individual i has found a job in a short period of time and if Y is equal to 1, it implies that individual i has found a job the first day after finishing the training program. 4.3. Identification and selection on observables The fundamental problem of causal identification makes us look for second best solutions in which researchers leave the estimation of the individual causal effects . The Average Treatment Effect of the program14 (ATE) is addressed in a partial equilibrium environment15 and, by using the potential outcome notation popularized by Rubin (1974), it is obtained as the average expected value from the difference between the potential values of Y1 (the case of an individual treated) and Y0 (the case of a non treated individual). Implicit in this notation is the stability assumption or SUTVA (Rubin, 1978) that individuals are not affected by receipt of treatment by others, and there is only one version of the treatment. As a consecuence, no general equilibrium effects are considered (Cameron and Trivedi, 2005). In the same way, the Average Treatment Effect of the program on the Treated (ATET) is defined as the average expected value from the difference between the potential values of Y1 and Y0 but only with respect to individuals who have received treatment. Given the fact that the validity of average effects can be damaged if participants and controls show different characteristics apart from their participation or non participation in the training program, these charateristcs must be controlled because of their effect on the values of the response variable. 12 A relatively broad period of time has been considered, three years (1095 days), due to specific problems of this collective in finding jobs. 13 The date fixed to start the test for the control group coincides with the starting date for many of the individuals in the participants group. 14 Although in this paper only the most well-known average effects are used, Imbens (2004) summarizes all the possible types of average effects of treatment in literature. 15 The ATE is addresed in a partial equilibrium environment different, for example, to the CGE model mentioned in Cansino, Cardenete y Román. (2007). Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 8 If the observed characteristics are the only individual (participants and controls) characteristics that differ, we can therefore control these differences. This is the base of the selection on observable model16. Selection on observables allows us to isolate the effect of a covariate (or a vector of covariates)17 maintaining the independence between the treatment indicator variable D and the response variable, Y. This condition can be expressed as ( Y1 , Y0 )  D | X (2) Selection on observables supports the independence assumption18 typical in randomized experiments, contributing to the comparison between participants and controls. Following Heckman and Hotz (1989), selection on observables is recommended when the independence between D and Y is because of the covariate X (or vector of covariates), which has influence on the individual selection process, so by controlling X we give a solution to possible biased selection, making the dependency between D and Y disappear. In the selection on observable context, when the independence assumption is guaranteed, we considered, according to Dehejia and Wahba (1999), that E [Y1 - Y0 | X ] = E [ Y | X, D = 1 ] - E [ Y | X, D = 0 ] (3) Equation 3 lets us express the ATE as              )(0,|1,| )(| 0101 XdPDXYEDXYE XdPXYYEYYEATE (4) Then, it is possible to determine ATE from the difference between the average observed value of the response variable of the participants and the controls by calculating the difference for every possible value of X. In a similar way, it is possible to calculate the average effect of the training program only for participants (ATET) as the difference between the average 16 To improve knowledge of selection on observables we recomend Barnow et al. (1980). When controlled and treated differ in unobserved characteristics like psicologycal ones, average effect can be estimated by the differences in differences estimator. For the context of this paper, see Cansino and Sánchez (2008a). Another alternative, when independence condition is not guaranteed, is to use the methods based on instrumental variables if an adecuated instrument to determinate D is available. Instrument must be uncorrelated with Y. 17 As an introduction to the framework of the observational methods we recommended the examples that are used by Rosenbaum (1995) in his exposition about these methods; this also can by said of Cochran (1968) and Cameron and Pauling (1976). We also recommend the papers of Billewicz (1965) and Moertel et al. (1985), both of them refering to the two previous examples. 18 This is also known as the unconfounddedness assumption (Rubin, 1978; Rosenbaum and Rubin, 1983). See also Barnow et al. (1980). Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 9 observed values in the response variable of participants and the average values of controls for every different value of X when D=1. That is what makes (5)        )1|(0,|1,| 1| 01 DXdPDXYEDXYE DYYEATET (5) 4.4. Definition of the vector X of covariates With D defined, X will be a covariate19 with respect to D if, for each of the individuals observed, its values remain the same for each value of D. That is to say X 0i=X 1i, being X0i the X value before the event D (D=0) and X1i the X value after happening D (D=1). X covariate is also named contaminant because of the fact that X can contamine Y by adding its own effects20 to those provoked by D. The fact that X is predetermined with respect to D does not imply that this independence is bidirectional because it is possible that, as a characteristic of considered population, dependence in an opposite direction can appear making the value of D be affected for X. From the sample information included in the database, we consider three predetermined characteristics to form the vector of covariates X. The database only allows us to include in the model a complete information about this three characteristics. Table 2 describes X. Table 2 Definition of Covariates of Vector X Characteristics Description Cova riates Potential values 1) Sex This binary covariate shows if the individual considered is male or female. Base category = female. X11 1 = male 0 = female 2) Age This covariate shows the individual’s age at the beginning of the observational period. In the case of participants, shows the individual’s age when the training program is over. For controls, shows the individual’s age as of January 1 st, 2000. Considering that age range for participants in one of the considered training programs is between 16 and 24 years old, and also considering that the program may extend for 1 or 2 years, this covariate will have values of between 17 and 26 years old. X2 17 ≤ X2 ≤ 26 19 We talk about one covariate but everything we state can be extrapolated for the case that X is a vector of n covariates, as X n = ( X1 , X2 , … , Xn ). 20 To read more, comments of Rubin (1978) about covariates are very interesting. Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 16 where )( ˆi X  is the estimated value of the propensity score for the i-individual on vector X, n is the sample size and n1 the number of participants. We can apply bootstrapping techniques to obtain estimates for the standard errors.The obtained results are shown in Table 8. Table 8 The calculation of ATE  ˆ and ATET  ˆestimators by using the “propensity score weighting” Estimator (Coefficient) Coef. Value Std. Error t -statistic Prob. ATE  ˆ 0’421280 *** 0’060 6’854666 0’0000 ATET  ˆ 0’430409 *** 0’056 7’685875 0’0000 Note: One, two or three asterisks indicate coefficient significance at the 10-percent, 5-percent and 1-percent levels, respectively. Source: Own elaboration The estimated value of the ATE is positive. In average, the sample’s individuals’ ability to find a job increases by 0’421280. In the case of the ATET, the estimated value is also positive, meaning that there is a favourable causal effect from the program. This result indicates that participant’s ability to find a job has increased, on average, by 0’430409 so the TSP reduces the time needed to find a job and no looking-in effect of trainees is shown. We can address results in the context of similar evaluations. Mato (2010) and Ramos et al. (2010) give an overview of the evaluations of training programs carried out in Spain recently. Compared with other research in Spain, our results show evidence in the same way as most of evaluation but stronger. A similar conclusion is derived from a comparison with international evidence summary in Kluve et al. (2007) and Card et al. (2009). 6. ESTIMATION OF THE AVERAGE EFFECT BY REGRESSION It is possible to determine the average effect of a training program on participants (ATET) by regression by using Least Squares, given that independence assumption is also guaranteed. According to Stock and Watson (2003), we can obtain an estimator of the average effect of the evaluated training program on partipants ( ATET  ˆ) using a linear model. Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 17 It is also possible to introduce covariates in the model as additional regresors. By doing that, we can measure the effects that covariates have on Y. The inclusion of covariates in the model is shown by the following expression (Wooldridge, 2002): vXDY        (11) where Y is the dependent variable and shows the potential results of the individual. D is a binary explanatory variable. X is the covariates’ vector and β is the parameter associated of the vector X. Parameter µ collects fixed effects in the model and the parameter v collects the random error of the model, with an average value equal to 0, E [  | D, X ] = 0. The parameter  will determine the average effect of the program on participants. This parameter is named the “differences estimator with additional regressors”. As independence assumption is guaranteed, if necessary assumptions23 for multiple regression by Least Squares are guaranteed too, this estimator will be unbiased and consistent. The inclusion of additional regressors in the model lets us improve the estimator’s efficiency by reducing the random error variance. On the other hand, the addition allows us to test the randomness in the individual assigning procedure between the participants group and the control group, in the case that the assigning procedure is related to the additional covariates. That is to say, by including the covariates in the model we can control the probability of individuals being assigned between the participants group and the control group by adding characteristics in which participants and controls differ. The inclusion of predetermined variables in the model will consist of inserting covariables included in the vector X as additional regressors. Table 9 shows the correlation matrix between explanatory variables included in the model, allowing us to analyze possible multicolineality. 23 These are the four assumptions: a) the conditioned distribution of the random error, given the explanatory variates X1i , X2i , …, Xki, is equal to 0 on average (in this case, the explanatory variables are Di , that indicates program participation, and the covariates). b) all the observations i = 1, 2, …, n are distributed both independently and identically random. C) X1i , X2i , …, Xki and  i have four moments. D) Perfect multicolineality does not exist. Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 18 Table 9 Matrix correlation between D and the covariables of vector X D X11 X2 X31 X32 X33 D 1’000000 0’194895 0’168779 -0’007731 0’006873 0’027864 X11 0’194895 1’000000 0’005540 0’218229 0’014691 -0’092845 X2 0’168779 0’005540 1’000000 0’143666 -0’105606 0’030735 X31 - 0’007731 0’218229 0’143666 1’000000 -0’334617 -0’323978 X32 0’006873 0’014691 -0’105606 -0’334617 1’000000 -0’410569 X33 0’027864 -0’092845 0’030735 -0’323978 -0’410569 1’000000 Source: Own elaboration Linear correlation between variables is not obvious because all the coefficients are very low and far from ±1. With regard to the relationship between X31, X32 and X33, these variables show slightly high correlation because of the fact that they have been constructed to include the zone variable in the model. However, values are never over ±0’5. On the other hand, the value of the determinant of the correlation matrix is 0’4464, far from 0. In addition, the condition number of the correlation matrix is C=2’3240, far from the limits that determine multicolineality. Everything we have exposed lets us indicate that multicolineality problems are not relevant in the model. With the previous specifications made, we implement the regression by Least Squares. Results are shown in Table 10. Table 10 The calculation of the “differences estimator with additional regressors” Dependent Variable: Y (Ability to find a job) Variable Coefficient Coef. Value Std. Error t -statistic Prob. Fixed effect μ -0’206095 0’216058 -0’953891 0’3412 D α 0’409173 *** 0’050124 8’163276 0’0000 X11 β11 0’128764 *** 0’045467 2’832055 0’0051 X2 β2 0’023647*** 0’010274 2’301544 0’0223 X31 β31 -0’040937 0’065065 -0’629174 0’5299 X32 β32 -0’028182 0’059367 -0’474715 0’6355 X33 β33 -0’075899 0’058980 -1’286872 0’1995 R-squared 0’360720 Ajusted R-squared 0’343125 Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 19 Mean dependent var 0’578147 S.D. dependent var 0’380657 S.E. of regression 0’308514 F-statistic -143’2157 Sum squared resid 20’74949 Prob (F-statistic) -0’598072 Log likelihood -51’10855 Note: Std. Errors adjusted by White’s method. One, two or three asterisks indicate coefficient significance at the 10-percent, 5-percent and 1-percent levels, respectively. Source: Own elaboration The α parameter (0’409173) is the differences estimator with additional regressors from the program’s effect on participants ( ATET  ˆ). The individual significance of the explanatory variables included in the model is demonstrated by the values obtained for the t-statistc and its associated probability. From these values D, X11 (sex) and X2 (age) appear to be significant variables. The X31, X32 and X33, defined to include the zone covariate, appear insignificant. The result of the log. likelihood ratio test to assess whether they are jointly significant is also negative (Prob. log. likelihood ratio: 0’617698). In any case, we have decided to mantain them because they help to improve the significance of all the estimated parameters and goodness of fit. The adjustment shows non significance of the fixed effect. With respect to goodness of fit, the R-squared statistic equals 0’3607 and shows that the explanatory power of the considered variables is equal to 36’07 percent, significantly improving the accuracy of the adjustment over the estimation without additional regressors. The joint significance of all model estimated parameters can also be tested from the value of the probability of the F-Snedecor contrast. In this case the probability is equal to 0’00000 meaning the acceptance of the joint significance of all the parameters of the model. This implies that we can consider all model parameters, jointly taken, significantly different from 0 with a very high probability. From the estimation we find that the meaning of parameters of the significant variables in the model is important:  The α coeficient associated with the explanatory variable D, shows that when an individual has participated in the program (D=1), the response variable increases by 0’409173. This is the effect on the response variable of participants and means that the ability to find a job increases by 0’409173 over non participants’ value.  The β11 coefficient, associated with the explanatory variable X11, shows that in the case of a male participant (X11=1), the response variable increases by 0’128794. This means that males have a better position than females in terms of Y, which is higher by 0’128794 than the registered value in the case of females. Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 20  The β2 coefficient, associated with the explanatory variable X2, collects the effect of age on Y. Due to variable X2 is a quantitive one, this effect will be related to possible values of this variable, adding 0’023647 to the value of the response variable Y for every unitary change registered by X2. Therefore, individuals belonging to the sample (individuals between 17 and 26 years old) show a higher ability to find a job as the value of X2 increases. To summarize, Tables 11 (for participants) and 12 (for controls) contain the model estimated values for the response variable Y for every possible value of explanatory variables. As figures show, male participants aged 26 from zone 4 have higher expected values for the response variable Y (though differences between zones are not significant). For any individual, higher age implies a higher expected values of Y. Additionally, is shown that women for all cases always register a lower value of the response variable. Higher expected values for the response variable Y for male participants must be related with the characteristics of the Spanish Labor Market in which probability to find a job is still higher for male than female (Blanchard and Jimeno, 1995) and (Dolado and Jimeno, 1997). Age seems to be well considered for employers (see the expected values of Y for the case of male participants aged 26). For political decisions the spread between male and female values of Y could be considered in the sense of Manski (2001) by focusing TSP on groups with higher values of ATET. However this option has legal and moral limits (Cansino and Román, 2007). If we compare results obtained in this section with papers refered at the end of section 5 a similar discussion can be made. Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 21 Table 11 Source: Own elaboration Table 12 Source: Own elaboration Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 22 7. CONCLUSIONS The TSP’s average effect estimated by weighting observations by the inverse of the estimation of the propensity score let us conclude that, for treated, the time needed to find a job is reduced in 471 days. As the program was designed to improved the employ between youngers unemployed; this result supports the effectiveness of this public policy. The TSP’s average effect estimated by the differences estimator let us conclude that, for treated, the time needed to find a job is reduced in 448 days. This result also supports the effectiveness of this public policy. Another conclusion can be obtained from using the covariates information contained in the BASEVAFOR. Males have a better position than females in terms the response variable considered. More specifically, on average, the period needed for a treated to find a job, is reduced in 141 days. By considering age, the same period is reduced in 26 days per year from 16 to 25 years old. According to obtained results, the effectiveness of TSP from the “differences estimator with additional regressor” is positive too, so both evaluations show evidence of that this program contribute to reduce the time needed to find a job. Summing up results of the analysis, TSP works as an active labour market policy with favourable effects on young unemployment. Although there is no consensus in literature related with training programs evaluation about its effects our results show evidence in the same way as most of evaluation but stronger. A similar conclusion is derived from a comparison with international evidence. So we have to be prudent with evidence founded. Further investigations might improve conclusions if public authorities let researchers extend the database information with data related with others individual characteristics. REFERENCIAS BIBLIOGRÁFICAS ASHENFELTER, O. C. (1978): “Estimating the Effect of Training Programs on Earnings”, Review of Economics and Statistics, 60(1), pp. 47-57. BARNOW, B., CAIN, G. and GOLDBERGER, A. (1980): “Selection on Observables”, in Stromsdorfer, E. W. and Farkas, G. (eds.), Evaluation Studies Review Annual, vol. 5, pp. 43-59,. BASSI, L. J. (1984): “Estimating the Effect of Training Programs with Non- Random Selection”, Review of Economics and Statistics, 66 (1), pp. 36-43. Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 23 BELL, S. H., ORR, L. L. and BLOMQUIST, J. D. (1995): Program applicants as a comparison group in evaluating training programs. W.E. Upiohn Institute for Employment Research. Kalamazoo, MI. BILLEWICZ, W. Z. (1965): “The Efficiency of Matched Samples: an Empirical Investigation”, Biometrics, 21 (3), pp. 623-644. BLANCHARD, O. and JIMENO, J. F. (1995): “Structural Unemployment. Spain and Portugal”, American Economic Review (Papers and Proceedings), 85 (2), pp. 212-218. BURTLESS, G. (1995): “The Case for Randomized Field Trials in Economic and Policy Research”, Journal of Economic Perspectives, 9 (2), pp. 63-84. CAMERON, E. and PAULING, L. (1976): “Supplemental Ascorbate in the Supportive Treatment of Cancer: Prolongation of Survival Times in Terminal Human Cancer”, Proceedings of the National Academy of Sciences of the United States of America, 73 (10), pp. 3685-3689. CAMERON, A. C. and TRIVEDI, P. K. (2005): Microeconometric. Methods and Applications, New York, Ed.: Cambridge University Press. CANSINO, J. M., CARDENETE, M. A. and ROMÁN, R. (2007): “Regional evaluation of a tax on retailer sales of some fuels through social accounting matrix”, Applied Economics Letters, 14 (12), pp. 877-880. CANSINO, J. M. y ROMÁN, R. (2007): “Evaluación de políticas públicas sobre poblaciones heterogéneas ¿pueden los órganos de control externo contribuir a su avance?” Revista Española de Control Externo, 9 (25), pp. 107-129. CANSINO, J. M. and SÁNCHEZ, A. (2008a): “Evaluación del programa de Escuelas Taller y Casas de Oficios a partir de su efecto sobre el tiempo de búsqueda del primer empleo. El caso de Sevilla”, Estudios de Economía Aplicada, vol. 27 (1) ref 27101. CANSINO, J. M. and SÁNCHEZ, A. (2008b): “Average effect of training programs on the time needed to find a job. The case of the training programs schools in the south of Spain (Seville, 1997-1999)”. Funcas Workpapers. Available in www.funcas.es. [Last access: November, 2010]. CAPARRÓS, A., NAVARRO, M. L. and RUEDA, M.F. (2010): “Rentabilidad salarial de la formación laboral: Un análisis con datos de panel”, Estudios de Economía Aplicada, 28 (2), pp. 1-20. CARD, D., KLUVE, J. and WEBER, A. (2009): “Active labor market policy evaluations: A meta-analysis”. CESIFO Working Paper nº 2570. COCHRAN, W. G. (1968): “The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies”, Biometrics, 24 (2), pp. 295-313. COX, D. R., (1992): “Causality: Some Statistical Aspects,” Journal of the Royal Statistical Society, Series A (Statistics in Society), 155 (2), pp. 291-301. Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1 JOSÉ M. CANSINO MUÑOZ-REPISO Y ANTONIO SÁNCHEZ BRAZA 24 CUETO, B. and MATO, F. J. (2009): “A nonexperimental evaluation of training programmes: regional evidence for Spain”, Annals of Regional Science, 43 (2), pp. 415-433. DAWID, A. P. (1979): “Conditional Independence in Statistical Theory”, Journal of the Royal Statistical Society, Series B (Statistics Methodological), 41 (1), pp. 1-31. DAWID, A. P. (2000): “Causal Inference Without Counterfactuals”, Journal of the American Statistical Association, 95 (2), pp. 407-448. DEHEJIA, R. H. and WAHBA, S. (1999): “Causal Effects in Non-Experimental Studies: Reevaluating the Evaluation of Training Programs”, Journal of the American Statistical Association, 94 (448), pp. 1053-1062. DOLADO, J. J. and JIMENO, J. F. (1997): “The causes of Spanish Unemployment: A Structural VAR Approach”, European Economic Review, 41 (7), pp. 1281-1307. FISHER, R. A. (1928): Statistical Methods for Research Workers, 2nd ed., London, Ed.: Oliver and Boyd. FISHER, R. A. (1935): The design of experiments, Edinburgh, Ed.: Oliver and Boyd. FRIEDLANDER, D., GREENBERG, D. H. and ROBINS, P. K. (1997): “Evaluating Government Training Programs for the Economically Disadvantaged”, Journal of Economic Literature, vol. 35 (december), pp. 1809-1855. HAHN, J. (1998): “On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects”, Ecometrica, 66 (2), pp. 315-331. HECKMAN, J. J. (1990): “Varities of Selection Bias”, American Economic Review, Papers and Proceedings of the Hundred and Second Annual Meeting of the American Economic Association, 80 (2), pp. 313-338. HECKMAN, J. J., CLEMENTS, N. and SMITH, J. (1997): “Making the Most out of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in Programme Impacts”, Review of Economic Studies, 64 (4), pp. 487-535. HECKMAN, J. J. and HOTZ, V. J. (1989): “Choosoing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training”, Journal of the American Statistical Association, 84 (408), pp. 862-874. HECKMAN, J. J., ICHIMURA, H., SMITH, J. and TODD, P. E. (1998): “Characterizing Selection Bias Using Experimental Data”, Econometrica, 66 (5), pp. 1017-1098. HECKMAN, J. J., ICHIMURA, H. and TODD, P. E. (1997): “Matching As an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme”, Review of Economics Studies, 64 (4), pp 605-654. Estudios de Economía Aplicada, 2011: 1-26  Vol. 29-1 EFFECTIVENESS OF PUBLIC TRAINING PROGRAMS RESUCING… 25 HECKMAN, J. J., ICHIMURA, H. and TODD, P. E. (1998): “Matching As an Econometric Evaluation Estimator”, Review of Economics Studies, 65 (2), pp. 261-294. HECKMAN, J., SMITH, J. and TABER, C. (1998): “Accounting for Dropouts in Evaluations of Social Programs”, Review of Economics and Statistics, 80(1), pp. 1-14. HIRANO, K., IMBENS, G. W. and RIDDER, G. (2003): “Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score”, Econometrica, 71(4), pp. 1161-1189. HOLLAND, P. W. (1986): “Statistics and Causal Inference (with discussion)”, Journal of the American Statistical Association, 81 (396), pp. 945-970. IMBENS, G. W. (2004): “Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review”, Review of Economics and Statistics, 86 (1), pp. 4-29. IMBENS, G.W. and WOOLDRIDGE, J. M.(2009): “Recent Developments in the Econometrics of Program Evaluation”, Journal of Economic Literature, 47 (1), pp. 5-86 KIEFER, N. (1979): Economic Benefits from Four Manpower Training Programs, Garland Series of Outstanding Dissertations in Economics, New York, Ed.: Garland Press. KLUVE, J., CARD, D. FERTIG, M., GORA, M., JAVOBI, L., JENSEN, P., LEETMAA, R., NIMA, L., PATACCHINI, E., SCHAFFNER, S., SCHMIDT, C., VAN DER LLAAUW, B., and WEBER, A. (2007): Active Labor Market Policies in Europe: Performance and perspectives, Berlin, Ed.: Springer. LITTLE, R. and RUBIN, D. (1987): Statistical Analysis with Missing Data, Nueva York, Ed.: Wiley. MANSKI, C. F. (2001): “Designing programs for heterogeneous populations: The value of covariate information, American Economic Review, 91 (2), pp. 103-106. MATO, F. J. (2010): “La formación continua en España desde una perspectiva comparada: Balance y propuestas de mejora”, Papeles de Economía Española, 124, pp. 266-280. MATO, F.J. and CUETO, B. (2008): “Efectos de las políticas de formación a desempleados”, Revista de Economía Aplicada, 16 (46), pp. 61-83. MOERTEL, C., FLEMING, T., CREAGAN, E., RUBIN, J., O’CONNELL, M. and AMES, M. (1985): “High-dose Vitamin C Versus Placebo in the Treatment of Patients with Advanced Cancer Who Have Had no Prior Chemotherapy: a Randomized Double-blind Comparison”, New England Journal of Medicine, 312, pp. 137-141. NEYMAN, J. (1923): “On the Application of Probability Theory to Agricultural Experiments. Essay on Principles.”, re-edited in Statistical Science (with discussion), 1990, 5 (4), pp. 465-472. Estudios de Economía Aplicada, 2011: 1-26  Vol 29-1