Evaluating content dimensions in entrepreneurship education

EVALUATING CONTENT DIMENSIONS IN ENTREPRENEURSHIP EDUCATION Kåre Moberg, PhD-students, Copenhagen Business School, Department of Strategic Entrepreneurship and Globalization [email protected], +46 760 478 733 Abstract Interest in entrepreneurship education is growing over the world, especially in innovation based economies, such as Denmark (GEM, 2010). However, we know rather little about the outcomes of entrepreneurship education, in particular with respect to which type of course content produces the best results (i.e. most high performing entrepreneurs) and how this affects different types of students. There is a great variety of different views in the field of research concerning the content and structure of entrepreneurship courses, but no comprehensive study has as yet been done in which these competing views are clearly articulated as rivals and tested against each other. There is also a lack of programme evaluations that use control groups and have a longitudinal design (Gorman, Hanlon & King, 1997; Matlay, 2008). Those that have this setup often experience methodological problems due to their conceptual framework (Krueger, 2009), or they have a view of entrepreneurship that does not take into account the advancements within research that have been made during the last decade (Sarasvathy, 2008). Thus, we clearly need to dig deeper into this field in order to create methods and models that allow us to evaluate the outcomes of different types of entrepreneurship courses. With the beginning of 2011, the Danish Foundation for Entrepreneurship – Young Enterprise1 initiated a research project with the aim to further our understanding of the type of impact entrepreneurship education and different educational designs have on different types of students. Two longitudinal quasi-experimental surveys, one with a focus on elementaryand secondary-level education and one with focus on tertiary-level education, will be performed and databases with students from all parts and levels of the Danish educational system will be created. The surveys will use entrepreneurial self-efficacy (Mauer, Neergaard & Kirketerp, 2009) as a performance indicator, but in order to generate robust results the development of new measurement tools is needed. In this paper the initial phases of this project and the research design of these two surveys will be presented. Introduction and disposition of the text Ever since education for entrepreneurship started during the 1940s, it has been questioned if it can be taught or not (Henry, 2005). Nevertheless, policy makers all around the world have come to recognise it as an important tool for societies to adapt to the “new economy” (GEM, 2010). In Denmark there was launched a major project in 2010 in which a great number of entrepreneurship organisations were consolidated into one major organisation. This organisation was given the name The Danish Foundation for Entrepreneurship – Young Enterprise, and has the mission to support entrepreneurship education at all levels of the educational system. The organisation was also given the assignment to assess which effects and impact entrepreneurship education have. It is this project that will be described in this paper. The text will begin with a contextual description and a short presentation of the Danish Foundation for Entrepreneurship – Young Enterprise. In order to identify the problem a 1 The Danish Foundation for Entrepreneurship – Young Enterprise was established in 2010 when the Danish government merged The Foundation for Entrepreneurship, Activities and Culture -Young Enterprise Denmark, IDEA Denmark and Øresund Entrepreneurship Academy into one organisation. It has the responsibility to develop entrepreneurship at all levels of the educational system in Denmark. discussion about the theoretical background of the field of entrepreneurship education and different types of outcome measurements will follow. We will then describe the methodological approaches that will be applied and two longitudinal surveys that will be performed. The text will end with a description of how we will develop new measurement tools and how these have the potential to further our understanding of which type of content (theoretical focus and didactical methods) in entrepreneurship education that fits different types of students. The case of Denmark In 2010, the Danish government brought together several organisations in order to create a new organisation which should have the responsibility of developing entrepreneurship education holistically trough-out the whole educational system in Denmark – from ABC to PhD, so to speak. The Foundation for Entrepreneurship, Activities and Culture -Young Enterprise Denmark, IDEA Denmark and Øresund Entrepreneurship Academy became one organisation with the name The Danish Foundation for Entrepreneurship – Young Enterprise. The organisation shall function as a coordinating actor and connect education within the field so that the progression runs like a red thread through all levels. In figure 1 the vision of the organisation is graphically illustrated. Insert figure 1 here To accomplish this outcome the organisation performs activities that both focus on the demand-side, such as information and inspiration campaigns, and on the supply-side, such as the development of new courses and further education for educators. It functions primarily as a fund for innovative initiatives, both curricular and extra-curricular, that are initiated by local actors within the educational system. In figure 2 the outcome line of the Danish Foundation for Entrepreneurship – Young Enterprise is presented. Insert figure 2 here One important assignment for this organisation is to assess educational outcomes, i.e. the effects and impact of the programme. Each year the organisation makes a survey of how the number of courses in entrepreneurship and the students taking these courses has developed. A specific coding-scheme that identifies the subject and phase in the entrepreneurial project that the course focuses on and which didactical methods that are used (see Moberg, Vintergaard and Vestergaard, 2008, for a description), allows the organisation to assess the quantitative progress of the field. However, this design reveals little regarding which types of impact and effects these initiatives have. In order to assess the outcomes of entrepreneurship education the organisation has put together a research group whose work will be presented in this paper, but before we can find the cure to a problem we first need to identify the problem. This will be done in the two following parts of the text, where the theoretical background, the diverging views and perspectives of entrepreneurship education, and the different ways to measure outcome within the field, are presented. Theoretical background Although the interest in entrepreneurship education has grown explosively in the recent years, the field still lags behind advances made within entrepreneurship research (Honig, 2004; Rasmussen & Sørheim, 2006). Much curricular design is based on atheoretical asumptions, and entrepreneurship viewed as an activity is often divided into two fields, the science of entrepreneurship and the art of entrepreneurship (Henry, Hill & Leitch, 2005). The science part, which is often being viewed as more or less being the same as business management skills, is percieved as being teachable, whereas the art part is being mystified as something that individuals learn by practice, experience and reflection, and is therefore not suitable for educational institutions to adress (Timmons & Stevenson, 1985). As the field can be said to have its roots within American business schools and the field of strategic managment (Katz, 2003, 2008), planning, management and business skills have traditionally been the main focus for educational programmes, and it has often been taught by using case-based learning methods and business plan development activities (Honig, 2004). This traditional perspective has been challanged primarily by British researchers who argue that the focus should not be on how to perform a business start-up but on how to act and live as an entrepreneur (e.g. Gibb, 2002; Gibb & Hannon 2006). This research tradition argues that entrepreneurship cannot be viewed as a discipline, that thus should be targeting a smal and specific group (Gibb, 2002). Entrepreneurship education should instead focus on providing students with entreprising skills, which are useful to all students, and it should thus be embedded in every programme (Gibb, 2002). An assessment of the impact of learning in the field should be broad and include all positive outcomes, such as increased motivation and intresest in learning, resulting in better educational results and higher work satisfaction later on. Another perspective that lately has influenced actors within the field is Saras Sarasvathy’s concept of effectuation. By studying how expert entrepreneurs reasoned about how to make decisions under true uncertainty (Knight, 1921), she found that they used a different logic that was based on effectuation rather than causation. The expert entrepreneurs tended to ignore predictive methods which focus on future goals such as market research, competitive analysis and calculation of future gains, and instead relied on means-based, non-predictive control methods such as partnerships, affordable loss and leverage of contingencies. Instead of relying on the traditional notion that “to the extent that we can predict the future, we can control it”, which is typical of management methods (e.g. Kotler, 1991), the effectual logic postulates that “to the extent that we can control the future, we do not need to predict it” (Sarasvathy, 2001). In this sense, the “art” part of entrepreneurship is demystified and understood as something that can be investigated and codified and, thus, taught. These new perspectives have rapidly gained ground within the field and many educators have moved away from a strict focus on start-up activities and altered their learning goals to a more skill-based approach of their educational programmes, both on elementaryand secondaryas well as on teritary level. Little is known, though, about which effects and outcomes this has (Baron, 2009). Another debate within the field revolves around the level of focus that should be given to either theory or practice (Fiet, 2001a; 2001b). In a simplified manner, the field is often divided in three groups: education about, for or in entrepreneurship, which is said to depend on what target group the programme has (Henry et al., 2005). Knowledge lacks, though, about how these learning methods should be combined in a progressive manner though-out the whole educational system or in an extensive entrepreneurship programme. Many researchers within the field acknowledge that entrepreneurship educators need to apply a different type of didactics in order to teach entrepreneurial skills effectively (Gorman et al., 1997). Entrepreneurship in this perspective is viewed as a practical activity that requires doing, and educational programmes in the subjects should thus be based on action-based didactics with a functioning focus such as those advocated by the educational researchers Biggs and Tang (2007), with classical declarative learning as solely a complement (Johannisson, 1991; Politis, 2005). Still, much curricular design within the field relies first and foremost on classic declarative teaching methods, often being the result of institutional pressure from study boards (Honig, 2004). This short review of the theoretical background of the field clearly shows that both the disciplinary content and didactical methods are heavily debated and no clear consensus can be found regarding which approach to entrepreneurship education that should be applied to what type of students. There is a lack of studies that dig deeper into this problem. The studies that have been performed mainly focus on whether entrepreneurship education has a positive impact or not, and do not problematize the lack of consensus. This, in combination with institutional pressure from both study boards and the business system, has led to the result that many educational programmes within the field stick to classic teaching methods and curriculum design and do not acknowledge the latest advancements within the field. In the next pages of the text I will discuss different ways that researchers within the field have used to measure the outcomes of entrepreneurship education. Different measurements To understand what type of content, that is, theoretical focus and didactical methods, that works best we need to be able to assess the outcomes of entrepreneurship education effectively (Gartner & Vesper, 1994; Gorman et al., 1997; Matlay, 2008). A common way to measure the outcome of entrepreneurship education is to assess the impact it has on students’ behaviours, intentions and skills (Krueger & Brazeal, 1994; Kolvereid & Isaksen, 2006; Kickul, Gundry, Barbosa & Whitcanack, 2009). Behaviour is hard to assess because there is often a significant time-lag between graduation and start-up activity (Bird, 1988; Lent, Brown & Hackett, 1994). Most surveys therefore focus on either nascent behaviour (e.g. Reynolds et al., 2004), intentions (e.g. Krueger & Brazeal, 1994) or skills (e.g. Chen, Greene & Crick 1998). Especially entrepreneurial intentions have gained a growing interest in the last decade and many rigorous studies have been performed in which social psychological theories have been applied, foremost Ajzen’s (1991) Theory of Planned Behaviour (e.g. Tkachev and Kolvereid, 1999; Peterman and Kennedy, 2003; Fayolle, Gailly & Lassas-Clerc, 2006; Souitaris Zerbinati & Al-Laham, 2007; Graevenitz, Harhoff & Weber, 2010). From the viewpoint of a policy maker, the measurement of entrepreneurial intentions and entrepreneurial behaviour is of special interest (European Commission, 2008). However, it is hard to argue from a normative point of view that the learning goals of a university course should concern these outcomes (Karlsson & Moberg, 2011). An enhancement of entrepreneurial skills should, though, fit learning goals well, because the enhancement of knowledge and skills is education’s raison d’être (Biggs & Tang, 2007), and thus, a model that allows us to measure this should be preferred. The Self-Efficacy model, developed by Bandura (1977; 1997), has been widely used within many fields to assess the impact of different programmes, and it has been applied extensively by researchers within the field of entrepreneurship education (Mauer et al., 2009). It is a model that allows us to measure “people's judgments of their capabilities to organize and execute courses of action required to attain designated types of performances’ to the extent that their level of motivation, affective states and actions are based more on what they believe than on what is objectively true” (Bandura, 1986, p. 391; 1997, p.2). It thus fits the field of entrepreneurship education well; because it to some extent has been established that individuals’ perception of their abilities have a greater impact on their behaviour than actual abilities do (Krueger & Dickson, 1994). To get precise measurements, we need to develop context specific scales (Bandura 1977; 1997). Researchers within the field of entrepreneurship education have mainly used scales developed by Chen et al. (1998) and De Noble, Jung & Ehrlich (1999) (Mauer et al. 2009). Cox, Mueller & Moss (2002) have taken the development a step further and anchored their entrepreneurial self-efficacy scale to Stevenson, Roberts & Grousbeck’s (1985) entrepreneurial stage model. This measurement design fits educational programmes better, because it allows us to follow the progression and development of the students in a clearer manner. This model was later refined by McGee et al. (2009). At Cambridge, UK, researchers at the faculty of education have for many years used entrepreneurial self-efficacy scales developed by EHGI2 (Cooper & Lucas, 2006a; 2006b Mclellan, Barakat & Winfield (2010). The scales mentioned above are fairly biased towards a traditional view of entrepreneurial activity, though, and little of the latest advancements within the field have been included, with perhaps Mclellan et al. (2010) as an exception. Kickul, et al. (2009) found that individuals with a cognitive preference for analysis scored higher than individuals with an intuitive cognitive style on the Cox et al. (2002) scale. This is perhaps not the common view we have of the entrepreneur. As a model, it thus remains empirically underdeveloped (Kolvereid & Isaksen, 2006), and although it has been established that ESE is strongly connected to entrepreneurial intentions (De Noble et al., 1999; Krueger et al., 2000; Jung, Ehrlich, De Noble & Baik, 2001), little is known about which ESE construct that relates strongest to entrepreneurial intentions, behaviour and performance (Kickul, et al., 2009). Although there are some examples of studies that have a longitudinal design and use control groups (e.g. Peterman and Kennedy, 2003; Fayolle et al., 2006; Souitaris et al., 2007, Mclelland et al., 2009; Graevenitz et al, 2010), this is more the exception than the rule (Gorman el al, 1997, Matlay, 2008), and a literature review of the field shows that no study within entrepreneurship education, known to me, that applies social psychological variables so far have followed their subjects for a sufficient time period (Matlay, 2008). All of the five 2 The Education for High Growth Industries Enterprise Project. See http://www.cmi.cam.ac.uk/ for further details. ESE scales mentioned above (Chen et al., 1998; De Nobel et al., 1999; Cox et al., 2002; McGee et al., 2009; Mclelland et al., 2009) use a phrasing that is very biased towards entrepreneurship and business startup, which makes them unsuitable to use with nonentrepreneurship oriented control groups. Consequently, they need to be refined in order to generate reliable data to a quasi-experimental comparative change survey (Mohr, 1995). The challenge for a researcher who wishes to assess the impact of educational programmes will therefore be to develop non-biased but still context specific measurement variables, and design a survey that allows for a longitudinal tracking of the subjects for many years. In the next part of the text we will describe how this type of survey has been designed by the research group at the Danish Foundation for Entrepreneurship - Young Enterprise, in order to evaluate the entrepreneurial initiatives in the Danish educational system. Two longitudinal surveys As the discussion above illustrates, there are quite many challenges posed to an evaluation of entrepreneurship programmes. The time-lag issue is one, the role of education another. In this final part of our text we will describe how we have chosen to handle these problems, and why we have chosen this particular research design. Two longitudinal surveys will be performed. One that focuses on elementary and secondary level where we follow students at lower-secondary level, and one that focuses on tertiary level where we follow university students at six entrepreneurship programmes and six non-entrepreneurship programmes. Even though the research design for the surveys has many commonalities, there are some important differences in the set up and in the outcome analysis. In both surveys we strive to use a quasi-experimental design (Campbell and Stanley, 1966), with a focus on how the students develop entrepreneurial self-efficacy (Bandura, 1977) and how this relates to start-up intention and entrepreneurial behaviours (Krueger& Dickson, 1994). The structure of the field at elementary and secondary level is very different from tertiary level, though. At tertiary level, the educational programmes are structured in a way that makes them suitable for a classical impact analysis, such as advocated by for example Mohr (1995) for example. At elementary and secondary level this is, unfortunately, not the case, and we will therefore use different research designs in the surveys. The goal for both of the surveys is to build databases which allows for accurate analysis and rigorous research. The survey at tertiary level, which allows for more sophisticated measurement tools, will be described first. Tertiary level At the tertiary level, we are foremost interested in understanding why, not just if entrepreneurship education works or not. A formative impact analysis will thus be performed in which we will pay significant attention to each sub-objective (see Figure 3). The programme evaluation is designed in accordance to Mohr’s (1995) impact analysis and we apply the quasi-experimental design that was pioneered by Campbell and Stanley (1966) “Experimental and quasi-experimental designs for research”, and later refined by Cook and Campbell (1979) and Cook, Campbell & Peracchio (1990); with some modifications that will be described below. The activity of interest in our impact analysis is various methods and ways of teaching entrepreneurship education to master level students at universities, technical universities and business schools. As illustrated in the outcome line in figure 3, the outcome of this activity will be assessed by measuring what effect the education programmes has on the students’ level of entrepreneurial self-efficacy. This performance indicator is presumed to have a positive effect on the following outcomes to the right in Figure 3 below, but this relationship still needs further empirical evidence. We will, thus, also measure the impact of entrepreneurship education on entrepreneurial intentions, nascency, behaviour and performance. Insert Figure 3 here We are also interested in finding out which other different effects entrepreneurship education and entrepreneurial self-efficacy have on students’ career choices. We will therefore measure variables such as work satisfaction, employment position, salary and wealth, in later stages. We will with the beginning of September 2011 collect primary data on approximately 500 master level students of six entrepreneurship programmes (experiment group) and six non-entrepreneurship programmes (control group) at three Danish universities and business schools will be followed for seven years (at the least). A classic comparative change design in a quasi-experiment are structured as follows (Mohr, 1995): A/C: X₁ₑ T Yₑ A/C: X₁c Yc This longitudinal design is subjected to various threats to internal validity, such as selection, history, spuriousness and contamination (Mohr, 1995). The threat of history, that something else besides the treatment (T) accounts for all or part of the change over time (Mohr, 1995; 67), is eliminated with the use of control groups (c). Eventually significant events will have the same impact on both of the groups (e and c). This is the main reason why we use this design. In our survey we are dealing with self-selecting groups. This is in conflict with the use of quasi-experimental design, because it generates selection bias and spuriousness. Our experiment group (e) and the control group (c) can be suspected to differ significantly regarding levels of initial entrepreneurial self-efficacy (X₁), the so called Pselection variables in programme assessment (Mohr, 1995), but also on other variables which are not expected to be affected by entrepreneurship education (T), the so called Q-selection variables. By the use of pre-test (X₁) and post-test (Y) we can measure the change in our two groups (e and c), and thus, the impact of the treatment (T). The problem is to control for the other variables that might affect the outcome (Y). These Q-selection variables can also be expected to differ significantly between the two groups due to the self-selection. In entrepreneurship research these variables are fairly known, though, and we will control for variables such as parents’ occupational status, entrepreneurial intentions, entrepreneurial experience, work experience, demographics such as age and gender, and educational background. Selection biases will thus be turned into selection effects, and the spuriousness will be eliminated in a large extent. The contamination problem that is a threat in all quasiexperimental designs (Mohr, 1995), will in our survey be controlled for simply by asking if the students have experienced any event that has had a significant impact on their entrepreneurial attitudes which cannot be related to their educational activities. How the treatment affects the students can also be expected to vary depending on initial characteristics. As illustrated in the equation below, we suspect that the level of initial entrepreneurial self-efficacy (X₁i), will affect how the educational process (Ti) affects them. Yi = ₁X₁i + TTi +₂X₁iTi + ui The outcome (Yi) is thus not only dependent on the effect (₁) of the treatment (Ti). A high initial level of entrepreneurial self-efficacy (X₁i) will probably lessen the effect of the treatment and thus render ₂ negative. ui is the disturbance term assumed to have mean of zero and to be randomly distributed across the subjects and is the Y intercept. Out of the twelve programmes (six belonging to the experiment group and six to the control group), six will target management students, four will target engineering students and two will target humanities students. During the first two years, when the students attend their programmes, they will be asked to fill in a questionnaire three times: before they start the programme, after the first year and after graduation. They will then be asked to fill in the questionnaire three more times: one year after graduation where the focus will be on nascent entrepreneurial behaviour; three years after graduation where focus will be on actual behaviour, and then, finally, five years after graduation where the focus will be on performance (see figure 3 for a graphic illustration of the time-line). Elementary and secondary level To assess the impact of entrepreneurship education on elementary students, in detail, we would have to follow them from the first day of school, which would be a very time consuming and impractical project. We have therefore decided to select students that are to begin their second year at lower secondary level (the same year the turn fifteen). Students at this level have their elementary schooling fresh in mind and are just one year from a very important decision: are they going to continue to upper secondary level or not? We will select 400 students at lower-secondary level, from 20 classes and 7 schools in Denmark, and analyse their experience with entrepreneurship education during elementary school. A pre-test that measures their initial entrepreneurial self-efficacy will allow us to analyse the effects of entrepreneurship education during the last year at elementary level. The students will be asked to fill in the questionnaire annually, which allows for an analysis of their experience with the field, their entrepreneurial progression and their decisions. With regarding to their decisions, special attention will be paid to their choice of school. In figure 4 a description of the outcome model for the survey is presented. At each stage the students can choose to either drop out from the educational process and get a job, become entrepreneurs, become unemployed, or choose to study further. In Denmark a political goal is that 95% of students at lowersecondary level should continue on to secondary level. It is therefore of interest to analyse if entrepreneurship education at elementary and on lower-secondary level increases the students’ propensity to continue on to secondary level, and whether entrepreneurship Figure 3: The activityand outcome-line of our programme assessment with all sub-goals included. Figure 4: Outcome model of the longitudinal survey of students at lower-secondary level. Primary level Secondary level Tertiary level Employment or start-up Unemployment Various types of entrepreneurship education Enhancement of students’ Entrepreneurial Self-Efficacy Enhancement of students’ Entrepreneurial Intentions More students chose to pursue a career as selfemployed Growth Enhancement of students’ Entrepreneurial Nacency PLANNING Causation VISIONARY ADAPTIVE TRANSFORMAIVE Effectuation Figure 5: A categorisation model that place perspectives according to their emphasis on prediction or control (Wiltbank et al. 2006) POSITIONING CONSTRUCTION Emphasis on prediction Emphasis on control Low High High Low TRADITIONAL UNIVERSITY EDUCATION ACTION AND REFLECTION ??? ACTION LEARNING Emphasis on functioning methods Emphasis on declarative methods High Low Low High Figure 6: A categorisation model that place courses according to their emphasis on declarative methods or functioning methods