SEDA IMPACT EVALUATION WESTERN CAPE SOUTH AFRICA Varsha

SEDA IMPACT EVALUATION WESTERN CAPE (SOUTH AFRICA) Varsha Harinath (the dti) Francisco Campos (World Bank) Finance and Private Sector Development IE Workshop Dakar, Senegal February 2010

Content Background Objective of the impact study Impact Evaluation methodology Process and timelines Challenges and lessons Next steps Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

South Africa background Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Unemployment is high: 19% for males 25% for females South Africa has identified the development of small, micro and medium enterprises (SMMEs) as means to mitigate the unemployment crisis in the country, as well as to ensure a more equitable distribution of income (Department of Trade and Industry, 1995).

SEDA Small Enterprise Development Services Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 The Small Enterprise Development Agency (seda) offers non-financial business development services to small businesses in South Africa. The agency, overseen by the Department of Trade and Industry (the dti), was created in 2004 and operates in all 9 provinces of South Africa. Business assessments, training as well as assistance with business plans, promotional material, technology systems, registering process are among a range of non-financial services that seda offers to clients. Eighty percent of its support to enterprises is focused on businesses with less than 20 employees.

Objective of the impact evaluation Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 The impact evaluation seeks to test if seda services in the Western Cape Province lead to improved business knowledge and practices by SMMEs and hence have a positive impact on growth, competitiveness, sustainability and formality.

Key outcomes Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Outcomes Indicators 1. Growth 1 a. Increase in turnover 1 b. Increase in number of employees 2. Competitiveness 2 a. Increase in productivity 2 b. Increase in net profits 2 c. Improvement in operational margins 2 d. Increase in number of customers 3. Sustainability 3 a. Survival rate 4. Formality 4 a. Registered 4 b. Certification of tax payment

Methodology Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 To measure the impact of seda services, we would like to observe the performance of the same enterprise with seda’s intervention and without seda’s intervention over the same period in time………. Both states of the world do not exist at the same time!

Methodology Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 We “construct” these states of the world by creating an “experiment”, whereby we randomly assign a representative group of small business to either: A treatment group - those that receive seda’s services. A control group - those that do not receive seda’s services. The treatment and control group is obtained from a representative sample of formal and informal small businesses in the Western Cape Province of South Africa. To date, there is no extensive list of formal and informal small businesses in the Western Cape. Hence, for this study a multi-stage sampling technique was used , including the sampling of households and the identification of small enterprises within those households.

Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Methodolody Populations of SMMEs SMME Sample (906 businesses) Treatment group (453 businesses) Average outcome for treatment group _ Control group (453 businesses) Average outcome for control group __ Impact of seda’s intervention

Process 1. 2. 3. 4. 5. Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Sample identification - ~9, 500 sampled households surveyed to identify SMMEs owned by household members Baseline survey - ~900 SMME owners surveyed on business characteristics, financial accounts, support from government, registration status Intervention - seda contacts the businesses in the treatment group to complete an office visit, assess their needs and provide appropriate services. Follow-up survey - information collected on outcomes for treatment and control group 12 months after the intervention Analysis - impacts measured with difference in differences

Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Timelines 2008 -2009 2010 2011 Set Objectives Roll out of intervention Follow-Up survey Design Methodology Baseline survey reports Impact evaluation final report Budget and Implementation Plan Design Questionnaire Sampling Baseline survey Roll out of intervention

Baseline reports Baseline data can be useful for decision-making by assessing the status of entrepreneurship, as well as differences between sub-groups and their corresponding needs Growth Series: Profile of businesses Profile of entrepreneurs Awareness of SEDA Informality Access to finance Performance (financials) Labor Markets Challenges to growth Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010

Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Baseline data Variable Treatment Control Difference Male business owners (%) 53. 0% 0 pp Non-white business owners (%) 80. 6% 79. 7% 0. 9 pp High-school business owners (%) 50. 0% 52. 5% -2. 5 pp Average business owner 40. 9 40. 2 0. 7 Average number of employees 5. 2 5. 1 0. 1 Wholesale & retail (%) 54. 1% 53. 4% 0. 7 pp Unregistered (%) 72. 9% 72. 1% 0. 8 pp 453 0 Number of observations

Challenges and lessons Challenges Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Lessons First rigorous IE study for the dti: buy-in for a large study is difficult Political head lobbying for IE fast tracks the process of buy-in IE can at first glance look expensive (especially if multi-stage sampling is done) Multi-stage sampling improves external validity and provides rich data at the household and enterprise level Pilot encouragement design, inviting treatment group to an event by seda at cinemas in the Western Cape yielded low turn-out rates and even lower take up rates of seda services Significant brainstorming with programme managers & marketing teams is crucial and flexibility when implementing an encouragement may be necessary. Encouragement can be a challenge, especially when invitees are dispersed geographically. Randomised outreach intervention techniques are also usually more successful

Next steps Impact Evaluation South Africa WB FPD Workshop, Dakar February 2010 Office visit intervention for the Western Cape to be completed by March 2010. Following the interest of this study in the Western Cape, the dti was also requested by seda to do an impact evaluation for the agency on a national scale. The baseline data for national seda study is piggybacking of a national small business study that is presently being conducted for South Africa. Follow–up survey for Western Cape and for the National study in 2011.

SEDA Impact Evaluation Finance and Private Sector Development IE Workshop Dakar, Senegal February 2010