Performance Management Policy Highlights Workforce Innovations 2005 July

Performance Management Policy Highlights Workforce Innovations 2005 July 12 and 13, 2005

Workforce System Performance Improving Performance Results § Reporting performance is a fundamental element of improving customer services and good public administration § Performance information should be easily understood by all customers, stakeholders, and operators of the workforce investment system § Establishing common measures and standardizing customer data collection improves the comparability and understanding of performance results § Ensuring the accuracy and timeliness of performance results is necessary for demonstrating system integrity and value 2

Workforce System Performance Benefits of Common Performance Measures § Focus on the core purposes of the workforce system– employment for adults and skill gain for youth § Break down barriers to integration resulting from different definitions, data and reports for each workforce program § Resolve questions raised by GAO and other oversight agencies regarding the consistency and reliability of data § Reduce confusion among our customers and stakeholders who want to know about results 3

Workforce System Performance Common Performance Measures Common measures focus on key results for the adult and youth programs; other management information collected will continue to be valuable § Management and oversight of programs will continue to require information that is relevant to each program § Information on statutorily-required performance measures will continue to be collected and reported § Additional information about participants, services and outcomes must supplement the story told by the common measures 4

Common Performance Measures Three common performance measures apply to programs serving adults: § Entered Employment § Employment Retention § Earnings Increase Three common measures apply to programs serving youth: § Placement in Employment or Education/Advanced Training § Attainment of a Degree or Certificate § Literacy and Numeracy Gains 5

Accountability WIA Adult and Dislocated Worker, Wagner-Peyser and Trade Programs States have negotiated performance goals for WIA programs for PY 2005 and PY 2006 WIA Adults Entered employment Employment retention Earnings change Employment and credential rate WIA Dislocated Workers Entered employment Employment retention Earnings change Employment and credential rate Note: In addition, states have negotiated performance goals for WIA for Employer customer satisfaction and Job seeker customer satisfaction PY 2005 will be a baseline year for these performance measures for W-P employment services. Goals will be negotiated beginning with PY 2006. W-P employment services Entered employment Employment retention Earnings change States are responsible for achieving National Goals for the Trade program. Trade Entered employment Employment retention Earnings change 6

Accountability WIA Youth Programs States have negotiated performance goals for WIA programs for PY 2005 and PY 2006 Older Youth Entered employment Employment retention Earnings change Employment and credential rate Younger Youth Skill attainment rate Diploma attainment rate Retention rate States have not negotiated performance goals for the youth common measures, however states will be expected to begin reporting on these measures All Youth- Common Measures Placement in employment or education/ advanced training (PY 05) Attainment of a degree or certificate (PY 05) Literacy and numeracy gains (PY 05/06) States must collect and report the first two measures beginning in PY 05. States are encouraged to begin data collection and reporting for the literacy/numeracy gains measure, although it will not be required until PY 06. 7

Common Performance Measures Key Definitions: § Participant – An individual who is determined eligible to participate in the program and receives a service funded by the program in a physical location. * § Exit – Participant does not receive a service funded by the program or a partner program for 90 consecutive calendar days and is not scheduled for future services. * States and grantees are encouraged to consider as participants those individuals who receive services that are available through the Internet and are not accessed through a physical location. 8

Adult Common Measures Summary of Major Policy Changes TEGL 15 -03 TEGL 28 -04 Old Policy New Policy ADULT MEASURES Entered Employment Rate v. No change in methodology from WIA or TEGL 15 -03 v. Permits grantees to use wage records and supplemental data Employment Retention Rate v. Measurement at 1 st, 2 nd, and 3 rd quarters v. No change in methodology from TEGL 15 -03 v. Permits grantees to use wage records and supplemental data Earnings Increase Measure Separated into two distinct indicators of performance 1. Percent change pre- to postprogram Qtr 1 2. Percent change post-program Qtr 1 to Qtr 3 v. Replaces the two-part earnings increase measure identified in TEGL 15 -03 with a single Six Month Earnings Increase measure (pre- to post-program) v. Wage records are the only data source v. Grantees that do not have access to wage records can use supplemental data as an interim means of reporting Program Efficiency videntified as a core common measure v. Eliminates program efficiency as a reportable measure at the grantee level 9

Adult Common Measures Entered Employment Measure: Of those who are not employed at the date of participation: Number of participants employed in the 1 st quarter after the exit quarter divided by the number of participants who exit during the quarter § Same as current WIA Adult and Dislocated Worker entered employment definitions § Includes only those not employed at entry § Employment status at participation is based on information from the individual, not wage record data 10

Adult Common Measures Employment Retention Measure: Of those who are employed in the 1 st quarter after the exit quarter: Number of participants employed in both the 2 nd and 3 rd quarters after the exit quarter divided by the number of participants who exit during the quarter § Additional measurement point (2 nd quarter) compared to current WIA Adult, Dislocated Worker, and TAA retention rates § Includes all participants employed in the 1 st quarter regardless of their employment status at participation § Employment in the 1 st, 2 nd, and 3 rd quarters does not have to be with the same employer 11

Adult Common Measures Earnings Increase Measure: Of those who are employed in the 1 st quarter after the exit quarter: [Total earnings in the 2 nd and 3 rd quarters after exit minus total earnings in the 2 nd and 3 rd quarters prior to participation] divided by the number of participants who exit during the quarter § Same as current WIA Adult Earnings Gain measure, but different than current wage replacement rate for WIA Dislocated Worker and TAA programs § Completely new measure for W-P & VETS programs § Designed as a six month pre- and post-program look at earnings change 12

Adult Common Measures Earnings Increase, Cont. § Calculation yields a dollar amount, not a rate § Negative dollar amounts are likely for WIA Dislocated Worker and TAA customers § Wage records will be the only data source for this measure • Grantees that do not have access to wage records may use supplemental data as an interim means of reporting on the measure § States and grantees may exclude participant earnings from each of the total quarterly amounts where the following occurs: • False or erroneous wage data has been reported to the grantee • Individual received severance pay or other earnings attributable to a termination from employment 13

Youth Common Measures Summary of Major Policy Changes TEGL 15 -03 TEGL 28 -04 Old Policy New Policy YOUTH MEASURES Placement in Employment or Education v Excluded individuals in secondary school at exit from performance calculations v Attainment of Degree or Certificate v Excluded individuals in secondary school at exit from performance calculations v Literacy and Numeracy Gains v Included all in-school and out-of-school youth who were basic skills deficient v Program Efficiency v identified as a core common measure v v v Individuals in secondary school at exit are now included in the measure Permits grantees to use wage records and supplemental data Individuals in secondary school at exit are now included in the measure Clarifies the definition of Diploma Includes degrees/certificates awarded by triballycontrolled colleges and universities In-school youth are now excluded from the measure Provides additional guidance on providing reasonable accommodations for testing youth with disabilities Eliminates Program Efficiency as a reportable measure at the grantee level 14

Youth Common Measures Placement in Employment or Education Measure: Of those who are not in post-secondary education, employed, or the military at the date of participation: Number of participants in employment, military, or enrolled in postsecondary education and/or advanced training/occupational skills training in the 1 st quarter after the exit quarter divided by the number of participants who exit during the quarter § Includes individuals in secondary school at participation and exit § Similar to current Younger Youth Retention measure in that it recognizes the same outcomes – improvement from Older Youth Entered Employment § Tighter definition of advanced training 15

Youth Common Measures Attainment of Degree or Certificate Measure: Of those who are enrolled in education at participation or any time during the program: Number of participants who attain a diploma, GED, or certificate by the end of the 3 rd quarter after the exit quarter divided by the number of participants who exit during the quarter § Similar to a combination of the Younger Youth Diploma Rate and Older Youth Credential Rate § Tighter definition of certificate; does not have employment as a component like the current credential measure § Diplomas, GEDs, or certificates can be obtained while a person is still receiving services or at any point prior to the end of the 3 rd quarter after exit Individuals in secondary school at exit will be included in the measure § Education refers to those in secondary, post-secondary, adult education, or any other organized program of study § Individuals in secondary school at participation and exit will be included in the measure 16

Youth Common Measures Attainment of Degree or Certificate § Diploma – any credential that the state education agency accepts as equivalent to a high school diploma § Certificate – awarded in recognition of attainment of measurable technical or occupational skills necessary to gain employment or advance within an occupation • Based on standards developed or endorsed by employers • Certificates awarded by workforce investment boards or in recognition of attaining generic pre-employment or work readiness skills are not included in this definition 17

Youth Common Measures Attainment of Degree or Certificates can be awarded by the following institutions: • State educational agency • Institutions of higher education (sec. 102 of Higher Education Act) • Professional, industry, or employer organization or a product manufacturer or developer • Registered apprenticeship program • Public regulatory agency • Department of Veterans Affairs • Office of Job Corps • Tribally-controlled colleges and universities 18

Youth Common Measures Literacy and Numeracy Gains Measure: Of those out-of-school youth who are basic skills deficient: Number of participants who increase one or more educational functioning levels divided by [the number of participants who have completed a year in the program plus the number of participants who exit before completing a year in the program] § In-school youth are excluded from this measure § Individuals who are not basic skills deficient are excluded from this measure § Includes individuals with learning disabilities, with accommodations and alternate testing options § Creates specific focus on improving basic skills rather than including basic skills among work readiness and occupational skills training § Does not mean work readiness and occupational skills no longer important (impact of those skills will be seen in the Placement in Employment and Education Rate) § Standard for success much higher than with less-defined skill attainment rate 19

Youth Common Measures Literacy and Numeracy Gains § To be included in the numerator, an individual must advance one or more Adult Basic Education (ABE) or English as a Second Language (ESL) functioning levels § Gain can occur in literacy or numeracy—can pre-test at different levels in each category § Educational functioning levels are consistent with Adult Basic Education’s National Reporting System (NRS) § Initial assessment test must occur within 60 days following the date of participation in the program § Individuals should be post-tested by the end of one year of participation and compared to pre-test results obtained during the initial assessment § Participants only included in measure a second time if they complete a second full year of participation 20

Youth Common Measures Literacy and Numeracy Gains § Must use one of NRS cross-walked assessment tests or equate alternate test to NRS scale § Tests cross-walked with ABE and ESL levels include: • Comprehensive Adult Student Assessment Instrument (CASAS) • Test of Adult Basic Education (TABE) • Adult Basic Learning Examination (ABLE) • Student Performance Levels for ESL • Basic English Skills Test (BEST) for ESL • Work. Keys (for the top 3 ABE levels) 21

Implementation § ETA received emergency approval in April 2005 to incorporate common measures into the WIA, W-P, VETS, and TAA program reporting systems • Remember, where possible the definitions for common performance measures are replacing “old” definitions § These changes are effective on July 1, 2005 for WIA, W-P, and VETS programs, and October 1, 2005, for the TAA program § ETA will be publishing three 60 -day notices on these reporting changes in July to provide opportunity for public comment as part of the full-approval process of these revisions § ETA is considering alternative definitions for the earnings measure 22

Highlights of Changes to Reporting WIA Performance Reporting System § Quarterly Report • • • § Annual Report • • § Collects aggregate counts of self-service participants, the number of participants in training, and youth (by age and school status) Collects current WIA youth measures plus common measures Separate reporting of common measures for Adults, Dislocated Workers, and NEG participants Rolling four-quarter reporting methodology to align with W-P employment services Reporting System Eliminated customer satisfaction reporting on a quarterly basis Collects similar aggregate counts of participants and exiters as the Quarterly Report New report table to collect youth common measures WIASRD • • • Collects additional data on participant services and outcomes to calculate common measures States do not submit WIASRD files on participants who receive only selfservices and informational activities States must report co-enrollment of individuals in Wagner-Peyser or TAA programs 23

Highlights of Changes to Reporting Wagner-Peyser Reporting System (ES 9002 and VETS 200) § § § Aligns performance reporting cohorts with WIA reporting cohorts Eliminates all customer satisfaction reporting Collects aggregate counts on participants, exiters, and those receiving workforce information services Collects aggregate outcomes for the adult common measures Collects aggregate data on services to Homeless Veterans (VETS 200 C report only) Eliminates reporting on each category of veteran (e. g. , campaign, disabled) by age grouping Trade Act Participant Report (TAPR) § § Collects data elements to calculate common measures Collects information on participant services as authorized under the TAA Reform Act of 2002 24

Questions or Comments? Questions about performance issues? Want updates on performance? Please contact us at: §ETAperforms@dol. gov or 202 -693 -3031 §www. doleta. gov/performance Office of Performance and Technology §Karen Staha, staha. karen@dol. gov Office of Workforce Investment §Evan Rosenberg, rosenberg. evan@dol. gov (Youth) §Gail Eulenstein, eulenstein. gail@dol. gov (Adult/Dislocated Workers) §Tony Dais, dais. anthony@dol. gov (W-P employment services) Office of National Response §Terry Clark, clark. terry@dol. gov (TAA) VETS §Pamela Langley, langley. pamela@dol. gov 25