Technical Skill Assessments Overarching Issues Common Concerns Intended

Technical Skill Assessments Overarching Issues Common Concerns Intended State Approaches Critical Definitions Data Quality Institute Savannah, Georgia May 19, 2007

Overarching Issues: Topics n Don Hilber, DTI – Technical Skills Assessment Study Group: advice for DQI agenda / associated research – Results of Questions for States: issues of greatest importance; stage of development; intentions in transition plan & beyond – Framework for approaching tech skills indicators n Bob Sheets, U of IL – Recap recommendations from 2005 -2006 DQI – Perkins IV realities – what still applies and what does not – Facilitated discussion; Q & A

Technical Skill Assessment Study Group n n n Spun off of Next Steps Working Group 25 states, OVAE, associations and consultants – about 50 get notices Accountability / data experts and curriculum / assessment expert; some administrators Eleven calls starting Jan 10, 2007 Invited private companies to address group and take questions Informed DQI; desired other research = knowledge of others, (deferred) inventory

Study Group: Most Active States Alaska Secondary North Carolina Secondary Arizona Secondary Ohio Both Arkansas Both Oklahoma Secondary Florida Secondary Oregon Secondary Hawaii Secondary South Carolina Secondary Indiana Postsecondary Tennessee Postsecondary Iowa Secondary Texas Both Michigan Secondary Utah Both Minnesota Postsecondary Virginia Secondary New York Postsecondary Wisconsin Both

Experts Invited to Address Technical Skill Attainment Study Group n NOCTI (Foster +) ACT (Klemme +) VTECS (Mc. Cage, Hattaway) XPAND Corp (Osman, Butler) OSU (Mahlman +) ABG / Skills USA (Mc. Clelland +) n Others Post-DQI ? n n n

Questions Asked of States n n n Level of Experience with Technical Skill Assessments – Secondary and Postsecondary Details from Experienced States Importance of 16 Issues Intentions as to: Methods, Types of Skills, When Assessment Will Occur Numerator / Denominator in Transition Plan – if Different than Guidance

Experience Among States Using Technical Skill Assessments Secondary Postsecondary Very Experienced; no need to make alterations 5 1 Somewhat Experienced; will need to make alterations 5 2 Experienced – with some methods or program areas 8 14 12 10 5 7 17 17 Just Starting to Gain Experience or Familiarity No Experience Whatsoever Did Not Submit

Alternative to Guidance 23 of 35 States (65%) Following Guidance on Technical Skill Attainment for Both Secondary and Postsecondary n Secondary Exceptions = 7 States n – 2 completion and 2 GPA (permanent or interim? ) – 3 specific changes – year or numerator n Postsecondary Exceptions = 9 States – 2 completion and 1 GPA (permanent or interim? ) – 2 waiting for data from colleges – 3 specific changes to student definitions, year or numerator

Issues of Importance – Top Tier Definition of Technical Skill n Determining Phase-in / Interim Reporting n Defining / Specifying Validity and Reliability n Data Collection and Reporting n End of Course / End of Program Assessments n Development / Administrative Costs n Value of Assessment to Student n Student Definitions (Concentrator) n Suitability of Assessment to Perkins n Alignment to Standards All of these 60%+ Critical or Very Important n

Issues of Importance – Second Tier n n n Distinguishing Among Foundational, Generic or Occupational-Specific Skills (may have confused w/ “Definition of Technical Skill”) Ability of and Devices for States to Collaborate and Pool Resources Dealing with Local v State Development or Selection of Assessments Developing a Seamless System for Secondary and Postsecondary Relationship Among Perkins Indicators Clarifying Clusters, Pathways and Programs of Study

Technical Skill Attainment: A Framework n WHO needs to be measured? n WHAT is to be measured? n WHEN should measurement occur? n HOW can measurement be accomplished? In general order of most uniformity to most flexibility

WHO Needs to be Measured? n Student Definitions: – Concentrator – Numerator / Denominator (also WHEN) Presented: Day One (OVAE) Discussed: Day One (Definitions) and Overarching Issues (Tech Skill Attainment Measure)

WHAT is to be Measured? n Definition of Technical Skill n Relationship of Technical Skill Attainment to Other Perkins Indicators: Academic, Completion n Distinguishing Among Foundational, Generic and Occupational Skills n Clarifying Clusters, Pathways and Programs of Study Presented: Day One (OVAE) and Two Discussed: Breakouts One (S & PS) and Two (Similar States)

WHEN Should Measurement Occur? n Data Collection and Reporting n Determining the Phase-In Period and Interim Reporting n End of Course versus End of Program n Developing a Seamless System for Secondary and Postsecondary (also HOW) Presented: Day Two - Panels Discussed: Breakouts One (S & PS) and Two (Similar States)

HOW Can Measurement be Accomplished? n Defining Validity / Reliability n Alignment to Standards n Suitability to Perkins n Value to the Student n Local v State Development / Selection n Development / Administrative Costs n Pooling / Sharing Resources Presented: Day Two - Panels Discussed: Breakouts One and Two

DQI: Past and Present n Quick Review of 05 -06 Data Quality Institute n Perkins IV Impact on: – Student Definition = Concentrator – Secondary Technical Skill Attainment Numerator and Denominator – Postsecondary Technical Skill Attainment Numerator and Denominator n Clarification on Overarching Issues – Student Definitions (Recap of Day One) – Phase-In Period and Interim Reporting: How? – Definition of Technical Skill • Versus Academic Skill and Completion (Secondary) • Versus Credential, Certificate or Degree (Postsecondary)