What Should You Consider When Talking About Technical Assessments? John M. Townsend Tennessee Board of Regents NACTEI 28 th Annual Conference Boise, Idaho
Why are you assessing? Most important question – For Perkins IV? – For Program Improvement? – For student attainment?
If For Student Attainment At what level is appropriate? – Entry level – Mastery level – Industry certification level
When do you assess? End-of-program End-of-course End of Pathway Cluster related
Norm- or Competency-referenced • Norm-referenced – Comparison among individuals – Demonstrates changes of students as a group • Competencyreferenced – Attainment of an individual – Measurable learning outcomes based upon industry standards
Buy or Develop? 1. Standardized instruments 2. Centralized state-developed instruments 3. Local assessments recognized by state
Performance or Cognate • Performance – What the student should be able to do – Checklist – Limited number of areas can be assessed in specified time • Cognate – What the student should know – Test-items – Many areas can be assessed in a specified time
Building an Assessment • Adopt – seek items or assessments that you can adopt as is • Adapt – items or assessments that can be modified to fit your standards • Build – items or assessments when others cannot be found
Table of Specifications (blueprint) • Determine competencies to be assessed • Determine competency groups (3 – 10) • Determine percentage of each cognitive typology per group
Development Questions • Is cost and time worth benefit of test? • Who should develop the assessment? • Test security issues?
Multiple-choice elements: Stem Correct response Foils or distractors
Fixed-form v Variable-form Fixed-form • High reliability • Needs high security Variable-form • Good reliability • Lesser security needed • Limited number of test • Requires a large items needed number of test items
Assessment Administration • Decide the media for administration • Identify who should conduct the assessment • Determine when the assessment activities should occur
Reporting issues: 1. Grading of the assessment 2. Assessment improvement needs 3. Reports needed
Assessment Review • Validity – the degree to which certain inferences can be made from test scores. • Reliability – the degree of consistency between two or more measures of the same thing.
Item & Assessment Analysis Find a good statistician, psychometrician or educational psychologist !
Ethical Issues in Testing Code of Fair Testing Practices in Education by the Joint Committee on Testing Practices, American Educational Research Association, American Psychological Association & National Council on Measurement in Education
Legal Issues • Accommodations for members of special populations • Access to the test-items by parents and others • Utilization of assessments in relation to passing or graduation
Postsecondary Issues: “Each eligible agency shall identify in the State plan core indicators of performance for career and technical education students at the postsecondary level that are valid and reliable, and that include, at a minimum measures of each of the following: ”
Postsecondary Issues: “(i) Student attainment of challenging career and technical skill proficiencies, including student achievement on technical assessments, that are aligned with industry-recognized standards, if available and appropriate. ”
Postsecondary Issues: OVAE • OVAE has denied the utilization of GPAs as an indicator • OVAE preference is a third-party developed and administered assessment (e. g. NOCTI or centralized state assessment unit) • OVAE has not defined what they will accept; burden of proof is on State
Postsecondary Issues: Systemic • Local colleges seek to serve local business/industry – diversity of curriculum • Cannot require anything beyond what the state requires for graduation – in TN most certification assessments are postprogram • Accreditation processes often evaluates the assessment process of a program of study
Postsecondary Issues: Challenge to OVAE • Goes against Perkins IV Sec. 8 “establishment” clause • An unfunded mandate – many postsecondary receive little Perkins IV funds • In pedagogy andragogy, a single assessment is not sound
Postsecondary Issues: Reference Wiseley, Chuck. (August 31, 2007) California Community Colleges NSWG response to the: Proposed DRAFT Guidelines for State Phase-in of Perkins IV Core Indicators 2 S 1 and 1 P 1 Technical Skills Assessments.
Contact Information: Dr. John M. Townsend Executive Director, Workforce Development Office of Academic Affairs Tennessee Board of Regents 1415 Murfreesboro Pike, Suite 350 Nashville, TN 37217 615 -366 -4428 John. Townsend@tbr. edu
Скачать презентацию What Should You Consider When Talking About Technical
06d7e3c6a6e3f023d2e2c57cdcbf5eaa.ppt