Kentucky GSEG General Supervision Enhancement Grant Caroline

Kentucky GSEG General Supervision Enhancement Grant © Caroline Gooden, Kentucky Early Childhood Data System, 2007

l l The GSEG Outcomes and Process Overview of Continuous Assessment Guide and implications for First Steps How KEDS is working for Preschool Behind the Scenes – Ensuring valid and reliable data © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Two Major Goals l Pilot the Standards and Continuous Assessment Guide to Determine…. – – – l how well CBA address the standards and outcomes how well children with disabilities fit into the framework supports needed to implement in programs state-wide Pilot the 3 National Early Childhood Outcomes for Part C and 619 (Preschool) of IDEA © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Continuous Assessment Guide l Diagnostic section designed to support Evaluation activities as outlined in IDEA with respect to eligibility – Norm-referenced standardized instruments (908 KAR 2: 130 & 635 of IDEA) © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Continuous Assessment Guide l Classroom/Instructional section designed to support Assessment activities as outlined 908 KAR 2: 130 with progress monitoring and program planning – – Criterion referenced or curriculum based tools “an ongoing process of observing a child's current competencies (including knowledge, skills, dispositions and attitudes) and using the information to help the child develop further in the context of family and caregiving and learning environments” © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Continuous Assessment Guide l Purpose of Classroom/Instructional Assessments 1. 2. 3. Identify the individualized needs of children to inform intervention and instructional planning, Develop individualized plans (e. g. , IFSPs) Inform families and other team members of child developmental status. © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Why CR/CB over Norm-Referenced l l Gather information for all the major developmental domains or dimensions of development. Tools were reviewed for their ability to: – – l Be developmentally and culturally valid; Yield comprehensive and useful information Allow for multiple methods from multiple sources without jeopardizing reliability or validity Connect to important learning represented in the Kentucky EC Standards, Provide outcomes that match family goals and cultural preferences. It is widely accepted that the tools included in the screening and diagnostic assessment section of CAG do not provide information sufficient for the purposes of classroom/instructional program planning and this is supported in 908 KAR 2: 130. © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Objectives of GSEG 1. 2. 3. 4. Determine current use of the standards and assessment guide Help programs develop assessment systems for data on outcomes Help programs use assessment data for child and program improvement Document assessment process for replication © Caroline Gooden, Kentucky Early Childhood Data System, 2007

GSEG Pilot Sites l Anderson Co. Preschool – – l Growing Together – – l Urban private early childhood center ~180 children, ages 0 -5 years 2 Rural/urban early intervention providers – l Rural public preschool center ~ 300 children ages 3 and 4 ~20 children, ages 0 -3 All serve children with & without disabilities © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Kentucky’s Approved Assessment Tools l Preschool Child Observation Record (COR) l l Child Observation Record for Infants & Toddlers (COR I-T) l l Creative Curriculum l l OUNCE Scale Work Sampling System (WSS) Brigance Inventory of Early Development-II (IED-II) Assessment, Evaluation, & Programming System (AEPS) Early Learning Accomplishment Profile (E- LAP) l l l Learning Accomplishment Profile– 3 (LAP– 3) Transdisciplinary Play Based Assessment (TPBA) Hawaii Early Learning Profile (HELP) Carolina Curriculum for Preschoolers with Special Needs ( CCPSN) Carolina Curriculum for Infants and Toddlers with Special Needs (CCITSN) © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Kentucky’s Approved Assessment Tools l l l l Preschool Child Observation Record (COR) Child Observation Record for Infants & Toddlers (COR I-T) Creative Curriculum OUNCE Scale Work Sampling System (WSS) Brigance Inventory of Early Development-II (IED-II) Assessment, Evaluation, & Programming System (AEPS) Early Learning Accomplishment Profile (ELAP) l l l Learning Accomplishment Profile– 3 (LAP– 3) Transdisciplinary Play Based Assessment (TPBA) Hawaii Early Learning Profile (HELP) Carolina Curriculum for Preschoolers with Special Needs ( CCPSN) Carolina Curriculum for Infants and Toddlers with Special Needs (CCITSN) © Caroline Gooden, Kentucky Early Childhood Data System, 2007

GSEG Activities l Year 1 (2004 -05) – – – l Training on standards and assessment guide Selection of assessment tools Training on assessment tools by publishers Year 2 (2005 -06) – – – Collection of data on all children served by pilots Training on assessment tools Developing data platform for correlations to standards and to OSEP child outcomes © Caroline Gooden, Kentucky Early Childhood Data System, 2007

What is KEDS? l Kentucky Early Childhood Data System l Provides a consistent platform for collection of data from: – – Diverse programs and providers across the state Diverse numbers of approved assessment tools © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Why KEDS? l Supports improved instruction through ongoing, continuous assessment – l l Gathering information in the context of everyday routines and activities to obtain a representative picture of children’s abilities and progress Improves outcomes for children Documents child outcomes for Office of Special Education © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How Does Continuous Assessment Work for Early Intervention Providers l l l Embedded with RBI Supported by the Consultative Model Requires ongoing communication among team members © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How Does Continuous Assessment Work for Early Intervention Providers l l l Providers continue to record observations, field notes, gather “evidence” following a home visit Providers continue to communicate with other members of the child’s team and family members on progress (IFSP goals and outcomes) Providers record mastery of skills as appropriate on assessment protocol. © Caroline Gooden, Kentucky Early Childhood Data System, 2007

l Example with Carolina Curriculum © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How Does Continuous Assessment Work for Early Intervention Providers l Providers record mastery of skills as appropriate on assessment protocol. © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How KEDS was Designed for Preschool l Ongoing meetings with OSEP, ECO and KDE staff (accountability and early childhood) to determine: – – how the system should integrate with state data system who should enter data and at what level what formats should be used by those in the field how training and support will be provided for assessments and data transmission © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How KEDS works for Preschool Log In Accounts created at two levels l Preschool Coordinators have access to all children in their program l Teachers have access to children on their caseload © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How KEDS works for Preschool l l Child level data is imported into the system from KDE STI – child tracking system © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How KEDS works for Preschool l Preschool Coordinators establish the teacher accounts and assign children to the teacher © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How KEDS Works for Preschool l l Teachers conduct quality assessments on each student: Teachers tally data twice a year – – l November May Coordinators monitor teacher assessments and data entry for accuracy © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How KEDS Works for Preschool l Teachers verify imported data and complete additional demographic data on each child © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Verification of Data l l l l l Child Unique Identifier Number Date of Birth Name Gender Ethnicity IEP Enrollment Date Exit Date District School © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Additional Data l Does child have an IEP – l Child Status – – – l Developmental Delay, Speech/Language, or Severe at-risk eligible (up to 150 % of poverty) over income LEP Assessment used with child and format of the assessment (on-line, diskette, or paper-and-pencil) © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Assessment Recording Formats l l l Online systems - direct administrative access is provided to UK Diskette or CD systems - electronic submission of information via Keds Online Paper-and-pencil versions – – Publisher permission to establish data entry system Transmit data via Excel via Keds Online © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Sample Paper – Pencil Entry System © Caroline Gooden, Kentucky Early Childhood Data System, 2007

What Happens Behind the Scenes Methodology l l Align individual assessment items to KY standards and benchmarks then to OSEP Outcomes Activity – – Conceptual framework 4 phase validity process © Caroline Gooden, Kentucky Early Childhood Data System, 2007

KY’s Conceptual Framework © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Benefits of the System l l Allows flexibility for providers to choose a tool that best meets the needs of individual children (with state guidance) Puts the focus on implementing high quality assessments that provide immediate data to support intervention Immediately responsive to changes in required outcome reporting at the federal level Allows for reporting across multiple programs © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How Does KEDS Work? © Caroline Gooden, Kentucky Early Childhood Data System, 2007

4 -Phase Validation Process l Expert Panel – – l l l General cross walk of all CB assessment tools to benchmarks and outcomes Detailed cross walks of all assessment items to benchmarks for preschool tools Statistical analysis of child data on benchmarks, standards and OSEP outcomes (Correlational and Factor Analysis) for three pilot assessments Concurrent validity studies of 2 assessments using BDI-II Teacher/Provider ratings and rankings of child progress on standards, benchmarks and OSEP outcomes © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Sample Crosswalk Arts & Humanities Standard 1: Participates and shows interest in a variety of visual art, dance, music and drama experiences Benchmark 1. 1: Develops skills in and appreciation of visual arts Developmental Continuum 1)Uses a variety of media and materials for exploration (e. g. paint, glue, threedimensional materials, technology, etc. ). 2)Uses a variety of art forms, elements and materials for representing people, places, and things in the environment. 3)Observes and responds to artwork produced by other individuals and/or cultures. Example Behaviors 1. Kenny chooses to paint at the easel three days in a row. He paints with one color, covering the entire page. 2. Sally uses finger paint to make swirls on black paper. 3. Shavon uses glue and ribbon on paper. 4. The teacher puts Q-tips at the art table and Monica dips them in paint to make spots on paper. 5. Ben uses a computer program to create a picture then glues on a tree-twigs picture frame after printing the picture. 1. Olivia uses markers to draw eyes, nose, and a mouth on a paper plate. 2. Jarrad paints a picture of his dog at the easel. 3. Maggie sticks leaves she has gathered outside to a tree trunk shaped from play-dough. 4. Trina builds her house with Popsicle sticks and glue. COR I. 1 J. 1, 2 l. 2 -5 J. 3 -5 X. 2 None CC WSS LAP-3 None 37. F through III None NOTE: FOR ALL WSS ITEMS, ALL ITEMS REFER TO 3’S AND 4’S UNLESS OTHERWI SE NOTED VI. A. 3 None VI. B. 1 FM 4, 9, 10, 16, 19, 25, 30, 32, 35, 39; PW 1, 2, 6, 8, 13, 16, 17 FM 34 PW 14, 29 None AEPS None FM B 2. 1 None Brigan ce None C-3. 6, 3. 7 None © Caroline Gooden, Kentucky Early Childhood Data System, 2007 CCPSN Help

Statistical Validation of Items l Data reduction – l Establishing reliability – l Are the scales measuring the content of the benchmark with repeated use? Ensuring internal consistency – l Based on crosswalk analysis, single items pulled together to create scales, a confirmatory factor analysis conducted to ensure items are measuring the same dimension Do the scales from the crosswalk “hang” together? Predictive validity – Regression analysis © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Statistical Validation of Items l Conduct sample tests* – – l Measures of central tendency Outlier analysis Test individual assessment tools – – – Measures of central tendency Outliers within tools Investigate age bands - If no age bands are available for individual assessments, investigate continuum * No group norms are available for B-3 tool selection, therefore in depth analysis of the Kentucky Early Childhood Data System, 2007 © Caroline Gooden, sample for B-3 will be the main focus

Statistical Validation of Items l Convert individual assessment scores to common metric – Allows for comparison between assessments that was not possible before conversion l – – Conversion formula x- µ Z=σ Conversion based solely on the individual assessment tool, and once converted, can be compared to any other z score Investigate the relationship of items and scales to KY benchmarks © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Statistical Validation of Items l l Align KY benchmarks to OSEP outcomes Report entry data on Outcomes based on OSEP’s five category reporting system © Caroline Gooden, Kentucky Early Childhood Data System, 2007

Final Conversations and Questions l l l Have the funding to support the system design for First Steps and other CHFS early childhood programs Need to complete the validation work with 0 – 3 assessment tools Need to identify specific program issues and considerations for implementation, specifically…. © Caroline Gooden, Kentucky Early Childhood Data System, 2007

How KEDS can be designed to support Early Intervention l Continued meetings and discussions with OSEP, ECO and EI staff to determine: 1. 2. 3. 4. 5. How the system should integrate with state data system – specifically CBIS Who should enter data and at what level What formats should be used by providers How training and support will be provided – have trainers available for most assessments Data transmission processes – have basic systems in place © Caroline Gooden, Kentucky Early Childhood Data System, 2007