2012 Texas Assessment Conference November 29 -30 2012

2012 Texas Assessment Conference November 29 -30, 2012 Criss Cloudt, Shannon Housson, and Ester Regalado Texas Education Agency | Office of Assessment and Accountability Division of Performance Reporting

2013 Accountability System Design

Objectives for Today’s Session 3 Questions that will be addressed in today’s session: § Why is Texas is developing a different framework for public school accountability? § What is a performance index? § How will performance indexes be used in the new rating system? § When will the various indicators be evaluated in the performance indexes? § How will Texas ensure that individual student groups are not ignored? § What are the additional distinction designations that will be developed for 2013?

Objectives for Today’s Session 4 Questions that cannot be addressed in today’s session: § Questions related to the STAAR assessment program that will be addressed in other sessions. § Certain details about the indicators or indexes that are more fully explained in the technical description document. § What the accountability targets will be for 2013, since final decisions about the system framework, indicators, and targets will be released by the commissioner in spring 2013.

2013 Accountability Development 5 § In 2009, the Texas Legislature passed House Bill (HB) 3, mandating the creation of an entirely new accountability system for 2013. § TEA produced a plan for implementing these changes in the House Bill (HB) 3 Transition Plan, published in December 2010. § In 2012, TEA began working with advisory committees to develop the new rating and distinction designations systems required by HB 3.

2013 Accountability Development 6 Accountability Technical Advisory Committee (ATAC) § In October 2011, the commissioner asked superintendents and ESC directors to submit nominations for educators to serve on the ATAC. § 156 nominations were received, 27 members were selected for the ATAC. § Since March, work groups of ATAC members have met to discuss, research, and propose solutions to key issues. § The ATAC will meet in February 2013 to review system safeguards and develop their recommendations on the accountability targets for 2013 and beyond.

2013 Accountability Development 7 Accountability Policy Advisory Committee (APAC) § In October 2011, the commissioner requested nominations from educator organizations, business organizations, and educational service centers for the APAC. Twenty-nine members were selected for the APAC, representing various educational and business organizations and legislative offices. § APAC members have met twice with the ATAC to develop recommendations on the accountability framework as well as rating levels and labels. § The APAC will meet again in early March 2013 to review the ATAC proposal on accountability targets and develop their recommendations to the commissioner on the accountability targets for 2013 and beyond.

2013 Accountability Goals 8 § Improving student achievement at all levels in the core subjects of the state curriculum. * § Ensuring the progress of all students toward achieving Advanced Academic Performance. * § Closing Advanced Academic Performance level gaps among groups. * § Closing gaps among groups in the percentage of students graduating under the recommended high school program and advanced high school program. * § Rewarding excellence based on other indicators in addition to state assessment results. § The committees adopted a set of Guiding Principles that will be used to inform the accountability development process. * These goals are specified in Chapter 39. 053(f) of the Texas Education Code.

Indicators: General 9 § The following indicators must be used in determining accountability ratings: STAAR grades 3 -8 English, STAAR grades 3 -5 Spanish, STAAR End-of-Course (EOC) assessments* including retests, Dropout Rates grades 9 -12 or district completion rates, and High School Graduation Rates. § Grade 11 TAKS performance must also be included in the 2013 ratings. * EOC results for students enrolled below grade 9 must be combined with assessment results for other students in the same grade.

Indicators: Assessment 10 § The assessment indicators must include evaluation of: Level II performance, and for students who do not meet the Level II standard, progress toward the Level II standard. Level III performance, and for students who do not meet the Level III standard, progress toward the Level III standard. § Level III performance cannot be evaluated for acceptable/unacceptable ratings in 2013. § Assessment indicators must combine performance across grades for each subject area. § Indicators must be based on information that is disaggregated by race, ethnicity, and socioeconomic status.

Previous State Accountability System 11 § In previous system from 2004 - 2011, districts and campuses were required to meet criteria on up to 25 separate assessment measures (five subjects X five student groups) and on up to 10 dropout and high school completion measures in order to achieve the Academically Acceptable rating. Thus, a district or campus could receive an Academically Unacceptable rating due to poor performance on a single measure even if all other measures indicated high performance. § The new state accountability system under development will allow accountability on a large number of measures, without the rating being dependent on a single measure.

Previous State Accountability System (2011) 12

Previous State Accountability System (2011) 13

Proposal for Accountability Framework 14 § The ATAC committee members developed the proposed framework based on the requirements of HB 3 and their expectation that the new accountability system should: Be comprehensive in nature, Improve student performance for every child, Focus on narrowing the performance gap between historically disadvantaged and advantaged students, Measure indicators that move a school/district toward higher performance, and Direct resources for improvement.

Proposal for Accountability Framework 15 Primary Factors Considered for Selecting Performance Index Framework §Accountability System Goals and Guiding Principles §Statutory Requirements of House Bill 3 (2009) Focus on Postsecondary Readiness Inclusion of Student Progress Emphasis on Closing Achievement Gaps §New STAAR program with EOC-based assessments for middle schools and high schools §Lessons learned from previous Texas public school accountability rating systems (1994– 2002 and 2004– 2011) §Successful models used by other states, e. g. , CA, CO, FL, GA, KY, LA, OH, OK, NC, and SC

STAAR Indicators –Proposed 2014 Performance Index Framework 16

Indicators for Proposed 2014 Performance Index Framework 17

Performance Index Framework 18 What is a Performance Index? § With a Performance Index each measure contributes points to an index score. § Districts and campuses are required to meet one accountability target— the total index score. § With a Performance Index, the resulting rating reflects overall performance for the campus or district rather than the weakest performance of one student group/subject area.

Performance Index Framework 19 § Features of a Performance Index Each indicator contributes points to the index score. Performance on all measures is included, but no single indicator can be the sole reason for a lower rating. Resulting rating reflects overall performance rather than the weakest areas. Multiple indexes can be used in the framework to ensure accountability for every student. Any number of indicators and student groups can be added to the system without creating additional targets for campuses and districts to meet.

Performance Index Framework 20 For 2013 and beyond, a framework of four Performance Indexes will include a broad set of measures that provide a comprehensive evaluation of the entire campus or district. Student Achievement Index II Index Postsecondary Readiness Index 4 Accountability System Closing Performance Gaps Index 3 Student Progress Index 2

Index 1: Student Achievement 21 Sample Campus STAAR Performance Results All African Amer. Indian Asian Hispanic Pacific Islander White Two or More ELL Special Ed. Econ. Disadv. Met Level III 25 20 * 2 0 * 7 4 1 7 40 Met Level II 25 0 * 2 20 * 6 3 9 4 40 Level I 50 0 * 2 20 * 9 5 10 11 0 100 20 * 6 40 * 22 12 28 22 80 % Met Level III (Advanced) 25% 100% * 33% 0% * 32% 33% 4% 32% 50% % Met Level II (Satisfactory) 25% 0% * 33% 50% * 27% 25% 32% 18% 50% % Met Level II or III 50% 100% * 67% 50% * 59% 58% 36% 50% 100% STAAR Reading Number Tested

Index 1: Student Achievement 22 Sample Campus STAAR Performance Results % Met Level II or III STAAR Performance Results % Met Growth Standard (L 2 or L 3) STAAR Reading 50% STAAR Reading 49% STAAR Mathematics 38% STAAR Writing 45% STAAR Writing 40% STAAR Science 25% STAAR Science TBD STAAR Social Studies 83% STAAR Social Studies TBD

Index 1: Student Achievement 23 STAAR Percent Met Level II Standard (2013 and Beyond) § STAAR Grades 3 -8 English and Spanish at final Level II performance standard for assessments administered in the spring; § EOC at final Level II performance standard for assessments administered in the spring and the previous fall and summer; § STAAR Grades 3 -8 and EOC Modified and Alternate at final Level II performance standard; § TAKS 2013: Grade 11 results at Met Standard performance 2014 and beyond: None

Index 1: Student Achievement 24 STAAR Percent Met Level II Standard (2013 and Beyond) §Combined over all subject areas: Reading, Mathematics, Writing, Science, and Social Studies §Student groups: All Students only § Students below Grade 9 taking EOC courses: Administrative rules for the assessment program will require that students be administered the EOC test rather than the STAAR grade level assessment for the subject.

Index 1: Student Achievement 25 Index 1 Construction Since Index 1 has only one indicator, the Total Index Points and Index Score are the same: Index Score = Total Index Points is the percentage of assessments that met the final Level II Standard. Each percent of students meeting the final Level II performance standard contributes one point to the index. Index scores range from 0 to 100 for all campuses and districts. Example Reading Students Met Level II 50 Students Tested 100 Index Score Mathematics + 38 Writing + 19 Social Studies Science + 10 + 19 Total = % Met Level II 136 45% + 100 + 42 + 40 + 23 = 45 305 45

Index 1: Student Achievement 26 Index 1 Construction Reading Students Met Level II 50 Students Tested 100 Index Score Mathematics + 38 Writing + 19 Social Studies Science + 10 + 19 Total = % Met Level II 136 45% + 100 + 42 + 40 + 23 = 45 305 45

Index 2: Student Progress 27 STAAR Percent Met Growth Standard (2014 and Beyond) §The STAAR growth measure will not available in time for use in the 2013 accountability ratings. Since the growth measure must be finalized based on the spring 2013 STAAR results, it is not possible to set the 2013 accountability targets for Index 2 prior to the release date of the 2013 ratings. §This graphic is an example of a transition table that divides the three STAAR performance levels (Level I, Level II, and Level III) into performance bands. The number of bands within a performance level may differ for the final growth measure adopted.

Index 2: Student Progress 28 Index 2 Construction §Ten Student Groups Evaluated: 1. 2. 3. All Students English language learners (ELLs) Students with Disabilities Race/Ethnicity: 4. African American 5. American Indian 6. Asian 7. Hispanic 8. Pacific Islander 9. White 10. Two or More Races

Index 2: Student Progress 29 Index 2 Construction §By Subject Area: Reading, Mathematics, and Writing ¢Credit given for meeting the student progress measure requirements for: Progress toward Satisfactory performance (Level II) or Progress toward Advanced performance (Level III)

Index 2: Student Progress 30 Index 2 Construction All African Amer. STAAR Reading % Met Growth Standard 49% STAAR Mathematics % Met Growth Standard 45% STAAR Writing % Met Growth Standard 36% STAAR Science % Met Growth Standard * * * * * STAAR Social Studies % Met Growth Standard * * * * * Indicator Amer. Indian Two or More ELL Special Ed. Total Points Max. Points 58% 40% 35% 56% 377 800 48% 52% 45% 30% 50% 366 800 30% 40% 134 400 * * * 877 2000 Asian Hispanic 36% 60% 43% 31% 65% Pacific White Islander Total Index Score (total points divided by maximum points) * Science and Social Studies will be evaluated if growth measures are developed for these subjects. 28% 44

Index 2: Student Progress 31 Index 2 Construction All African Amer. STAAR Reading % Met Growth Standard 49% STAAR Mathematics % Met Growth Standard 45% STAAR Writing % Met Growth Standard 36% STAAR Science % Met Growth Standard * * * * * STAAR Social Studies % Met Growth Standard * * * * * Indicator Amer. Indian Two or More ELL Special Ed. Total Points Max. Points 58% 40% 35% 56% 377 800 48% 52% 45% 30% 50% 366 800 30% 40% 134 400 * * * 877 2000 Asian Hispanic 36% 60% 43% 31% 65% Pacific White Islander 28% Total Index Score (total points divided by maximum points) * Science and Social Studies will be evaluated if growth measures are developed for these subjects. 44

Index 3: Closing Performance Gaps 32 STAAR Weighted Performance (2013 and beyond) § Two approaches to evaluating progress toward closing performance gaps: § Compare the performance of the lower performing student group to the performance of a higher performing student group over time, or Compare the performance of the lower performing student group to an external target, the performance target that is tied to the statutory and accountability goal that Texas will be among the top ten states in postsecondary readiness by 2020 with no significant achievement gaps by race, ethnicity, or socioeconomic status. Index 3 takes the second approach through a weighted performance index.

Index 3: Closing Performance Gaps 33 STAAR Weighted Performance (2013 and beyond) § Index 3 ensures that individual student groups are not ignored within the performance index framework. § Credit based on weighted performance: Level II satisfactory performance (2013 and beyond) One point for each percent of students at the final Level II satisfactory performance standard. Level III advanced performance (2014 and beyond) Two points for each percent of students at the final Level III advanced performance standard.

Index 3: Closing Performance Gaps 34 Index 3 Construction §Assessment results include all assessments that are included in the Index 1 student achievement indicator. §By Subject Area: Reading, Mathematics, Writing, Science, and Social Studies. §Student Groups Socioeconomic: Economically Disadvantaged Lowest Performing Race/Ethnicity: The two lowest performing race/ ethnicity student groups on the campus or district (based on prior-year assessment results). §The STAAR weighted performance rate calculation must be modified for 2013 because STAAR Level III advanced performance cannot be included in the indicator until 2014.

Index 3: Closing Performance Gaps 35 Index 3 Construction STAAR Weighted Performance Rate Economically Disadvantaged Lowest Performing Total Points Race/Ethnic Group - 1 Race/Ethnic Group - 2 Maximum Points Reading Weighted Performance Rate 150 50 200 400 600 Mathematics Weighted Performance Rate 125 100 90 315 600 Writing Weighted Performance Rate 80 90 125 295 600 Science Weighted Performance Rate 120 40 90 250 600 Social Studies Weighted Performance Rate 50 40 80 170 600 1430 3000 Total Index Score (total points divided by maximum points) 48

Index 3: Closing Performance Gaps 36 Index 3 Construction STAAR Reading Weighted Performance Rate Example Calculation for Reading Number of Tests Performance Results: Level II Satisfactory Number Percent Economically Disadvantaged Lowest Performing Race/Ethnic Group - 1 Race/Ethnic Group - 2 80 40 20 50% 40 50% 0 0% 50 (50% x 1) 0 (0% x 1) Level III Advanced (two point credit) 100 (50% x 2) 0 (0% x 2) 200 (100% x 2) 150 50 200 600 20 100% Weighted Results: Level II Satisfactory (one point credit) 400 0 0% Level III Advanced Number Percent Maximum Points 20 40 50% Total Points Reading Weighted Performance Rate 36

Index 4: Postsecondary Readiness 37 STAAR Percent Met Level III § 2014 and beyond (Level III performance is not included in accountability in 2013) §Assessment results include all assessments evaluated in Index I §Combined over All Subjects: Reading, Writing, Mathematics, Science, and Social Studies

Index 4: Postsecondary Readiness 38 STAAR Percent Met Level III §Eight Student Groups Evaluated: 1. All Students Race/Ethnicity: 2. African American 3. American Indian 4. Asian 5. Hispanic 6. Pacific Islander 7. White 8. Two or More Races

Index 4: Postsecondary Readiness 39 Graduation Rates §High School Graduation Four-year Graduation Rate or Five-year Graduation Rate (or Annual Dropout Rate if no graduation rate) Ten Student Groups Evaluated: 1. 2. 3. All Students English language learners (ELLs) Students with Disabilities Race/Ethnicity: 4. African American 5. American Indian 6. Asian 7. Hispanic 8. Pacific Islander 9. White 10. Two or More Races

Index 4: Postsecondary Readiness 40 Recommended High School Program/Advanced High School Program §RHSP/AHSP indicators are calculated for campuses and districts for which a graduation rate is calculated. §Eight Student Groups Evaluated: 1. All Students Race/Ethnicity: 2. African American 3. American Indian 4. Asian 5. Hispanic 6. Pacific Islander 7. White 8. Two or More Races

Index 4: Postsecondary Readiness 41 Index 4 Construction §Graduation Score: Combined performance across the graduation and dropout rates for Grade 9 -12 Four-Year Graduation Rate for All Students and all student groups OR Grade 9 -12 Five-Year Graduation Rate for All Students and all student groups, whichever contributes the higher number of points to the index. One of the two rates is used, not a mix of Four-Year Graduation Rate for one student group and Five-Year Graduation Rate for another student group. §RHSP/AHSP Graduates for All Students and race/ethnicity student groups §STAAR Score: STAAR Percent Met Level III for All Students and race/ethnicity student groups (2014 and beyond) §For high schools that do not have a graduation rate, the annual dropout rate and STAAR Level III performance contribute points to the index. For elementary and middle schools, only STAAR Level III performance contributes points to the index.

Index 4: Postsecondary Readiness 42 Index 4 Construction Indicator All African Amer. Indian Asian Hispanic Pacific White Islander Two or More ELL Special Ed. Total Points Max. Points 4 -year graduation 84. 3% rate 78. 8% 91. 6% 86. 0% 44. 2% 69. 8% 533. 5 700 5 -year graduation 85. 1% rate 78. 8% 80. 0% 92. 1% 84. 0% 48. 9% 77. 5% 546. 4 700 RHSP/AHSP 76. 4% 83. 6% 83. 0% 325. 7 400 872. 1 1100 82. 7% Graduation Total Graduation Score (graduation total points divided by maximum points) 2014 and beyond: STAAR All Subjects* % Met Level III 29% 16% 40% 23% 79 38% 36% 182 600 STAAR Score (STAAR total points divided by maximum points) 30 Index Score (average of Graduation Score and STAAR Score: 79 + 30 / 2 = 55) 55

Index 4: Postsecondary Readiness 43 Index 4 Construction Indicator All African Amer. Indian Asian Hispanic Pacific White Islander Two or More ELL Special Ed. Total Points Max. Points 4 -year graduation 84. 3% rate 78. 8% 91. 6% 86. 0% 44. 2% 69. 8% 533. 5 700 5 -year graduation 85. 1% rate 78. 8% 80. 0% 92. 1% 84. 0% 48. 9% 77. 5% 546. 4 700 RHSP/AHSP 76. 4% 83. 6% 83. 0% 325. 7 400 872. 1 1100 82. 7% Graduation Total Graduation Score (graduation total points divided by maximum points) 2014 and beyond: STAAR All Subjects* % Met Level III 29% 16% 40% 23% 79 38% 36% 182 600 STAAR Score (STAAR total points divided by maximum points) 30 Index Score (average of Graduation Score and STAAR Score: 79 + 30 / 2 = 55) 55

Overview of Proposed Performance Index Framework 44

Overview of Proposed 2014 Performance Index Framework (Sample Campus) 45

Possible Safeguards 46 Apply Safeguards to Specific Performance Indexes as needed: § Ensure reporting system disaggregates performance by student group, performance level, subject area, and grade, § Implement interventions focused on specific areas of weak performance, § Apply minimum performance requirements or performance floors, § Apply a limit on proficient results to STAAR Modified and STAAR Alternate, § Apply Participation Rate Targets, § Ensure Leaver Data Quality, or § Incorporate Grade 7 – 8 Annual Dropout Rate.

Additional Topics 47 Pending Issues For Consideration § Evaluation of the four indexes to produce single accountability rating for campus or district, § Rating levels and labels, § Application of system safeguards, § Evaluation of alternative education campuses, § Transition Issues between 2013 and 2014, § Inclusion of a performance measure for English Language Learners (ELLs), and § State and federal reporting.

Federal Accountability for 2013 48 § As indicated in the September 6, 2012, To The Administrator Addressed correspondence from the commissioner, the agency plans to submit a waiver request to the United States Department of Education (USDE) in January or February 2013. § The waiver will include a request to use the new state accountability system to evaluate campuses and districts in place of federal Adequate Yearly Progress (AYP) evaluations. § If denied, augment the proposed state accountability system to meet federal requirements. § If denied, use components (Reading and Mathematics) of the proposed performance index developed for state accountability to meet federal requirements.

Academic Achievement Distinction Designations

Academic Achievement Distinction Designations 50 Distinction Designation Indicators §Eighteen indicators will be used to determine outstanding academic achievement and will vary by type of campus and by subject. §Indicators evaluated include performance at the STAAR Level III (Advanced) standard for selected grades and subject areas in elementary and middle schools, and SAT/ACT, PSAT/PLAN, and AP/IB participation and performance for high schools. §For details, refer to the AADDC framework document at http: //ritter. tea. state. tx. us/perfreport/account/2013/20120625 mtg/frame. pdf

Academic Achievement Distinction Designations 51 Distinction Designation Framework Steps § The proposed framework for distinction designations uses four steps to determine a campus distinction. Step 1 identifies a campus comparison group for each campus and calculates campus performance for each distinction indicator by subject. Step 2 compares the performance of the target campus to the performance of the campuses in the comparison group for each indicator. For example, Campus A is in the top 25% of campuses among a 40 campus comparison group on a particular distinction indicator.

Academic Achievement Distinction Designations 52 Distinction Designation Framework Steps (continued) Step 3 generates a single outcome by subject for each campus. For example, Campus A achieved the top 25% in three of the six distinction indicators that were evaluated for the campus. Step 4 is a statewide evaluation of campus outcomes in order to identify the top campus distinction designations by subject. For example, campuses that outperformed their peers on 50% or more of the mathematics distinction indicators evaluated receive an academic distinction in mathematics.

Academic Achievement Distinction Designations 53 Recommended Targets § Campuses in the top 25% (top quartile) of their campus comparison group in Step 2 are eligible for a distinction designation for that subject area. § Elementary and middle school campuses in the top quartile on at least 50% of their eligible measures receive a distinction designation for that subject area. § High schools in the top quartile on at least 33% of their eligible measures receive a distinction designation for that subject area.

Calendar

Calendar 55 November 15, 2012 Academic Excellence Indicator System (AEIS) released on the TEA Secure Environment (TEASE) November 28, 2012 AEIS released on the TEA public website. § Reporting requirements for the School Report Card are suspended. § The Public Education Grant (PEG) list for 2013 -14 is planned to be developed based on TAKS-equivalent standards for Grades 3 -8. § Final 2012 AYP Status released § Preview of NCLB School Report Card data (Part I only) December 2012

Calendar 56 Public release of the 2011 -12 Texas NCLB Report Card January 2013 STAAR results are available for Grades 3 -8 from test contractor. March 2013 Commissioner releases final decisions on the state rating system by end of March 2013. Late Spring 2013 2012 STAAR accountability performance results released. August 8, 2013 Accountability ratings and AADD released on August 8, 2013.

2013 Accountability Development Website 57 Current postings to the 2013 development website include: § Overview of Proposed Performance Index Framework that provides a brief description of the features and safeguards of the proposed four performance indexes in the new accountability system. § Overview of Assessment Indicators in the Previous State Accountability System provides a visual overview of the previous system. http: //ritter. tea. state. tx. us/perfreport/account/2013/index. html § Meeting outcome summaries for the APAC, ATAC, and AADDC meetings are posted online at: http: //ritter. tea. state. tx. us/perfreport/account/2013/materials. html

2013 Accountability Development Website 58 § By next week, a “To The Administrator Addressed” letter will be sent to superintendents to invite comments on proposals made by the advisory groups. § Updated proposals will be posted online for educator review and comment at the 2013 Accountability Development page: http: //ritter. tea. state. tx. us/perfreport/account/2013/index. html § Click on the Recommendations link, scroll to the bottom of the page, and click on the Comments link.

Resources 59 § 2013 Development Site http: //ritter. tea. state. tx. us/perfreport/account/2013/index. html § Frequently Asked Questions About Adequate Yearly Progress (AYP) http: //ritter. tea. state. tx. us/ayp/faq. html § Performance Reporting Home Page http: //www. tea. state. tx. us/perfreport § Adequate Yearly Progress (AYP) Home Page http: //www. tea. state. tx. us/ayp § Performance Reporting Email performance. [email protected]. state. tx. us § Division of Performance Reporting Telephone (512) 463 -9704