Скачать презентацию Use SP Xpress to analyze students performance Скачать презентацию Use SP Xpress to analyze students performance

1c5bf288c1123ad79031b948c8c2f6be.ppt

  • Количество слайдов: 36

Use SP Xpress to analyze students’ performance & evaluate the question items setting St. Use SP Xpress to analyze students’ performance & evaluate the question items setting St. Anthony’s School 16 Jan 2013 1

Background information Assessment Curriculum 2 Background information Assessment Curriculum 2

Enhance the effectiveness of learning and teaching Curriculum Learning Teaching Assessment 3 Enhance the effectiveness of learning and teaching Curriculum Learning Teaching Assessment 3

Evaluation after Assessment Are the questions too difficult or too easy? What are the Evaluation after Assessment Are the questions too difficult or too easy? What are the follow-up actions? Why did students choose A, not B, C or D? Have the students learned? Have they studied hard? 4

Assessment Data SP Xpress MCI performance Chart MCI item P-value Chart 5 Assessment Data SP Xpress MCI performance Chart MCI item P-value Chart 5

Data – Good to start Student MCI Item MCI Cronbach Alpha Graph shape 6 Data – Good to start Student MCI Item MCI Cronbach Alpha Graph shape 6

2 Modified Caution Indexes Student Modified Caution Index Students’ Performance 0> MCI> 0. 3 2 Modified Caution Indexes Student Modified Caution Index Students’ Performance 0> MCI> 0. 3 Item Modified Caution Index Question items’ Performance 0> MCI> 0. 3 7

25 27 54 2012 -13 P. 4 1 st Reading Exam 58 16 18 25 27 54 2012 -13 P. 4 1 st Reading Exam 58 16 18 Student MCIx. P Chart 1. 0 30 63 67 71 MCI vs Performance of Student 35 0. 9 60 22 0. 8 26 Student Performance 0. 7 37 43 0. 6 6 28 0. 5 41 0. 4 51 70 0. 3 2 0. 2 12 38 0. 1 3 5 0. 0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 Student Modified Caution Index 0. 7 0. 8 0. 9 8 1. 0 14 32 11 8

Student MCIx. P Chart 2012 -13 P. 4 1 st Reading Exam 9 Student MCIx. P Chart 2012 -13 P. 4 1 st Reading Exam 9

More focused strategies Type A Type B The teaching and learning are suitable for More focused strategies Type A Type B The teaching and learning are suitable for them. Encourage them to keep on. Know the difficult questions Miss the simple questions Careless mistakes Don’t rely on luck Type D Type C Adjust the teaching content Raise their learning motivation Need more consolidation Answering techniques Careless mistakes Focus on the foundation Don’t rely on luck 10

Item MCI x. P Chart 2012 -13 P. 4 1 st Reading Exam 11 Item MCI x. P Chart 2012 -13 P. 4 1 st Reading Exam 11

Item MCI x. P Chart Need improvement Difficult questions Problem questions 12 Item MCI x. P Chart Need improvement Difficult questions Problem questions 12

MCI item P-value chart Item Type 1112 P. 3 Final 1213 P. 4 First MCI item P-value chart Item Type 1112 P. 3 Final 1213 P. 4 First Exam A 20 17 B 1 2 C 1 0 D 3 7 According theof items Total no. Item Modified Caution Index, most of the 26 set are normal. 25 13

Cronbach Alpha (Reliability) Cronbach Alpha 0 – 0. 3 0. 7 -0. 9 Reliability Cronbach Alpha (Reliability) Cronbach Alpha 0 – 0. 3 0. 7 -0. 9 Reliability > 0. 9 high low acceptable 14

2011 -2012 P. 3 First Exam Blue line Students Red line Items Purple line 2011 -2012 P. 3 First Exam Blue line Students Red line Items Purple line Average Slightly Easy 15

2012 -2013 P. 4 First Exam Blue line Students Red line Items Purple line 2012 -2013 P. 4 First Exam Blue line Students Red line Items Purple line Average Appropria te 16

Which question items should be improved? 17 Which question items should be improved? 17

Which question items are difficult to our students? 18 Which question items are difficult to our students? 18

SP Xpress analysis on ‘ 2011 -2012 Primary 3 Third Exam Paper’ SP Xpress SP Xpress analysis on ‘ 2011 -2012 Primary 3 Third Exam Paper’ SP Xpress analysis results: Type A: 20 items Type B: 1 item Type C: 1 item Type D: 3 items Let’s examine one item from each category (Type B, C, D) 19

Exam item example: Type B On which floor of the centre, can Grandpa find Exam item example: Type B On which floor of the centre, can Grandpa find presents for Jack? A. 1/F B. 2/F C. 3/F D. 4/F 20

Possible reasons for item being Type B: Item is not difficult but requires amendments Possible reasons for item being Type B: Item is not difficult but requires amendments because many students couldn’t answer it. Possible reasons: Item design flaws Not close to what teacher taught in class Students lack specific training in reading comic strips 21

Teacher examined the Type B item Students found it difficult because … Need to Teacher examined the Type B item Students found it difficult because … Need to process and connect many ideas all at the same time Question Comic strip Which floor? 1/F, 2/F, 3/F, 4/F floor directory … of the Centre … Star Shopping Centre … can Grandpa find presents … We can buy all the gifts … presents? presents robots toys(4/F) … can Grandpa … ‘We’ refers to ‘grandparents’ 22

Recommended solutions Increase students’ chances in reading English comic strips in classroom teaching or Recommended solutions Increase students’ chances in reading English comic strips in classroom teaching or leisure reading Teacher needs to illustrate more clearly in connecting ideas Teacher needs to teach more explicitly which means more visual demonstrations 23

Exam item example: Type C In which step does Jack’s mum add ‘soy sauce’ Exam item example: Type C In which step does Jack’s mum add ‘soy sauce’ and ‘wasabi’ to the sushi? A. Step 1 B. Step 3 C. Step 6 D. Step 10 24

Possible reasons for item being Type C: Item is difficult and requires amendments because Possible reasons for item being Type C: Item is difficult and requires amendments because many students couldn’t answer it. Possible reasons: Confusing description Question misleading students Question containing error(s) 25

Teacher examined the Type C item Students did not have the relevant personal experience Teacher examined the Type C item Students did not have the relevant personal experience in making and/or eating sushi The recipe did not mention dipping sushi into ‘soy sauce’ or ‘wasabi’ after the finish Without experience, students could not interpret the correct answer which is the final step This question contains the error which requires outside knowledge uncommon among the students 26

Exam item example: Type D After cutting the vegetables and crab meat, Jack’s mum Exam item example: Type D After cutting the vegetables and crab meat, Jack’s mum ____. C. takes some rice wine vinegar and mixes it with the sushi rice This is the answer but the option rephrased the instructions. 27

Features of Type D item Type D: Item is difficult and doesn’t require amendments. Features of Type D item Type D: Item is difficult and doesn’t require amendments. It is suitable for screening the high ability students. Low and mid- ability students can’t answer it. Type D items should be kept at a low proportion in an assessment. The rephrasing is difficult for students to interpret the correct answer. 28

Analysis of TSA P. 3 Reading Paper Predict P. 3 students’ results in the Analysis of TSA P. 3 Reading Paper Predict P. 3 students’ results in the upcoming TSA reading assessment Compare the overall students’ ability from different years Examine the neglected elements in our teaching Assumption: TSA assessment covers the basic competency that all students are supposed to have acquired by the end of a learning stage 29

SP Xpress analysis on ‘ 2011 TSA P. 3 Reading Paper (3 ER 1)’ SP Xpress analysis on ‘ 2011 TSA P. 3 Reading Paper (3 ER 1)’ 2011 -2012 P. 3 2012 -2013 P. 3 Mean 19. 28 18. 95 SD 4. 78 4. 76 Type A 18 20 Type B 6 3 Type C 0 1 Type D 2 2 30

An Interesting Fact: A common Type B item across 2 years Part 1 B An Interesting Fact: A common Type B item across 2 years Part 1 B Question #1 How many chapters are there in this book? ‘Chapter / Table of Contents’ can be an obstacle Students never encounter ‘Chapter’ in storybook teaching (replacing textbook teaching) 31

An Interesting Fact: A common Type B item across 2 years Sam buys two An Interesting Fact: A common Type B item across 2 years Sam buys two sandwiches with the Rainbow Fast Food coupon. How many sandwiches can he get? C. three sandwiches (= Buy 2 + 1 Free) Students are not familiar with ‘Buy 2 Get 1 Free’ concept even though they can understand every single word 32

An Interesting Fact: A common Type D item across 2 years “… too. In An Interesting Fact: A common Type D item across 2 years “… too. In the end, John wins the race. All the people cheer loudly …” Read lines 11 -12. What do the people do when John wins the race? C. shout happily Students are not familiar with synonyms cheer loudly shout happily 33

Personal reflection Teachers become aware of the quality of the assessments 1. Accumulate professional Personal reflection Teachers become aware of the quality of the assessments 1. Accumulate professional experience in setting better assessments Teachers prepared for the unexpected and unlearned 2. Teachers become cautious and adjust their teaching 34

Things to do Suggest Ts reflect on their exam paper using SP Xpress Facilitate Things to do Suggest Ts reflect on their exam paper using SP Xpress Facilitate Ts process the data with technical supports Set up a school based question bank Nurture a culture on giving a more constructive feedback to both Ts and Ss after assessment 35

Thank you so much for your time! Thank you so much for your time!