Friday, September 11, 2009

Are retests a waste of $500,000?

CMS Superintendent Peter Gorman says retesting kids who fail state exams may bump up scores for schools, but students aren't really learning much when they cram for a week.

He also estimates the state spent $500,000 in a tight budget year printing and mailing extra tests.

Here's his letter to the N.C. Board of Education urging them to drop the mandatory retesting, which began last school year:

August 24, 2009

NC State Board of Education
c/o William C. Harrison
6302 Mail Service Center
Raleigh, NC 27699

Dear State Board of Education,

We would like to share our serious concerns around the use of retesting for accountability purposes in North Carolina. After one year of retesting all students who performed at a Level II on any EOG test or its alternates, we have reviewed the data. We believe it raises serious questions about the process, outcomes and the impact of this testing.

In reviewing our test results, we found that almost two-thirds of our students who passed on the retest originally had scores within one standard error of measurement on the first test administration. We only have 180 instructional days to teach our students. Therefore, the less testing we do for summative assessments (which the state tests are), the more time we have to actually teach students.

We had to suspend generalized instruction after the first EOG test so we could provide a week’s worth of remediation to the students who scored a Level II. Can one week of remediation really advance students in a measurable way? Our percent passing went up around nine percentage points, mostly due to students who scored in the range where they may have actually known enough to pass in the first place. This year, we will need to test earlier to provide more remediation. We would much prefer to spend the time teaching instead of remediating.

Even in an optimal situation, the level of feedback from the state tests does not lend itself to use for remediation. We were left wondering what can we do in one week to move a child from:

“Students performing at this level demonstrate inconsistent mastery of knowledge and skills that are fundamental in this subject area and that are minimally sufficient to be successful at the next grade level.” –SBE policy GCS-C-018

to

“Students performing at this level consistently demonstrate mastery of grade level subject matter and skills and are well prepared for the next grade level.” –SBE policy GCS-C-018

If the point of the retesting is to give a better read on student abilities, we think it makes sense to either use a more accurate test (the 4 points that make up the 1 SEM, is in many cases, over 55% of the Level II range), or simply count the students who score within 1 SEM as proficient so we can focus on teaching instead of preparing for a second test.

Our Exceptional Children educators are concerned about the appropriateness of the NCEXTEND1 for a first administration. This issue is compounded when the state regulations require us to put these students through the ordeal again if they only score a Level II score. Several parents have questioned whether retesting hurts the child.

We are also concerned about the validity of retesting students on the NCCLAS assessment. Since this assessment requires materials collection and review, teachers often were asked to re-enter student scores without any real ability to collect new work samples. While we want to be sure student performance is properly assessed, it is hard to justify requiring teachers to rekey the same score.

Another concern was on NCEXTEND2 and the lack of varying test forms. The students who scored a Level II on the first administration were given the same form of the test about a week later, due to schedules and our attempts to maximize instructional time. This gives retested students an advantage due to previous exposure to the test.

Given the statewide budget issues, we think it would be wiser to save the resources for actual instruction. CMS administered almost 50,000 retests in the 08-09 school year. In 09-10 we expect that number to increase. Using an estimate of $1 per test (as NCDPI did when calculating the cost savings from the elimination of the third grade pretest) the printing and shipping costs across the state would approach $500,000. This does not include the staff effort for delivering, administering, scoring and reporting on these tests. Nor does this count the lost educational value of the time required for additional testing and the remediation process which interrupts school activities.

We prefer a test that measures accurately enough that a week of tutorial will not move a child to proficiency, if that is not an achievable goal, we would suggest that the state use the SEM in measuring proficiency rather than retesting. Our data and experience this year suggest that retesting is expensive and not effective.

Sincerely,


Peter C. Gorman
Superintendent

C: CMS Board of Education
Robert Avossa
Christopher Cobitz

- Ann Doss Helms

5 comments:

Anonymous said...

WOW it does not seem like Dr. Gormon actually knows how all of this works. 1. NCClass is no longer being used 2. Students take ther 1st restest immediately after they find they failed. Remediation does not happen UNTIL the 1st Retest is failed. THE irony behind this process is that the last safety net, a student generated portfolio of work completed throughout the year, is the last thing to be reviewed to determine if students meet the gateway. The Portfolio should be USED FIRST and then students should take the multiple guess test.

LMA said...

The apocalypse is nigh. Gorman is making sense.

I wish he'd have asked them to take it one step further and eliminate the tests altogether. As it stands, many students spend the first quarter reviewing skills learned the previous year, and the fourth quarter reviewing for the EOGs. That leaves a semester for new instruction.

Anon 8:38, I know my daughter's school offered before-school tutoring for students who the teachers believed might have trouble with the EOGs (based either on prior scores or on classroom performance). My daughter received it for math and it really helped her, and not just with the EOGs.

Anonymous said...

The majority of CMS students pass the test on their first attempt.

These students then watch movies and do other non-academic activities during their final weeks of school in order to allow time for the retesting to occur.

The waste isn't the $500,000 cost of retesting - it is real cost of cutting the school year short for the best learners.

Anonymous said...

Every day after testing when my children came out from school, I would ask them how the testing went. One day when I asked my third grader if he did his best he said, "No, my friend got done first and I wanted to be done too, so I just guessed on the rest." Unsurprisingly, he failed that portion of the test in a big way.

We had a good talk and he retook that portion of the test and passed beautifully.

Just because a child fails a test, does not mean they lack academic skills. There are many factors that play into failing a test. My son did not suddenly learn the material, he just focused and performed better.

If he had not been allowed to retest, he would be sitting in a third grade classroom uneccessarily again this year.

RFA52S said...

Thanks you. Very good post.Unless they can offer a really compelling reason for users to come back, it will be the next Bebo, MySpace