Are retests a waste of $500,000?

CMS Superintendent Peter Gorman says retesting kids who fail state exams may bump up scores for schools, but students aren't really learning much when they cram for a week.

He also estimates the state spent $500,000 in a tight budget year printing and mailing extra tests.

Here's his letter to the N.C. Board of Education urging them to drop the mandatory retesting, which began last school year:

August 24, 2009

NC State Board of Education
c/o William C. Harrison
6302 Mail Service Center
Raleigh, NC 27699

Dear State Board of Education,

We would like to share our serious concerns around the use of retesting for accountability purposes in North Carolina. After one year of retesting all students who performed at a Level II on any EOG test or its alternates, we have reviewed the data. We believe it raises serious questions about the process, outcomes and the impact of this testing.

In reviewing our test results, we found that almost two-thirds of our students who passed on the retest originally had scores within one standard error of measurement on the first test administration. We only have 180 instructional days to teach our students. Therefore, the less testing we do for summative assessments (which the state tests are), the more time we have to actually teach students.

We had to suspend generalized instruction after the first EOG test so we could provide a week’s worth of remediation to the students who scored a Level II. Can one week of remediation really advance students in a measurable way? Our percent passing went up around nine percentage points, mostly due to students who scored in the range where they may have actually known enough to pass in the first place. This year, we will need to test earlier to provide more remediation. We would much prefer to spend the time teaching instead of remediating.

Even in an optimal situation, the level of feedback from the state tests does not lend itself to use for remediation. We were left wondering what can we do in one week to move a child from:

“Students performing at this level demonstrate inconsistent mastery of knowledge and skills that are fundamental in this subject area and that are minimally sufficient to be successful at the next grade level.” –SBE policy GCS-C-018

to

“Students performing at this level consistently demonstrate mastery of grade level subject matter and skills and are well prepared for the next grade level.” –SBE policy GCS-C-018

If the point of the retesting is to give a better read on student abilities, we think it makes sense to either use a more accurate test (the 4 points that make up the 1 SEM, is in many cases, over 55% of the Level II range), or simply count the students who score within 1 SEM as proficient so we can focus on teaching instead of preparing for a second test.

Our Exceptional Children educators are concerned about the appropriateness of the NCEXTEND1 for a first administration. This issue is compounded when the state regulations require us to put these students through the ordeal again if they only score a Level II score. Several parents have questioned whether retesting hurts the child.

We are also concerned about the validity of retesting students on the NCCLAS assessment. Since this assessment requires materials collection and review, teachers often were asked to re-enter student scores without any real ability to collect new work samples. While we want to be sure student performance is properly assessed, it is hard to justify requiring teachers to rekey the same score.

Another concern was on NCEXTEND2 and the lack of varying test forms. The students who scored a Level II on the first administration were given the same form of the test about a week later, due to schedules and our attempts to maximize instructional time. This gives retested students an advantage due to previous exposure to the test.

Given the statewide budget issues, we think it would be wiser to save the resources for actual instruction. CMS administered almost 50,000 retests in the 08-09 school year. In 09-10 we expect that number to increase. Using an estimate of $1 per test (as NCDPI did when calculating the cost savings from the elimination of the third grade pretest) the printing and shipping costs across the state would approach $500,000. This does not include the staff effort for delivering, administering, scoring and reporting on these tests. Nor does this count the lost educational value of the time required for additional testing and the remediation process which interrupts school activities.

We prefer a test that measures accurately enough that a week of tutorial will not move a child to proficiency, if that is not an achievable goal, we would suggest that the state use the SEM in measuring proficiency rather than retesting. Our data and experience this year suggest that retesting is expensive and not effective.

Sincerely,


Peter C. Gorman
Superintendent

C: CMS Board of Education
Robert Avossa
Christopher Cobitz

- Ann Doss Helms