Things did not go smoothly last month when Lincoln Public Schools eighth- and 11th-grade students took the statewide writing test online.
In Jennifer Van Winkle’s class at Lux Middle School, for instance, one student had to log on 16 times, another lost half of the just-written work and yet another never did get logged on. And 2½ hours later, students were still working to finish the test that was scheduled to take 90 minutes.
It was frustrating for students and teachers who had been working all fall to prepare for the test, she said.
“It’s difficult to reduce all that (work) to ‘sorry, you’ll have to try to log in again,’” she said.
Teachers and students all across the district faced similar struggles, enough that LPS officials have asked the Nebraska Department of Education to invalidate the district’s writing test results.
“Because of the difficulty during administration we believe the validity of the scores have been compromised and should not be used to draw conclusions,” said Leslie Lukin, LPS’s director of assessment and evaluation. “It’s not something we take lightly.”
Many districts across the state have had similar problems, and it is not something the Nebraska Department of Education takes lightly either, said Valorie Foy, the state education department’s director of statewide assessment.
State education officials are working with the company that administers the test to determine the extent of the problem statewide, Foy said.
“We have to decide what to do but I have to have some facts and figures to work with,” Foy said. “We’ll bring policy people together to make a determination of what we want to do. Always, our goal is to be fair with students and fair with districts.”
The department contracts with Data Recognition Corp., a Minnesota-based company that helps the state develop and administer the tests. The state has worked with the company for about seven years and just entered into a new five-year, $25 million contract.
The writing test, along with statewide reading, math and science tests, reflect the performance of schools and districts in a state schools report and are submitted to the U.S. Department of Education. Districts and schools that don’t meet certain federal No Child Left Behind benchmarks can face sanctions, and the state is developing an accountability system based on a number of factors including statewide test scores.
State education officials also want to make sure there will be no problems with reading, math and science tests, which students take in March and April, Foy said. There were no problems with those online tests last year, all of which are multiple choice.
There were some problems with formatting on the writing tests last year, and state education officials graded the tests and posted results online, but did not use them for either state or federal accountability purposes. Last year, Foy said, Data Recognition Corp. subcontracted with another technology company, but it is handling the work themselves this year, Foy said.
This year’s problems, according to LPS officials, are far worse.
“Really, it was every school and some days were worse than other days," Lukin said. “It got downright nasty in some instances.”
In addition to getting kicked off the computer or not being able to log in, students noticed some work was lost, and in a few cases the work didn't get submitted at all.
The problems create stress on students -- especially special education students, those with medical issues or those who already get anxious taking tests -- and it’s impossible to tell how the interruptions and other problems might have affected their work, LPS officials said.
Expecting 14-year-olds to stay focused for 2½ hours is challenging, Van Winkle said.
Education Commissioner Matt Blomstedt said Data Recognition Corp. is working hard with the state to solve the problem.
“We need assurance from them that they’re diving in to do their work to assure us it’s going to work effectively,” he said. “It’s incumbent on them and on me to make sure that gets done.”
The problems present a conundrum for the state, Foy said, because some districts have had no problems and they want their writing scores to count.
“It’s really a tough spot,” she said. “It’s not a good situation.”
Nebraska isn’t the only state to experience problems trying to administer statewide tests online. Last year, Indiana, Kentucky, Minnesota and Oklahoma experienced numerous technical problems that forced their state education departments to extend testing windows. Problems in those states were traced back to other contractors.
LPS Associate Superintendent of Instruction Jane Stavem said some glitches are expected when administering a test online, but this went well beyond what could be considered normal.
“Assessments are important, but those assessments need to work because they take away from instruction,” she said. “So if they’re going to measure our instruction, we want it to be valid.”
Districts across the state had a three-week window beginning Jan. 20 to administer the writing tests, which are given to fourth-, eighth- and 11th-grade students. Fourth-graders take the test with paper and pencil.
David Smith, LPS English curriculum specialist for secondary education, said most middle school teachers use the writing test as part of students’ grades. About half the high school teachers do, he said. Now, district officials are advising teachers to use another piece of writing for students’ grades.