Those Who Can

Why did some students' scores (English, mostly) go in the wrong direction?

June 17, 2007

by Cal Lanier

The following students had a lower score on their actual ACT than on their first diagnostic. FTI students are in bold.

English	Math	Reading	Science	Composite
Frank (-1) Ursula (-1) Kiera(-1) Ettie(-1) Orlando (-2) Gerry (-3)	Gerry (-1)	Frank (-1) Wanda (-1)	Larry (-3) Quentin (-1) Pamela (-2)	None

Apart from four students' English scores, these declines are relatively unremarkable. Larry and Quentin had outstanding tests overall; both would have scored a full point higher on their composite if they'd not had the fall off in science. I haven't had a chance to talk to either of them; my guess is that Quentin just got slightly tired. Larry may have relaxed a bit too much after having aced the first three sections; science is usually his second best section. But then, the tendency to explain is hard to ignore--there could be some other factor entirely.

Wanda was just off by a point and had only taken two practice tests (including the diagnostic). While Wanda is an excellent math and science student, her English skills are considerably weaker. The reading test goes by fast, so a tentative reader could easily get stuck on a tough passage and have it affect her entire test.

Gerry and Pamela both had wildly varying scores throughout the class, improving tremendously in one section while doing worse in others. Pamela had a major improvement in English; Gerry's Science score was his best ever. Frank has always consistently scored well below his abilities and had never put together a good test. Orlando had a (relatively) solid, consistent test that was below his best effort but not wildly caroming all over the place, so in that light one point off in English is unexceptional. In all these cases (save for Gerry in English), their failure to improve or inconsistent performance is the major problem, not the particular section.

There seems to be a lot of English scores lower than the diagnostic? Really? I hadn't noticed. Unless you count my running howling to Ettie, Kiera, Ursula, and Gerry, saying "FORGET EVERYTHING I EVER SAID! ABORT! ABORT!" after Test 3 as "noticing" rather than, say, "panicking".

I was very conscious of these four students' alarming decline in English scores throughout the course. Two of them caught up to their diagnostic by the last test, but still scored a point off on the real thing. The other two didn't catch up on the practice tests; Gerry's was off by the greatest degree.

I can't even find a consistent reason for their decline. Kiera's results did offer a clue, and I could kick myself for missing it during the class. Each row of answers corresponds to an English "passage" (5 passages per section).

Kiera learned a great deal in class. In the second test, she took longer to answer a higher percentage of questions correctly (68% to 56%). She got all but one of the first passage questions correct. However, answering the questions correctly took more time, and she couldn't finish. Her next two tests show her trying to answer the questions more quickly to finish the test, while still using her new knowledge. Ironically, Kiera lost out by learning more. This is a common problem between the first and second test, but it rarely persists through to the actual test.

While in retrospect this seems obvious, I didn't see it earlier because I didn't realize that many of the students were having a very difficult time working the English passages in a timely fashion. The English section is, traditionally, the only section of the ACT that doesn't give students timing problems. The students were doing timed sessions in class without problems, but were working very differently on the test. I didn't figure this out until after test 4, at which point I required most of the students to take 3 timed drills in English passages and then check their error rate. These drills are, in my view, the reason that Pamela, Hannah, Tara, and Vanessa saw a nice boost in their English scores from their practice tests to the ACT.

However, I hadn't figured this out by test 2 and thus didn't see what was going on with Kiera. At the time, it seemed much more likely that her high accuracy and failure to finish were caused by a late arrival (something that, along with early departures, is common when teaching low income students). I asked her if she'd arrived late. She said yes, she'd come late. I suspect that she actually arived on time and was just nervous about my questions.

At least I have an explanation for Kiera. Ettie, Gerry, and Ursula show no such patterns. Obviously, something I said confused them. On the other hand, many students saw tremendous improvement in their English scores, so whatever confusion my teaching may have caused, it wasn't widespread.

I don't have an answer for this one, but will be watching the next course carefully for similar patterns. I'm also going to start the timed drills sooner in the course, as these helped a number of students understand they could work more quickly without sacrificing accuracy.