Those Who Can

C.1 Patterns of Improvement

June 14, 2007

by Cal Lanier

A student who fails to improve (henceforth labeled, obviously, FTI) becomes visible after the second practice test--her scores in all sections will be within the margin of error (the same or a point higher/lower). I meet with these students and discuss their problems. In almost all cases, they've experienced this frustrating problem before, and the most important thing to do is reassure them because, and I can't stress this enough improvement does not correlate with effort.

A student who does nothing more than sit in a class, do the in class work, and actively try on the tests will almost certainly improve (using my definition of 2 pts per section). If he doesn't, then, in my experience, his lack of improvement is independent of academic knowledge and effort.

Who Failed to Improve?

The two tables below depict student progress on two different variables. The first table lists the total number of points gained from the student's first test to his or her actual ACT. The second lists the total number of points gained from the first diagnostic to his or her best practice section score in each subject. (Since the primary benchmark is the ACT, I consider the 7 students (see note on Hannah) to be the students who failed to improve. Their names are in bold font in the other comparisons. )

Diagnostic to ACT Improved FTI name Net Pts name Net Pts Jane 28 Hannah* 8 Donna 27 Maxine 8 Betsy 26 Rita 7 Yasmine 26 Gerry 7 Ettie 22 Pamela 6 Quentin 22 Orlando 6 Kiera 19 Ursula 5 Vanessa 19 Frank 4 Sandra 16 Wanda 16 Larry 14 Tara 14 Inga 14

Diagnostic to Best Score Improved FTI name Net Pts name Net Pts Rita 37 Hannah 8> Jane 33 Vanessa 7 Donna 25 Frank 6 Yasmine 25 Pamela 6 Betsy 23 Inga 20 Maxine 17 Sandra 16 Ettie 14 Gerry 14 Kiera 14 Larry 14 Tara 14 Orlando 13 Quentin 12 Ursula 11 Wanda 10

The table on the right more accurately reflected my expectations at the end of class. Going into the ACT, I had no concerns about Rita and had hoped, based on their last diagnostic, that Gerry, Maxine, Orlando and Ursula had kicked their various FTI problems. Others who had caused concern and, with help, improved at class end were Ettie, Tara, Hannah*, and Kiera.

Frank, who is an exceptional case of underperformance (I put his actual abilities at between 22-25) spent the last two class sessions working on timed practice, working at different speeds to see if we could uncover what was going on. In Pamela's case, she'd actually kicked one major problem and was making progress on another. I had hopes for both of them, but realized they might not have enough experience to put together a solid test. Vanessa was making acceptable progress except in reading, where she was underperforming by about ten points based on her in-class ability. I'd proven this to her in a timed test, and I was optimistic that she'd make the connection.

Combining the two parameters and taking the delta of the point totals:

name	First to ACT	First to Best	Delta
Vanessa	19	7	12
Quentin	22	12	10
Ettie	22	14	8
Wanda	16	10	6
Kiera	19	14	5
Betsy	26	23	3
Donna	27	25	2
Yasmine	26	25	1
Sandra	16	16	0
Larry	14	14	0
Tara	14	14	0
Hannah	8	8	0
Pamela	6	6	0
Frank	4	6	-2
Jane	28	33	-5
Inga	14	20	-6
Ursula	5	11	-6
Gerry	7	14	-7
Orlando	6	13	-7
Maxine	8	17	-9
Rita	7	37	-30

By comparing their improvement from the diagnostic to their best score and their ACT score, the students who had a "bad day" are easier to identify. Five of the students originally identified as "fti" also had a bad day and were off their best scores by more than point per section on average. Three of them (including Hannah) were consistent in their pattern of non-improvement. Two students, Jane and Inga, had slightly off days but still locked in strong improvement from their diagnostic. Rita obviously had a disastrous test. In retrospect, her highly erratic practice tests signal this possibility. I've never seen anything as dramatic as her fall, although I don't hear of all my students' actual test scores so it might happen occasionally.

Rita's test profile isn't at all consistent with FTI. (It is far more accurately characterized as WTF?). The others' results are consistent with my experiences, although as a percentage they are much higher than usual. The underlying cause of the failure to improve varies widely. In short, the answer to "what students don't improve" is usually "the students who don't improve". At this point, I can only diagnose from the symptoms.

 

 

 

 

 

 

 

 

 

*Hannah's actual diagnostic test score in reading was a 1. When I asked her about it, she explained that she wasn't very good at reading tests and so she spent all her time on the math section. I explained to her gently that the English word for this was "cheating" and she could not do this on any more tests. (Her reaction made it clear she had no idea she was cheating. Needless to say, I had the proctor strengthen these instructions on the remaining tests.)

I didn't want to ignore Hannah's scores entirely, so I changed her reading score on the first test to her second test score, 11, which probably understates her progress. She received the same math score on her second test as the diagnostic, which suggests that her actual diagnostic score would have been lower. I almost removed her from this analysis, but decided to asterisk it instead. Hannah and Tara have very similar testing profiles and I put her actual progress at around 14 points.