In November 2006, I wrote to Steve Mancini, Director of Public Affairs at the KIPP Foundation, and asked him to address the questions/concerns I have about KIPP. He wrote back and turned me over to one of his colleagues. Despite assurances they would answer my questions/concerns, they have yet to do so. As you might recall, I am interested primarily in tracking data and score analysis. One of my colleagues uncovered enrollment data at KIPP schools in California which corroborated much of the anecdotal information that is said about KIPP and "creaming." According to 2005-2006 school data, KIPP Bridge College Preparatory in Oakland had 35 African-American boys in 5th grade, but only 8 of these kids remained in 8th grade. Despite whatever the intentions of the parents, 27 out of these 35 kids -- about 77% -- did not make it.
I asked Mancini for a statistical analysis of KIPP achievement data in order to determine the standard deviation of the scores. The obvious problem with averaging anything is that the average often does not depict the typical outcome. If there is one outcome that is very far from the rest of the data, then the average will be strongly affected by this outcome. In short, some really high achievers will make the others look pretty good, even if these others are not doing so well.
For example, let's say there were 5 students taking a test. The scores (out of 100 possible points) were as follows: 45, 47, 52, 98, 99. The average of these five scores is 68.2 So you could truthfully and accurately say, "Student scores were near the 70th percentile." But how many students scored a 70? None. If you look at the scores, 3 out of the 5 did really badly. But 2 of the 5 did really, really well. The result? It looks like great things are happening when, in fact, they are not.
This is especially relevant to the issue I mention above, i.e., the ever-shrinking number of kids at KIPP as they move from 5th to 8th grade. For example, let's say there are 20 kids at KIPP in the 5th grade. Two of the kids score really high - 98 and 99 out of 100. But the other 18 score either 47, 45, or 52. This results in an average of 51.5. The next year, in the 6th grade, there are only 15 kids left -- 5 dropped out, were "counseled out," or simply are no longer there. Same scenario: two of the kids score really high - 98 and 99 out of 100. But the other 13 score either 47, 45, or 52. But this results in a slightly higher average of 53.4. In the 7th grade, only 8 kids are left. Same scenario: two of the kids score really high - 98 and 99 out of 100. But the other 6 score either 47, 45, or 52. But this results in yet another higher average: 59.75. And by the time they reach 8th grade, there are 5 kids left. Now the 2 high scorers really skew the average, all the way up to 68.2. But the other 3 are still scoring 47,45, and 52.
What's going on here? KIPP is getting statistically better because more kids are dropping out. So should we blame KIPP for pushing them out or praise them for raising the scores of the two that remain?
KIPP needs to come clean and reveal what actually happens to its enrollments and whether or not the scores are skewed by a small number of high achievers.
5 comments:
If you've ever taught in an urban school, you should know that the rate of transience is very high. What KIPP experiences is pretty typical for the children they teach.
Drop-outs, or at least transients, are a common phenomenon in low-income schools, even good ones. So these numbers would not be surprising if they were associated with your average public school. But KIPP is not your average public school. Many supporters of KIPP see it as the answer to the problems that vex inner-city schools. But it seems, at least from what we can tell from the California enrollment data, that even KIPP cannot solve the drop-out/transient problem.
There were nine KIPP schools in California as of the 2005-2006 school year. Six of the nine schools saw decreases in enrollment as their 5th grade kids moved up from the 5th grade to the 7th grade (the 8th grade at one school).
KIPP Academy Fresno went from 60 to 48 kids from 5th to 6th grade, a 20% decrease in enrollment.
KIPP San Francisco Bay Academy in San Francisco went from 73 to 56 kids from 5th to 7th grade, a 23% decrease in enrollment.
KIPP Academy of Opportunity in LA went from 88 to 66 kids from 5th to 7th grade, a 25% decrease in enrollment.
KIPP Bayview Academy in San Francisco went from 81 to 55 kids from 5th to 7th grade, a 32% decrease in enrollment.
KIPP Los Angeles College Preparatory in LA went from 88 to 57 kids from 5th to 7th grade, a 35% decrease in enrollment.
KIPP Bridge College Preparatory in Oakland went from 87 to 36 kids from 5th to 8th grade, a whopping 59% decrease in enrollment.
KIPP students are required to go to school Monday to Friday from 7:30 in the morning until 5 in the afternoon. They go to school on Saturday from 9 in the morning until 1 in the afternoon. They are required to complete two hours of homework every night. They are required to attend an extra month of school in the summer. They are required to wear a uniform. All told, KIPP students spend about 70% more time in school than their regular public school peers. KIPP students are subject to a strict code of discipline that punishes offenders by forcing them to wear a sign around their neck that says "bench" or, according to one source, "miscreant."
So it seems reasonable to ask this question: is KIPP contributing to this drop-out/transient problem?
I doubt rather seriously that anyone at KIPP wants any of their students to drop out. But declining enrollments actually benefit KIPP by making their achievement data look better than it might actually be.
Are we contributing to the drop-out/transience rate? It's an excellent question - I know that the KIPP school that I teach at makes every attempt to keep kids here, but much of what is happening in their lives and our city is out of our control.
Students move out of our rapidly gentrifying district and attend public schools in surrounding districts (only district residents can attend, since we receive public funds). Students move even further away and attend schools out of state. Students have little siblings that turn 4 and have to attend Pre-K - they're responsible for taking their sibs and they can't pick them up at 2:30 if they're here until 5:00 pm.
BELIEVE ME - after investing the time we invest in our students NOBODY wants that "time-in" squandered by losing them to other districts.
Also, FYI - KIPP Foundation now reports results only for children who test/re-test. Only our students who have entering 5th grade and exiting 8th grade test scores are reported in the KIPP released achievement results. That controls for transience.
Unfortunately, the geniuses who drafted NCLB didn't do the same...so now there are two sets of scores to contend with when evaluating our schools.
Just my 2 cents.
As much as I'd like to believe you, there is no evidence for your claims. So I won't believe you until you can provide proof. You say:
"KIPP Foundation now reports results only for children who test/re-test. Only our students who have entering 5th grade and exiting 8th grade test scores are reported in the KIPP released achievement results. That controls for transience."
OK. Prove it. Do more than say, "Believe me." No offense, but why should we? You teach at a KIPP school. so what you claim is liable to be pretty biased, no?
Even if this were the case -- i.e., if it could be proven that KIPP controlled for transients so that high-achieving students who entered KIPP later did not inflate the average scores of those that entered in 5th grade --, it still does nothing to address the simple fact that KIPP scores go up more kids drop out.
You sound like a nice person, and god knows how you can work that many hours. But when you say,
"BELIEVE ME - after investing the time we invest in our students NOBODY wants that "time-in" squandered by losing them to other districts."
my response is the same as above. I don't believe you. I don't believe you because KIPP benefits from these drop-outs, push-outs, counsel-outs, move-outs, whatever you want to call them. It might be framed as a simple, diect, sincere conversation between a student and a principal, such as the one between the principal at KIPP 3D Academy in Houston and a student named Ray. Here's the transcript:
Hedrick Smith (voiceover) - At the start of school, Ray had his first confrontation with 3D Academy's principal, Dan Caesar.
Ray: We were going over our chants, and -- just being myself, still trying to figure out how this school works and everything . . .
Dan Caesar - We say, "Is 3D in the house?!?!" and all the kids raise up their hands and say, "YES!" and Reynaldo raised up his hands and said "NO!"
Ray: I waved my hand. I said, "No." And then he looked at me and he said it a second time. And I said "No" again.
Dan Caesar - I knew right then, "Here's the first test, the first person testing our culture." So I let him know in front of everybody in the room that that's not going to be tolerated. We all want to be here. We chose to be here. If you don't want to be here, find the door.
So much for KIPP's motto, "No excuses."
I think I may be the one who originally publicized the attrition figures, so it's interesting to see them get further attention. It seemed like the initial general response was: "Hey, that's weird; now let's forget it and get back to fawning over KIPP." My information was originally posted on www.sfschools.org
"Anonymous' " comment that the staggering attrition rate for African-American boys at Oakland's KIPP Bridge is just "pretty typical" overlooks the fact that that school's attrition is sharply higher for African-American boys than for girls. (African-American boys are generally a more academically challenged subgroup.) It's also sharply higher than attrition for the school's population overall, though that's very high too.
Also, note that the 8th-grade figure is for FALL of 8th grade; we don't know how many of the students reported for 8th grade actually graduated. This cannot be brushed aside as "pretty typical" -- it's pretty shocking, and should be questioned until KIPP explains fully.
I have another question about KIPP. It's very clear from my research (amateur research, BTW) that KIPP retains students to repeat grades at a high rate. Questions include: how high; how does that compare to traditional public schools; how much does that contribute to high test scores; and does retention confer genuine, lasting academic improvement?
KIPP schools are almost all grades 5-8 in districts where the feeder schools are almost all K-5. Does that indicate that KIPP schools are intended to encourage students to repeat 5th grade, and to weed out those who are unwilling to do that? And where KIPP retention rates are reported (I saw a Baltimore report that mentioned them), do those rates include students who finished 5th grade elsewhere and went through 5th grade again at KIPP?
I took my own 7th-grader in fall 2006 to visit KIPP San Francisco Bay Academy and ask about enrollment. The response included the message that we should not assume she's in 7th grade; she'll be in the grade she tests into. When we were originally looking at middle schools for my oldest child, I visited a number of schools, and never got that message before -- the norm is to assume students remain in their grade on schedule unless there's some significant reason that shouldn't happen. (My younger has continued to happily attend the same middle school her brother graduated from, for the record; after visiting KIPP and learning about its chilling discipline system, she refers to it as "the evil charter school of doom." But that's another story.)
Post a Comment