Oklahoma education doesn’t get much praise, but I find it hard to sit quietly when others take issue with our few accomplishments. Adam Schaeffer at the Cato Institute started a debate on Oklahoma’s Preschool “successes” as reported by USA Today last week. He states:

There’s just one tiny problem. Oklahoma’s achievement scores on National Assessment of Educational Progress (NAEP, AKA “the nation’s report-card”) suggest that the state’s universal preschool program is at best ineffective and at worst harmful to student achievement.

Sara Mead at the New America Foundation took issue with his comments in her own blog posting. While she sites various possible variables that may effect NEAP scores in OKlahoma, her last paragraph really hits the nail on the head:

More broadly, no one should look at pre-k as an “innoculation” that, administered once at age four, delivers improved academic performance without further follow-up through children’s schooling. That’s not how learning works. High-quality elementary schools must build on the base of improved skills and knowledge children bring with them from pre-k. If elementary schools are poor quality or otherwise unable to provide supports and curricula that build on pre-k learning gains, those gains will be squandered. That’s why efforts to improve early education can’t stop at pre-k, but need to continue through the early elementary years and indeed throughout a student’s K-12 education. But that doesn’t mean quality pre-k doesn’t make an important contribution in getting that process off on the right foot. Oklahoma has done a good job in putting a high-quality pre-k program in place for the vast majority of its students. Now it needs to work to improve its kindergarten and elementary programs in order to sustain the documented gains children are making in pre-k.

Mr. Schaeffer retorts with more facts, figures, and charts looking at hispanic populations, poverty rates, etc. (which, unfortunately, Ms. Mead initially threw onto the fire), but he is still missing the broader picture.

NCLB really is a two-edged sword. Accountability is great, but numbers often can be misleading. The first real assessment for preschool programs comes at the fourth grade, and Mr. Schaeffer seems bent on numbers telling the whole story:

So I challenge you, Sara, and any other preschool activist out there, to find the nefarious factor that has destroyed all the gains from pre-k. By all means, take this data and run it through statistical software with whatever controls you’d like related to documented demographic and education changes (as long as you include the national averages as a control).

NAEP and NCLB will not cure the woes of public education. Unfortunately, there just is no magic bullet. And no matter how you wrangle the current numbers, it will not provide you with the data needed to accurately assess student achievement. Mr. Schaeffer admits in another post:

And that is why non-experimental analysis can only provide suggestive evidence, with a heavy dose of uncertainty. Among the available research methods, the only way to be fairly certain an educational treatment has had an effect on students is to conduct a controlled experiment akin to those used in medicine or drug testing. Researchers randomly assign each person to either get the treatment or to not get the treatment.

Empirical research is very difficult, costly, and takes time. Ideally, all education innovations would be subject to large-scale, double-blind, studies. But with no time in the global competitive race and a government that balks at any research that doesn’t “protect our national interests” (apparently, education does not factor into national security), there will be little opportunity for understanding what works.

Education is a grassroots effort. For many in Oklahoma, preschool works. Certainly, it may need improvement, but given what I know from the front lines, it’s helping Oklahoma. An empirical study would be wonderful, and would do much to point out specific improvements needed in the program.

In the meantime, my challenge to Mr. Schaeffer and anyone wanting to really know the effectiveness of the preschool program would be to visit the Oklahoma communities. Ask the parents and kindergarten teachers what they feel. Observe some Kindergarten classes this Fall. It won’t be large-scale, but you might learn something.