It’s difficult to improve teacher quality and PD doesn’t help

(Part of my recent post on the conventional wisdom of education)

Another common hope for improving student outcomes is to improve teacher quality through rigorous systems of evaluation and professional development. In the school that I work for, I am constantly being observed, evaluated, and then pushed to improve my teaching through many hours of professional development each week after school.

I understand why schools feel like this would be effective. We have a bias towards thinking that doing something will always be better than doing nothing. This is called the action bias and we see it commonly in policy. The trouble is that our perception is acutely built for us to assume that our actions are paying off without considering the actual evidence that would show whether something was effective.

In education it can take a long time for interventions to pay off and those interventions will frequently have very small gains. I want it to be true that professional development and rigorous feedback system can improve teaching quality, however I suspect that we are very biased in our implementation of these actual programs.

In a recent Rand study on improving teacher effectiveness they found no gains in student performance from implementing rigorous systems of teacher feedback and mentorship all paired with intensive professional development.

“The Intensive Partnerships for Effective Teaching initiative, designed and funded by the Bill & Melinda Gates Foundation, was a multiyear effort to dramatically improve student outcomes by increasing students’ access to effective teaching. Participating sites adopted measures of teaching effectiveness (TE) that included both a teacher’s contribution to growth in student achievement and his or her teaching practices assessed with a structured observation rubric. The TE measures were to be used to improve staffing actions, identify teaching weaknesses and overcome them through effectiveness-linked professional development (PD), and employ compensation and career ladders (CLs) as incentives to retain the most-effective teachers and have them support the growth of other teachers. The developers believed that these mechanisms would lead to more-effective teaching, greater access to effective teaching for low-income minority (LIM) students, and greatly improved academic outcomes.

Beginning in 2009–2010, three school districts — Hillsborough County Public Schools (HCPS) in Florida; Memphis City Schools (MCS) in Tennessee (which merged with Shelby County Schools, or SCS, during the initiative); and Pittsburgh Public Schools (PPS) in Pennsylvania — and four charter management organizations (CMOs) — Alliance College-Ready Public Schools, Aspire Public Schools, Green Dot Public Schools, and Partnerships to Uplift Communities (PUC) Schools — participated in the Intensive Partnerships initiative. RAND and the American Institutes for Research conducted a six-year evaluation of the initiative, documenting the policies and practices each site enacted and their effects on student outcomes. This is the final evaluation report.”

Here are their main findings:

“The sites succeeded in implementing measures of effectiveness to evaluate teachers and made use of the measures in a range of human-resource (HR) decisions.

Every site adopted an observation rubric that established a common understanding of effective teaching. Sites devoted considerable time and effort to train and certify classroom observers and to observe teachers on a regular basis.

Every site implemented a composite measure of TE that included scores from direct classroom observations of teaching and a measure of growth in student achievement.

Every site used the composite measure to varying degrees to make decisions about HR matters, including recruitment, hiring, and placement; tenure and dismissal; PD; and compensation and CLs.

Overall, however, the initiative did not achieve its goals for student achievement or graduation, particularly for LIM students.

With minor exceptions, by 2014–2015, student achievement, access to effective teaching, and dropout rates were not dramatically better than they were for similar sites that did not participate in the Intensive Partnerships initiative.

There are several possible reasons that the initiative failed to produce the desired dramatic improvement in outcomes across all years: incomplete implementation of the key policies and practices; the influence of external factors, such as state-level policy changes during the Intensive Partnerships initiative; insufficient time for effects to appear; a flawed theory of action; or a combination of these factors.”

This was a very well-funded study supported by the Bill and Melinda Gates Foundation and it was held over the course of six years. It might be the case that a perfect storm of professional development could end up being effective in other circumstances. In general, it can be very hard to generalize the results of one policy experiment on other contexts. In this case, we have very good reason to believe that expensive systems of professional development do not have as much of a positive impact as we would assume. ‘

It might be more effective to get rid of professional development and drop many systems of feedback and observation because they can negatively affect teacher retention while at the same time being ineffective in improving teacher quality. A common stressor for teachers in high-quality Charter Schools is that they cannot stay in the school’s long because the cycles of observation, feedback, and professional development can take up so much time and energy that they will over-stress teachers. It might be worth dropping these systems all together, but there needs to be more research on the tradeoff.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s