THERE are few policy questions to which improving the quality of education is not a reasonable answer. Yet assessing teachers is far from straightforward. Pupils’ grades or test scores may reflect any of a host of influences, not just the standard of instruction. Neither can one take for granted that good teaching, however it is measured, will translate into better lives for its recipients. In two new working papers , Raj Chetty and John Friedman of Harvard University and Jonah Rockoff of Columbia University deploy some statistical wizardry to tease out the value of teaching (see sources below). Good teachers, they find, are worth their weight in gold.

School systems often try to assess the quality of their teachers by measuring “value added”: the effect a given teacher has on pupils’ test scores. Research is divided on whether this makes sense. Critics reckon that pupils with particular backgrounds—from richer families, for instance, or with more attentive parents—wind up in classrooms with better teachers. If so, the teachers who excel in assessments of value added may simply be teaching more privileged pupils. Messrs Chetty, Friedman and Rockoff try first to settle this debate.

To do so, the economists draw on a substantial data-set from a large, urban American school district, which they do not name. It covers 20 years of results and takes in more than 2.5m pupils. The data include teacher assignments and their pupils’ test scores from third to eighth grade (ages eight to 14, roughly speaking). The authors dig into the mountain of numbers to calculate the effect each teacher has on their pupils’ performance, after adjusting for demography and previous test scores. Previous scores, they reckon, do a good job of capturing the various external influences pupils bring to the classroom, from how well-nourished they are to how good their former teachers were. The authors also control for “drift”, or the fact that more recent test scores are better reflections of a teacher’s effectiveness than older ones, perhaps because the quality of an individual’s teaching tends to change a bit over time. In the end they produce a value-added score for every teacher in their sample.

The researchers then go hunting for bias: the possibility that good teacher scores actually reflect the lucky circumstances of their pupils. Using data from income-tax records, the authors show that the characteristics of pupils’ parents, such as family income, do not generally predict how teachers perform. They also use changes in the average quality of a school’s teaching staff from year to year (from the movement of a particularly good teacher to a different school, for instance) to check their findings. When the average quality of teachers in fourth grade falls from one year to the next, for example, the performance of the fourth-graders also drops as expected.

A bias-free measure of teacher quality allows the economists to wring fascinating conclusions from their data-set. They find, for instance, that the quality of teachers varies much more within schools than among them: a typical school has teachers spanning 85% of the spectrum for the school system as a whole. Not only do teachers matter, in other words, but the best teachers are not generally clumped within particular schools.

Across schools, however, better pupils are assigned to slightly better teachers on average. The common practice of “tracking” pupils (filtering good ones into more advanced courses) could be to blame, the authors reckon, though they abstain from drawing firm conclusions. Whatever the cause, getting more effective teachers to instruct better-performing pupils naturally exacerbates the gap in achievement. Making the best teachers work with the worst pupils could go a long way toward minimising the yawning differences in attainment within a school system, the authors contend.

At the very least, that change would be lucrative for the pupils who benefit from it, according to the researchers’ second paper. They compare their measure of teacher quality against pupils’ fortunes as adults, after again controlling for pupils’ previous test scores and demography. (Pupils from the earliest years of their sample are now in their late 20s.) Unsurprisingly, exposure to better teachers is associated with an increased probability of attending university and, among pupils who go on to university, with attendance at better ones, as well as with higher earnings. Somewhat more unexpectedly, good teachers also seem to reduce odds of teenage pregnancy and raise participation in retirement-savings plans. Effects seem to be stronger for girls than for boys, and English teachers have a longer-lasting influence on their pupils’ futures than maths teachers.
The authors reckon that swapping a teacher at the bottom of the value-added spectrum with one of average quality raises the collective lifetime income of each class they teach by $1.4m. That rise would apply across all the teacher’s classes and over the whole of his or her career.

Pay peanuts, create monkeys
Given the stakes, the salary rises needed to coax teachers into accepting rigorous assessment and performance-linked pay—often hard sells in unionised school systems—seem a bargain. Performance bonuses designed to retain good teachers make less sense. Since 91% of teachers in a third year of teaching stay for a fourth, most of the expense of a bonus programme is wasted. Replacing teachers who score badly is cheaper and more effective, provided that administrators can get around strict union rules on sackings. Though not easy, school reforms that identify good teachers and assign them to struggling pupils should pay off handsomely.

Sources
"Measuring the impacts of teachers I: evaluating bias in teacher value-added estimates", Raj Chetty, John Friedman, and Jonah Rockoff, NBER working paper 19423, September 2013
"Measuring the impacts of teachers II: teacher value-added and student outcomes in adulthood", Raj Cheety, John Friedman, and Jonah Rockoff, NBER working paper 19424, September 2013.