There’s an old saying: If you don’t have to pay someone to be good, then he’s probably good for nothing. Harvard economist Roland Fryer, Jr. is testing the reliability of this old saw using schoolchildren as test subjects.
The cover story in the April 19 issue of Time magazine gives an account of Fryer’s efforts to use the scientific method to zero in on exactly what motivates kids to do well in school. As many parents and teachers can attest, Fryer’s quest is tantamount to the search for the holy grail and verifiable results would be just as valuable.
According to the Time story, Fryer used about $6.3 million in “mostly” private money to pay 18,000 school kids in four cities (Dallas, Chicago, Washington, D.C., and New York City) to get better grades and do more homework. Predictably, Fryer got some flack as news of his little experiment leaked.
Fryer recalls receiving death threats from parents who read about his plans to bribe their kids to improve their school work. Fryer was run off from schools where he had begun the experiment as principals began feeling the pressure from politicians, wary of how a “pay for grades” scheme would play with their constituency.
{modulepos inner_text_ad}
Criticism of Fryer’s educational R&D came from other, more scientific, sources, as well. Psychologists around the country excoriated Fryer for carrying on with an experiment that in one version or another had failed repeatedly for years.
One of these researchers, Dr. Edward Deci of the University of Rochester, built his reputation on identifying factors that were effective motivators of people. Among his findings, Dr. Deci reported that “money — like other tangible rewards — does not work very well to motivate people over the long term, particularly for tasks that involve creativity.” Basically, Deci and many of his colleagues discovered that money alone was not enough to make people buckle down, especially when the desired work product was something intellectual or creative.
The article goes on to cite one of the experiments offered by Deci as evidence of the failure of rewards programs to motivate kids to do better in school. In that experiment, researchers at Stanford University divided 51 toddlers into groups and instructed all the kids to draw pictures. One group was informed that they would get a special certificate if they did their work as they were told. The other group, however, was not given such a promise.
When the scientists returned to the school a few weeks later to observe the kids through a one-way mirror, they found that the boys and girls that received the certificates at the beginning of the experiment spent half as much time drawing as the kids who drew without reward. Deci hypothesizes that the reward, instead of spurring the subjects on to increased output, actually had the opposite effect and diminished the intrinsic value of drawing for the kids that received the certificates. “What we really want is for people to value the activity of learning,” remarked Deci.
It would be difficult to find anyone, even a fan of social engineering, who would disagree with Deci’s hypothesis — even Fryer does not. He claims that learning for its own sake should be the ultimate goal of all instruction, but he claims that kids are not doing that, so something else has to be tried.
So after some of the political pressure that rose from his earlier attempts to implement his plan had died down, Fryer began seeking out principals willing to let Fryer and his team experiment with their young charges. Eventually, 143 schools volunteered to allow Fryer to conduct his controversial experiment on their students. Then, in the fall of 2007, the experiment began.
New York City was first. The parameters were set by Fryer and the staff of his education innovation laboratory at Harvard. Fourth-graders at the target school would be paid a maximum of $25 for every test on which they received higher marks than the previous one. Kids in seventh grade were paid up to $50 under the same scheme.
In an unexpected act of appropriate deference to the family, Fryer required potential participants to get their parents permission before they could enroll in the experiment. Remarkably, 82 percent of those who sought parental permission received it. The money was then directly deposited in accounts set up in the child’s name.
In Chicago, the plan was a littler simpler: Subject students would by paid $50 for an A, $35 for a B, and $20 for every C, up to a cap of $2,000 per academic year. The rules in Chicago were noticeably less complicated than those for the kids in New York. This is likely the result of the collaboration of Arne Duncan, President Obama’s Education Secretary, in the establishment of rules for the Chicago experiment.
Kids in the nation’s capital would get cash rewards based on ratings taken over five different subject areas, including attendance, grades, and good behavior. Depending on how well they performed in these areas, the children would be paid up to $100 every two weeks.
The kids in Dallas had the simplest set-up of all: Second-graders were told by their teachers that they would be paid $2 every time they read a book and took a computerized quiz on its content.
In the summer of 2009, Fryer and his associates returned to the schools to collect the data and the results were mixed, and that’s being generous. In New York City, over 8,000 fourth and seventh-graders were paid for their improved test scores and they “earned” a total of $1.5 million. Despite such a sum being doled out, the test scores did not improve overall. Simply put, the experiment was a failure.
In Chicago, while kids that got paid to be better students did show higher than average attendance, they scored no better than their colleagues that received no financial incentive.
The kids in Washington that were paid by Fryer under his scheme did improve their scores on standardized reading tests.
In Dallas, where the rules were the most straightforward, the results revealed that those boys and girls that were paid to read books did score better on standardized tests than those who read for the sake of reading.
Fryer went back to Harvard and began unravelling the seemingly contradictory results of his efforts. First, he analyzed the demographic composition of his control and treatments groups in the four target cities. This factor, Fryer reasoned, did not effect the outcome of the research as all the students in both groups in all four cities were mostly black or Hispanic and were from low-income families. Curiously, the full demographic breakdown of the results were not shared with Time magazine. Such a dissection has the potential of being embarrassingly politically incorrect. Perhaps when the full report of the experiment is published by Fryer such distinctions may be revealed and their implications may be deemed irrelevant or disturbingly dispositive. We’ll have to wait for the book.
One discovery Fryer claimed to have made while combing through the data he collected was that the simpler the reward system, the more successful it was. Kids in New York, where money was paid for improved test scores, reported that they were excited at the prospect of getting cash for doing better on tests, they just weren’t sure how to go about doing better on the tests. According to surveys conducted after the termination of the experiment, subjects in the treatment group in New York City commented that they could not think of any reliable methods of improving tests scores. “No one said they were going to stay after class and talk to the teacher,” laments Fryer.
Another thorny issue identified by Fryer in his post-mortem was the aspect of the subjectivity of grades. If students are paid for getting “better grades” there is a substantial role played by teachers and their subjective estimation of a student’s work and the resulting mark given for the work. In a scientifically controlled environment, grades would be assigned according to well-defined, unalterable criteria, but in the real world, teachers are the ultimate arbiters of what is “A” work and what is “A-” work. This relationship throws a monkey wrench into the sociological works and makes verifiable results difficult to attain.
The answer to that insuperable “obstacle” to the pay-for-grades success is the use of scores on standardized tests as a more objective metric. To Fryer’s dismay, even those students where grades or attendance improved during the duration of the experiment, the scores on standardized tests did not show similar improvement. If one is devoted to the scientific method and to experimenting with controversial educational “reforms,” then he must be equally prepared to accept the failure of the experiment to produce the desired results. In this case, if the hypothesis set forth at the beginning of the experiment was that a child will do better in school, as determined by that child’s scores on standardized tests, when he or she receives a cash reward for such improvement, then the hypothesis was not proven and the experiment failed.
As one familiar with this branch of pseudo-science and its practitioners would expect, Roland Fryer, Jr. of Harvard University wasn’t quite prepared to accept these findings with scientific sang-froid and move on to testing the next hypothesis. In fact, Fryer points to surreptitious racism and classism written into standardized test questions as possible sources of the skewed results. The article in Time claims that Fryer addresses this topic at length in the full report of his findings, but that they make no more mention of it in their summary.
Finally, Fryer reluctantly admits that much of the data he collected after his experiment perplexes him. He doesn’t have an explanation for why some children didn’t improve in school even when promised cash in return for better grades. Undaunted, however, Fryer is determined to continue conducting experiments on school kids in order to determine the scope of the rewards system that will finally prove successful. “One thing we can’t do,” Fryer says, “is we cannot restrict ourselves to a set of solutions that make adults comfortable.”
Dr. Fryer better get to work on a new paradigm, then, because there are few adults that are comfortable with either the inference or the implication of a program that pays kids for learning. Although it may be illusive, the ultimate aim of every parent for every child should be the inculcation of a life-long love of learning and an understanding that education is its own reward and it fills your life with riches, whether or not it fills your wallet.
