From Jonah Lehrer in the WSJ:
. . . We live in a society obsessed with maximum performance. Think of exams like the SAT and the GRE. Though these tests take only a few hours, they’re supposed to give schools and companies a snapshot of an individual’s abiding talents.
Or consider the NFL Scouting Combine, in which players entering the draft perform short physical and mental tasks, such as the 40-yard dash. The Combine is meant to measure physical ability; that’s why teams take the results so seriously.
It’s easy to understand the allure of such maximal measures. They don’t take very long, so we can quantify many people. Also, they make assessment seem relatively straightforward, reducing the uncertainty of selecting a college applicant or football player.
But as Mr. Sackett demonstrated with those supermarket cashiers, such high-stakes tests are often spectacularly bad at predicting performance in the real world. . . .
Even the NFL Combine is a big waste of time. According to a recent study by economists at the University of Louisville, there’s no “consistent statistical relationship” between the results of players at the Combine and subsequent NFL performance.
The reason maximal measures are such bad predictors is rooted in what these tests don’t measure. It turns out that many of the most important factors for life success are character traits, such as grit and self-control, and these can’t be measured quickly.
Consider grit, which reflects a person’s commitment to a long-term goal. As Angela Duckworth, a psychologist at the University of Pennsylvania, has demonstrated, levels of grit consistently predict levels of achievement, such as graduation from West Point and success in the National Spelling Bee.
The problem, of course, is that students don’t reveal their levels of grit while taking a brief test. Grit can only be assessed by tracking typical performance for an extended period. Do people persevere, even in the face of difficulty? How do they act when no one else is watching? Such traits often matter more than raw talent. We hear about them in letters of recommendation, but hard numbers take priority.
The larger lesson is that we’ve built our society around tests of performance that fail to predict what really matters: what happens once the test is over.