The Skeptical Tactician: Why soccer pundits need to learn to embrace uncertainty
Posted by Richard Whittall under Media on Jul 04, 2013

http://blogs.thescore.com/counterattack/2013/07/04/the-skeptical-tactician-why-soccer-pundits-need-to-learn-to-embrace-uncertainty/


I’ve been slowly reading Princeton psychology professor emeritus Daniel Kahneman’s Thinking, Fast and Slow over the last few months now, mostly because each and every chapter blows my mind in a slightly different, uniquely beautiful way and I don’t want to forget anything.
Yesterday while sitting under the smoggy sun in a nearby park, I finished a chapter entitled “The Illusion of Understanding.” Here, Kahneman follows up on an issue made famous in Nassim Taleb’s famous book, The Black Swan: the narrative fallacy.
This isn’t “narrative” in the strictly classical, third act literary sense. Rather, ‘narrative’ refers here to the natural post hoc explanation we give to success and failure you might recognize from all walks of life, from the reason you messed up that job interview to a classic newspaper match report. Here’s Kahneman:
You cannot help dealing with the limited information you have as if it were all there is to know. You build the best possible story from the information available to you, and if it is a good story, you believe it. Paradoxically, it is easier to construct a coherent story when you know little, when there are fewer pieces to fit into the puzzle. Our comforting conviction that the world makes sense rests on a secure foundation: our almost unlimited ability to ignore our ignorance.
Italics mine. This rather incredible paragraph got my mind racing at first about tactics writing, which tends to be overly reductive (“the team lost because the full-backs did not track back quickly enough”), but there are lessons here for fans wary of football punditry in general.
It’s already difficult enough for soccer fans to wade through the countless reams of articles on sports which ostensibly explain anything and everything you would ever want to know about football. Thanks largely to the giant free printing press that is the Internet, soccer writing is about as competitive as it’s ever been. In order to stand out from the competition, some sports writers have gone to incredible lengths to underline their “expertise,” from studying team tactics to watching a plethora of obscure leagues. And yet this confidence leads pundits to make the same mistakes, over and over again, because it makes for a more convincing story to readers, and themselves.
Here are some common examples.
Hindsight Bias
A team wins a match 3-1, with the winning side out-shooting the other. A tactics writer compares the two opposing formations and argues that the winning side dominated because their wingbacks were excellent in support of the attacking forwards and the losing side were far too narrow. In retrospect of course, it’s hard to see the result going any other way. Fans and other pundits agree. RTs all around! Now there is pressure from journalists on the coach of the losing side to adopt a more expansive formation against a similarly shaped opponent in future.

This scenario is so familiar it seems almost banal. We read these pundits regularly; often they’ve written popular books and are referred to as “boffins” in some of the broadsheets. While an individual football match might first appear to be naked chaos augmented by imperceptible differences in overall skill, apparently the entire thing is neatly ordered underneath, decided in large part by coaches who decide which player will play where as part of a static formation. Even though it seems these players are in constant flux, deviating from their set positions with some regularity, and even though it seems goals are generated by accidents of positioning, it’s all quite orderly underneath.
This is a bit of a caricature, but in truth the opinion of the expert was formed on very limited information. They watched the match from the press box. They were not privy to the specific player instructions of the manager. They do not know the fitness levels of either side. They were not aware that the first goal was conceded when a defender slipped ever so subtly in the attempt to track back, thus making it seem as if he was “wrong-footed.” They don’t know the effect of this goal, which was to cause the opposing side to press forward for an equalizer, allowing just a little more space behind their opposition defense to add a crucial second goal.
And yet their conviction in the totality of the tactical explanation seems so convincing to readers because the authors are so clearly convinced. In general, however, it’s a good idea to be wary of world-explaining, overly confident hindsight explanations for failure and success in a sport dominated by luck, particularly within a single match. Equally nefarious is the pundit who reverse engineers a tale of woe for a team with an empirically good process but a few missed crucial results.
Consider the case of Mourinho. Had he won La Liga once more with Real Madrid and made it to a few more Champions League finals, would the explanations for his supposed ‘failure’—also based on impossibly limited information—be less true? Would this alternate universe Mourinho have been better with his players? Would his jerk-face attitude be a strength then instead of a weakness?
Means Regression
A team buys several players who had tremendous preceding seasons, though each of these standout performances came after several decent-but-not-spectacular seasons at clubs in a slightly less competitive leagues. Fans of the team doing the buying get excited their club will now perform better than they did last season. They do end up performing better, although the players don’t match their previous season tallies. No matter; they’re just “adjusting” to the new team and league. A blogger points out that the team has a likely un-sustainably high PDO. No one knows who that blogger is except a bunch of goddamn nerds. Everyone praises the manager for acquiring these players and making the team great.
Except for some reason, the following season the club then ‘inexplicably’ drops four places. The manager gets sacked. He goes on to guide a Championship club to promotion.
So there are a lot of things going on here, but most of them revolve around an admittedly difficult, non-intuitive concept: regression to the mean. The manager saw a few players who had outstanding seasons, and bought them in the hope their form would continue. Instead, their form regressed to their basic innate skill ‘baseline.’ Maybe one or two regressed just a little, and maybe the other guys regressed a lot, or even performed below the mean. Yet the team as a whole did great. Success! Except the team’s shot and save percentage were unnaturally high. They were getting some incredible luck, particularly in scoring opening goals. This luck however was interpreted as skill, both of the players and the manager.
The team’s PDO regressed in the next season however and the team suddenly sucked. To the expert this was all somebody’s fault, probably the manager’s. In reality, it’s just the shifting sands of random variation. It’s incredible therefore that a phenomenon with such incredible power seems relegated by most pundits to penalty kicks. So beware when reading someone (including moi) who over-determines cause and effect and the influence of skill in football without paying credit to luck.
The Halo Effect
Manchester United’s success comes down entirely to the leadership of Sir Alex Ferguson.
And a myriad of other factors, including incredible luck, and individuals that put them in a position to dominate the league over two decades.
When a team or organization is successful, pundits tend to look to the strong work ethic and good values of a particular personality or set of personalities as an underlying cause. This is the ‘halo effect,’ first researched by Edward Thorndike in 1920. In organization terms, this is the over-attribution of a team’s success to the positive characteristics of the person or persons in charge.
You see this kind of thing when a team is successful. Pundits naturally seek to accentuate the positive characteristics of the guy in charge in such a way as to either directly or indirectly imply causality. The same happens when a team isn’t successful; the same pundits will accentuate negative characteristics of the manager as a root cause of the team’s failure as a whole, as if the XI guys playing 38 individual matches each with their own interminably unpredictable elements were mere bystanders.
Conclusion
Football pundits—whether amateur or professional, within the media or within football clubs themselves—aren’t bad people, and they’re not always wrong. But there is a tendency among many of them to gloss over the complexity of football in order to tell a stronger, more convincing story.
You might argue, “It’s just football, let us have our stories, even though they might not capture the whole truth.” Except no one tells these stories with the idea that they’re collectively recognized as bupkis. These stories are used to make decisions that can clearly undermine the future success of a soccer team. They matter.
Not only that, but the stories are getting old. We’re running our of space for all the guys who are desperate to convince us how much they know, and how confident they are in their knowledge. Kahneman mentions in his book Isaiah Berlin’s essay on Tolstoy entitled “The Hedgehog and the Fox.” The former is the historian for whom everything can be neatly explained according to a comprehensive intellectual framework that takes years to fully grasp. “For hedgehogs,” writes Kahneman, “a failed prediction is aalmos always ‘off only on timing” or “very nearly right.” These guys are made for TV, and the newspapers.
Foxes however are wary of world-historical explanations. They understand that success is the aggregate effect of a host of complex factors, including random variation. They are reticent to offer totalizing, all-encompassing explanations for single phenomena.
Right now in football, the foxes work mostly in analytics. We need far more of them in the papers, on the radio and on TV, expressing doubt, erring on the side of caution, and tearing down the comfortable certainty of the people in charge.