In 2003, a book entitled "Moneyball" (perhaps you've heard of it) upended the baseball apple-cart, effectively overturning over a century of entrenched dogma, including ingrained beliefs, valuations, and standard operating procedures.
Moneyball, as we all know, is about the existence of market inefficiencies and the manner in which the Oakland A's identified and exploited those inefficiencies. Statistical analysis was the tool by which the A's more accurately valued ballplayers than the market. What Moneyball did NOT explain was why these inefficiencies existed in the first place. That question was the precursor to the events of Moneyball.
Shortly after publication, Michael Lewis began to look into this very question, a journey he documented in the December 2011 issue of Vanity Fair. His journey necessarily led him to the work of psychologist Daniel Kahneman. Kahneman, while studying the decision-making process, discovered that people unknowingly incorporate irrelevant factors into their decision-making process. This discovery was directly at odds with economic theory that viewed market participants as being rational and selfish.
To identify the existence of market inefficiencies, you have to first conclude that the marketplace does not incorporate all readily available information of value. That the decisions of market participants are, in some respect, irrational. Kahneman's work made that conclusion easier to reach.
One of Kahneman's research experiments was to rig a wheel-of-fortune. It was marked from 0 to 100, but was rigged to land on only 10 or 65. Kahneman would stand in front of a group of people, spin the wheel and ask the people to write down the number on which it landed, which of course was 10 or 65. He would then ask two questions: (1) Is the percentage of African nations among UN members larger or smaller than the number you just wrote? (2) What is your best guess of the percentage of African nations in the UN?
The wheel was entirely unrelated to the question asked, but it affected the answers that were given. The average estimate of those who saw the number 10 was 25%. The average estimate of those who saw the number 65 was 45%. The group had included wholly irrelevant factors, which served as a kind of anchor for their thought processes/answers, in its decision-making.
Kahneman's work led to the establishment of a sub-field of economics known as "behavioral economics." That field ultimately sparked the interest of Harvard undergraduate Paul DePodesta who went on to exploit the irrational decision-making of baseball market participants.
While DePodesta put the work of Kahneman to good use, another premier baseball mind sought out Kahneman long before DePodesta discovered him.
On June 4, 1985, Bill James wrote a letter to Kahneman. James sought out Kahneman in hopes of determining why baseball professionals always tried to explain random events; why they tried to extract meaning from inexplicable events; why they tried to order a universe that frequently defied order. The letter read, in part, as follows:
"Baseball men, living from day to day in the clutch of carefully metered chance occurrences, have developed an entire bestiary of imagined causes to tie together and thus make sense of patterns that are in truth entirely accidental. They have an entire vocabulary of completely imaginary concepts used to tie together chance groupings. It includes 'momentum,' 'confidence,' 'seeing the ball well,' 'slumps,' 'guts,' 'clutch ability,' being 'hot' and 'cold,' 'not being aggressive' and my all time favorite the 'intangibles.' By such concepts, the baseball man gains a feeling of control over a universe that swings him up and down and tosses him from side to side like a yoyo in a high wind."
Whether or not ALL these phrases are "completely imaginary concepts" is still debatable, but those that are would be examples of irrelevant factors included in the decision-making process. The inclusion of such factors ultimately leads to the creation of market inefficiencies in Major League Baseball. On a micro level, it's almost impossible not to have James' premise in mind whenever you hear announcers, managers, or players prattle on about these concepts. I find that James' letter, to a certain extent, now frames my thinking on the subject.
In the modern game, statistical analysis has become the tool by which smart organizations attack and exploit those unfounded beliefs that arose out of the efforts of baseball men to explain the discrete events of the sport. In an effort to give practical meaning to those events, baseball men incorporated irrelevant factors into their decision-making process. The flawed decision-making process created market inefficiencies; inefficiencies that could ultimately be exploited if those irrelevant factors could be identified and stripped from the decision-making process.