The Tao of Data: How to Improve Your "Gut" Decisions

Why committing to a data-driven process is so essential for success, even when you aren't using data

About Lucky Maverick

About Jonathan Bales

I began playing daily fantasy golf in 2016 after my company FantasyLabs acquired a whole bunch of awesome golf data and hired a golf-focused data scientist (Colin Davy, who was just on Jeopardy…please read his post from the “Additional Reading” section at the end of this). I built a model and planned to gear it up for the Masters that year.

Knowing nothing about golf, I was utilizing some of our data that adjusted golfers based on what tour they were on—i.e. the courses they played and the competition they faced—and how much they dominated. The system was very useful for identifying golfers from the Euro and Asian tours on whom others might not have data.

Everything was leading me to one golfer: a German I had never heard of who seemed to be the most mispriced guy in the tournament by a large margin. I went all-in on Bernhard Langer.

As lineups locked, I felt pretty damn good about myself, especially when I saw how low-owned Langer was in tournaments. Why isn’t anyone using this stud!? Made no sense. Imagine my surprise when I clicked on his player card and saw this:

It turns out Langer was so dominant because he was playing on the Senior PGA Tour. He won the Masters twice, the first time coming before I was even born. I put every dollar on someone who was a legitimate threat to cut his round short because he had to go to bed. A golfer whose headshot made me jokingly say to my friend, “Dude, this guy is as old as my dad”—to which we shared a slight chuckle—before I looked it up and found out he’s actually older. A guy competing against a field whose most recent erection amongst the group came when George Bush was president. The first one.

In hindsight, I probably should have followed up more on the only piece of Langer data that didn’t look so hot: the fact that his driving distance was like 50 yards shorter than everyone else’s. I mean his driving is pretty good for a 5’9”, 160-pound senior citizen, but not as much for someone you’re betting lots of money on to win the Masters.

As you might expect, Langer stank it up. PSYCH! This motherfucker went OFF! Langer headed into the final day of play in third place, trailing leader Jordan Spieth by just two strokes. He had a poor showing on Sunday, but it was still an amazing performance for Langer, who at the time of the tournament was 10 years older than Spieth and Rory McIlroy combined.

So did I make a good play? I don’t know. I mean probably not. But it’s interesting to note that the model I created was designed to adjust for tour-specific data to put various golfers who might not normally face each other on even footing, and the numbers, at least, still liked Langer. But by most other measures, including common sense, you’d probably not want to be putting your money on someone with more wrinkles than dimples in a golf ball.

The point I’m trying to make is there are various ways you can try to decipher what’s “true” and it’s not always that easy. Sometimes you should trust models. Sometimes you should listen to others more knowledgeable than you. Sometimes you should trust your gut.

Your success in games, business, and life is highly dependent on your ability to overcome your own biases, sort through various data, and use the right tools in your decision-making arsenal at the right times to ultimately construct accurate assessments of reality.

Everything is Data: Pick the Right Times to Use Each Tool

Everything you use to make a decision is data. Your personal experiences are data. Patterns you think you see that might or might not exist are data.

So everyone is a data scientist. Your brain was literally designed to function as the computer you use to process that data. Some people are simply really shitty data scientists.

I suspect what most people mean by “data,” though, is when we rely on numbers or algorithms over other means of acquiring truth. In my Langer example, it’d be trusting the model over other factors, such as common sense or what should be a gut instinct to not play a senior citizen. Or when you see data that fly in the face of your personal experiences, i.e. “well Jordan Spieth has won me a lot of money and he seems like a great play in the Masters, but my model hates him.”

There are many ways through which we process and interpret information. If you think of it as a range, maybe at one end you have pure “instincts” and at the other you have blind trust in an algorithm or model. Beware of people who only live in one narrow range of that spectrum, or those who over-use one avenue to finding truth. One individual path might be the best one for any single problem, yes, but viewing the world through numerous lenses, as a general practice, leads to greater accuracy in assessing truth.

If you’re playing blackjack, it’s probably not the best time to use your instincts; you’d want to just rely on data and maybe some pattern recognition if you’re counting cards I suppose. But what if you’re deciding if you should marry someone? “Love” might not be an arena in which you should primarily use your analytics filter. You’re probably best off trusting your gut (but maybe not!) or the opinions of specific, qualified people who are close to you.

Here’s how to find “believable people” you can trust in specific areas. This can be applied to yourself, too, to know when you should trust your instincts and when you’re out of your league.

Nonetheless, whether we always use analytics or not, I believe becoming data-driven is an essential part of the prediction/truth puzzle, for two main reasons. One reason is that a data-driven process is self-correcting because math and logic are foundational and self-evident; to know the number 2 and the number 4 is to know that 2 + 2 = 4. And so we can apply data to itself to know which data, which models, which algorithms are good and bad. Now, some people do that pretty poorly, but that’s a human error, not a math error. A poor model can become a very useful one over time—it can be applied to itself to evolve—but how do we efficiently improve, say, our common sense, without data?

We can use data to test and improve our other lenses of viewing the world. If you believe you have an amazing gut feel for picking NFL games, you can and should test that versus other methods (and other people). If you have a hunch as to a particular diet that will work for you or specific way of studying or the best way to make ramen, you can test those ideas over time and work to refine your hypotheses.

The other reason being data-driven is so important is that the benefits spill out to other areas of decision-making, even without conscious effort…

The Data-Driven Process

The other day, I posted this on Twitter:

Jonathan Bales @BalesFootball

That’s in relation to sports, but can be applied to basically any area in which you’re using Big D Data to make decisions. The best thinkers and decision-makers I know effortlessly blend data-driven thinking with a natural subjective feel of how to proceed.

In his book The Art of Learning, Josh Waitzkin brilliantly explains how he became both a chess master (he’s the basis for the movie Searching for Bobby Fischer) and a world champion in martial arts through a truly art-and-science approach to learning, or what he calls “numbers to leave numbers.”

As I struggled for a more precise grasp of my own learning process, I was forced to retrace my steps and remember what had been internalized and forgotten. In both my chess and martial arts lives, there is a method of study that has been critical to my growth. I sometimes refer to it as the study of numbers to leave numbers, or form to leave form. A basic example of this process, which applies to any discipline, can easily be illustrated through chess: A chess student must initially become immersed in the fundamentals in order to have any potential to reach a high level of skill. He or she will learn the principles of endgame, middlegame, and opening play. Initially one or two critical themes will be considered at once but over time the intuition learns to integrate more and more principles into a sense of flow. Eventually the foundation is so deeply internalized that it is no longer consciously considered, but is lived. This process continuously cycles along as deeper layers of the art are soaked in.

To get to that point, you should get comfortable using and analyzing data because the process of scrutinizing numbers, interpreting the results of an algorithm, etc. leads to better decision-making in all areas; it makes your “gut” more accurate.

Being data-driven leads to better subjective choices when you aren’t using what we refer to as “data.” It helps sharpen the other means by which you come to finding the truth. It improves your “common sense.” It helps you know when to trust others’ experiences and when they’re worthless. Maybe most of all, it significantly improves pattern recognition, which I believe is probably the most vital component of what we’re all trying to do: make better predictions.

From Waitzkin:

It is important to understand that by numbers to leave numbers, or form to leave form I am describing a process in which technical information is integrated into what feels like natural intelligence. Sometimes there will literally be numbers. Other times there will be principles, patterns, variations, techniques, ideas. A good literal example of this process, one that does in fact involve numbers, is a beginner’s first chess lesson. All chess players learn that the pieces have numerical equivalents. Bishop and knights are worth three pawns, a rook is five pawns, a queen is nine. Novices are counting in their heads, or on their fingers, before they make exchanges. In time they will stop counting. The pieces will achieve a more flowing and integrated value system. They will move across the board as fields of force. What was once seen mathematically is now felt intuitively.

This is why I tweeted “Only a very small percentage of people should use their gut.” This level of “numbers to leave numbers” thinking can arise only after hours upon hours of practice, commitment to a data-driven process, an understanding of when to trust each of your filters—a level of brain interconnectedness to harmonize the personal with the impartial to a point that everything becomes one and you can just act.

Additional Reading and Some Quotes

As mentioned, you absolutely must read this post from Colin Davy: How I Won Jeopardy with Data Science.

He has another great post on why Sports Is Your Perfect Intro to Data Science. I also love sports—and sports gambling—as a way to play with the implementation of data in your decisions because it’s one of the few areas that provides quick, direct feedback. There’s no greater motivator to improve your thinking than when you lose tons of money because you forgot a zero…on a man’s age.

Another great quote from Waitzkin:

My numbers to leave numbers approach to chess study was my way of having a working relationship with the unconscious parts of my mind. I would take in vast amounts of technical information that my brain somehow put together into bursts of insight that felt more like music or wind than mathematical combinations. Increasingly, I had the sense that the key to these leaps was interconnectedness – some part of my being was harmonizing all my relevant knowledge, making it gel into one potent eruption, and suddenly the enigmatic was crystal-clear.

The Inner Game of Tennis is another great book with many Taoist-like principles related to “learning without learning,” a la Waitzkin.

From Ray Ralio, hedge fund manager and author of Principles:

Intuition, instinct, hunch…it's critical. And it's also, by the way, what the computer can't do well. In other words, it's the subconscious. As you know, man, although it's only 200,000 years old, the brain is much older. And we came programmed with many of these things in our brain, intuition and those things. And they're in our subconscious. And so by opening up one's subconscious to one's consciousness so they come up, you know, creativity comes from not working hard at it, it comes from relaxation. It bubbles up from the subconscious. But that intuition, that creativity, man is still unique at being able to do those things. So you let that bubble up, but you have to reconcile it with your logic. So when the subconscious creativity and intuition comes up and replicate it with your logic, it's fabulous.

And finally, one from me:

Dude I swear, it says here he was born in 1957.