Tuesday, October 15, 2013

Data Never Lies....right?

I hear it a lot these days - Data Never Lies. Whether it's a political debate, a conversation on your fantasy football team, or the latest research into a childhood illness, everyone seems to agree - Data Never Lies.

The issue is that people take raw data and interpret and intuit conclusions, decisions, and courses of action, all while spouting the concept that "Data Never Lies" as proof positive that they are most certainly right.

At its very core, the statement Data Never Lies is accurate. Raw data doesn't lie. Data points are merely moments in time, frozen facts that provide a very specific truth at a very specific time under very specific circumstances. It is currently 72 degrees. That's data. That's a fact. And on it's own, it is utterly useless. Bring in other, relevant data and you begin to build a story - at 2 pm on Wednesday, October 8th 2013 in Ann Arbor, Michigan, it is 72 degrees. Supporting data gives greater detail, and makes for a better decision. But how much data is enough? Based on the information provided, Bobby decided it was a perfect day to mow the lawn, but was sadly torn to pieces by the pack of wild coyotes in the front yard. A little more data would have gone a long way.

Not only must there be enough data to build a complete story, but the data must be relevant and in context. Knowing there's wild coyotes in the front yard might seem irrelevant to weather data, but not to the decision in question. Knowing that I had Raisin Bran for breakfast this morning might make for a more compelling story, but it is irrelevant to the decision to mow the lawn. Bad data - data that is not relevant to the decision - can often be difficult to ferret out, and can lead to the wrong conclusions if included.

Even with all the right data, it can lead you to the wrong decision. That's because data requires interpretation to have any value in plotting a course, predicting the future, or making a decision. This interpretation requires human intervention, human skills, and human thought. Just look at the current political situation and you can see how flawed our ability to interpret data and make decisions can be. The proof is simple. I could have made that statement in any month of any year of any decade of civilized history and you would have thought "I know just what he's talking about". Humans make assumptions, their brains take shortcuts, their intuition is often weak, and they see the data they want to see, based on their own agendas and the baggage they bring with them.

How do you avoid falling prey to the coyotes? The first step is to know what the question is you're trying to answer, and understand it first. If you don't understand the question, you'll never understand the answer. IF you don't understand the question, you won't gather enough (or appropriate) data, turning you into a coyote snack.

Don't gather too much data. This is not a contradiction to the previous point. Sometimes the goal is to be on a fishing expedition, looking for a question that you didn't know you had, and it's important to look at as much data as possible for trends. But that's a situation where you're searching for a question, not an answer. When you already have a question or decision to make, too much irrelevant and useless data will cloud and confuse the process. How many times have you seen a movie where one side in a legal battle swamps the other with millions of documents to hide the truth? Don't do it to yourself.

The toughest nut to crack is the correct interpretation of the data. It's important to remove as many pre-disposed ideas and obstacles that might lead a human to jump to a conclusion or select a course of action based on an agenda and not on the data. Organizations and individuals often give off subtle clues as to the answer they want, and this can drive those interpreting the data to make a bad call. This is an example of organizational group think. Avoid making up your mind in advance, and learn to recognize when you're gathering the data and seeing only what you want to see to get the answer you want, rather than letting the answer flow naturally from the data.

And the next time your spouse tells you that's it's the perfect day to mow the lawn because Data Never Lies, remember to ask about the coyotes.

---Michael Crawford

1 comment: